Seer: A Lightweight Online Failure Prediction Approach

In [1], we present a lightweight online failure prediction approach, called Seer, to predict the manifestation of failures at runtime, i.e., while the system is running and before the failures occur, so that preventive and/or protective measures can proactively be taken to improve software reliability. One way Seer differs from the other related approaches is that it collects information from inside program executions, from which the existing approaches generally refrain themselves due to the typically excessive runtime overheads incurred. Seer overcomes this issue by pushing the substantial parts of the data collection task onto the hardware with the help of hardware performance counters (HPCs) - CPU resident counters that record various low level events occurring on a CPU, such as the number of instructions executed and the number of branches taken. At a very high level, Seer operates as follows: functions, called seer functions, that can reliably distinguish failing executions from passing executions are determined; these functions are then instrumented in such a way that after every invocation of a seer function, a binary prediction (i.e., passing or failing) about the future of the execution is made; the instrumented system is deployed and the sequence of predictions made by the seer functions are analyzed at runtime using fixed-length sliding windows to predict the manifestation of failures. We have evaluated Seer by conducting a series of experiments on three software systems in the presence of both single and multiple defects. At the lowest level of runtime overheads, Seer predicted the failures about 54% way through the executions (when the duration of an execution is measured as the number of function calls made in the execution) with an F-measure of 0. 77 (computed by giving equal importance to precision and recall) and a runtime overhead of 1.98%, on average. At the highest level of prediction accuracies, Seer predicted the failures about 56% way through the executions with an F-measure of 0. 88 and a runtime overhead of 2.67%, on average. Furthermore, Seer performed significantly better than the other online failure prediction approaches used in the empirical studies.