Segmentation with area constraints Image segmentation approaches typically incorporate weak regularity conditions such as boundary length or curvature terms, or use shape information. High-level information such as a desired area or volume, or a particular topology are only implicitly specified. In this paper we develop a segmentation method with explicit bounds on the segmented area. Area constraints allow for the soft selection of meaningful solutions, and can counteract the shrinking bias of length-based regularization. We analyze the intrinsic problems of convex relaxations proposed in the literature for segmentation with size constraints. Hence, we formulate the area-constrained segmentation task as a mixed integer program, propose a branch and bound method for exact minimization, and use convex relaxations to obtain the required lower energy bounds on candidate solutions. We also provide a numerical scheme to solve the convex subproblems. We demonstrate the method for segmentations of vesicles from electron tomography images. Image segmentation is a fundamental task in image analysis. Consequentially, a large number of segmentation methods have been developed ranging from local thresholding to methods using statistical models of shape variation (Pham et al., 2000; Sonka et al., 2008) . The simplest available segmentation methods rely on local pixel-by-pixel segmentation decisions such as Otsu thresholding or methods based on clustering. These fully-local decisions are often not sufficient and because they neglect spatial dependencies, they are sensitive to noise and not directly applicable if an object is defined by its boundary surface only (e.g., if only the cell membrane or a cell membrane surrogate is imaged, but an image of the entire cell is desired). To overcome these limitations, nonlocal approaches have been proposed based on intelligent local merging decisions or by formulating optimization problems incorporating spatial dependencies. The former class of methods encompasses region growing approaches such as the popular watershed segmentation (Sonka et al., 2008) . The latter class of methods includes active-contours and -surfaces (Sapiro, 2001) as well as general parametric models which may use statistical information on shape and/or appearance (Cootes et al., 2001; Pizer et al., 2003) . When the object segmentation task is highly structured (i.e., expected shape variations are reasonably small and the approximate number and location of the objects are known) shape-or atlasbased segmentation methods are highly successful (Rohlfing et al., 2005) . However, for less structured cases these methods are not applicable. In microscopy, for example, images often contain hundreds or thousands of cells, cell nuclei, or organelles, with possibly large variations in shape and a priori unknown locations. While local thresholding or active-contour-type models may be applied in such cases, they are often too generic, too sensitive to noise, or require the judicial placement of seed points to assure an appropriate segmentation result to avoid over-or undersegmentations.. If shape-or atlas-based segmentation methods are too restrictive, and if general purpose segmentation methods such as active contours, region-growing or thresholding are not restrictive enough for a particular segmentation task, the question of how to incorporate additional domain information into a segmentation that lies between these two extremes arises. A possible option is to use information about simple geometric properties. In this paper we explore an approach for a segmentation with constraints on the segmentation area. Such a method can counteract potential leakage or shrinkage biases in a principled way. Such biases can be observed, for example, for active contour (Sapiro, 2001) or graph cut (Boykov and Funka-Lea, 2006) segmentations when boundary regularity is encouraged by penalizing a weighted length of the segmentation boundary. Area constraints may not be appropriate for all biomedical segmentation tasks; however there are a large number of problems in which reasonable area or volume intervals are known a priori. Our objective in this paper is not to perform an actual study for a particular biological problem, but rather to demonstrate the behavior of a segmentation method with areaconstraints on realistic image data. We use electron tomography datasets of synaptic vesicles and of double-membrane vesicles (DMVs) implicated in the SARS-coronavirus (severe acute respiratory syndrome coronavirus) replication (Knoops et al., 2008) .. Many recent segmentation approaches are formulated such that the optimization problems become convex so that globally optimal solutions can be obtained (Appleton and Talbot, 2006; Bresson et al., 2007) or so that they can be solved with discrete solution methods, such as graph-cuts (Boykov and Funka-Lea, 2006) . While area-constraints can formally easily be added to the optimization problems for segmentation, solving the problems is hard. However, if finding a globally optimal solution is not of concern and a good initial guess for a solution is available, one can resort to standard methods from constrained optimization. For example, a curve evolution approach with an area penalty can be used (Ayed et al., 2008) . Proposed numerical solution approaches to obtain a global optimum or a good approximation.. are limited to problems with small numbers of variables (Ji, 2004) , or require long computation times (Dahl and Flatberg, 2007) , use solution heuristics (Kernighan and Lin, 1970) , or use various forms of relaxations of the original problem to facilitate computations: e.g., spectral relaxations (Olsson et al., 2008) , semidefinite programming (Keuchel et al., 2003; Lisser and Rendl, 2003; Hager et al., 2009) , or variational inference approximations (Kropotov et al., 2010) .. Approaches have generally focused on equality constraints (i.e., exact size) in formulation (Lim et al., 2010; Eriksson et al., 2011; Falkner et al., 1994; Ayed et al., 2008) or for testing (Hager et al., 2009 ). However, equality constraints have only limited applicability when the exact object size is not known beforehand or when it is a desired measurement (as is frequently the case in biomedical imaging), because it would bias the segmentation towards the chosen area. We therefore formulate the segmentation problem with inequality constraints on the segmentation area.. Section 2 introduces the area-constrained segmentation problem. Section 3 outlines our solution approach. Sections 4-6 discuss its numerical solution. Segmentation results on real electron tomography images demonstrate the utility of the method in Section 7. The paper concludes with a summary and a discussion of future work. We developed a new method for image segmentation with area constraints. The method readily extends to higher dimensions using higher-dimensional generalizations of the selection regions. The proposed method relies on the solution of a mixed integer nonlinear program, which is solved using branch and bound. To reduce computational effort in solving the area-constrained segmentation, we proposed to use selection variables based on eroded super-pixels. This allows computation of the segmentations for practical problems. The behavior of the method was demonstrated for segmentations of vesicles from slices of electron tomography images. When area-constraints were available, statistically significant increases in segmentation quality were obtainable even in challenging cases. In particular, due to the global optimality properties of the algorithm, it performs well for noisy data.. Future directions include improvements on the optimization method: e.g., should computations be performed directly on super-pixels? Further, the sensitivity of the obtained results on the type and size of the superpixels should be explored. Another interesting direction would be to vary the area constraints to define an area-based scale space, which would allow us to automatically extract coherent sub-structures at different size levels from the images. Extensions to multiple objects or the inclusion of topological constraints (e.g., to enforce one connected component) are other possible research directions. We tested the area-constrained segmentation method for the segmentation of synaptic vesicles and for double membrane vesicles in epithelial cells infected with SARS-coronavirus (Knoops et al., 2008) . All images are slices of electron tomography images. Images for the epithelial cells were obtained from the cell centered database (CCDB) of the National Center for Microscopy and Imaging Research (NCMIR -http://ccdb.ucsd.edu). The images for the synaptic vesicles were approximately at a resolution of (1.0 nm by 1.0 nm)/pixel and for the SARS-coronavirus at (1.2 nm by 1.2 nm)/pixel. These examples were chosen to demonstrate the properties of our developed area-constrained segmentation method, because segmentations for these electron tomography images are known to be challenging. For example, for the synaptic vesicle segmentation task the vesicle wall is not directly visible in the electron tomography image. Instead it needs to be inferred from the location of proteins (which appear dark) penetrating the vesicle wall which results in a ''noisy'' appearance of the vesicle wall. Further, a large number of vesicles can be found in one image. Vesicles are closely packed in some areas, which even experts can have difficulty outlining precisely. In our experiments a user was asked to place individual seed points at the center of the objects to be segmented. The selection areas were obtained by eroding a quick-shift segmentation of the complete image. The selection region closest to the user-placed seed point was set as a foreground seed, and the selection areas at the boundaries of a 100 Â 100 pixel box centered at the seed point were set as background seeds. This box size was chosen to be sufficiently large to guarantee that the desired objects are contained within it. We used image intensities as edge terms (c st ) and set q = 0. We set c = 1 for all ADMM experiments.. Setting the selection areas at the boundaries as background seeds is meaningful for our experiment because the object will be, by construction, at the center of the image. However, this is not essential. The boundaries could be included into the optimization, albeit at the price of higher computational cost.. Given the selected segmentation area and the selection regions we compared the following methods for the vesicle datasets:. (1) UC: Area-constrained segmentation with a lower bound of 0.. This unconstrained case corresponds to a classical graph-cut segmentation with seed points. (2) LB/UB/LBUB: Area-constrained segmentation enforcing upper and lower bounds on the segmentation area separately and jointly. (3) BNC: Biased normalized cut (Maji et al., 2011) using foreground seeds. (4) NC: Normalized cut (Shi and Malik, 2000) . No seed regions are supported by this algorithm and hence none were used. (5) WS: Seeded watershed segmentation (Vincent and Soille, 1991) using foreground and background seeds. (6) RW: Random walker segmentation using foreground and background seeds.. To allow comparisons between the algorithms (i) we used the same seed regions for all algorithms, (ii) we used the same edge weights g for all algorithms except for the random walker algorithm, and (iii) determined the best possible thresholds for NC and BNC by searching which threshold results in the best value for the normalized cut. For the random walker segmentation algorithm, we used two sets of edge weights, g, because this is a segmentation model which will not exhibit discontinuities at the putative segmentation boundary and hence treats edge-weights differently than all the other tested models. We report randomwalker results using the same edge-weights as the other algorithms (RW) as well as using the more appropriate default settings for random walker segmentation (d-RW).

B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O


Learning a Channelized Observer for Image Quality Assessment Abstract-It is now widely accepted that image quality should be evaluated using task-based criteria, such as human-observer performance in a lesion-detection task. The channelized Hotelling observer (CHO) has been widely used as a surrogate for human observers in evaluating lesion detectability. In this paper, we propose that the problem of developing a numerical observer can be viewed as a system-identification or supervised-learning problem, in which the goal is to identify the unknown system of the human observer. Following this approach, we explore the possibility of replacing the Hotelling detector within the CHO with an algorithm that learns the relationship between measured channel features and human observer scores. Specifically, we develop a channelized support vector machine (CSVM) which we compare to the CHO in terms of its ability to predict human-observer performance. In the examples studied, we find that the CSVM is better able to generalize to unseen images than the CHO, and therefore may represent a useful improvement on the CHO methodology, while retaining its essential features.Index Terms-Channelized Hotelling observer (CHO), image quality, machine learning, numerical observer, support vector machine (SVM), task based image evaluation. We have introduced an alternative to the CHO called the channelized support vector machine (CSVM) (as well as the channelized linear regression model (CLIN) a special case of CSVM when a first order polynomial kernel is used for SVM), which is intended for use as a task-based measure of image quality. We found, at least for the data tested, that the CSVM, obtained by regressing against the human observer scores, can produce more accurate predictions of human-observer performance than the CHO. Our main finding is that, while the CHO and CSVM both generalized well to unseen images reconstructed in the same way as those in the training set, the CHO did not generalize well in this experiment to unseen images reconstructed in different ways. On the contrary, the CSVM performed well in predicting human-observer performance in all the cases tested. The CHO is a simpler detector than CSVM, which is more conducive to analytical analysis of observer performance. However, the good performance of CSVM may prove beneficial for numerical analysis of performance.. Of course, further studies will be needed to fully validate the CSVM technique; however, this initial study suggests that the CSVM is a promising method for calculation of accurate predictions of human-observer performance in lesion-detection tasks.. In future studies, we will evaluate whether the channel operator is the optimal pre-processor within the CSVM context, and whether other learning machines might perform better than the support vector machine. In addition we will also consider the feasibility of extending the proposed methodology for an observer localization task. We begin with a brief introduction to the well-known CHO, on which our proposed image-quality assessment method is based.

O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I B I I I B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Optimal sparse matrix dense vector multiplication in the I/O-model We analyze the problem of sparse-matrix dense-vector multiplication (SpMV) in the I/O-model. The task of SpMV is to compute y := Ax, where A is a sparse N × N matrix and x and y are vectors. Here, sparsity is expressed by the parameter k that states that A has a total of at most kN nonzeros, i.e., an average number of k nonzeros per column. The extreme choices for parameter k are well studied special cases, namely for k = 1 permuting and for k = N dense matrix-vector multiplication.We study the worst-case complexity of this computational task, i.e., what is the best possible upper bound on the number of I/Os depending on k and N only. We determine this complexity up to a constant factor for large ranges of the parameters. By our arguments, we find that most matrices with kN nonzeros require this number of I/Os, even if the program may depend on the structure of the matrix. The model of computation for the lower bound is a combination of the I/O-models of Aggarwal and Vitter, and of Hong and Kung.We study two variants of the problem, depending on the memory layout of A. If A is stored in column major layout, SpMV has I/O com- * Sparse-matrix dense-vector multiplication (SpMV) is one of the core operations in the computational sciences. The task of SpMV is to compute y := Ax, where A is a sparse matrix (most of its entries are zero) and x and y are vectors. Applications abound in scientific computing, computer science, and engineering, including iterative linear-system solvers, least-squares problems, eigenvalue problems, data mining, and web search (e.g., computing page rank). In these and other applications, the same sparse matrix is used repeatedly; only the vectors x and y change.. From a traditional algorithmic point of view (e.g., the RAM model), the problem is easily solved with a number of operations proportional to the number of entries in the matrix, which is optimal. In contrast, empirical studies show that this naive algorithm does not use the hardware efficiently, for example [18] reports that CPU-utilization is typically as low as 10% for this algorithm. The explanation for this observation lies in the memory system of a modern computer, where an access to a data item happens without delay if the item is stored in the fast processor cache, whereas it takes a significant amount of time if the item needs to be fetched from memory, or even worse if the item resides on disk. Today the trend in hardware development is that cache has more levels, and that the speed of CPU and innermost caches increase faster than the speed of the slower memory, in particular the disk. Hence, the goal of taking algorithmic advantage of the memory hierarchy will also remain important for hardware in the foreseeable future.

O B I I I I O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Eliminating false data dependences using the Omega test Array data dependence analysis methods currentlyin use generate false dependence that can prevent useful program transformations. These false dependence arise because the questions asked are conservative approximations to the questions we really should be asking. Unfortunately, the questions we really should be asking go beyond integer programming and require decision procedures for a subclass of Presburger formulas. In this paper, we describe how to extend the Omega test so that it can answer these queries and allow us to eliminate these false data dependences. Wehaveimplemented thetechniques described here and believe they are suitable for use in production compilers. Recent studies [HKK+91, CP91] suggest that array data dependence analysis methods currently in use generate false dependence that can prevent useful program transformations.. For the most part, these false dependence are not generated by the conservative nature of algorithms such as Banerjee's inequalities [SLY89, KPK90] .. These false dependences arise because the questions we ask of dependence analysis algorithms are ccmservative approximations to the questions we really should be asking (methods currently in use are unable to address the more complicated questions we should be asking). Several authors have explored methods for using integer programming methods to decide subclasses of Presburger formulas [Ble75, Sho77, JM87]. Previous approaches have not had any way to compute projections (and thereby handle embeded existential quantifiers) nor have they had special efficient methods for checking when an implication is a tautology. We have not yet looked at wider applications of our work or done any direct comparison of our implementation against implementations of other approaches. We believe our approach and implementation will compare favorably with previous ones. a(n) := . . . . In Example 1, the write to a(Ll ) kills the flow from the write of a(n) to the read of a(Ll):. If the first write were to a(m), we would not be able to verify the kill:. If n~m~n + 10 had been asserted by the user, we would be able to verify the kill. . In Example 2, the read of a(L2 ) is covered by the write to a(L2-1):. Since we have determined that this dependence is a covering dependence, there is no need to check for dependencies with writes that must precede a(L2-1 ) (such as the write to a(m)). However, for writes that may be executed after some executions of the cover (such as the write toa(L2)), we must check separately for a kill. In general, we may have to determine which loop carries the cover to know which write accesses must precede it. The cover in this example is loop independent, so we know that all executions of the write a(Ll ) must precede the cover. Note that we only find out that the cover is Ioop independent when we refine its dependence vector from (O+) to (0). For this reason, we perform refinement before coverage analysis.. In analyzing false array flow data dependence (caused by output dependence), there are two basic approaches:. e Extend scalar dataflow methods by recording which array sections are killed and/or defined [GS90, Ros90] .. e Extend the pair-wise methods typically used for array data dependence to recognize array kills [Bra88, Rib90].. Both approaches have merits. Our work is an example of the second approach, and we believe it corrects several limitations and flaws in previous work on that approach.

O O B O O O O B O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Temporal aggregation impacts on epidemiological simulations employing microcontact data Background: Microcontact datasets gathered automatically by electronic devices have the potential augment the study of the spread of contagious disease by providing detailed representations of the study population's contact dynamics. However, the impact of data collection experimental design on the subsequent simulation studies has not been adequately addressed. In particular, the impact of study duration and contact dynamics data aggregation on the ultimate outcome of epidemiological models has not been studied in detail, leaving the potential for erroneous conclusions to be made based on simulation outcomes. Methods: We employ a previously published data set covering 36 participants for 92 days and a previously published agent-based H1N1 infection model to analyze the impact of contact dynamics representation on the simulated outcome of H1N1 transmission. We compared simulated attack rates resulting from the empirically recorded contact dynamics (ground truth), aggregated, typical day, and artificially generated synthetic networks. We have demonstrated that the dynamics of empirical contact networks impact the outcome of simulation models, that aggregation over this data provides systematic overestimates of disease burden, that typical day data collection techniques employed elsewhere impose a risk. These findings are important as they inform both issues in microcontact study design and use of synthetic contact networks in calibrated models. However these findings are limited by the number of participants in the study and the single pathogen studied. In the future, we intend to investigate these impacts against larger, longer and more diverse datasets to determine if there is a point at which increasing study duration or sample size no longer materially affects simulation outcomes. We will also investigate the interaction between pathogen behavior and temporal sampling strategies. By providing this work we have established baseline insights into both the design of agent-based simulations of pathogen transmission and into the emerging discipline of contact dynamics acquisition and analysis. The results presented here simultaneously underscore the high rate of return in terms of model reliability of investing in the collection of micro-contact data, but suggest that not all such investments confer equivalent value. It is widely recognized that model aggregation (for example, in the imposition of random mixing assumptions) tends to overestimate the spread of contagion. The results presented here suggest the natural extension of this understanding to the network context, where simulations employing purely static networks -such as might be produced by traditional network reconstruction techniques based on contact tracing, diarying or survey instruments -are also demonstrated to be biased towards overestimation of contagion. While this overestimation is pronounced in our experiments, it appears likely that the degree of this overestimation will depend heavily on the characteristics of the contagion processparticularly the speed of transmission relative to the speed of contact formation and dissolution [15] . The distortions associated with imposing a static network for some types of contagion suggest the importance of collecting information regarding network dynamics at a fine temporal granularity for such contexts. For very short-lived pathogens, a typical day approach may be sufficient in that all members of the network will be infected quickly or the infectious transmission will dwindle quickly. For long-lived pathogens such as tuberculosis, an aggregate model may be appropriate, as infection rates are low and latent periods are long. For the flu-like infection we modeled, our results indicate that typical days are too volatile and aggregate representations too permissive.. In addition to highlighting the importance of fine-scale data collection, these results also suggest a minimum efficient scale for such data collection. Such results are important in that many healthcare researchers have sought to defray the cost and logistic burden of electronically mediated micro-data collection by restricting the duration of studies. While such study designs sidestep many challenges associated with power consumption and device failures, they do rely heavily on the assumption that the network dynamics observed during the data collection interval is in some sense representative of the network dynamics over the long term. While such assumptions hold greater plausibility when applied in highly structured settings such as secondary schools [9] , the results presented here highlight the elusiveness of identifying a typical day in some populations. Our results suggest that adopting a day as representative can be the source of major variability in simulation results, which can swamp the accuracy benefits conferred by using electronic micro-contact data. Diverse transmission modeling studies in the past several decades have revealed pronounced impacts of population heterogeneity on transmission dynamics, and have recognized the risks to model reliability of positing that the population is composed of homogenous typical persons. The results here suggest that -at least for some settings and sample populations -selecting a typical day can be fully as risky to model reliability as selecting a typical person -even when due diligence has been exercised to filter out manifestly unrepresentative days. While typical day assumptions appear to be hazardous in the context of our limited and highly clustered sample population, future data collection is required to determine whether similar risks extend from choosing typical days for studies with larger and more diverse populations.. Synthetic networks based on randomly generated networks and contact duration distributions are an appealing representation for extending results to larger populations. Our results highlight the need for care in the design of synthetic networks given that even high quality matches to the contact distribution using parametric distribution mixtures can lead to distortions of infection risks, primarily due to the impact of a few important outliers. When assessing the match fidelity between the synthetic results and results for the fully detailed data, it is easy to overlook an underestimation if it cancels out the overestimation due to an aggregated network whose outcomes misleadingly agree with historical or synthetic ground truth estimates. These biases could in turn distort the perceived trade-offs between interventions, or skew other important statistics regarding model dynamics. As is the case for random mixing models [4] , even apparently calibrated models which match baseline data well can yield very misleading results when applied to counter-factual scenarios, such as interventions. We have little doubt that it is both important and feasible to build synthetic networks to capture transmission dynamics with high accuracy, but the current study suggests that achieving this goal will require moving beyond even well-calibrated models of network structure and contact duration. We particularly highlight the potential need to scrutinize the convenient assumption (applied here) of independence in the centrality of an agent and that agent's contact duration distribution. While the close match in mean attack rates observed between the FullA and SWE scenarios suggests that this assumption may impose little distortion, further study is required to assess this assumption's reliability. The findings here have implications both for health research and epidemiological methodology. The results suggest that, while it is important to capture microcontact data in many contexts, attempting to economize by reducing study duration may be penny wise and pound foolish. Use of micro-contact data appears to confer substantial benefits, but those benefits will be compromised -and potentially reversedunless studies are of sufficient duration to capture day-to-day variability. While these results could be unique to smaller participant pools and flu-like infections, they likely apply to many important pathogens in high-risk populations of note such as institutional populations associated with long-term care facilities which are often not much larger than our participant pool.. We hope that the findings presented here will offer initial guidance in planning micro-contact studies, elevating the prospect that these studies will serve as a cost-effective means of enhancing health insight. Our results regarding the impact of network representation and parametric contact distribution approximations will also inform the data analysis methodologies used to analyze and generalize such data for transmission modeling. We have further described here a general methodology that can be replicated for future studies to assess the impact of aggregation on simulation results.. While our paper has made several important contributions to the field, as with all initial analyses, there are several shortcomings that should be addressed in future work. In fact, a major contribution of the paper is elucidating what the structure of future experiments should be to provide realistic samples of contact and disease dynamics for a given population. In particular, the role of sample size (N), versus study duration (T) is problematic.. Previous contributions [9, 15] have opted for larger N at the expense of T, and attempted to use various techniques to extend the duration of simulations beyond T artificially. We have demonstrated that the error introduced through replicating T is significant and perhaps larger than using a well-tuned aggregate model. However, the limited participant pool of our study prevented us from addressing the issue of sample size. This leads to the possibility that our results are a statistical fluke, and that larger sample size would begin to diminish the heterogeneity in sample days to the point where one day was like any other for a large enough group of people. We believe this outcome to be unlikely due to the smallworld nature of dynamic contact networks [36] . Rather than having an averaging effect, adding additional Nparticularly for heterogeneous study populations -is likely to capture more partially isolated sub-networks with their own particular contact dynamic patterns and life rhythms. A study employing a typical day strategy must then assume that the sampling period represents a typical day for new subnets. In fact, we conjecture that larger sample sizes will increase the sensitivity to T, as temporal variability is likely to rise with population heterogeneity.. The uncertainty over the apparent tradeoff between study size and duration is a strong motivation for further contact dynamic studies with substantially larger, heterogeneous populations observed over longer time periods. Larger longitudinal studies would permit the analysis of the relative sensitivity of different disease models to simplifications in sample size, population and study duration by permitting knock-out experiments where similar simulations could be performed over statistically significant subsets of the data, and compared against each other and the full data set. Computational models of contagion can provide insight and foresight into the behavior of particular pathogens for specific populations. The representation of contacts between population members has a critical impact on simulation outcomes [1, 2] . It has long been recognized that models that assume random mixing between aggregate population categories can alter the dynamics and results of the simulation of contagion spread [3] , and may inhibit insight into intervention trade-offs [4] . Reflecting such findings, the past decade has witnessed the growing use of agent-based transmission models, which can better capture the impact of individual-level contact patterns on both individual risk and populationwide transmission dynamics. Such models have frequently represented the inter-agent contact patterns as static graphs with edge weights corresponding to threat of infection drawn from uniform, power-law [5] , or other [5, 6] distributions.. More recently, researchers have used microelectronic devices such as motes [7] [8] [9] , RFID tags [10] , cell phones [11] [12] [13] [14] , or custom-built wireless technologies [15] to capture contact distance, duration and location, as well as other parameters, with significantly more accuracy than previous diarying [16, 17] or retrospective selfreport [18, 19] methods, and substantially greater accuracy than cellular [20, 21] or WiFi [22] location estimates whose modeling use entails assuming random mixing patterns between individuals in the same or nearby geographic regions. Data collected from such dynamic contact networks has been used to simulate the spread of a pathogen through the recorded network, using both the raw sample data [15, 23, 24] , and aggregated versions where infection probability is drawn from an empirical rather than functional distribution [9] . While electronically supported micro-contact data collection offers significant spatialtemporal resolution and compliance advantages over traditional techniques, the relatively high cost and logistical effort involved in a deployment of telemetry systems and their limited on-board energy capacity imposes design tensions between study duration, sampling rate, and participant count. While study design has been shaped by clear cost-economic considerations favoring shorter deployments with larger populations [9] or longer deployments with smaller populations [8, 23] , it has been been conducted absent a clear understanding as to how study design or post-study data aggregation affect transmission model accuracy. Reflecting the power hungry character of the data collection devices, studies have frequently favoured larger, high sampling-rate designs of just one [9] , two [10, 25] or a handful of days [26] in duration. While some researchers have argued that such short sampling periods capture one or more typical day(s) [9] , there remains a dearth of formal evidence for the representative character of short sampling periods, and the effects of limited sampling periods on the quality of subsequent simulation outcomes have not been explored.. Researchers have employed diverse strategies for using micro-contact data in transmission models. In some cases [25] , researchers have sought to use traditional static representations, with weights on the link connecting two individuals being proportional to the cumulative contact duration observed between those individuals over the study period. Other studies [9, 23, 24, 27] have employed strategies in which dynamic contact patterns collected over an interval of time are used directly as the contact pattern for that the same time horizon. In initial examinations of the effects of aggregation of microcontact data on simulation outcome, the authors of [28] noted the effects of aggregation on contagion using two sets of empirical data, and the authors of [25] explored the impact of changes in model parameter values and network aggregation measures -including alternate use of both weighted and unweighted static representations -to simulation outcomes. Because the dataset in [28] was collected over a relatively short (2 day) study duration, the researchers replicated the recorded data and variants over longer simulation horizons (60 or 100 days) to ensure sufficient duration to capture the dynamics of the simulated outbreak, and found that application of unweighted homogeneous aggregation schemes led to significant overestimation of infection spread. The study further examined three strategies for extending the time frame, with all three approaches positing that the observed contact recorded over the brief study period are representative of a longer period of time. The researchers found notable epidemic-phase-specific differences between the typical day [9] variants, reflecting whether contacts occur repeatedly with the same set of individuals or different individuals. Most importantly, simulated infection transmission on a duration-weighted aggregated static graph yielded results close to those from a fully disaggregated but replicated dynamic network. However, because of the short study duration, the researchers were unable to evaluate the fidelity of any of the three approaches to simulated infection transmission over a disaggregated dynamic network for the entire simulation time horizon. While this contribution yielded important insights into the effects of aggregation and replication strategies on simulation outcomes, the researchers emphasized the importance of analyzing data from longer-duration studies, to validate the fidelity of the proposed strategies for study data extrapolation.. To help address and expand upon the questions raised in [25] regarding the impact of temporal aggregation and extrapolation, this paper analyzes such effects for the Flunet dataset [8] in the context of an influenza-like illness (ILI). The Flunet dataset is characterized by a two minute-level temporal contact resolution, and a smaller population (N=36) observed over a much longer time period (T=92 days) than other studies [9, 10] . Taking advantage of the longer time duration of this study, we examine agent-based simulation using a previously contributed ILI model [29] , and both previously contributed [8, 23, 24] and novel dynamic network structures, and previously used but unevaluated strategies for extending the study timeframe. We report three important findings:. Even the weighted "typical-day" techniques advanced in [9, 25] are fragile because the variance in contact patterns between days can lead to highly variable simulation outcomes; Confirming with an extended temporal horizon and extending findings of [25] , aggregated network representations tend to overestimate the disease burden in the population; While static synthetic networks with contact distributions can produce results consistent with the full dynamic network, this apparent agreement can be due to an accumulation of counter-balancing inaccuracies leading to a deceptively correct outcome.. Overall, our findings suggest that longitudinal data collected over a prolonged period is required to accurately reconstruct dynamic or static contact networks for the purpose of simulating pathogen spread for at least some subpopulations, and that simulations based on summary networks tend to systematically overestimate the risk of infection. These findings are important because they suggest that traditionally calibrated disease models may contain biased disease parameter estimates to compensate for undetected inaccuracies in the underlying network and mixing models. These biases in disease parameter estimates might then lead to erroneous conclusions about the relative impact of interventions or the rate of disease propagation. Because such inaccuracies may not cancel in the same fashion when investigating intervention effects, additional caution should be employed when using such aggregation and generalization schemes. Unfortunately, while typical day [9, 25] contact data acquisition techniques ease study design, they do not appear to be an adequate solution, as they can yield even more erroneous disease burden estimates than traditional aggregate models. This leads us to finally conclude that larger-scale longer-term longitudinal studies may be required to generate sufficiently accurate descriptions of the contact dynamics in many populations. Such studies are urgently needed to help identify the balance between study size and length required to secure reliable insight into intervention tradeoffs. Given the H1N1 strain emergence and in anticipation of the significance of the 2009-2010 influenza season, the co-authors launched a previously-described [24] pilot study in the midwestern Canadian city of Saskatoon to electronically collect contact patterns between 36 participants, in addition to their influenza-related health status information. The study was conducted between November 9th, 2009 and February 9th, 2010 -a time period coinciding with the second rise of reported H1N1 cases in the province of Saskatchewan [30] . Each participant was requested to carry a proximity sensor whenever awake during the study period, and to respond to a sequence of weekly health surveys via a web browser. Participants filled out an informed consent form prior to joining the study, as required by the university research ethics board.. To study the impact of network representation on simulation outcomes, an H1N1 infection transmission model [24] was created and parameterized with empirical characteristics of the H1N1 pathogen [31] . The model simulated the spread of infection between agents through their daily interactions. Agents' daily connectivity -either from empirically collected or aggregate approximations -were imported into the model and defined the potential for infection between discordant agents. Overall, 616 different scenarios -each positing a different connectivity network derived from the datawere simulated, with each such scenario being run for 10,000 realizations by replaying a sequence of the 265,000 thirty-second time slots in the study period. Based on the 100,000 run ensembles with the same model and dynamic network reported in [24] we concluded that 10,000 runs were sufficient, as the maximum depth of infection in [24] was reached 10 times, implying that division by 10 would have a high probability of capturing even the most extreme infection events, while maintaining the probability distribution at significantly reduced computational cost. Because each run could be conducted relatively swiftly (typical run-times for an ensemble were 30 s) having a large number of runs was not computationally prohibitive given the resources available to us.. The underlying disease parameters and distributions were held constant across scenarios. Within a given scenario, stochastics associated with exogenous and endogenous infection transmission and duration of different phases in the natural history of infection induced variability in simulation output.

O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O


Quantifying the determinants of outbreak detection performance through simulation and machine learning Objective To develop a probabilistic model for discovering and quantifying determinants of outbreak detection and to use the model to predict detection performance for new outbreaks.We used an existing software platform to simulate waterborne disease outbreaks of varying duration and magnitude. The simulated data were overlaid on real data from visits to emergency department in Montreal for gastroenteritis. We analyzed the combined data using biosurveillance algorithms, varying their parameters over a wide range. We then applied structure and parameter learning algorithms to the resulting data set to build a Bayesian network model for predicting detection performance as a function of outbreak characteristics and surveillance system parameters. We evaluated the predictions of this model through 5-fold cross-validation.The model predicted performance metrics of commonly used outbreak detection methods with an accuracy greater than 0.80. The model also quantified the influence of different outbreak characteristics and parameters of biosurveillance algorithms on detection performance in practically relevant surveillance scenarios. In addition to identifying characteristics expected a priori to have a strong influence on detection performance, such as the alerting threshold and the peak size of the outbreak, the model suggested an important role for other algorithm features, such as adjustment for weekly patterns.We developed a model that accurately predicts how characteristics of disease outbreaks and detection methods will influence on detection. This model can be used to compare the performance of detection methods under different surveillance scenarios, to gain insight into which characteristics of outbreaks and biosurveillance algorithms drive detection performance, and to guide the configuration of surveillance systems. The past decade has seen the emergence of diseases caused by previously unrecognized threats and the sudden appearance of known diseases in new environments. Consequently, infectious diseases continue to cause high human and financial costs. In order to prevent the spread of infectious diseases, early detection of disease outbreaks is crucial. One approach to early detection is to use automated syndromic surveillance systems, which monitor health-related data from different sources to detect potential disease outbreaks in a timely fashion.. Syndromic surveillance systems continuously apply statistical algorithms to large volumes of data generated through health-related behaviours (e.g. counts of Emergency Department visits) to detect anomalies and support investigation and control measures.. Many outbreak detection algorithms have been proposed for use in syndromic surveillance. While it is clear that algorithms perform differently when they are applied to different data sources or used in different surveillance situations, there is insufficient empirical evidence regarding the effectiveness of algorithms under different conditions. Such evidence could guide public health practitioners in the choice of surveillance systems algorithms and configurations . The few existing studies evaluating detection performance are based on data that are not publicly available, making evaluations difficult to generalize or replicate [1] . Moreover, the performance of detection algorithms is influenced by many factors, including the nature of the disease, characteristics of the outbreak signal (such as peak size and intensity), baseline data (such as weekly mean and standard deviation), and parameters of the detection method used (such as alerting threshold). Some researchers [2] argue that the lack of a standardized framework for the assessment of outbreak detection methods and the diversity of factors that influence detection performance decreases the ability to compare detection methods.. The objective of this research is to develop and evaluate a model for quantitatively characterizing the determinants of outbreak detection performance and predicting the performance of detection methods. Earlier work [3] showed that it is possible to predict outbreak detection performance quantitatively with acceptable accuracy. That research developed a prediction model based on logistic regression, which assumes a multiplicative relationship between variables. While this model predicted the detection performance of the algorithms with reasonable accuracy, it could not model complex dependencies between variables and their relationship with multiple performance metrics. This limitation was due mainly to the nature of logistic regression, which implements a flat, linear model. In previous work [4] , we assessed the feasibility of addressing this limitation by developing a Bayesian network model using data generated thorough simulation. Many different algorithms could be used to model detection performance, such as support vector machines (SVM) and random forests. However, we chose to use a graphical model because it has the advantage of not only providing a prediction of performance, but also providing a representation of the different probabilistic dependencies between outbreak and algorithm characteristics, on one hand, and performance, on the other hand. This information can be useful when trying to understand which factors influence the ability of an outbreak detection algorithm to detect a type of outbreak accurately and in a timely manner. This paper significantly advances our prior work by combining outbreak data generated by a realistic simulation model with real healthcare utilization data and then evaluating the performance of a wider range of commonly used biosurveillance algorithms. The resulting dataset is used to build and evaluate a Bayesian network model for predicting detection performance. The developed Bayesian network can be used for predicting how well different outbreak detection methods will perform under different circumstances. We illustrate a variety of outbreak scenarios and use inference in the learned Bayesian network to find the best settings for detection methods and predict the detection performance in those scenarios.. While the Bayesian network is built using simulation data, it has two major advantages as a predictor over simply querying the simulation results. On one hand, the Bayesian network is efficient to query when new algorithms or scenarios need to be tested (As opposed to running an expensive simulation). On the other hand, the Bayesian network generalizes the information from the simulation data, allowing queries for outbreak characteristics and surveillance algorithm traits that have not been simulated. A secondary effect of the generalization is to smooth out noise and possible outliers in the simulation data.. The proposed framework for performance evaluation of outbreak detection methods under a wide variety of outbreak circumstances is general and can be used in further studies. We note that while we use the SnAP simulation platform developed at McGill, the same methodology can be used with other count data, provided through alternative simulation methods. We anticipate that the model for predicting detection performance can be used to develop new biosurveillance methods by identifying ideal algorithms characteristics, which may not exist in any currently available algorithms. However, building a new detection method is beyond the scope of this paper.. The structure of this paper is as follows: in the background section, we review the outbreak detection methods used in our study and describe common measures of detection performance. In the Methods section, we describe our simulated surveillance data for waterborne disease outbreaks used in this study and the development and evaluation of our Bayesian network model. In the Results section, we present the accuracy of our model for predicting detection performance and we illustrate its capability to identify factors that influence outbreak detection performance. We also present examples of how the model can be used in practical scenarios. We close with a discussion of the results, concluding remarks, and directions for future work. In this study, we analyzed outbreak detection performance for a range of algorithms that are commonly used in public health practice, considering a range of features related to the outbreak characteristics, baseline data, and the parameter settings for the detection methods. We assessed the performance of seven different outbreak detection algorithms using simulated and real surveillance data for GI outbreaks in eighteen outbreak scenarios and trained Bayesian networks to model the relationships between all surveillance attributes and the detection performance. Our evaluation results show that even when the outbreak characteristics were unknown a priori, the model was able to predict detection performance with high accuracy (AUC = 0.86).. The Bayesian network model developed in this paper allows quantifying the effect of outbreak characteristics and algorithm configurations on the performance of detection algorithms. As expected, the most informative determinants of detection performance were the alerting threshold, which is a parameter of the detection method, and the contamination level and the peak size of the outbreak. But our model also quantified the contribution of other algorithm features such as accounting for day-of-week and maintaining a guardband or memory. We demonstrated how inference performed using our model can help to develop what-if analyses for using detection methods in practice, or to find an appropriate algorithm configuration given the desired level of detection performance for outbreak scenarios. Such an inferential tool gives insight into the features of detection methods that are important to provide better performance. We also described how the model can be utilized to predict the expected performance of detection methods in different surveillance situations.. One limitation in comparing surveillance methods is the lack of data for benchmarking. This limitation was addressed in our work by using simulated data. Our approach is similar in spirit to the research reported by Lewis and colleagues [25] , who used simulation of influenza outbreaks to evaluate spatiotemporal outbreak detection methods. However, they did not quantify the effects of algorithm parameters on detection performance, and we believe this is an important contribution of our research.. Our approach can be extended to allow a coherent evaluation of new algorithms and new data sources as needed. In particular, using our current model, we can evaluate outbreak detection performance for new algorithms different than the C, W, and Adaptive Poisson algorithms. Any configuration of considered parameters in our model different than the ones belonging to these algorithms can be thought of as a new detection method and can also be evaluated.. We presented several scenarios of outbreaks and desired performance, and used inference to suggest the best algorithm and parameter setting to use, as well as to quantify the expected performance. These scenarios are by no means exhaustive, and are meant as examples of what kinds of inference can be performed. Of course, in order to use this model as a tool for what-if analysis in the public health sector, an adequate interface would also need to be developed, but this goes beyond the scope of our work.. We used emergency department visits as the baseline time series for outbreak detection. In recent years, nontraditional data sources have been introduced in public health and surveillance systems. This includes mobile phone data [26] , social data [27] , micro-blogging [28] , Twitter feeds and Google search queries [29] . While introducing and combining new data sources, especially in the era of big data, are promising directions for research in biosurveillance systems, the evaluation of their relevance and significance will be extremely important. Evaluation studies such as [30] are needed to compare these new data sources to the existing ones, and the approach that we describe could be used to consider the relative contribution to detection performance of data sources and algorithms.. A number of extensions to this work may improve the generalizability of the results. We used simulated outbreaks superimposed on real surveillance data; therefore, the results are affected by the quality of the simulation. Our approach can be extended by including more detection methods, using spatio-temporal data simulations, and considering health care utilization data sources in addition to ED visits. In public health practice, many approaches are used analyze time series of healthcare utilization records with the goal of detecting disease outbreaks. In this paper, we use a popular set of detection methods based on statistical process control charts, the C-family of detection algorithms [5] and Adaptive Poisson Regression. C1, C2, and C3 are adaptive algorithms included in the Early Aberration Reporting System (EARS) developed by the Centre of Disease Control and Prevention (CDC). The C-algorithms assume that the expected value of the time series for the given time t is the mean of the values observed in a sliding window. If the difference between the observed value at a given time t and the mean of the window divided by the standard deviation of the window is bigger than a threshold, an unusual event is flagged and the possibility of a disease outbreak is signalled.. The C-algorithms are distinguished by the configuration of two parameters: the guardband and the memory.. Gradually increasing outbreaks can bias the test statistic upward, so the detection algorithm may fail to flag the outbreak. To avoid this situation, C2 and C3 use a 2-day gap, called a guardband, between the sliding window and the test interval. C3 includes 2 recent observations in the computation of test statistic at time t, which is called memory. In the EARS system, the size of the window used for the calculation of the expected value is 7 days; however, this parameter can be varied. Detection algorithms can be configured using various alerting thresholds, which result in different sensitivity and false alarm rates.. Most surveillance tasks based on health care utilization are affected by weekly patterns. Many health-care facilities have fewer visits during weekends and there is a sharp increase in the number of visits on Mondays, which should not be considered an outbreak. The W2 algorithm is a modified version of C2 that takes weekly patterns into account [6] , by stratifying the baseline data into two distinct baselines: one for weekdays, the other for weekends.. The W3 algorithm is the similar counterpart of the C3 algorithm.. Another outbreak detection method, called Adaptive Poisson Regression, assumes that the distribution of health care utilization counts in the surveillance time series is Poisson and uses categorical variables to represent trends and patterns. Xing described Adaptive Poisson Regression, which uses a sliding window of 56 days for estimating the regression coefficients and alerting threshold [7] . The logarithmic link function estimates the expected value at time t as:. where c 0 is a constant intercept, the term ሾܿ ଵ × ݓ݀ ௦ ሺݐሻሿ captures the day-of-week effect, and the term ሾܿ ଶ × ݕܽ݀41 ௦ ሺݐሻሿ represents the current seasonal trends in cycles of 14 days. The Poisson regression algorithm is adaptive to recent changes in the data and the algorithm parameters. A 2-day guardband can be used to avoid the contamination of the sliding window and test interval.. We note that many other surveillance methods can be employed, such as the Shewhart control Chart [8] , CUSUM [9] , Exponential Weighted Moving Average algorithms (EWMA) [10] , the Shiryaew-Roberts method [11] , and likelihood ratio-based methods [12] . More details on these methods can be found in [13] and [14] . Our work does not include these algorithms at the moment, because doing an exhaustive study would be too time consuming. As a result, we picked approaches that seem to be used most extensively in practical surveillance settings. However, it would be conceptually straightforward to extend our framework to include other biosurveillance algorithms and their characteristics.. The performance of outbreak detection algorithms is evaluated in terms of the specificity and timeliness of detection.. Specificity is the probability that no alert will be given when no outbreak has occurred [15] . It is calculated as:. where ݊ሺ݈ܽܽ݉ݎ = 0, ݇ܽ݁ݎܾݐݑ = 0ሻ is the number of non-outbreak days in which the algorithm does not raise an alarm and ݇ܽ݁ݎܾݐݑ݊ሺ = 0ሻ is the number of non-outbreak days in an analysis interval.. Timeliness is the proportion of time saved by detection relative to the onset of an outbreak ݐ( ௦௧ ሻ. If an outbreak is detected, timeliness is defined as:. where ݊݅ݐܽݎݑܦ݇ܽ݁ݎܾݐݑ is the number of days for which outbreak cases occur. The ݐ ௗ௧௧ is the index of the day within the time series when the outbreak is detected and ݐ ௦௧ is the index of the day on which the outbreak starts. The proportion of delay is subtracted from 1, so higher values denote an earlier detection of the outbreak.. Timeliness is 1 if the outbreak is detected on the first day and 0 when the outbreak is not detected at all [15] .. In this paper, we measure the detection rate as a performance metric. Detection is a binary variable which indicates whether each outbreak is detected or not. It can be defined as sensitivity per outbreak where sensitivity is the probability of raising an alarm given that an outbreak occurred:. As there is only one outbreak in every time series generated by the simulator, the detection will be 1 if the outbreak is detected and 0 if the algorithm does not trigger any alert. We focus on detection rate because it measurs the overall ability of algorithms to detect different types of outbreaks. We are interested in predicting how the detection rate changes in different surveillance settings and under different algorithm parameter values. Using administrative data and simulated outbreaks, we created a data set for performance benchmarking of a number of detection algorithms and used a Bayesian network to model the performance of these algorithms.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


WBC-ALC: A Weak Blocking Coordinated Application-Level Checkpointing for MPI Programs As supercomputers increase in size, the mean time between failures (MTBF) of a system becomes shorter, and the reliability problem of supercomputers becomes more and more serious. MPI is currently the de facto standard used to build high-performance applications, and researches on the fault tolerance methods of MPI are always hot topics. However, due to the characteristics of MPI programs, most current checkpointing methods for MPI programs need to modify the MPI library (even operating system), or implement a complicated protocol by logging lots of messages. In this paper, we carry forward the idea of Application-Level Checkpointing (ALC). Based on the general fact that programmers are familiar with the communication characteristics of applications, we have developed BC-ALC, a new portable blocking coordinated ALC for MPI programs. BC-ALC neither modifies the MPI library (even operating system) nor logs any message. It implements coordination only by the Barrier operations instead of any complicated protocol. Furthermore, in order to reduce the cost of fault-tolerance, we reduce the synchronization range of the barrier, and design WBC-ALC, a weak blocking coordinated ALC utilizing group synchronization instead of global synchronization based on the communication relationship between processes. We also propose a faulttolerance framework developed on top of WBC-ALC and discuss an implementation of it. Experimental results on NPB3.3-MPI benchmarks validate BC-ALC and WBC-ALC, and show that compared with BC-ALC, the average coordination time and the average backup time of a single checkpoint in WBC-ALC are reduced by 44.5% and 5.7% respectively. In recent years, with the performance improvement of supercomputers, the reliability problem has become more and more serious. On the one hand, with the development of manufacturing technology, the single chip integrates more and more transistors, and it is more vulnerable to transient faults because of high temperature [1] ; on the other hand, in order to obtain higher performance, the scale of supercomputers keeps growing. For example, the top ten supercomputers in the top500 list of Jun. 2011 have more than 100 k nodes (cores) on average [2] . However, the reliability of a parallel computing system is inversely proportional to its degree of parallelism [3] . Some researchers have pointed out that the mean time between failures (MTBF) of future Exascale supercomputers will reduce to days or even hours. However, in order to improve the precision of results, some applications need to run several days even on the fastest su- percomputer [4] , and the execution times of these applications are much longer than MTBF. So fault tolerance methods are necessary for guaranteeing the execution of these applications.. In the distributed and parallel computing area, Checkpoint/Restart (C/R), i.e., checkpointing methods are widely used fault tolerance methods [5] , [6] . These methods periodically save the checkpoint data, namely the states of the computation, to stable storage. If some failure happens in system, the program will roll back to an appropriate checkpoint, and the computation is restarted after recovering the state. Generally, checkpointing can be classified along two orthogonal dimensions. Firstly, according to the level of saving states, checkpointing methods can be classified into two kinds: Systemlevel checkpointing (SLC) and Application-level checkpointing (ALC) [7] . SLC stores all the states of the whole system periodically. It is a transparent method for the programmer, but its implementation relies on the concrete hardware and software environments. ALC reduces the faulttolerance overheads with the help of programmers, who appoint the location and data of checkpoint. Checkpointing function codes are inserted into the original program, so ALC can satisfy the requirements of different hardware and operation systems.. Secondly, according to the manner of maintaining consistency, checkpointing methods can be classified into coordinated checkpointing and uncoordinated checkpointing [8] . Coordinated checkpointing needs to harmonize all processes to generate a consistent global state. It simplifies the recovery procedure, but its overhead expands as the system scale grows. In uncoordinated checkpointing, each process saves its checkpoint data independently, so each process can save its state at a convenient time. However, uncoordinated checkpointing may lead to the domino effect [9] . Furthermore, according to the manner of coordination, coordinated checkpointing can be classified into blocking coordinated checkpointing and non-blocking coordinated checkpointing. Blocking techniques bring all processes to a stop before taking a global checkpoint. Non-blocking coordinated checkpointing implements coordination by exchanging special markers or control tokens between processes. In order to form a global checkpoint, a global coordination protocol is used to orchestrate the saving of the states of individual processes and the contents of certain messages.. The Message Passing Interface (MPI) [10] is currently the de facto standard used to build high-performance appliCopyright c 2012 The Institute of Electronics, Information and Communication Engineers cations. How to perform fault-tolerance for MPI programs with C/R is always a hot topic. Many researches [11] - [14] have designed and optimized the checkpointing for MPI programs from different points of view. Nonetheless, in order to get a global consistency checkpoint, the methods above either modify the MPI library (even operating system) or implement a coordination protocol by logging lots of messages. These methods are complicated to implement and will bring heavy fault-tolerance overheads. Based on the ideas of ALC and blocking coordinated checkpointing, this paper develops BC-ALC, a blocking coordinated ALC for MPI programs. This method need not modify the MPI library or operating system, and programmers only need to appoint the positions of checkpoints in an MPI program. A global consistency checkpoint can be obtained by synchronization, implemented by Barrier operations, without any complicated coordination protocol.. The classical blocking coordinations are usually implemented by global barriers. However, the synchronization overhead of global barriers is very expensive, and all processes must backup data together after the global barrier, which will exacerbate the pressure on I/O bandwidth. So the blocking coordinated checkpointing based on global barrier is considered as a poor scalable checkpointing method. In order to reduce the overheads of synchronization and backup, based on the analysis of the communication characteristics of MPI programs, this paper develops WBC-ALC, a weak blocking coordinated ALC using group barrier to implement synchronization.. To the best of our knowledge, this is the first work investigating the development of checkpointing methods for MPI programs without modifying MPI library or logging MPI messages. The contributions of this paper are summarized below:. • We introduce BC-ALC, a new portable blocking coordinated ALC without modifying MPI library or logging any message, for MPI programs.. • We design WBC-ALC, a weak blocking coordinated ALC using group barrier instead of global barrier to implement synchronization for MPI programs to reduce fault-tolerance overheads.. • We present a WBC-ALC-based fault-tolerance framework for providing reliability guarantees for MPI programs, and describe an implementation of this framework. The rest of the paper is organized as follows. Section 2 introduces the blocking coordinated ALC method for MPI programs to overcome the difficulties in ALC of MPI programs. Section 3 reduces the synchronization range of BC-ALC, and introduces the basic idea and synchronization mechanism of WBC-ALC. Section 4 describes a faulttolerance framework developed based on WBC-ALC for MPI programs. Section 5 presents an implementation of WBC-ALC. Section 6 describes our evaluation methodology, demonstrates the correctness of BC-ALC and the effectiveness of WBC-ALC. Section 7 reviews the related work. Finally, Sect. 8 concludes the paper. For the difficulties in ALC of MPI programs, we introduce BC-ALC, a new portable blocking coordinated ALC for MPI programs, and with the reduction of synchronization range in blocking coordination, we develop WBC-ALC. Based on this new method, we have developed a compilerdirected fault tolerance framework for MPI programs and an implementation for it. Our experimental results obtained on NPB3.3-MPI benchmarks demonstrate that programmers can use BC-ALC and WBC-ALC easily, and these two methods are valid. Compared to BC-ALC, WBC-ALC can reduce the fault-tolerance overhead effectively. In order to implement blocking coordinated checkpointing, hardware blocking was used on the IBM SP-2 to take system-level checkpoints; Software blocking techniques exploit barriers -when processes reach a global barrier, each one saves its own state on stable storage [13] . Blocking coordinated checkpointing with global barrier has been well used in OpenMP programs [20] . However, this checkpointing method has not been used in MPI programs.. In coordinated checkpointing, the procedure of coordination is costly. In order to reduce this cost, minimal checkpoint coordination implements a protocol to reduce the range of coordination by only harmonizing the processes which it has communicated with since the last checkpoint [21] . Xavier Besseron developed an optimized coordinated protocol using a dataflow graph model for KAAPI applications [22] . In this optimized coordinated protocol, a process only synchronizes with its communication objects based on the dataflow graph. However, this protocol is only used for KAAPI applications.. For send-deterministic MPI applications, [11] developed an uncoordinated checkpointing to avoid domino effect by implementing a complicated protocol based on the characteristic of send-deterministic. However, most of the researches of checkpointing for MPI programs are focusing on coordinated checkpointing. [12] compared blocking with non-blocking coordinated checkpointing for largescale fault tolerant MPI programs. The authors found out that for high-speed networks, the blocking implementation gives the best performance for sensible checkpoint frequency.. The above researches of checkpointing for MPI are all SLC, which must modify the MPI library or operating system. Bronevetsky developed a automated ALC for MPI programs [13] . A coordination layer is added between the application and the MPI library, and a coordination protocol is implemented. Bronevetsky also improved the fault-tolerance method for collective operations in MPI programs [14] . However, Bronevetsky's methods must intercept all calls to the MPI library and save lots of information of MPI library. We demonstrate the validation of WBC-ALC (BC-ALC) and the superiority of WBC-ALC over BC-ALC using the MPI-version NPB benchmarks. Section 6.1 introduces the benchmarks and experiment platform used. Section 6.2 describes the methodology used for evaluating this work. Section 6.3 presents and analyzes our results.

O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O B O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O