Performance indicators Sensitivity(%) Timeliness(Day) EWMA CUSUM MPM* EWMA CUSUM MPM* EWMA CUSUM MPM False alarm rate(%) 56.02 58.72 69.71 2.40 2.52 1.65 5.09 5.85 4.93 53.08-58.95 55.81-61.63 66.46-72.96 2.20-2.62 2.31-2.73 1.45-1.85 4.39-5.78 5.08-6.63 4.28-5.58 22.04 <0.001 20.28 <0.001 1.88 0.153 * p < 0.001.According to Bonferroni’s procedure, the family wise error rate was 0.05,divided by the number of test. The significance level αfor an individual test was 0.05/3,being 0.017. outbreaks that an algorithm detected within several days from the start of the outbreak. Statistical inference and multiple linear regression For each of the eight diseases in each of the ten counties on each of the four out- break magnitudes, we computed the sensitivity, timeliness and false alarm rate across all 320 analysis runs. We used ANOVA to test for signifi- cant difference in the algorithms’ sensitivities and timeli- ness. The Bonferroni correction was applied for multiple comparisons to control the family wise error rate. The significance level α was 0.05. Finally, multiple linear re- gression was run to understand the relationship of the algorithms sensitivities and timeliness with disease incu- bation, baseline counts and outbreak magnitude. All the analyses were performed using R software. Kuang et al. BMC Public Health 2012, 12:418 http://www.biomedcentral.com/1471-2458/12/418 Table 3 Summary the performance of outbreak detection algorithms Mean 95% confidence F value P -value interval Page 5 of 9 Results Algorithm performance The respective proportion of detected outbreak signals of EWMA, CUSUM, MPM were 56.02%, 58.72%, 69.71%. Overall, the MPM was more sensitive than EWMA, CUSUM. In timeliness, the average lag days of all outbreak signals were: EWMA 2.40, CUSUM 2.52 and MPM 1.65. Compared with EWMA and CUSUM, the MPM needed shorter time to detect outbreak signals (P <0.001). The false alarm rate of EWMA, CUSUM and MPM were 5.09, 5.85 and 4.93. The false alarm rate did not differ in three algorithms (P = 0.153) (Table 3). Sensitivity-timeliness plot Figures 4, 5, 6 were sensitivity-timeliness plot to com- pare three algorithms performance in different incuba- tion categories, baseline counts levels and outbreak magnitudes. In the three incubation categories, the MPM had a higher probability of detecting the outbreak than CUSUM and EWMA within all days from the start of the outbreak; the sensitivity changes of CUSUM and EWMA over time were very close. In short incubation, the sensitivity of MPM reached 90% at the sixth day of a outbreaks start. The long incubation disease had a poorer sensitivity than short and medium incubation disease (Figure 5). We used the 33.33, 66.67 percentiles with the cuts off 0.05, 0.2 to divide average baseline counts into low, medium and high levels. At the three levels, the MPM had a higher probability of detecting the outbreak than CUSUM and EWMA within all days from the start of the outbreak; the sensitivity of CUSUM and EWMA was very closely changed over time. The sensitivity of low baseline counts was better than medium and high levels, with the MPM reaching higher than 90% (Figure 6). In the four outbreak magnitudes, the MPM had a higher probability of detecting outbreak than CUSUM Figure 5 Sensitivity versus timeliness for three detection algorithms in three incubation categories. EWMA, exponentially weighted moving average; CUSUM, cumulative sum; MPM, moving percentile method.