Gayán et al. BMC Genomics 2010, 11:403 http://www.biomedcentral.com/1471-2164/11/403 with a significance level of 10-3, and a three-replication- group analysis, with a significance level of 10-1 for each group. This second strategy was applied with and without the direction filter (consistency of direction of effect across groups). HFCC was employed to test for associa- tion between all two-locus SNP combinations and case- status (20 SNPs produce 190 two-locus combinations). For each bi-SNP combination, nine simple genetic mod- els were tested, where each test compares the frequency of one of the nine possible two-locus genotypes (AABB, AABb, AAbb, AaBB, AaBb, Aabb, aaBB, aaBb, and aabb) against the frequency of the remaining genotype classes. The simulations resulted in a Type I error rate of 0.00080 for the combined analysis, and 0.00086 for the 3- group analysis. The observed type I error rate is a bit con- servative compared to the theoretical rate (0.001), which is probably due to the fact that some of the genetic model tests are correlated. In any case, both analysis strategies are similar and conservative. Most importantly, when the direction filter is applied, the Type I error rate drops to 0.0002, much lower than for the two previous analyses, proving that the replication group analysis, when the direction filter is applied, reduces the false positive rate. We then analyzed three datasets simulated under three different effect sizes (Average OR of 1.67, 2.10, and 2.85). Table 1 summarizes these results, where we find that the combined analysis is, as expected and suggested in our original paper [4], always more powerful than the 3- group analyses. Nonetheless, and regarding Toplak et al.'s main claim [5], the false positive rate decreases when we Analysis Type Combined group (p = 10-3) 3 group No Direction (p = 10-1) 3 group Direction (p = 10-1) Combined group (p = 10-4) Power Power False positive rate False positive rate Power Power False positive rate False positive rate Page 3 of 5 use the 3-group strategy, and more so when we apply the direction filter. It has also been suggested [5] that the false positive rate can be reduced by increasing the significance level in a combined group analysis. The bottom row of Table 1 shows that a combined analysis using a significance level of 10-4 achieves a significance level similar than the 3- group strategy, and a greater power. But, of course, this is not a fair comparison: If we apply this new significance level with a 3-group strategy, we would obtain again a reduction in the false positive rate. A final comparison is reported in the last column of Table 1. We simulated a mixed population, where one replication group produces an association signal, while two other replication samples are simulated under the hypothesis of no association. We found that the com- bined group analysis detected the association 71 and 50 percent of the time, but the 3-group analysis did not detect it even once. In addition, the false positive rate was lowest for the 3-group analysis with direction filter. Discussion We have evaluated the power and Type I error rate of two possible analysis strategies: a combined group analysis of the full sample, and a replication group analysis. The results presented in Table 1 and in the Results section show that applying a replication group analysis with a direction filter reduces the false positive rate relative to a combined group analysis. It is important to note here that the combined analysis is more powerful, something 1.67 90 57 57 66 0.00379 0.00197 0.00121 0.00103 2.10 0.01144 0.00538 100 100 100 100 0.00486 0.00571 OR 2.85 100 0.02448 100 0.01713 100 100 0.01645 0.01824 1.57 Mixed 71 0.00297 0.00108 0 0 0.00039 50 0.00070 Table 1: Power and Type I error rates for several simulated datasets and analysis strategies. Each dataset consists of 3000 cases and 3000 controls genotyped for 20 SNPs, and each experimental dataset was simulated 100 times. The simulated effect size for each experiment (a pair of SNPs per simulation) had an average Odds-Ratio (OR) of 1.67, 2.10, and 2.85. A final mixed model (overall OR of 1.57) resulted from the combination of two simulations with a simulation with a pair of SNPs associated with case- status. The combined-group strategy analyzed the full sample size simultaneously (3000 cases and 3000 controls), and was applied for two different significance levels (p = 10-3 and p = 10-4). The 3-replication-group strategy analyzed three replication groups, each with 1000 cases and 1000 controls, with a significance level of p = 10-1 in each group. Results with and without applying a direction filter (consistency of results in size and direction of effect across replication groups) are shown. Power is expressed as a percentage.