e105 Nucleic Acids Research, 2010, Vol. 38, No. 9 cause real CNVs to merge with regions of noise, creating CNVs larger than simulated especially for simulated CNVs of 10-15 SNPs. The Nexus Rank method also had larger CNVs with larger variation, averaging two SNPs larger than the model in simulation 2 and one SNP larger than the model in simulation 1. Both the CBS and Nexus programs are based on seg- mentation methods and have similar rankings in terms of performance. The CBS program had strong boundary calling, but had trouble detecting CNVs that were detected by the majority of CNV detection methods. The Nexus Rank and Nexus SNPRank programs are two commercially available segmentation algorithms that are well-documented and user-friendly for CNV analyses and provide visualization of the results in a biological context. While our evaluation indicators such as NSR and ROC curve residuals were promising for the Nexus methods, there are concerns about these methods. Rank’s ROC residuals were inﬂated by the abundant number of calls as reﬂected in simulation 2 with a false positive rate of 44% at the CNV level and in the analysis dataset with the lowest speciﬁcity of all methods. Rank also has boundary calling problems, with most CNVs called 1–2 SNPs too long and 3–4 SNP CNVs called as 6–7 SNP CNVs. Both Nexus methods have diﬃculty achieving suﬃcient power at moderate false positive rates. For example, in simulation 2 and the neutral and Aﬀymetrix simulations, SNPRank can only achieve moderate power while Rank only has high power with a high false positive rate. The correlation between the results at the CNV and CNV SNP levels are inﬂuenced by the boundary calling properties of the method and by the statistical properties of the speciﬁcity calculation at the two levels. For example, in simulation 1, QuantiSNP’s inaccuracies in boundary calling give a moderately high-false positive rate at the CNV SNP level, while at the CNV level there are zero false positives. This is one factor contributing to QuantiSNP’s diﬀerent optimal parameters for the two levels. The second factor is the speciﬁcity calculation. For example, because a single false positive CNV has more impact than a single false positive CNV SNP on both false positive rate and sensitivity, Rank’s overabun- dant CNV calls made the optimal parameter for CNV SNPs (1E–3) less stringent than the one for CNVs (1E–4). Is there an inherent advantage for QuantiSNP, because it was developed for Illumina SNP arrays, while CBS, GLAD and CNVFinder were developed for other plat- forms? A deﬁnite answer cannot be given here, because testing on real data from multiple platforms was not done here. QuantiSNP could have this advantage, but there are other possibilities. One possibility is that methods developed for SNP array data, like HMMs, are more sophisticated than the methods developed for other platforms, like segmentation and SD-based methods, which would indicate that the advantage of QuantiSNP is method based instead of platform based. Another pos- sibility is that the inherent quality of QuantiSNP, regard- less of platform or method type, is higher than that of other methods. There is evidence that the latter is true. First, CBS ranks higher than PennCNV in the analysis, PAGE 12 OF 14 simulation 2, Aﬀymetrix simulation and neutral simula- tion datasets. This shows that not all HMM methods are superior to segmentation methods like CBS and Nexus. Furthermore, this shows that methods not developed for SNP arrays can outperform methods created for SNP arrays. Second, CBS both has good per- formance in this SNP array study, and it also had the best performance in the Lai et al. 2005 (10) CGH study, giving additional evidence for platform-independent ranking performance. The PennCNV program is a HMM-based algorithm like the QuantiSNP program and is probably the most frequently used program for CNV studies in recent publi- cations (14–16). This is in part due to the user-friendly design of the program and free access to users. However, our evaluation ranked the performance of PennCNV at the intermediate level. Simulation 2 showed that the PennCNV program has moderate power. However, its low false positive rate is a promising aspect. The NSR and ROC curve residuals were moderate on real data. It generally detects less CNVs than other methods except cnvPartition in the analysis dataset. It also has some trouble detecting small CNVs (three to four SNP CNVs), which is exacerbated by using stricter than optimal parameters used in our study. These observations serve as good references for the appli- cation of PennCNV. In tuning parameters, the NSR is a more straightfor- ward measure than using ROC curves. It is database inde- pendent and does not require the calculation of multiple statistics. As a database independent measure, the NSR does not have to take into account CNV database issues such as limited CNV studies in certain populations, largely unknown CNV population frequencies, and the fact that studies without dense marker spacing call a higher number of CNVs relative to real CNV counts. For instance, we found very low sensitivity between our population and the corresponding ethnic population in the Database of Genomic Variants (3). This is largely due to the large number of CNVs and the lack of CNV validation in DGV. The NSR is robust to the tested Illumina SNP array platforms, to gender bias, and to genotyping quality levels, since it was minimal at the same parameters on multiple values of these characteristics of the training dataset. Our study showed that the same conclusion can be drawn based on NSR and ROC curve residuals, which validates the usage of the NSR. In comparing method quality, the NSR overcomes the correlation between the number of CNV calls and sensitivity, speciﬁcity and kappa, which allows every method to be set at any param- eter setting. Since datasets vary in their log R ratio SDs, B allele frequency distributions and other factors, there is no single parameter setting for all datasets. Therefore, the determination of parameter setting for a CNV detection method can be a challenge. Without a priori knowledge, one may choose the default setting of the program, which may not be optimal. In this study, we proposed to stan- dardize the parameters in a small subset of samples before the full analysis. Since half of the methods tested take at least 15 minutes per sample, it is important to ﬁnd a