Genesifter, leading the blind

Dear Sir, Properly dealing with DNA microarray data involves employing statistical methods beyond that which we apply to regular experimental findings. For example, inferring statistically significant changes in gene expression requires taking into account the multiple testing problem inherent to microarray-based experiments (Allison et al., 2006). Ignoring this aspect alone has led to significant problems in reproducing results from several microarray-based melanoma research programs (Hoek, 2007). Therefore, it is no surprise that user-friendly softwares promising to deal appropriately with microarray data are gaining popularity. Such tools aim to eliminate the need for laboratories to contract specialist staff and, if properly executed, have the potential to rapidly expedite the translation of high-throughput data into useful hypotheses. One on-line data analysis tool is GENESIFTER (http:// www.genesifter.net) which is produced by VizX laboratories and marketed towards biologists who are having difficulty coming to grips with very large DNA microarray data sets. The GENESIFTER layout is relatively straightforward and there is a section of the website where one can try out the software (following helpful tips in the margins) free of charge using publicly accessible data sets. One simply uploads data and (prompted appropriately) applies the desired algorithms. The results are shown in ranked tabular form and may be readily downloaded. Currently, the going subscription rate is $4000 a year which is substantially cheaper than hiring a dedicated biostatistician. With this model it would be no surprise that GENESIFTER is, as it claims, ‘the world’s fastest growing gene expression analysis system. ’ However, there are critical problems with GENESIFTER’s pairwise analysis algorithm as it commits at least two errors in data handling: the insufficient normalization of data and the introduction of selection bias while calculating significance. During the course of microarray experimentation, errors in measurement can arise from (just to name a few) between-sample differences in the quantity of RNA, labelling efficiency, errors in platform manufacture and systematic measurement bias. To control for these errors requires normalization of the raw data so that chip experiments may be comparable (Quackenbush, 2002). There are several ways of going about this but GENESIFTER’s method is to conduct normalization so that differences between chips are not accounted for. Instead, normalization is conducted only within each chip such that every raw data point is divided by the median (or mean) of all data points on the chip. This half-way approach, which omits an additional division of each data point by the median (or mean) of its performance across different chips to account for betweensample error, makes the inadvisable assumption that there is no significant between-chip variation to account for. Inferring significance for a gene’s differential expression between sample classes involves performing a statistical test. For DNA microarray analyses, a critical aspect requires taking into account the multiple testing problem. Each probe on a microarray platform represents an individual experiment, and due to biological and systematic variation the performance of thousands of such experiments ensures that many apparently significant results will be the result of chance events (false positives). To control this various multiple testing correction measures can be applied. GENESIFTER allows users to choose a statistical test and apply a suitable multiple testing correction. Additionally, users may apply a foldchange filter to further narrow down the list of genes with significantly changed expression. However, including the fold-change filter with statistical testing is problematic, because GENESIFTER applies the fold-change filter to averaged data before performing statistical tests. This practise is a form of selection bias and serves to increase false positives in the data when statistical testing is complete. When seeking to identify which of a population of genes undergo significant changes between sample classes, it is important to remember that the entire available population of genes must be subject to statistical analysis. By first cherry-picking for genes which (on average) show a fold-change between sample classes, subsequent testing is biased towards incorrectly identifying outcomes as having rejected the