GeneSifter, not so blind after all

Dear Sir, We are writing this letter in response to ‘GeneSifter, leading the blind,’ a letter published last fall (Hoek, 2008) about GeneSifter , a software product for analyzing data from microarray and next generation DNA sequencing experiments (http://www.geospiza.com). We wish to address the areas of concern described in the article: the methods used for data normalization and concerns regarding how fold-change cut-offs and statistical tests are applied. Hoek raises valid points regarding both the need for between chip normalization and that the order of filtering steps can have dramatic affects on the control for false positives; however, we maintain that Hoek’s criticisms were based on an incomplete knowledge of GeneSifter features, capabilities and proper operation. We believe this misunderstanding most likely resulted from a failure on the part of VizXLabs (the previous owners of GeneSifter) to adequately respond to Hoek’s concerns. As the new owners of GeneSifter, we would like to address the criticisms that Hoek raised. Hoek pointed out the necessity for normalizing microarray measurements both within a chip and between multiple chips and wrote that GeneSifter fails to account for the differences between chips when normalizing data (Hoek, 2008). It’s probable that Hoek missed seeing GeneSifter’s options for normalization because they are not visible from the pairwise analysis interface. GeneSifter does include methods for normalizing data within chips and between chips, but also gives users a choice to upload normalized data without further processing; this option allows users to circumvent errors that would result from re-processing already normalized data. The normalization options within GeneSifter include methods such as RMA (Robust Microarray Analysis) and GC-RMA (Irizarry et al., 2003; Wu et al., 2004) for normalizing data between multiple Affymetrix chips. It should be noted, however, that these methods must be applied at the time of loading the data rather than at the time of analysis. Consequently, these methods are not available through the pairwise analysis interface and could be missed by someone who is new to the software. We believe these details were not communicated to Hoek, leaving him with the impression that these more robust normalization methods are missing from the GeneSifter package. Hoek’s second concern was with the order in which a fold-change cut-off is applied and the affects on corrections for multiple testing. Hoek stated that the order of statistical operations in GeneSifter was incorrect because users could apply a fold-change filter to averaged data before performing statistical tests and that users were left unable to change the options or perform steps in the appropriate manner, resulting in weak control for false positives. While it is true that this ordering can be used in GeneSifter, the product has always allowed the statistical test and corrections to be performed prior to applying a fold-change cut-off. As with the normalization concerns, we believe this misunderstanding arose from a failure to communicate the options available in GeneSifter. The default settings for pairwise statistics did perform a threshold cut-off first and then apply a correction based on the number of genes that passed the initial cut-off. These settings were used to facilitate discovery by decreasing the possibility of false negatives. In contrast to the statement in the article; however, GeneSifter users have always had the option to change the order of steps by using the preference settings in the GeneSifter program. The last point we wish to address is the comparison between data analyzed with GeneSifter and the same data analyzed with GeneSpring (Agilent Technologies). Hoek analyzed a publicly available colon cancer data set (GEO accession no. GDS756) with both GeneSifter and GeneSpring. The GeneSpring analysis yielded 449 genes with significant differences in expression where analyzing the data with GeneSifter produced a list of 1556 genes. This difference however, did not result from a true difference in the software platforms, the difference in gene number resulted because different methods were used for the analyses. When we use the same analysis procedure with GeneSifter, that Hoek used with GeneSpring, we obtain similar numbers of significantly expressed genes. Since the publication of Hoek’s letter; we have made changes in the default settings in the pairwise analysis interface to reduce the likelihood that users would inadvertently lower their control for false positives. Users still have the option however, to change their preferences settings and reverse the order should the need arise.