Gene–Environment Interactions: How Many False Positives?
Authors: Giuseppe Matullo, Marianne Berwick, Paolo Vineis
Journal: JNCI Journal of the National Cancer Institute (2005)

In this issue of the Journal, Hung et al. ( 1 ) describe the results of a large study on the association between base excision repair gene polymorphisms and lung cancer risk. Their study demonstrates problems typical of investigations of gene – environment interactions, in particular, the fact that among the generally negative results, some seemingly noteworthy associations are identifi ed in subgroups of subjects who are defi ned on the basis of their tumor histology or smoking habits. The unique aspect of this study is that the authors have estimated the probability that the associations found are attributable to chance (i.e., false positives). The idea of evaluating new observations in the light of existing evidence is not novel and belongs to at least two traditions: Bayesian statistics and clinical epidemiology. In clinical epidemiology, formal treatment of “proof ” from prior evidence is updated into posterior probability according to Bayes’ theorem. This approach has proved extremely useful because ( a ) it has revealed that diagnosis (and, by extension, clinical and scientifi c reasoning) develops in a sequence of steps that incorporate explicitly or implicitly previous knowledge ( 2 ) and ( b ) it has trained physicians to assign probabilities to the different components of such reasoning, namely, prior knowledge and “likelihood,” or the probability, of a positive test result in the presence of disease. Wacholder et al. ( 3 ) have proposed these same approaches to assess the validity of the increasing number of associations that are being identifi ed among genetic variants, environmental exposures, and disease (i.e., gene – environment and gene – gene interactions). A large number of such associations have been reported and even more are expected to emerge in the future. Tens of thousands of single nucleotide polymorphisms (SNPs) are or will be investigated for their association with cancer, and many of the observed results will be false positives. The challenge is to distinguish the false-positive associations from the true positives. Wacholder et al. ( 3 ) proposed a simple Bayesian approach that is based on the estimation of a prior probability and the calculation of posterior probability. The prior probability can be estimated from results of previous studies, from biochemical or molecular information (e.g., gene expression data) that supports the function of a SNP, or from other types of evidence such as sequence homology ( 4 ) . The general idea is to weight a new observation with the available prior evidence to derive a posterior probability. Recently, in an extension of the model proposed by Wacholder et al. ( 3 ) , Ioannidis ( 5 ) has incorporated selective reporting and other biases and the fact that specifi c hypotheses, including gene – disease associations and gene – environment interactions, are usually tested by many teams worldwide. By taking these parameters into account when scrutinizing all the evidence, one can show that very few identifi ed gene – disease associations and gene – environment interactions are probably real. The approach proposed by Wacholder et al. is ingenious and more convincing than other approaches, such as those based on Bonferroni’s correction for multiple comparisons or similar statistical methods ( 6 , 7 ) . Although the study by Hung et al. is a clever application of the proposal by Wacholder et al., execution of that proposal is not free from problems. The main problem is how best to calculate the prior probabilities. First, there is often no information available to use. Second, any available information may not be easy to evaluate, either because it is indirect (for example, from sequence homology) or because it is contradictory. The latter is very often the case with the epidemiologic evidence of gene – environment interactions (see below). Third, methods to quantify prior probabilities are not available, even when we have good evidence. The attempt made by Hung et al. is a good step forward, but it is still imperfect. The authors have considered and applied fi ve categories of prior probability (from 50% to 0.1%); however, the basis for these numbers and categories is unclear. A further step forward might consist of using results of welldone meta-analyses or pooled analyses of the available studies. In this case, we have good examples from the clinical literature: a good meta-analysis can allow clinicians to judge how much a new study changes the estimate of effi cacy of a drug. For example, if a meta-analysis provides an odds ratio of 0.9 and a new study reports an odds ratio of 0.2, then the real contribution of the new study to the advancement of knowledge will be judged by its ability to change the odds ratio provided by the meta-analysis. This ability will depend on both qualitative considerations (such as potential biases and the response rate) and the size of the study. Metaanalyses can be extremely useful for obtaining prior estimates against which the new evidence can be challenged because they include ( a ) an overall odds ratio from virtually all the available studies, ( b ) an evaluation of the quality of the studies (a quality score is usually assigned to each study included in a metaanalysis), ( c ) a weight that depends on the study size, and ( d ) a measure of heterogeneity. In a meta-analysis, a large and well-conducted positive study can thus outweigh several small negative studies. Figure 1 shows an example of a meta-analysis of some of the genes included in the Hung et al. article that we performed using data on gene–environment interactions, with a specifi c focus on DNA repair polymorphisms (summarized at http://perseus.isi.it/ huge [last accessed: March 24, 2005]). A network of investigative groups involved in human genome epidemiology research has been developed to combine data from many studies to overcome the defi ciencies of currently available, relatively small datasets (Ioannidis JPA, Altman R, Boffetta P, Danesh J, Hartge P, Little J, et al.: unpublished data). Overall, the results shown in the fi gure support the choices made by Hung et al. in assigning prior