Interpreting P Values in Pharmacogenetic Studies: A Call for Process and Perspective

To our knowledge, the cancer pharmacogenetics article by Marsh et al in this issue of the Journal of Clinical Oncology is the largest published pharmacogenetic study of carboplatin and taxanes, focusing on the potential association of candidate polymorphisms with treatment toxicity and disease response outcomes. It is a model for how collection of whole blood samples for DNA in a phase III clinical trial (which, by design, prospectively collects important phenotype data with unbiased treatment assignments) enables high quality pharmacogenetic research. Strikingly, but appropriately, it is published with no positive findings to report. Attaining the proposed benefits of the Human Genome Project requires an iterative process of hypothesis generation and hypothesis testing. It is not disconcerting that this study failed to confirm previously published associations between various polymorphisms and outcomes from combined carboplatin and docetaxel or paclitaxel regimens. It is a signal that more work might be done, and should be a warning to those investigators who have jumped from small, hypothesis-generating pharmacogenetic studies with borderline P values into large prospective randomized trials to test the utility of the putative pharmacogenetic test, but without first attempting to validate the association. The first two pharmacogenetic tests in oncology, for thiopurine methyltransferase (TPMT) and UGT1A1 deficiencies, were developed through a candidate gene/polymorphism approach. In each case, clinical observations led to multiple preclinical studies, including testing of the relationship between specific functional polymorphisms of TPMT and UGT1A1 and enzyme function. Ultimately, small prospective clinical trials were conducted where the primary objective was to confirm the relationship between the genetic variant and excess exposure and consequent toxicities from 6-mercaptopurine and irinotecan, respectively. Use of this approach does not ensure successful development of a pharmacogenetic test. While dihydropyrimidine dehydrogenase deficiency is heritable and leads to excess exposure to fluorouracil and consequent toxicity, it has been associated with numerous polymorphisms in the DPYD gene, but with each single nucleotide polymorphism (SNP) found at different frequencies in different populations. Hence, any single genotype has insufficient negative predictive value to be clinically useful. At the same time, adhering to the principles of the candidate gene/polymorphism approach has instead led investigators to focus efforts on developing phenotype-based assays. Marsh et al utilized an approach likely to be of increasing importance in pharmacogenetics. They were able to test specific hypotheses regarding candidate polymorphisms in a large clinical trial data set because DNA had been collected and was available for analysis. By removing biases of treatment assignment, completing data collection within a specified timeframe, and by careful phenotyping for features of immediate clinical relevance, large clinical trials of cancer therapy provide important data sets for developing pharmacogenetic tests. Since preclinical testing does not assess the magnitude of effect for a candidate gene/polymorphism in the clinical setting, genotyping for associations with previously collected phenotypic data provides a crucial opportunity for validating the candidate polymorphism, without incurring the expense of a new prospective study. Investigators could change the development plan based on these tests and increase the likelihood of success in the prospective setting, for example by increasing the number of polymorphisms typed to capture better the relevant genetic variants affecting a treatment phenotype or by switching to a phenotype test altogether. The study by Marsh et al is also a lesson in how to assess appropriately the relationship between candidate polymorphisms and clinical outcomes. One critical obstacle to successful development of a genotype-based diagnostic test is the high number of spurious associations. Each patient has 10 base pairs (bp) in his/her genome. With SNPs occurring every 100 to 300 bp, taken to the extreme, each patient has 10 potential hypotheses to test for association with each clinically relevant phenotype of interest. With a P value cutoff of .05, 500,000 SNPs will demonstrate a significant association with one phenotype by chance alone. As increasing numbers of phenotypes or combinations of polymorphisms are considered, the multiple-hypothesis testing in the same data set increases the number of false-positive associations. With so many expected false-positive associations, a nominally significant P value merely suggests, but does not confirm, a hypothesis that would warrant additional testing. Marsh et al addressed this problem for their association of 27 candidate polymorphisms with six clinical phenotypes first by predetermining an acceptable falsediscovery rate and screening these SNPs in 609 randomly selected patients with univariate models. They then incorporated the two SNPs passing the false-discovery rate criteria into a multivariate model that was then tested in 305 patients, and the two SNPs proved to be JOURNAL OF CLINICAL ONCOLOGY E D I T O R I A L VOLUME 25 NUMBER 29 OCTOBER 1