Author reply
Authors: Emily Y. Chew, Michael L. Klein, Traci E. Clemons, Elvira Agrón, Gonçalo R. Abecasis
Journal: Ophthalmology (2015)

Drs Awh and Zanke continue to provide “selective” interpretations of our analyses of data regarding the influence of genetics on the response to the Age-Related Eye Disease Study (AREDS) supplement. We would like to emphasize the exceptional editorial by Drs Wittes and Musch,1Wittes J. Musch D.C. Should we test for genotype in deciding on Age-Related Eye Disease Study Supplementation?.Ophthalmology. 2015; 122: 3-5Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar who are experienced methodologists (a statistician and an epidemiologist, respectively). They summarized the findings from reports that analyzed the AREDS data with differing results.2Awh C.C. Lane A.-M. Hawken S. et al.CFH and ARMS2 genetic polymorphism predict response to antioxidants and zinc in patients with age-related macular degeneration.Ophthalmology. 2013; 120: 2317-2323Abstract Full Text Full Text PDF PubMed Scopus (84) Google Scholar, 3Chew E.Y. Klein M.L. Clemons T.E. et al.No clinically significant association between CFH and ARMS2 genotypes and response to nutritional supplements: AREDS Report Number 38.Ophthalmology. 2014; 121: 2173-2180Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar, 4Awh C.C. Hawken S. Zanke B.W. Treatment response to antioxidants and zinc based on CFH and ARMS2 genetic risk allele number in the Age-Related Eye Disease Study.Ophthalmology. 2015; 122: 162-169Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar, 5Chew E.Y. Klein M.L. Clemons T.E. Genetic testing in persons with age-related macular degeneration and the use of AREDS supplements: to test or not to test?.Ophthalmology. 2015; 122: 212-215Abstract Full Text Full Text PDF PubMed Scopus (40) Google Scholar Specifically, Awh et al reported that the response to therapy with the AREDS supplement (antioxidants, zinc, or both) depended on the participant's genotype.2Awh C.C. Lane A.-M. Hawken S. et al.CFH and ARMS2 genetic polymorphism predict response to antioxidants and zinc in patients with age-related macular degeneration.Ophthalmology. 2013; 120: 2317-2323Abstract Full Text Full Text PDF PubMed Scopus (84) Google Scholar We, the AREDS investigators, found that the responses of the AREDS participants to the AREDS supplement were not significantly different by genotype, whether we used genotypes originally analyzed by Klein et al, or genotypes analyzed by Awh et al (interaction P values were 0.06 and 0.45, respectively).3Chew E.Y. Klein M.L. Clemons T.E. et al.No clinically significant association between CFH and ARMS2 genotypes and response to nutritional supplements: AREDS Report Number 38.Ophthalmology. 2014; 121: 2173-2180Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar Awh et al presented yet another set of analyses in a subsequent report, suggesting that harmful effects of zinc and antioxidants of the AREDS supplement may be associated with specific genotypes.4Awh C.C. Hawken S. Zanke B.W. Treatment response to antioxidants and zinc based on CFH and ARMS2 genetic risk allele number in the Age-Related Eye Disease Study.Ophthalmology. 2015; 122: 162-169Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar We then assembled a separate group of AREDS participants not included in the analyses by Awh et al and attempted to replicate their analyses. In this newly assembled group of AREDS participants, every genotype benefited from the AREDS supplements,5Chew E.Y. Klein M.L. Clemons T.E. Genetic testing in persons with age-related macular degeneration and the use of AREDS supplements: to test or not to test?.Ophthalmology. 2015; 122: 212-215Abstract Full Text Full Text PDF PubMed Scopus (40) Google Scholar contrary to the findings of Awh et al.4Awh C.C. Hawken S. Zanke B.W. Treatment response to antioxidants and zinc based on CFH and ARMS2 genetic risk allele number in the Age-Related Eye Disease Study.Ophthalmology. 2015; 122: 162-169Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar Wittes and Musch provided an excellent guideline for interpreting all the data that have been presented. They specifically criticized Awh et al for doing multiple testing that, in essence, allowed them to shoot multiple arrows at a target, which provide even a poor archer a high probability of hitting a bull's eye. To be statistically rigorous, one would have to adjust for the large number of arrows (i.e., the many multiple tests). However, it is not possible to adjust in the case of Awh et al as they do not report the number of subgroups examined and whether they were prespecified or post hoc (after the fact). To quote Wittes and Musch, “Post hoc exploration of subgroups is treacherous because one never knows whether the results are due to inadvertent selection bias or if they are ‘true.’ Many people search post hoc for subgroup effects. Such ‘data dredging’ rewards the searcher with a high probability of finding differences between subgroups, but the probability is high that the finding is spurious. Replication is the only reliable way of assessing the validity of a post hoc finding.” Our analyses of an independent cohort did not validate the results of the second set of analyses by Awh et al. If we were to find similar results as Awh et al, despite all the biases, we would agree that their conclusions might be reasonable. But this was not the case. The authors of the editorial then explain that Awh et al based their subgroup analyses on the outcome. This was conducting the analyses “after the fact.” Thus, Awh et al have ensured themselves of finding significant results. Wittes and Musch agreed with our clinical recommendation. Treatment with the AREDS supplement is indicated in persons with bilateral intermediate age-related macular degeneration (AMD) or those with late AMD in 1 eye. In fact, if they personally had ocular findings that warrant AREDS supplement, Wittes and Musch state that they hope their ophthalmologists would prescribe the AREDS supplement without genetic testing. We, the AREDS investigators, and Wittes and Musch, agree: genotype testing before advising treatment with AREDS supplements for a patient at risk of developing late AMD is not recommended. The current letter from Drs Awh and Zanke criticized the lack of finding a beneficial effect of the AREDS supplement in AREDS Report 38.3Chew E.Y. Klein M.L. Clemons T.E. et al.No clinically significant association between CFH and ARMS2 genotypes and response to nutritional supplements: AREDS Report Number 38.Ophthalmology. 2014; 121: 2173-2180Abstract Full Text Full Text PDF PubMed Scopus (65) Google Scholar This insignificant finding is not surprising, because the report was based on a subgroup analysis, utilizing less than one-quarter of the original cohort required to show a beneficial effect in the original AREDS report. Their letter also contains a new claim based on further subgroup analyses to support their mistaken view of genetic testing. This quote reinforces the thoughts expressed by Wittes and Musch: “While postulating subgroup effects a priori allows formal statistical hypothesis testing, much less credence is due to formulating hypotheses after examining the data. Such analysis smacks of betting on a horse after the race is over.” Let's not bet again. Re: Chew et al.: No clinically significant association between CFH and ARMS2 genotypes and response to nutritional supplements: AREDS report number 38 (Ophthalmology 2014;121:2173-80)OphthalmologyVol. 122Issue 8PreviewData found in the work of Chew et al1 provides evidence that (1) the Age-Related Eye Disease Study (AREDS) Formulation provided no demonstrated overall benefit for this subset of 1237 patients and (2) that genotype-directed nutritional supplementation could provide substantial benefits. Full-Text PDF