Is It Necessary to Correct for Measurement Error in Nutritional Epidemiology?
Authors: Anne C.M. Thiébaut, Laurence S. Freedman, Raymond J. Carroll, Victor Kipnis
Journal: Annals of Internal Medicine (2007)

Editorials2 January 2007Is It Necessary to Correct for Measurement Error in Nutritional Epidemiology?FREEAnne C.M. Thiébaut, PhD, Laurence S. Freedman, PhD, Raymond J. Carroll, PhD, and Victor Kipnis, PhDAnne C.M. Thiébaut, PhDFrom National Cancer Institute, Bethesda, Maryland; Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel; and Texas A&M University, College Station, Texas., Laurence S. Freedman, PhDFrom National Cancer Institute, Bethesda, Maryland; Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel; and Texas A&M University, College Station, Texas., Raymond J. Carroll, PhDFrom National Cancer Institute, Bethesda, Maryland; Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel; and Texas A&M University, College Station, Texas., and Victor Kipnis, PhDFrom National Cancer Institute, Bethesda, Maryland; Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel; and Texas A&M University, College Station, Texas.Author, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-146-1-200701020-00012 SectionsAboutVisual AbstractPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Relationships between diet and chronic disease have become the focus of many analytic studies in nutritional epidemiology over the past several decades. Given often relatively limited variation in dietary intake within study populations, the results of observational studies critically depend on accurate assessment of the dietary exposure and other adjustment covariates, especially in view of the moderate-sized relative risks seen in many nutrition studies. Error in measuring exposure leads to a biased and inefficient estimate of the relationship of the exposure to disease (1). In this issue, Beulens and colleagues (2) present a prospective analysis of the relationship between alcohol intake, which participants self-reported using a food-frequency questionnaire, and cardiovascular events among men with hypertension in the Health Professionals Follow-Up Study. The authors report the findings before and after correction for measurement error, which provides a welcome opportunity to comment on the importance of measurement error in nutritional epidemiology.Measurement error in covariate assessment of prospective studies is generally assumed to be nondifferential with respect to disease, which means that the relationship between true and reported diet is the same for persons who develop the disease as for those who do not. If this assumption is true, most authors in epidemiologic literature assume that measurement error attenuates the true relative risk—that is, it biases its estimate toward no effect. Consequently, authors may interpret null results as perhaps being due to exposure measurement error. In a similar manner, because of measurement error, the true relative risk in a statistically significant association should, if anything, be larger than that reported. This attenuation of effect always occurs when only the main exposure is measured with error, and this error follows the "classical model": It is additive; it is independent of the size of the true exposure; and it has a mean of zero and a variance that does not change with the size of the true exposure (1).Measurement error that occurs when assessing a person's usual diet with a food-frequency questionnaire does not follow the classical model. Instead, it generally involves bias related to true intake in addition to random variation (3). Bias related to true intake often occurs as the "flattened-slope phenomenon" (consumers with high levels of intake tend to underreport and low-level consumers tend to over-report), which biases relative risk upward (4). On the other hand, random between-person and within-person variation biases the relative risk downward. The effect of random variation is more important in most situations, with the overall effect leading to an attenuated relative risk when a single error-prone exposure is present (5). In this situation, the nominal significance level of statistical tests of the exposure effect remains unaffected by the measurement error so that the usual significance tests of the relative risk are valid tests of the exposure–disease relationship. Nevertheless, measurement error causes statistical power to fall by a factor that increases as the amount of measurement error increases (6). This means that studies become less able to detect exposure–disease relationships as measurement error increases.The effect of measurement error becomes much more complex if the disease model includes more than 1 exposure measured with error. This factor is important to consider because most analyses of diet–disease relationships, as in Beulens and colleagues' study (2), include adjustment for energy intake (7) and other potentially error-prone adjustment covariates. In such cases, error in the main exposure attenuates the observed association with disease, and error in the adjustment covariates "contaminates" the observed association because of residual confounding (3). As a result, the bias in the estimated effect could be in either direction and not simply toward the null. Moreover, uncorrected statistical tests no longer accurately measure the degree of significance, making it impossible to reliably interpret the observed diet–disease association. Adjustment for multivariate measurement error is therefore necessary not only for unbiased estimation of the true diet–disease relationship (for example, the relative risk), but also for accurately testing whether the relative risk statistically significantly differs from 1.0.To adjust for measurement error, Beulens and colleagues (2) applied the linear regression calibration approach, which is a state-of-the-art method in nutritional epidemiology (8, 9). In principle, this method requires performing accurate measures of nutritional intake in a calibration substudy and fitting multiple regressions of each true (that is, accurately measured in the calibration subsample) exposure variable on the error-prone and precisely measured covariates included in the disease model. The slopes in these multiple regressions are then applied to correct the observed relative risks in the main study. Because usual dietary intakes are impossible to measure accurately in free-living populations, the regression slopes in the calibration subsample may still be estimated by using reference measurements of usual intakes in place of their exact values. These reference measurements may contain errors of their own, but those errors should ideally be purely random, independent of the true covariates and errors in the reported intakes on the food-frequency questionnaire (3). As we explain below, the availability of such reference measurements in nutritional studies is limited.To date, epidemiologic studies have used more intensive self-report methods as the reference instrument for the calibration subsample, for example, multiple-day food records, as in the Health Professionals Follow-Up Study (2), or several recalls of food intake in the previous 24 hours. Cumulative evidence from studies using "recovery" biomarkers, such as doubly-labeled water, as a reference standard for total energy intake (recovery biomarkers do meet requirements for a reference instrument [10]) shows that these intensive self-report reference methods can have systematic errors that are correlated with food-frequency questionnaire errors, therefore violating a key requirement of reference instruments (11). Unfortunately, reliable recovery biomarkers are available for only a few dietary factors—in addition to doubly labeled water. Urinary nitrogen is a marker for protein intake, and urinary potassium may be a marker for dietary potassium intake (10). Therefore, in most studies, the use of self-report instruments in a calibration subsample as the reference standard is currently our best, although imperfect, approach to adjustment for measurement error.In their uncorrected analysis of the association between alcohol consumption and myocardial infarction, Beulens and colleagues (2) found a statistically significant relative risk of 0.85 (P < 0.001). After adjustment for measurement error in most dietary covariates and body mass index, the relative risk was 0.68 but with only borderline statistical significance (P = 0.051). We expect confidence intervals to widen after measurement error correction, partly because of added uncertainty in the estimated regression calibration slopes as a result of taking measurement error into account (9). Had the authors considered all error-prone variables (including trans and ω-3 fatty acids and physical activity) in the same analysis, the estimated confidence intervals would have been even wider (12). Larger calibration substudies, such as in the European Prospective Investigation into Cancer and Nutrition (EPIC) (13) and the National Institutes of Health–AARP [American Association of Retired Persons] Diet and Health Study (14), allow a more accurate calibration and, together with selection of more parsimonious diet–disease models, help limit the extent to which confidence intervals become wider after correction for measurement error.Furthermore, the linear regression calibration method assumes that regressions relating reference measurements of intake to reported intakes on the food-frequency questionnaire are linear and homoscedastic. Meeting this condition often requires transformation of the involved variables (for example, log transformation). Moreover, we need a new method for foods that may be episodically or never consumed by a substantial proportion of individuals, alcohol being 1 important example (15). Beulens and colleagues do not discuss how they dealt with reports of no alcohol consumption and skewed distributions of positive alcohol intake and intakes of other dietary factors (for example, energy) in the disease model (2), which could affect the adjusted relative risks for cardiovascular outcomes and their corresponding confidence intervals.Although current execution of regression calibration may not fully correct for measurement error, it does express some of the uncertainty arising from measurement error and therefore remains an important practice. To improve our understanding of dietary measurement error, we need to identify new recovery biomarkers and to investigate "concentration" biomarkers (for example, serum carotenoids), which do not meet the requirements for reference measurements but do correlate with intake (10, 16). We need new methods for combining concentration markers with dietary-assessment methods (17).Our discussion of measurement error in nutrition studies should convince the reader to exercise caution when interpreting relationships between diet and clinical outcomes. Owing to the current limitations of available procedures and reference instruments, we cannot assume that corrected estimates of diet–disease associations in any single study are definitive. To firmly establish a hypothesis, we need carefully conducted studies in diverse populations with different dietary patterns and ranges of intake, incidence rates, and sociocultural histories. In addition, we must use different dietary assessment and reference instruments, which may result in different measurement error structures and different methods of error correction.The uncertainties surrounding measurement error should send a strong message to those who formulate recommendations about nutrition. With unexplained inconsistencies in results, the discretion of silence may be preferable to the valor of setting recommendations. Conversely, when several studies of the same topic have consistent, robust results, we can be confident that the association in question is strong enough to withstand the impact of measurement error. Results that satisfy these requirements can be a basis for future dietary guidelines.References1. Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu CM. Measurement Error in Nonlinear Models: A Modern Perspective. 2nd ed. London: Chapman & Hall; 2006. Google Scholar2. Beulens JW, Rimm EB, Ascherio A, Spiegelman D, Hendriks HF, Mukamal KJ. Alcohol consumption and risk for coronary heart disease among men with hypertension. Ann Intern Med. 2007;146:10-9. LinkGoogle Scholar3. Kipnis V, Carroll RJ, Freedman LS, Li L. Implications of a new dietary measurement error model for estimation of relative risk: application to four calibration studies. Am J Epidemiol. 1999;150:642-51. [PMID: 10490004] CrossrefMedlineGoogle Scholar4. Wacholder S. When measurement errors correlate with truth: surprising effects of nondifferential misclassification. Epidemiology. 1995;6:157-61. [PMID: 7742402] CrossrefMedlineGoogle Scholar5. Kipnis V, Subar AF, Midthune D, Freedman LS, Ballard-Barbash R, Troiano RP, et al. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;158:14-21; discussion 22-6. [PMID: 12835281] Google Scholar6. Kaaks R, Riboli E, van Staveren W. Calibration of dietary intake measurements in prospective cohort studies. Am J Epidemiol. 1995;142:548-56. [PMID: 7677134] CrossrefMedlineGoogle Scholar7. Willett W, Stampfer MJ. Total energy intake: implications for epidemiologic analyses. Am J Epidemiol. 1986;124:17-27. [PMID: 3521261] CrossrefMedlineGoogle Scholar8. Rosner B, Willett WC, Spiegelman D. Correction of logistic regression relative risk estimates and confidence intervals for systematic within-person measurement error. Stat Med. 1989;8:1051-69; discussion 1071-3. [PMID: 2799131] Google Scholar9. Rosner B, Spiegelman D, Willett WC. Correction of logistic regression relative risk estimates and confidence intervals for measurement error: the case of multiple covariates measured with error. Am J Epidemiol. 1990;132:734-45. [PMID: 2403114] CrossrefMedlineGoogle Scholar10. Kaaks R, Ferrari P, Ciampi A, Plummer M, Riboli E. Uses and limitations of statistical accounting for random error correlations, in the validation of dietary questionnaire assessments. Public Health Nutr. 2002;5:969-76. [PMID: 12638598] CrossrefMedlineGoogle Scholar11. Kipnis V, Midthune D, Freedman LS, Bingham S, Schatzkin A, Subar A, et al. Empirical evidence of correlated biases in dietary assessment instruments and its implications. Am J Epidemiol. 2001;153:394-403. [PMID: 11207158] CrossrefMedlineGoogle Scholar12. Schatzkin A, Kipnis V. Could exposure assessment problems give us wrong answers to nutrition and cancer questions? [Editorial]. J Natl Cancer Inst. 2004;96:1564-5. [PMID: 15523078] CrossrefMedlineGoogle Scholar13. Slimani N, Kaaks R, Ferrari P, Casagrande C, Clavel-Chapelon F, Lotze G, et al. European Prospective Investigation into Cancer and Nutrition (EPIC) calibration study: rationale, design and population characteristics. Public Health Nutr. 2002;5:1125-45. [PMID: 12639223] CrossrefMedlineGoogle Scholar14. Schatzkin A, Subar AF, Thompson FE, Harlan LC, Tangrea J, Hollenbeck AR, et al. Design and serendipity in establishing a large cohort with wide dietary intake distributions: the National Institutes of Health-American Association of Retired Persons Diet and Health Study. Am J Epidemiol. 2001;154:1119-25. [PMID: 11744517] CrossrefMedlineGoogle Scholar15. Tooze JA, Midthune D, Dodd KW, Freedman LS, Krebs-Smith SM, Subar AF, et al. A new statistical method for estimating the usual intake of episodically consumed foods with application to their distribution. J Am Diet Assoc. 2006;106:1575-87. [PMID: 17000190] CrossrefMedlineGoogle Scholar16. Potischman N, Freudenheim JL. Biomarkers of nutritional exposure and nutritional status: an overview. J Nutr. 2003;133 Suppl 3 873S-874S. [PMID: 12612172] CrossrefMedlineGoogle Scholar17. Prentice RL, Willett WC, Greenwald P, Alberts D, Bernstein L, Boyd NF, et al. Nutrition and physical activity and chronic disease prevention: research strategies and recommendations. J Natl Cancer Inst. 2004;96:1276-87. [PMID: 15339966] CrossrefMedlineGoogle Scholar Comments0 CommentsSign In to Submit A Comment Author, Article, and Disclosure InformationAffiliations: From National Cancer Institute, Bethesda, Maryland; Gertner Institute for Epidemiology and Health Policy Research, Tel Hashomer, Israel; and Texas A&M University, College Station, Texas.Grant Support: Dr. Carroll's research was supported in part by a grant from the National Cancer Institute (CA-57030) and the Texas A&M Center for Environmental and Rural Health via a grant from the National Institute of Environmental Health Sciences (P30-ES09106).Disclosures: None disclosed.Corresponding Author: Victor Kipnis, PhD, National Cancer Institute, Biometry Research Group, Division of Cancer Prevention, Bethesda, MD 20892-7354; e-mail, [email protected]nih.gov.Current Author Addresses: Dr. Thiébaut: National Cancer Institute, Nutritional Epidemiology Branch, Division of Cancer Epidemiology and Genetics, Bethesda, MD 20892-7232.Dr. Freedman: Gertner Institute for Epidemiology and Health Policy Research, Biostatistics Unit, Sheba Medical Center Tel Hashomer 52161, Israel.Dr. Carroll: Texas A&M University, Department of Statistics, Blocker Building Room 447; College Station, TX 77843-3143.Dr. Kipnis: National Cancer Institute, Biometry Research Group, Division of Cancer Prevention, Bethesda, MD 20892-7354. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoAlcohol Consumption and Risk for Coronary Heart Disease among Men with Hypertension Joline W.J. Beulens , Eric B. Rimm , Alberto Ascherio , Donna Spiegelman , Henk F.J. Hendriks , and Kenneth J. Mukamal Metrics Cited bySTRATOS guidance document on measurement error and misclassification of variables in observational epidemiology: Part 2—More complex methods of adjustment and advanced topicsSTRATOS guidance document on measurement error and misclassification of variables in observational epidemiology: Part 1—Basic theory and simple methods of adjustmentPlant Foods, Antioxidant Biomarkers, and the Risk of Cardiovascular Disease, Cancer, and Mortality: A Review of the EvidenceTwenty-Four-Hour Diet recall and Diet records compared with 24-hour urinary excretion to predict an individual's sodium consumption: A Systematic ReviewAssessment of dietary sodium intake using a food frequency questionnaire and 24-hour urinary sodium excretion: a systematic literature reviewAssociation between Mediterranean diet and head and neck cancer: results of a large case–control study in ItalyMeasurement Error and Misclassification: IntroductionMethods to assess measurement error in questionnaires of sedentary behaviorGene–environment interaction and risk of breast cancerMediterranean and Nordic diet scores and long-term changes in body weight and waist circumference: results from a large cohort studyResearch gaps in evaluating the relationship of meat and healthAdherence to a healthy Nordic food index and breast cancer risk: results from a Swedish cohort studyUse of a Urinary Sugars Biomarker to Assess Measurement Error in Self-Reported Sugars Intake in the Nutrition and Physical Activity Assessment Study (NPAAS)Measurement ErrorCan efficiency be gained by correcting for misclassification?Mediterranean Dietary Pattern and Risk of Breast CancerAspirin, Nonsteroidal Anti-inflammatory Drug Use, and Risk for Crohn Disease and Ulcerative Colitis A Cohort StudyAshwin N. Ananthakrishnan, MD, MPH, Leslie M. Higuchi, MD, MPH, Edward S. Huang, MD, MPH, Hamed Khalili, MD, James M. Richter, MD, Charles S. Fuchs, MD, MPH, and Andrew T. Chan, MD, MPHSerious psychological distress, health risk behaviors, and diabetes care among adults with type 2 diabetes: The California Health Interview Survey 2007Calibration of the dietary data obtained from the Brazilian center of the Natural History of HPV Infection in Men study: the HIM StudyContribution of specific dietary factors to CHD in US femalesA Bayesian model for estimating the effects of drug use when drug use may be under-reportedDo Flavonoid Intakes of Postmenopausal Women With Breast Cancer Vary on Very Low Fat Diets?Relative Effectiveness of Osteoporosis Drugs for Preventing Nonvertebral FractureSuzanne M. Cadarette, PhD, Jeffrey N. Katz, MD, MS, M. Alan Brookhart, PhD, Til Stürmer, MD, MPH, Margaret R. Stedman, MPH, and Daniel H. Solomon, MD, MPHAdopting Moderate Alcohol Consumption in Middle Age: Subsequent Cardiovascular EventsAlcohol Consumption and Risk for Coronary Heart Disease among Men with HypertensionAlcohol Consumption and Risk for Coronary Heart Disease among Men with HypertensionDiet and Alzheimer's disease 2 January 2007Volume 146, Issue 1Page: 65-67KeywordsAlcoholsCancer preventionConfidence intervalsDietEpidemiologyNutritionNutritional epidemiologyPopulation statisticsQuestionnaires ePublished: 2 January 2007 Issue Published: 2 January 2007 PDF downloadLoading ...