Sensitivity analysis for misclassification in logistic regression via likelihood methods and predictive value weighting.

The potential for bias due to misclassification error in regression analysis is well understood by statisticians and epidemiologists. Assuming little or no available data for estimating misclassification probabilities, investigators sometimes seek to gauge the sensitivity of an estimated effect to variations in the assumed values of those probabilities. We present an intuitive and flexible approach to such a sensitivity analysis, assuming an underlying logistic regression model. For outcome misclassification, we argue that a likelihood-based analysis is the cleanest and the most preferable approach. In the case of covariate misclassification, we combine observed data on the outcome, error-prone binary covariate of interest, and other covariates measured without error, together with investigator-supplied values for sensitivity and specificity parameters, to produce corresponding positive and negative predictive values. These values serve as estimated weights to be used in fitting the model of interest to an appropriately defined expanded data set using standard statistical software. Jackknifing provides a convenient tool for incorporating uncertainty in the estimated weights into valid standard errors to accompany log odds ratio estimates obtained from the sensitivity analysis. Examples illustrate the flexibility of this unified strategy, and simulations suggest that it performs well relative to a maximum likelihood approach carried out via numerical optimization.

Related papers:

title: Measurement Error in Nonlinear Models
abstract: Preface Guide to Notation 1.  Regression and Attenuation 3.  Calibration 4. Simulation Extrapolation 5. Instrumental Variables 6.  Likelihood and Quasilikelihood 8. Bayesian Methods 9. Semiparametric  Unknown Link Functions 11.  Testing 12. Density Estimation and Nonparametric Regression 13. Response Variable Error 14.  Fitting Methods and Models References Author Index Subject Index

title: The value of latent class analysis in medical diagnosis.
abstract: Assessment of the value of diagnostic indicators such as symptoms and laboratory tests results from calculation of the sensitivity and specificity of the indicators. Knowledge of the rate of occurrence of the disease allows for additional calculations of the error rates in using an indicator. These calculations are accurate only when the data on which they are based are reliable. If the diagnosis, which is used as the criterion for computing the sensitivity and specificity, is not accurate, then the resulting calculations will be in error. We show how a statistical method, latent class analysis, allows for the estimation of the characteristics of indicators even when an accurate diagnosis is unavailable. In addition, the method deals with several indicators at once, and provides a way to combine the information from all the indicators to make a diagnosis.

title: Adjustment for non-differential misclassification error in the generalized linear model.
abstract: It is well known that estimates of association between an outcome variable and a set of categorical covariates, some of which are measured with misclassification, tend to be biased upon application of the usual methods of estimation that ignore the classification error. We propose a method to adjust for misclassification in covariates when one applies the generalized linear model. In the case where one can observe some true covariates only through surrogates, we combine a latent class analysis with the approach to incorporate multiple surrogates into the model. We include discussion on the efficacy of repeated measurements which one can view as a special case of multiple surrogates with identical distribution. We provide two examples to demonstrate the applicability of the method and the efficacy of multiple replicates for a covariate subject to misclassification in a regression framework.

title: A method to automate probabilistic sensitivity analyses of misclassified binary variables.
abstract: Misclassification bias is present in most studies, yet uncertainty about its magnitude or direction is rarely quantified.



The authors present a method for probabilistic sensitivity analysis to quantify likely effects of misclassification of a dichotomous outcome, exposure or covariate. This method involves reconstructing the data that would have been observed had the misclassified variable been correctly classified, given the sensitivity and specificity of classification. The accompanying SAS macro implements the method and allows users to specify ranges of sensitivity and specificity of misclassification parameters to yield simulation intervals that incorporate both systematic and random error.



The authors illustrate the method and the accompanying SAS macro code by applying it to a study of the relation between occupational resin exposure and lung-cancer deaths. The authors compare the results using this method with the conventional result, which accounts for random error only, and with the original sensitivity analysis results.



By accounting for plausible degrees of misclassification, investigators can present study results in a way that incorporates uncertainty about the bias due to misclassification, and so avoid misleadingly precise-looking results.

title: Logistic regression when the outcome is measured with uncertainty.
abstract: In epidemiologic research, logistic regression is often used to estimate the odds of some outcome of interest as a function of predictors. However, in some datasets, the outcome of interest is measured with imperfect sensitivity and specificity. It is well known that the misclassification induced by such an imperfect diagnostic test will lead to biased estimates of the odds ratios and their variances. In this paper, the authors show that when the sensitivity and specificity of a diagnostic test are known, it is straightforward to incorporate this information into the fitting of logistic regression models. An EM algorithm that produces unbiased estimates of the odds ratios and their variances is described. The resulting odds ratio estimates tend to be farther from the null but have greater variance than estimates found by ignoring the imperfections of the test. The method can be extended to the situation where the sensitivity and specificity differ for different study subjects, i.e., nondifferential misclassification. The method is useful even when the sensitivity and specificity are not known, as a way to see the degree to which various assumptions about sensitivity and specificity affect one's estimates. The method can also be used to estimate sensitivity and specificity under certain assumptions or when a validation subsample is available. Several examples are provided to compare the results of this method with those obtained by standard logistic regression. A SAS macro that implements the method is available on the World Wide Web at http:@som1.ab.umd.edu/Epidemiology/software.h tml.

title: Semi-Automated Sensitivity Analysis to Assess Systematic Errors in Observational Data
abstract: epidemiologic research usually provides a quantitative assessment of random error for effect estimates, but no quantitative assessment of systematic error. Sensitivity analysis can provide such an assessment.  We describe a method to reconstruct epidemiologic data, accounting for biases, and to display the results of repeated reconstructions as an assessment of error. We illustrate with a study of the effect of less-than-definitive therapy on breast cancer mortality.  We developed SAS code to reconstruct the data that would have been observed had a set of systematic errors been absent, and to convey the results. After 4,000 reconstructions of the example data, we obtained a median estimate of relative hazard equal to 1.5 with a 95% simulation interval of 0.8-2.8. The relative hazard obtained by conventional analysis equaled 2.0, with a 95% confidence interval of 1.2-3.4.  Our method of sensitivity analysis can be used to quantify the systematic error for an estimate of effect and to describe that error in figures, tables, or text. In the example, the sources of error biased the conventional relative hazard away from the null, and that error was not accurately communicated by the conventional confidence interval.

title: Exposure measurement error: influence on exposure-disease. Relationships and methods of correction.
abstract: Epidemiology, being primarily an observational rather than experimental science, has often encountered problems with measurement of the study variables, which are not under the control of the investigator. Difficulties arise in part because the variables under study are often subjective and often must be ascertained by subjects' self reports or from records of dubious quality, because biological variability and laboratory error can occur, and because studies are often conducted retrospectively, thus requiring informa­ tion on events that occurred long ago. Epidemiologists have long recognized that measurement errors have been among the major weaknesses of their studies and have gone to great lengths to try to assess the magnitude of such errors and their likely impact on the conclusions . This problem has spurred much methodologic research, initially on understanding the effects of measurement errors on exposure-response relationships and more recently on developing methods to correct for such errors. The results of the former line of research have often been used qualitatively in the interpretation of epidemiologic findings, but the quantitative predictions have seldom been applied. Thus, the latter line of research appears to be particularly promising in terms of practical applications. Because most of this work is relatively

title: A note on estimating crude odds ratios in case-control studies with differentially misclassified exposure.
abstract: Morrissey and Spiegelman (1999, Biometrics 55, 338 344) provided a comparative study of adjustment methods for exposure misclassification in case-control studies equipped with an internal validation sample. In addition to the maximum likelihood (ML) approach, they considered two intuitive procedures based on proposals in the literature. Despite appealing ease of computation associated with the latter two methods, efficiency calculations suggested that ML was often to be recommended for the analyst with access to a numerical routine to facilitate it. Here, a reparameterization of the likelihood reveals that one of the intuitive approaches, the inverse matrix method, is in fact ML under differential misclassification. This correction is intended to alert readers to the existence of a simple closed-form ML estimator for the odds ratio in this setting so that they may avoid assuming that a commercially inaccessible optimization routine must be sought to implement ML.

title: Correcting for misclassification in two-way tables and matched-pair studies.
abstract: Barron has presented a matrix formula that allows correction for nondifferential misclassification in two-by-two tables. We show how extensions of his result can be applied to situations involving differential misclassification, matched data, and arbitrary two-way tables. We also discuss some important limitations in the application of algebraic corrections for misclassification. In particular, we show that study misclassification rates need not reflect population misclassification rates if the distribution of the misclassified variable is determined by aspects of the study design.

title: Variance estimation for epidemiologic effect estimates under misclassification.
abstract: This paper presents basic methods for construction of variance estimators for epidemiologic effect estimates when adjusting for misclassification. Methods are described for differential and non-differential misclassification, and for external and internal estimates of classification rates. The methods take account of sampling variability in both the observed data and the estimated classification rates. The methods are illustrated in a case-control analysis of the association of antibiotic use during pregnancy with sudden infant death syndrome.

title: Maximum-likelihood and closed-form estimators of epidemiologic measures under misclassification
abstract: There is a large literature on estimation under misclassification. The present paper reviews epidemiologic inference under misclassification in the multiway contingency-table setting, and addresses a few controversial issues. In the 1990s, claims of inefficiency of early closed-form estimators of odds ratios under misclassification arose from misapplication of the estimators to studies with internal validation. In reality, these estimators are maximum likelihood (ML) and hence efficient under the external-validation assumptions used for their derivation. For the internal-validation case, a new closed-form estimator is derived that incorporates the nondifferentiality constraint into the predictive-value (“direct” or “inverse-matrix”) estimator. Results are presented in a general framework that applies to misclassification in models for multiway tables, and that allows the target parameter to be any measure of association or effect.

title: Matrix methods for estimating odds ratios with misclassified exposure data: extensions and comparisons.
abstract: Misclassification of exposure variables is a common problem in epidemiologic studies. This paper compares the matrix method (Barron, 1977, Biometrics 33, 414-418; Greenland, 1988a, Statistics in Medicine 7, 745-757) and the inverse matrix method (Marshall, 1990, ) to the maximum likelihood estimator (MLE) that corrects the odds ratio for bias due to a misclassified binary covariate. Under the assumption of differential misclassification, the inverse matrix method is always more efficient than the matrix method; however, the efficiency depends strongly on the values of the sensitivity, specificity, baseline probability of exposure, the odds ratio, case-control ratio, and validation sampling fraction. In a study on sudden infant death syndrome (SIDS), an estimate of the asymptotic relative efficiency (ARE) of the inverse matrix estimate was 0.99, while the matrix method's ARE was 0.19. Under nondifferential misclassification, neither the matrix nor the inverse matrix estimator is uniformly more efficient than the other; the efficiencies again depend on the underlying parameters. In the SIDS data, the MLE was more efficient than the matrix method (ARE = 0.39). In a study investigating the effect of vitamin A intake on the incidence of breast cancer, the MLE was more efficient than the matrix method (ARE = 0.75).

title: Analysis of clustered and longitudinal binary data subject to response misclassification.
abstract: Misclassified clustered and longitudinal data arise in studies where the response indicates a condition identified through an imperfect diagnostic procedure. Examples include longitudinal studies that use an imperfect diagnostic test to assess whether or not an individual has been infected with a specific virus. This article presents methods to implement both population-averaged and cluster-specific analyses of such data when the misclassification rates are known. The methods exploit the fact that the class of generalized linear models enjoys a closure property in the case of misclassified responses. Data from longitudinal studies of infectious disease will illustrate the findings.

title: Sensitivity analysis of misclassification: a graphical and a Bayesian approach.
abstract: Misclassification can produce bias in measures of association. Sensitivity analyses have been suggested to explore the impact of such bias, but do not supply formally justified interval estimates.



To account for exposure misclassification, recently developed Bayesian approaches were extended to incorporate prior uncertainty and correlation of sensitivity and specificity. Under nondifferential misclassification, a contour plot is used to depict relations among the corrected odds ratio, sensitivity, and specificity.



Methods are illustrated by application to a case-control study of cigarette smoking and invasive pneumococcal disease while varying the distributional assumptions about sensitivity and specificity. Results are compared with those of conventional methods, which do not account for misclassification, and a sensitivity analysis, which assumes fixed sensitivity and specificity.



By using Bayesian methods, investigators can incorporate uncertainty about misclassification into probabilistic inferences.

title: A Tool for Deterministic and Probabilistic Sensitivity Analysis of Epidemiologic Studies
abstract: Classification errors, selection bias, and uncontrolled confounders are likely to be present in most epidemiologic studies, but the uncertainty introduced by these types of biases is seldom quantified. The authors present a simple yet easy-to-use Stata command to adjust the relative risk for exposure misclassification, selection bias, and an unmeasured confounder. This command implements both deterministic and probabilistic sensitivity analysis. It allows the user to specify a variety of probability distributions for the bias parameters, which are used to simulate distributions for the bias-adjusted exposure–disease relative risk. We illustrate the command by applying it to a case–control study of occupational resin exposure and lung-cancer deaths. By using plausible probability distributions for the bias parameters, investigators can report results that incorporate their uncertainties regarding systematic errors and thus avoid overstating their certainty about the effect under study. These results can supplement conventional results and can help pinpoint major sources of conflict in study interpretations.

title: Case-control analysis with partial knowledge of exposure misclassification probabilities.
abstract: Consider case control analysis with a dichotomous exposure variable that is subject to misclassification. If the classification probabilities are known, then methods are available to adjust odds-ratio estimates in light of the misclassification. We study the realistic scenario where reasonable guesses, but not exact values, are available for the classification probabilities. If the analysis proceeds by simply treating the guesses as exact, then even small discrepancies between the guesses and the actual probabilities can seriously degrade odds-ratio estimates. We show that this problem is mitigated by a Bayes analysis that incorporates uncertainty about the classification probabilities as prior information.

title: Recall bias in a case-control study of sudden infant death syndrome.
abstract: Case-control differences in the accuracy of maternal recall may create spurious associations between suspected risk factors and perinatal conditions. We examined case-control differences in the accuracy of maternal recall and evaluated the impact of maternal reporting errors on observed measures of association. We compared interview information with information recorded on medical records for the mothers of 226 cases of Sudden Infant Death Syndrome (SIDS) and the mothers of 226 living controls. We found that having a child die from SIDS increased the sensitivity of recall for less than half of the 25 study variables. However, for 18 of the 25 variables, the mothers of SIDS cases were more likely than the mothers of living controls to report events that could not be confirmed on medical records. Case-control differences in recall accuracy did not appear to create spurious associations with SIDS or to bias most associations away from the null value.

title: Applied Logistic Regression
abstract: "A new edition of the definitive guide to logistic regression modeling for health science and other applicationsThis thoroughly expanded Third Edition provides an easily accessible introduction to the logistic regression (LR) model and highlights the power of this model by examining the relationship between a dichotomous outcome and a set of covariables. Applied Logistic Regression, Third Edition emphasizes applications in the health sciences and handpicks topics that best suit the use of modern statistical software. The book provides readers with state-of-the-art techniques for building, interpreting, and assessing the performance of LR models. New and updated features include: A chapter on the analysis of correlated outcome data. A wealth of additional material for topics ranging from Bayesian methods to assessing model fit Rich data sets from real-world studies that demonstrate each method under discussion. Detailed examples and interpretation of the presented results as well as exercises throughout Applied Logistic Regression, Third Edition is a must-have guide for professionals and researchers who need to model nominal or ordinal scaled outcome variables in public health, medicine, and the social sciences as well as a wide range of other fields and disciplines"--

title: A conditional model for incomplete covariates in parametric regression models
abstract: Incomplete covariate data arise in many data sets. When the missing covariates are categorical, a useful technique for obtaining parameter estimates is the EM algorithm by the method of weights proposed in Ibrahim (1990). This method requires the estimation of many nuisance parameters for the distribution of the covariates. Unfortunately, in data sets when the percentage of missing data is high, and the missing covariate patterns are highly non-monotone, the estimates of the nuisance parameters can lead to highly unstable estimates of the parameters of interest. We propose a conditional model for the covariate distribution that has several modelling advantages for the E-step and provides a reduction in the number of nuisance parameters, thus providing more stable estimates in finite samples. We present a clinical trials example with six covariates, five of which have some missing values.

title: A practical approach to computing power for generalized linear models with nominal, count, or ordinal responses.
abstract: Data analysts facing study design questions on a regular basis could derive substantial benefit from a straightforward and unified approach to power calculations for generalized linear models. Many current proposals for dealing with binary, ordinal, or count outcomes are conceptually or computationally demanding, limited in terms of accommodating covariates, and/or have not been extensively assessed for accuracy assuming moderate sample sizes. Here, we present a simple method for estimating conditional power that requires only standard software for fitting the desired generalized linear model for a non-continuous outcome. The model is fit to an appropriate expanded data set using easily calculated weights that represent response probabilities given the assumed values of the parameters. The variance-covariance matrix resulting from this fit is then used in conjunction with an established non-central chi square approximation to the distribution of the Wald statistic. Alternatively, the model can be re-fit under the null hypothesis to approximate power based on the likelihood ratio statistic. We provide guidelines for constructing a representative expanded data set to allow close approximation of unconditional power based on the assumed joint distribution of the covariates. Relative to prior proposals, the approach proves particularly flexible for handling one or more continuous covariates without any need for discretizing. We illustrate the method for a variety of outcome types and covariate patterns, using simulations to demonstrate its accuracy for realistic sample sizes.

title: Factors affecting sensitivity and specificity of exercise electrocardiography. Multivariable analysis.
abstract: Unlike the predictive value of a diagnostic test, which depends on the prevalence of disease in the population tested, its sensitivity and specificity have been assumed to be constants. This assumption was examined in patients who had both exercise electrocardiography and cardiac catheterization. The effects on sensitivity of factors from clinical history, catheterization, and exercise performance were defined by multivariable logistic regression analysis in 1,401 patients with coronary disease; effects on specificity were defined by a similar analysis in 868 patients without coronary disease. Five factors had significant, independent effects on exercise electrocardiographic sensitivity: maximal exercise heart rate, number of diseased coronary arteries, type of angina, and the patient's age and sex. Only maximal exercise heart rate had a significant, independent effect on exercise electrocardiographic specificity. Thus, the sensitivity and specificity of exercise electrocardiography vary with clinical history, extent of disease, and treadmill performance; the sensitivity and specificity of other diagnostic tests may also vary.

title: Predictive ability of fasting plasma glucose for a diabetic 2-h postload glucose value in oral glucose tolerance test: spectrum effect.
abstract: The performance of diagnostic tests may vary according to patient characteristics. The aim of this study is to find out the factors, if any, that may affect the performance of fasting plasma glucose (FPG) to predict a diabetic 2-h postload glucose level (> or =200 mg/dl) in oral glucose tolerance test (OGTT). One hundred ninety-six patients with known risk factors for diabetes mellitus to whom OGTT was applied were included. Factors that may have an effect on the performance of FPG in prediction of a diabetic value in OGTT were determined by using logistic regression and likelihood ratios (LRs). The cutoff of FPG predicting a 2-h postload glucose of > or =200 mg/dl was calculated by receiver operating characteristic curve as 110 mg/dl (sensitivity, 76.7%; specificity, 75.9%). Waist-to-hip ratio (WHR) and body mass index (BMI) influenced sensitivity, whereas age, family history, and presence of hyperlipidemia affected specificity of FPG. Significant factors for positive LR were age and hyperlipidemia, whereas sex, smoking, hyperlipidemia, physical inactivity, WHR, and BMI influenced negative LR. Fasting plasma glucose performance as a diagnostic test can be affected by many factors that are clearly stated as risk factors for diabetes mellitus. These data emphasize how the interpretation of a diagnostic test varies as the patient characteristics vary; the criteria that we confidently rely on may not be that reliable, changing between just two different patients.