The Risk of Determining Risk with Multivariable Models

Although most physicians have received no instruction in multivariable methods of statistical analysis, the methods now commonly appear in medical literature. The results of multivariable analyses are often expressed in statements such as, When other risk factors are controlled, a decrease of 5 units in substance X reduced disease by 10%, or After adjustment for age and stage of disease, treatment with procedure Y reduced mortality by 25%. Our purpose in the current research was to note the frequency with which multivariable analyses now appear in general medical journals, to identify some common problems and desirable precautions in the analyses, and to determine how well the challenges are being met. The investigation also provided a framework for a brief reviewintended for clinical readersof commonly used multivariable statistical methods. General Principles Format of Multivariable Analysis In the types of multivariable analyses discussed here, the mathematical expressions described in Appendix 1 are used to relate two or more independent variables to an outcome or dependent variable. In those expressions, a linear regression coefficient indicates the impact of each independent variable on the outcome in the context of (or adjusting for) all other variables. The values of the regression coefficients are obtained as the best mathematical fit for the specified model, although the selected model and the multivariable analysis may or may not provide a good absolute fit for the data. The four main multivariable methods in medical literature have many mathematical similarities but differ in the expression and format of the outcome expressed as the dependent variable: 1. In multiple linear regression, the outcome variable is a continuous quantity, such as blood pressure in millimeters of mercury or sodium concentration in milliequivalents per liter. 2. In multiple logistic regression, the observed outcome variable is usually a binary event, such as alive versus dead or case versus control. The event occurs at a fixed point in time, such as mortality 1 year after surgery. 3. In discriminant function analysis, the outcome variable is a category or group to which a subject belongs. For example, patients may be classified as having obstructive, restrictive, vascular, or other forms of pulmonary dysfunction. The analytic results are often converted to a score used to classify observations into one of the categorical groups. For only two categories (such as healthy or diseased), this form of multivariable analysis produces results similar to logistic regression. 4. In proportional hazards regression, which is also known as Cox regression [1], the outcome variable is the duration of time to the occurrence of a binary failure event during a follow-up period of observation. The most common event in such analyses is death, but other failures can also be modeled. For example, each person's final state can be classified as either dead at a specified time or as censored if lost to follow-up or alive at the end of the study period. The results of a Cox model may be considered as an instantaneous incidence rate for the failure event. The technical details of all four methods have been described elsewhere [2-6], and the current review is limited to the way two of the methodsmultiple logistic regression and proportional hazards analysisare used in medical literature. These multivariable methods have become particularly popular because binary dependent variables, such as survival state, are frequently used in clinical research. (Although the term multivariate is often used interchangeably with multivariable, the former does not apply to the most common medical situations, where a single outcome variable is studied.) Purposes of Multivariable Analysis Multivariable methods can be used for at least five major purposes. Bivariate Confirmation Before multivariable analyses are initiated, most investigators have previously done simple bivariate analyses in which each independent variable is evaluated, one at a time, for its association with the outcome variable. (Because of the one-at-a-time examination, the analyses are often called univariate, although two variables are being associated.) After the important independent variables have been identified in the simple analyses, the multivariable examination is usually done to confirm that the independent variables retain their importance in the simultaneous context of the other variables. For example, the effect of smoking on cardiovascular mortality can be evaluated along with the concomitant effects of age, systolic blood pressure, serum cholesterol level, and serum triglyceride value. If the initial bivariate impact of smoking is changed in the multivariable context, specific relationships can be investigated further. Multivariable Confirmation A second purpose of the multivariable methods is to confirm the results of non-regression analyses such as adjustment by cross-stratification [7] or standardization [8]. For example, data on survival after prostatectomy, independent of type of surgery, can be examined for subjects cross-stratified in categorical groups of age and severity of illness. After suitable composite strata are formed, long-term mortality can then be examined in each stratum for transurethral compared with open prostatectomy, and the results can be checked with a multivariable regression model [9]. Screening With large administrative data bases, the total number of variables may make bivariate analyses too time-consuming to obtain and difficult to assimilate. In such situations, multivariable analysis can be used as an initial screening process. Important variables can be suggested by the screening, with thresholds of quantitative and statistical importance (significance) determined by the researcher; more detailed analyses can then be performed. Software programs typically have default values that can be modified for levels of statistical significance, but quantitative significance is usually ignored by the computer and must be recognized and specified by the analyst. Creating Risk Scores The multiple variables that seem important may be assigned simple rating scores and combined into a single risk score used for predicting outcomes of individual patients. The choice of these ratings can be aided by the regression coefficients found in the multivariable analysis. A well-known simple, single combination of arbitrary ratings is the Apgar score, created by Virginia Apgar, who assigned integer ratings of 0, 1, or 2 to each of five variables in the assessment of a newborn infant [10]. Although Apgar's initial decisions were made using clinical judgment and were not related to infants' outcome, a multivariable analysis might have been used to suggest suitable weights for the ratings. Quantifying Risk of Individual Variables In many instances, particularly in studies of risk factors, the impact of an individual variable is quantified for its specific effect among the other independent variables. For example, proportional hazards analysis was used to determine that a 19% reduction in coronary heart disease risk was associated with each decrement of 8% of total plasma cholesterol [11], adjusting for age, systolic blood pressure, cigarette smoking, and other factors. In these situations, the regression coefficients found in the multivariable analysis are converted to expressions of relative risks or hazards (with Cox regression) or odds ratios (with multiple logistic regression), indicating the individual variable's effect on the outcome. Assumptions and Limitations In the first three purposes just cited, the multivariable analyses are used mainly to confirm results that have already been documented with other methods or to suggest important variables that will be evaluated later in greater detail. In the fourth purpose, the risk score will combine the different variables and will demonstrate the impact of the combinations rather than each variable alone. Although the assumptions and limitations of multivariable models are important, they may be considered as having a secondary role when the combined scores are formed. In the fifth purpose, when individual variables are examined to determine the magnitude of their impact or risk, the estimates will vary with the structure of the mathematical model and the coding of variables. The assumptions and limitations of the multivariable methods then become especially important for ensuring accurate results and valid interpretations. Methods Selection of Articles The BRS Colleague complete text computer data base [12] was used to identify the use of the four cited multivariable methods in articles from The Lancet and The New England Journal of Medicine from 1985 to 1989. The search was limited to these two journals because they include a variety of general clinical topics, have large circulations, and have complete texts (not just keywords) available in the data base. The computer search was designed to focus on original and special articles while excluding editorials, reviews, letters, and so forth. The search process included the text but not the references of published articles. The main terms used in the search included multiple linear, discriminant function, logistic, proportional hazards, or Cox. Each main term was paired with regression, analysis, function, model, and method. The accuracy of the computer search was checked with a manual inspection of articles published in each journal during a 6-month period. After the initial search confirmed that the use of logistic regression and proportional hazards analysis was particularly frequent, these two methods were selected for further review. For this purpose, a random, stratified sample of 60 articles [28-87] was selected to provide three articles for each of the two methods, for each of the two journals, for each of the 5 years 1985 through 1989. With information about each usage ex