The Discrepancy between Observational Studies and Randomized Trials of Menopausal Hormone Therapy: Did Expectations Shape Experience?

Although many observational studies have found that postmenopausal hormone therapy (HT) reduced the risk for coronary heart disease (CHD) (1-3), recent randomized trials have challenged this truth. When a randomized study of HT (4) reported an increased risk for recurrent CHD among HT users, the discrepancy was interpreted as evidence that experimentation trumps observation (5). The conclusions of dozens of observational studies were attributed to a consistent bias because consistent biases are expected to yield consistent results (5). The choice by healthier women to use HTthe healthy-user effectwas proposed as the principal explanation (6-8). Women who choose to use HT have better cardiovascular risk profiles and are more educated than women who do not choose to use HT (9-11). Although many observational studies were adjusted for several potential confounders, not all differences between users and nonusers could be measured, nor could adjustment be complete. Few studies adjusted for socioeconomic status, an important predictor of morbidity and mortality (12). Studies that corrected for socioeconomic status reported a higher relative risk for CHD events compared with that reported in studies that did not correct for socioeconomic status (relative risk, 0.97 vs. 0.71) (13). Even studies that corrected for socioeconomic status, however, continued to find that current HT use prevented CHD death (relative risk, 0.62) and had no effect on incident CHD, suggesting that differences in socioeconomic status among HT users only partially account for discrepant CHD findings. Other related biases have been described. Hormone therapy users adhere to treatment regimens more often than nonusers, and adherent patients have a lower risk for CHD than nonadherent patients (14, 15). Hormone therapy users have more frequent contact with physicians than nonusers, which may lead to better clinical outcomes. Selection biases should affect all clinical outcomes; the healthy-user effect should especially affect clinical outcomes with important modifiable risk factors. The magnitude of the effect is expected to be proportional to the extent to which lifestyle factors affect risk. However, risk estimates from the observational Nurses' Health Study (NHS) (2, 16) were concordant with those from the randomized Women's Health Initiative (WHI) trial (17, 18) on all major clinical outcomes except CHD (Figure 1). Coronary heart disease and stroke share the same modifiable risk factors (19-21), and pulmonary embolism is closely associated with atherosclerosis (22). Therefore, any biases that select women at lower risk for CHD should also select women at lower risk for stroke and pulmonary embolism, although to varying degrees. Figure 1. Pairwise comparison of the relative risk for various neoplastic and vascular clinical outcomes associated with menopausal hormone therapy use reported in the Nurses' Health Study ( NHS ) and the Women's Health Initiative ( WHI ) trial. Differences in the diagnostic criteria for CHD in the NHS and the WHI trial could result in an isolated discrepancy in CHD end points. Diagnostic (or reporting) bias would occur if the diagnosis or classification of an outcome were influenced by knowledge of the patient's exposure status (23-25). The WHI trial and the NHS defined CHD as the combined end point of nonfatal myocardial infarction and CHD death. However, their definitions of these end points differed substantially in ways that could have introduced bias (26, 27) (Appendix Table). Appendix Table. Comparison of the Definition of Coronary Heart Disease between the Nurses' Health Study and the Women's Health Initiative Trial Nonfatal Myocardial Infarction Approximately one third of myocardial infarctions among women are unrecognized (or silent) and are detected only by routine electrocardiography (28-30). About half of affected patients have no symptoms; the remainder have atypical symptoms. The WHI trial included silent infarctions. The NHS did not. Excluding silent infarctions could affect the comparative risk for nonfatal CHD if HT use influenced ascertainment of silent infarctions (Figure 2). Because they often present with atypical ischemic symptoms (31-33), women who believed that HT reduced the risks for CHD might be less likely to interpret ischemic symptoms as related to CHD, might not seek medical attention, and might present differently to their physicians, leading to more unrecognized infarctions among HT users. Figure 2. Effect of excluding silent myocardial infarction. NHS WHI MIs Because mortality rates after silent and recognized infarctions are similar (28, 30, 34), excluding silent infarctions should not substantially affect the risk for CHD death. However, exclusion of unrecognized infarctions could lead to systematic undercounting of nonfatal CHD events, which account for more than 75% of CHD events. No analyses have explored the prevalence of silent infarctions according to HT use. However, the higher rate of silent events among women has been attributed to the belief that CHD is uncommon among women (35, 36). The only observational study to conclude that HT substantially and significantly increased incident CHD risk, the Framingham Study (37), included silent infarctions in the count of CHD events. CHD Death Deciding whether a death, especially a sudden death, resulted from CHD can be subjective. Approximately 70% of CHD deaths occur suddenly outside the hospital, and only half are associated with previous angina or infarction (38). Because cardiac arrest is the common final pathway to death from other diseases, many death certificates list CHD as a cause of death. Whether CHD was listed as the primary cause of death in the NHS, however, was at the discretion of the study investigators who used medical records and interviews with family members to ascertain the most plausible cause of death. Inferring cause of death often involves subjective reasoning and the discretion of the physician preparing the death certificate. If the physician, the family member interviewed, or the investigator interpreting the death certificate were aware that a patient used HT and believed that HT decreased the risk for CHD, ascertainment of CHD death among HT users could be biased. Although both the NHS and the WHI trial reported that their investigators were blinded to participants' HT use, the protocol for observational studies makes true blinding nearly impossible. The NHS reports that in no case was the cause listed on the death certificate used as the sole criterion for coronary death and for all deaths possibly attributable to cardiovascular causes, we requested permission to review the medical records (16). Most records were reviewed by 2 senior investigators, and references to HT exposure status were not expunged before medical record review. Physicians who prepared death certificates were not blinded to patients' HT use. In contrast, investigators and physicians who prepared death certificates in the WHI trial were truly blinded because the medical records did not contain patients' treatment assignments. Although 40% of women assigned to HT (to manage vaginal bleeding) in the WHI trial were unblinded to gynecologists, the gynecologists were not typically involved in preparing death certificates. Because ascertaining the cause of death often requires subjectively interpreting medical records and death certificates, any implicit bias of the reviewer or the physician preparing the death certificate could lead to a spurious protective effect of HT. Mathematical Modeling We developed a mathematical model to simulate the effect of excluding silent infarctions and misclassifying CHD deaths on the observed risk for CHD. Using standard spreadsheet software (Microsoft Excel, Microsoft, Inc., Redmond, Washington), we simulated 2 cohorts of women (1 cohort using HT, the other cohort not using HT) and counted CHD events with and without the hypothesized biases. The proportion of unrecognized nonfatal infarctions was varied from the base rate of 34% according to HT use. Proportional deviations from this base rate were applied to both HT users and nonusers. Only recognized infarctions were counted in the observational study simulation, whereas both recognized and silent events were counted in the randomized study. The effect of misclassifying CHD deaths was modeled by altering the percentage of deaths that are misclassified as CHD deaths according to HT use. Differences in socioeconomic status were modeled by comparing the summary relative risk for CHD among current HT users in studies that adjusted for socioeconomic status with that for studies that did not adjust for socioeconomic status (relative risk, 0.97 vs. 0.71) (13). Thus, we assumed that unadjusted differences in socioeconomic status decrease the observed relative risk by 1 (0.71/0.97), or 27%. Effect of Excluding Silent Infarctions If the likelihood of recognizing an infarction is 50% lower among HT users, the observed relative risk decreases from 1.29 to 0.89. The greater the proportion of silent nonfatal infarctions, the greater the decrease in the observed relative risk (Figure 3, top). Figure 3. Sensitivity analyses on the effect of misclassifying silent myocardial infarctions ( MIs ) and coronary heart disease ( CHD ) deaths on the observed relative risk for CHD. Effect of Misclassifying CHD Deaths Misclassifying 50% of CHD deaths decreases the observed relative risk from 1.29 to 0.96 (Figure 3, middle). The greater the ratio of fatal to nonfatal CHD events, the greater the effect on the observed risk. Combined Effect Both biases are probably present to some extent. Misclassifying 20% of CHD deaths and differentially recognizing 20% of nonfatal infarctions reverse the apparent direction of risk from 1.29 to 0.99 (Figure 3, bottom). These biases are synergistic. To change the relative risk for CHD among HT users from 1.29 to 0.6 (the risk observed in the NHS) (16