Influence of Patient Education on Profiles of Physician Practices

The widespread implementation of managed care in the United States has called attention to the quality of the health care delivered by health maintenance organizations (HMOs) and their participating providers (1). Quality indicators are often seen in HMO report cards (2-4) or physician profiles (5), which usually include a measure of patient satisfaction and a combination of process and outcome measures. Recently, interest has developed in using various health status measures to evaluate the performance of HMOs and providers (7, 8). For example, the Health Care Financing Administration's Health of Seniors program has begun to use the Medical Outcomes Study Short Form 36-Item Health Survey (SF-36) to assess changes in the physical and mental health status of elderly patients enrolled in HMOs across the United States (9). However, performance reports and profiles are not usually adjusted for patients' level of education (10). Because patients' education level often differs by both physical and mental health status (11-16) and by provider, it may confound providers' health outcome profiles. Level of education may also be associated with patient satisfaction (17) and may similarly confound performance measures of this variable. Consequently, practice profiles of physicians who care for poorer patients may reflect lower mean health status and satisfaction simply because of patients' demographic characteristics. In other words, current performance measures may unfairly penalize providers who care for poorer patients. In contrast to case-mix adjustment, limited empirical data are available about the effect of patients' education level on performance profiles (18). We evaluated the effect of patients' education level on the rank of physician practices, which was determined by three performance measures: patient physical health status, patient mental health status, and patient satisfaction. Methods Physician Sample We analyzed data taken from a study of the referral behavior of primary care physicians (19). As part of the original study, 100 primary care physicians (internists and family physicians) were randomly selected for participation if they had been practicing for at least 2 years before the office survey, had at least 100 patients enrolled in the largest local managed care organization (which enrolls more than 50% of the local population) during both of those years, and reported no area of specialization that prompted referrals from other physicians. Of the 457 primary care physicians practicing in the metropolitan Rochester, New York, area, 274 met these criteria. All eligible physicians who agreed to participate in an office survey and respond to a questionnaire were enrolled, and 182 (66%) agreed to participate. To reach previously determined power requirements, we selected a subsample of eligible physicians by using random numbers for the office survey component of the study. Profiles of Physician Practices Physicians were ranked on the basis of mean patient physical and mental health status scores and patient satisfaction scores; these scores were derived from patient surveys conducted in the physicians' offices. All ranks were adjusted for age, sex, case mix, and insurance status. We examined the effect of additional adjustment for patients' education level on ranks. Patient Selection and Survey Research assistants distributed self-administered surveys to at least 50 consecutive patients 30 years of age or older who visited the offices of each of the participating physicians. Patients who could not complete the questionnaire by themselves were given assistance. In addition to completing health status, patient satisfaction, and morbidity instruments, patients were asked to provide their age, sex, and level of education. Patient Health Status Patient physical and mental health status were assessed by using the Short Form 12-Item Health Survey (SF-12), a self-report instrument that allows construction of a physical component summary (PCS) and a mental component summary (MCS) (20). Scores for each component ranged from 0 to 100; higher scores indicated better health. All items were derived from the SF-36 to reproduce the PCS and MCS scores of this instrument in a U.S. population sample (20). For predicting the SF-36 PCS and MCS scores, initial studies showed that the regression coefficient was 0.91 for the physical health component subscale and 0.92 for the mental health component subscale. Test-retest reliabilities of the two scales were 0.89 and 0.76, respectively (20). Several subsequent studies conducted in national and international samples, including samples similar to ours, have confirmed the reliability and validity of the SF-12 (20-23). Patient Satisfaction Patients completed the general satisfaction subscale from the Patient Satisfaction Questionnaire (24). Higher scores indicated greater satisfaction (range, 1 to 5). Nunnally and Bernstein (25) suggested that an internal scale consistency greater than 0.8 provides a satisfactory level of reliability for research purposes. In our study, the internal consistency (measured by using the Cronbach value) was 0.84 (95% CI, 0.83 to 0.85). Case-Mix Adjustment We used the method described by Katz and colleagues (26) to assess patient morbidity from a checklist of chronic diseases and their treatments. Among hospitalized patients older than 50 years of age, patient-generated indexes were found to be reliable and valid compared with chart review and were equally predictive of resource utilization (26). We used self-reported years of education as a measure of socioeconomic position. Statistical Analysis Three sets of mixed-effects regression analyses were fitted (27), one for each of the three performance outcomes (physical health status, mental health status, and satisfaction). Because these outcomes were approximately normally distributed, the models were fitted with an identity link and a Gaussian error structure. To account for the natural correlation of patients in a particular physician's practice, the models assumed a compound symmetric (exchangeable) correlation structure. These models estimated the variation in patient outcomes by physician (random effect) after adjusting for patient characteristics (fixed effects). Patient factors included age, sex, insurance status, clinical condition (case mix), and education. An interaction term for education level of an individual patient and a term for the mean education level of all patients in the practice were also included as a fixed effect. Ranks of physician practices for each outcome were determined from the estimates of variables for the physician random effects. Practices with the largest positive random-effects estimates, which represented the largest positive departure from the average of the 100 practices, were ranked the highest. Those with the largest negative random-effects estimates were ranked the lowest. These mixed-effects, or hierarchical, models adjust for the clustering induced by the nesting of patients within practices and also tend to produce better, more stable estimates of relative performance (profiling) by shrinking all of the physician-specific estimates toward the overall mean (28-30). By including or deleting patients' education level from the models and then examining the changes in ranks, we tested the effect of education on observed physician performance. By including an interaction term between the education level of individual patients and the mean education level of all patients in the practice, we examined the differences in the effect of adjusting for education between practices whose patients had a higher mean education level and those whose patients had a lower mean education level. All analyses were conducted by using the MIXED procedure (SAS, version 6.12, SAS Institute, Inc., Cary, North Carolina). Role of the Funding Source The original study protocol was approved by the institutional review board at the University of Rochester and was supported by a grant from the Agency for Health Care Policy and Research. The funding source had no role in the design, conduct, or reporting of the study. Results The mean physician age was 46.6 years. Eighteen percent of physicians were women, and 66% were internists. Compared with the 357 local primary care physicians who did not participate in the office survey, those who participated had more patients who were enrolled in the largest local managed care organization (mean, 586 patients compared with 421 patients; P<0.001); participating physicians also generated lower costs per patient per year in that organization ($1091 compared with $1422; P=0.02). No statistically significant differences were seen in mean physician age (47 years compared with 48 years), sex (82% men compared with 79% men), number of practice partners (3.4 for both), or average age of patients in the managed care organization (45 years for both). Of patients who were approached to participate in the study, 98% (n=5155) agreed to enroll. The mean ( SD) PCS score was 44.5 11.5, the MCS score was 49.1 10.7, and the mean satisfaction score was 4.02 0.39; all of these variables were approximately normally distributed. Table 1 shows the average mean ( SD) and range of patient characteristics for the approximately 50 patients sampled in each practice. Table 1. Patient Characteristics by Practice (n=100) The change in physician rank associated with adjustment for education is shown in Table 2. Ranks ranged from 1 to 100; 1 represents the best rank. For the outcome of physical health status, the rank of 18 physicians changed by 10 or more; for the outcome of mental health status, the rank of 11 physicians changed by 10 or more; and for the outcome of satisfaction, the rank of 2 physicians changed by 10 or more. These effects are reflected in the regression analyses, in which education had the largest effect on the analysis of physical health status (F=10.6; P<0.001), a m