Systematic Review: The Relationship between Clinical Experience and Quality of Health Care

Context While practice makes perfect in some situations, physicians' knowledge and performance may decline with the passage of time. Contribution Of 62 published studies that measured physician knowledge or quality of care and described time since medical school graduation or age, more than half suggested that physician performance declined over time for all outcomes measured. Only 1 study showed improved performance for all outcomes measured. Implications This review should provoke careful study of the relationship of physician experience and the quality of care. It also raises concerns about the adequacy of continuing professional education in medicine. The Editors Quality assurance and performance evaluation have become central issues in medicine. Care is suboptimal in many different medical conditions and clinical settings (1-6). Although delivering high-quality care is important to all clinicians, this issue may be particularly relevant to certain subgroups, such as physicians with less specialized training and those who see a smaller volume of patients (7-10). Physicians who have been in practice for more years may also be less likely to deliver high-quality care (11, 12). Medical advances occur frequently, and the explicit knowledge that physicians possess may easily become out of date. Therefore, although it is generally assumed that the tacit knowledge and skills accumulated by physicians during years of practice lead to superior clinical abilities (13), it is also plausible that physicians with more experience may paradoxically be less likely to provide technically appropriate care. Few existing studies have had the specific goal of evaluating the effects of experience on the quality of medical care (11). However, length of time in clinical practice has been included as part of a set of physician characteristics that might explain variations in quality or that may be confounders of the association between quality and other factors (13-18). The purpose of this paper is to assess the robustness of the relationship between clinical experience and quality of care by systematically reviewing empirical studies. Although we define experience as the number of years a physician has been in practice, physician age and time in practice are highly correlated (11, 19, 20); therefore, for the purposes of this paper, we consider these variables to be interchangeable. Methods We searched MEDLINE (Ovid Technologies, 1966 to June 2004; English language) for terms describing physician experience (keywords: physician age, clinician age, physician experience, clinician experience), physician demographic characteristics (keywords: physician characteristics, clinician characteristics), practice variation (subject heading: physician's practice patterns), and performance in various domains (subject headings: clinical competence, health knowledge, attitudes and practice, outcomes assessment [health care]; keywords: knowledge, guideline adherence, appropriateness, outcomes). We retrieved potentially relevant articles and reviewed their reference lists to identify studies that our search strategy may have missed (Figure 1). We also searched our personal archives to identify additional studies. We included studies if they 1) were original reports providing empirical results; 2) measured knowledge, guideline adherence, mortality, or some other quality-of-care process or outcome; and 3) included years since graduation from medical school, years since certification, or physician age as a potential explanatory variable. We excluded studies if they described practice variation that is not known to affect quality of care (for example, assessed test-ordering behavior in clinical situations where optimal practice is unknown) or evaluated the performance of fewer than 20 physicians. For studies that examined several different end points, we included only those outcomes that are linked to knowledge or quality of care. Figure 1. Selection process for studies included in analysis. We used a standardized data extraction form to obtain data on study design and relevant results. We categorized studies into 4 groups on the basis of whether they evaluated knowledge (for example, knowledge of indications for blood transfusion), adherence to standards of care for diagnosis, screening, or prevention (for example, adherence to preventive care guidelines), adherence to standards of care for therapy (for example, appropriate prescribing), or health outcomes (for example, mortality). We classified the results of each study into 6 groups on the basis of the nature of the association between length of time in practice or age and performance: consistently negative, partially negative, no effect, mixed effect, partially positive, and consistently positive. Consistently negative studies were those for which all reported outcomes demonstrated a statistically significant decrease in performance with increasing years in practice or age. Partially negative studies showed decreasing performance with increasing experience for some outcomes and no association for others. We used similar definitions for consistently positive and partially positive studies. Concave studies found performance to initially improve with years in practice or age, then peak, and subsequently decrease. We did not use formal meta-analytic techniques because the included studies used many different effect measures and some did not report parameter estimates. Since studies based on self-reported practice may suffer from social desirability bias (21), we explored the effect of study quality on results by subcategorizing studies according to whether they measured outcomes with self-reports (that is, using surveys and interviews) or observed practice (that is, using chart audits or administrative data review). We also compared studies according to whether they performed multivariable modeling to adjust for patient and physician covariates. We used the Fisher exact test to compare the observed frequencies. We conducted all analyses with SAS, version 8.2 (SAS Institute, Inc., Cary, North Carolina). Role of the Funding Source The Harvard Pilgrim Health Care Foundation supported this study. It had no role in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication. Results Fifty-nine articles that reported data on 62 groups of relevant outcomes formed the basis of our analysis. Overall, 32 of the 62 evaluations (52%) demonstrated a negative association between increasing experience and performance (that is, performance decreased as experience increased) for all outcomes assessed; 13 (21%) reported a negative association for some outcomes but no association for other outcomes; 2 (3%) reported a concave relationship (that is, performance initially increased as experience increased, then peaked, and subsequently decreased); 13 (21%) reported no association; 1 (2%) reported a positive association (that is, performance increased as experience increased) for some outcomes but no association for other outcomes; and 1 (2%) reported a positive association for all outcomes assessed (Figure 2). Figure 2. Distribution of study results relating physician age to clinical performance in various domains. Knowledge Twelve studies assessed the knowledge of practicing physicians, and all studies reported a negative association between knowledge and experience (Table 1). Studies by Ayanian and colleagues (7) and Salem-Schatz and colleagues (22) had large sample sizes, high response rates, and good sampling methods; used rigorous criteria to evaluate knowledge; and performed multivariate analysis. Table 1. Studies Relating Length of Time in Practice or Physician Age to Knowledge Ayanian and colleagues (7) surveyed physicians to assess their beliefs about the survival benefit of therapies for acute myocardial infarction; the appropriate use of these therapies has been well-established in randomized, controlled trials. Specialists were more knowledgeable than generalists; however, after adjustment for this and other variables, physicians younger than 40 years of age were more likely to correctly believe in the value of therapies that improve survival (for example, thrombolytic agents, aspirin, and -blockers). They were also significantly less likely to believe in the value of therapies that have been disproved (for example, prophylactic lidocaine) (P< 0.05). Salem-Schatz and colleagues (22) interviewed surgeons and anesthesiologists to assess their knowledge of the risks associated with and indications for the transfusion of blood products. They found a highly significant negative association between knowledge and the number of years the physicians had been in practice (P< 0.001). Adherence to Standards of Practice for Diagnosis, Screening, and Prevention Twenty-four studies have assessed the appropriateness of physician use of diagnostic and screening tests, as well as preventive health care (Table 2). Overall, 15 (63%) of these studies demonstrated that physicians in practice for more years were less likely to adhere to standards of practice in this domain. Table 2. Studies Relating Length of Time in Practice or Physician Age to Adherence to Standards of Appropriate Diagnosis, Screening, and Preventive Health Care In the largest of these studies, Czaja and colleagues (33) surveyed participants to assess their adherence to cancer screening guidelines endorsed by the American Cancer Society and the National Cancer Institute. Physicians who had graduated more than 20 years before the survey were consistently less likely to adhere to recommended practices (odds ratio, 0.62 to 0.72; P< 0.05). Using more objective measures of guideline adherence, Aubin and colleagues (17) assessed the practice of 21 physicians and found that after adjustment for patient and physician covariates, younger physicians were more likely to appropriately screen for hypertension (odds ratio, 1.11 [95