The unspoken tyranny of regulatory agencies: a commentary on the ACGME Resident Survey.

Residency training directors dread various moments— for instance, when the Residency Review Committee (RRC) reviews their program, or Match Day. Another such moment of anxiety is when residents answer the Accreditation Council for Graduate Medical Education (ACGME) Resident Survey. This survey asks residents about their compliance with duty hours and their opinions of faculty, the feedback they receive, the educational content of the teaching, their program’s resources, and, finally, their overall experience. These are seemingly fair, important questions. Why would anyone dread seeing the answers? What would be wrong with a survey asking this information about the program? Why would anybody think that this is not a good and useful survey? In my opinion, at least four cardinal problems with the ACGME Resident Survey may trigger training director anxiety and dread. The first problem is the validity and comprehensibility of the survey questions. Fahy and colleagues (1) suggested that the responses obtained on the ACGME Survey might inaccurately reflect the magnitude of noncompliance found in certain areas, and they proposed that this discrepancy might be due to the limited range of responses available on the survey. Sticca and colleagues (2) reported that 14% of residents admitted to not answering the questions truthfully, and 37% of residents felt that the survey did not provide an accurate evaluation of their work-hours. These authors felt that a tool in which 1 in 7 responders admits to answering questions falsely and 1 in 5 responders had difficulty interpreting the questions may not be a valid evaluation tool. In contrast, Holt and colleagues (3) found that the ACGME Survey demonstrated a high degree of internal reliability and thus felt that this survey was a reliable, valid, and useful tool for evaluating residency programs. Holt and colleagues (3) also reported that programs having resident-identified duty-hour issues were more likely than those without such issues to have received duty-hour citations from residency review committees (sic). These are contradictory results. Some might question the first two studies (1, 2) because they were conducted by surgeons who have been unhappy with the implementation of the 80 duty-hour limit. Othersmay question the third study (3) because it was conducted by the main stakeholder and creator of this survey: the ACGME itself. Unfortunately, these studies, as well as many critics of this survey, focus on just one part of the survey—duty-hours —which is probably the most objective part of the ACGME Resident Survey. I have a harder time with the interpretation of some questions in the other parts of the survey. For instance: “How sufficient is the supervision you receive from faculty and staff in your program?” (Possible answers: Extremely, Very, Somewhat, Slightly, Not At All); “Thinking about the faculty and staff in your program overall, how interested are they in your residency education?” (same answers); or “Thinking about the faculty and staff in your program overall, how effective are they in creating an environment of scholarship and inquiry?” (same answers) are all very subjective questions. Their interpretation could vary profoundly from resident to resident. Similarly, the question “In your opinion, howoften do your rotations and other major assignments provide an appropriate balance between your residency education and other clinical demands?” sounds ambiguous. What constitutes an appropriate balance is a matter of opinion, and this question raises further questions: What is considered education? Is seeing patients under attending supervision “clinical demand”or “education”? Is classroom teaching the only form of education? Or consider another question—“Howoften do youwork in interdisciplinary teams to care for patients?” What does the answer “Somewhat/Sometimes” mean? Are we expecting that residents work always in interdisciplinary teams? If they see patients in individual therapy (i.e., no interdisciplinary team), does it mean “noncompliance”? I could discuss Received April 6, 2011; revised June 28, November 2, 2011; accepted November 2, 2011. Send correspondence to Richard Balon, M.D., Wayne State Univ., Univ. Psychiatric Center; e-mail: rbalon@wayne.edu Copyright © 2012 Academic Psychiatry