A New Era in Graduate Medical Education

Field: Health Professions (Health Sciences)

Data from a 2018 program director survey shows that general surgery residency programs receive an average of 742 applications with only 36% receiving an in-depth review.1 Historically, a minimum USMLE Step 1 score has been used as a cutoff to quickly reduce the applicant pool to a manageable number to review for potential interview.1 This uses the USMLE in a way that was not originally intended. Because these scores negatively impact medical student well-being and have innate biases, a change to a pass/fail system was announced for USMLE Step 1 scoring.2 This forces the applicant screening and interview selection process to evolve. The development of a standardized assessment for surgical subinternships can serve as a practical and efficient method to streamline the identification of applicants appropriate for a specific surgical training program. SHIFTING THE PROBLEM Although the Step 1 score will be reported as pass/fail, a 3-digit numeric score will continue to be reported for Step 2 clinical knowledge (CK).2 Currently, Step 1 is more heavily utilized than Step 2 CK in the screening process.1,3 Although pass/fail grading can have significant benefits to the well-being of students by de-emphasizing the impact of USMLE Step 1 scores on residency options, it eliminates the most simplistic method for programs to reduce the large number of applications received.4 Therefore, a shift toward utilizing Step 2 CK scores could occur to keep the screening process simple. Additionally, similar to Step 1, Step 2 CK has been shown to correlate with the American Board of Surgery In-training Examination scores, which positively correlate with the American Board of Surgery pass rates.5,6 However, this again misuses a licensing exam to stratify applicants. This calls for surgical educators to find a pragmatic solution now, irrespective of the changes to the USMLE scoring. INADEQUATE SOLUTIONS Surgery clerkship performance could provide a robust replacement for USMLE scores because they are based on a longitudinal assessment rather than a snapshot in time. However, there are limitations to this method. First, surgery clerkships are designed to provide basic surgical knowledge and skills to all medical students regardless of specialty interests. Second, studies have demonstrated substantial variation and inflation in medical school grading systems.7 Because there is no standardization, the comparison of students across, or even within, institutions based on surgery clerkship grades is problematic. The lack of data regarding the grading system distribution for each medical school is even more troublesome. In 2016, the Association of American Medical Colleges Medical Student Performance Evaluation (MSPE) Task Force made recommendations to revise the MSPE, which included "a level of standardization and transparency that facilitates the residency selection process" and "comparative information on applicants."8 However, a recent study showed MSPE reviewers thought this still required significant improvement.9 The findings highlight the dichotomy between undergraduate medical education (UME) and graduate medical education. UME has a responsibility to help students match into the "best" residency programs. Therefore, with increased transparency, students could suffer if their school's standards are not on par with other medical schools or the revealed grading system is found to be skewed. However, this information would be extremely valuable to residency programs in their selection process. Due to this current lack of UME transparency, programs must sift through the coded messages buried in applications to discover those they think are the top applicants for their residency. A unified grading system would help eliminate this code so that residency programs can make more informed decisions. Yet, this seems unlikely given the wide variability in curriculum structure and assessment between institutions. USMLE Step 2 clinical skills scores and standardized letters of recommendation (SLOR) could be substitutes for the USMLE Step 1 score. Although these may potentially have advantages in other specialties, general surgery programs would likely not experience the intended benefits. Patient care scenarios utilized on Step 2 clinical skills are not primarily surgery-specific and emphasize preventative care, weakening the argument for its use within general surgery. The American College of Surgeons and Association for Program Directors in Surgery created a SLOR for general surgery.10 However, there is a lack of widespread acceptance. This highlights the limitations of SLORs and diminishes its applicability in the general surgery residency selection process. Surgical residency preparatory courses ("boot camps") are another potential consideration. Students matching into surgical residency can participate in these programs and undergo a rigorous and immersive curriculum that trains them in the necessary skills and knowledge required for a postgraduate year (PGY)-1 resident. However, these courses tend to occur in a simulated environment after a student has matched, thereby providing no value in the selection process. NOVEL ROLE FOR SURGICAL SUBINTERNSHIPS One area of untapped potential in general surgery is the utilization of subinternships (also known as "acting internships"), where students are placed in the role of a PGY-1 resident. The majority of subinternships are completed before submitting residency applications, allowing them to be utilized for the selection of interviewees. Moreover, students can be evaluated on their current aptitude, offering better insight to their potential in residency. By identifying specific areas of strength, applicant and program compatibility can be established. We propose that subinternships become a nationally standardized, performance-based process for screening and comparing potential general surgery residency candidates. We recognize that the subinternship experience varies at each institution and reflects the program's unique values. This component is vital for applicants to assess their own compatibility with the program. Therefore, we are not proposing a standardized experience but rather a standardized evaluation, as subinternships provide students with the opportunity to be assessed on clinical acumen, basic surgical skills, and professionalism in the true surgical environment. In this sense, programs would be able to utilize subinternships as objective measures for identifying applicants that best fit their residency, exclusive of other factors. Although bias in completing these evaluations is a risk, this would be mitigated by standardization. Systems such as this seem to be effective in other elite selection processes, such as those used by the Navy Sea, Air, and Land Forces (SEALs) – basic underwater demolition/SEAL (BUD/S) training. In BUD/S training, the personnel are placed under the extreme physical conditions and evaluated in the true environment typically required of this job. By all accounts, this is exceptionally advantageous for those involved in the selection process and training, as those candidates who cannot adequately perform the tasks required of the job are not advanced. This benefits the trainees, as they recognize they are not appropriately matched for the position, and the program because they only select those who can carry out the required responsibilities. In this fashion, subinternships could help provide students with objective feedback that is needed in this process to help create compatibility between students and programs. Although students can only complete a small number of subinternships before submitting residency applications, this standardized assessment should translate to all programs. Moreover, a rationale can be presented for students to complete these subinternships at different institutions ("away rotations") to reduce evaluator bias, as outside faculty would not have incentives to inflate their abilities. However, with standardization, students will be afforded the opportunity to complete subinternships at their home institution, alleviating potential financial concerns or scheduling restrictions. We recognize that BUD/S training has its own inherent bias and is an extreme example. Furthermore, while surgeons can work in many different environments, such as private, academic, government, or military institutions, BUD/S only trains and selects for military personnel. Yet, the Navy SEAL in-situ selection process through the use of BUD/S training could establish a precedent for the role of subinternships in the comparative process for general surgery applicants and produce similar results in regards to student evaluation and program fit. We are not advocating to place these students under undue stress. Instead, we are simply calling for an honest assessment of students' performance with a standardized grading system to better determine their projected career path in general surgery. If a standardized assessment system within surgical subinternships is developed, it holds much promise in reliably evaluating students applying to general surgery residency and could play a significant role in the interview selection process. We propose that students completing surgical subinternships be assessed on 3 core standards: clinical practice, surgical practice, and professionalism (Fig. 1). These standards are fundamental in trainee assessment within medical school and residency. In this respect, we can bridge the gap between UME and graduate medical education, providing the transparency needed to critically assess medical students' achievement of core entrustable professional activities for entering residency and potentially predicting their aptitude as a future PGY-1 surgical resident. These 3 standards can then be incorporated into an assessment system that can be implemented by all institutions. This would unify the evaluation process of medical students applying to general surgery residency and aid programs in identifying well-suited applicants that warrant an in-depth review for interview. It is our hope that this perspective can stimulate the conversation regarding surgical subinternships and their potential role as a meaningful entity that can benefit both general surgery residency applicants and programs.FIGURE 1: Proposed surgical subinternship core standards. These are the proposed core standards on which to assess students during their surgical subinternship. They are based on an overlap between the Association of American Medical Colleges (AAMC) Core EPAs for Entering Residency and the ACGME Surgery Residency Level 1 Milestones 2.0.