Entrustable professional activities for gastroenterology fellowship training
Authors: Suzanne Rose, Oren K. Fix, Brijen J. Shah, Tamara N. Jones, Ronald D. Szyjkowski
Journal: Hepatology (2014)

The development of entrustable professional activities (EPAs) for gastroenterology (GI) fellowship training has followed a model engagement of 5 GI societies (American Association for the Study of Liver Diseases [AASLD], American College of Gastroenterology [ACG], American Gastroenterological Association [AGA], American Neurogastroenterology and Motility Society [ANMS], and American Society for Gastrointestinal Endoscopy [ASGE]) to work collaboratively toward the development of a committee called the Oversight Working Network (OWN). A sixth GI society (North American Society for Pediatric Gastroenterology Hepatology and Nutrition. [NASPGHAN]) helped provide direction and feedback throughout the process; in addition, the committee had representation from the GI Program Directors Caucus from the beginning of the project, with input and feedback from the GI Program Directors Caucus Steering Committee and the governing boards and education and training committees of the societies before finalization. The acronym "OWN" clearly confirms the strong belief of the societies that we need to take charge of the educational components of our subspecialty and ensure that the needs of our trainees, program directors, and educators are met. The OWN product is intended to support the need to complete the reporting milestone requirement of the Accreditation Council for Graduate Medical Education (ACGME).1 Reporting milestones are a key component of the Next Accreditation System (NAS) and will be a required component of fellow and program evaluation. All internal medicine (IM) subspecialty societies have agreed to accept a common set of reporting milestones, which were released by the ACGME in February 2014. These were developed for subspecialties in IM based on the IM Reporting Milestones2 and were further reviewed and revised with input from all subspecialty societies, including those in gastroenterology and hepatology.3 It should be noted that hepatology is included in general GI fellowship training, and therefore the EPAs by consensus are referred to as EPAs for gastroenterology. This will help differentiate other efforts in the creation of EPAs for transplant hepatology. The OWN project supports a transforming paradigm in medical education toward competency-based medical education, which focuses on the desired outcomes of training rather than a time- or process-based curriculum that does not ensure attainment of competency. Moving forward, the standard for educational assessment will support this competency concept, which has been evolving in graduate medical education (GME) since 2000 (Table 1) with the development of the 6 core competencies (Table 2).4 This will ensure that all trainees are competent in all defined areas before certification for independent practice. The OWN timeline is shown in Table 3, and the assignments of the topics to the societies are listed in Table 4. Thirteen EPAs were identified by OWN and linked with observable behaviors and objectives. A checklist of the applicable ACGME competencies is identified, along with the subcompetencies of the reporting milestones needed to achieve mastery of the EPA. The following are provided with each EPA: a prompt for each program to identify when a typical fellow should be ready for unsupervised practice, a list of potential resources and assessments, identification of whom in the program may be most likely to inform the Clinical Competency Committee (CCC) of an entrustment determination, and a description of the implications of entrustment. This white paper has been developed in 3 parts. Part 1 includes the background of this project in light of the history of competencies and milestones and describes in detail the process by which this project was achieved. This may serve as a model for other collaborative projects across the GI and hepatology societies and GME. Part 2 is a guide to using the EPAs for programs and fellows. This product is not intended to mandate specific curricular formats or pedagogies and is only meant to identify a core set of EPAs that all GI fellows must achieve. This section clearly identifies limitations of the materials generated and outlines opportunities for program-specific innovation. Part 3 includes the list of 13 EPAs, which is accompanied by the comprehensive tool for each EPA to assist program directors and fellows. All societies have reviewed and agree in consensus that these EPAs should be applied to all fellows completing training in gastroenterology. The comprehensive tools associated with part 3 are available online in the Supplementary Material. In February 2012, Nasca et al outlined the NAS, a plan from the ACGME to advance the reality of competency-based medical education, which started in 2001 by defining the 6 core competencies (Table 2).5 The NAS is composed of 3 major changes in GME: (1) the development and use of reporting milestones for group assessment in CCCs, (2) changes to program accreditation, and (3) on-site review of each institution's clinical learning environment (Clinical Learning Environment Review or CLER) every 18 to 24 months. Milestones will be one additional element that will be used by the ACGME Residency Review Committee in Internal Medicine to assess program quality. The reporting milestones went into effect in July 2013 for IM residency programs and will be implemented for IM subspecialties in July 2014. The ACGME, American Board of Internal Medicine (ABIM), and Alliance for Academic Internal Medicine (AAIM) (the partners in the milestones venture) solicited the input and wisdom of educational leaders in the subspecialties to refine the IM Reporting Milestones for the subspecialties. The ACGME and ABIM, in collaboration with the Association of Specialty Professors and AAIM, contacted subspecialty societies and asked their leadership to recommend individuals to participate in this initiative. The first IM Subspecialty Milestones Summit was held February 11 to 12, 2013, in Alexandria, Virginia, with the goal of considering appropriate strategies to efficiently develop IM Subspecialty Reporting Milestones, capitalizing on the development work of existing IM Reporting Milestones. After much work and the development of a new scholarship subcompetency, the AAIM, ABIM, and ACGME hosted the final IM Subspecialty Reporting Milestones Summit on November 11, 2013, in Chicago, Illinois. The meeting brought together 25 societies and stakeholders to hear the recommendations of the working groups about scholarly activity components and aspirational and critical deficiency stages in the reporting milestones as well as the perspectives of multispecialty groups. Discussions continued on merging the work of the groups and improving the language in the text. Final approval of the IM Subspecialty Reporting Milestones was announced on February 22, 2014.3 Before the NAS, there were 2 requirements for summative evaluation of trainees: completion of the ABIM FasTrack form and a narrative summative evaluation attesting to the trainee's ability to practice independently.6 These attestations were made without a common frame of reference and relied heavily on subjective 9-point rating scales. The shift to reporting milestones intends to provide a common language and descriptors for formative and summative evaluation of the fellow and assessment of program quality. Because the reporting milestones are generic, the GI EPAs are a critical tool to accomplish the task of fellow assessment in the field of gastroenterology. The creation of OWN preceded the first IM Subspecialty Milestones Summit in an effort for the GI and hepatology societies to take charge of the education of our trainees. Representatives from the AASLD, ACG, AGA, ANMS, ASGE, and NASPGHAN worked together to develop tools to aid program directors in using the IM Subspecialty Reporting Milestones. Educational milestones are observable developmental steps that describe a trajectory of progress from novice (eg, the entering fellow) to proficient (eg, the graduating fellow) and ultimately to expert/master. These reporting milestones are "context-free" and intentionally constructed to be used by all IM subspecialties. Each competency has multiple subcompetencies, each with its own narrative stream. The narrative streams can be used by program directors and fellows as the roadmap toward achieving competency and independent practice. It is recognized that each program, using standard assessment tools and evaluation measures, will continue to assess its trainees in accordance with its curriculum as presently occurs. Using all available data semiannually, the CCCs will advise the program directors of the learner's trajectory toward independent practice using the IM Subspecialty Reporting Milestones as a guide. These data are reported to the ACGME twice a year, fulfilling one of the requirements of the NAS. Knowing that each program has unique systems, metrics, evaluation tools, and processes, the OWN group created a guide and a toolbox focusing on 13 EPAs for gastroenterology for use in completing the IM Subspecialty Reporting Milestones. When developing this toolbox, OWN considered different assessment methods, including curricular milestones and EPAs. Curricular milestones are granular descriptions of the knowledge, skills, and attitudes or behaviors that define the content of the 6 general competencies at each level of training. These can provide and define the specialty-specific content that is taught by a specialty and that is potentially unique to that specialty. They are inherently modifiable to meet the structure and needs of an individual program. OWN believes that using curricular milestones would be impractical. In addition, curricular milestones provide less flexibility for programs because they indicate when certain knowledge, skills, and attitudes should be attained. EPAs define the core activities of a profession and provide an assessment tool that is task based and clinically oriented. EPAs provide a means to translate theoretical competencies into clinical practice. In contrast to competencies, which define traits of individual trainees, EPAs describe the work.7 Specific knowledge, skills, and attitudes are components that comprise an individual EPA. An example of an EPA is "breaking bad news," which would include the relevant knowledge of the disease or problem, communication skills to impart the news to the patient and family, and professional attitudes to accomplish this. Programs may wish to organize curricular milestones to define the specific knowledge, skills, and attitudes at each point in training required to counsel the patient about bad news. OWN believes that development of curricular milestones is not a necessary step to inform the reporting milestone decisions. Rather, we believe that EPAs and tools related to EPAs can inform the reporting milestones to summarize the trainee's competence in performing a task. As descriptions of work-based activities, EPAs can serve as meaningful and manageable points of assessment, defined by ten Cate and Scheele as "a critical part of professional work that can be identified as a unit to be entrusted to a trainee once sufficient competence has been reached."7 EPAs are clear tasks that patients, fellows, and faculty can recognize. By associating them with the IM Subspecialty Reporting Milestones, this OWN product can help program directors accomplish the reporting milestones task in the NAS. In summary, OWN has developed 13 EPAs, with a tool accompanying each EPA that facilitates the systematic assessment of the fellow's incremental progression through all expected levels of training specific to gastroenterology. The cornerstone of this tool is the set of 13 EPAs specific to GI training programs, which have been developed based on the nationally recognized Gastroenterology Core Curriculum.8 First published in 1996 and last revised in 2007, the Core Curriculum is a living document that represents the best practices in GI training and, as such, was an ideal framework for developing this toolbox. The EPAs embody the curricular scope of training in gastroenterology, recognize program-specific assessment, and are mapped to specific subcompetencies of the IM Subspecialty Reporting Milestones. The next section offers a practical guide to the use of the EPAs with the reporting milestone toolbox. The GI EPAs enable program directors to assess and improve their existing training curriculum. In competency-based medical education, the curriculum should be designed after determining the health system needs and the desired outcomes.9 The EPAs define the needs and desired outcomes for a gastroenterologist practicing in the United States today. We offer suggestions for how to use the GI EPAs for curriculum development and assessment. Table 5 shows a list of definitions. Most training programs have a curriculum that was built on the Gastroenterology Core Curriculum.8 The EPAs serve as a tool for global program review at this crucial time in GME. After a global assessment, programs can identify which of the EPAs can be targeted for further development, depending on local resources and need. These EPAs can aid program faculty in self-assessment of their existing curriculum using Kern's 6-step approach for curriculum development as a framework.10 Programs could ask if there is a disease state or topic that needs further attention and compare what is currently being done with what is outlined for knowledge, skills, and attitudes in the respective EPA. Feedback from current fellows and an assessment of their needs should be included as part of this. Program directors or education faculty can use this information to improve objectives for the rotations being discussed. The Program Evaluation Committee is a committee prescribed by the NAS, which can use these EPAs and tools to refine its program's curriculum. The EPAs provide measurable objectives that can be incorporated into existing rotations and learning activities. Over time, the GI training community will identify educational strategies and tools that can be used for instruction. For example, a program has decided to evaluate the fellows' current nutrition rotation. The hospital has an increased volume of patients with morbid obesity. Currently, the fellows' rotation in nutrition focuses on enteral access and inpatient parenteral support. The program faculty have identified a need for increased training of fellows to improve the care of obese patients and have received feedback from the fellows consistent with this. The program director can use the EPA for nutrition to develop didactics (knowledge objectives) and clinical experiences with a bariatric surgeon and nutritionist (skills/attitude objectives) to help improve the curriculum related to nutrition. Because the EPAs are task based, a program can link rotations and learning activities to the relevant EPAs. This would aid in curriculum development and gathering assessment data in the course of teaching and clinical care. Examples of these maps are provided in Tables 6 and 7. Specifying which elements of each EPA will be part of the curriculum for that rotation could provide greater detail. Program directors are encouraged to think broadly when listing learning loci, looking beyond clinical rotations and including conferences, research activities, and other educational programs. Programs can use the EPAs to provide greater clarity to existing rotation objectives and deepen the level of training through current clinical experiences by incorporating some of the skills and attitudes elements into teaching and training. Because the EPAs are task oriented, one approach could be to align EPAs with common GI fellowship rotations; for example, the EPA for management of GI bleeding has several elements that could be linked to an inpatient consultative rotation. The faculty who teach on this rotation can use the EPA to understand not only the knowledge items but also the specific skills and attitudes that need to be taught during the rotation. Similar to the Gastroenterology Core Curriculum, the GI EPAs can be used to structure didactic content. The objectives can be used for lectures, for development of reading lists, and to highlight self-study materials (Web-based modules, question-based review), which can guide learners in the achievement of the EPA. In the previous example, the program director may develop a new reading list and suggest the use of the AGA Nutrition Toolkit, an online case-based self-study module, to fulfill instruction of some of the knowledge and attitudes objectives. The distinction between competencies as an educational framework and as evaluation tools has become blurred.11 Attempts to reliably measure the competencies separately from one another have not been successful, and this has frustrated clinical educators tasked with assessing trainees in the new competency-based training paradigm.12 The concept of EPAs was developed for medical education as a work-based assessment tool. Adequate completion of the critical activities defined by EPAs requires the possession of several competencies. Therefore, as a relevant representation of the day-to-day activities of a specific medical specialist, EPAs "bridge the gap" between the theory of competency-based training and the application of those competencies in discrete activities that can be observed and assessed.7 Assessment through entrustment embraces the subjectivity of experienced evaluators and validates subjective, holistic impressions of trainees based on specific knowledge, skills, and attitudes. To meet the reporting requirements of the NAS, we developed the GI EPAs as the focus of assessment among a toolbox of multiple assessment tools.13 The GI EPA tool was created to facilitate completion of the ACGME reporting milestone requirement. Table 8 lists each of the 13 EPAs and identifies the subcompetencies that can be tracked via each EPA. Table 9 lists the 23 subcompetencies and identifies the EPAs that can be used for the reporting requirement. These 2 tables show the close relationship between the EPAs and reporting milestones, which can be leveraged to complete the task of assessment in the NAS. EPAs incorporate multiple competencies and can therefore be a powerful way to inform clinical educators on the general competence of trainees. We do not recommend, however, that each program abandon use of existing, effective tools. Rather, training programs should continue to use these tools and begin to adapt or modify them, either to directly inform the reporting milestones or to inform entrustment decisions within the EPAs. Each GI EPA contains suggestions for assessment task and data sources, which program directors can use to gauge milestone attainment for a particular trainee. In addition to traditional assessment instruments and sources (as outlined in the ACGME Toolbox, 2001), the group considered nonphysician evaluators and quality and patient safety data where appropriate (Figure 1). The EPAs are not intended to be prescriptive but rather were developed to allow for an individualized approach to assessment and data collection. An assessment map (Table 7), similar to the curricular map (Table 6), can be created to assist faculty in understanding how reporting milestones, EPAs, and other assessment tools are aligned with specific learning activities. Relationship between GI EPAs and the IM Subspecialty Reporting Milestones. Using EPAs as the cornerstone of assessment is strongly encouraged in part because these critical activities and the concept of entrustment should both be familiar to clinical faculty, even inexperienced evaluators. Furthermore, use of EPAs emphasizes direct observation as the most important assessment method, which should lead to increased opportunities for faculty-trainee interaction to form a more complete basis for assessment. One approach to incorporating EPAs into an assessment program is to begin by mapping current rotations and assessment tools to specific EPAs. This process can be further refined by mapping rotations and assessment tools to the specific knowledge, skills, and attitudes included with each EPA. Note areas of overlap and identify rotations best suited to learn and assess them. Once this is accomplished, the next step is to incorporate specific knowledge, skills, and attitudes statements into new or existing assessment forms. It would be inappropriate to ask each faculty member to assess trainees on each EPA in every rotation. Instead, the program should focus on those EPAs that are best learned and evaluated in a particular rotation. Also consider the strengths and expertise of individual faculty members who may be best suited to assess specific EPAs (eg, hepatologists for the general hepatology EPAs or motility specialists for the motility disorders EPA). Keep the assessment forms manageable and emphasize the importance of narrative comments rather than relying on a Likert-based rating scale. The EPAs provide clear behaviorally based objectives for faculty to reflect on a fellow's performance to aid in assessment. Faculty in a certain rotation might review the relevant portions of the EPAs related to their clinical setting and then review the related subcompetencies of the reporting milestones. Group discussion can lead to a shared mental framework and allow for more consistent assessment across faculty. For each fellow, assessment data can be aggregated and summarized for each of the 6 ACGME core competencies. For example, a chart-stimulated recall used for 2 EPAs will contain information about the patient care competency. This can be added to the patient care elements of an existing faculty evaluation from one or more rotations, allowing the CCC to assess the patient care–related subcompetencies and milestones. Some commercial residency management systems are developing the capacity to do this through individualized portfolios. Ultimately, the aggregate of assessment tools, including EPAs, will be used to inform the CCC, which is tasked with evaluating trainee competence and completing the semiannual NAS reporting milestones. The job of the CCC will be easier, with more detailed data available from multiple sources and the development of a shared mental model of assessment within the program. To assist the CCC, each GI EPA indicates the main subcompetencies of the reporting milestones that are needed to achieve mastery. Each program may wish to map specific knowledge, skills, and attitudes statements within each EPA to the subcompetencies within relevant reporting milestones. By taking advantage of significant areas of overlap (an intended consequence of EPAs is that they each encompass multiple competencies), the CCC can identify and refine the specific reporting milestones for each trainee. With experience, the CCC will be able to identify gaps in the curriculum and/or missing assessments that are needed to complete the reporting milestones and can advise the program director accordingly. The role of the CCC cannot be overstated. The assessment of competency is inherently subjective, and no single person should make judgments of competence. This is particularly critical in summative decisions (eg, need for academic probation, graduation from the program), which will become relevant when ACGME reporting milestones eventually coincide with ABIM evaluations for certification. Faculty development and education is critical to ensure successful implementation of the IM Subspecialty Reporting Milestones and EPAs as part of the NAS. Faculty will need education about these changes in GME. More importantly, faculty using similar assessment tools should be trained on what to observe and how to rate the learner so there is greater interobserver agreement. Members of the CCC should also have specific faculty development on how to aggregate assessment data, understanding the reporting milestones, and the role of the EPAs. The time to achieve entrustment will vary depending on the EPA, the curriculum, the trainee, and the teacher. We encourage individual programs to track and ultimately define the time expected to achieve each EPA. Deviation from this expectation should be evaluated. Slow learners require additional assessments and, if appropriate, remediation opportunities should be made available to the trainee. For rapid learners, the curriculum should be flexible and allow for achievement of proficiency level 5 ("may act as a supervisor and instructor") in each EPA or enable the trainee to focus on other areas of interest (eg, therapeutic endoscopy, motility, inflammatory bowel disease, or transplant hepatology).7 The EPAs can operationalize the concept of graded and progressive responsibility as identified in the ACGME Program Requirement for Graduate Medical Education in Gastroenterology.14 Each program is encouraged to define for itself the implications for entrustment of each EPA, which can simply mean that the trainee can perform a particular EPA with indirect or oversight supervision. This avoids the unfortunate common experience in which trainees are not allowed to perform critical professional activities independently until very late in training or not at all during training yet are expected to be able to perform all professional activities on the day they graduate from fellowship.7 Training programs must actively engage the fellow in the assessment process.15 Quality formative feedback is essential for learning, and programs are expected to provide each fellow with evaluation of performance with feedback. In turn, fellows are expected to be able to incorporate formative feedback into daily practice.14 Feedback should focus on specific knowledge, skills, and attitudes that the evaluator has witnessed directly16; therefore, EPAs are ideally suited to facilitate the feedback process. Feedback is generally welcomed when it is based on performance and tailored to the learner's goals.17 We recommend using EPAs to help trainees pinpoint the specific knowledge, skills, and attitudes that, once mastered, would lead to an entrustment decision, which necessarily leads to development of competence. A key competency in medical education is the ability to identify gaps in knowledge, skills, and attitudes and self-direct one's learning to fill those gaps, which is embodied in the "practice-based learning and improvement" core competency. EPAs can help trainees identify the goals and expectations of training, not just from individual programs but also from the specialty and society as a whole. The GI EPAs were authored with the program director and trainee in mind, who are both end users of this product. Trainees can use the EPAs to identify the specific knowledge, skills, and attitudes they need to master to achieve entrustment. The EPAs are a useful tool as fellows engage in self-reflection or develop learning plans for each stage of training. In consultation with their program director and core faculty, trainees can help direct their own learning and experiences to focus on areas of deficiency. EPAs can motivate trainees to earn entrustment and thereby competence as early as possible in training, provided that the implications of entrustment are clearly defined and meaningful and that opportunities for supervisory roles and advanced learning exist. The EPAs do not define a time for achievement of each EPA or component of the EPA. This tool does not offer a "quick fix" formula to completing the reporting milestones from a set of evaluations. The EPAs are intended to aid in creating a shared framework for the end product of GI training among fellows, faculty, and other stakeholders. In prior education and training documents, concerns for special populations, ethics, quality/patient safety, health care economics, research, and practice management were separated into distinct sections. The authors recognize that these are key tasks gastroenterologists perform as part of their job. However, these elements have been integrated into each EPA to make learning and assessment seamless. For example, calculating an adenoma detection rate is part of performing a lower endoscopy, which has been listed in the endoscopy EPA as opposed to a separate EPA on quality in gastroenterology. One criticism of our decision to focus on EPAs rather than curricular milestones is that EPAs provide a description of the outcome of GI training without describing the developmental trajectory (milestones) along the way to competence and expertise. On the other hand, the traditional competency-based approach has been criticized because there is a tendency to split each objective into more detail and create complexity that confuses many clinicians.7 This matter is further complicated in assessment when we oversimplify competence through reductionism that instead threatens validity.18 This can be avoided by adopting an integrated, holistic approach to competence, which is the very nature of EPAs. The developmental trajectory of the trainee should be assessed by the CCC in the context of the knowledge, skills, and attitudes acquired within each EPA and the level of supervision required. The following are the 13 EPAs for gastroenterology. The accompanying toolbox is provided as Supplementary Material and can also be found at: www.ownyourfellowship.com. The AASLD, ACG, AGA, ANMS, and ASGE, with support from NASPGHAN, have worked together to provide supportive materials for program directors and fellows to enhance assessment of our trainees and to comply with the requirements to submit reporting milestone documentation for each fellow every 6 months. We believe that the 13 EPAs identified for GI training with the accompanying toolbox will be an asset for educators and trainees. The OWN Committee respects the autonomy of GI fellowship programs and the creativity and innovation that may relate to faculty talent, institutional resources, and programmatic strengths. We also recognize that there may be opportunities to share creative resources and integrate efforts to resolve needs and challenges. As an example, OWN plans to collate innovative assessment tools created by GI program directors who are willing to share their resources on our Web site. New advances in technology, both in clinical medicine and in education, as well as changing demographics and a changing health care environment will provide opportunities to define the future direction in GI and hepatology education. Finally, we acknowledge the value of bringing the societies together to work toward a common purpose and believe that the EPAs and the toolbox have been enriched as a resource with collaborative multi-society input. OWN would like to thank the following staff participants: Lori Marks, Ph.D. (AGA), Cory Harlow, Administrative Organizer (AGA), Maria Susano, Administrative Organizer (ACG), Janeil Klett (AASLD), Lori Ennis (ANMS), and Diane Alberson (ASGE); our colleagues at NASPGHAN who provided direction and feedback: Cary Sauer, M.D., and Toba Weinstein; and Olle ten Cate, Ph.D., for his expert review of our EPA list and providing feedback. Note: To access the supplementary material accompanying this article, visit the online version of Gastroenterology at www.gastrojournal.org, and at http://dx.doi.org/10.1053/j.gastro.2014.04.038. Supplementary Information Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.