Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment
Authors: Mink Richard B. , Carraccio Carol L. , Herman Bruce E. , Weiss Pnina , Turner David A. , Stafford Diane E. J. , McGann Kathleen A. , Kesselheim Jennifer , Hsu Deborah C. , High Pamela C. , Fussell Jill J. , Curran Megan L. , Chess Patricia R. , Sauer Cary , Pitts Sarah , Myers Angela L. , Mahan John D. , Dammann Christiane E. L. , Aye Tandy , Schwartz Alan ,   for the Subspecialty Pediatrics Investigator Network, Havalad Vinod , Pinheiro Joaquim , Alderman Elizabeth , Fuloria Mamta , McCabe Megan E. , Mehta Jay , Rivas Yolanda , Rosenberg Maris , Doughty Cara , Hergenroeder Albert , Kale Arundhati , Lee-Kim YoungNa , Rama Jennifer A. , Steuber Phil , Voigt Bob , Hardy Karen , Johnston Samantha , Boyer Debra , Mauras Carrie , Schonwald Alison , Sharma Tanvi , Barron Christine , Dennehy Penny , Jacobs Elizabeth S. , Welch Jennifer , Kumar Deepak , Mason Katherine , Roizen Nancy , Rose Jerri A. , Bokor Brooke , Chapman Jennifer I. , Frank Lowell , Sami Iman , Schuette Jennifer , Lutes Ramona E. , Savelli Stephanie , Amirnovin Rambod , Harb Rula , Kato Roberta , Marzan Karen , Monzavi Roshanak , Vanderbilt Doug , Doughty Lesley , McAneney Constance , Rice Ward , Widdice Lea , Erenberg Fran , Gonzalez Blanca E. , Adkins Deanna , Green Deanna , Narayan Aditee , Rehder Kyle , Clingenpeel Joel , Starling Suzanne , Karpen Heidi Eigenrauch , Rouster-Stevens Kelly , Bhatia Jatinder , Fuqua John , Anders Jennifer , Trent Maria , Ramanathan Rangasamy , Nicolau Yona , Dozor Allen J. , Kinane Thomas Bernard , Stanley Takara , Rao Amulya Nageswara , Bone Meredith , Camarda Lauren , Heffner Viday , Kim Olivia , Nocton Jay , Rabbitt Angela L. , Tower Richard , Amaya Michelle , Jaroscak Jennifer , Kiger James , Macias Michelle , Titus Olivia , Awonuga Modupe , Vogt Karen , Warwick Anne , Coury Dan , Hall Mark , Letson Megan , Rose Melissa , Glickstein Julie , Lusman Sarah , Roskind Cindy , Soren Karen , Katz Jason , Siqueira Lorena , Atlas Mark , Blaufox Andrew , Gottleib Beth , Meryash David , Vuguin Patricia , Weinstein Toba , Armsby Laurie , Madison Lisa , Scottoline Brian , Shereck Evan , Henry Michael , Teaford Patricia A. , Long Sarah , Varlotta Laurie , Zubrow Alan , Barlow Courtenay , Feldman Heidi , Ganz Hayley , Grimm Paul , Lee Tzielan , Weiner Leonard B. , Molle-Rios Zarela , Slamon Nicholas , Guillen Ursula , Miller Karen , Federman Myke , Cron Randy , Hoover Wyn , Simpson Tina , Winkler Margaret , Harik Nada , Ross Ashley , Al-Ibrahim Omar , Carnevale Frank P. , Waz Wayne , Bany-Mohammed Fayez , Kim Jae H. , Printz Beth , Brook Mike , Hermiston Michelle , Lawson Erica , van Schaik Sandrijn , McQueen Alisa , Vander Ploeg Booth Karin , Tesher Melissa , Barker Jennifer , Friedman Sandra , Mohon Ricky , Sirotnak Andrew , Brancato John , Sayej Wael N. , Maraqa Nizar , Haller Michael , Stryjewski Brenda , Brophy Pat , Rahhal Riad , Reinking Ben , Volk Paige , Bryant Kristina , Currie Melissa , Potter Katherine , Falck Alison , Weiner Joel , Carney Michele M. , Felt Barbara , Barnes Andy , Bendel Catherine M. , Binstadt Bryce , Carlson Karina , Garrison Carol , Moffatt Mary , Rosen John , Sharma Jotishna , Tieves Kelly S. , Hsu Hao , Kugler John , Simonsen Kari , Fastle Rebecca K. , Dannaway Doug , Krishnan Sowmya , McGuinn Laura , Lowe Mark , Witchel Selma Feldman , Matheo Loreta , Abell Rebecca , Caserta Mary , Nazarian Emily , Yussman Susan , Thomas Alicia Diaz , Hains David S. , Talati Ajay J. , Adderson Elisabeth , Kellogg Nancy , Vasquez Margarita , Allen Coburn , Brion Luc P. , Green Michael , Journeycake Janna , Yen Kenneth , Quigley Ray , Blaschke Anne , Bratton Susan L. , Yost Christian Con , Etheridge Susan P. , Laskey Toni , Pohl John , Soprano Joyce , Fairchild Karen , Norwood Vicky , Johnston Troy Alan , Klein Eileen , Kronman Matthew , Nanda Kabita , Smith Lincoln , Allen David , Frohna John G. , Patel Neha , Estrada Cristina , Fleming Geoffrey M. , Gillam-Krakauer Maria , Moore Paul , El-Khoury Joseph Chaker , Helderman Jennifer , Barretto Greg , Levasseur Kelly , Johnston Lindsay 
Published: October 2023

Background

Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one’s abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment.

Objectives

We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME).

Methods

In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale.

Results

One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59–0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05).

Conclusions

We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.