Scaling Entrustable Professional Activities: insights from a national-level software pilot for the Emirati Board
Creators: name: Shersad, Fouzia
name: Ucenic, Teodora
name: Balla, Ibrahim
name: Syeda, Asma
name: Alhanouf Khthail
name: AlHouqani, Mohammed
Publisher: name: F1000 Research Limited
Year: 2024

description: <b>Background</b><br>
Implementing a large-scale Entrustable Professional Activities (EPA) assessment tool at a national level necessitates a thorough knowledge of the pros and cons and mitigation of risks. Accepting such a tool requires early identification and prevention of any shortcomings. This pilot study sought to identify identify the critical features and factors contributing to the success of a mobile application designed for the systematic assessment of EPAs and competency attainment as part of workplace-based assessments (WBAs) in residency programs.<br>
<br>
<b>Summary of Work</b><br>
We selected three medical specialties for the initial implementation. In total, 144 residents across 8 programs in 4 centers were included in the pilot study. The application's effectiveness and feasibility were evaluated through focus group discussions with program directors and residents. Following these insights, we refined the application to enhance the efficacy of documenting entrustment decisions and feedback. For 12 weeks, the application was deployed using an adapted version of the Royal College of Canada's validated questionnaire and a user interface rating scale with open-ended questions. <br>
<br>
<b>Summary of Results</b> <br>
The focus group discussion revealed that the software's capabilities were introduced to all facilitators and program directors. Residents initiated the assessments on the mobile app, which supervisors then completed to document the level of entrustment. Data visualization was made accessible on a dashboard for the competency committee and program directors. Indicators such as speed, technical support, acceptance rate, faculty support, and satisfaction rate were evaluated using an online questionnaire. The response rate was 56.9 %. Among the responders, the adoption rate was 59.8%. The user-friendliness satisfaction rate is 71.4% and the availability of support is 62.5% (percentage of respondents who score 4-5).  The best time for assessment was identified to be after a bedside procedure. The overall results indicated that they were acceptable in addition to positive attitudes of faculty. They suggested reducing the tasks under each EPA to reduce the time taken for the assessment. The adoption rate was below expectation and the reason was attributed to the optional nature of a pilot study. It was inferred that a national mandate would enhance the acceptance of EPAs. <br>
<br>
<b>Discussion and Conclusion</b><br>
The pilot successfully demonstrated the EPA assessment process, with clear data visualization for program directors and the accreditation body. Analysis of the feedback highlighted several considerations for the broader implementation of the system.<br>
<br>
<b>Take Home Message</b><br>
The user-friendliness, reliability, and precision of the e-portfolio serve as motivators for EPA participants. Key success factors identified include 

 Mandating attainment of entrustment for certification
 Stakeholder engagement through continuous support​
 Robust documentation of assessment​
 Efficiency and reliability of software