The utility of high-stakes technical performance assessments in surgical training: establishing a framework across the continuum from selection to certification
Creators: name: Toale, Conor
Publisher: name: Royal College of Surgeons in Ireland
Year: 2025
Subjects: classificationCode: 320226
subject: Surgery
classificationCode: 390110
subject: Medicine, nursing and health curriculum and pedagogy

description: The traditional Halstedian of apprenticeship training in surgery has undergone significant revolution in recent decades <sup>1</sup>. Evolving patient expectations regarding the role of surgical trainees in their care<sup>2</sup>, an increased emphasis on theatre efficiency <sup>3</sup>, and well explored concerns regarding perceived operative competence and confidence of graduating trainees <sup>4-6</sup>, has led to a re-evaluation of the training paradigm. ‘Competency-based’ approaches to outcome-driven training and assessment are well established across training jurisdictions<sup>7 8</sup>, leading to the development of nominally time-independent postgraduate surgical programs <sup>9</sup>. The surgical specialties pose a unique challenge in this regard, due to the requirement for robust and reliable methods of teaching and assessing competence in operative skill<sup> 10</sup>. Determinations of competence, in the context of the surgical training programme of Ireland and the United Kingdom, are made at Annual Review of Competence Progression (ARCP) meetings and are informed by performance in workplace-based assessments (WBAs), operative numbers (as recorded by trainees in an electronic logbook), and educational supervisor reports.  From August of 2021, surgical training in this jurisdiction has moved to an explicitly outcome-basedcurriculum<sup>11</sup>. Trainees are measured against high level outcomes-‘Capabilities in Practice’ and ‘Generic Professional Capabilities’. This training programme is nominally time-independent<sup>11</sup>. In order to transition to a truly competency-based paradigm, operative skill will need to be assessed and certified through an objective, standardised, and valid approach. 
The role of simulation in surgery has evolved in tandem with competency-based education<sup>12</sup>.  Simulation is commonplace in surgical training curricula<sup> 13</sup>, and is increasingly being deployed as a method of assessing a trainee’s competence <sup>14 15</sup>. Studies have demonstrated the safe transfer of simulator-acquired operative skill to the operating theatre <sup>16</sup>. Whether simulation-training ultimately translates to improved patient safety and surgical outcome measures has not yet been firmly established <sup>17</sup>. Simulation has most notably been used in high-stake assessments through the implementation of the Colorectal Objective Structured Assessment of Technical Skill by the American Board of Colon and Rectal Surgery<sup>14 18</sup>. Such assessments have not been used for higher specialist training in the UK and Ireland to date. Beyond end-of-training certification, simulation (or other technical performance assessments) could be used as a measure of skill or competence at a number of key checkpoints or high-stakes scenarios in surgery, namely: trainee selection, progression through training, and autonomy granting. 
This thesis will seek to explore the use of high-stakes technical performance assessments of operative skill and competence in surgical training. The validity, acceptability and reliability of technical performance assessments will be explored, using both quantitative and qualitative methods, at three key time points: trainee selection, in-training progression, and competence certification in senior training. For the purpose of this thesis, ‘performance assessments’ are defined as assessments that take place outside of the workplace context, in comparison to ‘workplace-based assessments’. The validity of psychomotor assessments of fundamental technical aptitude will be examined at entry to Core Surgical Training. The validity of simulation-based assessment will be assessed during Core Surgical Training. Finally, we will seek to develop and validate a simulation-based assessment curriculum suitable for the assessment of operative competence in senior Higher Specialist trainees in General Surgery. 
Findings from this work will be used to inform a blueprint, or proposed framework, regarding the use of technical performance assessments in high stakes scenarios surgical training, from trainee selection to certification as an independent practitioner.