Title: Developing Evidence-Based Standards for Competency in General Surgical Training
Funder: Department of Health and Social Care

Abstract: Background Surgical training in the UK is a competency-based model governed by the Intercollegiate Surgical Curriculum Programme (ISCP). Trainees must collect evidence of competency throughout training by completing mandatory work-based assessments and keeping a record of operative experience. In order to be awarded a Certificate for Completion of Training (and subsequently practice independently), trainees must meet certain standards including minimum operative numbers for 6 defined index procedures (appendicectomy, inguinal hernia repair, cholecystectomy, segmental colectomy, Hartmanns and emergency laparotomy). Trainees must be deemed competent to perform the same index procedures. This is defined by achieving three level 4 ratings in procedure- based assessments (PBA), judged by 3 independent assessors each level 4 PBA indicates independent operative competency in the particular procedure that was observed. These competency standards are based on minimal evidence. Furthermore, there is a paucity of data relating to the operative experiences, competency levels and learning curves (using standardised skill assessment tools) in surgical training that are currently provided by trainees at the time of completion of their training. Objectives In UK general surgery training: How many index and total operations are completed through, and at completion of, training? What index procedure-based assessment levels do trainees achieve through, and at completion of, training? What is the construct validity of the procedure-based assessment with respect to index procedures? What are the learning curves and numerical operative experience thresholds for competency in index procedures? Design Analysis of two linked, electronic, surgical training databases (ISCP and eLogbook) will be undertaken using Stata software. All general surgical trainees registered with ISCP since 2007 will be included. Trainees who have completed training during the time period will form a complete training cohort for analysis with those still in training forming a stage-specific cohort. Statistical methods will be used to address the objectives listed including multi-level, mixed effects, ordered logistic regression modelling. Outcomes The findings of this study will provide an accurate account of the operative experiences and assessment outcomes in UK general surgery training. I will describe the validity of an existing tool of assessment (PBA) in index procedures. I will provide an evidence-base to be used for future standard setting in the training of safe surgeons in the UK.

Lay abstract: Background There are approximately 1400 general surgeons in training in the UK with approximately 150 new surgeons starting and finishing training each year respectively. The way in which these general surgical trainees should be trained is defined by the Intercollegiate Surgical Curriculum Programme (ISCP). To qualify for independent Consultant practice, trainees must perform a minimum number of 1600 operations along with defined minimum numbers of 6 key operations. They must also achieve 3 top-level (Level 4) competency assessments (defined as Procedure-Based Assessments (PBAs), a standardised measure of competency), judged by 3 different assessors, in the same key operations. There are few studies describing how much operating a surgeon in training currently does or should do. There are no prior studies relating numbers of operations during surgical training to the achievement of competency (using procedure-assessments as a standardised measure). Objectives For UK general surgical trainees:I will describe operative experiences during and at completion of surgical training.I will describe the numbers of competency assessments during and by completion of training.I will analyse the relationship between numbers of operations performed and the competency assessment levels achieved with respect to 6 key operations.I will determine the numbers of operations, i.e. the threshold, needed to achieve competency in the 6 key operations.The analysis in objectives 1 and 2 will accurately describe the operative experiences and levels of competency assessment in UK general surgery training. The results of objective 3 will validate the current tools used in the assessment of competency in key operations in general surgical training. No prior studies have done this for these 6 operations. The results of objective 4 will inform the design of the next surgical curriculum and provide an evidence base for the standards to be achieved, for both numbers of operations and levels of assessments, throughout and at completion of general surgery training. This will ensure that all general surgeons in the UK, who are deemed suitable for independent practice, have been adequately trained and have met defined, evidence-based standards reflecting attainment of operative competency. Design and methods I will use linked, anonymised data from 2 national training databases; the ISCP (which collects data relating to trainee demographics, training placements and assessments) and an online record of operative experience ( the eLogbook). I will assess variation by age, gender and training region for all objectives. Objective 1: I will take each key operation and count the number of operations performed within each year of training and at completion of training. Objective 2: I will calculate the number of procedure assessments performed for each trainee stage and the assessment levels that are given for each of the key operations. Objective 3: I will use statistical methods to assess the association between the numbers of operations performed and the levels of assessments achieved for each key operation. Objective 4: I will use a statistical model to describe the numbers of operations required to reach competency in each of the key operations. Participants I will study all general surgical trainees registered in surgical training since 2007. I estimate that I will have data from approximately 450 surgical trainees who have completed surgical training since 2007 forming a complete training cohort. I estimate that the operation records in relation to the complete training cohort will be in the region of 640,000 operations. I will create a further study population relating to trainees who have not yet completed surgical training. I expect this population to be approximately 1400 trainees and will use this for training-stage data analysis. Patient and public involvement I met with patients and public from a local NIHR unit patient and public involvement group. The group felt the research was essential for understanding what makes a surgeon competent and ensuring that safe surgeons are trained in future. They agreed the research was timely and with sound research objectives. The lay representative from the ISCP has been involved in reviewing the research plan and plain English summary. Dissemination I will publish 3 papers relating to the findings of this research. I will present the findings of this research at national and international surgical conferences. I will consult the lay representative from ISCP regarding disseminating findings to the public. The findings of this research will be presented to the surgical curriculum management group at the Royal College of Surgeons of England. I anticipate the results of this research will form evidence-based standard setting for the next ISCP curriculum in relation to target minimum operative numbers and competency assessments.

Research activity codes: 8.1 ORGANISATION AND DELIVERY OF SERVICES
Health categories: GENERIC HEALTH RELEVANCE