The Use of Interventional Cardiovascular Simulation to Evaluate Operator Performance: The Carotid Assessment of Operator Performance by the Simbionix Carotid StEnting Simulator Study (ASSESS). (2007)

BACKGROUND: The percutaneous endovascular carotid stenting procedure has only recently been approved by the FDA, and interventional operators from multiple disciplines (Cardiology, Radiology, Vascular Surgery) are getting training in this field. The professional societies of these disciplines are looking for novel ways to assess the competency of an operator, before they grant authorization to perform this complex procedure. Simulators have been suggested as tools to evaluate operators in specific procedures and operative techniques. The virtual reality machines can be used to capture numerous factors that translate into the success and safety of the procedure. These include selection of devices, accuracy of manual performance, reaction to interactive problems, use of adjuvant pharmacotherapy, management of hemodynamic changes, and prevention and resolution of complications. All of these can be automatically evaluated and scored by a complex software designed virtual reality simulator. The simulators that are used in Interventional Cardiology and endovascular therapy have not been validated yet as such assessment tool. OBJECTIVE: The main objective of the ASSESS study is to validate the simulator-based metrics as a tool to discriminate between different levels of operators. METHODS: We used the Angio-Mentor carotid Stenting simulator system (Simbionix). The simulators had been loaded with predefined cases demonstrating increasing complexity for stenting intervention. Total of 34 operators (10 experts, 12 intermediate level, and 12 novice; based on prior reported experience) were asked each to do three cases of carotid stenting on the simulator. All the operators had the same cases that were selected to demonstrate increased complexity to judge anatomy, correct performance of the technical steps, use of wires, catheters, embolic protection device, balloons, and stents, adjuvant pharmacotherapy, and management of interactive hemodynamic response. During the procedure, multiple features of all of the above elements were automatically captured by the simulator. Each step and parameter captured had a pre-defined score. Scoring was based on penalty points for mal-performance, as pre-defined by a panel of independent experts in carotid stenting, prior to conducting the study. RESULTS: The study has been completed, and all the data that were captured is being analyzed. Full results will be available at the time of the presentation. CONCLUSIONS: Being able to demonstrate the ability of a hi-tech, virtual reality simulator to differentiate between different levels of interventional operators, may enlighten the simulator as a potential objective “examination tool”, to evaluate the performance and competence of an interventional operator. These will justify the use of Interventional Cardiovascular Simulation for certification and credentialing DISCLOSURE STATEMENT: Giora Weisz - Consultant to Simbionix; Jacque Devaud - employee of eTrinsic, a division of Simbionix; other authors NONE