A study of crowdsourced segment-level surgical skill assessment using pairwise rankings
Currently available methods for surgical skills assessment are either subjective or only provide global evaluations for the overall task. Such global evaluations do not inform trainees about where in the task they need to perform better. In this study, we investigated the reliability and validity of a framework to generate objective skill assessments for segments within a task, and compared assessments from our framework using crowdsourced segment ratings from surgically untrained individuals and expert surgeons against manually assigned global rating scores.
crowdsourcing skill assessment robotic surgery training feedback task flow pairwise comparisons activity segments task decomposition