CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
Creators: name: Nwoye, Chinedu Innocent
name: Yu, Tong
name: Sharma, Saurav
name: Murali, Aditya
name: Alapatt, Deepak
name: Vardazaryan, Armine
name: Yuan, Kun
name: Hajek, Jonas
name: Reiter, Wolfgang
name: Yamlahi, Amine
name: Smidt, Finn-Henri
name: Zou, Xiaoyang
name: Zheng, Guoyan
name: Oliveira, Bruno
name: Torres, Helena R.
name: Kondo, Satoshi
name: Kasai, Satoshi
name: Holm, Felix
name: Özsoy, Ege
name: Gui, Shuangchun
name: Li, Han
name: Raviteja, Sista
name: Sathish, Rachana
name: Poudel, Pranav
name: Bhattarai, Binod
name: Wang, Ziheng
name: Rui, Guo
name: Schellenberg, Melanie
name: Vilaça, João L.
name: Czempiel, Tobias
name: Wang, Zhenkun
name: Sheet, Debdoot
name: Thapa, Shrawan Kumar
name: Berniker, Max
name: Godau, Patrick
name: Morais, Pedro
name: Regmi, Sudarshan
name: Tran, Thuy Nuong
name: Fonseca, Jaime
name: Nölke, Jan-Hinrich
name: Lima, Estevão
name: Vazquez, Eduard
name: Maier-Hein, Lena
name: Navab, Nassir
name: Mascagni, Pietro
name: Seeliger, Barbara
name: Gonzalez, Cristians
name: Mutter, Didier
name: Padoy, Nicolas
Publisher: name: arXiv
Year: 2023
Subjects: subject: Image and Video Processing (eess.IV)
subject: Computer Vision and Pattern Recognition (cs.CV)
subject: Machine Learning (cs.LG)
subject: FOS: Electrical engineering, electronic engineering, information engineering
subject: FOS: Electrical engineering, electronic engineering, information engineering
subject: FOS: Computer and information sciences
subject: FOS: Computer and information sciences

description: Formalizing surgical activities as triplets of the used instruments, actions performed, and target anatomies is becoming a gold standard approach for surgical activity modeling. The benefit is that this formalization helps to obtain a more detailed understanding of tool-tissue interaction which can be used to develop better Artificial Intelligence assistance for image-guided surgery. Earlier efforts and the CholecTriplet challenge introduced in 2021 have put together techniques aimed at recognizing these triplets from surgical footage. Estimating also the spatial locations of the triplets would offer a more precise intraoperative context-aware decision support for computer-assisted intervention. This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection. It includes weakly-supervised bounding box localization of every visible surgical instrument (or tool), as the key actors, and the modeling of each tool-activity in the form of triplet. The paper describes a baseline method and 10 new deep learning algorithms presented at the challenge to solve the task. It also provides thorough methodological comparisons of the methods, an in-depth analysis of the obtained results across multiple metrics, visual and procedural challenges; their significance, and useful insights for future research directions and applications in surgery.
description: MICCAI EndoVis CholecTriplet2022 challenge report. Published at Elsevier journal of Medical Image Analysis. 25 pages, 15 figures, 8 tables