# Development of an ontology for laparoscopic transabdominal adrenalectomy via a comprehensive modified Delphi survey and its validation on a multicentric pilot data set for surgical training and future video analysis with machine learning algorithms

Barbara Seeliger<fnref n="1" />,<fnref n="2" />,<fnref n="3" />,<fnref n="4" />,† ID, Sofia Di Lorenzo<fnref n="5" />,<fnref n="6" />,†, Pier F. Alesina<fnref n="7" />, Laurent Brunaud<fnref n="8" />,<fnref n="9" /> ID, Costanza Chiapponi<fnref n="10" />, Carmela De Crea<fnref n="6" />,<fnref n="11" />, Gianluca Donatini<fnref n="12" />,<fnref n="13" />, Maurizio Iacobone<fnref n="14" /> ID, Özer Makay<fnref n="15" /> ID, Radu Mihai<fnref n="16" /> ID, Martina T. Mogl<fnref n="17" />, Didier Mutter<fnref n="1" />,<fnref n="2" />,<fnref n="3" />, Nicolas Padoy<fnref n="1" />,<fnref n="4" />, Fausto Palazzo<fnref n="18" />, Oscar Vidal<fnref n="19" />, Francesco Pennestrí<fnref n="5" />,<fnref n="6" /> ID, Jacques Marescaux<fnref n="3" />, Michel Vix<fnref n="2" />,<fnref n="3" /> and Marco Raffaelli<fnref n="5" />,<fnref n="6" /> ID

<figure><img src="image_1.png" /></figure>

## Lay summary

Surgical technique is essential to ensure safe minimally invasive adrenalectomy. Due to the relative rarity of adrenal surgery, it is challenging to ensure adequate exposure in surgical training. Surgical video analysis supports auto-evaluation, expert assessment and could be a target for automatization. The developed ontology was validated by a European expert consensus and is applicable across the surgical techniques encountered in all participating centres, with an exemplary demonstration in bi-centric recordings. Standardization of adrenalectomy video analysis may foster surgical training and enable machine learning training for automated safety alerts.

Right and left transabdominal lateral adrenalectomy (RTLA/LTLA) are relatively rare procedures<fnref n="1" />, requiring appropriate training and experience, with a long learning curve<fnref n="2" />. Procedure-related complications can be serious, including liver, duodenal or inferior vena cava injury on the right side, or colonic, pancreatic, gastric, splenic and splenic vascular injury on the left. A video recording of the procedure provides documentation and may help identify

and avoid causes of intraoperative complications. Meanwhile, structured video analysis can provide individual training and would pave the way for automatic workflow recognition and safety alerts via artificial intelligence (AI) systems<fnref n="3" />.

Laparoscopic and retroperitoneoscopic adrenalectomy are the 'gold standard' for most surgically treated adrenal diseases<fnref n="4" />. A surgical ontology, outlining the procedure via its hierarchical

<footnote n="1">¹Institute of Image-Guided Surgery, IHU Strasbourg, Strasbourg, France</footnote>

<footnote n="2">²Department of Digestive and Endocrine Surgery, University Hospitals of Strasbourg, Strasbourg, France</footnote>

<footnote n="3">³Research Institute Against Digestive Cancer, IRCAD, Strasbourg, France</footnote>

<footnote n="4">⁴ICube, UMR7357, CNRS, INSERM U1328 RODIN, Université de Strasbourg, Strasbourg, France</footnote>

<footnote n="5">⁵Division of Endocrine and Metabolic Surgery, Fondazione Policlinico Universitario Agostino Gemelli—IRCCS, Rome, Italy</footnote>

<footnote n="6">⁶Centro di Ricerca in Chirurgia delle Ghiandole Endocrine e dell'Obesità, Università Cattolica del Sacro Cuore, Rome, Italy</footnote>

<footnote n="7">⁷Department of Surgery, Division of Endocrine Surgery, Helios Universitätsklinikum Wuppertal, Universität Witten-Herdecke, Wuppertal, Germany</footnote>

<footnote n="8">⁸Unit of Endocrine and Metabolic Surgery, Department of Surgery (CVMC), CHU Nancy—Hospital Brabois Adultes, University of Lorraine, Nancy, France</footnote>

<footnote n="9">⁹Faculty of Medicine, INSERM NGERE/U1256 'Nutrition, Genetics, Environmental Risks' University of Lorraine, Nancy, France</footnote>

<footnote n="10">¹⁰Department of General, Visceral, Cancer and Transplant Surgery, University Hospital Cologne, Cologne, Germany</footnote>

<footnote n="11">¹¹Division of Endocrine Surgery, Ospedale Isola Tiberina—Gemelli Isola, Rome, Italy</footnote>

<footnote n="12">¹²Department of General and Endocrine Surgery, Centre Hospitalier Universitaire de Poitiers, Poitiers, France</footnote>

<footnote n="13">¹³INSERM U1313-IRMETIST, Université de Poitiers, Poitiers, France</footnote>

<footnote n="14">¹⁴Endocrine Surgery Unit, Department of Surgery, Oncology, and Gastroenterology, University Hospital, University of Padua, Padua, Italy</footnote>

<footnote n="15">¹⁵Center for Endocrine Surgery, Ozel Saglik Hospital, Izmir, Turkey</footnote>

<footnote n="16">¹⁶Department of Endocrine Surgery, Churchill Cancer Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK</footnote>

<footnote n="17">¹⁷Department of Surgery, CCM | CVK, Charité—Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany</footnote>

<footnote n="18">¹⁸Department of Endocrine Surgery, Hammersmith Hospital, Imperial College Healthcare NHS Trust, London, UK</footnote>

<footnote n="19">¹⁹Department of General and Endocrine Surgery, Institute of Digestive and Metabolic Diseases, Hospital Clínic Barcelona, Universitat de Barcelona, Barcelona, Spain</footnote>

<footnote>*Correspondence to: Barbara Seeliger, IHU Strasbourg 1, Place de l'Hôpital, 67091 Strasbourg CEDEX, France (e-mail: barbara.seeliger@ihu-strasbourg.eu;</footnote>

<footnote>@DocSeeliger)</footnote>

<footnote>†Contributed equally.</footnote>

<footnote>**Upcoming communication:** The study was presented at the 10th Biennial Congress of the European Society of Endocrine Surgeons, Rome, Italy, in May 2024 and was selected as a candidate for the BJS Prize.</footnote>
<figure><img src="image_2.png" /><figcaption>**Fig. 1** Duration of the phases and steps in relation to the total duration of the video recordings of 4 LTLA and 4 RTLA from Strasbourg and Rome
Mandatory steps are coloured, facultative steps are grey. In case of P6 (Phase 6—Other intervention), no adrenalectomy steps are present. Each video was annotated by two surgeons. The inter-annotator agreement is illustrated by an almost complete match of phase and step durations for each video.</figcaption></figure>

decomposition into different levels of granularity with a
standardized vocabulary, is therefore required to provide a
machine-readable process model as a prerequisite for AI analysis
of the surgical workflow⁵. To address this, we developed an
ontology for multicentre use and validated its application for
standardized RTLA/LTLA video analysis in a clinical pilot.

An ontology was developed by a steering committee (BS/SDL/MR), hierarchically structuring RTLA/LTLA into five discrete phases—Preparation, Exposure, Dissection of the main adrenal vein, Dissection of the adrenal gland, Extraction and disassembling—with 27 RTLA and 25 LTLA steps (Figs S1 and S2). For both RTLA and LTLA, a sixth phase named other intervention was added for cases of simultaneous procedures.

Senior academics (n = 17) from across Europe with >10 years of
experience in adrenal surgery were invited to take part in a
modified Delphi process across two rounds (Fig. S3). Panellist
response rate was 88% in the first round (15/17) and 100%
(15/15) in the second round. Consensus, defined as ≥80%
agreement for each statement, was reached for LTLA in all 38
statements and for RTLA in 39/40 statements (97.5%; Tables S1
and S2). The content analysis of the panellists' comments
identified the recurring theme of the identification of the right
renal vein during RTLA as a facultative step rather than a
mandatory one, depending on tumour and anatomical factors.

Two surgeons then applied the defined ontology to annotate
each phase and step in 4 RTLA and 4 LTLA videos from two
centres in a machine-readable format. Near-complete agreement
between both surgeons was observed for the phase and step
durations (Fig. 1). Video annotations revealed that completion
of mandatory steps is required to proceed to the next phase,

although the sequence of steps may be adapted to the individual
anatomy. Procedure progression is not necessarily linear, as
dissection may require interruption of a phase/step, and progress
one step forward with completion of the previous step later. The
supplementary material to this letter describes the key phases
and steps illustrated in Video S1 (RTLA) and Video S2 (LTLA) as
they would appear in a dedicated video annotation software.

In this study, we have defined a novel consensus-derived
ontology for RTLA and LTLA. The ontology is suitable for
standardized and multicentric video assessments as a baseline
for stepwise understanding, surgical training and video analysis.
The ontology will aid machine learning algorithms for future AI
applications such as automated phase and step recognition,
surgical skill assessment, procedural training, and ultimately
'real-time' intraoperative guidance.

Author contributions

Barbara Seeliger (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Visualization, Writing—original draft, Writing—review & editing), Sofia Di Lorenzo (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Writing—original draft, Writing—review & editing), Pier Francesco Alesina (Investigation, Validation, Writing—review & editing), Laurent Brunaud (Investigation, Validation, Writing—review & editing), Costanza Chiapponi (Investigation, Validation, Writing—review & editing), Carmela De Crea (Investigation, Validation, Writing—review & editing), Gianluca Donatini (Investigation, Validation, Writing—review & editing), Maurizio Iacobone
(Investigation, Validation, Writing—review & editing), Özer Makay (Investigation, Validation, Writing—review & editing), Radu Mihai (Investigation, Validation, Writing—review & editing), Martina T. Mogl (Investigation, Validation, Writing—review & editing), Didier Mutter (Investigation, Validation, Writing—review & editing), Nicolas Padoy (Investigation, Validation, Writing—review & editing), Fausto F. Palazzo (Investigation, Validation, Writing—review & editing), Oscar Vidal (Investigation, Validation, Writing—review & editing), Francesco Pennestrí (Investigation, Validation, Writing—review & editing), Jacques Marescaux (Investigation, Resources, Validation, Writing—review & editing), Michel Vix (Investigation, Validation, Writing—review & editing), and Marco Raffaelli (Conceptualization, Investigation, Methodology, Resources, Supervision, Validation, Writing—review & editing)

## Funding

This work was supported by French state funds managed by the ANR within the 'Programme d'investissements d'avenir' France 2030 (reference ANR-10-IAHU-02).

## Acknowledgements

The authors thank Mathieu Haller for his invaluable help in the illustration of adrenal video annotations.

## Disclosures

Sofia Di Lorenzo, Pier F. Alesina, Laurent Brunaud, Costanza Chiapponi, Carmela De Crea, Maurizio Iacobone, Özer Makay, Radu Mihai, Martina T. Mogl, Didier Mutter, Nicolas Padoy, Fausto Palazzo, Oscar Vidal, Francesco Pennestrí and Michel Vix have no conflicts of interest or financial ties to disclose. Barbara Seeliger is the recipient of a grant from the French National Agency for Research (Agence Nationale de la Recherche (ANR), 86 rue Regnault, 75013 Paris, France) within the framework of the project AI-DIAL—Diagnostic Imaging of Adrenal Lesions

(ANR-22-CE17-0019-01) and has a consultant agreement with Intuitive Surgical. Gianluca Donatini is Scientific Board Advisor for Fluoptics-Getinge. Jacques Marescaux is the President and Founder of the IRCAD Institute, which receives grants from Karl Storz and Medtronic. Marco Raffaelli has a consultant agreement with Medtronic, AB Medica, and Intuitive Surgical.

## Supplementary material

Supplementary material is available at BJS online.

## Data availability

Raw data are available on request, and upon approval of a data-sharing agreement and analysis plan.

## References

1. Gaujoux S, Mihai R; Joint Working Group of ESES and ENSAT. European Society of Endocrine Surgeons (ESES) and European Network for the Study of Adrenal Tumours (ENSAT) recommendations for the surgical management of adrenocortical carcinoma. *Br J Surg* 2017;104:358–376

2. Mihai R, Donatini G, Vidal O, Brunaud L. Volume–outcome correlation in adrenal surgery—an ESES consensus statement. *Langenbecks Arch Surg* 2019;404:795–806

3. Mascagni P, Alapatt D, Lapergola A, Vardazaryan A, Mazellier JP, Dallemagne B et al. Early-stage clinical evaluation of real-time artificial intelligence assistance for laparoscopic cholecystectomy. *Br J Surg* 2024;111:znad353

4. Raffaelli M, De Crea C, Bellantone R. Laparoscopic adrenalectomy. *Gland Surg* 2019;8(Suppl 1):S41–S52

5. Meireles OR, Rosman G, Altieri MS, Hager CL, Madani G, Padoy A et al. SAGES consensus recommendations on an annotation framework for surgical video. *Surg Endosc* 2021;35:4918–4929