# Narrative Acts in the Operating Theater

Sandrine Barbois, Rémi Ronfard, Nicolas Szilas

► To cite this version:

Sandrine Barbois, Rémi Ronfard, Nicolas Szilas. Narrative Acts in the Operating Theater. CMN 2025 - 8th International Workshop on Computational Models of Narrative, May 2025, Geneva, Switzerland. pp.1-15. hal-05354212

HAL Id: hal-05354212

https://inria.hal.science/hal-05354212v1

Submitted on 7 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Narrative Acts in the Operating Theater

Sandrine Barbois<fnref n="1" />,<fnref n="2" />*, Rémi Ronfard<fnref n="2" /> and Nicolas Szilas<fnref n="3" />

**Abstract**

We propose a method for analyzing a surgical procedure as a narrative, considering not only technical acts but also dialogues between the different protagonists. Using a corpus of films of surgical procedures, we build a vocabulary of narrative acts that concisely represents the required non-technical skills of surgical staff. We suggest that this approach can be used to simulate the drama taking place in the operating theater and teach the required non-technical skills to students.

**Keywords**

Narrative acts, Simulation game, Medical education

## 1. Introduction

The operating theater is the place where surgery is performed. It is a unique and complex place because of the team effort involving several professions (surgeon, anesthetists, specialist nurses), with the same goal (treating the patient) but different procedural objectives. In this paper, we argue that the operating theater can also be considered as the theater or stage where a drama is played out, with characters, roles, challenging goals, and a spatial setting [1]. It is the site of human, social, economic, and scientific issues, with serious consequences in medical training and education. Recent research indicates that only 25% of the quality of a surgical operation depends on the technique. The remaining 75% are related to non-technical skills or knowledge [2], [3]. Developing non-technical skills consists of developing leadership, collective knowledge, practices, and behaviors such as communication, mutual aid, permanent evaluation of the situation concerning the environment, the patient and the actions in progress, and information sharing. These skills, also known as "cognitive-behavioral", are accessible to simulation-based learning, where professional actors and medical students play out a surgical operation together in a fake operating theater, with benefits now clearly demonstrated [4], [5]. A worthwhile goal for medical education would be to implement simulation-based learning using serious games in shared, virtual environments, in order to recreate "a safe place to make errors and learn as individuals and as teams" [6]. Computer modeling and simulation of surgical operations in virtual reality has become an active area of research in recent years [7], [8], [9], [10], [11]. For that purpose, we need to build a systematic method for the elaboration of coherent, interactive and educational scenarios. To properly include non-technical skills, we need to describe them in a systematic way with a rigorous and finite vocabulary, suitable for computer simulation. Towards that goal, we propose a description language for representing a surgical operation as a narrative sequence of technical and non-technical actions, and we illustrate the usefulness of the

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<footnote n="1">¹ Emergency and digestive surgery Unit, Hôpital Edouard Herriot, Hospices Civils de Lyon, Lyon, France</footnote>

<footnote n="2">² Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France</footnote>

<footnote n="3">³ TECFA, University of Geneva, Switzerland</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>† These authors contributed equally.</footnote>

<footnote>sandrine.barbois@chu-lyon.fr (S. Barbois); remi.ronfard@inria.fr (R. Ronfard); nicolas.szilas@unige.ch (N. Szilas)</footnote>

<footnote>0000-0002-0688-9723 (S. Barbois); 0000-0003-4830-5690 (R. Ronfard); 0000-0002-4565-8805 (N. Szilas)</footnote>

<footnote>© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).</footnote>
language in the task of describing videos of actual surgical operations in the operating theater. The paper is organized as follows. In Section 2, we explain our approach for representing the surgical procedure as a narrative sequence and describe our method based on the actions and behaviors observed in video recordings. We present our experimental results in Section 3 and discuss the perspectives and limitations of our work in Section 4.

# 2. The Surgical Procedure as a Singular Drama

## 2.1. A Narrative Point of View of Surgical Procedure

We have discussed the relevance of finding a formal description of non-technical skills in the context of the operating theater, and the constitution of a finite and relevant vocabulary for creating educational scenarios involving non-technical skills. While a classical viewpoint on educational simulation may consider that a simulation should be as close as possible to reality as possible [12], [13], “grounded in an authentic clinical context” [14], so that the learner could train in a realistic situation, a more thoughtful approach considers that not only perfect realism is not feasible, but more importantly that a educational simulation has a purpose [15], [16] : to teach the learner some knowledge and know-how in a given domain. In this latter view, the analysis of surgical procedure cannot be neutral —some coding assumptions need to be made anyway— and could take advantage of seeing the operating theater as a narrative. Following the narrative hypothesis in cognition [17], this approach would allow a greater impact of the simulation. A narrative is, by definition, made to impact its reader, to have a point of view on the reality to be portrayed.

## 2.2. Common Characteristics Between Drama and Surgical Procedure

A drama can be seen as a particular form of narrative, involving direct perception of the characters in action. And if we look closely, we can find several common characteristics between a drama and the description of what composes a surgical procedure. Indeed, “medical drama” is a highly topical television drama genre, having some success in fiction such as the TV series “ER”, “Scrubs” or “Grey’s Anatomy”. Although these series also deal with private lives and characters’ romances, they are fundamentally focused on the medical world, without censoring either the vocabulary or the technical medical procedures.

First of all, in the surgical procedure, we find roles: surgeons, nurses, anesthetists, and of course the patient. All these characters share a set of goals (to care for the patient and ensure his/her safety, guarantee optimal and well-reasoned organization of the technical platform). However, they have different training and perform different but complementary tasks, and they also have different hierarchical statuses, which may represent an important stake in a drama. Then, the entire surgical procedure is a succession of tasks, in a rational order, that the surgeon and his team must accomplish to carry out the operation. A succession of surgical tasks can be seen as a narrative sequence. Indeed, this succession of tasks can be considered as a series of transformations that bring the "quest" from point A to point B. Furthermore, non-technical behaviors can be considered as triggers, generic actions for generating transitions necessary for the surgical procedure. Surgical actions and tasks cannot currently be programmed or fully robotized. They are performed by human beings, who need indications and communication elements to coordinate, rectify, plan and execute these actions. This collaboration and its expressed way, involving non-technical skills, can be seen as a way of performing the main action, achieving the objective.
For example, when the surgeon has to cut the skin with a scalpel, s/he can take it himself/herself or interact with the scrub nurse to obtain it. He can ask for it, or order it. The scrub nurse may also propose the instrument even before the surgeon asks for it. These behaviors reveal non-technical skills such as communication, anticipation, leadership.

**2.3. The Concepts of Narrative Acts and Meta-Actions**

Meta-actions are a transversal concept that can be described as “an action about an action”. For example, when the surgeon requests a scalpel, she performs the meta action of requesting someone to give her a scalpel, “giving the scalpel” being a simple action.

This concept is borrowed from a narrative theory, in an attempt to unify the various motifs, processes and narrative transformations proposed by narratologists [18].

Considering a central action in the story, characterized by a goal to be achieved, the meta-actions that compose it and allow its transformation are narrative acts: “A narrative act is a type of meta-action in which the embedded action is one of the core actions of the story (should this embedded action be performed or not)” [19]. According to this definition, narrative acts are not only represented by speech acts, but are defined by their relation to the actions constituting the core narrative sequence, in our case the surgical procedure.

**2.4. Technical and Non-Technical Actions**

Like a drama, a surgical procedure is structured into acts and scenes. Young surgeons learn the various stages involved in the procedure by companionship in the operating theater and from surgical manuals [20]. There is therefore a succession of operative steps, composed of actions, that represent a succession of transformations that enable the procedure to progress towards its goal: removal of an organ, repair of an anatomical structure, implantation of a pharmaceutical device, etc.

**3. Building a Set of Narrative Acts Dedicated to Surgery**

Our goal is to build a set of narrative acts that covers the “drama” occurring in the operating theater. There already exist extensive classifications of narrative acts or speech acts [19], [21], [22], but they would contain a large quantity of acts that would not occur in the field. Conversely, the specificity of the surgical environment might contain narrative acts not found in general classifications. Therefore, we decided to build this set directly from the observation of the operating theater.

We chose to focus on a laparoscopic cholecystectomy (i.e. gallbladder removal), a very common procedure. It is very well described and codified, with technical guidelines validated by the surgical community [20], leaving little opportunity for inter-operator technical variability.

Schematically, an intervention could be represented at several levels of granularity. The more detailed the level of description, the lower the level. At the highest level of granularity, we might therefore find the procedure itself (e.g. appendectomy, heart valve replacement, etc.). Then, we can describe the various phases, steps, tasks, and finally actions, at the lowest level of granularity [23], [24].
In addition to "technical" actions, there are actions that could be described as "non-technical". These include acts of communication. In our work, we will mainly focus on verbal acts. Other forms of non-verbal communication are limited: the characters wear masks, which limits their facial expressions, and to ensure the sterility of the operating area, contact between characters and between characters and objects in the operating theater is also limited.

**3.1. Method**

We chose video recordings as a modeling strategy to satisfy our need for a comprehensive formalism, using an « off-line » observation method. This procedure allows us to grasp the subtlety of verbal and non-verbal interactions. The main difficulty of this type of data acquisition in laparoscopic surgery is often a description of activities with a high level of granularity, considering only the images from the operating camera, i.e. the “laparoscopic” camera inserted in the abdomen and used by the surgeon to operate. To overcome this difficulty, we have chosen to combine two different film shots, from the laparoscopic camera (used during the operation, see inset on upper right corner of Figure 1) and from the operating room, by the setting of a mini camera (Figure 1).

To respect the patient's anonymity, the procedure does not begin when they arrive in the operating room but just after surgical drapes placement, and ends before removing them.

We chose to use a spreadsheet to collect the data. Each line of the table corresponded to one second of film and describes the start of an observed action. Several actions could begin at the same time. The duration of the actions was also reported. The choice to use a spreadsheet was motivated by the great flexibility of support adaptation, allowing an inductive approach to data collection.

So far, three surgical operations were filmed and annotated, for a total of 1h57 of recording. After annotation, this corpus contains 3430 technical and non-technical actions. The videos were annotated by the first author of this article who is also a surgeon and therefore the medical expert for this work. This made annotation easier, particularly for recognizing instruments and interpreting characters actions in videos.

<figure><img src="image_5.png" /><figcaption>**Figure 1:** Film shot of the operating room and laparoscopic camera (embedded) : (a) surgeon, (b) resident, (c) nursing assistant, (d) scrub nurse, (e) anesthetist.</figcaption></figure>
## 3.2. Results

Describing the videos has allowed us to develop a vocabulary of technical acts that can be identified for that type of surgical procedure. We found 37 core technical actions needed to perform the surgical procedure. These 37 technical actions cover all the actions observed in the videos. They are listed in Table 1 in predicative form.

<caption>**Table 1**</caption>

Technical actions. X,Y denote characters; o,o' objects (equipment, furniture, tool, anatomical structure, etc.), p a place, and d a direction

<table><thead><tr><td>Technical acts (EN)</td><td>Meaning</td></tr></thead><tbody><tr><td>Activate (X,o)</td><td>X activates an object o using a pedal or handle</td></tr><tr><td>Adjust (X,o)</td><td>X adjusts an object o</td></tr><tr><td>Attach (X,o,o')</td><td>X attaches 2 object parts o and o' together</td></tr><tr><td>Clean up (X,o,o')</td><td>X cleans up an object o with another object o'</td></tr><tr><td>Connect (X,o,o')</td><td>X connects 2 object parts o and o' together</td></tr><tr><td>Count (X,o)</td><td>X counts object o (ex: surgical pads)</td></tr><tr><td>Cut (X,o,o')</td><td>X cuts an object o with a cutting object o'</td></tr><tr><td>Dab (X,o,o')</td><td>X dabs an object o with another object o'</td></tr><tr><td>Deflate (X,o,o')</td><td>X deflates an object o with another object o'</td></tr><tr><td>Direct to (X,o,d or Y)</td><td>X directs an object o to a direction d or character Y</td></tr><tr><td>Disconnect (X,o,o')</td><td>X disconnects 2 object parts o and o'</td></tr><tr><td>Fit on (X,o)</td><td>X fits a protection on an object o</td></tr><tr><td>Give (X,Y,o)</td><td>X gives an object o to a character Y</td></tr><tr><td>Hold with (X,o,o') / towards (X,o,d or Y)</td><td>X holds an object o with another object o' or towards a direction d or a character Y</td></tr><tr><td>Inflate (X,o,o')</td><td>X inflates an object o with another object o'</td></tr><tr><td>Inject in (X,o,o')</td><td>X injects a liquid object o (substance) into an (anatomical) object o'</td></tr><tr><td>Insert in (X,o,o')</td><td>X inserts a (part of an) object o in another object o'</td></tr><tr><td>Make an incision (X,o,o')</td><td>X makes an incision on an anatomical object o with another sharp object o'</td></tr><tr><td>Move to (X,p)</td><td>X moves to a place p</td></tr><tr><td>Note (X,i,o)</td><td>X notes information i on an object o</td></tr><tr><td>Open (X,o)</td><td>X opens an object o</td></tr><tr><td>Pour in (X,o,o')</td><td>X pours a liquid object o in another object o'</td></tr><tr><td>Put in/on (X,o,o')</td><td>X puts an object o in/on another object o'</td></tr><tr><td>Release (X,o)</td><td>X releases an object o</td></tr><tr><td>Remove from (X,o,o')</td><td>X removes a (part of an) object o from another tool o'</td></tr><tr><td>Set in/on (X,o,o')</td><td>X sets an object o in/on another object o'</td></tr><tr><td>Spread (X,o,o')</td><td>X spreads an anatomical object o with another object o'</td></tr><tr><td>Stand at (X,p)</td><td>X stands at a place p</td></tr><tr><td>Suture with (X,o,o')</td><td>X sutures an anatomical object o with another object o'</td></tr><tr><td>Swab with (X,o,o')</td><td>X swabs an anatomical structure o with an object o'</td></tr><tr><td>Take with/from (X,o,o')</td><td>X takes an object o with a tool or from an object o'</td></tr><tr><td>Throw in (X,o)</td><td>X throws an object o in garbage can</td></tr><tr><td>Touch with (X,o,o')</td><td>X touches an object o with another object o'</td></tr></tbody></table>
<table><tr><td>Turn off (X,o)</td><td>X switches off a generator o or a tool</td></tr><tr><td>Turn on (X,o)</td><td>X switches on a generator o or a tool</td></tr></table>

To these technical actions can be added certain non-technical acts, in particular acts of communication. In our work, we call these "dialogues".

We have extracted 30 narrative acts which, when combined with our central procedural actions (technical action OR dialogue) or state, allow us to describe how how the procedure is completed.

These narrative acts have been organized hierarchically, according to a logical description in a taxonomy of actions domains initially proposed by Szilas et al [19], based on the moment when the narrative act occurs in relation to the action it contains: Decision, Influence, Preparation, Performance, Sanction, and a sixth domain called Knowledge management.

<caption>**Table 2**</caption>

Narrative acts identified in the videos, their meaning and classification according to the one proposed by Szilas et al. [18], [19]. "a" denote an action; "s" a state.

<table><thead><tr><td>Narrative acts</td><td>Meaning</td><td>Domain (Szilas 2018)</td></tr></thead><tbody><tr><td>AcceptToPerform (X,a)</td><td>X accepts to perform action a</td><td>Decision</td></tr><tr><td>AcceptHelp (X,Y,a)</td><td>X accepts help from Y to perform action a</td><td>Preparation</td></tr><tr><td>AcceptAndPerform (X,Y,a)*</td><td>X accepts Y's request and perform action a</td><td>Performance</td></tr><tr><td>Admit (X,s)</td><td>X admits that state s is true</td><td>Knowledge management</td></tr><tr><td>Advise (X,Y,a)</td><td>X advises Y to perform action a</td><td>Influence</td></tr><tr><td>Apologize (X,Y,a)</td><td>X apologizes to Y for performing action a</td><td>Sanction</td></tr><tr><td>AskAbout (X,Y,s)</td><td>X asks Y what he knows about state s</td><td>Knowledge management</td></tr><tr><td>AskForHelp (X,Y,a)</td><td>X asks Y for help to perform action a</td><td>Preparation</td></tr><tr><td>Assert (X,Y,s)</td><td>X asserts to Y that state s is true</td><td>Knowledge management</td></tr><tr><td>Confirm (X,Y,s)</td><td>X confirms to Y that state s is true</td><td>Knowledge management</td></tr><tr><td>DeclareIntention (X,Y,a)</td><td>X declares to Y his/her intention to perform action a</td><td>Decision</td></tr><tr><td>Dissuade (X,Y,a)</td><td>X dissuades Y to perform action a</td><td>Influence</td></tr><tr><td>Doubt (X,s)</td><td>X doubts that state s is true</td><td>Knowledge management</td></tr><tr><td>Encourage (X,Y,a)</td><td>X encourages Y to perform action a</td><td>Influence</td></tr><tr><td>Inform (X,Y,predicate)</td><td>X informs Y that (predicate)</td><td>Knowledge management</td></tr><tr><td>InformExecution (X,Y,a)</td><td>X declares to Y that s/he is performing action a</td><td>Knowledge management</td></tr><tr><td>Insist (X,Y,a)</td><td>X insists that Y perform action a</td><td>Influence</td></tr><tr><td>Observe (X,Y,a)</td><td>X observes Y performing action a</td><td>Knowledge management</td></tr><tr><td>Order (X,Y,a)</td><td>X orders Y to perform action a</td><td>Influence</td></tr><tr><td>Permit (X,Y,a)</td><td>X permits Y to perform action a</td><td>Influence</td></tr><tr><td>ProposeHelp (X,Y,a)</td><td>X proposes to help Y to perform action a</td><td>Preparation</td></tr><tr><td>Propose (X,Y,a)</td><td>X proposes that Y perform action a</td><td>Decision</td></tr><tr><td>RefuseHelp (X,Y,a)</td><td>X refuses Y's help to perform action a</td><td>Preparation</td></tr><tr><td>RefusePerformance (X,Y,a)</td><td>X refuses that Y perform action a</td><td>Collaboration</td></tr><tr><td>Regret (X,a)</td><td>X regrets performing action a</td><td>Sanction</td></tr><tr><td>Renounce (X,a)</td><td>X renounces to perform action a</td><td>Decision</td></tr></tbody></table>
<table><tr><td>Request (X,Y,a)</td><td>X requests Y to perform action a</td><td>Influence</td></tr><tr><td>Thank (X,Y,a)</td><td>X thanks Y for performing action a</td><td>Sanction</td></tr><tr><td>Think (X,Y,s)</td><td>X informs Y that he thinks state s is true</td><td>Knowledge management</td></tr><tr><td>Undertake (X,a)</td><td>X undertakes to perform action a</td><td>Performance</td></tr></table>

### 3.3. Examples of Narrative Sequences

To illustrate our analysis of "surgical drama" with these narrative acts, we present below several examples taken from videos, focused on a fairly generic task called "Using an instrument". To operate, the surgeon uses instruments. These instruments are placed on a table (back table). They are easily accessible by the nursing assistant.

To hold these instruments, the surgeon can either take them alone or interact with the nurse. Then, he or she uses it and returns it (or puts it on the back table himself or herself).

Figure 2 shows a first example, where the surgeon (a) says "Give me the scalpel". The nurse (b) takes the scalpel from the table and gives it to the surgeon. This is the most common situation where the surgeon orders an action and the nurse performs it.

A simplified example of our data collection table is shown in Figure 2.

<figure><img src="image_6.png" /></figure>

<table><caption>**Figure 2:** First example of task "Use an instrument"</caption><thead><tr><th>Timeline</th><th>Character</th><th>To</th><th>Action</th><th>Dialogues</th><th>Narrative act</th></tr></thead><tbody><tr><td>01:23</td><td>Surgeon</td><td>Nurse</td><td></td><td>Give me the scalpel</td><td>Order (X,Y,Give)</td></tr><tr><td>01:26</td><td>Nurse</td><td></td><td>Take scalpel on table</td><td></td><td>Undertake (X, Take)</td></tr><tr><td>01:27</td><td>Nurse</td><td>Surgeon</td><td></td><td></td><td>AcceptAndPerform (X,Y,Give)</td></tr></tbody></table>

Top: screenshots of the scene (a: surgeon; b: nurse; c: resident)
Bottom: simplified data collection model for this example

Figure 3 shows a second example. This time, the nurse (b) takes instruments from the table. This action is not preceded by any request from the surgeon (a) or his resident (c). In this example, the resident agrees to the nurse giving him the instruments, without having asked for them. In this case, accepting is not described as a narrative act in our table. The nurse's action is noted as "successful".
The nurse showed anticipation and initiative by offering the surgeon the instrument he needed even before he asked for it.

<figure><img src="image_7.png" /></figure>

<table><caption>**Figure 3:** Second example of task “Use an instrument”</caption><thead><tr><td>Timeline</td><td>Character</td><td>To</td><td>Action</td><td>Dialogues</td><td>Narrative act</td></tr></thead><tbody><tr><td>01:11</td><td>Nurse</td><td></td><td>Take retractors on table</td><td></td><td>Undertake (X,Take)</td></tr><tr><td>01:13</td><td>Nurse</td><td>Resident</td><td>Give retractors</td><td></td><td>Propose(X,Y,Give)</td></tr></tbody></table>

Top: screenshots of the scene (a: surgeon; b: nurse; c: resident)

Bottom: simplified data collection model for this example

Figure 4 shows a third example, where the nurse (b) takes an instrument from the table and asks the surgeon "Do you want this instrument? The surgeon replies "No".

<figure><img src="image_8.png" /></figure>

<table><caption>**Figure 4:** Third example of task “Use an instrument”</caption><thead><tr><td>Timeline</td><td>Character</td><td>To</td><td>Action</td><td>Dialogues</td><td>Narrative act</td></tr></thead><tbody><tr><td>02:39</td><td>Nurse</td><td></td><td>Take forceps on table</td><td></td><td>Undertake (X,Take)</td></tr><tr><td>02:40</td><td>Nurse</td><td>Surgeon</td><td></td><td>Do you want forceps?</td><td>Propose (X,Y,Give)</td></tr><tr><td>02:42</td><td>Surgeon</td><td>Nurse</td><td></td><td>No</td><td>RefusePerformance (X,Y,Give)</td></tr></tbody></table>

Top: screenshots of the scene (a: surgeon; b: nurse; c: resident)

Bottom: simplified data collection model for this example
In this example, the nurse fails to suggest the most appropriate instrument to the surgeon. This may be due to a lack of knowledge of the instrument best suited to the action, or a lack of knowledge of the surgeon's working habits. The surgeon's refusal is described as a narrative act because it is explicit ("No"), unlike the previous example, and the nurse's action is noted as a failure. When unsuccessful suggestions are repeated, the surgeon may become irritated.

Finally, Figure 5 shows a fourth example. The surgeon (a) takes his instruments directly from the back table.

<figure><img src="image_9.png" /></figure>

<table><caption>**Figure 5:** Fourth example of task "Use an instrument"</caption><thead><tr><td>Timeline</td><td>Character</td><td>To</td><td>Action</td><td>Dialogues</td><td>Narrative act</td></tr></thead><tbody><tr><td>03:34</td><td>Surgeon</td><td></td><td>Take scalpel on table</td><td></td><td>Undertake (X,Take)</td></tr><tr><td>04:02</td><td>Surgeon</td><td></td><td>Make incision</td><td></td><td>Undertake (X, Make an Incision)</td></tr><tr><td>04:15</td><td>Surgeon</td><td></td><td>Put scalpel on table</td><td></td><td>Undertake (X,Put)</td></tr></tbody></table>

Top: screenshots of the scene (a: surgeon; b: nurse; c: resident)

Bottom: simplified data collection model for this example

In this example, the surgeon performs all the actions alone and does not communicate with his operating assistants. Taking and putting instruments on the back table requires him to move and turn around, which is not recommended (movements must be limited to avoid aseptic errors). His attitude can lead to a lack of initiative from his operating assistants. Note the resident (c) standing back when the surgeon makes the incision in the skin, passive, as deprived of his "helping" role.

These simple examples show how, starting from the actions of a generic task, we can observe different behaviors, and suggest skills that are also expressed differently. For example, a sense of initiative on the nurses' part can facilitate the surgeon's work. Lack of communication can create a climate of tension. These different behaviors are well expressed through the narrative acts that incorporate technical actions.

# 4. Discussion

## 4.1. Findings and Observation

In this work, we proposed a method for analyzing a surgical procedure as a narrative, considering not only actions but also dialogues between the different protagonists.
Video recordings offered the advantage of a more exhaustive analysis than the simple note-taking of an observer who witnessed the intervention and would have lost some information. This work was an incremental and adaptive process, with numerous modifications and versions in the data collection table, to keep only relevant actions and to highlight the non-technical skills required for their implementation.

From a surgical point of view, while technical actions are central to describing the procedure, they are not sufficient to describe the “drama” whose story we want to tell. With this original method, we wanted to describe more precisely how they are introduced, accompanied and followed by “non-technical” coordination acts.

We extracted a set of 30 narrative acts, generic types of actions interpreted in a narrative framework, as a vocabulary that concisely represents the required non-technical skills of surgical staff. Most of these acts are already covered by the existing classification that served as a starting point, but it is noticeable that our analysis enabled a focalization on a short number of narrative acts, given the fact that some existing classifications contain more than six hundred narrative acts [21]. These 30 narrative acts, combined with the 37 technical actions constitute a manageable vocabulary to generate an extensive set of possible actions in a given narrative domain. The originality of this approach is that narrative act refers to a technical action and describes how this action is carried out collaboratively by team's members. They are not always speech acts.

Particular attention has been dedicated to the choice of narrative acts, in particular to satisfy the need to describe highly procedural actions. Indeed, before choosing “Undertake”, we had initially proposed the term “Decide”, with the intention of describing autonomous, self-initiated intention. However, the intellectual process that drives a character to act, especially if he is not acting in response to a command, is not visible. What is visible is the result of this choice, i.e. the action itself.

Following the same explanatory and demonstrative constraints, we have created the “AcceptAndPerform” narrative act, to describe the performance of an action in response to a request or order. This necessity reflects the highly technical nature of the surgical environment, where actions are organized in a particularly procedural sequence (action-reaction sequence).

We also found an original narrative act, originally named “DeclarePerformance”, to describe when a character states aloud what he or she is doing. This attitude corresponds to a best practice rule and aims to keep the rest of the team informed of an operative step that is, if not dangerous, at least crucial for the rest of the procedure. Effectively, it is not something that generally occurs in stories — it is even in opposition with the well-known writing tip in drama: “show, don’t tell”—, which illustrates a specificity of the context of the operation theater. After some discussion, we finally decided that “InformExecution” seemed to be appropriate too, and this action was preferred to the previous one.

## 4.2. Related work

Surgical procedures have long been studied for their technicality, complexity and risk. The data collected during surgical procedures is used to establish Surgical Procedure Models (SPM), which is defined as “a simplified pattern of a surgical process that reflects a predefined subset of interest of the surgical process in a formal or semi-formal representation” [25]. These models are used for evaluation of surgical tools/systems/approaches, training and assessment of surgeons, optimization of operative room management, context-aware systems and robotic assistance [24].
The OntoSPM project [26] is a European collaborative action whose aim is to provide a common ontology for SPM. It provides a common vocabulary for a large and relevant knowledge corpus and allows it to be shared in a semantically consistent, machine-readable form. There are many possible application areas of this description formalism, including computer simulation [9]. While there are many knowledge areas formalized in this way, such as procedures division into tasks, identification of the actors and anatomical structures involved, surgical instruments used, trajectories, movements, etc. This ontology includes 33 technical human actions ("manipulating an object"), such as "Cut an object", "Take an object" or "Inject a substance". There is currently no conceptual representation of soft skills in OntoSPM, except some speech acts such as " Authorize ", "Ask", "Inform" and observations as "Assessment action" or "Reasoning action". In our work, we were focused not only on the surgeon's gesture, but also on the actions of the other characters and their interactions. We therefore believe that our work could contribute to the previous work of OntoSPM with a more detailed and logical description of the non-technical (soft) skills occuring in the operating theater.

The learning and maintenance of non-technical skills is a new issue in surgery. Many teams have been working on the development of simulation scenarios involving these non-technical skills. High-fidelity simulation, such as a realistic physical reproduction of an operating theater environment, has the advantage of being able to consider both technical and non-technical skills simulation [27], [28]. An alternative is the development of collaborative virtual environments, which are likely to be less organizationally demanding to implement.

Among recent research works that have been interested in these simulation scenarios, we observe a great heterogeneity in terms of the educational objectives (management of a crisis situation, risk prevention [29], [30], procedural coordination [31]), the initial situations (equipment failure, hemorrhage [29], staffing issues [30], etc.), the source of information [32] (expert opinion [29], surgical videos, mix of both [9]), and the nature of information (technical knowledge, speech acts, operating theater environment). A standardized analysis method would be useful to organize this large body of work into a more coherent corpus.

Our work proposed to use this representation, based on narrative acts, for teaching non-technical skills to students.

We observed a strong semantic similarity between the taxonomy proposed by [19], presented in Table 2 and some rating systems used to evaluate non-technical skills in surgery. The NOTSS (Non-Technical Skills for Surgeons) [33] and NOTECHS II (modified Oxford Non-technical Skills) [34] provide a common vocabulary for the description of non-technical — implicit and informal — skills in surgery, and thus a degree of formality. Like many behavioral marker systems in demanding environments (medicine, aeronautics, nuclear energy), these rating scales present skills in a taxonomy. The four main categories of skills are quite similar between these two systems. These are Situation Awareness, Decision Making (and Problem Solving), Communication and Teamwork (Co-operation), and Leadership (Management). Examples of positive and negative behaviors are given in the NOTSS system. For example, "gathering information" is an element of "Situation awareness" skill, defined by "seeking information in the operating theater from the operative findings, theater environment, equipment and people". One of the good behaviors could be "Ask anesthetist for update", about medical patient situation. In Table 2, we found "AskAbout", a narrative act which, within the narrative framework of a surgical procedure, could refer to a behavior that emphasizes information sharing for a better knowledge of the situation. In future work, we are planning to evaluate sequences of narrative acts in terms of NOTSS and use them in simulation-based teaching of NOTSS.

Other taxonomies have been developed for other operating theater actors, including nurses [35] and anesthesiologists [36], based on observation of reality, preceding the development of expert's
consensus. It is important to note that the same broad skill categories are present, although expressed differently, through different behaviors, according to roles in the operating theater. They are mainly used in practice for the evaluation of novices and/or teams [37], especially during simulation courses ("exhibits this behavior" / "does not exhibit this behavior" / "exhibits poor behavior"). However, they cannot be recommended for formal assessment [33], and, more importantly, they lack the necessary precision for a computer simulation.

Finally, we believe that our taxonomy of narrative acts presented in this work open new directions for using previous work in social physics and interactive drama [38], [39], [40], [41], [42], [43], [44] toward the long term goal of authoring computer simulations of team work in surgical operations.

### 4.3. Limitations and Future Work

During the annotation process, we encountered a few methodological difficulties. The first difficulty is related to the data collection method. To avoid disturbance, we used a battery-powered mini sports camera, placed in the operating theater. Because of the fixed shot, some actions are difficult to observe (e.g. which instrument is taken from the table). Some are not seen. The coherence of the scenarios therefore depends on the expertise of those who annotate videos. Similarly, it is not always possible to transcribe all the words spoken: the surgical instruments and anesthetic monitors cause noise pollution, several protagonists may be speaking at the same time, and wearing a mask sometimes makes it difficult to understand what is being said.

Another limitation is that only one expert annotated the videos. In future work, we will need to have the proposed vocabulary of technical and non-technical actions validated by other surgeons, and even by other non-physician researchers, in order to check that the annotations proposed are consistent. In addition, we plan to include additional videos in our data to check that the lists proposed in this article, both the list of technical actions and the list of narrative acts, are complete. However, we assume that the corpus of narrative acts does not to be enriched with other acts from videos. Our aim is pedagogical: it will probably be more efficient to build synthetic examples by adding, removing or replacing narrative acts in existing examples.

Our next step will be to design a minimalist ontology, NAOT (Narrative Acts in the Operating Theater), containing the characters, the objects (surgical equipment, furniture, pharmaceutical products, etc.), the different phases of the procedure, the actions required to perform them and the relevant meta-actions that we have extracted to formalize non-technical skills. By extending this method to other surgical procedures, we expect the number of technical acts to increase. However, we believe that the narrative acts observed will probably remain the same.

Our vocabulary of narrative acts also offers interesting potential for constructing educational simulation scenarios where the learner is invited not only to work on his procedural knowledge, but also to solicit the relational, communication and teamwork skills that we are trying to highlight. Our approach is well suited to simulating an operation, either with actors or with video game non-player characters (NPCs). We believe that even in a low-realism environment where technical actions would not be represented, a scenario with narrative acts alone would be sufficient to learn non-technical skills.

Many of the simulation scenarios already in use have focused on reproducing crisis situations. Although these situations require perfect coordination between team members, they are fortunately unusual. We preferred to observe frequent, routine, well-coded interventions with a low risk of a major problem occurring. The aim was to show how these non-technical skills materialize in everyday
teamwork. We hypothesize, for our future work, that non-crisis scenarios can be just as instructive
and interesting for the training of these skills [14].

5. Conclusion

In this paper, we have proposed a method for analyzing a surgical procedure as a narrative, considering not only technical acts but also dialogues between the different protagonists. To this effect, we have shown that surgery and drama share some characteristics: characters with roles, a technical "challenge" comparable to a quest, some actions, which combine with each other to form narrative sequences. Using a corpus of films of surgical procedures, we have built a vocabulary of narrative acts that concisely represents the required non-technical skills of surgical staff. By leveraging recent results in narrative theory and interactive storytelling, we hope to make a useful contribution to the fast-growing field of surgical process modeling, and open new directions for simulation-based medical training and education.

Declaration on Generative AI

The author(s) have not employed any Generative AI tools.

References

[1] R. Wilson, 'Teamwork in the Operating Room', *Human Organization*, vol. 12, no. 4, pp. 9-14, Dec. 1953, doi: 10.17730/humo.12.4.33300646734k2435.

[2] J. C. Hall, C. Ellis, and J. Hamdorf, 'Surgeons and cognitive processes', *Br J Surg*, vol. 90, no. 1, pp. 10–16, Jan. 2003, doi: 10.1002/bjs.4020.

[3] L. Hull, S. Arora, R. Aggarwal, A. Darzi, C. Vincent, and N. Sevdalis, 'The impact of nontechnical skills on technical performance in surgery: a systematic review', *J Am Coll Surg*, vol. 214, no. 2, pp. 214–230, Feb. 2012, doi: 10.1016/j.jamcollsurg.2011.10.016.

[4] J. Neily et al., 'Association between implementation of a medical team training program and surgical mortality', JAMA, vol. 304, no. 15, pp. 1693–1700, Oct. 2010, doi: 10.1001/jama.2010.1506.

[5] L. Fung et al., 'Impact of crisis resource management simulation-based training for interprofessional and interdisciplinary teams: A systematic review', *J Interprof Care*, vol. 29, no. 5, pp. 433–444, 2015, doi: 10.3109/13561820.2015.1017555.

[6] E. W. Riddle, D. Kewalramani, M. Narayan, and D. B. Jones, 'Surgical Simulation: Virtual Reality to Artificial Intelligence', *Curr Probl Surg*, vol. 61, no. 11, p. 101625, Nov. 2024, doi: 10.1016/j.cpsurg.2024.101625.

[7] V. Boccara, R. Delmas, and F. Darses, 'Ergo-Scripting in Activity-Based Training Design: An Illustration from the Design of a Virtual Environment', in *Simulation Training through the Lens of Experience and Activity Analysis*, vol. 30, S. Flandin, C. Vidal-Gomel, and R. Becerril Ortega, Eds., Cham: Springer International Publishing, 2022, pp. 129–149. doi: 10.1007/978-3-030-89567-9_7.

[8] G. Claude, V. Gouranton, and B. Arnaldi, 'Roles in collaborative virtual environments for training', presented at the Eurographics Symposium on Virtual Environments, 2015.

[9] G. Claude, V. Gouranton, B. Caillaud, B. Gibaud, P. Jannin, and B. Arnaldi, 'From Observations to Collaborative Simulation: Application to Surgical Training', in *ICAT-EGVE 2016 Proceedings*, Little Rock, Arkansas, United States, Dec. 2016. Accessed: Jan. 13, 2025. [Online]. Available: https://hal.science/hal-01391776

[10] F. Lécuyer, V. Gouranton, A. Lamercerie, A. Reuzeau, B. Caillaud, and B. Arnaldi, 'Unveiling the implicit knowledge, one scenario at a time', *Vis Comput*, vol. 36, no. 10–12, pp. 1951–1963, Oct.
2020, doi: 10.1007/s00371-020-01904-7.

[11] M. Risy, B. Arnaldi, and V. Gouranton, 'SAMPO, a Scenario Authoring Model for Virtual Reality with Pedagogical Objectives: An Authoring Perspective', *Communications in Computer and Information Science*, vol. 12, inPress.

[12] S. B. Issenberg, W. C. McGaghie, E. R. Petrusa, D. Lee Gordon, and R. J. Scalese, 'Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review', *Med Teach*, vol. 27, no. 1, pp. 10-28, Jan. 2005, doi: 10.1080/01421590500046924.

[13] G. Alinier, 'A typology of educationally focused medical simulation tools', *Med Teach*, vol. 29, no. 8, pp. e243-250, Oct. 2007, doi: 10.1080/01421590701551185.

[14] R. L. Kneebone, D. Nestel, C. Vincent, and A. Darzi, 'Complexity, risk and simulation in learning procedural skills', *Med Educ*, vol. 41, no. 8, pp. 808-814, Aug. 2007, doi: 10.1111/j.1365-2923.2007.02799.x.

[15] D. Richards and N. Szilas, 'Challenging reality using techniques from interactive drama to support social simulations in virtual worlds', in *Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System*, Auckland New Zealand: ACM, Jul. 2012, pp. 1-10. doi: 10.1145/2336727.2336739.

[16] L. Vadcard, 'Réflexions à propos de la conception d'environnements de formation par la simulation : le cas de la formation médico-chirurgicale:', *Raisons éducatives*, vol. N° 21, no. 1, pp. 81-96, Nov. 2017, doi: 10.3917/raised.021.0081.

[17] J. Bruner, 'Self-Making and World-Making', *Journal of Aesthetic Education*, vol. 25, no. 1, p. 67, 1991, doi: 10.2307/3333092.

[18] N. Szilas, 'Les actes narratifs : définition et typologie', *narratologie*, no. 41, May 2022, doi: 10.4000/narratologie.13794.

[19] N. Szilas, M. Marano, and S. Estupiñán, 'A Tool for Interactive Visualization of Narrative Acts', in *Interactive Storytelling*, vol. 11318, R. Rouse, H. Koenitz, and M. Haahr, Eds., Cham: Springer International Publishing, 2018, pp. 176-180. doi: 10.1007/978-3-030-04028-4_17.

[20] L. Barbier, N. Tabchouri, and E. Salame, 'Technique de la cholécystectomie', *Techniques chirurgicales - Appareil digestif*, vol. 38, no. 1, pp. 1-11, 2021.

[21] N. Szilas, 'Narrative Act Catalog'. [Online]. Available: https://teclacl3.unige.ch/nacl/

[22] T. T. Ballmer and W. Brennenstuhl, *Speech act classification: a study in the lexical analysis of English speech activity verbs*. in Springer series in language and communication, no. 8. Berlin Heidelberg: Springer, 1981.

[23] L. MacKenzie, J. A. Ibbotson, C. G. L. Cao, and A. J. Lomax, 'Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment', *Minimally Invasive Therapy & Allied Technologies*, vol. 10, no. 3, pp. 121-127, Jan. 2001, doi: 10.1080/136457001753192222.

[24] F. Lalys and P. Jannin, 'Surgical process modelling: a review', *Int J Comput Assist Radiol Surg*, vol. 9, no. 3, pp. 495-511, May 2014, doi: 10.1007/s11548-013-0940-5.

[25] T. Neumuth, P. Jannin, G. Strauss, J. Meixensberger, and O. Burgert, 'Validation of Knowledge Acquisition for Surgical Process Models', *Journal of the American Medical Informatics Association*, vol. 16, no. 1, pp. 72-80, Jan. 2009, doi: 10.1197/jamia.M2748.

[26] B. Gibaud et al., 'Toward a standard ontology of surgical process models', *Int J Comput Assist Radiol Surg*, vol. 13, no. 9, pp. 1397-1408, Sep. 2018, doi: 10.1007/s11548-018-1824-5.

[27] N. Nguyen, J. O. Elliott, W. D. Watson, and E. Dominguez, 'Simulation Improves Nontechnical Skills Performance of Residents During the Perioperative and Intraoperative Phases of Surgery', *J Surg Educ*, vol. 72, no. 5, pp. 957-963, 2015, doi: 10.1016/j.jsurg.2015.03.005.

[28] H. Truong et al., 'Operating room team training using simulation: Hope or hype?', *Am J Surg*, vol. 222, no. 6, pp. 1146-1153, Dec. 2021, doi: 10.1016/j.amjsurg.2021.01.044.
[29] D. Michelet et al., 'Benefits of Screen-Based Postpartum Hemorrhage Simulation on Nontechnical Skills Training: A Randomized Simulation Study', *Simul Healthc*, vol. 14, no. 6, pp. 391-397, Dec. 2019, doi: 10.1097/SIH.0000000000000395.

[30] M.-S. Bracq, E. Michinov, M. L. Duff, B. Arnaldi, V. Gouranton, and P. Jannin, "Doctor, please": Educating Nurses to Speak Up With Interactive Digital Simulation Tablets', *Clinical Simulation in Nursing*, vol. 54, pp. 97-104, May 2021, doi: 10.1016/j.ecns.2021.01.007.

[31] J. Barré et al., 'Virtual Reality Single-Port Sleeve Gastrectomy Training Decreases Physical and Mental Workload in Novice Surgeons: An Exploratory Study', *Obes Surg*, vol. 29, no. 4, pp. 1309-1316, Apr. 2019, doi: 10.1007/s11695-018-03680-9.

[32] M. Gholinejad, A. J Loeve, and J. Dankelman, 'Surgical process modelling strategies: which method to choose for determining workflow?', *Minim Invasive Ther Allied Technol*, vol. 28, no. 2, pp. 91-104, Apr. 2019, doi: 10.1080/13645706.2019.1591457.

[33] S. Yule, R. Flin, N. Maran, D. Rowley, G. Youngson, and S. Paterson-Brown, 'Surgeons' Non-technical Skills in the Operating Room: Reliability Testing of the NOTSS Behavior Rating System', *World j. surg.*, vol. 32, no. 4, pp. 548-556, Apr. 2008, doi: 10.1007/s00268-007-9320-z.

[34] E. R. Robertson et al., 'Oxford NOTECHS II: a modified theatre team non-technical skills scoring system', *PLoS One*, vol. 9, no. 3, p. e90320, 2014, doi: 10.1371/journal.pone.0090320.

[35] L. Mitchell and R. Flin, 'Non-technical skills of the operating theatre scrub nurse: literature review', *J Adv Nurs*, vol. 63, no. 1, pp. 15-24, Jul. 2008, doi: 10.1111/j.1365-2648.2008.04695.x.

[36] G. Fletcher, R. Flin, P. McGeorge, R. Glavin, N. Maran, and R. Patey, 'Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system', *Br J Anaesth*, vol. 90, no. 5, pp. 580-588, May 2003, doi: 10.1093/bja/aeg112.

[37] L. Hull, S. Arora, E. Kassab, R. Kneebone, and N. Sevdalis, 'Observational teamwork assessment for surgery: content validation and tool refinement', *J Am Coll Surg*, vol. 212, no. 2, pp. 234-243.e1-5, Feb. 2011, doi: 10.1016/j.jamcollsurg.2010.11.001.

[38] J. O. Ryan, A. Summerville, M. Mateas, and N. Wardrip-Fruin, 'Toward characters who observe, tell, misremember, and lie.', presented at the Artificial Intelligence and Interactive Digital Entertainment Conference, 2015.

[39] R. Evans and E. Short, 'Versu-A Simulationist Storytelling System', *IEEE Trans. Comput. Intell. AI Games*, vol. 6, no. 2, pp. 113-130, Jun. 2014, doi: 10.1109/TCIAIG.2013.2287297.

[40] M. Si, S. C. Marsella, and D. V. Pynadath, 'Thespian: using multi-agent fitting to craft interactive drama', in *Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems*, The Netherlands: ACM, Jul. 2005, pp. 21-28. doi: 10.1145/1082473.1082477.

[41] J. McCoy, M. Treanor, B. Samuel, M. Mateas, and N. Wardrip-Fruin, 'Prom Week: social physics as gameplay', in *Proceedings of the 6th International Conference on Foundations of Digital Games*, Bordeaux France: ACM, Jun. 2011, pp. 319-321. doi: 10.1145/2159365.2159425.

[42] S. Azad and C. Martens, 'Lyra: Simulating Believable Opinionated Virtual Characters', *AIIDE*, vol. 15, no. 1, pp. 108-115, Oct. 2019, doi: 10.1609/aiide.v15i1.5232.

[43] K. Tan and A. Mitchell, 'Dramatic Narrative Logics: Integrating Drama into Storygames with Operational Logics', in *Interactive Storytelling*, vol. 12497, A.-G. Bosser, D. E. Millard, and C. Hargood, Eds., Cham: Springer International Publishing, 2020, pp. 190-202. doi: 10.1007/978-3-030-62516-0_17.

[44] R. Damiano, V. Lombardo, and A. Pizzo, 'The ontology of drama', *AO*, vol. 14, no. 1, pp. 79-118, Feb. 2019, doi: 10.3233/AO-190204.