# Blocking notation: a tool for annotating and directing theater

Rémi Ronfard, Camelia Guerraoui

<figure><img src="image_1.png" /></figure>

## To cite this version:

Rémi Ronfard, Camelia Guerraoui. Blocking notation: a tool for annotating and directing theater. Journées d'Informatique Théâtrale, ENSATT, Oct 2022, Lyon, France. hal-04130264

HAL Id: hal-04130264

https://inria.hal.science/hal-04130264

Submitted on 15 Jun 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# BLOCKING NOTATION
## A TOOL FOR ANNOTATING AND DIRECTING THEATER

Camélia Guerraoui

INSA Lyon, France

Rémi Ronfard

Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France

## ABSTRACT

We introduce the *Blocking Notation* as a minimal formal system to describe a complete theater performance, and we make recommendations for a user-friendly interface dedicated to notating performances based on video recordings. This interface should give stage directors the opportunity to visualize the play at the earliest stages. It should also enable stage managers to easily take notes in order to make sure that every aspect of the production runs just as the director intended time after time. Preliminary experiments demonstrate the usefulness of the notation on an experimental data set of three short scenes in four different productions of the Marivaux play, "The Island of Slaves".

## 1. INTRODUCTION

Blocking notations are an overlooked yet important aspect of theater production, which are seldom conserved or published. They are useful to the theater crew during rehearsals. In case of a last-minute cast member replacement, the new performer can quickly rehearse understandies. In this paper, we argue that blocking notations can also be useful to researchers: they can be used to analyze staging and to identify specific patterns of a stage director. All in all, notations are simply another kind of record of the production.

Even if notations are very useful, not every play is systematically notated. Notating an entire live performance is indeed time-consuming, mostly due to the lack of usability regarding the chosen notation system. Most of the notation systems are not digital i.e. have to be written manually. However digital notation system limits writing and spelling mistakes by checking the input and can accelerate annotating by automatizing some notations. With a formal digital notation system, private shorthand or signs are not allowed, which guarantees the integrity and the clarity of notations.

Notations can be made during rehearsal but they have to be updated regularly as the staging evolves. If the writer wants to avoid these frequent changes, he will have to note the play after the last performance in order to have the very last version of the production. But in this case, annotations could only be used to analyze the play. Even if the notations are made after the last performance, annotating is still a hard task, as the writer has to focus on each performer while annotating at the same time. It is only

achievable if many writers are mobilized or if the play is interrupted.

The idea behind our *Blocking Notation* is on one hand to propose a formal manner to represent a play, and, on the other hand, to elaborate recommendations for a user-friendly interface dedicated to notate plays based on video recordings. The *Blocking Notation* system should allow stage directors during the creative process to visualize the play at its earliest stages. It also should allow stage managers to easily take notes in order to make sure that every aspect of the production runs just as the director intended time after time.

The requirement for our *Blocking Notation* system are the following:

1. **Formal**: It should give a formal representation of actions which happens during a play. This includes actors movements, dialogues, monologues, lighting, sound and sets cues.

2. **Intuitive**: It should be understandable by a human. A new actor should be able to easily read the *Blocking Notations*.

3. **Non ambiguous and precise**: A new actor should be able to know exactly what he should do on stage. The *Blocking Notation* system should also be understandable and read by a machine, in order to correct any writing mistakes and auto-complete some notations.

4. **Compact**: It should be shorter than a real script. It is tempting when staring out to record every single small move that every performer makes, but it is not necessary.

5. **Non-intrusive**: It is a tool to help stage directors and managers to keep tracks of actions performed on stage. So the *Blocking Notation* should not restrain the creative process.

After reviewing the previous work related to the *Blocking Notation* project, we present the *Blocking Notation Ontology* and secondly the *Blocking Notation Tools*. Then we evaluate the *Blocking Notation* system by annotating different stagings of the first scene of *The Island of Slaves* written by Marivaux (Figure 1).

We decided to leave apart **subtexts** (i.e. the underlying emotion, thoughts, and meanings underneath what is said by characters in a play), because they are very hard to objectively determine. Indeed **subtexts** can be various and depend on the notation writer's analyzes of the staging.

In the future, the *Blocking Notation* project could be used to generate 3D animations of plays as in [17]. This generation could also be used to recreate previous stagings
<figure><img src="image_3.png" /><figcaption>**Figure 1:** Recorded performances of “The island of slaves” by directors Jean-Luc Lagarce (1994), Giorgio Strehler (1995), Irina Brook (2005), Francisco Suarez (2019), Gaële Boghossian (2010), Gerold Schumann (2016), Jacques Vincey (2019), Sébastien Biessy. (2015), Gilberte Tsai (2012), Gilles Droulez (2012), Axel Joucla (2009) and Jean-Thomas Bouillaguet (2016). All frames show the same line from scene 1 in the original play : “We are alone on the island of slaves”.</figcaption></figure>

of plays which were not recorded [12, 11]. It could also
be a way of producing immersive and/or interactive plays
in mixed reality [18].

## 2. RELATED WORK

In the tradition of English and American theater, blocking is the description of stage positions and movements of the actors during a performance [7]. Blocking notations and related technical information such as props' movement, light and sound cues, are generally written in a prompt book, which is a dynamic document, as it is changed during and after each rehearsal.

The stage manager has the responsibility to keep up-to-date the prompt book. It is a crucial responsibility, because the prompt book is considered as a complete guide and record of the production. The notations need to be clear: a new stage manager should be able to easily read and understand the prompt book.

This document is often a paper book with on the left side script and on the right side a blank page for notations. It is critical to have enough place to write the notations. Peter Maccoy [9] even recommends to have a

hard-covered ring binder to make the prompt copy last longer. The prompt book is useful because it is based on hand writing, which is fast and efficient, and very modular (each stage manager writes the prompt book in his own way). However stage managers can be tempted to use private shorthand or signs, which will lead to confusion.

As it is a paper book, it can be easily damaged or torn. Moreover copies of the book are rarely made, as it changes frequently. Digital prompt copies help to back up the notations and to send them to other crew members. Commercial software tools such as Stage Write <fnref n="1" /> provide facilities for blocking, adding notes, cues to a digital script, and drawing "spacing charts" with floor plan views of the stage, but they are not (yet) based on a commonly agreed semantics. A common notation shared by stage managers would be a valuable contribution to the state of the art.

There are many formal notation systems to annotate an artistic live performance specifically in the field of dance. Dance notation systems such as Benesh notation [2] and Labanotation [6] use abstract symbols to define precisely the direction, the level, the involved part of the body, the

<footnote n="1">¹https://www.stagewritesoftware.com</footnote>
duration and the dynamic quality of the movement. However, writing down the notation by observing the dance movement is a difficult task which requires patience and skill. While dance notation can be used in theory to notate dramatic theater performances, it provides limited support for the complex interplay between verbal and non-verbal interaction which is central to theater expressions. As a result, it has not been widely adopted by theater professionals.

Specific theater notation systems have been designed for the creation of multimedia art work involving technology [10, 4]. In the future, we expect such notation systems to become more prevalent to support mixed reality theater production [18]. In contrast, our paper deals predominantly with traditional theater notation.

Our Blocking Notation system is more specifically related to the Schneider Notation System [15]. This system introduces symbols, abbreviations and other conventions for noting the blocking of stage movement. This system is easier to learn and use than dance notation. The purpose of the Schneider Notation system is to make notating stage movement quick, and accurate. However some actions defined in this system are not perfectly disjoint and this can lead to ambiguous notations.

Closer to our work, DRAMA [5] and DRAMMAR [3] are two attempts to digitally annotate play scripts with semantic annotation of stage directions including entries, exits, movements and positions of actors; characters, costumes, morphology and props; non-verbal actions, language actions, emotions, voice qualities and types of interactions. The DRAMA and DRAMMAR ontologies are quite accurate and exhaustive, but they are based on play scripts and not video recordings. Notations based on play scripts reflects more the author's point of view rather than the director's staging. In contrast, our Blocking Notation attempts to notate what happens on stage during the performance, from the point of view of the audience. To the best of our knowledge, there is no existing system of notation that addresses this particular problem.

## 3. METHODS AND TOOLS

The purpose of the *Blocking Notation* project is to develop a formal representation of a theater play. This representation relies on an ontology to describe theater performances, and a graphical representation of the play.

### 3.1. Blocking Notation Ontology

The *Blocking Notation* ontology includes actors identified by their roles; set elements, costumes, props; technical elements performed during the play (such as light, video, sound, etc.) and actions performed on stage during a play.

We organize actions into five categories:

1. **Move:** Actions which involve the movement of an actor or a group of actors to a specific location on stage.

2. **Say:** Actions which involve the use of an actor's voice (or of a group of actors).

3. **Touch:** Actions which involve the contact of an actor or a group of actors with an item present on stage. The item must be close to the actor or group of actors otherwise a Move Action to get closer must be performed before touching it.

4. ** Gesture:** Bodily actions of an actor or a group of actors which are neither a movement, nor an interaction with an item.

5. **Look:** Actions which involve the eye movement of an actor or a group of actors.

Two actions from the same category cannot be performed at the same time: an actor cannot simultaneously walk (Category Move) and run (Category Move) whereas he can at the same time walk and talk (Category Say). This property is mostly guaranteed for physiological reasons. However it could be possible to do two Touch Actions at the same time as an actor has two arms. But as it doesn't frequently happen and for integrity reasons, we chose to restrict one Touch Action at a time per actor.

At the time of writing, our ontology counts twenty nine different actions which are illustrated from Figures 4 to 8.

The actions "LookAt" and "Pose" are a bit particular, because actors continually looks around and change postures. In order to not overburden the notations, we only notate gaze directions and postures when they are charged with a dramatic significance. In many cases, gaze directions and postures can be easily recreated following a common set of rules described by Christine Talbot and G. Michael Youngblood[16] : an actor looks (or changes his posture according to) the direction where he walks, who he is talking to, where he is pointing at. They do not need to be notated.

A location can either be a general location (i.e. a global position on stage), or a relative location (i.e a position defined by another actor, a costume, a prop or a set element). We decided to divide the stage into nine blocks. If a stage object or an actor is located on a border between two blocks, we arbitrarily chose to write the block which is the closest to the audience and the most in the center. To describe an object's or an actor's position with a relative location, prepositions of place such as "Left To" and "Right To" can be used. These orientations are marked with the audience's point of view (and not from the actor's). But all nine locations on stage are from the perspective of the performer ("Down Left", "Up Right", etc).

### 3.2. Blocking Notation Template

To write our *Blocking Notations*, we created an Excel template with eight different sheets: Actor, Costume, Prop, Set, Stagecraft, Step by Step, Statistics, and Vocabulary.

We used the first five sheets to describe the different stage elements (Description phase) and how they appear at the beginning of the scene (Initialization phase). In the
<table><thead><tr><th rowspan="3">TIME</th><th colspan="10">Iphicrate</th></tr><tr><th colspan="3">MOVE</th><th colspan="3">SAY</th><th colspan="3">TOUCH</th><th colspan="3">GESTURE</th><th colspan="3">LOOK</th></tr><tr><th>State</th><th>Type</th><th>Direction</th><th>State</th><th>Type</th><th>Listener</th><th>State</th><th>Type</th><th>Object</th><th>To/With</th><th>State</th><th>Type</th><th>Direction</th><th>State</th><th>Type</th><th>Direction</th></tr></thead><tbody><tr><td>0min 0s</td><td>Success</td><td>Stand</td><td>Floor</td><td></td><td></td><td></td><td>Start</td><td>Use</td><td>Paper</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>0min 2s</td><td>Start</td><td>Walk</td><td>Down Center</td><td></td><td></td><td></td><td>Success</td><td>Use</td><td>Paper</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>0min 4s</td><td>Success</td><td>Walk</td><td>Down Center</td><td>Start</td><td>Talk</td><td>Arlequin</td><td></td><td></td><td></td><td></td><td>Success</td><td>Gesticulate</td><td></td><td>Start</td><td>Look At</td><td>Audience</td></tr><tr><td>0min 6s</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Success</td><td>Gesticulate</td><td></td><td></td><td></td><td></td></tr><tr><td>0min 8s</td><td></td><td></td><td></td><td>Suspend</td><td>Talk</td><td>Arlequin</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>0min 10s</td><td></td><td></td><td></td><td>Start</td><td>Shout</td><td>Arlequin</td><td>Success</td><td>Take</td><td>Skull</td><td>Arlequin</td><td></td><td></td><td></td><td>Success</td><td>Look At</td><td>Audience</td></tr><tr><td>0min 12s</td><td></td><td></td><td></td><td>Success</td><td>Shout</td><td>Arlequin</td><td>Start</td><td>Use</td><td>Skull</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>0min 14s</td><td></td><td></td><td></td><td>Resume</td><td>Talk</td><td>Arlequin</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>0min 16s</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

**Figure 2:** Interface to visualize the staging of a play using a floor plan view of the stage. Blocking actions can be used to track the positions and orientations of actors and objects on the stage. The visualization can be used to check that the description is consistent with the video recording at any given time during the performance and to complete the description by adding all necessary actions.

Vocabulary sheet, we defined the words that our ontology accepts. For example, we can find every allowed actions, their category and their possible states. We added Excel formulas in the other sheets to check if the correct words were used.

The most important sheet is the Step by Step sheet which contains every *Blocking Notation* (Figure 2). This sheet is a 2D chart: the horizontal axis displays all actors and their actions and the vertical axis represents the time reference (2 seconds per line). We use a total of sixteen columns per actor for notating their actions (three columns each for Move, Say, Gesture and Look actions and four columns for Touch actions).

Each category has one column to write the action name (for example "Walk"), one column to write the action state, and several columns to write the action parameters (Figures 4 to 8). If several parameters are needed, they should be defined as a group i.e. every parameter are written in the same cell and are separated with a semicolon without any extra space.

For our proof of concept, we decided to have a fixed time reference: each row represent two seconds of the video. But in the future, we should have an abstract time reference. We chose to have 2 seconds per notations, because we estimate that a discrete action lasts about 2 seconds. If a continuous action happens very fast (i.e. in 2 seconds or less), only the notation to indicate the end of the action should be written.

The Statistics sheet gives a simple overview of our notations. We can see the number of actors, costumes, props, set and stagecraft elements, performed actions per category and per actor, and written notations. With this sheet, we can estimate if the *Blocking Notation* system requires a lot of notations or not. We can also compare different staging and to see which actor is more active during a scene (cf Figure 9).

### 3.3. Blocking Notation Visualization.

To visualize in a better way our notation, we implemented an interface in Python. In this interface (Figure 3) we can

see on the left side the video and on the right side a sketch representing the stage with a bird's-eye view. Actors are shown with icons, whereas objects are represented with rectangles. When we launch this interface, we see the actors moving on the sketch and on the video. That way we could efficiently compare our notations and the real video.

<figure><img src="image_4.png" /><figcaption>**Figure 3:** Visualization tool used to monitor and validate the annotation.</figcaption></figure>

As it is only a proof of concept, the interface only shows the actors movement (i.e. every Move Action). We leave for future work the implementation of other action's categories. This visualization was useful for reviewing and updating our notation, by comparing our result with the real video.

It is important when annotating to visualize every protagonist at the same time. However it is not necessarily to see the equivalent of our 16 columns in our Excel file for every actors all the time. The visualization used Figure 3 is a good approach to notate a theater play with many actors.

## 4. EXPERIMENTAL RESULTS

To see if our model can properly represent a theater play, we annotated video recordings of the play *The island of slaves* by Marivaux staged by different directors in different languages. More specifically, we annotated the first
scene of the play, a dialogue between Arlequin and Iphicrate, in a French version by Irina Brook and a Spanish version by Francisco Suárez.

The French version by Irina Brook is about 5 minute long whereas the Spanish version by Francisco Suarez is 8 minute long. To annotate a play, we first tried to notate the two actors at the same time, but moving from one actor to the other was complicated and slowed down the whole process. Then we tried to focus on one actor and write all his notations; then repeat the same process for the second actor. As a result, the video needs to be replayed for every actor. In our experience, this process is a bit quicker and reduces the risk of notation errors, because we are allowed to focus on one actor at a time. We needed one hour per actor per minute of video to notate. In comparison, Benesh notator Dany Leveque needs around 8 to 12 hours to notate one minute of a dance video [8].

Even if notating with *Blocking Notations* is quicker than usual notation systems, it is still time-consuming. Moving between the video player and the Excel file slows down the annotation process. In future work, to reduce the burden of annotation, we would like to provide a graphic interface for blocking integrated with a video player as in KinoAi [14, 13] or Advene [1].

## 5. CONCLUSION

The *Blocking Notation* is a concise and clear notation system for theater performance, with a minimal vocabulary of 29 actions. To test the relevance of our ontology, we notated two versions of the same scene and created an interactive visualization for them. The template and the ontology helped us reduce the time needed to annotate. However even with our template, annotating requires concentration and patience. In future work, we plan to improve our visualization interface and transform it into a notation interface, and we believe this will further accelerate the notation process.

## 6. REFERENCES

[1] Olivier Aubert and Yannick Prié. Advene: active reading through hypervideo. In *Proceedings of ACM Hypertext*, pages 235–244, 2005.

[2] Rudolf Benesh and Joan Benesh. *Reading Dance: The Birth of Choreology*. McGraw-Hill, 1983.

[3] Rossana Damiano, Vincenzo Lombardo, and Antonio Pizzo. The ontology of drama. *Applied Ontology*, 14(1):79–118, 2019.

[4] Robert Faguy. Modéliser et numériser l'écriture d'une production scénique complexe. In *Journées d'Informatique Théâtrale, Performance Lab, Univ. Grenoble Alpes*, Feb 2020.

[5] Sanda Golopentia, Monique Martinez Thomas, and Matthieu Pouget. Les étiquettes du personnage de

théâtre. In *La notation informatique du personnage théâtral*. Lansman, 2010.

[6] Ann Hutchinson Guest. *Labanotation*. Routledge Chapman & Hall, 1977.

[7] Laurie Kincman. *The Stage Manager's Toolkit*. Focal Press Toolkit Series, Routledge, 2016.

[8] Dany Lévêque. *Angelin Preljocaj, de la création à la mémoire de la danse*. Les belles Lettres/Archimbaud, 2011.

[9] Peter Maccoy. *Essentials of Stage Management*. A & C Black, 2004.

[10] Marc Philippe Parent and Robert Faguy. Préfiguration d'un logiciel de dramaturgie scénique plurielle. In *La notation du travail théâtral du manuscrit au numérique*, pages 147–167. Lansman, Oct 2016.

[11] Jennifer Roberts-Smith and al. Visualizing theatrical text: From watching the script to the simulated environment for theatre (set). *Digital Humanities. Quaterly*, 7, 2013.

[12] Jennifer Roberts-Smith, Sandra Gabriele, Stan Ruecker, and Stéfan Sinclair. The text and the line of action: Re-conceiving watching the script. *Scholarly and Research Communication*, 3(3), 2012.

[13] Rémi Ronfard and Rémi Colin de Verdière. OpenKinoAI: A Framework for Intelligent Cinematography and Editing of Live Performances. *Leonardo*, 55(4):373–377, August 2022.

[14] Rémi Ronfard and Julie Valero. Kinoai et le carnet audiovisuel : une solution plurielle pour l'étude des répétitions. *European Journal of Theatre and Performance*, pages 334–375, 2020.

[15] Doris Schneider. *The Art and Craft of Stage Management*. Wadsworth Publishing, 1996.

[16] Christine Talbot and G. Michael Youngblood. Shakespearean spatial rules. In *Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems*, page 587–594, 2013.

[17] Christine Talbot and G. Michael Youngblood. Virtual environment positioning utilizing play-script spatiotemporal reasoning. *IEEE Transactions on Games*, 12(3):225–235, 2020.

[18] Joris Weijdom. *Mixed Reality and the Theatre of the Future: Fresh Perspectives on Arts and New Technologies*. IETM - International Network for Contemporary Performing Arts, Brussels, 2017.
<figure><img src="image_5.png" /><figcaption>Figure 4: Move Actions</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 5: Say Actions</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6: Look Actions</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 7: Gesture Actions</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 8: Touch Actions</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>(a) French Version by Irina Brook</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(b) Spanish Version by Francisco Suárez</figcaption></figure>

<caption>**Figure 9:** Comparison between two versions of *The Island of slaves's* opening scene</caption>