# Strategies for the Reconciliation of Artistic Intent and Technical Constraints in Mixed Reality Performances

Pierrick Uro, Florent Berthaut, Thomas Pietrzak, Laurent Grisoni, Marcelo Wanderley

► To cite this version:

Pierrick Uro, Florent Berthaut, Thomas Pietrzak, Laurent Grisoni, Marcelo Wanderley. Strategies for the Reconciliation of Artistic Intent and Technical Constraints in Mixed Reality Performances. DIS 2025 - ACM Designing Interactive Systems Conference, ACM, Jul 2025, Funchal (Madeira Island), Portugal. 10.1145/3715336.3735738 . hal-05126258

HAL Id: hal-05126258

https://inria.hal.science/hal-05126258v1

Submitted on 23 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Strategies for the Reconciliation of Artistic Intent and Technical Constraints in Mixed Reality Performances

Pierrick Uro

Florent Berthaut

Thomas Pietrzak

Laurent Grisoni

pierrick.uro@univ-lille.fr

florent.berthaut@univ-lille.fr

thomas.pietrzak@univ-lille.fr

laurent.grisoni@univ-lille.fr

Université de Lille

France

Marcelo M. Wanderley

marcelo.wanderley@mail.mcgill.ca

McGill University

Canada

## Abstract

As immersive technologies advance, Mixed Reality Performances (MRP) increasingly integrate them but often face technical challenges that must balance artistic vision and practical constraints. These constraints sometimes lead to unavoidable limitations. The diversity of technologies and artistic goals in MRP prevents a one-size-fits-all solution to such dilemmas. This paper presents a model unifying strategies for addressing compromises between artistic intent and technological feasibility. Using reflexive thematic analysis of a performance-led case study, interviews, and independent case studies, we identify recurring strategies in Mixed Reality experiences with varying constraints. These strategies fall on an axis based on the audience's awareness of limitations and are categorized into five approaches: Avoid, Disguise, Tolerate, Integrate, or Leverage. We argue that this framework helps designers better navigate the limitations inherent in creating MRPs, offering practical pathways to align technological capabilities with creative objectives.

## CCS Concepts

* Human-centered computing → Mixed / augmented reality; Collaborative interaction; * Applied computing → Performing arts; Media arts.

## Keywords

Interactive Performance, Mixed Reality, Mixed Reality Performance, Multi-user

## ACM Reference Format:

Pierrick Uro, Florent Berthaut, Thomas Pietrzak, Laurent Grisoni, and Marcelo M. Wanderley. 2025. Strategies for the Reconciliation of Artistic Intent and Technical Constraints in Mixed Reality Performances. In *Designing Interactive Systems Conference (DIS '25), July 5-9, 2025, Funchal, Portugal*. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3715336.3735738

## 1 Introduction

As an ever-increasing number of artistic productions employ immersive displays in Mixed Reality (MR) or Virtual Reality (VR), they often run into the issue of compromising between what the technology enables and what the production team envisions. When such a satisfying compromise is found, it can rarely be applied to other productions, even with similar technological and artistic contexts. For instance, the choice of having a large audience sit down for a narrative experience can be made to avoid tracking or collision issues [16], and reduce the amount of data needed to represent every participant as avatars [21]. This approach would however not fit in other contexts requiring more freedom for the participants. Another example is how a low-quality black and white passthrough camera can still appropriately be used to immerse a spectator in Augmented Reality (AR) without breaking the flow of the performance [22]. If the limited quality can be made to fit within the aesthetic context of the piece, it will be accepted by the audience and can even become more of a feature than a limitation. Yet again, not just any production can fit such a sequence, and some may be forced to find an alternative camera. This is why the designers must answer the limitations while respecting both the technical and artistic constraints of the production. This means that a similar issue might find very different answers from project to project.

Still, designers need to be able to draw from the productions of their peers for their own practice despite the diversity in contexts. Benford et al.'s trajectory framework has been used consistently in the design and analysis of Mixed Reality Performances (MRP) [6]. In particular, the framework underlines transitions as potentially immersion-breaking moments of the continuous experience of a participant when a trajectory shifts from one state to an other in space, time, roles, and interfaces. While transitions may help understand a subset of limitations, some challenges step outside of this concept, such as a restricted field of view [20] or the above example of a low quality passthrough camera [22]. Promising strategies were also proposed by Benford et al. to face the issue of uncertainty [3], however, they are specific to the experience and may not be applicable to MRPs at large.

In this paper, we address the difficulty in learning from productions with various contexts by looking for the overarching strategies
they follow to re-conciliate constraints of immersive technologies and artistic intents. This is accomplished through three iterations of a reflexive thematic analysis focusing on a performance-led case study, two interviews, and independent case studies. This analysis reveals five strategies, which constitute a design space organized according to how much of the limitation is revealed to the audience. We show how this design space allows designers to consider alternative strategies when facing conflicts between limitations and ambitions during the creation of MRPs with varying constraints.

## 2 Related Work

This section defines the concept of Mixed Reality Performance (MRP) in order to delimit the scope of the study. We highlight the diversity of the productions that can be included in this scope. Then, we present existing frameworks that provide guidelines for the design of productions belonging to MRPs or adjacent contexts.

### 2.1 Mixed Reality Performances

The term Mixed Reality Performance is defined by Benford and Giannachi as a theatrical genre that mixes "real and virtual worlds", and combines "live performance of actors or participants with interactive digital media" [5]. Numerous examples of MRPs stem from the long-term collaboration of the University of Nottingham's Mixed Reality Lab<fnref n="1" /> and the artist collective Blast Theory in the form of *Desert Rain, Can You See Me Now?, Day of the Figurines*, or *Ulrike and Eamon Compliant*, dating from 1997 to 2009, though some of these experiences have also been qualified as "extended theatrical performance" [4]. These productions are extremely varied in their form, ranging from a 45-minute performance with projections on a water screen in *Desert Rain*, to a mobile phone test-messaging adventure game in *Day of the Figurines*. It is important to note that Benford and Giannachi's definition of MRP does not restrict its scope to headset-centric MR and remains flexible as to the involvement or not of actors as part of the performance.

A different group of researchers, New York University's Future Reality Lab<fnref n="2" />, was also able to develop a number of narrative experiences, in this case more centered around immersive technologies. Among them, *Holojam in Wonderland* features 2 actors and 4 spectators sharing a virtual environment as part of a narrative experience categorized as Immersive Mixed Reality Theater [17]. Deciding to focus on scale, they went on to produce *CAVE* with the *CAVRN* system to immerse up to 30 participants [21, 24]. Their later production, *Mary and the Monster*, intended for participants to share a narrative experience while immersed either in AR or VR [20]. Again here, even within the same group of creators focusing on immersion through headsets, the variety in technological and artistic contexts is great. The three productions used different headsets, and while the first involved actors and a small audience freely roaming in a shared space, the others featured no live actors, and large seated audiences.

A similar variety is to be expected with more independent productions. The diversity of their creative stakes can be quickly seen through how they were designated. For example, *Debussy3.0* is described as either an augmented performance, ballet, or show and

presents an interaction between stereoscopic 3D and the performance of two dancers [14]. With a similar screen-based immersion, Marner *et al.* presented *Half Real* as "*a live action interactive theater show employing spatial augmented reality*" [25]. The show immerses the actors into a projected virtual environment and allows its audience to select directions of the narrative via a voting system.

While some productions such as the previous two define themselves around the artistic discipline they are built around, some can be described with more technological details. This is the case of *Wake*, which is presented as an "*asymmetric, co-located, co-present, mixed reality social experience*". It lets a headset-wearing participant interact with the volumetric projection of a performer inside a virtual environment [19].

The productions described here range from a single participant to an audience occupying an entire theater room, with or without live performers, or using different means of immersion. Yet, they all arguably fall under the concept of MRP, and as such have the potential to draw lessons from each other.

### 2.2 Analyzing Mixed Reality Performances

A recurring framework in the conception and analysis of MRP is Benford *et al.'s* trajectories, which were derived from the projects created at the Mixed Reality Lab [4]. This analytical framework describes the experience of the participants through their journey across space, time, roles, and interfaces. One of its uses is the identification of *transitions*, key points in the experience where a participant changes their state in one of their trajectories. These moments are important since they are where the risk of breaking the immersion of the participant is highest. Transitions include the beginning and end of the experience, changes in roles and interfaces, shifts from physical to virtual worlds, or the presence of *seams*.

*Seams* are what Chalmers and Galani call the noticeable boundaries and differences in a system [13], which in Benford *et al.'s* framework could cause a problematic transition. This does not however mean that any seam must be eliminated, since Chalmers and Galani argue for a seamful design, a concept that aims to recognize the existence of these seams and to exploit them.

In one of the works that defined MRPs, *Can You See Me Now?*, Benford *et al.* were able to draw from this idea [3]. In the piece, GPS and WiFi signals proved to be unreliable and brought a level of uncertainty to the experience. They were able to identify the various states of connectivity and proposed five strategies for coping with them: Remove, Hide, Manage, Reveal, and Exploit. While these strategies may be informative for the design of other experiences, they have not yet been applied to contexts broader than the connectivity issues encountered in *Can You See Me Now?*.

In a case study of two MRPs, Rostami and McMillan [27] document challenges and strategies artists and performers encountered and adopted through video analysis. They regroup them under four categories: *Stage Managing VR Performances*, *Choreographing for Cables*, *Consistency*, and *Charging*, *Improvising Interventions*, and *Priming Participants*. While the detail provided by the video analysis of the incidents during the performances is valuable, such a method may not capture issues and answers that occurred earlier in the design process.

<footnote n="1">¹https://www.nottingham.ac.uk/research/groups/mixedrealitylab/</footnote>

<footnote n="2">²https://frl.nyu.edu/</footnote>
Weijdom built on top of Benford and Giannachi's definition of MRP to define a Performative Mixed Reality Experience (PMRE) [29], which draws more focus on the body becoming part of the aesthetic of the piece. This serves as a basis to study the design processes of two productions in order to propose a performative prototyping method, which uses a combination of bodystorming, Wizard of Oz, and puppetteering techniques along with the development of an interdisciplinary vocabulary as a way to achieve a common ground for designers of PMREs.

MRPs can also be considered through the adjacent scope of im-
mersive musical performances. In this regard, Zappi et al. proposed
dimensions to analyze the scenography of Immersive Virtual Mu-
sical Instruments (IVMI) [30]. Though this framework focuses on
IVMIs and may not fit all MRPs, it introduces dimensions that aim to
describe the family of IVMIs, which is also subject to vast diversity
in artistic and technological contexts.

The scope of MRP can also be broadened to non-performance
MR applications and the challenges faced by their designers. Eight
key challenges were identified by Ashtari et al. [2] for authoring
AR/VR applications along with some approaches taken by creators.
Similarly, Krauß et al. [23] revealed three key challenges for AR/VR
designers, and potential solutions for them. They also identify roles
taken by the designers and give guidelines for the design of author-
ing tools. These findings relate to a scope encompassing that of
MRPs, and as such may be applied by MRP designers, yet they might
miss challenges caused by the particular context of a performance,
or strategies afforded by it.

In this paper, we draw from these frameworks and attempt to
adapt their models to the specific context of MRPs.

3 Iterative Process

The goal of the process described here is to identify overarching
elements in the ways designers of MRP approach critical challenges
they encounter in their productions. In this context, we consider as
challenges situations where the available immersive technology is
not capable of accomplishing a desired artistic outcome, resulting
in a compromise between what is wanted and what is possible.
Designers thus need to come up with an answer, which is how they
decide to approach and work around the compromise imposed by
the challenge. Since the challenge relies on the technology being
used, and the desired result, so does the answer given.

We aim to model the *answers* brought by various designers into *strategies*, which describe potential approaches regardless of their artistic or technological context.

This will be undertaken through a three-phase reflexive thematic
analysis attempting to associate the approaches in various MRP
productions as described by their designers. The phases consist of a
performance-led case study, two interviews, and independent case
studies (see Figure 1). During each step, we perform an analysis
following Byrne's example [11] of Braun and Clarke's reflexive
thematic analysis [8, 9]. As such, underlying theoretical assump-
tions within this study should be addressed. Since this analysis is
of qualitative essence, it follows constructionist epistemological
considerations, implying that both recurrence and meaningfulness
in the data are taken into account in the generation of themes. The

approach adopted leans on both critical and experiential orienta-
tions, since the identification of challenges relies on a description
of the experience of the subjects, while the identification of strate-
gies is related to the finding of an underlying structure within
the testimonies. Coding was accomplished both in a semantic and
latent manner, as the data in the entire corpus already comes in
analytical form as communicated by experts, which can be inter-
preted explicitly. Still, as the goal is to find an overarching model
in the approaches of the subjects, latent coding also found value
where deeper meaning could be constructed by the researcher's
interpretation. The first phase adopts a predominantly inductive
approach, while the second and third reflect previous iterations and
thus present a stronger level of deductive process.

The first phase of the analysis focuses on the experience of the first author as the Lead VR Developer on the immersive theater piece *Animate* through the analysis of a presentation that was given shortly after the premiere of the show. This helps gaining an initial understanding of potential answers, which is used in the next phase, and reveals bootstrap strategies.

The second phase consists in the analysis of two interviews with
designers of MRPs, in which they were asked about the various
challenges they encountered and how they answered them. The
second interviewee is the second author of this paper, and did not
participate in the preparation or analysis of the second phase. The
insight gained during the previous phase allows to find challenges
that were not explicitly mentioned in the documentation of the
pieces. The analysis of the interviews also draws from the initial
case study when devising its themes, resulting in refined strategies.

Finally, several independent case studies are accomplished by
completing an analysis of published material extensively detail-
ing the design process of various recent MRPs. This serves as a
validation step to the previous results by challenging them with
independent perspectives.

The perspectives that were selected were chosen with the goal to
represent the diversity of what can be included in a MRP, as can be
seen through their representation in Figure 2. This includes various
immersive technologies, various sizes of audience, and either live
or pre-recorded performances. The focus was also put on picking
relatively recent projects in order to keep a similar technological
landscape throughout the various productions.

The involvement of two of the authors in two of the studied
projects grants a familiar point of view on the decisions that were
taken throughout their respective design. That same involvement
is also what motivated this study. Moreover, the auto-ethnographic

<figure><img src="image_2.png" /><figcaption>Figure 1: Method used for the iterative process.</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Figure 2: Top-view representations of the mixed-reality setups for parts of the studied performances: P correspond to performers, A to audience members. Solid lines correspond to physical persons/objects while dotted lines represent virtual avatars/objects. Audience members only see gray elements and elements of the same color as themselves. Audience members in dotted lines with headsets do not see the physical space, while those in solid lines see both virtual and physical spaces.</figcaption></figure>

aspect of the initial case study, and the potential bias of the inter-
views being led by the same author were addressed through the
cross-validation step of the independent case studies, which bring
a larger variety of points of view on the implementation of MRPs.

Throughout each step of this process, challenges that could be
identified are listed in Table 1 in order to keep track of the variety
of challenges found within the corpus. For each project, one such
challenge will be detailed along with the approach taken by the
authors in response.

4 Performance-Led Case Study

The first step of the iteration is a case study of *Animate* through the
experience of the first author on its production as part of a team of
artists and developers with a research-creation approach.

4.1 Methodology

The data set for this iteration of the analysis is a fifteen-minute
speech that was given by the first author in October 2022, a month
after the show premiered. The presentation was given at a workshop
focused on audio and MR. It was structured around the challenges
that were encountered, and how the resulting imperfections were
addressed to achieve a working show. For this reason, analyzing
it under the larger scope of strategies can reveal underlying pat-
terns that were not obvious at the time of the presentation. This
means that in this analysis, while the approach is generally induc-
tive, attention is put towards identifying strategies, which brings
deductive components.

4.2 Animate

Animate has been described as an Extended Reality-based theater
work and premiered in 2022³. In it, an audience of up to 12 people
wore headsets and were plunged in VR while holding a rope. Actors
pull on the rope to guide the spectators while the story unfolds,

with the audience unable to see them. Eventually, the physical
environment is revealed through the black and white cameras on
the headset and the audience can watch and interact with floating
virtual rocks as they fly around the room.

A key challenge in *Animate* was the sharing of several hundred
flying rocks. Each rock followed a physical model and followed
a predefined choreography, while the audience could push them.
Such a large amount of independent objects proved to be too much
to be shared over the network, and an approach was taken to let
each headset have their own instance of the scene. Each participant
thus saw their own rocks, and they ran into the risk of noticing
their neighbor interacting with rocks they could not see. The an-
swer to this limitation was to highly reduce the amount of impact a
participant could have on a rock. The interaction went from being
collision based, to relying on a force field around each hand that
could slightly push the rocks away. This ensured that the macro-
scopic behavior of the rocks would remain close to the scripted
choreography, and that the audience would thus be unlikely to no-
tice the difference. This choice was also made possible by a parallel
decision to let go of live spatial audio generated from the collisions
of the rocks as it was deemed too unreliable. Instead, a pre-mixed
audio was created in order to guarantee the effect desired by the
designers.

4.3 Resulting Themes

The analysis accounts for two themes as seen in Fig. 3 : Strategic Management of Limitations and Complementary Considerations.

*Strategic Management of Limitations*. The first theme reveals three potential approaches in its sub-themes, starting with the use the audience's awareness. The first entails that designers keep in mind what their audience will notice or not, in order to make the limitations of the piece less discernible and make use of channels they do not have access to for orchestration. The challenge of sharing a large amount of objects is answered this way by altering

<footnote n="3">³https://montreal.mutek.org/en/artists/animate</footnote>
<table><caption>**Table 1: Specific challenges that were identified in each project**</caption><thead><tr><th>Projects</th><th>Identified Challenges</th></tr></thead><tbody><tr><td rowspan="7">Animate</td><td>Low reproduction fidelity</td></tr><tr><td>No occlusion from physical elements</td></tr><tr><td>Occasional loss of alignment</td></tr><tr><td>Complexity of large scale shared interaction</td></tr><tr><td>Complexity of shared audio</td></tr><tr><td>Un-immersed performers</td></tr><tr><td>Encumbrance of emotion sensing equipment</td></tr><tr><td>Discreet haptic feedback</td></tr><tr><td rowspan="4">Terres Rares</td><td>Partially immersed performers</td></tr><tr><td>Restricted effective area for display/capture</td></tr><tr><td>Respective occlusion of performers</td></tr><tr><td>Capture-display latency</td></tr><tr><td rowspan="3">Debussy3.0</td><td>Inaccurate intersection with a prop</td></tr><tr><td>Inaccurate mechanized control of a prop</td></tr><tr><td>Disturbance of capture by quick movements</td></tr><tr><td rowspan="5">CAVE</td><td>Capture-display latency</td></tr><tr><td>Impact of point of view</td></tr><tr><td>Occultation of physical and virtual elements</td></tr><tr><td>Single body-tracking suit</td></tr><tr><td>Restricted effective area for capture</td></tr><tr><td rowspan="2">Mary and the Monster</td><td>Complexity of large scale tracking</td></tr><tr><td>Complexity of large scale shared interaction</td></tr><tr><td rowspan="7">Wake</td><td>High bandwidth for large scale hand tracking</td></tr><tr><td>Occasional loss of alignment</td></tr><tr><td>Performance-costly avatars</td></tr><tr><td>Restricted field of view</td></tr><tr><td>Multiple viewing paradigms</td></tr><tr><td>Restricted exploration while sitting</td></tr><tr><td>Fatigue-inducing interaction</td></tr><tr><td>Complex interaction</td></tr><tr><td rowspan="5">Bury The Wren</td><td>Restricted field of view</td></tr><tr><td>Restricted movements of tethered headset</td></tr><tr><td>Capture-display latency</td></tr><tr><td>Imperfect positioning of dancer projection</td></tr><tr><td>Interference between trackers and infrared depth camera</td></tr><tr><td rowspan="10">Wake</td><td>Low reproduction fidelity</td></tr><tr><td>Uncanny avatars</td></tr><tr><td>Uncanny animation</td></tr><tr><td>Weightless virtual objects</td></tr><tr><td>Restricted effective area for capture</td></tr><tr><td>Disruptive VR to AR transition</td></tr><tr><td>Unstable VR to AR transition</td></tr><tr><td>Imperfect Photogrammetry</td></tr><tr><td>Abrupt transition between virtual objects</td></tr><tr><td>Distraction of the object tracker before immersion</td></tr><tr><td>Limited rendering budget</td></tr><tr><td>Inconsistent shape of physical handle and virtual objects</td></tr></tbody></table>

the interaction of the rocks in order to make their inconsistent position less perceivable.

A second approach is to scale down the ambitions of the project. This can be done by considering the trade off between complexity in the implementation or running effort and the effect granted on the audience. A compromise can then be found by reducing or removing certain features, in particular the complexity of the interaction or of the virtual environment. An example of this is the foregoing of haptic feedback in the production, as the acquired technology added too much complexity in relation to the effect it provided.

The third option is to encourage the audience to accept the limitation as part of the story. This way, issues deemed unsolvable can be addressed by only altering them enough to make them part of the

aesthetics of the piece. For instance, the passthrough cameras used for the AR sequence were in black and white, with poor resolution. The decided approach was to shift the entire aesthetic of the virtual environment towards how the camera displayed the physical world, supporting the visual merge of both environments.

*Complementary Considerations.* The second theme involves additional considerations that can inform the choice of an approach. First, designers should consider the finality of the piece during the implementation. This means that technological decisions should be driven by dramaturgical decisions based on the intended narrative experience. Moreover, breaking points should be preemptively assessed and safeguards should be implemented based on their likeability to happen. For example, a tendency for the headsets to lose their alignment with the physical world caused by the light and the repetitive nature of the setting was identified. In response, the alignment protocol was simplified, sacrificing accuracy for speed and ease of use in order to let orchestrators realign headsets during the performance with limited interruption for the participant.

Secondly, decisions are rarely independent, whether they happen at the choice of a technology, or during implementation. As such, deciding to limit an aspect of a performance may also solve or cause other issues in the project, much like the decision to instantiate each headset, which impacted the audio design.

## 5 Interviews

For the second phase of the iterative process, two interviews were conducted with researchers that worked on an augmented ballet, and a cyber opera. In both cases, the interviewees were assigned to the implementation of the immersive technology within the performance and remained in close collaboration with the directors of the pieces. As such they can provide insight on the technological details, the decisions that were made and the motivations behind them, as well as their potential outcomes.

### 5.1 Methodology

The interviews were tailor-made for each interviewee and thus had different questions, but followed a common pattern. Using available literature on each production, we identified details in the implementation that revealed explicit or implicit challenges. The first questions were about the challenges that were explicitly presented in the written documentation and requested more details on how answers were devised and compromises were found. The next questions focused on implicit challenges, which are potential issues that were quickly mentioned or could be hypothesized from the written documentation, with little to no details on how they were addressed. These questions served to open up the scope of the interview for the final part, where the interviewee were asked to share any other challenges that may have come up during the production, and what the answer of their team was. While a plan was prepared for both interviews, the order of the questions was not strictly upheld, and some questions were asked when they were more appropriate with the topic of the conversation. Both interviews were conducted in french, any following citation was thus translated.

The following analysis did not use the same codebook as the first iteration in order to allow different expressions to come out
of the independent discourse of the interviewees. However, during
the analytical phases starting from theme generation, codes from
the previous iteration were included and potentially merged with
the new codes. This allows the iterations to build on top of each
other while keeping the resulting themes and their structure flexible
when facing new data.

## 5.2 Terres Rares

*Terres Rares* is a cyber-opera featuring four on-stage performers and four musicians <fnref n="4" />, which premiered in 2022 [1]. One of its authors, a researcher who designed the immersive part of the performance and is also the second author of this paper, was interviewed for 40 minutes. The interviewee for this production will be further referred to as P1. The interview was structured around four challenges that were explicitly stated, and one implicit.

The third act of *Terres Rares* relies on the use of a Spatial Augmented Reality (SAR) display, across four sequences. This system consists in the combination of a projector and depth camera to reveal slices of physically located volumetric content [7, 12, 18]. Using either their own bodies or props as a screen, actors can intersect the virtual shapes in order to reveal the resulting slice on the corresponding physical surface. They use this system to finely control playback of video recordings by positioning props or themselves inside the virtual shapes as they perform.

An example of the limitations that the production came across is the latency between the detection of a surface and the projection of the revealed content on the surface. The issue would have been problematic if the performers moved quickly, as the projection would not follow them correctly. Coincidentally, the performers were meant to act as memories or ghosts, and as such would move slowly, thus avoiding the issue.

## 5.3 Debussy3.0

Debussy3.0 is described as an augmented ballet and consists in the performance of three people on and behind the stage for over 400 spectators [14]. The first author of the article presenting the ballet was interviewed for a total of an hour. The interviewee for this production will be further referred to as P2. The interview was structured around the challenges that were identified within the paper, three of which were explicit, and two were implicit. P2 was also able to mention a sixth challenge, which was the limited range in which the body-tracking suits could be used, and which was not found from the paper alone.

Across three acts, two dancers perform a choreography in front of stereoscopic 3D screens, one of which is being tracked using a XSens MVN motion capture suit. This tracking is used to control virtual elements that are being displayed, such as an avatar mirroring the movements of the dancer, or a trail following their hand depending on the act. An "*Augmented Reality Engineer*" directly controls the timing and visual filters applied to various virtual elements in order to match the performance of the dancers.

One of the challenges met by the production involved the use of stereoscopic 3D with a static virtual camera, which meant that the point of view of a spectator impacted the position of the virtual objects they saw. For instance, a person close to the stage could

see an object in its center, while someone further away could see
the object above the audience. This prevented the designers from
establishing a spatial relationship between the dancers and the
virtual components. In response, the scenography was adapted so
that most virtual elements appeared as behind the screen, where
this impact was limited, and some on-stage virtual elements were
removed, such as sculptures.

## 5.4 Resulting Themes

The second iteration resulted in two themes (see Fig. 3): *Strategic Management of Limitations* and *Controlling What is Shown*.

*Strategic Management of Limitations*. The first theme inherits from the previous step, expanding it with new considerations found in the interviews, which leads to its five sub-themes, corresponding to five potential strategies.

A common strategy is the *avoidance of the issue*, for instance, P2 expressed about the magnetic sensitivity of the sensors on the tracking suit "*So on a stage with an all-metal structure, in the end, it was fine, we had a few places where we couldn't go, but we avoided them, and that's it*". Another instance of this strategy is in *Terres Rares*, where a large inflatable ball was supposed to fall from above and was meant to be grabbed and manipulated by the performers as a screen. This was deemed dangerous in the dark, and too impractical with an inaccurate timing, since only one of the performers could see the projection on the prop, and was subsequently abandoned.

The second strategy has the same aim of preventing the audience from perceiving a limitation, and relies on *disguising the limitation*. This can be achieved by relying on the limitation not being disturbing for the experience of the audience, like P2 who commented "*But that's not very bothersome because it's really a small thing*" about the inconsistent positioning of virtual objects. Similarly, P1 shared "*Anyway, the background of the stage is black, precisely to absorb any potential mismatches between the captured area and the projection*" about how the issue of lag was answered.

In some cases, there is no choice but to tolerate the limitation. This case may happen due to cost, logistic or technology maturity causes. For example, P2 explained "*We did not have enough time at all, since we had only 3 months and received the retro-projected screens the day before the premiere.*" when asked about the slight lag in *Debussy3.0* between the capture and display which was reported in their paper.

The next strategy is the *integration of the limitation* within the context of the performance. For example, flags were used as props in *Terres Rares*, though they were a slow and less than ideal projection surface. Still they were kept, as they provided an ethereal look to the sequence. P1 commented "*it worked well with the kind of content we wanted to display*".

The next step is to bring the limitation forward and make it part of the purpose of the piece. Though it was not used for the issue of tracking during jumps in *Debussy3.0*, P2 mentions the possibility of having the choreography feature jumps meant to trigger glitches. In that case, the glitches would become part of the message of the performance revealing the limitations of the technology.

*Controlling What is Shown*. The second theme relates to the projected effect of the piece on the audience. It is the result of the

<footnote n="4">⁴https://terrev.univ-lille.fr/terres-rares/</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 3: Theme map after each iteration: case study (left), interviews (center), and focused literature review (right).</figcaption></figure>

grouping of codes attributed to the *Consideration of the final result* sub-theme and codes found in the current iteration. The second sub-theme, *Choice interconnection* was trimmed from the final theme map since it found little resonance with the new data set and was deemed less representative of the overarching narrative within it.

As discussed in the previous iteration, designers should carefully consider what they mean to show, which includes the limitations. In a similar way as public interfaces in the taxonomy of Reeves et al. [26], strategies can be put on an axis based on the resulting effect of the limitation (Fig. 4). As such, a limitation that must not be seen will be avoided. If it is deemed less problematic, it can be hidden, or even tolerated. It then crosses into being more constructive, and can be revealed through an integration into the context of the piece. A limitation can even become part of the message itself by being amplified and completely becoming a feature to play with, with the complete awareness of the audience.

Considering immersive experiences through the scope of how elements are seen also finds echo in the literature. The necessity to make concrete dramaturgical decisions as to what is shown or hidden to the audience can be linked to the dimensions of visibility and immersion in the scenography of immersive virtual musical instruments [30]. Choices in the design of the piece that can be placed among these dimensions can have consequences on the entire experience, and designers should stay keenly aware of how they intend their production to fit within this frame.

# 6 Independent Case Studies

The third iteration aims to validate and complete the previous findings, in particular the strategies, by broadening the sources with material from the literature.

## 6.1 Methodology

The data set is the result of a selection among a corpus built through queries in the form of combinations of the keywords *Mixed / Augmented / Virtual Reality Performance / Experience / Theater*, and papers they cite or are cited by. A strict manual selection was then made based on several criteria. The data set represents the diversity of authors, technology, and artistic context of the field, while being recent enough that a majority of the technical challenges would remain relevant to current MRP designers. Each piece of material provides sufficient detail and justification of the implementation

process and design choices to enable a thematic analysis. Coincidentally, this selection process resulted in only Ph.D. and M.S. theses being part of the final data set, which can be seen as a symptom that this nature of report tends to be more narrative and to include more details on processes.

Using case studies unrelated to the authors as the validation step also helps to break out of our previous framing of the strategies. Still, as part of the reflexive approach of the analysis, it is important to note at this point that while an effort is made in taking various perspectives into account, the results remain a reflection of our interpretation when conducting the analysis.

Like the previous iteration, the analysis began with a new codebook, but integrated the previous results in its later stages.

## 6.2 CAVE

The shared VR narrative *CAVE* premiered in 2018 and presents the opportunity to be studied through two points of view. Both DeFanti and Herscher have worked on it and describe aspects of its implementation in their respective Ph.D. theses [16, 20]. In DeFanti's thesis, we selected the part dedicated to *CAVE* (pages 33 to 36), and in Herscher's thesis, for which the show occupies a larger portion, we focused on chapter 2 covering the constraints, implementation, and results of the experience (pages 6 to 34).

Though *CAVE* features no live performer, it gathers up to 30 users in a shared environment as a co-present audience. Each spectator wears a stand-alone headset which communicates with a server in order to share their position among the entire group. The audience can thus be represented as avatars in the VR environment during the narration.

An example of a challenge answered in the piece is related to large scale shared immersion. A large number of users immersed with headsets while in physical proximity would normally lead to collisions and tracking issues. This could however be circumvented by having the audience sit down. This approach is also able to draw from the similarity of the experience to live theater, which allowed the audience to better accept this restriction.

## 6.3 *Mary and the Monster*

Following the production of *CAVE*, Herscher *et al.* began working on *Mary and the Monster* with two objectives: allowing for more interaction for the audience without encroaching on the experience
of the group, and designing an experience for both VR and AR. This production is analyzed through the third chapter of Herscher’s Ph.D thesis [20] (pages 35 to 64), which is dedicated to this multi-user experience that premiered in 2019.

Much like *CAVE*, the show is a recorded narrative experience with no live performer for 25 spectators in VR and 8 in AR. VR participants can see each other as customizable avatars arranged into three half-circular rows and are immersed in the content in full scale. Alternatively, the AR audience has no avatar since they can see each other, and sit in a single row around a round table on top of which the content is displayed at a 1/6th scale.

An example of challenge encountered by this production is what caused this smaller scale. The AR headset that was selected for the show featured a limited field of view for virtual elements. For this reason, spectators would be unable to see the entire virtual stage at life scale, which is why the designers chose to shrink the stage.

## 6.4 Wake

Wake is described as an asymmetric, co-located, co-present, mixed reality social experience. It is included in the analysis through the parts of Henson’s Ph.D thesis [19] that describe the preliminary experiments, the experience itself, and its design (pages 24 to 29, and 43 to 60).

The experience consists in the interaction of a participant and a dancer, along with a facilitator. The participant is immersed in a VR headset and is eventually able to see the non-immersed dancer as a volumetric projection on a plane using a depth camera. The hands of the participant and dancer are tracked and represented by rocks tethered through ropes either to other objects, their own hands, or the hands of the other person.

An example of challenge in *Wake* is what is suggested to be an interference between the infrared light used by the trackers and the camera. This resulted in a poor performance of the trackers when they were raised in front of the camera. While Henson notes that switching to a different kind of camera could solve the issue, she opted to deal with the technological limitation. The participants were warned that the interaction may be “*sticky*” so they could expect a “*weird*” behavior and not have their immersion broken. She also notes the helpfulness of a performer being present, so they can smoothen the overall experience.

## 6.5 Bury the Wren

Bury The Wren is designated as an extended reality project and theatrical performance, and was co-created by Kates and Christensen. The data set that was selected to study this project is a portion of the fourth chapter in Kates’ M.S. thesis [22] (pages 167 to 283). This chapter describes in detail the design and technical choices that were made for the production of the project through Kates’ *practice as research* method.

The show is described as a “*one-on-one experience*”, where a spectator and a performer interact along a narrative structure. Initially, the spectator is fully immersed in VR and can not see the performer, who manipulates a tracker on which virtual objects are anchored. These objects, captured and rendered through photogrammetry, can also be handed to the spectator. In the final sequence of the performance, the cameras of the headset are used to display the physical

environment along with the performer, bringing the spectator into AR.

An example of challenge in *Bury The Wren* is directly caused by the constraints of the choice of technology. Kates and Christensen chose to use a headset that featured external trackers, but were exposed to some inherent limitations brought by it, namely the passthrough camera. Its latency, resolution, and black and white image could have prevented any AR feature in the performance. However, they chose to look for a way to make use of the limitation, and decided to embrace the daguerreotype aesthetic it brought. As a result, they filtered and shaded the image during the AR sequence so it leaned even more towards their newly intended look.

## 6.6 Resulting Themes

As seen in Fig. 3, the third iteration of reflexive thematic analysis resulted in three themes: *Strategic Management of Limitations*, *Aligning Performance with Technological Constraints*, and *Managing Outcomes*. The final state of the themes, sub-themes, and retained codes can be found in Table 2.

*Strategic Management of Limitations*. We found the same sub-themes as in the previous iteration, with one modification on the fifth sub-theme that we changed from *Play with the Limitation* to *Leverage Limitations*. This change stems from new codes found in the independent case studies pointing towards using a limitation as a tool for generating ideas, especially in Kates’ work as she expressed “*I wonder if the disruptive seams might have provided us with some unknown usefulness*” or “*Working resolutely against technical common sense created the happy incident of the digitally-tarnished candlestick*”.

*Aligning Performance with Technological Constraints*. The second theme is the result of the reintroduction of codes and sub-themes that were discarded in the previous iteration. They were then not considered representative enough to remain within the final themes, but the addition of new themes from the independent case studies showed their legitimacy. The theme depicts the inherent challenges in projects involving both performances and technologies.

Working with collaborators from other disciplines is a recurrent matter in MRP, as it usually merges at least the domains of performing arts and technological design. During their interview, P2 recalled “*After some time we manage to understand what the other expects or how they view this relationship, thus moving beyond the initial perspective of the other discipline*”. This reflects how cross-disciplinary collaboration may not be trivial and how efforts should be put towards reaching a common understanding of the objectives in the project. These efforts can be made either as part of the creative process or outside of it. For example, both Henson and P2 integrated their collaborators’ communities by taking part in their classes and workshops on respectively Physical Integration and Dance. Kates undertakes steps promoting the collaboration as part of her process, reporting “*we ran the risk of [the developer] becoming disconnected from the primary objective of Bury The Wren: to tell a compelling story*”, and making sure in response that the developer also fully takes part in the rehearsals. Following this approach then allows both the creation and implementation processes to inform each other throughout the life cycle of the piece.
<table><caption>Table 2: The final codebook after the third iteration.</caption><thead><tr><th>Themes</th><th>Sub-themes</th><th>Codes</th></tr></thead><tbody><tr><td rowspan="10">Strategic Management of Limitations</td><td rowspan="4">Avoid Limitations</td><td>Simplify Interaction</td></tr><tr><td>Restrict Physical and Virtual Space</td></tr><tr><td>Restrict Environmental Complexity</td></tr><tr><td>Omit Features</td></tr><tr><td rowspan="3">Disguise Limitations</td><td>Rely on Limitations Remaining Unnoticed</td></tr><tr><td>Actively Conceal Limitations</td></tr><tr><td>Manage Attention to Prevent Noticing Limitations</td></tr><tr><td rowspan="2">Tolerate Limitations</td><td>Accept Temporal Constraints</td></tr><tr><td>Account for Technological Maturity</td></tr><tr><td rowspan="2">Integrate Limitations</td><td>Align Limitations with Purpose</td></tr><tr><td>Incorporate Limitations into Aesthetic Design</td></tr><tr><td rowspan="3">Leverage Limitations</td><td>Embrace Limitations as Creative Catalysts</td></tr><tr><td>Push Against Limitations for Artistic Discovery</td></tr><tr><td>Allow Limitations to Shape Project Goals and Features</td></tr><tr><td rowspan="4">Aligning Performance with Technological Constraints</td><td rowspan="2">Cross-disciplinary Collaboration</td><td>Foster Understanding Among Collaborators</td></tr><tr><td>Include Developers in the Creative Process</td></tr><tr><td rowspan="2">Dramaturgy and Technology</td><td>Unintentional Integration and Creative Solutions</td></tr><tr><td>Adapting Theater Techniques to Mixed Reality</td></tr><tr><td rowspan="5">Managing Outcomes</td><td rowspan="2">Collaboration with Live</td><td>Support and Flexibility of Live Performers</td></tr><tr><td>Simplifying the Audience Experience</td></tr><tr><td rowspan="3">Performers Controlling What is Shown</td><td>Optimizing Visual and Technical Impact</td></tr><tr><td>Managing Complexity</td></tr><tr><td>Maintaining Seamlessness and Minimizing Disruptions</td></tr><tr><td>Preserving Continuity</td><td>Handling Discontinuities and Failures</td></tr></tbody></table>

Even with all collaborators on the same page, the production needs to accord dramaturgical and technological processes of conception and implementation. Comparisons between theater and MRP were made at several points, noting the potential of traditional techniques, with for instance Herscher describing how *Mary and the Monster* drew from the traditional practice of a “blackout” to adapt it in VR, with the improvement of making physically impractical changes to the set. Kates also comments on what technology added to her practice of scenography, writing “This is a significant departure from designing in the traditional theatrical space where modifications on a large scale are usually impossible due to time and resources”. Here, she refers to the design of specific software tools that permitted great flexibility during the creation of the show. The design and use of such tools can thus become key in establishing an efficient workflow in the design and implementation of a MRP.

In the case of productions involving live performers, the role they play involves important specificities. First, they may not always be immersed in the same conditions as the audience, and might not be able to completely perceive the virtual environment. This is the case of *Terres Rares*, on which P1 described the various elements they put in place to help the performers interact with objects of limited visibility to them, such as marks on the floor. The involvement of live performers can however also bring initially unsuspected value. For instance, Kates shares “In an effort to overcome uncanny programmatic movement we chose to map our object to a tracker so that when they were manipulated by a human holding the tracker

they reproduced that believable movement in VR”. Henson also discusses how the presence of live performers could help mitigate the issue of infrared interference between the depth camera and the trackers through orchestration. This use of a performer shows that bringing live performance into a production, despite creating some challenges, can also enable potential answers to previously complex issues.

*Managing Outcomes*. The last theme encompasses both the control of what is shown, which was present in the previous iteration, and the preserving of continuity. Continuity relates to Benford and Giannachi’s trajectory framework [6], which helped Kates identify limitations related to transitions and their potential ruptures during the experience of the spectator. This consideration led to the modification of the transitional space between VR and AR into a version inducing a smoother visual transition. This same care about continuity throughout the show and the identification of problematic transitions also helped them identify potentially overlooked limitations. By considering the arrival of the participant as part of the experience, Kates was able to notice that the presence of the object tracker before immersion could be distracting and hid it in the darkness by altering the lighting. Preserving continuity also involves preparing for failures, such as in *Mary and the Monster* for which a monitoring feature was implemented in order to quickly identify any issue with the calibration of the headsets. The same commitment to continuity also informed design choices in the production of *Animate*. In the face of the high risk of a headset losing
its alignment with the physical space, the procedure to realign it
was made simpler and thus quicker at the expense of precision.

# 7 Discussion

We build on our results to provide a set of strategies for reconciling
technical constraints and artistic intent, which can be used to ana-
lyze choices made during the design of existing MRPs but also to
envision new solutions for challenges when designing novel MRPs.
We also discuss limitations of this study and how future work may
build from these results.

## 7.1 MRP strategies design space

The main result of the analysis is the five strategies under which
answers to challenges can be organized. These strategies build a
spectrum based on the level of awareness of the audience, as seen
in Fig. 4.

(1) *Avoid* aims to side step the issue, or scale down the desired effect, for instance by reducing the complexity of either the environment or the interaction. Therefore, it is associated with the lowest amount of awareness.

(2) *Disguise* is the reliance on the limited capacity of the audience to notice a limitation. It corresponds to a slight acceptance that the audience could notice the issue. This can be accomplished with more or less effort, since some limitations may already be hardly perceivable by participants.

(3) *Tolerate* corresponds to a neutral stance where no action is undertaken in reaction to the limitation. This is usually the marker of constraints preventing a different answer to a limitation, such as time or maturity of a selected technology.

(4) *Integrate* aims to alter either the limitation or the piece in order to match them together. This allows the limitation to be perceived by the audience as more of a feature than an unsolved issue.

(5) *Leverage* makes the limitation a key part of the show. Where the previous strategy simply merges the limitation with the rest, this one brings the limitation forward, and may even amplify it to convey a message.

It is important to note that these strategies do not define discrete categories, as some solutions may be seen as in-between two strategies. One should also consider that the interconnection of challenges and answers can lead to a challenge being answered in several ways, or to a decision answering several challenges.

Interestingly, this spectrum aligns with Reeves et al.'s taxonomy of public interfaces, which organizes them around two axes depicting how much effects and manipulations are revealed to observers [26].

Parallels can be drawn between these strategies and those proposed by Benford et al. as potential reactions to the connectivity uncertainty in *Can You See Me Now?* [3]. *Remove, Hide, and Exploit* are highly similar to *Avoid, Disguise, and Leverage*. *Manage* is close to *Tolerate* and *Disguise*, but focuses on establishing an alternative experience when the issue threatens to break it. This follows one of the recommendations in the sub-theme *Preserving Continuity*. *Reveal* has a key difference to *Integrate* in how it is used, which may stem from the difference in context of both strategies. Instead of matching the limitation to the theme of the experience, it pushes

to inform the user on the uncertainties of the system in order for
them to make better decisions.

<figure><img src="image_5.png" /><figcaption>**Figure 4: Five strategies arranged in relation to the projected awareness of the audience towards the limitation**</figcaption></figure>

## 7.2 Using the design space

Challenges that were previously identified can now be examined
again under the scope of these strategies. Formalizing these
strategies also amounts to defining a generative design space. As
designers become aware of how their first approach may be placed
on the spectrum, they can imagine other potential answers that
would be placed elsewhere. This is not an attempt to point out bet-
ter alternatives to choices in the productions, as some constraints
may not have been considered, and some changes may have reper-
cussions on the purpose of the pieces. Instead, we try to show the
other possible approaches that can be taken in the face of such
challenges.

The approach in *Animate* to let go of a complete synchronization of the virtual objects and to reduce the interaction in order to mask it can be placed between *Avoid* and *Hide*. This can be argued by the issue of synchronization being avoided, while inducing another limitation. The audience can expect to see the same environment as their neighbor, yet can tell from their behavior that there are differences. This resulting secondary limitation thus finds a complementary answer in the form of reducing the interactivity of the scene, effectively hiding the difference in the environments from the audience. There was an opportunity to embrace the asymmetry instead, and leaning towards *Leverage* by having the performers also seemingly interact with unseen rocks, while maybe even commenting that they can not see what the other sees. This would bring the focus of the audience precisely on the asymmetry, and allow them to experiment with their neighbors and discuss how their environments differ.

In *Terres Rares*, the latency between capture and projection could cause issues, had the performers moved too quickly. P1 commented on this issue that the answer was coincidental since the conception of the show already meant for the performers to move slowly, necessitating no further action in response to the limitation. Still, the back wall was made dark in order to limit the visibility of any potential light not hitting the intended screens. This leads to this combination of approaches being placed between *Disguise* and *Tolerate*, since the designers relied on the limitation not being noticeable. A challenge like this could also have been met with an approach of amplifying the latency in order to use it, leaning more towards *Leverage*. It is thus possible to imagine a performer interacting with a highly delayed projection in creative ways.

In *Bury The Wren*, the aesthetic brought by a camera with poor resolution, latency, and in black and white was embraced. By adding shaders to the image, its imperfection became part of the feel of the scene, corresponding to an *Integrate* strategy. Alternatively, the use
of an external camera would have allowed a feed of better quality, side-stepping the issue, and corresponding to an *Avoid* strategy.

## 7.3 Other dimensions in the design of MRP

In the analytic process, some elements were dismissed as themes or sub-themes, but may be interesting to consider in the description of strategies. The appearance of a challenge and application of its answer has appeared to be possible at various stages of the life cycle of the production. For instance, challenges result from the choice of technology, or the budget available like in *Debussy3.0*, which typically happen early on. Other challenges emerge very late, like a lag that appeared when setting up on stage in the same production. Answers can also vary in timing, from early in the implementation process with the design of adapted authoring tools [23, 28] or sketching [10], to the preparation of counter measures such as in *Animate* or *Can You See Me Now?* [3], or even improvised interventions [27].

Drawing from Benford *et al.*'s trajectories [6], other potential dimensions to consider are the spaces, interfaces, and roles involved in the challenges and answers. Challenges may relate to either the physical, the virtual world, or both, and so do their potential answers. For example, the glitches caused by jumps in *Debussy3.0* relate to the physical world, while the problematic positioning of the performers in the virtual environment of *Wake* shares both spaces. More specifically, how participants are immersed and perceive each other can be considered, for instance by adapting Zappi *et al.*'s dimensions for immersive virtual musical instrument scenography [30]. Challenges can also involve different parts of the interface of the immersive technology and be related to tracking (*CAVE*), capture (*Debussy3.0*), display (*Mary and the Monster*), or networking (*Mary and the Monster*). Finally, strategies can be discussed through the roles involved in the performance, as people occupying different roles may encounter different challenges and be able to answer them differently. In this context, various roles can be taken by spectators [6], actors [27], or even designers [23].

The themes also bring up the existence of non-technological challenges stemming from the interdisciplinary collaboration of art and science. The need for an establishment of a common vocabulary, as pointed out by P2 and Kates is a well known issue both in the global context of art/science collaboration [15], and the specific context of MR applications, with strategies adapted to it [23].

## 7.4 Limitations and Future Work

The testimonies that were collected for this analysis are single perspectives on their respective projects, except for *CAVE* which was represented by two designers. Since the limitations and how they were addressed may not always be reported for reasons of size limits or perceived value for publishing, their identification often relies on details and anecdotes spread out through the documentation. A richer analysis would heavily rely on interviews in order to draw undisclosed ordeals that the productions went through. It could also greatly benefit from looking at different perspectives of the same project, particularly from designers who played different roles in the production. Furthermore, a greater reliance on in-depth interviews would allow one to include in the analysis productions that were not described in publicly available literature as thoroughly

as those in this study. Most notably, this would enable the inclusion of productions with a more commercial outlook than the ones analysed here.

A notable limitation is that the iterative analysis presented in this paper stems partially from productions involving the first and second authors. This is important to keep in mind as it had a strong opportunity to influence the continuation of the process in light of the authors' own practice, with a focus on specific aspects. Still, the analysis was conducted with this bias in mind and includes the third step as a way to address it.

The data corpus included works discussing productions between 2019 and 2021 in order to have a coherent technological background for the challenges they encountered. Future work can find value in looking at the potential evolution in challenges and strategies before and after this period.

Additionally, value can be found by investigating the generative power of the strategies presented in this paper. This could be accomplished in future work by accompanying the production of a MRP and observing how using this framework can guide the design process, and conversely if and how the framework needs to be expanded.

## 8 Conclusion

Through three steps of an iterative reflexive thematic analysis on a performance-led case study, two interviews, and independent case studies, we identified five potential strategies when approaching challenges in MRPs involving the friction between a desired outcome and the constraints of immersive technologies. These strategies are organized depending on the extent to which the audience should be aware of the limitations, and include: *Avoid, Disguise, Tolerate, Integrate, and Leverage*.

We showed how this design space can be used for the analysis of previous MRPs. We believe that when faced with such compromises, designers of novel MRPs can draw from these results to envision new approaches to their challenges. Though these strategies were found in the context of MRPs, they may have broader applications. For instance, in the context of an educative environment *Disguise* might be used to maintain engagement or *Leverage* may turn a constraint into a learning moment. We also hope that by outlining the similarities within the field of MRPs, designers can draw inspiration from other works that would seem at first sight too different, and extend this exchange of ideas to and from fields beyond MRPs.

## Acknowledgments

This work was partially funded by grants from the Fonds de Recherche du Québec (FRQ PRISME-ART) and the Hauts de France region. The authors would also like to thank Chris Salter and the rest of the team for the work accomplished on the production of *Animate*, and the Éolie Songe theatre company for the collaboration on *Terres Rares*.

## References

[1] Cagan Arslan and Florent Berthaut. 2025. Examining the Design Process for 3D Interactions in Performing Arts: A Spatial Augmented Reality Cyber-Opera Case Study. *IEEE Transactions on Visualization and Computer Graphics* (March 2025), 1–10. doi:10.1109/TVCG.2025.3549194

[2] Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K. Chilana. 2020. Creating Augmented and Virtual Reality Applications: Current
Practices, Challenges, and Opportunities. In Proc. ACM CHI'20. Honolulu, HI, USA, 1-13. doi:10.1145/3313831.3376722

[3] Steve Benford, Andy Crabtree, Martin Flintham, Adam Drozd, Rob Anastasi, Mark Paxton, Nick Tandavani, Matt Adams, and Ju Row-Farr. 2006. Can you see me now? *ACM Trans. Comput.-Hum. Interact.* 13, 1 (Mar. 2006), 100-133. doi:10.1145/1143518.1143522

[4] Steve Benford and Gabriella Giannachi. 2011. *Performing Mixed Reality*. The MIT Press.

[5] Steve Benford and Gabriella Giannachi. 2012. Interaction as performance. *Interactions* 19, 3 (May 2012), 38-43. doi:10.1145/2168931.2168941

[6] Steve Benford, Gabriella Giannachi, Boriana Koleva, and Tom Rodden. 2009. From interaction to trajectories: designing coherent journeys through user experiences. In Proc. CHI'09. New York, NY, USA, 709-718. doi:10.1145/1518701.1518812

[7] Florent Berthaut, Diego Martinez Plasencia, Martin Hachet, and Sriram Subramanian. 2015. Reflets: Combining and Revealing Spaces for Musical Performances. In Proc. NIME'15. Baton Rouge, LA, USA.

[8] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. *Qualitative Research in Psychology* 3, 2 (Jan. 2006), 77-101. doi:10.1191/1478088706qp0630a

[9] Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. *Qualitative Research in Sport, Exercise and Health* (Aug. 2019).

[10] Bill Buxton. 2010. *Sketching User Experiences: Getting the Design Right and the Right Design*. Morgan Kaufmann.

[11] David Byrne. 2022. A worked example of Braun and Clarke's approach to reflexive thematic analysis. *Quality & Quantity* 56, 3 (Jun. 2022), 1391-1412. doi:10.1007/s11135-021-01182-y

[12] Alvaro Cassinelli and Masatoshi Ishikawa. 2009. Volume slicing display. In Proc. ACM SIGGRAPH ASIA'09 (SIGGRAPH ASIA'09). New York, NY, USA, 88. doi:10.1145/1665137.1665207

[13] Matthew Chalmers and Areti Galani. 2004. Seamful interweaving: heterogeneity in the theory and design of interactive systems. In Proc. DIS'04. Cambridge, MA, USA, 243-252. doi:10.1145/1013115.1013149

[14] Alexis Clay, Gael Domenger, Julien Conan, Axel Domenger, and Nadine Couture. 2014. Integrating augmented reality to enhance expression, interaction &amp; collaboration in live performances: A ballet dance case study. In Proc. IEEEISMAR-MASH'D'14. Munich, Germany, 21-29. doi:10.1109/ISMAR-AMH.2014.6935434

[15] D. J. Cox. 1991. Collaborations in art/science: Renaissance teams. *The Journal of biocommunication* 18, 2 (1991), 15-24.

[16] Connor DeFanti. 2019. *Co-Located Augmented and Virtual Reality Systems*. Ph.D. Dissertation. Dept. Comp. Sci., New York Univ., New York, NY, USA.

[17] David Gochfeld, Corinne Brenner, Kris Layng, Sebastian Herscher, Connor DeFanti, Marta Olko, David Shinn, Stephanie Riggs, Clara Fernández-Vara, and Ken Perlin. 2018. *Holojam in Wonderland: Immersive Mixed Reality Theater*. Leonardo 51, 4 (Aug. 2018), 362-367. doi:10.1162/leon_a_01644

[18] Fatma Ben Guefrech, Florent Berthaut, Patricia Plénacoste, Yvan Peter, and Laurent Grisoni. 2021. Revealable Volume Displays: 3D Exploration of Mixed-Reality Public Exhibitions. In Proc. IEEEVR'21. Lisbonne, Portugal, 31-39. doi:10.1109/VR50410.2021.00023

[19] Anna Henson. 2019. *We're In This Together: Embodied Interaction, Affect, and Design Methods in Asymmetric, Co-Located, Co-Present Mixed Reality*. M.S. thesis. Dept. of Architecture, Carnegie Mellon Univ. doi:10.1184/R1/8235539.v1

[20] Sebastian Herscher. 2020. *Scaling Multi-user Virtual and Augmented Reality*. Ph.D. Dissertation. Dept. Comp. Sci., New York Univ., New York, NY, USA.

[21] Sebastian Herscher, Connor DeFanti, Nicholas Gregory Vitovitch, Corinne Brenner, Haijun Xia, Kris Layng, and Ken Perlin. 2019. CAVRN: An Exploration and Evaluation of a Collective Audience Virtual Reality Nexus Experience. In Proc. ACM UIST'19. New Orleans, LA, USA, 1137-1150. doi:10.1145/3332165.3347929

[22] Beth Kates. 2020. *Incandescent Edges of the Future: Performance Creation with Virtual, Augmented and Carbon Realities*. Master's thesis. Drama, Calgary Univ., Calgary, AB, Canada. doi:10.11575/PRISM/38180

[23] Veronika Krauß, Alexander Boden, Leif Oppermann, and René Reiners. 2021. Current Practices, Challenges, and Design Implications for Collaborative AR/VR Application Development. In Proc. ACM CHI'21. Yokohama, Japan, 1-15. doi:10.1145/3411764.3445335

[24] Kris Layng, Ken Perlin, Sebastian Herscher, Corinne Brenner, and Thomas Meduri. 2019. *CAVE: Making Collective Virtual Narrative: Best Paper Award*. Leonardo 52, 4 (Aug. 2019), 349-356. doi:10.1162/leon_a_01776

[25] Michael R. Marner, Sam Haren, Matthew Gardiner, and Bruce H. Thomas. 2012. Exploring interactivity and augmented reality in theater: A case study of Half Real. In Proc. ISMAR-AMH 2012. Atlanta, GA, USA, 81-86. doi:10.1109/ISMAR-AMH.2012.6483993

[26] Stuart Reeves, Steve Benford, Claire O'Malley, and Mike Fraser. 2005. Designing the spectator experience. In Proc. ACM SIGCHI'05. Portland, OR, USA, 741-750. doi:10.1145/1054972.1055074

[27] Asreen Rostami and Donald McMillan. 2022. The Normal Natural Troubles of Virtual Reality in Mixed-Reality Performances. In Proc. ACM CHI'22. New Orleans, LA, USA, 1-22. doi:10.1145/3491102.3502139

[28] Ben Shneiderman. 2002. Creativity support tools. *Commun. ACM* 45, 10 (Oct. 2002), 116-120. doi:10.1145/570907.570945

[29] Joris Weijdom. 2022. Performative prototyping in collaborative mixed reality environments: an embodied design method for ideation and development in virtual reality. In Proc. ACM TEI'22. Daejeon, Republic of Korea, 1-13. doi:10.1145/3490149.3501316

[30] Victor Zappi, Florent Berthaut, and Dario Mazzanti. 2022. From the Lab to the Stage: Practical Considerations on Designing Performances with Immersive Virtual Musical Instruments. In *Sonic Interactions In Virtual Environments*. Springer VS.