# The digital performance between user experience design and the metaverse

Federica Patti

<figure><img src="image_1.png" /></figure>

## To cite this version:

Federica Patti. The digital performance between user experience design and the metaverse. Journées d'informatique théâtrale, Oct 2024, Avignon, France. hal-05138770

HAL Id: hal-05138770

https://hal.science/hal-05138770v1

Submitted on 1 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# THE DIGITAL PERFORMANCE BETWEEN USER EXPERIENCE DESIGN AND THE METAVERSE

Federica Patti

Università di Torino

f.patti@unito.it

## ABSTRACT

The Metaverse (Ball 2022) is now a highly performative, interactive and community-based ecosystem. It represents a fundamental shift in today's notion of 'being online', redefining the boundaries between physical and virtual, private and collective, human and more than human (Barad 2007; Braidotti 2013). Before and after the pandemic, it has emerged as a crucial site of social experience, facilitating conversations, creative projects and collective interactions. Can it, therefore, be conceived as a Fifth Wall (Steyerl 2021), a virtual theatròn (Del Gaudio 2020) where digital performances (Dixon 2007) are staged?

This article proposes an analysis of digital performances presented in 3D virtual online environments, following the user experience design approach: by proposing Martin Romeo's performance "Humanverse" on Spatial (2023) as a case study, the influence of the chosen technological system on the poetics of this type of virtual event will be identified. This contribution proposes to determine the mental model (Norman 2013), the kind of interactive experience Romeo proposed to the audience, and why. How he designed this human-computer interaction system to create the collective performative action and scenic environment (Patti 2024), and whether these choices could generate a narrative dramaturgy.

## 1. INTRODUCTION

The Internet has encouraged the emergence of virtual communities and forms of creative participation (Bazzichelli 2006; Monteverdi 2023). Before and after the pandemic, advanced online virtual environments have emerged as crucial sites for social and cultural experiences, structuring and facilitating conversations, creative projects, collective endeavours and commercial exchanges.

Dixon has defined digital performance as a specific field of artistic production and investigation in which the technological gesture is the protagonist and co-author of the creation of increasingly augmented, hyper-connected, decentralised and collaborative works (2007). In this context, Boccia Artieri has recently described

online theatre experiments as 'unidentified online theatre objects' - OTONI (Boccia Artieri 2023). OTONI are narratives conceived, realised, distributed and consumed online within a theatrical framework. They straddle the line between performative objects and extended spectator experiences (streaming video on a platform, immersion in a metaverse, chatting on social platforms, etc.). They are hybrids in terms of theatrical genres and adaptive and transdisciplinary formats by vocation. To fully understand the OTONI experience, it is essential to examine it through the quality of the mediatised theatrical experience, which has to do with the forms of construction of the relationship between spectator and performative gesture in the digital, that is, with a different empathic pact with the spectator, modulated in the relationship between a 'here and now' of a different nature, with a pervasive sense of presence and with an increased perception and representation of body and identity. According to Boccia Artieri, the 'live' performative dimension has been progressively influenced by the media, by the communicative possibilities opened up by a technologically mediated body-scene relationship, but also by the pervasiveness of the performative form that embraces mass media according to transmedial logics (ibid.) and gradients of liveness (Gemini and Brilli 2023) that are now common and shared experiences in digital spaces. Thus, the User Experience design's dramaturgical importance in online digital performance has yet to be recognised. After Eva and Franco Mattes' historical 'Reenactments' on the 3D virtual platform Second Life in 2007, some recent digital performances on online platforms force a re-evaluation of the nature of being online, together with a re-contextualisation of the concepts of reality, simulation, liveness (Auslander 2012) and community.

## 2. METHODOLOGY

Since the mid-1990s, the user has increasingly been invited to live an unusual experience in which the physical and the digital merge. The noun 'user experience' refers to how people experience an encounter with a system; it extends traditional human-computer interaction design by addressing all aspects of the product or service as perceived by the user. The UX classification system is based on the user's feelings, preferences, attitudes and emotional responses. Over the last thirty years and more, interactivity, conceived as a two-way
relationship between man and machine, has given users
the power to manipulate and transform the digital object
with which they interact. In turn, users got the
opportunity to influence it, taking an active role in the
creation
process.
Interactive systems have been studied in Human-
Computer Interaction (HCI) since the early 1980s. As a
branch of HCI, 'user experience' (UX) explicitly refers to
the experience(s) resulting from the encounter with
systems (Law, Van Schaik, and Roto 2014). According to
Löwgren (2007), the end product of interaction design is
always interactive products and services. UX is every
aspect of a person's - or group of people's - interaction
with a particular computer system, including interface,
graphics, industrial design, and physical and manual
interaction. UX design encompasses and extends
traditional human-computer interaction design,
addressing all aspects of the product or service as
perceived by the user.
UX evaluation can focus on methods that provide an
overall qualitative measure of the experience of
performing a particular task or using a system. UX can be
described as the totality of the user's feelings,
perceptions, motivations, preferences, beliefs, attitudes
and emotional responses that result from encountering or
interacting with an interactive technological artefact at a
given time and in a given usage context (Hussain,
Mkpojiogu, and Zabidin Husin 2021). The dimensions of
UX can be grouped into main categories such as
functionality, findability, accessibility, desirability,
usability, credibility and usefulness, each of which can be
refined by qualitative characteristics such as sensuality,
emotionality, spatiotemporal, instrumentality, sensory,
behavioural, utilitarian and affectivity. The UX
classification system is based on the user's emotions,
preferences, attitudes and emotional responses (ibid.).
Although initially driven by a strong emphasis on
functionality and usability, UX research has recently
focused on pleasure, beauty, emotion and experience.
Experience-oriented designers try to create specific
interaction events that can influence the user's emotions
and reactions. In doing so, they create the conceptual
model that underlies the interaction and experience of the
system. According to Norman (2013), users develop a
mental model of how they think the system works by
interacting with it. This model is used to reason about the
system, anticipate its behaviour, and explain why it reacts
the way it does. On the other hand, the designer reifies
 materialises) their mental model of a given design, for
example, a computer system, in a system model, which
becomes the only means of communicating their mental
model to the user.

Starting from these theoretical premises, I have
selected artistic case studies conceived on different
platforms, in different eras. I have analysed them, trying
to determine whether or not the technological system
influences the performative purposes of this type of
event. I tried to define how the interaction systems
designed by the artists allow the audience to interact with

the experience and what UX dimensions are generated. I
investigated if and how the artist's intervention hacks
each system, for what purpose, and with what effect on
the UX itself. Finally, I asked myself whether these
criteria for describing UX are also suitable for describing
digital online performances.

My doctoral research aims to structure an up-to-date
conceptual framework to describe these OTONI,
conceived and developed by implementing the design
possibilities of virtual environments and objects through
native software or external graphic design software. The
initial hypothesis is that artists stress the characteristics of
the platforms they use, disrupting the native conceptual
model to then check whether this operation generates a
specific dramaturgy, & how. How did the artist design
these systems to create performative actions and scenic
environments? What mental model and what kind of
experience (UX) did they use, design and propose to the
audience, and why? Can these spaces, communities and
UX generate a narrative dramaturgy? To answer these
questions, I am developing an interdisciplinary method of
art-historical analysis and interaction design. The method
involves an aesthetic reading of the performance object, a
historical contextualisation relating to artistic and
technological evolution, and a description of the object
according to UX dimensions.

3. ANALYSIS

Focusing on the concept of the hedonic experience (Diefenbach, Kolb, and Hassenzahl 2014) lived by users, this article proposes to read one such case based on the way users interact with technology, understand, perceive and describe the experience, emphasising psychological well-being through non-instrumental, user-oriented product attributes. Emphasising the dynamics of extended authorship and the DIWO (do it with others - or do it together) philosophy, an attempt is made to define how these systems enable the viewer to interact with each specific virtual environment, what setting has been created and what UX qualities have been generated by the system or the artist's intervention. Reading a digital performance means identifying its tools, the human-machine dynamics it activates, and the experiential effects it produces (Pizzo, Lombardo, and Damiano 2021). My professional experience as a curator of digital performances for Italian festivals and as a tutor of Italian online theatre residency projects allows me to have an up-to-date view of contemporary artistic production in this liminal field, in particular of the Italian scene of the last 20 years, within which I selected the proposed case. The analysis of Martin Romeo's performance "Humanverse" on Spatial (2023) starts by identifying and describing the influence of each technological system on the performative, participatory and community-building poetics of the virtual event.
<figure><img src="image_3.png" /><figcaption>**Picture 1.** Martin Romeo, *Humanverse*, online performance on Spatial, 2023. Still from video (courtesy the artist).</figcaption></figure>

As part of the “Residenze Digitali” project in 2023, I closely followed the creation, development and documentation of “Humanverse” on Spatial, a virtual reality platform for collaboration and content creation, to identify the effect of the system on dramaturgy. Guided by an artificial voice, five anonymous travellers wear a VR headset and form a virtual relationship by performing simple group actions, as well as augmented gestures (such as flying), collaborating at a distance and learning to move together in virtual space. Invited to traverse the spaces designed by Romeo on Spatial, the users reflect on their virtual physicality and the ‘average void’ of a platform like Spatial. Describing “Humanverse” with the HCI philosophy can be seen as a metaphor for Dante’s Divine Comedy: while the narrator’s voice gives directions, he ensures that the group remains united. In “Humanverse”, users are asked to learn to move, listen and collaborate, to recognise each other by overcoming awkwardness, and to explore together the spaces modelled by Romeo. The aim is to overcome initial awkwardness, to listen and work together by forming a group and following a set path. Compared to the standard native functionality of Spatial, group formation and collective dynamics are new. It should be noted that Spatial, as a 3D space, is in itself a metaphor, but it is a new metaphor (Baía Reis and Ashmore 2022): it does not refer to any previous standards, which makes it difficult for the user to interact with it for the first time. In other words, the mental model and the conceptual model do not coincide. The buttons are not very intuitive, and the functionality requires a medium to high learning time: this means low usability and learnability. I, therefore, hypothesised that Romeo’s innovative choices stressed Spatial’s native system and disturbed its conceptual model. I argued that these dissonant choices were the basis of the dramaturgy.

To verify this, I collected user feedback through several steps and tools borrowed from the user feedback collection and analysis system used during user experience design:

1) Initially, I sought to test users' level of confidence in technology by asking them to complete Franke, Attig and Wessel's (2019) Affinity for Technology Interaction (ATI) scale.<fnref n="1" />

2) I presented users with a standard multiple-choice user experience questionnaire, the UEQ User Experience Questionnaire, with responses geared towards describing the experience using the most common dimensions. In addition, I asked them to rank the selected dimensions according to their importance in describing the service. For example, ‘I find it useful to describe this service according to its level of ease, 1 - difficulty 5’.<fnref n="2" />

3) I conducted a 1:1 interview to collect qualitative data on the UX dimensions in parallel (questions such as ‘Describe your experience of the service according to (one dimension) to understand how people interpret the different dimensions and whether the service “makes sense” to them.’

*"I found myself with a new interface, which I had to learn to manage and control. It was like going back to primordial origins, to the essence of what is important, looking around to return to oneself".*

*"My physical being began to walk together with the avatar, in the tower scene, we were one! I climbed the mountain with a bit of vertigo."*

*"Initially, I felt that emptiness in my stomach when I threw myself or jumped beyond gravity. Later, I rationalised that it was related to emotions. It wasn't true! I couldn't die! I continued doing it with a certain intoxication."<fnref n="3" />*

4) I did a meta-analysis of the feedback to see which dimensions were relevant and useful to describe the case, highlighting most of the responses. By comparing this information, I can highlight whether and how the conceptual model was disregarded or not. Based on the collected data, I can describe the performance according to most of the UX dimensions based on identifiable patterns:

*"I found the experience ENJOYABLE 4.3 EASY 2.3 APPRECIATABLE 4.1 SUSTAINABLE 3.7 SUPPORTIVE 4.3 SAFE 1.5 EFFICIENT 3.8 CONGENIAL 1.5 AVERAGE VALUE 2.2 Meets expectations 2.4"*

*"I feel INTERESTED 4.6 SPECTACULAR 1.4 ENTHUSIASTIC 3.4, EMBARRASSED 1.2 ATTENTIVE 3.7"*

By comparing this data with the conceptual model underlying Spatial's UX design, I can highlight the similarities and discrepancies with the Humanverse design, if and how the user's mental model was disregarded.

<footnote n="1">¹ https://forms.gle/txfpE7iThcPHtSEN7</footnote>

<footnote n="2">² https://forms.gle/PYuN5YgNxiwz6pof7</footnote>

<footnote n="3">³ Examples of 1:1 interview response.</footnote>
<table><caption>**Table 1:** extract of feedback collected via UEQ questionnaire.</caption><thead><tr><th>FACILE/OFFICIO DA APPRENDERE</th><th>DI GRANDE/POCO VALORE</th><th>USUALE/MODERNO</th><th>CONFORME/NON CONFORME/ALI ASPECTATIVE</th><th>NON PRAGMATICO/PRAGMATICO</th><th>INTERESSATI</th><th>ECOITATI</th><th>SCONVOLTI</th><th>ENTUSIASTA</th><th>MBARAZZATI</th></tr></thead><tbody><tr><td>1</td><td>3</td><td>5</td><td>3</td><td>3</td><td>5</td><td>4</td><td>1</td><td>3</td><td>5</td></tr><tr><td>1</td><td>3</td><td>5</td><td>3</td><td>3</td><td>5</td><td>4</td><td>3</td><td>5</td><td>5</td></tr><tr><td>3</td><td>3</td><td>4</td><td>3</td><td>3</td><td>5</td><td>3</td><td>2</td><td>3</td><td>1</td></tr><tr><td>3</td><td>3</td><td>5</td><td>4</td><td>3</td><td>4</td><td>2</td><td>1</td><td>2</td><td>2</td></tr><tr><td>2</td><td>2</td><td>5</td><td>1</td><td>3</td><td>5</td><td>4</td><td>2</td><td>5</td><td>1</td></tr><tr><td>2</td><td>2</td><td>4</td><td>2</td><td>4</td><td>5</td><td>4</td><td>4</td><td>2</td><td>5</td></tr><tr><td>3</td><td>3</td><td>4</td><td>2</td><td>3</td><td>3</td><td>3</td><td>1</td><td>3</td><td>2</td></tr><tr><td>3</td><td>3</td><td>4</td><td>2</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>2</td></tr><tr><td>4</td><td>3</td><td>4</td><td>1</td><td>3</td><td>5</td><td>3</td><td>1</td><td>5</td><td>1</td></tr><tr><td>1</td><td>2</td><td>4</td><td>1</td><td>4</td><td>5</td><td>3</td><td>1</td><td>5</td><td>1</td></tr><tr><td>2</td><td>2</td><td>5</td><td>2</td><td>3</td><td>5</td><td>4</td><td>3</td><td>3</td><td>5</td></tr><tr><td>4</td><td>3</td><td>3</td><td>2</td><td>3</td><td>5</td><td>2</td><td>1</td><td>4</td><td>1</td></tr><tr><td>2</td><td>2</td><td>3</td><td>2</td><td>3</td><td>5</td><td>2</td><td>1</td><td>4</td><td>1</td></tr><tr><td>3</td><td>3</td><td>4</td><td>3</td><td>2</td><td>4</td><td>3</td><td>1</td><td>3</td><td>2</td></tr><tr><td>3</td><td>3</td><td>4</td><td>3</td><td>2</td><td>4</td><td>3</td><td>1</td><td>3</td><td>2</td></tr></tbody><tfoot><tr><td>2.333333333</td><td>2.2</td><td>4.533333333</td><td>2.4</td><td>3.333333333</td><td>4.733333333</td><td>3.6</td><td>1.733333333</td><td>4</td><td>1.2</td></tr></tfoot></table>

## 4. CONCLUSION

According to user feedback, during the “Humanverse” participatory performance, user satisfaction compensates for low usability, along with a strong incentive for exploration, creativity and collaboration. Even if the interaction is initially difficult through the functionality of the headset and the Spatial platform, the group dynamics and originality make the experience satisfying and overcome the sense of discomfort. They help to achieve the dramaturgical effect desired by the artist: the hedonic sensations encourage the discovery of the virtual capabilities of a body in a 3D environment, unite the group and push them to explore virtual spaces in an extended way, even flying.

The story aims to make people understand the possibilities and limitations of the virtual dimension by inviting five participants to freely interact with a pre-established ecosystem: overcoming physical flesh barriers, experiencing falling without consequences, and observing their own and others' more-than-human bodies from the outside. The use of the headset better connects the digital body with the physical one, synchronising them at times when movements and actions occur simultaneously in both dimensions. But it also creates glaring discrepancies: to climb the mountain, one has to walk around it; in physical space, on the other hand, one has to move around oneself to move one's avatar and reach the virtual summit. When you jump as if you were on the planet Ceres, with very low gravity, the weight of the gravity still present in physical space gives you a sense of compression and emptiness in your stomach. If one jumps from different planes of virtual height while wearing the headset, the material, the physical body, tends to close in on itself, as if to dampen the imprinted force, to protect itself from the unknown. Realising a performance allows all participants to become aware, present to the circumstances and sensitive to the enunciated reflections. “Humanverse” allows us to explore the narrative, interactive, multisensory and performative potential of the Metaverse; the main objective is to analyse the role and expansion of the body in the digital dimension. An environment takes shape based on the movements of avatars, which are managed

by the viewers via a visor or keyboard. The experience does not focus on the singularity of the individual but on the multiple possibilities of the body in the virtual environment, seen as a receptor of sensations and permeable to interactions.

The whole vision of the Metaverse is still decades away. It could be described as a network of interconnected experiences and applications, devices and products, tools and infrastructures. Nevertheless, the term has become so popular recently because we can feel it beginning. As an evolution of the Internet, the rise of the Metaverse represents a fundamental shift in today's notion of digital presence: quoting Ball, the metaverse is a highly performative, interactive and community-based ecosystem. It is redefining the boundaries between physical and virtual, private and collective, human and more than human towards mass interconnectivity, universal interoperability and persistent synchronicity. Also, in the Metaverse, a performance is developed to translate the concept of hic et nunc into the digital format, providing participants with a constant sensation of presence. This is an example of the method I use to show how artists characterise their creations in the metaverse by emphasising the conceptual model underlying the platform and how this subversion generates dramaturgy. This approach derives from the application of interaction design theories to humanistic studies, which deeply influence epistemologies and practices in the field. To conclude, quoting Hito Steyerl, I would like to emphasise how the metaverse can be seen as a theatrical backdrop that raises both aesthetic and ethical questions. Artistic action is crucial to enable storytelling, interaction and user participation in shaping a non-commercial dynamic outside the clamour of marketing. As Gene Youngblood has argued, in the hands of artists, technologies can be sites of re-socialisation, a γυμνάσιον<fnref n="4" /> for the construction of conscious communities.

<figure><img src="image_4.png" /><figcaption>Picture 2. Selfie on Martin Romeo’s *Humanverse*, online performance on Spatial, 2023. Still from video (courtesy the author).</figcaption></figure>

<footnote n="4">⁴ Gymnasium: exercise ground; primary school.</footnote>
BIBLIOGRAPHY AND WEBOGRAPHY

Auslander, Philip. 2012. "Digital Liveness A Historico-
Philosophical Perspective."

Baía Reis, António, and Mark Ashmore. 2022. "From Video Streaming to Virtual Reality Worlds: An Academic, Reflective, and Creative Study on Live Theatre and Performance in the Metaverse." *International Journal of Performance Arts and Digital Media* 18 (1): 7–28.
https://doi.org/10.1080/14794713.2021.2024398.

Ball, Matthew. 2022. *Metaverso. Cosa Significa, Chi Lo Controllerà e Perché Sta Rivoluzionando Le Nostre Vite*. Milano: Garzanti.

Bazzichelli, Tatiana. 2006. *NETWORKING. La Rete Come Arte*. costa & nolan.

Boccia Artieri, Giovanni. 2023. "OTONI - Oggetti Teatrali Online Non-Identificati." *Connessioni Remote*. *Artivismo_Teatro_Tecnologia*, no. 6 (December).
https://doi.org/10.54103/connessioni/22459.

Diefenbach, Sarah, Nina Kolb, and Marc Hassenzahl. 2014. "The 'hedonic' in Human-Computer Interaction - History, Contributions and Future Research Directions." *Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS*, 305-14.
https://doi.org/10.1145/2598510.2598549.

Dixon, Steve. 2007. *Digital Performance*. The MIT Press.
https://doi.org/10.7551/mitpress/2429.001.0001.

Gaudio, Vincenzo Del. 2020. *Théatron. Verso Una Mediologia Del Teatro e Della Performance*. Melteni.

Gemini, Laura, and Stefano Brilli. 2023. *Gradienti Di Liveness. Performance e Comunicazione Dal Vivo Nei Contesti Mediatizzati*. FrancoAngeli.

Hussain, A, E Mkpojiogu, and M Zabidin Husin. 2021. "Dimensioning UX Models for Design and Evaluation." *Turkish Journal of Computer and Mathematics Education (TURCOMAT)* 12 (3): 1878-83.
https://doi.org/10.17762/turcomat.v12i3.1018.

Law, Effie Lai Chong, Paul Van Schaik, and Virpi Roto. 2014. "Attitudes towards User Experience (UX) Measurement." *International Journal of Human Computer Studies* 72 (6): 526-41.
https://doi.org/10.1016/j.ijhcs.2013.09.006.

Monteverdi, Anna Maria. 2023. *GIACOMO VERDE. Installazioni Interattive, Tv Comunitarie, Net Art, Tecnoteatro (1992-2002)*. Milano: University Press.
https://doi.org/https://doi.org/10.54103/milanoup.134.

Norman, Donald. 2013. *The Design of Everyday Things*. New York: Basic Books.

Patti, Federica. 2024. "The FIFTH WALL. On DIGITAL PERFORMANCE between USER EXPERIENCE DESIGN and The." *Body, Space &

Technology, January.
https://doi.org/10.16995/bst.11230.

Pizzo, Antonio, Vincenzo Lombardo, and Rossella Damiano. 2021. *Interactive Storytelling. Teorie e Pratiche Del Racconto Dagli Ipertesti All'Intelligenza Artificiale*. Audino.

Steyerl, Hito. 2021. "The Fifth Wall." 2021.
https://www.hebbel-am-ufer.de/en/hau3000/hito-steyerl-the-fifth-wall.