Aesthetic Perspectives on Computational Media Design

Rui Craveirinha, Luís Lucas Pereira, Mariana Seiça, Licínio Roque

► To cite this version:

Rui Craveirinha, Luís Lucas Pereira, Mariana Seiça, Licínio Roque. Aesthetic Perspectives on Com-
putational Media Design. 20th International Conference on Entertainment Computing (ICEC), Nov
2021, Coimbra, Portugal. pp.482-488, 10.1007/978-3-030-89394-1_42 . hal-04144402

HAL Id: hal-04144402

https://inria.hal.science/hal-04144402

Submitted on 28 Jun 2023

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Aesthetic Perspectives on Computational Media Design

Rui Craveirinha[0000-0002-8017-489X], Luís Lucas Pereira[0000-0002-3161-6495], Mariana Seiça[0000-0001-5450-5459], and Licínio Roque[0000-0002-1911-2788]

University of Coimbra

Centre for Informatics and Systems of the University of Coimbra

Department of Informatics Engineering

rui.craveirinha@gmail.com, {lpereira, marianac, lir}@dei.uc.pt

**Abstract.** The main goal of this workshop is to open a collaborative space for reflection and debate on how Aesthetics can inform the design of computational media, in an attempt to grasp the multiplicity of aesthetic dimensions that can influence the reception, experience, and interpretation of such sociotechnical objects and experiences. We will approach the goal from the entertainment computing field where such concerns are of vital influence.

**Keywords:** Aesthetics · Computational Media Design · Digital Media · User Experience

## 1 Introduction

Computer technologies have entered our everyday lives as structural tools for our professional, personal and social dimensions. Society has become an information society, as the need to grasp, apprehend and manage information has inevitably entered our daily routines. It is this process for creating, changing and internalising this constant info-stimulus that Manovich [5] called info-aesthetics, in a “global information society (...) where the use of computers for design and production gives rise to new cultural forms”.

Aesthetics emerged as a relevant field to help us understand how we can creatively and innovatively expand universes of experience in a new digital world. But the plethora of dimensions in Aesthetics has centuries of expanding concepts and diverse perspectives. From the Greek *aisthētikós*, which means “something of or for the perception of the senses”, we will take it here broadly, to include any contribution relevant to understand how we perceive objects and their context, but also how they influence our *being-in-the-world* [14].

Notions of beauty and aesthetics have been intrinsically linked with long-held views of what constitutes an artistic object and an aesthetic experience. For example, an artistic object can be somewhat simplistically defined by its capacity to move subjects emotionally and prompt them to reflect critically on a given subject [3]. Yet, such aesthetic experiences are not only sought out in the context of artistic object fruition. Just as a film, or a painting, can be praised for its
ability to convey emotion and sprout critical reflection – for being beautiful, for being artistic, for its aestheticness – so can a TED talk on the impending climate crisis, a play from the world's top football player, a video game about the loss of a loved one, a medium article on the rise of new authoritarianism, a university's lecture on identity discrimination, a civil rights discussion on Twitter, a Facebook story showing a wedding proposal, a viral cat meme, a face swapping photo app, an algorithm that generates poetry and music, etc, etc, etc... all these things can move us, touch us emotionally, make us think, change our lives. They can be aesthetic; they can be beautiful.

Understanding the nature of this elusive aesthetic value, how it can be conceptualised, studied, and nurtured within digital media creation practices is therefore of the utmost relevance. What are the new Aesthetics? How does digital media's interactive nature shape them? What exquisite new shapes do Aesthetics take in the digital space? What objects do they encompass? And how can Aesthetics elevate our relationships with computational artefacts? With this workshop, we intend to reflect upon this challenges, resulting in a collective research manifest submission to a journal such as Entertainment Computing.

## 2 Plurality of Perspectives

With this workshop, we will explore several perspectives on how aesthetics can inform the design of computational media artefacts/experiences:

A. How can its experience shape subjective perceptions of digital artifacts;

B. How can it inform the process, tools and models used in designing digital artifacts;

C. How can it offer a lens to both conceptualise and evaluate the subjective experience with digital artifacts.

These are just the perspectives we have studied in the past, and wish to expand upon in the future. We will start with three examples of perspectives we pursued in our recent work. With these, we aim to illustrate and invite others to bring in their research contributions and unique perspectives on how aesthetics can inform the design of computational media.

### 2.1 Perspective A

Seiça et al. [12] propose an alternative view over Aesthetics applied to Sonification, rooted in the foundational applications in HCI of the human as a bodily, multisensorial agent. Driven by the notion of aesthetic experience that may offer an expansion of the perception field for the experiencer, it is based on a model proposed by Pelowski & Akiba [6] describing a 5-stage cognitive flow that leads to self-transformation. The differing piece of this model is the need for discrepancy to occur: in a circular process of understanding-evaluating the artefact, it rises when the comparison between what is expected and what is perceived
fails. This failure raises doubts and uncertainties that demands a reevaluation
of her own expectations and former knowledge, inviting her to look outside the
problem and out of herself. This is the transformational stage that leads to the
essence of an aesthetic experience, from which a new, expanded self emerges.

The notion of expanded self is built over Merleau-Ponty's statement that “per-
ception requires action” [14]. To fully grasp the behaviour and inherent mean-
ing(s) of an artefact, one must look, listen, touch, embody and interact with it
through a sequence of interpretive exchanges. As a conceptual space for this to
occur, systemism [1] and the living systems theory emerged with Capra's six per-
ceptual shifts [2] for designing sustainable societies based on nature's ecosystems.
We propose an alternative to classical sonification, grounded on the concept of
a sonification as a living system, designated as a systemic sonification.

A systemic sonification is regarded as a dynamic, ever-evolving system, com-
posed by an ensemble of intertwined sound beings who actively respond to
changes in its environment and interactions with surrounding elements. These
constant exchanges determine the course of evolution of the system, with which
each human can engage with and weave his/her own experience. The intertwined
bonds born in these interactions allow for unique experiences to arise for each
human, surpassing initial expectations, enabling perceptions and insights over
the dataset and, ultimately, an internal transcendental process which offers an
aesthetic experience of sonification. The experience itself becomes the purpose of
a systemic sonification, and the exploration of emergent elements and sonic uni-
verses that adapt and transform to each participant, while being transformed
as well with each interchange. Aesthetics embodies the role of an entity that
drives the multiple acoustic journeys for each individual, offering the expansion
of perceptual horizons and consciousness of each participant.

## 2.2 Perspective B

The Participation-centered Gameplay Experience Model [7] consists of both a
player experience model and a set of canvases intent on supporting the game de-
sign activity. While competing theories focus on psychological motivation or how
games can be designed to drive engagement or stimulate hedonic experiences,
this model sought to assist designers in conceptualising gameplay as a form of
player participation.

Inspired by postmodern artistic and poetic practices, the player is conceived
as an active, meaning-making participant in the experience. Games are framed
as artifacts, networks of mediators that enable said participation. Designers were
then asked to think, formulate and plan out their design in the form of intentions
for player participation. Once expressed, these intentions could guide the creation
of the artifact and its qualities.

To assist in that process, six participatory dimensions were outlined, 3 of
which being inherently aesthetic in their nature:

1. **Sensoriality:** the videogame as a context of multisensory involvement; par-
ticipation as feeling, perceiving, contemplation, sensorial expression, won-
dering, etc.
2. **Embodiment**: the videogame as a context of physical participation, both virtual and real; participation as physical performance, physical involvement, physical coordination, movement, dancing, etc.

3. **Sensemaking**: the videogame as a context of significant participation, of creation of meaning; participation as interpretation, understanding, role-playing, self-expression, critical-thinking, etc.

In trying to support the education of a new generation of designers so as to leave the comfort zone of familiar objects, the model's inspiration in artistic practices helped reframe pre-existing assumptions of what constitutes a game and its constituting elements (as that might constrain the designer's creativity) and thus push for the creation of imminently aesthetic artifacts. Aesthetics was, in that sense, both premise and goal for the model. Several design exercises were carried out [10, 9, 8] with the model and its support tools, resulting in innovative artifacts and aesthetic experience.

## 2.3 Perspective C

In [4], the authors set out to study how the presence of traditional game rules could impact the perceived artistic valuation of a videogame experience. Aesthetic theories provided both grounds for the research question, and the basis for a player self-report method for evaluating player experience, beyond hedonic facets of player experience. This research project ended up testing (among other things) if Aesthetic Theories of Art – which define art in terms of the attitude objects elicit in their audience – had any empirical basis, when applied to a specific videogame.

‘Aesthetic’ theories postulate that object perception is shaped by the function we attribute to them. They conceive of two different attitudes: the aesthetic and the practical. When engaging in a practical attitude, our attention swerves from the object towards our goal of use – “I see a pen as something I can write with”[13]. Conversely, if we are engaged in an aesthetic attitude, we pay attention to the object only, in disinterested and complacent manner, to “enjoy its visual features, the way it sounds, or feels to the touch”[13]; and only in those cases can an aesthetic experience emerge.

To verify these theories merits in the context of video game appreciation, the authors created two versions of renowned interactive art-game experience Passage [11], one which gave players explicit tasks to achieve (to stimulate a practical attitude), and one with no such goals (to stimulate an aesthetic attitude). A self-report based on cluster art theories was used to assess players’ aesthetic experience.

Results showed that the perception of the video game's meaning decreased in the practical version of the game, suggesting that aesthetic theories of art had at least some empirical confirmation: that the perception of meaning was dependent on the nature of the task the user was involved in. Thus, for an experience to be deemed aesthetic, users may need to approach it with an aesthetic attitude, a “contemplation of the object for its own sake”.
## 3 Process (at the workshop and beyond)

The overall goal of this process is to prepare a collective research manifest sub-mission to a major journal such as the Entertainment Computing, to be drafted from contributions at the workshop and subsequent interactions. As a collaboratively created research manifest, it will feature an integrative reflection with new questions for a renewed research program in computational media design. Participants are invited to submit a positional statement or case study of their own research practice into how aesthetics impacted digital artifact creation in their research practice. They will be invited to give a short introduction of their position as preparation for the debate during the workshop. For the expected duration, unless we receive a great volume of contributions, we expect that 4h should be enough to run through the meeting process (step 2 below).

1. *Call for and Gather Contributions:*

(a) Individual presentations of each participant, covering their definition of aesthetics and research influence;

2. *Group Debate of Contributions:*

(a) Discussion among groups of participants lead by questions designed to spark group interaction and mapping of bridges between research strands;

(b) Summary of group work and joint debate of positions presented;

(c) Draft from contributions and emergent questions for the production of a submission draft;

3. *Aggregate and Update Perspective:*

(a) Collective update of an aesthetics framework within the field and how it can influence digital artifact creation, experience and research;

4. *Propose Integration of Perspectives:*

(a) Outline research avenues for the future based on the framework;

## 4 Profile of participants

Intended participants will be any researcher or practitioner interested in the role of aesthetics in computational media design and experience. Participants interested in contributing with an aesthetic perspective should make the organizers aware of that intention by submitting a position paper in the workshop call.

## 5 Proponents Info

**Common Affiliation:** University of Coimbra, Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering

**Common Address:** Pólo II - Pinhal de Marrocos, 3030-290 Coimbra
**5.1 Rui Craveirinha**

**Email:** rui.craveirinha@gmail.com

**Biography:** Rui Craveirinha is a Games User Researcher at Player Research. He has lectured HCI, Game Design and Game Criticism courses at the University level. His research interests cover both the study of artistic experiences in video games and conceptualising authorial approaches to procedural content generation in games. He also loves cats and memes, which is to say, he loves cats twice.

**5.2 Luís Pereira**

**Email:** lpereira@dei.uc.pt

**Biography:** Luís Pereira is pursuing his PhD studies in the field of HCI, focusing on the notion of participation as a key perspective in game design. Having developed an hexadimensional model of participation from which he created and tested a design canvas, his work tries to address game design from an aesthetic perspective, rather than the more usual object mechanics. Luís is also a lecturer in interactive multimedia project courses with experience in multimedia development processes.

**5.3 Mariana Seiça**

**Email:** marianac@dei.uc.pt

**Biography:** Mariana Seiça is an audiovisual designer, with a Master's Degree in Design and Multimedia from the University of Coimbra. She is currently pursuing a PhD in Information Science and Technology, studying the influence of aesthetics on interactive systemic sonifications. Her research and artistic practices focus on sound as a communication tool, embodying her enthusiasm for musical forms that travel between the fields of Sonification, Sound Design, Human Computer Interaction and Computational Music.

**5.4 Licínio Roque**

**Email:** lir@dei.uc.pt

**Biography:** Licínio Roque is an Associate Professor, coordinator of the CISUC Information Systems group and Computational Media Design PhD Program. Current research focuses on Participatory Media, mainly the design and evaluation of shared computational media objects to empower creative collaborations. He was principal researcher on various research/industry projects, and has an extensive methodological and training experience in the areas of HCI, SE, ISD, Interaction Design, Game Design and related technologies. He was also an Associate Professor at Carnegie Mellon University, visiting professor at Aarhus University's CMA group, co-Founder of SPCVIDEOJOGOS and Tapestry Software.
References

1. Bunge, M.: Emergence and convergence: Qualitative novelty and the unity of knowledge. University of Toronto Press (2003)

2. Capra, F.: Speaking nature's language: Principles for sustainability. Ecological literacy: Educating our children for a sustainable world pp. 18–29 (2005)

3. Carroll, N.: Philosophy of Art: A Contemporary Introduction. Routledge (1999)

4. Craveirinha, R., Roque, L.: Impact of game elements in players artistic experience. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. p. 1–6. CHI EA ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3290607.3313049, https://doi.org/10.1145/3290607.3313049

5. Manovich, L.: Introduction to info-aesthetics. p. 333–344 (2008), https://doi.org/10.1215/9780822389330-022

6. Pelowski, M., Akiba, F.: A model of art perception, evaluation and emotion in transformative aesthetic experience. New Ideas in Psychology **29**(2), 80–97 (2011)

7. Pereira, L.L., Craveirinha, R., Roque, L.: A canvas for participation-centered game design. In: Proceedings of the Annual Symposium on Computer-Human Interaction in Play. p. 521–532. CHI PLAY ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3311350.3347154, https://doi.org/10.1145/3311350.3347154

8. Pereira, L.L., Portela, M., Roque, L.: Machines of disquiet: Textual experience in the Idod archive. MATLIT: Materialidades Da Literatura **6**(3), 59–71 (2018)

9. Pereira, L.L., Roque, L.: Fátima postmortem. Online-Heidelberg Journal of Religions on the Internet **5** (2014)

10. Pires, D., Furtado, B., Carregã, T., Reis, L., Pereira, L.L., Craveirinha, R., Roque, L.: The blindfold soundscape game: A case for participation-centered gameplay experience design and evaluation. In: Proceedings of the 8th Audio Mostly Conference. AM ’13, Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2544114.2544122, https://doi.org/10.1145/2544114.2544122

11. Rohrer, J.: *Passage*. Game [PC] (2007)

12. Seiça, M., Roque, L., Martins, P., Cardoso, F.A.: A systemic perspective for sonification aesthetics. In: Proceedings of the 26th International Conference on Auditorary Display (ICAD 2021). Georgia Institute of Technology (2021)

13. Stolnitz, J.: A atitude est?tica. In: D’Orey, C. (ed.) O que ? a Arte, A Perspectiva Anal?tica. Carmo d’Orey e Dinalivro (2007)

14. Svanaes, D.: Interaction design for and with the lived body: Some implications of merleau-ponty’s phenomenology. ACM Transactions on Computer-Human Interaction (TOCHI) **20**(1), 8 (2013)