# Spheres of Play: Designing Games and Interfaces for Media Architectures

Michael Schmitz, Dominik Scholl, Julian Saraceni, Pascal Klein, Carsten

Blaser, Jorge Olmeda, Soenke Zehle, André Miede

► To cite this version:

Michael Schmitz, Dominik Scholl, Julian Saraceni, Pascal Klein, Carsten Blaser, et al. Spheres of Play: Designing Games and Interfaces for Media Architectures. 14th International Conference on Entertainment Computing (ICEC), Sep 2015, Trondheim, Norway. pp.490-495, 10.1007/978-3-319-24589-8_45. hal-01758445

HAL Id: hal-01758445

https://inria.hal.science/hal-01758445

Submitted on 4 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Spheres of Play:
Designing Games and Interfaces for
Media Architectures

Michael Schmitz<fnref n="2" />, Dominik Scholl<fnref n="1" />, Julian Saraceni<fnref n="1" />, Pascal Klein<fnref n="2" />, Carsten Blaser<fnref n="1" />, Jorge Olmeda<fnref n="1" />, Soenke Zehle<fnref n="2" />, and André Miede<fnref n="1" />

**Abstract.** The paper describes a game-based interaction scenario around an existing media architecture, developed to integrate aesthetic, social, and technological dynamics. On screen, the game unfolded as users moved across a globe, using a spherical input device to direct their avatars across a dynamic world of obstacles. Recalling the singularity and site-specificity of a performative intervention, the multidisciplinary project is part of a larger research effort that explores the use of media facades as an infras- tructural core of complex interfaces for multiple forms of engagement and the co-creation of transmedial scenarios.

**Keywords:** media facade, urban HCI, user interfaces, game design

# 1 Introduction and Motivation

Part of the trend toward ambient media, the permanent integration of displays into buildings has facilitated the emergence of new aesthetic practices that combine visual media with a wide range of interaction technologies [3,6]. In these experiments, urban screens are no longer used simply as video walls, but as an infrastructural core of complex interfaces allowing for multiple forms of engagement and interaction [8]. The site of interaction is no longer the screen alone but the hybrid space created through the combination of a (permanent) media architecture and (temporary) forms of use, sometimes in combination with tangible interfaces to facilitate intuitive forms of interaction across a heterogeneous audience. Such approaches create scenarios not usually associated with digital media: rather than lossless reproduction and infinite replay across an anonymous network of users, these collaborative scenarios recall the singularity and site-specificity of a performative intervention. As interface design strategies have to take this into account, they become increasingly multidisciplinary: the focus of design is not only the “content” to be displayed on a screen but the creation of a hybrid situation integrating aesthetic, social, and technological dynamics.

The fundamental principle of tangible user interfaces (TUIs), the tangibility of the interface and the material form of a digital resource [5], have been applied to media facades in a few prototypes [2,4]. This approach maintains interaction

<footnote n="1">¹ htw saar, Systemtechniklabor, Saarbrücken, Germany, http://www.htwsaar.de</footnote>

<footnote n="2">² HBKsaar, xm:lab, Saarbrücken, Germany, http://www.xmlab.org</footnote>
with our everyday environment and situates the interface into real world context. This has the effect that users are enabled to utilise their experiences with the real world, the threshold for activity is lowered and the access bottleneck of common input devices bypassed (see e. g., [7]).

This paper presents an experimental game-based scenario that integrates a TUI to create such a hybrid scenario around an existing media architecture. On screen, the game unfolded as users move across a globe, using a spherical input device to direct their avatars across a dynamic world of obstacles. The input device (Sphero) was chosen because its design mimics the design of the game world, and its features and functionality allowed intuitive interaction.

The multidisciplinary design team included members from two research organizations: xm:lab (Experimental Media Lab) is an institute of the Academy of Fine Arts Saar (HBKsaar) – combining the autonomous aesthetic practices of art and design with research in information technologies,xm:lab develops and coordinates a wide spectrum of research projects at the intersections of art, design and technology. Saarland University of Applied Sciences (htw saar) is a research- and application-oriented university with strong regional ties, e. g., in the information and communication technology sector.

The following section of this paper describes the general technological and game design choices and outlines the implementation. The paper closes with a summary and general conclusions in Section 3.

## 2 Concept and Implementation

***Technical Background:*** The HBKsaar media facade is a back projection screen, operated with five projectors inside the building. Accordingly, the whole canvas is subdivided into five smaller ones, where every small canvas covers a single window. In our setup, a single projector is used for every window, such that five projectors in total cover the whole media facade.

When looking at the enclosed edge (see Figure 1), a spectator gets the impression of volumetrically spreading 3D objects, which seem to be caught within the room confined by the projection canvases. The projectors have a resolution of 800 × 600 pixels each, summing up to a bit less than 4000 × 600 pixels, since some parts are covered at the shorter side (see Figure 1a). The projectors are connected to a single PC, which maps the computer desktop to a 3 × 2 grid (as seen in Figure 1b). The sixth part of the grid is fed to a monitor inside to facilitate the configuration of the system without interrupting the media facade output.

As a result of the back projection, the displayed content appears mirrored from outside of the building – thus, the render output must be mirrored by the active applications. There are different ways to communicate with the media facade. Its host is connected to the university's local network with access to the Internet. There is also a Wi-Fi hotspot with access to the university's network in front of the building. For this project, a Bluetooth dongle was installed at the roof of the building.
<figure><img src="image_2.png" /><figcaption>Fig. 1: How the render output is put on the media facade</figcaption></figure>

## 2.1 Game Concept

In this section we focus on game design decisions specific to the media facade and do not discuss the general game design in detail. A major challenge when designing a game for our media facade was to find a concept that fits to the particular perspective, which is required for the 3D effect to work. The viewer has to stand at a specific position across the street in order to see the projection in an optimal way. Due to the fact, that the projection screen is located on the first floor of the building, the viewer is looking upward to the stage, which poses certain constraints to the 3D scene displayed at the facade, such that it is e.g. impossible to see the ground floor of the virtual space. This makes it also impossible to see the feet or the legs of a character walking inside this space. Additionally, the front bottom corner of the building partly covers the back walls of the virtual room. The only locations where a possible character would entirely be visible, would be nearby the windows of the building or in the center of the room some way above the floor. Another challenge is the placement of objects or obstacles that appear in a game. However, these are the requirements for creating a challenging game and the major source of interaction and fun.

A possible solution to the challenge of the limited view is to use the top and the back walls of the room as mirrors. Obviously, this is rather complicated for the player, since she would have to transfer/mirror the control commands.
<figure><img src="image_3.png" /><figcaption>Fig. 2: The controllable character is placed at the center of the screen</figcaption></figure>

We also considered to drop the 3D effect and to implement a 2D side-scrolling game including the edge or to use just the left, single screen for playing, while using the right and smaller part to display additional information. However, this would disregard the main characteristics of the media facade, which should be a central element of this installation.

The final solution we choose, was placing the character in a *fixed* position at the center of the screen as shown in Figure 2. For the game world, we chose the shape of a sphere, which resulted in two major benefits:

1. Due to the fixed camera position, it looks like the character is standing still and the world is moving. In combination with the sphere shaped world and the Sphero as an input device, we can transfer the way the player rotates the Sphero to the rotation of the game world, which yields a very simple and intuitive interaction metaphor.

2. All relevant parts of the sphere fit into the virtual space: The player sees only the part of the world, which is directly in front of the character, the rest of the world is hidden behind the horizon. This simplifies the scene and also simplifies the game by giving the player a better overview of the world.

## 2.2 Implementation

The implementation of the game was realized with Unity<fnref n="3" />, a game development platform including a 3D game engine, scripting and animation environment. It allows to develop games for many common platforms, including Windows, Linux, Android, and game consoles. It is also often used for the development interactive art installations, such as virtual or augmented reality applications.

From a technical point of view, the game is structured in just a few main components. As shown in Figure 3, there is a central “GameMain” component, which initializes the game, handles the game progress using some utility classes

<footnote n="3">³ http://www.unity3d.com</footnote>
<figure><img src="image_4.png" /><figcaption>Fig. 3: Overview of the game architecture</figcaption></figure>

and holds some general information such as the current score. During initialization, there is also a switch to differentiate between the development mode on PC and the production mode for the media facade.

The game flow consists of several scenes. Each kind of scene is represented by a *GameplayMode*. Such a *GameplayMode* can be used in multiple scenes, i. e., the actual levels. There are the following modes:

1. "Waiting" mode, which starts a new game as soon as it recognizes user input.

2. "NameInput" mode, where the player can enter her name (with three digits).

3. "Scoreboard" mode, which shows the high scores after each level.

4. "Level" modes, which contain the actual game play.

The Sphero<fnref n="4" /> ball contains both sensors and actors (motors), such that it can sense its orientation and move independently on the ground. It can connect via Bluetooth to the host PC controlling the rendering on the media facade. There is a documented API for the Sphero with several SDKs including one for Unity3D. Unfortunately, these are designed to control the Sphero from mobile devices. The other way around, reading Sphero's sensor data for input purposes is poorly supported and documented, e. g., the APIs, including the Unity plugin, only work with mobile devices and are not usable on a Windows PC. Therefore, we had to develop new tools to read the Sphero's streamed sensor data and map the values to a set of keyboard commands. The game maps these input commands into one of seven possible horizontal rotation values (center and three steps for left and right each). For the user, this creates the illusion of analog control in order to achieve an important design goal: controlling a spherical world using a spherical input device. Due to the space constraints of this paper, further technical details are omitted.

<footnote n="4">⁴ http://www.gosphero.com/sphero/</footnote>
# 3 Summary and Outlook

The scenario developed in this project exemplifies our cross-institutional approach to designing games and interfaces for media architectures. We conduct these experiments as part of a broader inquiry into the future of immersion, with a particular emphasis on immersive storytelling and transmedial scenarios that unfold across platforms. This kind of research not only requires a multidisciplinary methodology but the use of complementary research infrastructures – some of the experiments we conduct on the media facade, some in the media theater (immersion), and some in the green box using a markerless motion capture system. As we see it, the future of media experience design will have to take the becoming-ambient of display and interaction technologies into account; users will expect games and stories that are both situation-specific and unfold across devices. Our iterative research process links individual experiments conducted in different settings and allows us to compare and contrast the ways in which the use of different media architectures affects the aesthetic practices involved, from interface design to character development.

Future projects include collaborative research on game-based transmedial scenario development with xm:lab partners from the Connecting Cities Network, an international network of cultural organizations working with media architectures [1]. In this context, the academy's media facade serves both as a local infrastructure and as a node in an international network of arts-and-technology research.

# References

1. Connecting Cities Network, http://www.connectingcities.net

2. Boring, S., Gehring, S., Wiethoff, A., Blöckner, A.M., Schöning, J., Butz, A.: Multi-user Interaction on Media Facades Through Live Video on Mobile Devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2721–2724. CHI ’11, ACM, New York, NY, USA (2011)

3. Brynskov, M., Tscherteu, G.: Media Architecture Compendium, Vienna: Media Architecture Institute (2014), http://catalog.mediaarchitecture.org

4. Fischer, P.T., Zöllner, C., Hoffmann, T. and Piatza, S.: VR/Urban: SMSlingshot. In: Proceedings of TEI’10. pp. 381–382. ACM (2010)

5. Hornecker, E., Buur, J.: Getting a grip on tangible interaction: a framework on physical space and social interaction. In: CHI ’06: Proceedings of the SIGCHI conference on Human Factors in computing systems. pp. 437–446. ACM, New York, NY, USA (2006)

6. Schoch, O.: My building is my display. In: Proc. eCAAD’06 (2006)

7. Shaer, O., Hornecker, E.: Tangible user interfaces: past, present, and future directions. Foundations and Trends in Human-Computer Interaction 3(1–2), 1–137 (2010)

8. Struppek M.: Urban Screens – The Urbane Potential of Public Screens for Interaction. In: “Screenarcadia” – Microwave – International New Media Arts Festival, festival book. Hong Kong (2010)