# Player Control in a Real-Time Mobile Augmented Reality Game

Mareike Picklum, Georg Modzelewski, Susanne Knoop, Toke Lichtenberg, Philipp Dittmann, Tammo Böhme, Volker Fehn, Christian John, Johannes Kenkel, Philipp Krieter, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Mareike Picklum, Georg Modzelewski, Susanne Knoop, Toke Lichtenberg, Philipp Dittmann, et al. Player Control in a Real-Time Mobile Augmented Reality Game. 11th International Conference on Entertainment Computing (ICEC), Sep 2012, Bremen, Germany. pp.393-396, 10.1007/978-3-642-33542-6_36. [https://inria.hal.science/hal-01556179](https://inria.hal.science/hal-01556179)

HAL Id: hal-01556179

https://inria.hal.science/hal-01556179

Submitted on 4 Jul 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
Player Control In A Real-Time Mobile
Augmented Reality Game

M. Picklum, G. Modzelewski, S. Knoop, T. Lichtenberg, P. Dittmann,
T. Böhme, V. Fehn, C. John, J. Kenkel, P. Krieter, P. Niethen, N. Pampuch,
M. Schnelle, Y. Schwarte, S. Stark, A. Steenbergen, M. Stehr, H. Wielenberg,
M. Yildirim, C. Yüzüncü, F. Pollmann, D. Wenig, and R. Malaka

Research Group Digital Media, TZI,
University of Bremen, Bibliothekstr. 1, 28359 Bremen, Germany
Projekt Movirwelt
http://movir.informatik.uni-bremen.de

**Abstract.** Controlling virtual characters in AR games for modern smartphones is even more challenging than controls for 'pure' VR games because the player has to keep the AR world in view. We propose six interaction concepts based on combinations of both physical and virtual buttons and sensor input and suggest an evaluation according to game experience criteria.

**Keywords:** mobile gaming, augmented reality, AR, user interaction, gestures

# 1 Introduction

On smartphones, controlling characters in augmented reality games has a lot in common with touch-based games but there is an additional problem: To receive visual feedback of his actions the player needs to keep the AR world in view, so one is not entirely free to choose his device's position.

Within a students' project we developed a real-time mobile augmented reality game in which two virtual characters fight against each other. Each player controls a character in the AR world, which is created by having both devices track the same marker (Fig. 1) and communicate wirelessly. While each player can see both characters, one only controls his own. Additional information about the current game state as well as buttons to initiate actions are displayed on the screen (Fig. 1). The player can control the avatar's movement and initiate primary and secondary attacks. He is also able to block the enemy's attacks. Primary attacks have a short range, e.g. kicks or punches while secondary attacks can be long ranged e.g. fireballs. The movement speed of the character is limited to prevent the avatar from moving instantly to the point the user clicked on.

In this paper we suggest different input options for the best game experience in our AR game and propose further research to evaluate them.
## 2 Related Work

Hürst and van Wezel [1] developed a system in which the user's own hands or fingers are used to interact with the 3D objects. Avatars can be moved or knocked over by using the finger behind the device to 'push' the models instead of touching the display. Usability tests showed that users interacting with objects while holding a mobile device with one hand and moving the other one in front of the camera results '... in an awkward hand position or even forces people to switch the hand in which they hold the device.' Harviainen et al. [2] used camera movement and accelerometers to control virtual characters. In one example an animated model of a dog reacts to the camera movement, while another implementation uses the accelerometer to detect shaking or tilting of the phone. Gu et al. [3] developed a game similar to ours and used the accelerometer to move a character and touches on the screen to initiate actions. They did not evaluate the effects of different input options on the game experience. Calvillo-Gámez et al. [4] developed a questionnaire to measure game experience.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** left: AR mockup; middle: actual prototype; right: screen capture of the prototype. a) player/device 1, b) player/device 2, c) character of player 1, d) character of player 2, e) AR marker, f) GUI buttons, g) AR arena</figcaption></figure>

## 3 Interaction concepts

When developing mobile interaction concepts several factors need to be considered to provide the best user experience. Often the screen is both input and output device. The more control elements are placed on the screen the higher is the risk of occluding important parts of the game with the user's fingers. This might have a negative effect on the game experience and make it difficult to control it in a real-time environment. As the touch screen provides no inherent haptic feedback, the user has to actively make sure his fingers are in contact with the displayed control elements which lowers the immersion by forcing him to focus not on game content but on input modalities. An additional difficulty in AR games on smartphones is to permanently keep the tracked marker in the field of view of the device's camera.

The control of character movement in a game can be relative, with respect to the position of the player, or absolute, with respect to absolute coordinates of the
AR world. Absolute control means that the player sets the goal of the movement and the avatar moves to the destination by itself without further interaction of the player. For continuous control the input is read in each update cycle of the game, forcing the player to give continuous input until the desired position is reached. Discrete input starts a perpetual movement until it is interrupted by further user input or the destination is reached. To improve user experience, feedback can be provided, e.g. to notify the player if his fingers are no longer in contact with the control elements on the touch screen or if the marker can no longer be tracked. While this feedback could use all output channels of the device it is important that the user can distinguish between the provided input feedback and the regular game output at any time. Feedback can be provided using vibration, audio signals and visual hints like a flashing screen or a some kind of in-game feedback of the character (e.g. stumbling).

The following concepts concentrate on controlling the movement of the char-
acter. Primary and secondary attacks as well as blocking are initiated by touch-
ing buttons displayed on the screen. The concepts focus on the touch screen and
other built-in sensors of the devices.

**Touch-based virtual joystick** Some mobile games use a concept similar to a joystick but without the need for the actual hardware device. A circle is drawn on the corner of the screen and the position of a touch in relation to the center of the circle defines the position of the virtual joystick. It is usually used with a thumb. Virtual joysticks provide movement control relative to the character's position. They require the continuous and correct placement of the player's finger on the control element, occluding only a small part of the screen. Feedback can be provided if the finger is no longer touching the control element.

**Touch-gesture control** Swipe gestures are well established to interact with smartphones. They are used to initiate the movement of the character in the indicated direction. Tapping on the screen stops the movement.

**Physical control buttons** Some Android devices like the Motorola Milestone have physical keyboards which can be used to move the player model relatively to its current position with a set of predefined keys, analogous to many desktop games. This easy and intuitively understandable technique is established in many other applications. As most of the common devices currently on the market do not have a physical keyboard integrated, this concept is not suitable in general but provides the best haptic feedback.

**Touch-based absolute control** The user sets the location in the AR world by tapping on a point on the screen, occluding it only for a short period of time. The destination is set in the AR world, so moving the device does not change the target position. Depending on the distance between the current and the desired position of the avatar it might take some time before the movement is finished.
It could be helpful to place a visual hint such as a flag or cross-hair in the AR
world to remind the player about the current destination.

**Field-of-view-based continuous control** Instead of requiring the user to select a specific point, the destination is continuously updated to the center of the camera's current field of view. Moving the device and thereby the AR viewport allows the player to control the character. This is even true if the player does not intend to move the character. An alternative to the continuous updating could be realized by pressing a button to set the new destination. Again a visual hint can be helpful to indicate the current movement target.

**Physical gestures** Modern smartphones offer additional sensors like gyroscopes or accelerometers which can be used as input in games. They react to movements like tilting, rotating or shaking of the device. These gestures would also change the viewport on the AR world because the camera is fixed on the device. Further research has to show if the accompanying viewport change has a detrimental effect on the game experience in a real-time game.

## 4 Conclusion & Future Work

We presented several possible concepts for player control in mobile AR scenarios. Using visual, auditive or haptic feedback might mitigate the shortcomings of some of these concepts. Prototypical implementations will be used to conduct further game experience studies.

## References

1. Hürst, W., Wezel, C.: 15. In: Multimodal Interaction Concepts for Mobile Augmented Reality Applications. Volume 6524 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg, Berlin, Heidelberg (2011) 157–167

2. Harviainen, T., Korkalo, O., Woodward, C.: Camera-based interactions for augmented reality. In: Proceedings of the International Conference on Advances in Computer Entertainment Technology. ACE ’09, New York, NY, USA, ACM (2009) 307–310

3. Gu, J., Duh, H.B., Kitazawa, S.: 7. In: A Platform for Mobile Collaborative Augmented Reality Game: A Case Study of ” AR Fighter”. Springer New York, New York, NY (2011) 99–108

4. Calvillo-Gámez, E.H., Cairns, P., Cox, A.L.: 4. Human–Computer Interaction Series. In: Assessing the Core Elements of the Gaming Experience. Springer London, London (2010) 47–71