<figure><img src="image_1.png" /></figure>

HAL
open science

# WAR Bots: Combining Virtual and Augmented Realities for an Immersive and Enjoyable Gaming Experience

Jaryd Urbani, Mohammed Al-Sada, Shubhankar Ranade, Mingshu Zhang, Tatsuo Nakajima

<figure><img src="image_2.png" /></figure>

## To cite this version:

Jaryd Urbani, Mohammed Al-Sada, Shubhankar Ranade, Mingshu Zhang, Tatsuo Nakajima. WAR Bots: Combining Virtual and Augmented Realities for an Immersive and Enjoyable Gaming Experience. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.379-382, 10.1007/978-3-319-66715-7_44. hal-01771269

HAL Id: hal-01771269

https://inria.hal.science/hal-01771269

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# WAR Bots: Combining Virtual and Augmented Realities for an Immersive and Enjoyable Gaming Experience.

Jaryd Urbani\*¹, Mohammed Al-Sada¹, Shubhankar Ranade¹, Mingshu Zhang¹ and
Tatsuo Nakajima¹

¹Department of Computer Science and Engineering
Waseda University, Tokyo, Japan
{Jaryd, Alsada, Shubi, Momochi, Tatsuo}@dcl.cs.waseda.ac.jp

**Abstract.** Despite the popularity of Augmented and Virtual Reality within gaming, the full potential of such technologies is yet to be fully taken advantage of. Thus, we introduce WAR Bots, a cyber-physical game which intends to investigate the use of both Augmented and Virtual Reality within a coherent gaming experience. We present the underlying architecture to realize our approach, and accordingly, implemented our proposed method in a multiplayer game. Finally, we present the future direction of our approach.

**Keywords:** Mixed Reality · Augmented Reality · Virtual Reality

## 1 Introduction

Nowadays, Virtual and Augmented Reality Head Mounted Displays (VR/AR HMDs) are more popular than ever due to their relative high quality and affordability. However, despite the wide availability of such technology, it is still in its infancy when it comes to gaming. For instance, most VR/AR games attempt to focus on a singular experience, such as the sole use of VR or AR, with limited interaction methods. While AR enables adding of digital contents to a real world's visual view, VR enables immersive virtual environments and experiences that engage multiple senses. Taking advantage of both VR and AR within a gaming context has high potential to deliver a gaming experience that leverages the advances of both interaction mediums. Thus, we take the first steps to investigate a gaming experience that attempts to combine elements from both VR and AR.

We introduce WAR Bots, a multiplayer battle game where users pilot a physical robot vehicle from within a virtual cockpit in first person. We discuss various design elements and explain how such experience is delivered to players. Finally, we present our future direction of this project.
## 2 Related Work

Previous work and projects have investigated the use of either AR or VR separately to enhance the gaming experience. "AR Drone"[2] and "AR Quake"[3,4], have introduced AR based gaming elements to piloting drones. Players can engage in a drone fight using the drones equipped cameras and sensors. The drone's camera feed, viewed from a smartphone, is used to visualize missiles and opponents as well as other match related contents.

## 3 Approach

<figure><img src="image_4.png" /><figcaption>Fig. 1. Player's view in cockpit, with live camera feed and AR contents in the middle.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 2. Left and Right views of the cockpit as seen through the HMD.</figcaption></figure>

Improving on previous works, WAR Bots emphasizes the engagement of both VR and AR within a gaming experience for controlling robots. We extend each medium in the following method:

VR is utilized to immerse the player within the game's vehicle cockpit (see Fig. 1&2). Here users are able to naturally view the vehicle cockpit, within which we are able to introduce interaction methods to enhance the player's immersiveness and fun. Accordingly, this method enables flexible implementation of various interaction techniques and play mechanics that would suit various game types.
AR is used to extend the robot's live camera feed of the real world by adding various 3D models, effects and other gaming elements. First, AR is used to enhance the opponents' robot, by augmenting its physical appearance with a 3D model and energy shield (see Fig. 1). Elements such as player's Heads-Up Display (HUD), and damage and weapon effects are also shown as part of the AR environment, enabling players to view weapon trajectories, executions and outcomes.

As a result, we believe that VR can be utilized to enhance immersiveness and introduce increasingly interesting interaction mechanisms to enhance the gaming experience. Likewise, AR enhances the physical camera view by enhancing real-world game elements whilst also introducing additional game related information.

# 4 Implementation

The game is implemented in a client-server design structure, each host maintaining it's own connection to a single dedicated robot with all communications happening over wireless network (see Fig. 3). Game components such as hit points, shield points, ammunition, etc. are all held locally and are transmitted between corresponding clients, via the server, on such events as Shot Hit.

<figure><img src="image_6.png" /><figcaption>Fig. 3. WAR Bots System Architecture.</figcaption></figure>

## 4.1 Hardware

The game will successfully run on any VR capable PC, such were those used in design and implementation. The HMD used during testing and implementation was the Oculus DK2 coupled an Xbox 360 controller which was mimiced in the cockpit design for added immersion (see Fig. 1). Two EZ-Robot Adventure Bots[1], strapped with HD webcam, used as the robot battle vehicles. Each of the robots was also fitted with an AR-marker shell (see Fig. 4 Left).
## 4.2 Software

The entire project was developed within Unity3D game engine [7]. Vuforia [8] was the basis for all AR contents (see Fig. 4). Oculus DK2 HMD was utilized to deliver the VR experience (see Fig. 1-2). The robots were controlled wirelessly via a client-server architecture software script written between the robot and Unity3D.

<figure><img src="image_7.png" /><figcaption>Fig. 4. Robot fitted with AR-marker shell and webcam (left) and overlayed AR model (right).</figcaption></figure>

# 5 Conclusion and Future Work

WAR Bots is a multiplayer game that combines elements of AR and VR. The game creates a sense of tele-presence for the user by having the player control a surrogate robot. The ability to tangibly interact with the robot in a cyber-physical environment, allows for a deeply immersive and highly enjoyable experience for all.

Future functionality includes upgraded interaction methods, such as motion and gesture control, customizable robots which are digitally represented in game including type benefits, augmenting environments with kinect sensors and projectors, and em-bedding nested markers.

## References

1. Adventure Bot - Products - EZ-Robot, last accessed 2017/04/04
https://www.ez-robot.com/Shop/AccessoriesDetails.aspx?productNumber=34.

2. ARDrone 2.0 Elite Edition, last accessed 2017/04/04
https://www.parrot.com/us/drones/parrot-ardrone-20-elite-%C3%A9dition.

3. Thomas, B., Close, B., Donoghue, J., Squires, J., Bondi, P. d., Morris, M., and Piekarski, W.: "ARQuake: An Outdoor/Indoor Augmented Reality First Person Application". 4th IEEE International Symposium on Wearable Computers, 139, 2000.

4. Piekarski, W. and Thomas, B.: "ARQuake: the outdoor augmented reality gaming system". Communications of the ACM vol. 45, No. 1, 36-38, 2002.

5. Aukstakalnis, S., Blatner, D.: "Silicon Mirage: The Art and Science of Virtual Reality". Peachpit Press Berkeley, CA, USA, 1992.

6. Tachi, S. "Tele-Existence". Research Center for Advanced Science and Technology, University of Tokyo, 1992.

7. Unity - Game Engine, https://unity3d.com, last accessed 2017/04/04.

8. Vuforia SDK, https://www.vuforia.com, last accessed 2017/04/04.