# Effects of Image-Based Rendering and Reconstruction on Game Developers Efficiency, Game Performance, and Gaming Experience

George E. Raptis, Christina Katsini, Christos Fidas, Nikolaos Avouris

► To cite this version:

George E. Raptis, Christina Katsini, Christos Fidas, Nikolaos Avouris. Effects of Image-Based Rendering and Reconstruction on Game Developers Efficiency, Game Performance, and Gaming Experience. 16th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2017, Bombay, India. pp.87-96, 10.1007/978-3-319-67684-5_6. hal-01678498

HAL Id: hal-01678498

https://inria.hal.science/hal-01678498

Submitted on 9 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Effects of Image-Based Rendering and Reconstruction on Game Developers Efficiency, Game Performance, and Gaming Experience

George E. Raptis<fnref n="1" />, Christina Katsini<fnref n="1" />, Christos Fidas<fnref n="1" />,<fnref n="2" />, Nikolaos Avouris<fnref n="1" />

**Abstract.** Image-based rendering and reconstruction (IBR) approaches minimize time and costs to develop video-game assets, aiming to assist small game studios and indie game developers survive in the competitive video-game industry. To further investigate the interplay of IBR on developers’ efficiency, game performance, and players’ gaming experience we conducted two evaluation studies: a comparative, ecologically valid study with professional game developers who created games with and without an IBR-based game development pipeline, and a user study, based on eye-tracking and A/B testing, with gamers who played the developed games. The analysis of the results indicates that IBR tools provide a credible solution for creating low cost video game assets in short time, sacrificing game performance though. From a player’s perspective, we note that the IBR approach influenced players’ preference and gaming experience within contexts of varying levels of player’s visual intersections related to the IBR-created game assets.

**Keywords:** evaluation study; video games; image-based rendering and reconstruction; game development efficiency; game performance; gaming experience

## 1 Introduction

The video game industry is continuously growing with revenue over $90 billion worldwide [1, 2]. Thousands of people worldwide work on creating video games for entertainment, as well as for education, business and art [3]. The video game market is very competitive, with an increasing demand for sophisticated and more realistic games. Such requirements are barrier for small game studios and indie developers, as they need more resources, in terms of cost, time, expertise, and technology, to keep up with the competition. Therefore, small game studios and indie developers struggle to survive, and this has a direct negative consequence on the video game industry and the economy, as these developers bring in creativity and innovation.

One way to overcome this, is to introduce novel cost-effective technologies which are primarily based on image [4–7] and video [8–11] reconstruction techniques. The users of such techniques can capture real-life scenes and objects with conventional apparatus (e.g., smartphones, photo cameras). The captured images/videos are then used

<footnote n="1">¹HCI Group, Dept. of Electrical and Computer Engineering, University of Patras, Greece
raptisg@upnet.gr, katsinic@upnet.gr, avouris@upatras.gr</footnote>

<footnote n="2">²Dept. of Cultural Heritage Management and New Technologies, University of Patras, Greece
fidas@upatras.gr</footnote>
to build the 3D realistic models of assets. Image-based reconstruction (IBR) seems to
produce better results [12].

**Motivation and research question.** Although it is known that IBR and photogrammetric methods provide a short-cut for creating 3D assets for video-games development [4–6, 13–15], to the authors’ knowledge, there is no thorough ecological study involving different stakeholders (game developers and game players) and investigating the interplay of a state-of-the-art IBR game development pipeline on game developers’ efficiency, game performance, and gaming experience. In the context of the reported study a fully integrated research-based state-of-the-art pipeline for games development bootstrapped to game developers’ requirements [16] was used.

Hence, the research questions investigated in this paper are: a) to explore the effects of games created with IBR on game developers’ efficiency, in terms of time required to create video-games by also considering game performance metrics (game size, loading time, and scalability), and b) to explore the effects of games created with IBR on gaming experience, in terms of gamers’ preference and graphics quality.

**2 Method of Study**

**2.1 Procedure**

We conducted two user studies: a) a comparative evaluation study during which video game prototypes were developed with the traditional vs. IBR methods aiming to gather quantitative and qualitative feedback related to efficiency and performance measures, and b) an eye-tracking A/B testing study with gamers playing the game prototypes created during the comparative evaluation study.

**Comparative study of creating game assets with and without IBR.** For this study, two professional game developers participated and developed two different games over a period of six months (May to October 2016). The participants developed their games within their own working conditions (i.e., ecologically valid conditions), without the intervention or help of our research team. They were asked to keep track of their activities and the time needed to perform the necessary tasks, adopting an activity diary-log approach. We provided them with a diary template. Moreover, several semi-structured interviews were conducted during and after the game development phase.

**Comparative study of playing games created with and without IBR.** For this in-lab study twelve gamers were recruited (5 females, 7 males), aged 20-32, and played different versions of the games created with the traditional and the IBR approach. They were recruited through email invitations, and they were undergraduate and postgraduate university students. All individuals were experienced gamers, playing video games more than twelve hours per week, and they had never played any of the games used in the study before. After the recruitment of the participants, twelve in-lab sessions were scheduled (i.e., one for each participant). In each session, the participant was asked to
play the four game versions, wearing the eye-tracking apparatus. To avoid bias effects, six gamers played first the one game, and the other six played the other game first. In both groups, three participants played the traditional version first, and the rest three played the IBR version first. After the play-session, each participant was asked to fill a user experience questionnaire. Finally, each participant was asked to provide feedback and comments on their gaming experience through an unstructured interview. The participants were not informed that the two versions were developed using different technologies until the end of the interview.

## 2.2 Instruments and Apparatus

**IBR-based game development integrated pipeline.** The IBR pipeline used in our study was the CR-PLAY environment [17–21]. CR-PLAY is a state-of-the-art mixed pipeline for creating game assets and integrating them in the game development workflow, using IBR techniques [18, 22]. CR-PLAY users follow a three-step approach to build a game asset: a) capture real-life objects (e.g., a building) by taking a sequence of photos from multiple angles; b) reconstruct them as 3D point cloud models; c) and import them into their standard game development workflow.

**The games.** Two video-games were created and played in our studies: *Survive the Weekend* and *Basketball Stars*. *Survive the Weekend* is a hidden object game in which the player is asked to find a number of items in a play-room within a given time. *Basketball Stars* is a first-person sport shooting game, and the players’ goal is to shoot as many baskets as possible in a predefined amount of time against an opponent. The two games have a different intersection between the game assets and the game play in terms of visual search and visual attention, as *Survive the Weekend* requires constant visual attention of players through the whole scene, because they are searching for the hidden items; while *Basketball Stars* requires players’ attention on specific game assets.

**Apparatus:** Game developers used their own computer systems to develop the games. Gamers played the games on a desktop computer (with Intel Core i7-4500U at 2.40GHz, and 8GB RAM). The screen used was the LG 22MP48D, at a screen resolution of 1920x1080 pixels. To investigate the participants’ visual behavior, Tobii Pro Glasses 2 were used to record their eye movement patterns.

**Gaming experience questionnaire.** To measure gaming experience, we used the Immersive Experience Questionnaire (IEQ) [23]. IEQ is a credible and highly validated tool to evaluate players’ experience [24, 25]. It consists of 31 questions of Likert type, including questions about graphics quality.
# 3 Results

## 3.1 Effects of IBR on game developers efficiency in creating video-game assets

The development of a game prototype with IBR took approximately 12 man-hours, regardless of the complexity of the scene. However, the time needed to develop a game prototype with the traditional approach was highly dependent on the number of assets to be included in the game, their complexity, and whether they have been used in other projects or were available from third-party providers. In our case study, the game developers were free to re-use any game asset and/or use third-party assets, as they would normally do, strengthening the ecological validity of the study. To create a game based on a simple scene with few items (N = 3), approximately 26 man-hours were required, while for a game based on a complex scene with many items (N = 19), 160 man-hours were required. In both cases, the developers required less time to create the IBR version. The interviews revealed that IBR reversed the process the game developers followed to develop a game, as the IBR game scenario was based on the reconstructed scene, while traditionally they created the scenes based on the scenario.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Game developers created the IBR game versions in less time than the traditional game versions, in both complexity levels.</figcaption></figure>

## 3.2 Effects of IBR on game performance (size, loading-time, and scalability).

Game performance is a multidimensional variable, which can be measured on game file-size, loading-time, and scalability. In terms of file-size, the executable was bigger for the IBR versions (300MB for a simple scene, and 330MB for a complex scene) compared to the traditional versions (15MB for a simple scene, 20MB for a complex scene). The loading times were in-line with the file-size findings, and they were increased for the IBR versions (10 seconds for a simple scene, 20 seconds for a complex scene) compared to the traditional versions (2 seconds for both a simple and a complex scene). The aforementioned results reflect on the scene complexity and the size of point cloud models, introduced in IBR. Therefore, this has an impact on the scalability of a game, as developers could not build many levels.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** IBR games were bigger in size than the traditional versions for both simple and complex scenes (left). IBR games had also longer loading times than the traditional versions for both simple and complex scenes (right).</figcaption></figure>

## 3.3 Effects of IBR on gaming experience

**Effects of IBR on game players' preference.** The game players were asked to state which version they preferred: IBR or traditional. Ten participants stated that they preferred the traditional version of *Survive the Weekend*, a significant difference according to binomial statistical test (p = .019). In contrary, eleven participants stated that they preferred the IBR version of *Basketball Stars*, a significant difference according to the binomial statistical test (p = .003). When playing the IBR version of *Survive the Weekend*, the participants had to explore the whole scene to find and collect the hidden items, and they stated that most of the game elements were blurry when they were navigating through the scene. They also stated that many objects were not naturally placed and lighted in the scene. On the other hand, the *Basketball Stars* scene was more static, as the players could not navigate through the scene. Therefore, no blurring objects were noticed in the IBR version, and the participants found it to be more photorealistic, with smooth transitions and nice interaction.

**Graphics quality.** To measure how much the players valued the graphics quality, we conducted an independent-samples t-test for each game, with the technology factor (traditional and IBR) as the independent variable, and the graphics quality score (calculated using IEQ) as the dependent variable. In both tests, the assumptions were met. Regarding the *Survive the Weekend* game, the participants preferred the graphics of the traditional version (5.11 ± 1.48 vs. 4.01 ± 1.42, p = .046). Regarding the *Basketball Stars* game, the participants preferred the graphics of the IBR version (5.09 ± 1.34 vs. 4.32 ± 1.17, p = .042). This was also reflected on the overall immersion score, as the participants were more immersed when playing the traditional version of *Survive the Weekend* (4.72 ± 0.66 vs. 4.64 ± 0.57, p = .061), and the IBR version of *Basketball Stars* (5.19 ± 0.51 vs. 4.84 ± 0.59, p = .019).
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Gamers liked more the graphics of the traditional version of *Survive the Weekend*, and the graphics of the IBR version of *Basketball Stars* (left); Study participants preferred the traditional version of the visual-search games (e.g., *Survive the Weekend*), and the IBR version of the point-and-shoot games (e.g., *Basketball Stars*).</figcaption></figure>

**Eye-tracking analysis.** The eye-tracking analysis verified the aforementioned finding, as the participants fixated many times and for long time periods on the whole backdrop scene, visually scanning for the hidden objects, in both versions of *Survive the Weekend*. On the other hand, they fixated intensively in specific areas of interest (i.e., basket, shooter, power bar) when playing *Basketball Stars*, as their goals was to shoot and score. The heat-maps of both versions of both games are depicted in **Fig. 4**.

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** The participants visually interacted with the whole scene when playing *Survive the Weekend*; but, they visually focused on specific areas of interest when playing *Basketball Stars*.</figcaption></figure>
# 4 Findings and Discussion

The findings of our studies are summarized into three categories: a) efficient creation of game assets with IBR, b) the bottleneck in performance when using IBR, and c) effective use of IBR depends on game type (Table 1).

**Efficient creation of game assets with IBR.** The IBR technology provided a faster method to create photorealistic game assets in comparison to the traditional approach (Fig. 1). The main benefit for the game developers is that IBR time to delivery is independent of the assets complexity, as our study indicated that the asset creation times were similar for creating games of varying game elements complexity.

**The bottleneck in performance when using IBR.** In the presented study, the game performance reflects on the executable file-size, the loading time, and the scalability of the game. In both games, the IBR versions were bigger in size, had longer loading periods, and were less scalable than the traditional ones (Fig. 2). Despite the fact that some of the aforementioned issues are technology-related and can be resolved (e.g., reduce file-size by incorporating sophisticated image compression algorithms), the IBR game performance will remain poor since game assets are not based on polygons. Game developers can create gamers quickly when they use IBR, but the game performance (in terms of file-size, loading time, and scalability) is a bottleneck.

**Game genre affects preference of IBR video-games.** Game players preferred the graphics quality of traditional *Survive the Weekend* (i.e., a game with heavy interaction with assets throughout the scene) than the graphics of the IBR version; and the graphics of IBR *Basketball Stars* (i.e., a game with light interaction with the scene), than the graphics of the traditional version. This was also reflected on their answers when they were asked explicitly which of the two versions (traditional or IBR) they preferred. Games that require players to interact with the largest part of the scene or with most of the game assets (e.g., hidden objects games) require high visual attention. Therefore, as users visually explore the game environment, they can detect IBR-related flaws (e.g., bad lightning, non-normal position of an object, blurred assets), which influence their gaming experience (Fig. 3). On the other hand, when the players are engaged with a video game with more static scenes (e.g., point-and-click games), visually focusing on specific areas of the scene, they typically do not detect IBR-related flaws, and they enjoy the game more, as it is more realistic and aesthetically pleasant (Fig. 3).

Our eye-tracking analysis reflects on this finding (Fig. 4), as the players tend to fixate on various spots of the visual scene evenly, when playing a visual search game. On the other hand, their fixations are more intense and focused on specific areas of interest (e.g., the game assets they interact with) when playing games which use the background scenes to provide a more photorealistic perception, but they are not part of the game-play or the players are not required to interact with them. Overall, the effect of IBR on gaming experience depends on the game type and the degree of the visual search activity (e.g., traditional methods will be preferred for games with heavy visual search, while IBR will be preferred for games with lighter visual interaction).
<table><caption>**Table 1.** The main findings of our studies on developers' efficiency, gaming performance and gaming experience</caption><thead><tr><td>Metric</td><td>Finding</td></tr></thead><tbody><tr><td>Developers' Efficiency</td><td>IBR provided a faster approach for creating video-game objects, for both simple and complex scenes. However, traditional approaches can be considered for simple scenes.</td></tr><tr><td>Gaming Performance</td><td>Traditional approach outperformed IBR in terms of file size, loading times, and scalability. IBR games scenes and objects were bigger in size, had increased loading times, and were less scalable.</td></tr><tr><td>Gaming Experience</td><td>The game type plays an important role on the effective use of IBR. Gaming experience was better for traditional games which require heavy visual interaction with the game assets and the scene (e.g., hidden-objects games). However, gaming experience was better for IBR games which require light visual interaction effect with the game scene (e.g., point-and-click at a static target), as they provide a more photorealistic environment.</td></tr></tbody></table>

## 4.1 Limitations and future work

The main limitation of our work is the relatively small sample sizes used in our studies. However, the statistical tests performed met all the required assumptions, providing credible results. Another limitation is the use of a research, but state-of-the-art IBR game development pipeline (CR-PLAY). As a future work, we will engage more end-users to evaluate the quality of the produced game elements and measure the gaming experience on games of varying types and characteristics.

## 5 Conclusion

In this paper, we investigated, through two evaluation studies, the impact of image-based rendering and reconstruction (IBR) on game developers efficiency, games performance and game players' experience. In the first study, we engaged two professional game developers who created, during a six-month period and within ecologically valid conditions, two versions (traditional and IBR) of two different games. In the second study, twelve experienced gamers played the four game versions in the context of an in-lab, A/B testing, eye-tracking approach. The analysis of the results revealed benefits of the IBR approach, such as reduced time and costs for creating photorealistic game assets. Drawbacks were also revealed, such as increased loading times, huge file sizes of deployed games and scalability issues which can be credited to the very nature of the IBR technology. Regarding, the players' gaming experience, the analysis of the results revealed mixed outcome regarding players' preference of games created with the traditional vs. IBR approach which should be further investigated.
References

1. Newzoo: Top 100 Countries by 2015 Game Revenues. (2015).

2. Entertainment Software Association (ESA): Essential Facts about the Computer and Video Game Industry. (2016).

3. Fuchs, M.: The video game industry: Formation, present state, and future. 6, 83–87 (2014). doi: 10.1386/jgvw.6.1.83_5.

4. Ackermann, J., Langguth, F., Fuhrmann, S., Goesele, M.: Photometric stereo for outdoor webcams. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 262–269 (2012). doi: 10.1109/CVPR.2012.6247684.

5. Laffont, P.Y., Bousseau, A., Drettakis, G.: Rich intrinsic image decomposition of outdoor scenes from multiple views. IEEE Trans. Vis. Comput. Graph. 19, 210–224 (2013). doi: 10.1109/TVCG.2012.112.

6. Chaurasia, G., Duchene, S., Sorkine-Hornung, O., Drettakis, G.: Depth synthesis and local warps for plausible image-based navigation. ACM Trans. Graph. 32, 1–12 (2013). doi: 10.1145/2487228.2487238.

7. Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Darmstadt, T.: Ambient point clouds for view interpolation. ACM Trans. Graph. 29, 1 (2010). doi: 10.1145/1833351.1778832.

8. Tompkin, J., Kim, K.I., Kautz, J., Theobalt, C.: Videoscapes: Exploring Sparse, Unstructured Video Collections. Acm Trans. Graph. 31, 12 (2012). doi: 10.1145/2185520.2185564.

9. Lipski, C., Linz, C., Berger, K., Sellent, A., Magnor, M.: Virtual video camera: Image-based viewpoint navigation through space and time. Comput. Graph. Forum. 29, 2555–2568 (2010). doi: 10.1111/j.1467-8659.2010.01824.x.

10. Ballan, L., Brostow, G.J., Puwein, J., Pollefeys, M.: Unstructured Video-Based Rendering: Interactive Exploration of Casually Captured Videos. Acm Trans. Graph. 29, 11 (2010). doi: 10.1145/1778765.1778824.

11. Levieux, P., Tompkin, J., Kautz, J.: Interactive viewpoint video textures. ACM Int. Conf. Proceeding Ser. 11–17 (2012). doi: 10.1145/2414688.2414690.

12. Starck, J., Kilner, J., Hilton, A.: Objective Quality Assessment in Free-Viewpoint Video Production. 3DTV Conf. True Vis. - Capture, Transm. Disp. 3D Video. 225–228 (2008). doi: 10.1109/3DTV.2008.4547849.

13. Capturing Reality, https://www.capturingreality.com/.

14. Meerstone Archa eological Consultancy: 3D Modelling with AgiSoft PhotoScan -Tests to determine the suitability of AgiSoft PhotoScan for archaeological recording. 1–4 (2010).

15. Raof, A.N.A., Setan, H., Chong, A., Majid, Z.: Three dimensional modeling of archaeological artifact using photomodeler scanner. J. Teknol. 75, 143–153 (2015). doi: 10.11113/jt.v75.5283

16. Fidas, C., Avouris, N., Orvieto, I.: Requirements Elicitation for new Video Game Development Tools: A Case Study. 1–18 (2015).

17. CR-PLAY, http://www.cr-play.eu/.

18. Cayon, R.O., Djelouah, A., Drettakis, G.: A Bayesian Approach for Selective Image-Based Rendering Using Superpixels. Proc. - 2015 Int. Conf. 3D Vision, 3DV 2015. 469–477 (2015). doi: 10.1109/3DV.2015.59.

19. Fuhrmann, S., Goesele, M.: Floating scale surface reconstruction. ACM Trans.
Graph. 33, 1-11 (2014). doi: 10.1145/2601097.2601163.

20. Gryka, M., Terry, M., Brostow, G.J.: Learning to Remove Soft Shadows. ACM Trans. Graph. 34, 1-15 (2015). doi: 10.1145/2732407.

21. Langguth, F., Goesele, M.: Guided Capturing of Multi-view Stereo Datasets. Eurographics 2013 - Short Pap. Proc. 93-96 (2013). doi: 10.2312/conf/EG2013/short/093-096.

22. Fuhrmann, S., Langguth, F., Moehrle, N., Waechter, M., Goesele, M.: MVE—An image-based reconstruction environment. Comput. Graph. 53, 44-53 (2015). doi: 10.1016/j.cag.2015.09.003.

23. Jennett, C., Cox, A.L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., Walton, A.: Measuring and defining the experience of immersion in games. Int. J. Hum. Comput. Stud. 66, 641-661 (2008). doi: 10.1016/j.ijhcs.2008.04.004.

24. Denisova, A., Nordin, A.I., Cairns, P.: The Convergence of Player Experience Questionnaires. In: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '16. pp. 33-37. ACM Press, New York, New York, USA (2016). doi: 10.1145/2967934.2968095.

25. Nordin, A.I., Denisova, A., Cairns, P.: Too Many Questionnaires: Measuring Player Experience Whilst Playing Digital Games. Seventh York Dr. Symp. Comput. Sci. Electron. 69-75 (2014). doi: 10.1007/s11031-006-9051-8.