<figure><img src="image_1.png" /></figure>

HAL
open science

# Analyzing Video Game Completion Achievements
## Implications for Game Project Scope

Eric Nelson Bailey, Kazunori Miyata

► To cite this version:

Eric Nelson Bailey, Kazunori Miyata. Analyzing Video Game Completion Achievements Implications for Game Project Scope. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.469-472, 10.1007/978-3-319-66715-7_65. hal-01771293

HAL Id: hal-01771293

https://inria.hal.science/hal-01771293

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Analyzing Video Game Completion Achievements
## Implications for Game Project Scope

Eric Nelson Bailey [0000-0001-9147-9579] and Kazunori Miyata [0000-0002-1582-0058]

Japan Advanced Institute of Science and Technology, Nomi, Japan
bailey@jaist.ac.jp

**Abstract.** Game development project managers and product owners, such as directors, producers, and studio heads, rely on experience-based tacit knowledge to decide how much content to create for players. However, they could be operating on a misunderstanding of the way their players consume game content and how much game content is even desirable to players.

This paper presents the initial findings of our efforts to mine video game achievement data to discover trends in game completion rates and correlations to factors outside of the length of the game itself. Through tagging a sample of game achievements that signal a player has “finished” the primary single-player content, we discover that, for most games, few players will consume all of the provided content. With a better understanding of how players consume game content, project managers and product owners can make more informed decisions on project scope, which could reduce game budgets, make schedules easier to meet, and improve overall production efficiency.

**Keywords:** Game development · Project scope · Player retention · Completion rates · Achievements · Overdesign · Project management

# 1 Introduction

The video game industry generated over $100 billion in revenues in 2016 and is projected to continue growing in the coming years [1]. However, pressures from free-to-play, mobile, and independent games as well as competing entertainment products are forcing video game publishers and studios into difficult positions. The market, despite its size, is growing more difficult to profitably compete in, and video game budgets are growing to the point where many existing companies are one miscalculated project away from bankruptcy. With so much money at stake, publishers and studios in the video game industry need methods for determining the value of production tasks for their consumers and for deciding a scope for their projects that will generate the best return on investment.
# 2 Background

## 2.1 Video Game Project Scope Issues

Project scope is important because it is tied to schedules and development budgets. However, optimum scope is difficult to determine for any software development project, let alone for video game development projects, in which value can be much more difficult to define.

The video game industry suffers from poor project management practices, so improvements could not only improve the quality of life for game developers, but could also improve the quality of games and raise profits [2]. Of development issues, project scope is a critical problem for video game developers [3].

Game development might differ in some ways from other forms of software development, but software engineering principles can still be applied [4, 5], and a well-defined scope can remain flexible, while avoiding overdesign, feature creep and other risks [3].

The reality that players are not finishing their games and the development waste this might entail has come to some attention [6-8]. With regards to playtimes, prior research into player engagement discovered that player interest tends to fall according to a Weibull distribution [9], regardless of the properties of the game itself [10].

## 2.2 Achievement Data

Video game achievements reward players with digital badges for triggering events linked to game tasks such as single-player story progression, skill tests, or multiplayer-based challenges. Microsoft, Valve, Sony, Apple, Google, and some other platform holders have all implemented some form of achievement system for their games.

Valve makes much of their Steam service's game data available to the public through an API, which has inspired people such as Kyle Orland, at Ars Technica, Sergey Galyonkin, creator of Steam Spy, and “xPaw” and “Marlamin”, creators of SteamDB, to scrape the available data to discover various trends in sales and usage patterns. Through this data, it has been discovered that over a quarter of games (26.1%) have never been played, and, including the unplayed games, almost half (45%) have been played less than an hour, with just over a quarter of games (27.4%) having been played for more than 10 hours [11].

Of the data Valve provides, achievement data provides a rich source of information about the way players engage with their games that could be analyzed to help improve project scope decisions.

# 3 Methodology

Valve's Steam service keeps publicly visible information regarding achievement varieties. Using publicly available data from Steam, a list of 31,962 store entries, 5,489 of which had both single-player mode and achievements, was obtained. The achieve-
ment statistics data for each of these games was downloaded via Steam's API. Subsequent data analysis focused on progression-based achievements that indicated the primary, or “main path,” single-player content was completed by the player, excluding achievements that relied on game difficulty settings or branching endings.

In some cases, the achievement descriptions were enough to determine the appropriate game completion achievement, but often the entries needed to be checked against other sources, such as game walkthroughs or achievement guides, to determine what, if any, achievement marked completing the single-player content and whether any conditions existed. If the completion achievement was unclear, or the overall achievement rarity pattern indicated broken or post-release achievement implementation, the entry was pruned from further statistic calculations.

# 4 Results

Currently, we have checked 15% of the 5,489 candidate samples obtained from Steam. Of the 825 entries checked, 412 samples were identified as having completion achievements that did not rely on any special conditions other than clearing the game. Completion rates ranged from 0% to 56% completion with a mean of 14% (95% confidence interval of 13-16% with a margin of error of 1%), a median of 10%, and a standard deviation of 13%.

The distribution frequency in Figure 1 shows that, for nearly half of the games measured (202), 10% or fewer players finish the main game. Only about a quarter of games (106) had more than 20% of their players finishing the game, and only about 2% of games (9) had more than half seeing the game through to the end.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Distribution of video game completion ratings based on achievements.</figcaption></figure>
# 5 Conclusions and Future Work

Player retention falls over time, and few games, even short ones, have most of their players reaching the end of their single-player content. With an understanding of how players consume their game content, developers can take steps to examine their games against the backdrop of the market and decide whether the project scopes they have are appropriate to the value they wish to offer their players. By better tailoring project scope to the needs of their market, developers have the potential to avoid resource waste and schedule overruns.

In addition to analyzing more samples, we will continue our preliminary work testing correlations between completion rates and other data factors.

# References

1. McDonald, E.: The global games market will reach $108.9 billion in 2017 with mobile taking 42%. Newzoo, https://newzoo.com/insights/articles/the-global-games-market-will-reach-108-9-billion-in-2017-with-mobile-taking-42/, last accessed 2017/05/24.

2. Della Rocca, J.: Friction costs: How immature production practices and poor quality of life are bankrupting the game industry. Escapist Magazine, http://www.escapistmagazine.com/articles/view/video-games/issues/issue_40/243-Friction-Costs, last accessed 2017/05/24.

3. Petrillo, F., Pimenta, M., Trindade, F., Dietrich, C.: What went wrong? A survey of problems in game development. Computers in Entertainment (CIE) 7(1), 13 (2009). doi: 10.1145/1486508.1486521

4. Kanode, C. M., Haddad, H. M.: Software engineering challenges in game development. In: Proceedings of the 2009 Sixth International Conference on Information Technology: New Generations, pp. 260-265. IEEE (2009). doi: 10.1109/ITNG.2009.74

5. Musil, J., Schweda, A., Winkler, D., Biffl, S.: A survey on the state of the practice in video game software development. In: Technical report, QSE-IFS-10/04, TU Wien (2010).

6. Phillips, B.: Peering into the black box of player behavior: The player experience panel at Microsoft Game Studios. In: Games Developers Conference 2010 (GDC 2010).

7. Hullett, K., Nagappan, N., Schuh, E., Hopson, J.: Data analytics for game development: NIER track. In: Proceedings of the 33rd International Conference on Software Engineering (ICSE), pp. 940-943. IEEE (2011). doi: 10.1145/1985793.1985952

8. Zimmermann, T., Phillips, B., Nagappan, N., Harrison, C.: Data-driven games user research. In: CHI Workshop on Game User Research (CHI-GUR 2012), pp. 1-4. ACM (2012).

9. Bauckhage, C., Kersting, K., Sifa, R., Thurau, C., Drachen, A., Canossa, A.: How players lose interest in playing a game: An empirical study based on distributions of total playing times. In: 2012 IEEE Conference on Computational Intelligence and Games (CIG), pp. 139-146. IEEE (2012). doi: 10.1109/CIG.2012.6374148

10. Sifa, R., Bauckhage, C., Drachen, A.: The Playtime Principle: Large-scale cross-games interest modeling. In: 2014 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1-8. IEEE (2014). doi: 10.1109/CIG.2014.6932906

11. Orland, K.: Steam Gauge Ars Series, https://arstechnica.com/series/steam-gauge/, last retrieved 2017/05/24.