<figure><img src="image_1.png" /></figure>

HAL
open science

# Players Perception of Loot Boxes

Albert Sakhapov, Joseph Alexander Brown

► To cite this version:

Albert Sakhapov, Joseph Alexander Brown. Players Perception of Loot Boxes. 19th International Conference on Entertainment Computing (ICEC), Nov 2020, Xi'an, China. pp.134-141, 10.1007/978-3-030-65736-9_12. hal-03686044

HAL Id: hal-03686044

https://inria.hal.science/hal-03686044

Submitted on 2 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Players Perception of Loot Boxes

Albert Sakhapov and Joseph Alexander Brown[0000-0002-6513-4929]

Artificial Intelligence in Games Development Lab, Innopolis University, Republic of Tatarstan, Russia

{a.sahapov,j.brown}@innopolis.ru

**Abstract.** Loot boxes are a monetization technique in games where a player pays for a randomized chance at receiving an in-game item, either cosmetic or functional. They have recently been examined as a potential object for gambling, and government regulators are examining the issue of their use in games. To explore the issue and analyze players perceptions 53 people were surveyed and played a simple loot box game with different settings. Findings include that showing item drop probabilities directly influences the opinion of players regarding the fairness of loot boxes and can affect further choice whether to open them, which supports the need for regulations on loot box use.

**Keywords:** Loot Boxes · Gambling · Gaming · User Survey

## 1 Introduction

The phenomenon that is called loot box has existed for more than a decade. Originally, loot boxes were created to make the players game process easier or interesting for a certain amount of money. At the next stage, loot boxes have become a tool to keep the players interest within the game by offering new virtual items such as new characters, guns, or skins. The item varies from one game to another game, and it might be either customization for a players character that does not affect the balance of the game or booster that makes the game process easier and more comfortable for a target player[12].

The study presented in this paper discusses the perception of fairness on loot boxes by the essential people in game development – players. Currently, there is little data on how they perceive loot boxes from the perspective of fairness. Moreover, players opinions might differ according to certain factors of loot boxes, such as the cost of the box, the odds of particular drops, and the outcome. This study aims at finding out and analyzing how players perceive loot boxes and what they expect and want from developers to be adjusted. Our goal is to find the data and learn how players opinions change regarding the factors.

Behind the term loot box, there is a concept which Pawel Grabarczyk and Rune Nielsen named as random reward mechanisms (RRM) [10]. Even though loot boxes appeared about 15 years ago, RRM has existed for decades and first appeared in baseball collecting cards in 1875 in the United States. One card pack contains several randomly selected pieces, ranging from standard to rare. Rare
cards can reach prices in the millions of dollars. At that time, children and ado-
lescents were spending hundreds and even thousands of dollars to receive cards
of their favourite player, and now they are trying to open their favourite weapon
or character skin in video games. In video games RRM came with the first RPG
games like *Diablo*, where killing an enemy gives the player some random reward.
That system was used in many games because it is an easy way to introduce the
feeling of novelty and difference[20]. Moreover, *Blizzard Entertainment* had an
idea to sell individual game assets pack, which would contain several items of
different rarity. It was inspired by a collectible card game of *Magic: The Gath-
ering* that offers players to build their own physical desk from booster packs.
However, due to the unavailability of fast Internet technologies and the high cost
of distribution of these booster packs on physical discs, the idea was not adopted
in the final version of the game[2].

After a decade, with the progress of technologies, the concept of RRM began
to be used in mobile games. Since most of them are free to download and to play,
companies added “microtransactions” that have been an alternative monetiza-
tion for developers by providing extra items and functions that make the game
process more fun and exciting for some amount of money in turn. Currently, 30
games out of 31 contain in-app purchases [16]. Microtransactions have become
an integral part of game marketing and monetization[17].

To increase profits developers started to add loot boxes even in full-priced
video games. This has made it easier for companies to recoup multimillion bud-
gets for games, while keeping the initial purchase prices of games relatively the
same. The reasons people buy loot boxes were explained by Juho Hamari et
al. [11]. They highlighted six main reasons for opening loot boxes and buy-
ing in-game content: Performance, Personalization, Obtainment Achievements,
“Showing-off”, Low Perceived Cost, and Unlocking Content. The behaviour of
players is explained in a Hooked Model and examines how companies keep peo-
ple’s interest [5].

In the Autumn of 2017, EA Games went further than other companies, by
partially locking the content in their upcoming game Battlefront II, so that the
players spent their money or time to unlock it. Players researched that to open all
the content, the player should play at least 4578 hours or spend 2100 dollars[7].
This situation led to a huge video game scandal, and since then, governments of
many countries began to regulate loot boxes[4]. Since loot boxes use a random
chance of winning and players pay real-world funds, games may be considered
as a form of gambling[13]. As a result of the scandal, Gaming Commissions of
Belgium called game companies to have a “clear indication of the chances of
winning for the various item values” [9]. The same requirement made by Chinas
Ministry of Culture in December 2016 [3]. Singapore’s Parliament passed The
Remote Gambling Act, which obliges game companies to obtain a license to
operate if their games have random awards and real-money payments[18]. Aus-
tralian Gambling Commission of Victoria State suggested restricting the games
that contain loot boxes for children under eighteen [19]. On the other hand of
the dispute is UK Gambling Commission that concluded that loot boxes do not
need regulations because virtual items are *prizes*, and they cannot be used out-
side the game[8]. Entertainment Software Ratings Board (ESRB) organization
that provides video game age and content ratings for North America agreed
with UKs Gambling Commission and added that the players are guaranteed to
receive the in-game item. In May 2019, a bill to ban loot boxes and pay-to-win
microtransactions in video games for kids under 18 years was introduced in the
US. Senator Josh Hawley explained that game developers should not monetize
addiction when a game is designed for kids. After a day, The Entertainment
Software Association stated that numerous countries do not consider loot boxes
as gambling, and the ESRB organization “does not consider loot boxes to be
gambling [14]. They desired to share with the senator the tools “that keeps the
control of in-game spending in parents hands” [15]. Though researchers have rec-
ommended game developers to avoid giving players real money predicting this
could lead to problems with gambling laws [6] and associations with real gam-
bling [1]. The study presented brings into a discussion this controversial topic -
*loot boxes*, namely, how players perceive them.

## 2 Methodology

An experiment was started with the development of software that simulates opening a loot box. This system represents a loot box that provides a random item. As the idea was to examine players of Innopolis University, all items hypothetically can be used only within the University, so that students can easily get involved in the system and estimate its fairness without long introductions.

To cover all aspects and not miss important factors, we studied the principles
of loot box system in details. The following are loot box factors that were applied
in the software: *Payment Type*. There are three types of payment of loot boxes in
video games: fully free, ones that can be purchased with in-game currency, and
that can be bought with real-world currencies. Sometimes developers combine
all types to cover a wider audience. *Outcome*. Game development companies use
many different outcomes in their loot boxes. In our software, there are three
types of outcome: Innopolis University apparel that represents itself an analogy
to skins in video games, items that make the process more interesting, but are
optional and do not introduce imbalance, and grades for the hypothetical course
at University are like boosters that directly affect the result in video games.
*Rarity*. Mostly in all video games, items in loot boxes are divided by rarity.
It usually varies from very common to very rare, where the higher rarity, the
more valuable and desired item becomes. All rarity types are highlighted in some
colour so that players could easily see and identify the rarity. *Odds Distribution*.
Items of different rarity have different odds distribution. Therefore, common
items have the highest probability of drop, whereas rare items - the lowest. *Odds
Display*. As was mentioned in the previous section, some governments require
game companies to show loot box odds in video games. Developers added odds
display only in those countries in which it was obliged, whereas players from other
sides of our planet still cannot see the probabilities with which items fall. This
factor is important because it might be the new standard in loot box systems. Therefore, it is present in software that was developed and can be enabled or disabled in its settings.

The second part of the study implies gathering of the information of partic-
ipants. Subjects were asked questions about their experience and preferences in
loot boxes regarding fairness in probabilities, rarity division, and outcome type.
We collected this information to make overall analysis and to see in the end how
participants views have changed during the experiment. Most of the questions
in surveys use a scale with seven choices, where one completely disagrees/no, 4
is the neutral answer and 7 - completely agree/yes. This scale clearly shows the
opinions of participants and allows us to assess and compare the results of the
experiment easily.

All subjects that participated in initial data collection were divided into
several groups. Each group covers some of the factors of loot boxes. We chose two
the most important and widely discussed factors - items drop probabilities and
displaying them. These factors were the subject of discussion of the governments
because they form the opinion of people whether to open loot boxes. Thus, we
developed two probabilities distributions:

Fair. Probabilities: *Common* - 0.600, *Uncommon* - 0.350 *Rare* - 0.050

Unfair. Probabilities: ,Common - 0.849, Uncommon - 0.150, Rare - 0.001

as a result we examined a full factorial design on the factors of probabili-
ties and showing the values. To gather real information and avoid manipulating
results distribution of participants between groups was random. All the partici-
pants received their version of the game according to their experiment group.

The final round of collection started with asking participants their opinion
of loot boxes and whether they see loot boxes as gambling. These questions
were already asked in the initial data collection and would be compared to their
previous responses. Further, the participants were asked to describe their ideal
loot boxes. The type that they would not mind if removing loot boxes from video
games is not an option. Answers on this question might allow players, game
companies, and governments to come to a single solution of loot box problem if
the requirements and interests of all three sides will be satisfied.

3 Analysis

The main idea of the study was to understand the players perception of loot
boxed, considering many factors of the loot box system. During the experiment,
we were trying to influence participants opinions by changing two loot box factors
mentioned in previous section. As a result of the experiment, we expected to see
the change of participants views.

We created a list of questions on which we will look for changes of opinions:
1) What is your opinion about loot boxes?, 2) Do you agree with the statement: Loot boxes are gambling?, 3) Would you like to see the probabilities of item drops?, 4) Do you agree with the statement: Probability distributions in loot boxes are fair?, 5) Would you like to see the history of your previous openings?,
6) Do you like that items are divided into different groups of rarity?, 7) Which type of items do you like more?, and 8) Which type of loot box payment do you prefer?

*Group 1* In the first group with fair probabilities and without display, we did not expect changes in opinion, because odds distribution was hidden from the participants, and they did not know the probabilities. But results were unexpected. Participants raised their opinion regarding fairness in loot box probabilities.

*Group 2* The second group had fair probabilities and they were displayed. We observed changes in opinion regarding fairness of probabilities in loot boxes. Three participants in this group changed their opinion from *one* that equals to “strongly disagree that probabilities in loot boxes are fair” to *seven* that stands for “strongly agree”.

*Group 3* The third group had unfair probabilities without showing them. For the participants being in this group was challenging, we received feedback from 10 participants from this group that they did not open even one rare item. Some of them thought that the actual probability of getting a rare item in the game is *zero*. As a result, 71.4% of the group raised their desire to see probabilities in loot boxes.

*Group 4* The fourth group had unfair probabilities with showing them, resulted with a probability distribution for the respondents which was found to be not fair, and 72.2% of the participants in the fourth group decreased their trust in the fairness of loot box odds.

The results *showed* significance in question four that is stated: Do you agree with the statement: Probability distributions in loot boxes are fair?. The significance level were if *p* < .05, where *p value* is an evidence against our hypothesis. For this question, we hypothesized that with different probabilities distributions in groups 1,2 and 3,4, there would be changes in opinion regarding the fairness of odds. For the first and second groups, we expected to see an increasing level of trust and for third and fourth groups – decreasing level. So, players perception of fairness of probability distributions is formed when they face and try loot boxes. The amount of dropped valuable items and odds distribution display helps them to decide if the probability distribution is fair in loot boxes or not. We observed an increasing level of trust in loot box probabilities in groups one and two, and a decreasing level in groups three and four. The biggest changes were in the fourth group, where the participants saw probabilities, their average changes are -1.44, proving the hypothesis.

After analyzing the third question, we received *p value* equals to .000974 that makes results *significant*. The hypothesis for this question was that participants in groups with shown probabilities would increase their opinion regarding the display of probabilities. Therefore, our hypothesis was rejected because we observed changes conversely in groups that do *not* show probabilities. There was not a significant result that manipulations with probabilities and their display can influence players opinions whether they consider loot boxes as gambling.
<table><thead><tr><th>Question</th><th>Strong</th><th>No</th><th>Leaning</th><th>No</th><th>Neutral</th><th>Leaning</th><th>Yes</th><th>Yes</th><th>Strong</th></tr></thead><tbody><tr><td>Would you like to see probabilities of item drops?</td><td>0</td><td>5</td><td>0</td><td>1</td><td>7</td><td>9</td><td>31</td><td></td><td></td></tr><tr><td>Do you think probability distributions in loot boxes are fair?</td><td>4</td><td>6</td><td>13</td><td>12</td><td>11</td><td>3</td><td>4</td><td></td><td></td></tr><tr><td>Would you like to see the history of your previous openings?</td><td>0</td><td>1</td><td>4</td><td>5</td><td>5</td><td>9</td><td>29</td><td></td><td></td></tr><tr><td>Do you like that items are divided in different groups of rarity?</td><td>3</td><td>2</td><td>1</td><td>3</td><td>7</td><td>18</td><td>19</td><td></td><td></td></tr></tbody></table>

<table><thead><tr><th>Question</th><th>Only items that affect balance</th><th>Prefer Items that affect balance</th><th>Slightly for items that affect balance</th><th>Neutral</th><th>Slightly for items that do not affect balance</th><th>Prefer Items that do not affect balance</th><th>Only items that do not affect balance</th></tr></thead><tbody><tr><td>Which type of items do you like more?</td><td>3</td><td>1</td><td>2</td><td>3</td><td>5</td><td>8</td><td>31</td></tr></tbody></table>

<table><caption>Table 1. Distribution of answers regarding loot box preferences questions.</caption><thead><tr><th>Question</th><th>Only real-World Currency</th><th>Prefer Real-World Currency</th><th>Slightly for real-World Currency</th><th>Neutral</th><th>Slightly for in-Game Currency/Free</th><th>Prefer In-Game Currency/Free</th><th>Only in-Game Currency/Free</th></tr></thead><tbody><tr><td>Which type of loot box payment do you prefer?</td><td>2</td><td>0</td><td>0</td><td>2</td><td>4</td><td>18</td><td>27</td></tr></tbody></table>

It was considered that players dislike loot boxes because loot boxes deteriorate video games. However, the distribution is evenly split across all opinions. They explained their belief that loot boxes are an additional way for developers to monetize video games and to earn money. Moreover, if game companies do not overuse loot boxes, players mostly are not against the presence of loot boxes in video games. Moreover, some of the participants noted that sometimes the process of opening loot boxes could be fun and enjoyable if the odds distribution of dropped items is balanced, and loot boxes do not affect game balance. Negative parts of loot boxes that were mentioned by participants are that loot boxes are very similar to gambling and are present in almost every game. However, even though players are neutral to loot boxes, overwhelming majority of the participants — 73.6% — agreed that loot boxes are the form of gambling and must be regulated. They mentioned that it could lead to addiction. Only 15.1% of interviewed students did not agree or slightly disagree.

Overall satisfaction of results of loot box openings was also highly rated. So that, only 22.6% of the participants were not happy with results of their loot box trials, 26.4% were neither happy nor sad about the outcome, and 51% — happy and would open loot boxes in future. Even though 73.6% of the participants agreed with the statement that loot boxes are the form of gambling, most of the participants still open them some time, because they can give new experience, and the opening process might be fun and joyful. Only 25.6% spend money on loot boxes; others earn them in video games without investing money. As for preferences in loot boxes, all the questions with amount of people regarding each answer are provided in Table 1.
## 4 Compromise Loot Boxes

Loot boxes provide additional monetization for game companies so they could pay back the money spent on development and supporting the game even if it was not very successful. This is why loot boxes appear in almost every modern video game, and game companies start to overuse them. Sometimes it might negatively affect overall video game quality because companies focus on earnings from loot boxes. Such cases lead to an adverse reaction among the gaming community.

*Payment Type*— 88.6% of the participants said the loot boxes should be not only paid with real-world money but should be received during the game.

*Outcome*— As for this factor, the participants chose different outcome types for two types of online games: PvP and PvE. These types are contrasting, so in PvP games, the participants offered to use cosmetics that do not affect a players statistics and game balance as the only outcome in loot boxes. In PvE, game companies can use any of the outcomes, including booster packs and unique items that can help to improve game character because in PvE, all the players act together, and such things will not spoil the game experience for the others.

*Rarity*— Participants responses yield that 90.5% claim items distribution in different groups of a rarity to be important for loot boxes. If all the items were the same type of rarity, players would be much less likely to buy them, because there would not be rare, valuable items that players wanted to get.

*Odds Distribution*— As for odds distribution, participants wanted to be odds distribution fair, so that they do not need to open thousands of loot boxes to get one rare item. However, fairness is a subjective concept, so participants were asked to say the number of loot boxes they agree to open to get a rare item. Participants would like to see one rare item at least every 30 loot box.

*Odds Display*— 88.6% of the participants said that compromise loot boxes should show odds to the players.

## 5 Conclusions

In the study conducted, we discussed one of the controversial phenomenons in the video game industry – loot boxes. The majority of the participants face loot boxes almost every day in video games. Opinions are devised on if loot boxes negatively affect the game industry. However, there is the thing that unites those in the survey – the indifference to the game industry. The overall number of people who participated in the study is fifty-three, forty-six male and six females. All of the respondents had experience in video games and faced with loot boxes at least once. The perception of fairness in the study has highlighted many factors that broaden the understanding of peoples preferences in loot boxes. The results so far from the available sample size and covered loot box factors suggest that respondents’ loot box experience influenced their preferences. Finally, the participants proposed their vision of ideal loot boxes that can satisfy all the sides of the conflict - giving a picture of *ideal* loot box that would generate income for game companies, meet governments’ regulations and be accepted by video games community.
References

1. A.Brooks, G., Clark, L.: Associations between loot box use, problematic gaming and gambling, and gambling-related cognitions. Addictive Behaviors **96**, 26–34 (2019)

2. Brown, J.A.: Pitching diablo. ICGA Journal **40**(4), 417–424 (2018)

3. China's Ministry of Culture: Online game regulations, http://www.mcprc.gov.cn/whzx/bnsjdt/whscs/201612/t20161205_464422.html

4. Cross, K.: How the legal battle around loot boxes will change video games forever, https://www.theverge.com/2017/12/19/16783136/loot-boxes-video-games-gambling-legal

5. Eyal, N.: Hooked: How to Build Habit-Forming Products. Portfolio (2014)

6. Fields, T.: Mobile and Social Game Design: Monetization Methods and Design, chap. 8. CRC Press (2014)

7. Frank, A.: Star wars battlefront 2 content might take years to unlock, but ea wont say, https://www.polygon.com/2017/11/15/16656478/star-wars-battlefront-2-content-unlock-time-cost

8. Gambling Commission of UK: Virtual currencies, esports and social casino gaming position paper (2017)

9. Gaming Commission of Belgium: Research report on loot boxes (2018)

10. Grabarczyk, P., Nielsen, R.K.L.: Are loot boxes gambling? random reward mechanisms in video games (2018)

11. Hamari, J., Alha, K., Jarvela, S., Kivikangas, J.M., Koivisto, J., Paavilainen, J.: Why do players buy in-game content? an empirical study on concrete purchase motivations (2016)

12. Lawrence, N.: The troubling psychology of pay-to-loot systems, http://www.ign.com/articles/2017/04/24/the-troubling-psychology-of-pay-to-loot-systems

13. Olivetti, J.: The perfect ten: The truth about lockboxes (2012), https://www.engadget.com/2012/05/17/the-perfect-ten-the-truth-about-lockboxes/

14. Schreier, J.: ESRB says it doesn't see 'loot boxes' as gambling, https://kotaku.com/esrb-says-it-doesnt-see-loot-boxes-as-gambling-1819363091

15. Schreier, J.: U.S. senator introduces bill to ban loot boxes and pay-to-win microtransactions, https://kotaku.com/u-s-senator-introduces-bill-to-ban-loot-boxes-and-pay-1834612226

16. Shibuya, A., Teramoto, M., Shoun, A.: Systematic analysis of in-game purchases and social features of mobile social games in Japan (2015)

17. Stenros, J., Sotamaa, O.: Commoditization of helping players play: Rise of the service paradigm (2009)

18. Times, S.: Remote gambling bill could have 'negative effects' for digital games, https://www.straitstimes.com/singapore/remote-gambling-bill-could-have-negative-effects-for-digital-games

19. Walker, A.: Victoria's gambling regulator: Loot boxes 'constitute gambling', https://www.kotaku.com.au/2017/11/victorias-gambling-regulator-loot-boxes-constitute-gambling/

20. Wright, S.T.: The evolution of loot boxes, https://www.pcgamer.com/the-evolution-of-loot-boxes/