# Identifying Influences of Game Upgrades on Profitable Players Behavior in MMORPGs

Luiz Bernardo Martins Kummer, Hiroyuki Iida, Julio Cesar Nievola, Emerson
Cabrera Paraiso

► To cite this version:

Luiz Bernardo Martins Kummer, Hiroyuki Iida, Julio Cesar Nievola, Emerson Cabrera Paraiso. Identifying Influences of Game Upgrades on Profitable Players Behavior in MMORPGs. 1st Joint International Conference on Entertainment Computing and Serious Games (ICEC-JCSG), Nov 2019, Arequipa, Peru. pp.299-310, 10.1007/978-3-030-34644-7_24. hal-03652046

HAL Id: hal-03652046

https://inria.hal.science/hal-03652046

Submitted on 26 Apr 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Identifying Influences of Game Upgrades on
Profitable Players Behavior in MMORPGs

Luiz Bernardo Martins Kummer<fnref n="1" />[0000-0001-8793-9325], Hiroyuki Iida<fnref n="2" />[0000-0001-8412-9837], Julio Cesar Nievola<fnref n="1" />[0000-0002-2212-4499], and Emerson Cabrera Paraiso<fnref n="1" />[0000-0002-6740-7855]

**Abstract.** Players can change their interest in continuing playing due to many reasons, such as the game content available to them. Therefore, game upgrades play an important role as they have the potential to influence players, being it a “double-edged sword”, as players may like the new challenges or not. Among the active players, “whales” are those players that are the most profitable ones. The goal of this paper is to answer the following research question: “What are the influences of game upgrades on profitable players behavior?”. To do that, we propose to apply and jointly interpret the results of four metrics (or KPIs): Commitment, Key Risk Indicator, Available Motivational Growth, and Game Refinement Value. This approach was applied to an MMORPG dataset that contains four kinds of upgrades. As results, the proposed joining identified three key influences and two players aspects, showing the potential to be used by game producers to evaluate the acceptance and influences of their upgrades in real situations.

**Keywords:** Game Analytics · Profitable Players · Players’ Commitment · Players’ KPIs · Players’ Metrics · Game Refinement Theory

## 1 Introduction

The game producers' role can be portrayed by two main challenges: the acquisition of new players, and the maintenance of the active ones. The acquisition of new players is usually done through advertisements, while the maintenance of the active ones is done through releases of new game contents (known as game upgrades) [14], that affect the players' interest in continuing playing [2, 5, 17]. In addition, some players can be more profitable than others [14], being the very profitable players named as “whales” [9]. This fact highlights the importance of game producers being able to identify, comprehend, and satisfy this kind of players, as the more pleased they are, the more profitable a game is. The problem

<footnote n="1">¹ Pontifícia Universidade Catolica do Parana, Rua Imaculada Conceicao, 1155, Curitiba-PR, Brazil
{luiz.kummer, nievola, paraiso}@ppgia.pucpr.br,</footnote>

<footnote n="2">² Japan Advanced Institute of Science and Technology, Asahidai 1-1, Nomi, Ishikawa, Japan
iida@jaist.ac.jp</footnote>
is, satisfaction aspects of players are usually interpreted empirically through raw
metrics that consider only the amount of played time, like MAU (monthly active
users), which can lead them to risk situations (e.g., due to misinterpretations of
a new upgrade acceptance) [5, 7, 14].

In academia, the research field that focuses on modeling players' behavior is the Game Analytics [3]. One of its approaches regards the identification of changes in players' engagement to help producers identifying risk situations, being it usually based on two features present on usage data, the amount of time spent playing and the obtained score [5, 7, 12]. As examples, one can cite the Commitment metric and Key Risk Indicator (KRI) from Kummer et al. [5].

The Commitment metric was proposed to give an alternative to the empirical
analyses done by game producers in reference to the “how motivated are the
active players?” concern. This metric splits the active players into three different
groups regarding low, average, and high commitment degrees. Moving to the
KRI, this metric is based on the Commitment one, where positive and negative
transitions of players between the commitment degrees are identified and then
a view of it is presented in a comparative perspective.

An analogous approach is the called Game Refinement Theory (GRT) of Iida
et al. [4], which regards the measurement of the players’ entertainment based on
the game rules, the players’ strength, and the outcome uncertainty. Its concepts
are represented by the called Game Refinement Value (GRV).

In this work, the aforementioned metrics are studied and improved, resulting
in changes of the GRV, KRI and the proposition of a new metric, named Avail-
able Motivational Growth (AMG). It was proposed then to jointly apply the
Commitment, KRI, GRV, and AMG metrics and use their values as an answer
to the following research question (RQ): “What are the influences of game up-
grades on profitable players behavior?”. As results, three key influences and two
players aspects were identified. Players’ behavior is a complex universe, where
different points of view can be used to try to understand it. In this approach, the
influences of upgrades are interpreted in the engagement (increase or decrease)
and enjoyment (considering the uncertainty aspect) contexts.

This paper is organized as follows: Section 2 presents the adopted metrics, Section 3 shows the metrics’ previous applications, Section 4 depicts the metrics’ improvements, Section 5 presents the dataset, experiments and discussions, and Section 6 concludes the paper with the RQ answer and future works.

## 2 Metrics

This section presents how each metric used in this work was conceived.

### 2.1 Commitment

The Commitment metric was initially proposed by Kummer et al. in [5], being
improved by themselves in [7]. Its motivation regarded the fact that analyses over
the number of active players may hide risk situations, such when there is a high
number of active players and they are not engaged in continuing playing, leading game producers to a false sense of “comfort”, while in fact, it is a dangerous situation. To deal with that, this metric segregates the active players into three degrees of commitment (low, average, and high), highlighting when players are disengaged to a considered game (e.g., when the majority of players has a low commitment degree).

This metric is generated through a Machine Learning approach that contains unsupervised and supervised steps, where the amount of time spent playing and the obtained score are used considering a sense of speed progress. The method to generate it requires a definition of a time-span size, which can be weekly or monthly. For each time-span and for each player, his/her commitment degree is identified, and then a sum of all players on each degree is presented as the final result, usually being compared with metrics like MAU.

The Commitment input is represented by the following vector: $V_i = \{id_i, d_i, S_{i-min}, S_{i-max}, \Delta S_i\}$, where $id_i$ is the identification of the player $i$, $d_i$ the number of days played in a week (or in a month; it depends on the adopted time-span size), $S_{i-min}$ the minimum obtained score, $S_{i-max}$ the maximum obtained score, and $\Delta S_i$ the number of times that the score was increased.

The Commitment metric assumes that if a player likes a game, he/she will play for longer, improving his/her abilities. According to this rationale, the commitment to a game is something built and maintained over time. The definitions of low, average, and high degrees are updated based on all time-spans, therefore, the method presents tolerance to outliers and also the capacity to adapt to new and stable behaviors (due to an Ensemble with the majority vote policy).

The low committed profile is understood as players that started to play and are usually in a learning phase about the game mechanisms. The average profile represents the players that are evolving their performance and spending more time playing than the low committed ones, however, with a yet different behavior compared to the high profile. Lastly, the high profile represents the players who play for the longest period of time and present the best performances.

## 2.2 Key Risk Indicator - KRI

The KRI metric aims at depicting the general increase or decrease in players' commitment over time [5]. Its application starts considering the first pair of consecutive time-spans (e.g., two consecutive months or weeks), where the players that increased or decreased their commitment are identified. Next, each player is grouped according to definitions of Table 1. Then, the members of each group are summed up and the RawKRI value is computed through the (1) application. Finally, after computing the RawKRI value for each consecutive time-spans, the normalized and final value is obtained through the (2) application.

$$ \text{RawKRI} = (LA + AH + LH) - (AL + HA + HL) \quad (1) $$

$$ \text{KRI} = \frac{\text{RawKRI}}{\max(\text{RawKRI})} \quad (2) $$
<table><caption>**Table 1.** Possible changes in commitment</caption><thead><tr><td>Change Group</td><td>Label</td><td>Engagement</td></tr></thead><tbody><tr><td>Low to Average</td><td>LA</td><td>Increase</td></tr><tr><td>Average to High</td><td>AH</td><td>Increase</td></tr><tr><td>Low to High</td><td>LH</td><td>Increase</td></tr><tr><td>Average to Low</td><td>AL</td><td>Decrease</td></tr><tr><td>High to Average</td><td>HA</td><td>Decrease</td></tr><tr><td>High to Low</td><td>HL</td><td>Decrease</td></tr></tbody></table>

Where $max(RawKRI)$ is the biggest RawKRI of the series. This metric's range is from 0 to 1, where 0 means the worst decrease in players' engagement, while 1 is the best increase. It is interesting to highlight that every-time a new time-span presents a new $max(RawKRI)$, this time-span receives the value 1 and all the other time-spans have their values updated according to it.

## 2.3 Game Refinement Value - GRV

The GRT was initially proposed by Iida et al. in [4] to try to measure a degree of excitement of board games. It is interesting to notice that this theory avoids the empirical considerations about interest, founding its concepts on the classical mechanic physics of Newton [10]. In the same way that a roller-coaster can provide more or less fun according to settings of G-force, speed, and height, a board game may have a similar structure, as we have interests in both kinds of activities. On the one hand, the roller-coaster applies an acceleration into our body which affects our sensation of pleasure, on the other hand, a board game also provides such sensation which can be the result of an acceleration, but in that case, applied to our minds. This is the essential idea behind the GRT (physics in mind).

The acceleration in mind was modeled as the variations of the outcome uncertainty of a match, therefore, it is assumed that in an exciting game, the winner must be unknown until the very end. Based on this concept, the comprehension about the game result is a function of time (number of moves) $t$, as the result becomes more determined as time passes. Therefore, the amount of solved uncertainty can be represented by the function $x(t)$ (the game information progress model). Let B and D be the average branching factor and the average depth of a game respectively. If B and D are known for a match, the game information progress $x(t)$ will be given as a linear function of time $t$ with $0 \le t \le D$ and $0 \le x(t) \le B$, as presented in (3).

$$x(t) = (B/D)t \qquad (3)$$

However, the game information progress is usually unknown during a match. Hence, it is assumed to be exponential, due to its uncertainty until the very end of a game. Therefore, a more realistic model is given by (4).
$$x(t) = B(t/D)^n \qquad (4)$$

Where n is a constant given by an observer of the considered game. In this new model, the acceleration of the game information progress can be obtained applying the second derivative of (4). Solving it at $t = D$ (the end game period).

$$x(D) = (Bn(n-1)/D^n)D^{n-2} = (B/D^2)n(n-1) \qquad (5)$$

It is assumed that when a match is happening, the acceleration of game information is happening somehow in our minds, being it enjoyable or not. The physics in mind is not yet fully understood, but according to Newton's laws, if there was an acceleration, there was a force acting, so when there is an acceleration in our minds (i.e., stimulus-response), there is also a force acting. Therefore, it is expected that the larger the value $B/D^2$, the more exciting a game is, due to the outcome uncertainty. Lastly, the GRV is assumed as described in (6).

$$GRV = \sqrt{B}/D \qquad (6)$$

The key concept to apply the GRT to a game consists of identifying the game information progress model, i.e., game features that contain uncertainty. For example, in the Soccer and Basketball cases, the average values regarding the number of shots and the number of successful shots are used [15], while for the RPG case, the values of options available and turns can be used [11]. All of these values are conceptually applied somehow to the structure proposed in (6).

According to the applications of GRT [4,11,15,16], there is a common agreement that regards the called “Sophisticated Zone”, which is the GRV between 0.07 and 0.08. Games in this zone tend to be more enjoyable than games outside of it. Moreover, values lower than 0.07 mean that a game is more competitive (more based on players' skill), while values above 0.08 mean that a game is more stochastic (i.e., more based on chance), therefore, the Sophisticated Zone is a balance between players' skill and chance. Another aspect regards the game length, where games too long can be seen as boring (lower GRV) while games too short as unfair (higher GRV). As a final remark, the same game may have different GRVs. For example, in an MMORPG, it is possible to compute the GRV for PvP battles, loot boxes, and the reforging system.

## 3 Metrics' Previous Applications

Starting with the Commitment metric, it was applied by Kummer et al. [7] to identify when a game enters in its last stage of the usage lifecycle, known as the Niche stage [2]. In this approach, the authors identified an increasing trend of the number of high committed players together with a decreasing trend of the low committed ones in an MMORPG. Proposing the Niche occurrence when a game has a number of high committed players greater than the number of low committed ones.
In the KRI case, Kummer et al. [5] applied it in an MMORPG to identify variances on players' behavior according to game upgrades. It was possible to identify situations where the MAU did not highlight a risk situation while the KRI did. For example, when the MAU increased due to a game upgrade but the players' commitment decreased (i.e., an initial expectancy that was not attained after the new content consumption). In addition, this approach did not distinguish the characteristics of each upgrade.

Moving to the GRT applications, even though it was successfully applied to several game genres [4, 11, 15, 16], it has never been applied to MMORPGs. Thus this paper has the opportunity to assess the theory's concepts in a new context.

It is interesting to notice that none of these metrics were applied to the profitable players perspective. In addition, this perspective is usually linked to the predictions of players' churn, remaining lifetime, and lifetime value [1, 6, 8, 13].

In conclusion, even though there are interesting approaches to deal with usage lifecycle risks, none of them focused on the identification and interpretation of the influences of game upgrades on profitable players' behavior. Therefore, as far as our knowledge goes, this paper proposal is improving the state-of-art.

## 4 Metrics' Improvements

This section presents the proposed improvements of the GRV and KRI metrics, and also the proposition of a new one, named Available Motivational Growth.

Starting with the GRV metric, it was proposed to join it with the Commitment approach, allowing in that way to interpret the GRVs of low, average, and high committed players. Moving to the KRI metric, it shows the variations of players' commitment over time, however, it is not clear when the majority of players increased or decreased their commitment degree. In view of it, we propose to expand the metric's range from -1 to 1, where negative values regard the case when the majority of players decreased their commitment degree, while the positive ones the opposite of it. The updated KRI Equation is depicted in (7).

$$KRI = \frac{RawKRI}{max_{positive}(RawKRI)} \lor max_{negative}(RawKRI) \quad (7)$$

In view of the equation (1), when the sum ($LA + AH + LH$) is bigger than the sum ($AL + HA + HL$), the $max_{positive}(RawKRI)$ is used, being the $max_{negative}(RawKRI)$ used otherwise.

Another point regards the idea of potential growth of commitment per time-span. As it is neither depicted by Commitment nor by KRI, it was proposed to represent it through a new metric named Available Motivational Growth (AMG), which consists of a sum of the number of players that are not in the high commitment degree, such as depicted in (8) and normalized in (9).

$$RawAMG = P_{low} + P_{average} \quad (8)$$

$$AMG = \frac{RawAMG}{max(RawAMG)} \quad (9)$$
Where $P_{low}$ and $P_{average}$ are the number of players with low and average commitment respectively, and $max(RawAMG)$ is the biggest RawAMG of the series. The metric's range is from 0 to 1, where 0 means no potential growth (i.e., all players already have a high commitment) and 1 the opposite.

As final remarks, the Commitment, KRI, and AMG metrics demand data containing players' identification, obtained score, and the time-stamp associated to each score, while the GRV one requires the total number of attempts, the total number of successes, and the associated time-span linked to a game feature.

# 5 Experiments and Discussions

This section shows the metrics' applications to an MMORPG that aim at identifying behavioral changes of players through different game upgrades to answer the proposed RQ. As the dataset is divided in weeks, all metrics were applied considering this perspective. All computations were performed through SQL queries in a MySQL database and Machine Learning algorithms written in Java.

## 5.1 The Blade&Soul Dataset

The proposed metrics were applied to the Blade&Soul game, an MMORPG that presents four game upgrades (see Table 2). This dataset was provided by a Data Mining competition [8] and is divided into three subsets. Table 3 summarizes each subset aspects, such as the number of players (all selected by its game producer; all whales), the number of instances (each instance refers to one action did by a player in a given time), the collection period, and the number of weeks.

<table><caption>**Table 2.** Blade&Soul upgrades summary</caption><thead><tr><td>Date</td><td>Subset</td><td>Week</td><td>Observations</td></tr></thead><tbody><tr><td>2016-04-27</td><td>1</td><td>5</td><td>New PvP mode.</td></tr><tr><td>2016-07-20</td><td>2</td><td>8</td><td>More acts (stories).</td></tr><tr><td>2016-08-24</td><td>2</td><td>13</td><td>PvP improvements.</td></tr><tr><td>2017-01-18</td><td>3</td><td>21</td><td>Cosmetic and item drop modifications.</td></tr></tbody></table>

<table><caption>**Table 3.** Summary of the Blade&Soul dataset</caption><thead><tr><td>Subset</td><td># Players</td><td># Instances</td><td>Collection Period</td><td># Weeks</td></tr></thead><tbody><tr><td>1</td><td>4000</td><td>175,139,564</td><td>2016-03-30 until 2016-05-10</td><td>6</td></tr><tr><td>2</td><td>3000</td><td>197,661,989</td><td>2016-07-13 until 2016-09-13</td><td>9</td></tr><tr><td>3</td><td>3000</td><td>206,758,995</td><td>2016-12-14 until 2017-02-07</td><td>8</td></tr></tbody></table>

Note that there are gaps between the total of 23 weeks. However, regardless of these gaps, the weeks are numbered as a sequence. The 15th-week presents a
very distinct behavior compared to the others, as it has only one day of usage.
Thus, to not add a bias in the overall analysis, we opted to not consider it.

## 5.2 Metrics' Results and Analysis

All findings presented in this section are understood as detailed answers to the proposed RQ, where each metric contributes in terms of its concepts. Starting with the Commitment one, the input vector $V_i$ was configured such as follows: $id_i$ as the player identification, $d_i$ as the number of days played in a week, $S_{i-min}$ as the minimum level, $S_{i-max}$ as the maximum level, and $\Delta S_i$ as the amount of “level up” occurrences. As we can see in Fig. 1, all weeks presented a number of high committed players greater than the other degrees. This disposition highlights the whales characteristic of this dataset, where it is expected that the majority of players will be from the high commitment profile.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Commitment assignment</figcaption></figure>

The first week had a similar number of average and high committed players due to a previous upgrade that improved the players' max level. This week can be seen as a “rush” to achieve the new highest level, entailing in a greater number of level up events per player. Moving to the Christmas holiday, it presented an increase of the high committed players. We understand this increase as the players’ enjoyment due to the opportunity to play longer than usual.

Unpaired *t* tests were applied (considering *p* < 0.05) over the percentage of players on each commitment degree of each week per subset to identify if their distribution (i.e., mean value) changed from one subset to another. The identified significant differences regarded the low committed players between subsets 1 and 2, and the average and high ones between subsets 1 and 3 and also between 2 and 3. In conclusion, these differences highlight the fact that upgrades and holidays can influence the players’ commitment over time. Next, the KRI and the AMG values are presented in Fig. 2.

In the KRI perspective, it is possible to notice that the 4th-week presents an improvement of players' commitment, which can be seen as a positive expectancy regarding the upcoming upgrade. However, it was not kept in the following two
<figure><img src="image_3.png" /><figcaption>Fig. 2. KRI and AMG assignments</figcaption></figure>

weeks. We found two possible reasons for that, (1) after the players consuming the new content, they lost motivation, or (2) their expectancy was not attained. In the next upgrade (8th-week), the players' commitment increased compared to the previous week, but again, it decreased in the following weeks, due to game content consumption or expectancies not attained. The 13th-week and 21st-week upgrades can be considered as unsuccessful because the players' commitment dropped, in addition, the 21st-week is the starting point of the worst decrease of players' engagement. Even though the level upgrade week is not present in the dataset, its effects could be measured 2 weeks after its release as the best improvement of players' commitment (2nd-week).

Moving to the AMG perspective, none of the upgrades was engaging enough to have an AMG value lower than 0.4 (the lower, the better). The highest value was identified in the week after the first upgrade (6th-week), highlighting the drop of players' engagement in this time-span. Additionally, the best KRI value happened together with an AMG value of 0.66. It means that even though it was the best increase in players' engagement, there was a hidden potential to get better results. As a final remark, the AMG mean value was 0.55, it means that there was a chance to captivate players to higher degrees of commitment over the whole dataset period, highlighting the opportunity to promote upgrades.

Turning to the GRV perspective, we could manage to apply it to two game features, which were the PvP battles and the Reforging system. For the PvP case, we used conceptually the number of fought battles as *D* and the number of won battles as *B*, while for the Reforging case, the number of attempts was assumed as *D* and the number of successes as *B*. Next, Fig. 3 shows their results.

Firstly observing the PvP perspective (Fig. 3 a), it is possible to see that the identified GRVs are not in the Sophisticated Zone. After assimilating the zone's concepts to the current data, the PvP battles can be seen as unfair and based on chance due to the high GRVs. However, we understand that it is not fully true in this case, as the PvP battles in the considered game have an extra aspect attached to the players' skill that we named "evolutionary strength". During a battle, the players' skill can be seen as the players' effectiveness and efficiency in using the options available to them (i.e, playing well), while the evolutionary strength is the players' acquired level until a given moment. In addition, the evolutionary strength is accumulative, thus once it is acquired (i.e., a "level up"
<figure><img src="image_4.png" /><figcaption>Fig. 3. (a) PvP GRVs assignments; (b) Reforging GRVs assignments</figcaption></figure>

event) it is not possible to lose (i.e., there is no “level down” event), what does not happen to the players’ skill, that may improve with training or reduce due to a lack of practice. As an example, the combination of such characteristics can generate situations where a very skilled player may lose a battle against a less skilled one if the other player’s level is higher. We understand that this kind of situation influences the concepts around the Sophisticated Zone, for example, the fact that a player having more levels than the other does not mean that the battle is based on chance, therefore, we considered the PvP of the Blade&Soul game as possibly unfair and based on a combination of players’ skill and evolutionary strength. As a final remark, unpaired *t* tests with *p* < 0.05 were performed on each subset (considering the GRV of each commitment degree per week) and statistically significant differences were identified between all degrees for all subsets (except by the low and the average groups).

Moving to the Reforging perspective (Fig. 3 b), it was not possible to distinguish the effects of the different kinds of upgrades to any commitment degree, being a possible explanation for that the lack of improvements in the Reforging system. However, as occurred to the PvP battles, all the GRVs identified are not in the Sophisticated Zone. A reason for that regards the general success rate of approximately 94% that reduced the sense of outcome uncertainty. Moreover, all means of the commitment degrees presented statistically significant differences between them (unpaired *t* tests with *p* < 0.05). Thus, according to the identified values, we consider the Reforging system of Blade&Soul as stochastic.

# 6 Conclusions and Future Works

This work improved the state-of-art in four main points: (1) by improving the GRV and KRI metrics, (2) by proposing a new metric (AMG), (3) by identifying new points of analysis through the metrics combinations, and (4) by assessing the Sophisticated Zone concepts to the MMORPG context.

According to the Blade&Soul dataset, the KRI indicates that PvP upgrades tend to improve the players’ commitment before the upgrade, presenting then a drop of engagement after the new content consumption, while story upgrades can keep players motivated for longer compared to the PvP ones, presenting picks of motivation during the upgrade week. Moreover, the upgrade regarding cosmetic
and item drop modifications presents an initial increase before the upgrade and a strong decrease after it, similar to the PvP one. According to these influences we could point three reasons: (1) new PvP battles are opportunities to players became in evidence by winning the most battles as possible, thus an initial preparation is a good idea to have a good performance when the new competition starts, moreover, the competitions seem to excite or demotivate players fast (i.e., one week); (2) there is no preparation to accomplish story upgrades, therefore players become more committed only when the new content is available; and (3) cosmetic and item drop upgrades generate initial expectancies or preparation, but after players obtaining their new costumes or desired items, their engagement drops as the objectives were achieved.

In the GRT perspective, the higher a GRV, the more stochastic, while the lower, the more competitive. Thus, it is possible to state that high committed players present a more competitive behavior than the lower committed ones. Moreover, even though there were different kinds of upgrades, their GRVs kept similar. We rationale two possible reasons for that: (1) the degree of challenge offered to them keeps the same over time, or (2) the players were able to adapt and maintain their performance. By contrast, low and average committed players present more changes in their sense of entertaining considering PvP battles and the Reforging system, which means that they are more sensitive to changes.

The summarized answer to the RQ can be represented by the following three key influences: (1) different upgrades entail in expectations to consume the new content, moreover, some kinds of them allow a preparation while others do not; (2) depending on the considered upgrade, players can increase their commitment to the game for longer; and (3) upgrades that present a preparation aspect tend to keep players engaged for less time than the ones that do not allow it. Also, two aspects of players were identified: (1) the more committed a player is, the more resilient he/she is to changes on game features, and (2) the higher the commitment of a player, the more competitive his/her behavior is.

In conclusion, we understand that the proposed combination of metrics can be used by game producers to better evaluate the acceptance of their upgrades. As future works, we wish to: replicate these measurements considering players besides the whales and compare the results (validating with game producers a possible generalization over the findings), and propose new metrics to explore other aspects not covered in this paper, such as the players' motivations.

## Acknowledgments

We would like to thank NCSOFT for turning the dataset of Blade&Soul available, and CAPES-Brazil (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior) and Fundação Araucária (CP 09/2016) for their support in this research.
References

1. Chen, P.P., Guitart, A., del Río, A.F., Periáñez, Á.: Customer lifetime value in video games using deep learning and parametric models. In: 2018 IEEE International Conference on Big Data (Big Data). pp. 2134–2140. IEEE (2018)

2. Cook, D.: The circle of life: An analysis of the game product lifecycle (may 2007), https://www.gamasutra.com/view/feature/129880/the_circle_of_life_an_analysis_of_.php

3. El-Nasr, M.S., Drachen, A., Canossa, A.: Game analytics. Springer (2016)

4. Iida, H., Takeshita, N., Yoshimura, J.: A metric for entertainment of boardgames: its implication for evolution of chess variants. In: Entertainment Computing, pp. 65–72. Springer (2003)

5. Kummer, L.B.M., Nievola, J.C., Paraiso, E.C.: A key risk indicator for the game usage lifecycle. In: AAAI Publications, The Thirtieth International Flairs Conference (FLAIRS). pp. 394–399. AAAI (2017)

6. Kummer, L.B.M., Nievola, J.C., Paraiso, E.C.: Applying commitment to churn and remaining players lifetime prediction. In: Computational Intelligence and Games (CIG), 2018 IEEE Conference on. pp. 213–220. IEEE (2018)

7. Kummer, L.B.M., Nievola, J.C., Paraiso, E.C.: Identifying niche stage in mmorpgs using ensemble classifier. In: AAAI Publications, The Thirty first International Flairs Conference (FLAIRS). pp. 288–293. AAAI (2018)

8. Lee, E., Jang, Y., Yoon, D.M., Jeon, J., Yang, S.i., Lee, S., Kim, D.W., Chen, P.P., Guitart, A., Bertens, P., et al.: Game data mining competition on churn prediction and survival analysis using commercial game log data. IEEE Transactions on Games (2018)

9. McAloon, A.: Analyst: 'whale' spenders aren't as crucial to f2p mobile games as they once were (Sep 2018), http://www.gamasutra.com/view/news/326173/Analyst_Whale_spenders_arent_as_crucial_to_F2P_mobile games_as_they_o nce_were.php

10. Newton, I.: Philosophiae naturalis principia mathematica. London (1687)

11. Panumate, C.: Various Approaches to Improving Entertainment Impact in Games. Ph.D. thesis, Japan Advanced Institute of Science and Technology (2016)

12. Periáñez, Á., Saas, A., Guitart, A., Magne, C.: Churn prediction in mobile social games: towards a complete assessment using survival ensembles. In: Data Science and Advanced Analytics (DSAA), 2016 IEEE International Conference on. pp. 564–573. IEEE (2016)

13. Runge, J., Gao, P., Garcin, F., Faltings, B.: Churn prediction for high-value players in casual social games. In: Computational Intelligence and Games (CIG), 2014 IEEE Conference on. pp. 1–8. IEEE (2014)

14. Speller III, T.H.: The business and dynamics of free-to-play social-casual game apps. Ph.D. thesis, Massachusetts Institute of Technology (2012)

15. Sutiono, A.P., Purwarianti, A., Iida, H.: A mathematical model of game refinement. In: International Conference on Intelligent Technologies for Interactive Entertainment. pp. 148–151. Springer (2014)

16. Xiong, S.: A New Paradigm in Game Design using Game Refinement Theory and Cross Media. Ph.D. thesis, Japan Advanced Institute of Science and Technology (2017)

17. Zhu, L., Li, Y., Zhao, G.: Exploring the online-game life cycle stages. In: E-Business and E-Government (ICEE), 2010 International Conference on. pp. 2436–2438. IEEE (2010)