# A Framework to Determine the Suitability of Software Development Methodologies for the Development of Location-Based Games

Jacques Barnard, Günther Drevin, Magda Huisman

► To cite this version:

Jacques Barnard, Günther Drevin, Magda Huisman. A Framework to Determine the Suitability of Software Development Methodologies for the Development of Location-Based Games. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.335-342, 10.1007/978-3-319-66715-7_36. hal-01771267

HAL Id: hal-01771267

https://inria.hal.science/hal-01771267

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Framework to Determine the Suitability of Software Development Methodologies for the Development of Location-Based Games

Jacques Barnard¹, Günther Drevin¹✉, and Magda Huisman¹

School of Computer, Statistical and Mathematical Sciences,
North-West University,
Potchefstroom Campus,
Potchefstroom,
South Africa

gunther.drevin@nwu.ac.za

magda.huisman@nwu.ac.za

**Abstract.** To determine the suitability of an SDM (software development methodology) for the development of location-based games, it has to be determined to what degree SDMs address aspects that need to be addressed in the process of developing location-based games. These aspects have been identified from information gathered from the literature and were then validated using a survey. Their importance in the development process was also measured using the survey.

A framework that uses these aspects to determine the suitability of SDMs for the development of location-based games was then developed. Applying this framework to a number of existing SDMs it was found, among other, that in general SDMs are well suited for the development of location-based games as far as value to the player is concerned but that they are very lacking when it comes to security and privacy.

**Keywords:** Software Development Methodology, Mobile Games

## 1 Introduction

The gaming industry is expanding rapidly with mobile games becoming an increasingly larger part of the industry. In the 2015 annual report of the Entertainment Software Association (ESA) [10] it was reported that 34% of commuters with a round-trip of more than 3 hours play video games while commuting, with smart phones being used the most (80%) followed by tablets (43%). In 2016 the ESA reported that smart phones were used by 36% of gamers, with PCs being used by 56% of gamers and dedicated game consoles by 53% [11].

At the same time the complexity of developing games has increased exponentially [20] with one of the elements that contribute to this complexity being the multidisciplinary development process [8]. The interaction of different specialist areas, such as art, gameplay, sound, control systems, and human factors,
with traditional software development, indicates that a specialised software engineering methodology is needed for this domain [8]. Elements that lead to the complexity of game development are the technical challenges for the developers as well as tools, project size, workflow and other technical aspects [6].

This paper focuses on location-based games, which can be defined both as a type of ubigame as well as a type of mobile game. Location-based games make use of the player's location and incorporates it in the game play thereby immersing the player even more into the game world [7].

In a previous study unique aspects that need to be addressed during the development of location-based games were identified from the literature and were then validated using a survey. The survey was also used to determine the importance of each of the identified aspects [3, 4].

In this paper these identified aspects are incorporated into a framework that can be used to determine the suitability of an SDM for the development of location-based games. A number of SDMs are then evaluated using the framework.

## 2 Identified Aspects

A list of 15 aspects was identified from literature that focuses on the development process of games, mobile applications and mobile games. These aspects, as well as their importance (column w), are given in Table 1 [3, 4]. The aspects are listed in decreasing order of importance.

The security and privacy aspect scored an average of 7.11, which is lower than the average of 7.33 for all the aspects. From this it would seem that although security and privacy is important to the developers it is not as important as one would expect in the light of security concerns regarding online activity [15, 24]. To ensure the development of secure location-based games, there should be more emphasis on security and privacy in the development process.

## 3 Software Development Methodologies

It is important to understand what SDMs are and how they work in order to develop a framework that can be used to evaluate the suitability of an SDM. An SDM can be defined as the totality of a systems development approach in that it is a set of recommended rules, processes and/or steps that need to be followed [2]. Each of these forms part of the total development process that is usually governed by an underlying philosophy that supports, justifies and incorporates coherent context for a specific development project. Furthermore, an SDM identifies the best procedures, phases, tools, techniques, rules, guidelines, documentation and tasks that are to be used, as well as the best manner to execute each of these.

An SDM in itself consists of four main components, namely a philosophical approach, method, process model and tools and techniques that work together to achieve an integrated process with the objective of improving the development of systems [2, 13]. These components make it possible to evaluate, analyse,
compare and develop SDMs. The philosophical approach defines the perspective
through which the system is developed [2, 20]. The next part of an SDM is the
development method and consists of the steps that must be followed during the
development process. These steps are executed as specified by the process model.
This model therefore dictates the strictness of execution, dependencies and iter-
ations of the steps. Finally, the tools and techniques facilitate the development
process [2, 13].

A number of SDMs were evaluated using the framework developed in this
study. The SDMs that were chosen for the study are usually associated with
game development, mobile development or mobile game development [1, 2, 5,
14, 16, 18]. Furthermore, each SDM has a different combination of philosophical
approach, method, process and set of tools and techniques to better represent
SDMs that are used for these types of development. The SDMs used in this
study are:

- Traditional Systems Development Life Cycle [2, 21]:

The Traditional Systems Development Life Cycle (SDLC), or Waterfall model,
was one of the first SDMs used to develop information systems and was later
adapted and used to develop games.

- Scrum [17]:

The Scrum framework is part of the agile development methodology family
along with Extreme Programming and Feature Driven Development.

- MASAM [16]:

Mobile Application Software Based on Agile Methodology (MASAM) is an
agile methodology that focuses on rapid application development. The key
goal of this methodology is simple development and fast deployment of new
mobile applications.

- Mobile-D [1]:

Mobile-D is based on Extreme Programming, Crystal Methodologies and
Rational Unified Process and is considered to be an agile development ap-
proach. This methodology focuses on developing a mobile application in the
shortest possible time.

- RAD with DSDM Atern [5, 9]:

Rapid Application Development (RAD) focuses on developing and delivering
high quality systems in the shortest possible time and at the lowest cost.
RAD has been implemented as a new methodology, RAD Dynamic Systems
Development Method (DSDM).

- MDA Framework [14]:

Mechanics, Dynamics and Aesthetics framework (MDA) focuses on design-
ing games by breaking the game down into its distinct components; that is
rules that lead to the system and in turn lead to fun. After the components
have been identified their counterparts for designing the game are identified,
namely mechanics that lead to dynamics and in turn lead to aesthetics. The
fundamental idea of MDA is that games are more like artifacts than media.

- Playability and Re-playability [18]:

This methodology is based on the design process of the two key factors
that contribute to the success of a game, viz. playability and re-playability.
Playability and re-playability can be described in terms of six aspects: social reasons, challenge, experience, mastery, impact, and completion. These six aspects of playability and re-playability are used to weigh game features and adjust the design phase of the methodology.

- Player-Centred game design [23]:

The basis for this method is that games differ from business applications in that a business application is developed to satisfy the requirements for which the client asked, whereas games are not developed to satisfy the requirements of a client. This complicates the development process, as the game should satisfy different types of game requirements without having been specifically developed for a specific group of direct clients. This is why this method of game design focuses on the players of the game.

# 4 Evaluation of SDMs

Each of the SDMs chosen for this study was analysed and measured against the aspects identified earlier. Each SDM was awarded a score, which was based on the extent to which the SDM supports all of the aspects. The results of the evaluation are presented in Table 1. It should be noted that the first author, as well as three independent mobile games developers did the evaluation of the SDMs. The developers were given a blank copy of Table 1 and were asked to score, as a percentage, the extent to which each of the SDMs support each of the aspects in the development process.

## 4.1 Inter-Coder Reliability

To determine to what extent the four evaluators scored the aspects for each SDM the same, Krippendorff's alpha was used. The norm for a very good reliability is $α >= 0.8$, while $0.8 > α >= 0.67$ indicates a good reliability and $α < 0.67$ should be rejected [12].

The Krippendorff's alphas that were obtained are given in the last line of Table 1. Each of the Krippendorff's alphas in the table represents the reliability of the evaluators' scores with regard to all of the aspects for each of the SDMs. The Krippendorff's alphas are between 0.706 and 0.828 with an average of 0.734. This confirms good inter-coder reliability for the scoring of the aspects and SDMs.

# 5 Results of Evaluating the SDMs

The summarised results of the evaluation of the SDMs across the aspects are given in Table 1. The weight ($w$) for each aspect is the importance that the aspect received in the survey. In the first row of each aspect the average scores given by the evaluators for each of the SDMs are given as percentages. The second row of each aspect gives the weighted scores of each of the SDMs for that aspect. The calculations for the framework are discussed below.
Using the weight (w) assigned to an aspect and the average score for an SDM, the framework can be used to score the suitability of each SDM for each aspect. This suitability is in the form of a weighted score and is obtained by multiplying the weight (w) for an aspect by the average score that each SDM received for that aspect. For example: the suitability of SDLC for Playability is the weight w = 8.81 multiplied by the average score of 8.75 giving the weighted score of 77.1 as shown in Table 1.

Table 1 also indicates the suitability that an SDM has for all the aspects, as well as the suitability all the SDMs have for a single aspect. Both these suitabilities are given as percentages. The suitability of an SDM for the development of a location-based game is the sum of the weighted scores that the SDM received for each of the aspects. These suitability scores are shown as percentages in Table 1. As can be seen in Table 1, the SDM that was most suitable, with a score of 63%, was Playability and Re-playability, with Mobile-D the least suitable with a score of 32%.

Also given in Table 1 is the degree, on average, to which the SDMs address the different aspects. This value is given in the last column of the table and is the average of the scores assigned by the evaluators. As can be seen in Table 1 the aspect, value to the player, received the highest score (88%), while the aspect, security and privacy, scored the lowest (20%). It is critical that the SDM ensures that the games developers incorporate security and privacy in the development of location-based games.

By evaluating the results of the framework, it is clear that there are a number of inadequacies in the available SDMs. The framework shows this by indicating that the highest suitability percentage for any of the SDMs was 64%. Although the highest scoring aspect across all the SDMs in this framework scored 88%, there are still a number of inadequacies that need to be addressed in the development of location-based games.

# 6 Conclusions and Future Work

Aspects that need to be addressed during the development of location-based games were identified from the literature and were then validated using an on-line survey to gather information from the gaming industry to determine the importance of each aspect. These aspects were then incorporated into a framework, which was used to evaluate a number of SDMs.

It was also noted that security and privacy was not regarded as an important aspect in the development process of location-bases games. An example of where more attention to security could have prevented players from getting into harm's way is the recently released Pokémon GO game [19, 22]. The issue of online safety of children has also been raised by the ESA [10].

The result of this study is a framework that can be used to determine the shortcomings of current SDMs for the development of location-based games and can therefore be used as a guide in the development of an SDM that is more suited for the development of location-based games.
<table><caption>**Table 1. Suitability analysis results**</caption><thead><tr><th></th><th>w</th><th>SDLC</th><th>Scrum</th><th>MASAM</th><th>Mobile-D</th><th>RAD</th><th>MDA</th><th>Playability&<br>Re-playability</th><th>Player</th><th>Aspect<br>centered suitability</th></tr></thead><tbody><tr><td>Playability</td><td>8.81</td><td>8.75<br>77.1</td><td>20<br>176.2</td><td>12.5<br>110.1</td><td>12.5<br>110.1</td><td>16.25<br>143.2</td><td>41.25<br>363.4</td><td>93.75<br>825.9</td><td>58.75<br>517.6</td><td>33%</td></tr><tr><td>Player<br>Experience</td><td>8.70</td><td>67.5<br>587.3</td><td>55<br>478.5</td><td>83.75<br>728.6</td><td>87.5<br>761.35</td><td>84.5<br>735.2</td><td>86.25<br>750.4</td><td>95.75<br>833.0</td><td>95<br>826.5</td><td>82%</td></tr><tr><td>Usability</td><td>7.93</td><td>75.25<br>596.7</td><td>66.25<br>525.4</td><td>75<br>594.8</td><td>22.5<br>178.4</td><td>84.25<br>668.1</td><td>84.5<br>670.1</td><td>90<br>713.7</td><td>88.75<br>703.8</td><td>73%</td></tr><tr><td>Value to<br>Player</td><td>7.67</td><td>78.75<br>604.0</td><td>82.5<br>632.8</td><td>88.75<br>680.7</td><td>90<br>690.3</td><td>87.5<br>671.1</td><td>88.75<br>680.7</td><td>93.75<br>719.1</td><td>93.75<br>719.1</td><td>88%</td></tr><tr><td>UI Design</td><td>7.47</td><td>17.5<br>130.7</td><td>55<br>410.9</td><td>81.25<br>606.9</td><td>20<br>149.4</td><td>78.75<br>588.3</td><td>86.25<br>644.3</td><td>73.75<br>550.9</td><td>77.5<br>578.9</td><td>61%</td></tr><tr><td>Development<br>Team</td><td>7.37</td><td>11.25<br>82.9</td><td>87.5<br>644.9</td><td>52.5<br>386.9</td><td>86.25<br>635.7</td><td>88.75<br>654.1</td><td>13.75<br>101.3</td><td>56.25<br>414.6</td><td>86.25<br>635.7</td><td>60%</td></tr><tr><td>Re-Playability</td><td>7.34</td><td>7.5<br>55.1</td><td>23.75<br>174.3</td><td>10<br>73.4</td><td>5<br>36.7</td><td>12.5<br>91.8</td><td>56.25<br>412.9</td><td>95<br>697.3</td><td>73.75<br>541.3</td><td>35%</td></tr><tr><td>Learnability</td><td>7.15</td><td>7.5<br>53.6</td><td>15<br>107.3</td><td>47.5<br>339.7</td><td>7.5<br>53.6</td><td>41.25<br>295.0</td><td>50<br>357.5</td><td>57.5<br>411.1</td><td>55<br>393.3</td><td>35%</td></tr><tr><td>Efficiency</td><td>7.11</td><td>81.25<br>580.1</td><td>7.5<br>53.6</td><td>5<br>35.7</td><td>6.25<br>44.6</td><td>47.5<br>339.2</td><td>10<br>71.4</td><td>7.5<br>53.6</td><td>7.5<br>53.6</td><td>22%</td></tr><tr><td>Security &<br>Privacy</td><td>7.11</td><td>77.5<br>551.0</td><td>8.75<br>62.2</td><td>5<br>35.6</td><td>7.5<br>53.3</td><td>43.75<br>311.1</td><td>5<br>35.6</td><td>5<br>35.6</td><td>11.25<br>80.0</td><td>20%</td></tr><tr><td>Availability &<br>Accessibility</td><td>7.07</td><td>10<br>70.7</td><td>5<br>35.4</td><td>70<br>494.9</td><td>2.5<br>17.7</td><td>87.5<br>618.7</td><td>2.5<br>17.7</td><td>7.5<br>53.0</td><td>7.5<br>53.0</td><td>24%</td></tr><tr><td>Cognitive<br>Support</td><td>6.90</td><td>68.75<br>474.4</td><td>10<br>69.0</td><td>56.25<br>388.1</td><td>5<br>34.5</td><td>42.5<br>293.3</td><td>77.5<br>534.8</td><td>87.5<br>603.8</td><td>86.25<br>595.1</td><td>54%</td></tr><tr><td>Compatibility</td><td>6.62</td><td>81.25<br>537.9</td><td>7.5<br>49.7</td><td>6.25<br>41.4</td><td>55<br>364.1</td><td>18.75<br>124.1</td><td>26.25<br>173.8</td><td>25<br>165.5</td><td>10<br>66.2</td><td>29%</td></tr><tr><td>Adaptability</td><td>6.40</td><td>71.25<br>456.0</td><td>20<br>128.0</td><td>51.25<br>328.0</td><td>5<br>32.0</td><td>13.75<br>88.0</td><td>81.25<br>520.0</td><td>82.5<br>528.0</td><td>82.5<br>528.0</td><td>51%</td></tr><tr><td>Organisational<br>Structure</td><td>6.23</td><td>50<br>311.5</td><td>78.75<br>490.6</td><td>50<br>311.5</td><td>50<br>311.5</td><td>78.75<br>490.6</td><td>18.75<br>116.8</td><td>53.75<br>334.9</td><td>81.25<br>506.2</td><td>58%</td></tr><tr><td>Suitability<br>of SDM</td><td></td><td>47%</td><td>37%</td><td>47%</td><td>32%</td><td>56%</td><td>50%</td><td>63%</td><td>62%</td><td></td></tr><tr><td>α</td><td></td><td>0.726</td><td>0.743</td><td>0.709</td><td>0.706</td><td>0.828</td><td>0.763</td><td>0.738</td><td>0.733</td><td></td></tr></tbody></table>
References

1. Abrahamsson, P., Hanhineva, A., Hulkko, H., Ihme, T., Jäälinoja, J., Korkala, M., Koskela, J., Kyllönen, P., Salo, O.: Mobile-D: An agile approach for mobile application development. In: Companion to the 19th Annual ACM SIGPLAN Conference on Object-oriented Programming Systems, Languages, and Applications. pp. 174–175. OOPSLA ’04, ACM, New York, NY, USA (2004), http://doi.acm.org/10.1145/1028664.1028736

2. Avison, D., Fitzgerald, G.: Information systems development: methodologies, techniques and tools (3rd edition). McGraw Hill (2003)

3. Barnard, J.: A systems development methodology for developing location based games. Ph.D. thesis, Potchefstroom Campus of the North-West University (2017)

4. Barnard, J., Drevin, G., Huisman, M.: Aspects that need to be addressed during the development of location-based games. Tech. Rep. FABWI-N-RKW-2017-525, School of Computer, Statistical and Mathematical Sciences, Potchefstroom Campus, North-West University (2017)

5. Beynon-Davies, P., Holmes, S.: Design breakdowns, scenarios and rapid application development. Information and Software Technology 44(10), 579–592 (2002)

6. Blow, J.: Game development: Harder than you think. Queue 1(10), 28–37 (Feb 2004), http://doi.acm.org/10.1145/971564.971590

7. Buzeto, F.N., e Silva, T.B.P., Castanho, C.D., Jacobi, R.P.: Reconfigurable games: Games that change with the environment. In: 2014 Brazilian Symposium on Computer Games and Digital Entertainment. pp. 61–70 (Nov 2014)

8. Callele, D., Neufeld, E., Schneider, K.: Requirements engineering and the creative process in the video game industry. In: Proceedings of the 13th IEEE International Conference on Requirements Engineering. pp. 240–252. RE ’05, IEEE Computer Society, Washington, DC, USA (2005), http://dx.doi.org/10.1109/RE.2005.58

9. DSDM-Consortium: DSDM Atern: the Handbook. (2008), https://www.dsdm.org/resources/dsdm-handbooks/the-dsdm-agile-project-framework-2014-onwards

10. Entertainment Software Association: Annual report. A year of innovation and achievement (2015), http://www.theesa.com/wp-content/uploads/2016/04/ESA-Annual-Report-2015-1.pdf

11. Entertainment Software Association: Sales, demographic and usage data. Essential facts about the computer and video game industry (2016), http://essentialfacts.theesa.com/Essential-Facts-2016.pdf

12. Hayes, A.F., Krippendorff, K.: Answering the call for a standard reliability measure for coding data. Communication methods and measures 1(1), 77–89 (2007)

13. Huisman, M., Iivari, J.: Deployment of systems development methodologies: Perceptual congruence between is managers and systems developers. Inf. Manage. 43(1), 29–49 (Jan 2006), http://dx.doi.org/10.1016/j.im.2005.01.005

14. Hunicke, R., Leblanc, M., Zubek, R.: MDA: A formal approach to game design and game research. In: Proceedings of the Challenges in Games AI Workshop, Nineteenth National Conference of Artificial Intelligence. pp. 1–5 (2004)

15. Jain, A.K., Shanbhag, D.: Addressing security and privacy risks in mobile applications. IT Professional 14, 28–33 (2012)

16. Jeong, Y.J., Lee, J.H., Shin, G.S.: Development process of mobile application SW based on agile methodology. In: Advanced Communication Technology, 2008. ICACT 2008. 10th International Conference on. vol. 1, pp. 362–366. IEEE (2008)

17. Keith, C.: Agile game development with Scrum. Pearson Education (2010)
18. Krall, J., Menzies, T.: Aspects of replayability and software engineering: Towards a methodology of developing games (2012)

19. Reuters: Pokémon GO: US senator probes maker over data privacy concerns (July 2016), http://gadgets.ndtv.com/apps/news/pokemon-go-us-senator-probes-maker-over-data-privacy-concerns-860167

20. Reyno, E.M., Cubel, J.A.C.: Model-driven game development: 2d platform game prototyping. In: Proceedings of the 13th IEEE International Conference on Requirements Engineering. pp. 5–7. GAMEON’2008, EUROSIS, Ostend, Belgium (2008)

21. Royce, W.W., et al.: Managing the development of large software systems. In: Proceedings of IEEE WESCON. vol. 26, pp. 1–9. Los Angeles (1970)

22. Summers, N.: UK children’s charity says ‘Pokémon GO’ needs more safety features (July 2016), https://www.engadget.com/2016/07/13/nspcc-letter-niantic-pokeemon-go/

23. Sykes, J., Federoff, M.: Player-centred game design. In: CHI’06 extended abstracts on Human factors in computing systems. pp. 1731–1734. ACM (2006)

24. Zhu, H., Xiong, H., Ge, Y., Chen, E.: Mobile app recommendations with security and privacy awareness. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 951–960. KDD ’14, ACM, New York, NY, USA (2014), http://doi.acm.org/10.1145/2623330.2623705