# An Application of a Game Development Framework inHigher Education

Authors: Alf Inge Wang; Bian Wu
Journal: International Journal of Computer Games Technology
 (2009)

---

## Abstract

This paper describes how a game development framework was used as a learning aid in a software engineering
. Games can be used within higher education in various ways to promote student participation, enable variation in how lectures are taught, and improve student interest. In this paper, we describe a case study at the Norwegian University of Science and Technology (NTNU) where a game development framework was applied to make students learn software architecture by developing a computer game. We provide a model for how game development frameworks can be integrated with a software engineering or computer science course. We describe important requirements to consider when choosing a game development framework for a course and an evaluation of four frameworks based on these requirements. Further, we describe some extensions we made to the existing game development framework to let the students focus more on software architectural issues than the technical implementation issues. Finally, we describe a case study of how a game development framework was integrated in a software architecture course and the experiences from doing so.


---

## Body

## 1. Introduction

Games have been used in schools for many years to help
children learn skills in math, language, geography, science, and other domains
in an interesting and motivating way. Research shows that integrating games
within a classroom with children can be beneficial for academic achievement,
motivation, and classroom dynamics [1]. There is also evidence that the
teaching methods based on educational games are not only attractive to
schoolchildren, but also to university students [2]. There have been conducted researcheson games concept and game development used in higher education before,
for example, [3–5], but we believe there is an untapped potential that needs to
be explored. Games can provide teachers in higher education with teaching aids that
can promote more active students, provide alternative teaching methods to improve
variation, and enable social learning through multiplayer learning games.Games can be integrated in higher education in three
ways. First, games can be used
instead of traditional exercises motivating students to put extra effort in
doing the exercises and giving the teacher and/or teaching assistants an
opportunity to monitor how the students work with the exercises in real time
[6, 7]. Second, games can be used
within lectures to improve the participation and motivation of students [8, 9]. In this approach, the students and the teacher participate in
knowledge-based games. Third, the
students are required to develop a game as a part of a course using a game
development framework (GDF) to learn skills within computer science or software
engineering [10]. This paper focuses on the latter, where game development and
a GDF are used in student projects to learn software engineering skills,
extending the use of games as a teaching aid in higher education. The
motivation of making students develop games to learn software engineering is to
bring the students’ enthusiasm from playing games to learn  courses through
game development. In addition, we wanted to investigate if the specific
features of a GDF are suitable for teaching software engineering and how game
development can be integrated with the education process. More specifically, we
wanted to explore how the use of game development and the GDF would affect the
learning of software architecture with focus on the technical aspects of the
GDF.This paper focuses on how the technical aspects of a
GDF affect the learning of software architecture, the selection of appropriate
GDF for a software architecture course, and how a GDF can be applied in a
software engineering course. The main contribution of this paper is a
presentation of a novel GDF concept that can be used in courses that includes
software development, experiences from actual usage of the GDF, and some course
design considerations.The rest of the paper is organized as follows. Section2 describes and motivates for how a GDF can be used in higher education and
what criteria should be considered when choosing one. Section 3 describes a
case study of applying a GDF in a software architecture course. Section 4
describes experiences from using a GDF in a software course. Section 5
describes similar approaches, and Section 6 concludes the paper.


## 2. Game Development Frameworks in Higher Education

This section presents the motivation for applying GDFs
in higher education, a model for how GDFs can be integrated with a course, and
requirements for how to choose the appropriate GDF for educational purposes.


### 2.1. GDF and Education

The main motivation for introducing GDF in software
engineering (SE) or computer science (CS) courses is to motivate students to
put more effort into software development project in order to improve software
development skills. Game development offers an interesting way of learning and
applying the course theory. By introducing a game development project in a
course, the students have to establish and describe most of the functional
requirements themselves (what the game should be like). This can be a
motivating factor especially for group-based projects, as each group will
develop a unique application (the game); it will encourage creativity, and it
will require different skills from the group members (art, programming, story,
audio/music). The result will be that the students will have a stronger feeling
of ownership to the project. Furthermore, students also could learn about game
development technology. The main disadvantages by introducing a game
development project and a GDF into a SE or CS course is that the student might
spend too much time on game-specific issues and that the project results might
be difficult to compare.  It is critical
that the students get motivated applying a GDF in a course and that they get
increased motivation for learning and applying course theory through a game
development project.Tom Malone has listed three main characteristics that
make things fun to learn: they should provide the appropriate level of
challenge, they should use fantasy and abstractions to make it more
interesting, and they should trigger the player’s curiosity [11]. These
characteristics can directly be applied when developing a game for learning
purposes. However, we can also consider these characteristics when introducing
a GDF in a SE or CS course.  By allowing
the students to develop their own games using a GDF, such projects are likely
to trigger students’ curiosity as well as provide a challenge for students to
design fun games with their knowledge, skills, imagination, and creativity. The
level of the challenge can be adjusted according to the project requirements
given in courses by the teacher. Thus, the challenge level can not only be
adjusted to the right level for most participants, but also tailored for
individual differences. As the students will work in groups, group members
helping other group members can compensate for the individual differences. An
open platform and agile courses requirements should be provided for students to
design their own games, combined with their ability, fantasy, and comprehension
of lecture content.The main benefit of using a GDF as a teaching aid is
that it can be a motivating initiative in courses to learn about various topics
such as software requirements, software design, software architecture,
programming, 2D and 3D graphic representation, graphic programming, artificial
intelligence, physics, animation, user interfaces, and many other areas within
computer science and software engineering. It is most useful for learning new
skills and methods within a specific domain but also useful for testing and
rehearsing theory by applying know skills and knowledge in a project using a
GDF.


### 2.2. Circulatory Model of Applying a GDF in a Course

There are several good reasons for introducing a GDF
and game development projects in CS and SE courses as described in previous
section, but in order to make it a success it is important that the GDF is well
integrated with the course. Based on our experiences, we have developed a
circular model for how to apply a GDF in a CS or SE course through six steps
(see Figure1).  The model is intended
for courses where a software development project is a major part of the course.Figure 1
Circulatory model of GDF’s application in courses.To choose one appropriate development platform
according to the course content, it is important to consider the process of the
course related to the development project. This process starts with choosing an
appropriate GDF (step A) for the course related to some requirements (described
in the next section). Next, the design of exercises and projects (step B) must
reflect the limitations and constraints of the chosen GDF. In the initial phase
of the student project, it is important that the students get the required
technical guidance and appropriate requirements (step C) related to the GDF. It
is important that the students get to know the GDF early, for example, by introducing
an exercise to implement a simple game in the GDF. It is critical that there is
sufficient course staff that knows the GDF well enough to give the required
feedback. The next step is for the students to start designing and implementing
(step D) their own game according to the constraints within the course and the
GDF. After the students have delivered their final version of their project
implementation and documentation, the students should get the chance to
evaluate and analyze (step E) their own projects to learn from their successes
and mistakes. This information should then be used to provide feedback in order
to improve the course (step F). The feedback from the students might indicate
that another GDF should be used or that the course constraints on the projects
should be altered. The core of this model is that the teacher should encourage
the students to explore the course theory through a game development project
using a GDF and give the opportunity to improve the game development project
through feedback from the students.


### 2.3. Criteria for Choosing the Right GDF

How to
choose an appropriate GDF that easily can be integrated with course content
should be based on the educational goals of the course, the technical level and
skills of students, and the time available for projects and/or exercises. Based
on experiences from using GDFs and from student projects in CS and SE courses,
we have come up with the following requirements for choosing a GDF for a CS or
SE course.(1)
It must be easy to learn and allow rapid development. According to Malone’s recommendation of how to make things fun to learn, it is
crucial that we provide the appropriate level of challenge. If the GDF is too
much of a challenge and requires too much to learn before becoming productive,
the whole idea of game development will be wasted, as the student will lose
motivation. An important aspect of this is that the GDF offers high-level APIs
that makes it possible for the students to develop impressive results without
writing too many lines of code. This is especially critical in the first phase
of the project.(2)
It must provide an open development environment to attract students’
curiosity. Malone claims that fantasy and curiosity are other
important factors that make things fun to learn. By providing a relatively open
GDF without too many restrictions on what you can produce, the students get a
chance to realize the game of their dreams. This means that the GDF itself
should not restrict what kind of game the students can make. This requirement
would typically rule out GDFs that are tailored for producing only one game
genre such as adventure games, platform games, or board games. In addition,
ideally an open development environment should offer public and practical
interfaces for developers to extend their own functions. In this respect, open
source game development platforms are preferred.(3)
It must support programming languages that are familiar to the students. The students should not be burdened to have to learn a new programming
language from scratch in addition to the course content. This would take away
the focus of the educational goals of the course. We suggest to choose GDFs
that support popular programming languages that the students know like C++, C#,
or Java. It is also important that the programming languages supported by the
GDF have high-level constructs and libraries that enable the programmers to be
more productive as less code is required to produce fully functional systems.
From an educational point of view, programming languages like Java and C# are
better suited than C and C++, as they have more constraints that force the
programmers to write cleaner code, and there is less concern related to issues
like pointers and memory leakage. From a game development perspective, programming
languages like C and C++ are more attractive as they generally produce faster
executables and thus faster games.(4)
It must not conflict with the educational goals of the course. When
choosing a GDF it is important that the inherent patterns, procedures, design,
and architecture of the GDF are not in conflict with the theory taught in the
course. One example of such a conflict could be that the way the GDF enforces
event handling in an application is given as an example of bad design in the
textbook.(5)
It must
have a stable implementation. When a GDF is used in a course, it is
essential that the GDF has few bugs so the students do not have to fight a lot
of technical issues instead of focusing on the course topics. This requirement
indicates that it is important that the GDF is supported by a company or a
development community that has enough resources to eliminate serious technical
insufficiencies. It is also important that the development of the GDF is not a
dead project, as this will lead to compatibility issues for future releases of
operating systems, software components, and hardware drivers.(6)
It must have sufficient documentation. This requirement is
important for both the course staff and the students. The documentation should
both give a good overview of the GDF as well as document all the features
provided. Further, it is important that the GDF provides tutorials and examples
to demonstrate how to use the GDF and its features. The frameworks should
provide documentation and tutorials of high quality enabling self-study.(7)
It should be inexpensive (low costs) to use and acquire. Ideally,
the GDFs should be free or have very low associated cost to avoid extra costs
running the course. This requirement also involves investigating additional
costs related to the GDF such as requirements for extra or more powerful
hardware and/or requirements for additional software.The goal
of the requirements above is to save the time and effort the students have to
spend on coding and understanding the framework, making them concentrate on the
course content and software design. Thus, an appropriate GDF could provide the
students with exciting experiences and offer a new way of learning through a new
domain (games). The requirements above are also important for the course staff,
as they will help to find a GDF that would cause less effort spent on technical
issues, and incompatibility between GDF and the course contents.From the
requirements above, we acknowledge that there is a conflict between requirements
one and two. The level of the freedom the developer is given to make whatever
game he likes could be in conflict with providing a development environment
that allows rapid development and is easy to learn. A more open GDF usually
means that the developer must learn more APIs as well as the APIs themselves
are usually of lower level, and thus harder to use. However, it is possible to get
a bit of both worlds by offering high-level APIs that are relatively easy to
use but still allow the developer to access underlying APIs that give the
developer the freedom in what kind of games  can be made. This means that
the GDF can allow inexperienced developers to just modify simple APIs or
example code to make variants of existing games, or to allow more experienced
developers to make unique games by using more of the provided underlying APIs. How
hard the GDF is to use will then really depend on the ambition of the game
developer and not on the GDF itself. This can also be a motivating factor to
learn more about the GDF’s APIs.


## 2.1. GDF and Education

The main motivation for introducing GDF in software
engineering (SE) or computer science (CS) courses is to motivate students to
put more effort into software development project in order to improve software
development skills. Game development offers an interesting way of learning and
applying the course theory. By introducing a game development project in a
course, the students have to establish and describe most of the functional
requirements themselves (what the game should be like). This can be a
motivating factor especially for group-based projects, as each group will
develop a unique application (the game); it will encourage creativity, and it
will require different skills from the group members (art, programming, story,
audio/music). The result will be that the students will have a stronger feeling
of ownership to the project. Furthermore, students also could learn about game
development technology. The main disadvantages by introducing a game
development project and a GDF into a SE or CS course is that the student might
spend too much time on game-specific issues and that the project results might
be difficult to compare.  It is critical
that the students get motivated applying a GDF in a course and that they get
increased motivation for learning and applying course theory through a game
development project.Tom Malone has listed three main characteristics that
make things fun to learn: they should provide the appropriate level of
challenge, they should use fantasy and abstractions to make it more
interesting, and they should trigger the player’s curiosity [11]. These
characteristics can directly be applied when developing a game for learning
purposes. However, we can also consider these characteristics when introducing
a GDF in a SE or CS course.  By allowing
the students to develop their own games using a GDF, such projects are likely
to trigger students’ curiosity as well as provide a challenge for students to
design fun games with their knowledge, skills, imagination, and creativity. The
level of the challenge can be adjusted according to the project requirements
given in courses by the teacher. Thus, the challenge level can not only be
adjusted to the right level for most participants, but also tailored for
individual differences. As the students will work in groups, group members
helping other group members can compensate for the individual differences. An
open platform and agile courses requirements should be provided for students to
design their own games, combined with their ability, fantasy, and comprehension
of lecture content.The main benefit of using a GDF as a teaching aid is
that it can be a motivating initiative in courses to learn about various topics
such as software requirements, software design, software architecture,
programming, 2D and 3D graphic representation, graphic programming, artificial
intelligence, physics, animation, user interfaces, and many other areas within
computer science and software engineering. It is most useful for learning new
skills and methods within a specific domain but also useful for testing and
rehearsing theory by applying know skills and knowledge in a project using a
GDF.


## 2.2. Circulatory Model of Applying a GDF in a Course

There are several good reasons for introducing a GDF
and game development projects in CS and SE courses as described in previous
section, but in order to make it a success it is important that the GDF is well
integrated with the course. Based on our experiences, we have developed a
circular model for how to apply a GDF in a CS or SE course through six steps
(see Figure1).  The model is intended
for courses where a software development project is a major part of the course.Figure 1
Circulatory model of GDF’s application in courses.To choose one appropriate development platform
according to the course content, it is important to consider the process of the
course related to the development project. This process starts with choosing an
appropriate GDF (step A) for the course related to some requirements (described
in the next section). Next, the design of exercises and projects (step B) must
reflect the limitations and constraints of the chosen GDF. In the initial phase
of the student project, it is important that the students get the required
technical guidance and appropriate requirements (step C) related to the GDF. It
is important that the students get to know the GDF early, for example, by introducing
an exercise to implement a simple game in the GDF. It is critical that there is
sufficient course staff that knows the GDF well enough to give the required
feedback. The next step is for the students to start designing and implementing
(step D) their own game according to the constraints within the course and the
GDF. After the students have delivered their final version of their project
implementation and documentation, the students should get the chance to
evaluate and analyze (step E) their own projects to learn from their successes
and mistakes. This information should then be used to provide feedback in order
to improve the course (step F). The feedback from the students might indicate
that another GDF should be used or that the course constraints on the projects
should be altered. The core of this model is that the teacher should encourage
the students to explore the course theory through a game development project
using a GDF and give the opportunity to improve the game development project
through feedback from the students.


## 2.3. Criteria for Choosing the Right GDF

How to
choose an appropriate GDF that easily can be integrated with course content
should be based on the educational goals of the course, the technical level and
skills of students, and the time available for projects and/or exercises. Based
on experiences from using GDFs and from student projects in CS and SE courses,
we have come up with the following requirements for choosing a GDF for a CS or
SE course.(1)
It must be easy to learn and allow rapid development. According to Malone’s recommendation of how to make things fun to learn, it is
crucial that we provide the appropriate level of challenge. If the GDF is too
much of a challenge and requires too much to learn before becoming productive,
the whole idea of game development will be wasted, as the student will lose
motivation. An important aspect of this is that the GDF offers high-level APIs
that makes it possible for the students to develop impressive results without
writing too many lines of code. This is especially critical in the first phase
of the project.(2)
It must provide an open development environment to attract students’
curiosity. Malone claims that fantasy and curiosity are other
important factors that make things fun to learn. By providing a relatively open
GDF without too many restrictions on what you can produce, the students get a
chance to realize the game of their dreams. This means that the GDF itself
should not restrict what kind of game the students can make. This requirement
would typically rule out GDFs that are tailored for producing only one game
genre such as adventure games, platform games, or board games. In addition,
ideally an open development environment should offer public and practical
interfaces for developers to extend their own functions. In this respect, open
source game development platforms are preferred.(3)
It must support programming languages that are familiar to the students. The students should not be burdened to have to learn a new programming
language from scratch in addition to the course content. This would take away
the focus of the educational goals of the course. We suggest to choose GDFs
that support popular programming languages that the students know like C++, C#,
or Java. It is also important that the programming languages supported by the
GDF have high-level constructs and libraries that enable the programmers to be
more productive as less code is required to produce fully functional systems.
From an educational point of view, programming languages like Java and C# are
better suited than C and C++, as they have more constraints that force the
programmers to write cleaner code, and there is less concern related to issues
like pointers and memory leakage. From a game development perspective, programming
languages like C and C++ are more attractive as they generally produce faster
executables and thus faster games.(4)
It must not conflict with the educational goals of the course. When
choosing a GDF it is important that the inherent patterns, procedures, design,
and architecture of the GDF are not in conflict with the theory taught in the
course. One example of such a conflict could be that the way the GDF enforces
event handling in an application is given as an example of bad design in the
textbook.(5)
It must
have a stable implementation. When a GDF is used in a course, it is
essential that the GDF has few bugs so the students do not have to fight a lot
of technical issues instead of focusing on the course topics. This requirement
indicates that it is important that the GDF is supported by a company or a
development community that has enough resources to eliminate serious technical
insufficiencies. It is also important that the development of the GDF is not a
dead project, as this will lead to compatibility issues for future releases of
operating systems, software components, and hardware drivers.(6)
It must have sufficient documentation. This requirement is
important for both the course staff and the students. The documentation should
both give a good overview of the GDF as well as document all the features
provided. Further, it is important that the GDF provides tutorials and examples
to demonstrate how to use the GDF and its features. The frameworks should
provide documentation and tutorials of high quality enabling self-study.(7)
It should be inexpensive (low costs) to use and acquire. Ideally,
the GDFs should be free or have very low associated cost to avoid extra costs
running the course. This requirement also involves investigating additional
costs related to the GDF such as requirements for extra or more powerful
hardware and/or requirements for additional software.The goal
of the requirements above is to save the time and effort the students have to
spend on coding and understanding the framework, making them concentrate on the
course content and software design. Thus, an appropriate GDF could provide the
students with exciting experiences and offer a new way of learning through a new
domain (games). The requirements above are also important for the course staff,
as they will help to find a GDF that would cause less effort spent on technical
issues, and incompatibility between GDF and the course contents.From the
requirements above, we acknowledge that there is a conflict between requirements
one and two. The level of the freedom the developer is given to make whatever
game he likes could be in conflict with providing a development environment
that allows rapid development and is easy to learn. A more open GDF usually
means that the developer must learn more APIs as well as the APIs themselves
are usually of lower level, and thus harder to use. However, it is possible to get
a bit of both worlds by offering high-level APIs that are relatively easy to
use but still allow the developer to access underlying APIs that give the
developer the freedom in what kind of games  can be made. This means that
the GDF can allow inexperienced developers to just modify simple APIs or
example code to make variants of existing games, or to allow more experienced
developers to make unique games by using more of the provided underlying APIs. How
hard the GDF is to use will then really depend on the ambition of the game
developer and not on the GDF itself. This can also be a motivating factor to
learn more about the GDF’s APIs.


## 3. Case Study: Applying a GDF in a Software Architercture Course

This section describes a case study of a software
architecture course at the Norwegian University of Science and Technology
(NTNU) where a GDF was introduced.


### 3.1. The Software Architecture Course

The software architecture course is a postgraduate
course offered to CS and SE students at NTNU. The course is taught every
spring, its workload is 25% of one semester, and about 70 postgraduate students
attend the course every semester. The students in the course are mostly of
Norwegian students (about 80%), but there are about 20% foreign students mostly
from EU-countries.  The textbook used in
this course is the “Software Architecture in Practice, Second Edition”, by
Bass, Clements et al. [12]. Additional papers are used to cover topics that
are not sufficiently covered by the book such as design patterns, software
architecture documentation standards, view models, and postmortem analysis [13–16]. The education goal of the course is:“The students
should be able todefine and explain central concepts in software architecture literature and be able to use and describe design/architectural
patterns, methods to design software architectures, methods/techniques to
achieve software qualities, methods to document software architecture, and
methods to evaluate software architecture.”The course is taught in four main ways:(1)
ordinary lectures given in English;(2)
invited guest lectures from the software industry;(3)
exercise in design patterns;(4)
a software development project with emphasis on software architecture.
30% of the grade is based on an evaluation of a software
architecture project that all students have to do, while 70% is given from the
results of a written examination. The goal of the project is for the students
to apply the methods and theory in the course to design a software architecture
and to implement a system according to the architecture. The project consists
of the following phases.(1)
COTS (Commercial Off-The-Shelf) exercise: learn the development platform
to be used in the project by developing some simple test applications.(2)
Design pattern: learn how to utilize design pattern by making changes in
an existing system designed with and without design patterns.(3)
Requirements and architecture: describe the functional and the quality
requirements, and design the software architecture for the application in the
project.(4)
Architecture evaluation: use the Architecture Trade-off Analysis Method
(ATAM) [12, 17] to evaluate the software architecture in regards to the quality
requirements. Here one student group will evaluate another student group’s
project.(5)
Implementation: do a detailed design and implement the application based
on the created architecture and based on the results from a previous phase.(6)
Project evaluation: evaluate the project after it has been completed
using a Post-Mortem Analysis (PMA) method.
In the two first phases of the project, the students
work on their own or in pairs. For the phases 4–6, the students work in
self-composed groups of four students. The students spend most time on the
implementation phase (6 weeks), and they are also encouraged to start the
implementation in earlier phases to test their architectural choices
(incremental development). In previous years, the goal of the project has been
to develop a robot controller for a robot simulator in Java with emphasis on an
assigned quality attribute such as availability, performance, modifiability, or
testability.


### 3.2. Choosing a GDF for the Software Architecture Course

In Fall
2007, we started to look for appropriate GDFs to be used in the software
architecture course in spring 2008. We looked for both GDFs where the programmer
had to write the source code as well as visual drag-and-drop programming
environments. The selection of candidates was based on GDFs we were familiar
with and GDFs that had developer support. Further, we wanted to compare both
commercial and open source GDFs. From an initial long list candidate GDFs, we
chose to evaluate the following GDFs more in detail.(i)XNA:
XNA is a GDF from Microsoft that enables
development of homebrew cross-platform games for Windows and the XBOX 360
using the C# programming language. The initial version of Microsoft XNA
Game Studio was released in 2006 [18], and in 2008 Microsoft XNA Game
studio 3.0 was released that includes support for making games for XBOX
Live. XNA features a set of high-level API enabling the development of
advanced games in 2D or 3D with advanced graphical effects with little
effort. The XNA platform is free and allows developers to create games
for Windows, Xbox 360, and Zune using the same GDF [19]. XNA consists
of an integrated development environment (IDE) along with several tools
for managing audio and graphics.(ii)JGame:
JGame is a high-level
framework for developing 2D games in Java [20]. JGame is an open source
project and enables developers to develop games fast using few lines of
code as JGame will take care of typical game functionality such as
sprite handling, collision detection, and tile handling. JGame games can be
run as stand-alone Java games, Java applets games running in a web browser
or on mobile devices (Java ME). JGame does not provide a separate IDE but
is integrated with Eclipse.(iii)Flash:
Flash is a high-level
framework for interactive applications including games developed by Adobe
[21]. Most programming in Flash is carried out in Action script (a textual
programming language), but the Flash environment also provides a powerful
graphical editor for managing graphical objects and animation. Flash
applications can run as stand-alone applications or in a web browser.
Flash applications can run on many different operating systems like
Windows, Mac OS X, and Linux as well as on mobile devices and game consoles
(Nintendo Wii and Sony Playstation 3). Programming in Flash is partly
visual by manipulating graphical objects, but most code is written
textually. Flash supports development of both 2D and 3D applications.(iv)Scratch:
is a visual
programming environment developed by MIT Media Lab in collaboration with
UCLA that makes it easy to create interactive stories, animations, games,
music, and art and to share the creations on the web [22].
Scratch works similar to Alice [23] allowing you to program by placing
sprites or objects on a screen and manipulate them by drag-and-drop
programming. The main difference between Scratch and Alice is that Scratch
is in 2D while Alice is in 3D. Scratch provides its own graphical IDE that
includes a set of programming primitives and functionality to import
various multimedia objects.An evaluation
of the four GDF candidates is shown in Table1. From
the four candidates, we found Scratch to be the least appropriate candidate.
The main disadvantage with Scratch was that it would be very difficult to teach
software architecture using this GDF, as the framework did not allow exploring
various software architectures. Further, Scratch was also very limited in what
kind of games that could be produced, limiting the options for the students.
The main advantage using Scratch is that it is very easy to learn and use.
JGame suffered also from some of the same limitations as Scratch, as it put
some restrictions on what software architecture  could be used, and it had
little flexibility in producing a variety of types of games. The main advantage
using JGame was that it was an open source project with access to the source
code and that all the programming was done in Java. All CS and SE students at
NTNU learn Java in the two first introductory programming courses. An attractive
alternative would be to use Flash as a GDF. Many developers use Flash to create
games for kids as well as games for the Web. Flash puts little restrictions on
what kind of games you can develop (both 2D and 3D), but there are some
restrictions on what kind of software architecture  you can use in your
applications. The programming language used in Flash, Action Script, is not
very different from Java so it should be rather easy for the students to learn.
The main disadvantage using Flash in the software architecture course was the
license costs. As the computer and information science department does not have
a site license for the Flash development kit, it would be too expensive to use.
XNA was found an attractive alternative for the students, as it made it
possible for them to create their own XBOX 360 games. XNA puts little
restrictions on what kinds of software architectures you apply in your software,
and it enables the developers to create almost any game. XNA has strong support
from its developer (Microsoft) and has a strong community of developers along
with a lot of resources (graphics, examples, etc.). The main disadvantages using
XNA as a GDF in the course were that the students had to learn C# and that the
software could only run on Windows machines. Compared to JGame and other
Java-based GDFs, XNA has a richer set of high-level APIs and a more mature
architecture.Table 1
Evaluation of  four GDF candidates.
Selection requirementXNAJGameFlashScratch(1) Easy to learnRelatively easy to learn, but requires to learn
several core concepts to utilize the offered possibilities.Easy to learn, but requires to learn a small set of
core concepts.Relatively easy to learn, but requires to learn
several core concepts to utilize the offered possibilities.Very easy and intuitive to learn and supports
dynamic changes to the game in run time.(2) Open develop environmentXNA puts little restrictions on what kind of games can be developed and supports development of both 2D and 3D games. Not open source project.JGame supports a limited set of games, mainly classical 2D arcade games. Open source project.Flash puts little restrictions on what kind of games can be developed and supports development of both 2D and 3D. Not open source project.Scratch limits the options of what kind of games the user can make through the limited options provided in the graphical programming environment. Not open source project.(3) Familiar programming languageAll programming is done in C#.All programming is done in JavaSome programming can be done using drag-and-drop, but most will be written in Action Scripts.All programming is done in the visual drag-and-drop programming language Scratch.(4) Not in conflict with educational goalsXNA putslittle restrictions on what kinds of software architectures can be used.JGame putssome restrictions on what kinds of software architecture  can be used.Flash putssome restrictions on what kinds of software architectures can be used.Scratch putsstrict restrictions on what kinds of software architectures  can be used.(5) Stable implementationXNA has a very stable implementation and is updated regularly.JGame has a relatively stable implementation and is updated regularly.Flash has a very stable implementation and is updated regularly.Scratch has a relatively stable implementation and is updated regularly.(6) Sufficient documentationXNA is well documented and offers several tutorials and examples. Many books on XNA are available.JGame is not well documented, but some examples exist.Flash is well documented and offers several tutorials and examples. Many books on Flash are available.Scratch is ok documented and has some examples and tutorials available.(7) Low costsXNA is free to use. A $99 for a year of membership is required to develop games for XBOX 360.JGame is free to use.The Flash development kit costs $199 per license (university license).Scratch is free to use.Based on the evaluation described above, we chose XNA as a GDF for our course. From
previous experience we knew that it does not require much effort and time to learn C# for
students that already know Java.


### 3.3. XQUEST—An Extension of the Chosen GDF

After we had decided to use XNA as a GDF in the
software architecture course, we launched a project to extend XNA to make XNA
even easier to use in the student project. This project implemented XQUEST (XNA
QUick & Easy Starter Template) [24], which is a small and lightweight 2D
game library/game template developed at NTNU that contains convenient game
components, helper classes, and other classes that can be used in the XNA game
projects (see Figure 2). The goal of XQUEST was to identify and abstract common
game programming tasks and create a set of components that could be used by
students of the course to make their life easier. We choose to focus only on
2D. There are a few reasons for this. First,
the focus of the student projects is software architecture, not making a game
with fancy 3D graphics. Second,
students unfamiliar with game programming and 3D programming may find it
daunting to have to learn the concepts needed for doing full-blown 3D in XNA,
such as shader programming and 3D modeling, in addition to software
architectures. To keep the projects in 2D may reduce the effect of students
focus only on the game development and not on the software architecture issues.Figure 2
The XQUEST library shown in the XNA development environment.


### 3.4. Teaching Software Architecture Using XNA

XNA was introduced in the software architecture course
to motivate students to put extra effort in the student project with the goal
to learn the course content such as attribute driven design, design and
architectural patterns, ATAM, design of software architecture, view points, and
implementation of software architecture. This section will go through the
different phases of this project and describe how XNA affected these phases.


### 3.4.1. Introduction of XNA Exercises

In the start of the semester the course staff gave an
introduction to course where the software architecture project was presented.
Before the students started with their project, they had to do an exercise
individually or in pairs where they got to choose their own partner. The goal
of the first exercise was to get familiar with the XNA framework and
environment, and the students were asked to complete four tasks.(1)
Draw a helicopter sprite on the screen and make it move around on its
own.(2)
Move around the helicopter sprite from previous task using the keyboard,
change the size of the sprite when a key was pressed, rotate the sprite when
another key was pressed, and write the position of the sprite on the screen.(3)
Animate the helicopter sprite using several frames and do sprite
collision with other sprites.(4)
Create the classical Pong game in XNA.Before the students started on their XNA introduction
exercise, they got a two-hour technical introduction to XNA. During the
semester, two technical assistants were assigned to help students with issues
related to XNA. These assistants had scheduled two hours per week to help
students with problems, in addition to answering emails about XNA issues.


### 3.4.2. Requirement and Architecture for the Game Project

After the
introduction exercise was delivered, the students formed groups of four students.
Students that did not knew anyone were assigned to groups. The course staff
then issued the project task where the goal was to make a functioning game
using XNA based on students’ own defined game concept. However, the game had to
be designed and implemented according to their specified and designed software
architecture. Further, the students had to develop a software architecture that
focused on one particular quality attribute assigned by the course staff. We
used the following definitions for the quality attributes in the game projects:Modifiability, the game architecture
and implementation should be easy to change in order to add or modify
functionality;  Testability, the
game architecture and implementation should be easy to test in order to detect
possible faults and failures. These two quality attributes were related to the
course content and the textbook. A perfect implementation was not the ultimate
quest of this XNA game project, but it was critical that the implementation
reflected the architectural description. It was also important that the final
delivery was well structured, easy to read, and made according to the template
provided by the course staff.The first
phase of the project was the requirement and architecture phase where the
students should deliver requirements and the software architecture of the game
along with a skeleton code reflecting the architecture. The requirements
document focused on a complete functional requirement description of the game
and several quality requirements for the game described as scenario focusing on
one particular quality attribute. The architectural description was the most
important part of the final delivery of  the game project, and the students
had to document their architecture according to IEEE 1471-2000 [25]. The
architecture documentation could be altered several times before its final
delivery. Table 2 lists
main attributes required in the architectural description in the game projects.Table 2
List of architecture description for the game project.
#Architectural description attributesDetails of the implementation(1)Architectural driversThe main drivers that affect the system
mostly, including the attribute on which the students focus.(2)Stakeholders and concernsStakeholders of the system, and their
concerns.(3)Selection of architectural viewpointA list of the viewpoints used, and  their purpose,
target audience, and form of description. Places to look in for possible
viewpoints including the book [12] and the 4 + 1 article by Kruchten [26].(4)Quality tacticsIncluding all attributes and more details for the
focused ones.(5)Architectural patternsThe major patterns of your architecture, both
architectural and major design ones.(6)ViewsA separate section for each required views:
logic, process, and development views or other views added by students.(7)Consistency among viewsDiscuss the consistency between each described view.(8)Architectural rationaleIn this section and subsections, add why
things are chosen.We also
required that the students wrote the code skeleton for the architecture they
had designed. This was done to emphasize the importance of starting the
implementation early and to ensure that students designed an architecture that
was possible to implement.


### 3.4.3. Evaluation of the Game Project

After the
requirements, the architecture and the code skeleton were delivered; the
student groups were assigned to evaluate each other’s architecture using ATAM.
The whole idea was for one project group to evaluate the architecture of the
other group’s game to give feedback on the architecture related to the quality
focus of the software architecture [27]. It included attribute utility tree,
analysis of architectural approach, sensitivity points, trade-off points, risks
and non-risks, and risk themes.


### 3.4.4. Detailed Design and Implementation

The focus
of implementation phase was to design, implement, and test the game application.
The documentation delivered in this phase focused on the test results from
running the game related to the specified requirements and the discussion of
the relationship between the implemented game and the architectural
documentation [14, 15]. Table 3 lists
what should be delivered in the implementation phase.Table 3
Design & implementation phase description.
#Implementation deliverablesDetails of implementation(1)Design and implementationA more detailed view of the various parts of the
architecture description of game design.(2)User’s manualTo guide the users;
the steps to compile and run the game.(3)Test reportContains both
functional requirements and quality requirements (quality scenarios).(4)Relationship with the architectureList the
inconsistencies between the game architecture and the implementation and the
reasons for these inconsistencies.(5)Problems, issues, and points learnedListing problems and
issues with the document or with the implementation process.For the test report part in the Table3, the
functional requirements and quality requirements had the attributes like shown
in Table 4 and Table 5. The test reports should also include a discussion about the
observation of the test unless there was nothing to discuss about the test
results.Table 4
Attributes of functional requirements.
F1: The role in game should be able to jump along happilyExecutorSuper Mario IIIDate23.3.2005Time used5 minEvaluationFail: White role cannot jump!Table 5
Attributes of quality requirements.
A1: The role in game should not get stuckExecutorSnurre SprettDate24.3.2005StimuliThe role should be able to move around for 10 minExpected responseSuccess in 8 of 10 executionsObserved responseSuccess in 3 of 10 executionsEvaluationFailAt the end of this phase, the students had to submit
their final delivery of their projects that included all documents, code, and
other material from all project phases. The course staff evaluated all the
groups’ deliveries and gave grades by judging document and implementation
quality, document and implementation completeness, architecture design, and
readability and structure of code and report.


### 3.4.5. The Game Project Workshop

In this workshop, selected groups had to give short
presentations about the project goal, quality attribute focus, proposed
architectural solution with some diagrams or explanations, and an evaluation of
how well did the solution worked related to functional requirements and quality
focus. Further, the selected groups ran demos of their games, and it was opened
for questions from the audience.The workshop provided an open mind environment to let
students give each other feedback, brainstorm about improvements and ideas, and
to discuss their ideas to give a better understanding of the course content and
game architecture design.


### 3.4.6. Post-Mortem Analysis

In the final task in the project, every group had to
perform a post-mortem analysis of their project. The focus of the PMA was to
analyze successes and problems of the project. The PMA was documented in a
short report that included a positive (successes) and a negative (problems)
KJ diagram (structured brainstorm map), a positive and a negative causal map (a
diagram that shows cause-effect relationships), and experiences from using PMA
[13].  The PMA made the students reflect
on their performance in the project and gave them useful feedback to improve in the
future projects and inputs for the course staff to improve the course. The main
topics analyzed in the PMA were issues related to group dynamics, time
management, technical issues, software architecture issues, project
constraints, and personal conflicts.


## 3.1. The Software Architecture Course

The software architecture course is a postgraduate
course offered to CS and SE students at NTNU. The course is taught every
spring, its workload is 25% of one semester, and about 70 postgraduate students
attend the course every semester. The students in the course are mostly of
Norwegian students (about 80%), but there are about 20% foreign students mostly
from EU-countries.  The textbook used in
this course is the “Software Architecture in Practice, Second Edition”, by
Bass, Clements et al. [12]. Additional papers are used to cover topics that
are not sufficiently covered by the book such as design patterns, software
architecture documentation standards, view models, and postmortem analysis [13–16]. The education goal of the course is:“The students
should be able todefine and explain central concepts in software architecture literature and be able to use and describe design/architectural
patterns, methods to design software architectures, methods/techniques to
achieve software qualities, methods to document software architecture, and
methods to evaluate software architecture.”The course is taught in four main ways:(1)
ordinary lectures given in English;(2)
invited guest lectures from the software industry;(3)
exercise in design patterns;(4)
a software development project with emphasis on software architecture.
30% of the grade is based on an evaluation of a software
architecture project that all students have to do, while 70% is given from the
results of a written examination. The goal of the project is for the students
to apply the methods and theory in the course to design a software architecture
and to implement a system according to the architecture. The project consists
of the following phases.(1)
COTS (Commercial Off-The-Shelf) exercise: learn the development platform
to be used in the project by developing some simple test applications.(2)
Design pattern: learn how to utilize design pattern by making changes in
an existing system designed with and without design patterns.(3)
Requirements and architecture: describe the functional and the quality
requirements, and design the software architecture for the application in the
project.(4)
Architecture evaluation: use the Architecture Trade-off Analysis Method
(ATAM) [12, 17] to evaluate the software architecture in regards to the quality
requirements. Here one student group will evaluate another student group’s
project.(5)
Implementation: do a detailed design and implement the application based
on the created architecture and based on the results from a previous phase.(6)
Project evaluation: evaluate the project after it has been completed
using a Post-Mortem Analysis (PMA) method.
In the two first phases of the project, the students
work on their own or in pairs. For the phases 4–6, the students work in
self-composed groups of four students. The students spend most time on the
implementation phase (6 weeks), and they are also encouraged to start the
implementation in earlier phases to test their architectural choices
(incremental development). In previous years, the goal of the project has been
to develop a robot controller for a robot simulator in Java with emphasis on an
assigned quality attribute such as availability, performance, modifiability, or
testability.


## 3.2. Choosing a GDF for the Software Architecture Course

In Fall
2007, we started to look for appropriate GDFs to be used in the software
architecture course in spring 2008. We looked for both GDFs where the programmer
had to write the source code as well as visual drag-and-drop programming
environments. The selection of candidates was based on GDFs we were familiar
with and GDFs that had developer support. Further, we wanted to compare both
commercial and open source GDFs. From an initial long list candidate GDFs, we
chose to evaluate the following GDFs more in detail.(i)XNA:
XNA is a GDF from Microsoft that enables
development of homebrew cross-platform games for Windows and the XBOX 360
using the C# programming language. The initial version of Microsoft XNA
Game Studio was released in 2006 [18], and in 2008 Microsoft XNA Game
studio 3.0 was released that includes support for making games for XBOX
Live. XNA features a set of high-level API enabling the development of
advanced games in 2D or 3D with advanced graphical effects with little
effort. The XNA platform is free and allows developers to create games
for Windows, Xbox 360, and Zune using the same GDF [19]. XNA consists
of an integrated development environment (IDE) along with several tools
for managing audio and graphics.(ii)JGame:
JGame is a high-level
framework for developing 2D games in Java [20]. JGame is an open source
project and enables developers to develop games fast using few lines of
code as JGame will take care of typical game functionality such as
sprite handling, collision detection, and tile handling. JGame games can be
run as stand-alone Java games, Java applets games running in a web browser
or on mobile devices (Java ME). JGame does not provide a separate IDE but
is integrated with Eclipse.(iii)Flash:
Flash is a high-level
framework for interactive applications including games developed by Adobe
[21]. Most programming in Flash is carried out in Action script (a textual
programming language), but the Flash environment also provides a powerful
graphical editor for managing graphical objects and animation. Flash
applications can run as stand-alone applications or in a web browser.
Flash applications can run on many different operating systems like
Windows, Mac OS X, and Linux as well as on mobile devices and game consoles
(Nintendo Wii and Sony Playstation 3). Programming in Flash is partly
visual by manipulating graphical objects, but most code is written
textually. Flash supports development of both 2D and 3D applications.(iv)Scratch:
is a visual
programming environment developed by MIT Media Lab in collaboration with
UCLA that makes it easy to create interactive stories, animations, games,
music, and art and to share the creations on the web [22].
Scratch works similar to Alice [23] allowing you to program by placing
sprites or objects on a screen and manipulate them by drag-and-drop
programming. The main difference between Scratch and Alice is that Scratch
is in 2D while Alice is in 3D. Scratch provides its own graphical IDE that
includes a set of programming primitives and functionality to import
various multimedia objects.An evaluation
of the four GDF candidates is shown in Table1. From
the four candidates, we found Scratch to be the least appropriate candidate.
The main disadvantage with Scratch was that it would be very difficult to teach
software architecture using this GDF, as the framework did not allow exploring
various software architectures. Further, Scratch was also very limited in what
kind of games that could be produced, limiting the options for the students.
The main advantage using Scratch is that it is very easy to learn and use.
JGame suffered also from some of the same limitations as Scratch, as it put
some restrictions on what software architecture  could be used, and it had
little flexibility in producing a variety of types of games. The main advantage
using JGame was that it was an open source project with access to the source
code and that all the programming was done in Java. All CS and SE students at
NTNU learn Java in the two first introductory programming courses. An attractive
alternative would be to use Flash as a GDF. Many developers use Flash to create
games for kids as well as games for the Web. Flash puts little restrictions on
what kind of games you can develop (both 2D and 3D), but there are some
restrictions on what kind of software architecture  you can use in your
applications. The programming language used in Flash, Action Script, is not
very different from Java so it should be rather easy for the students to learn.
The main disadvantage using Flash in the software architecture course was the
license costs. As the computer and information science department does not have
a site license for the Flash development kit, it would be too expensive to use.
XNA was found an attractive alternative for the students, as it made it
possible for them to create their own XBOX 360 games. XNA puts little
restrictions on what kinds of software architectures you apply in your software,
and it enables the developers to create almost any game. XNA has strong support
from its developer (Microsoft) and has a strong community of developers along
with a lot of resources (graphics, examples, etc.). The main disadvantages using
XNA as a GDF in the course were that the students had to learn C# and that the
software could only run on Windows machines. Compared to JGame and other
Java-based GDFs, XNA has a richer set of high-level APIs and a more mature
architecture.Table 1
Evaluation of  four GDF candidates.
Selection requirementXNAJGameFlashScratch(1) Easy to learnRelatively easy to learn, but requires to learn
several core concepts to utilize the offered possibilities.Easy to learn, but requires to learn a small set of
core concepts.Relatively easy to learn, but requires to learn
several core concepts to utilize the offered possibilities.Very easy and intuitive to learn and supports
dynamic changes to the game in run time.(2) Open develop environmentXNA puts little restrictions on what kind of games can be developed and supports development of both 2D and 3D games. Not open source project.JGame supports a limited set of games, mainly classical 2D arcade games. Open source project.Flash puts little restrictions on what kind of games can be developed and supports development of both 2D and 3D. Not open source project.Scratch limits the options of what kind of games the user can make through the limited options provided in the graphical programming environment. Not open source project.(3) Familiar programming languageAll programming is done in C#.All programming is done in JavaSome programming can be done using drag-and-drop, but most will be written in Action Scripts.All programming is done in the visual drag-and-drop programming language Scratch.(4) Not in conflict with educational goalsXNA putslittle restrictions on what kinds of software architectures can be used.JGame putssome restrictions on what kinds of software architecture  can be used.Flash putssome restrictions on what kinds of software architectures can be used.Scratch putsstrict restrictions on what kinds of software architectures  can be used.(5) Stable implementationXNA has a very stable implementation and is updated regularly.JGame has a relatively stable implementation and is updated regularly.Flash has a very stable implementation and is updated regularly.Scratch has a relatively stable implementation and is updated regularly.(6) Sufficient documentationXNA is well documented and offers several tutorials and examples. Many books on XNA are available.JGame is not well documented, but some examples exist.Flash is well documented and offers several tutorials and examples. Many books on Flash are available.Scratch is ok documented and has some examples and tutorials available.(7) Low costsXNA is free to use. A $99 for a year of membership is required to develop games for XBOX 360.JGame is free to use.The Flash development kit costs $199 per license (university license).Scratch is free to use.Based on the evaluation described above, we chose XNA as a GDF for our course. From
previous experience we knew that it does not require much effort and time to learn C# for
students that already know Java.


## 3.3. XQUEST—An Extension of the Chosen GDF

After we had decided to use XNA as a GDF in the
software architecture course, we launched a project to extend XNA to make XNA
even easier to use in the student project. This project implemented XQUEST (XNA
QUick & Easy Starter Template) [24], which is a small and lightweight 2D
game library/game template developed at NTNU that contains convenient game
components, helper classes, and other classes that can be used in the XNA game
projects (see Figure 2). The goal of XQUEST was to identify and abstract common
game programming tasks and create a set of components that could be used by
students of the course to make their life easier. We choose to focus only on
2D. There are a few reasons for this. First,
the focus of the student projects is software architecture, not making a game
with fancy 3D graphics. Second,
students unfamiliar with game programming and 3D programming may find it
daunting to have to learn the concepts needed for doing full-blown 3D in XNA,
such as shader programming and 3D modeling, in addition to software
architectures. To keep the projects in 2D may reduce the effect of students
focus only on the game development and not on the software architecture issues.Figure 2
The XQUEST library shown in the XNA development environment.


## 3.4. Teaching Software Architecture Using XNA

XNA was introduced in the software architecture course
to motivate students to put extra effort in the student project with the goal
to learn the course content such as attribute driven design, design and
architectural patterns, ATAM, design of software architecture, view points, and
implementation of software architecture. This section will go through the
different phases of this project and describe how XNA affected these phases.


## 3.4.1. Introduction of XNA Exercises

In the start of the semester the course staff gave an
introduction to course where the software architecture project was presented.
Before the students started with their project, they had to do an exercise
individually or in pairs where they got to choose their own partner. The goal
of the first exercise was to get familiar with the XNA framework and
environment, and the students were asked to complete four tasks.(1)
Draw a helicopter sprite on the screen and make it move around on its
own.(2)
Move around the helicopter sprite from previous task using the keyboard,
change the size of the sprite when a key was pressed, rotate the sprite when
another key was pressed, and write the position of the sprite on the screen.(3)
Animate the helicopter sprite using several frames and do sprite
collision with other sprites.(4)
Create the classical Pong game in XNA.Before the students started on their XNA introduction
exercise, they got a two-hour technical introduction to XNA. During the
semester, two technical assistants were assigned to help students with issues
related to XNA. These assistants had scheduled two hours per week to help
students with problems, in addition to answering emails about XNA issues.


## 3.4.2. Requirement and Architecture for the Game Project

After the
introduction exercise was delivered, the students formed groups of four students.
Students that did not knew anyone were assigned to groups. The course staff
then issued the project task where the goal was to make a functioning game
using XNA based on students’ own defined game concept. However, the game had to
be designed and implemented according to their specified and designed software
architecture. Further, the students had to develop a software architecture that
focused on one particular quality attribute assigned by the course staff. We
used the following definitions for the quality attributes in the game projects:Modifiability, the game architecture
and implementation should be easy to change in order to add or modify
functionality;  Testability, the
game architecture and implementation should be easy to test in order to detect
possible faults and failures. These two quality attributes were related to the
course content and the textbook. A perfect implementation was not the ultimate
quest of this XNA game project, but it was critical that the implementation
reflected the architectural description. It was also important that the final
delivery was well structured, easy to read, and made according to the template
provided by the course staff.The first
phase of the project was the requirement and architecture phase where the
students should deliver requirements and the software architecture of the game
along with a skeleton code reflecting the architecture. The requirements
document focused on a complete functional requirement description of the game
and several quality requirements for the game described as scenario focusing on
one particular quality attribute. The architectural description was the most
important part of the final delivery of  the game project, and the students
had to document their architecture according to IEEE 1471-2000 [25]. The
architecture documentation could be altered several times before its final
delivery. Table 2 lists
main attributes required in the architectural description in the game projects.Table 2
List of architecture description for the game project.
#Architectural description attributesDetails of the implementation(1)Architectural driversThe main drivers that affect the system
mostly, including the attribute on which the students focus.(2)Stakeholders and concernsStakeholders of the system, and their
concerns.(3)Selection of architectural viewpointA list of the viewpoints used, and  their purpose,
target audience, and form of description. Places to look in for possible
viewpoints including the book [12] and the 4 + 1 article by Kruchten [26].(4)Quality tacticsIncluding all attributes and more details for the
focused ones.(5)Architectural patternsThe major patterns of your architecture, both
architectural and major design ones.(6)ViewsA separate section for each required views:
logic, process, and development views or other views added by students.(7)Consistency among viewsDiscuss the consistency between each described view.(8)Architectural rationaleIn this section and subsections, add why
things are chosen.We also
required that the students wrote the code skeleton for the architecture they
had designed. This was done to emphasize the importance of starting the
implementation early and to ensure that students designed an architecture that
was possible to implement.


## 3.4.3. Evaluation of the Game Project

After the
requirements, the architecture and the code skeleton were delivered; the
student groups were assigned to evaluate each other’s architecture using ATAM.
The whole idea was for one project group to evaluate the architecture of the
other group’s game to give feedback on the architecture related to the quality
focus of the software architecture [27]. It included attribute utility tree,
analysis of architectural approach, sensitivity points, trade-off points, risks
and non-risks, and risk themes.


## 3.4.4. Detailed Design and Implementation

The focus
of implementation phase was to design, implement, and test the game application.
The documentation delivered in this phase focused on the test results from
running the game related to the specified requirements and the discussion of
the relationship between the implemented game and the architectural
documentation [14, 15]. Table 3 lists
what should be delivered in the implementation phase.Table 3
Design & implementation phase description.
#Implementation deliverablesDetails of implementation(1)Design and implementationA more detailed view of the various parts of the
architecture description of game design.(2)User’s manualTo guide the users;
the steps to compile and run the game.(3)Test reportContains both
functional requirements and quality requirements (quality scenarios).(4)Relationship with the architectureList the
inconsistencies between the game architecture and the implementation and the
reasons for these inconsistencies.(5)Problems, issues, and points learnedListing problems and
issues with the document or with the implementation process.For the test report part in the Table3, the
functional requirements and quality requirements had the attributes like shown
in Table 4 and Table 5. The test reports should also include a discussion about the
observation of the test unless there was nothing to discuss about the test
results.Table 4
Attributes of functional requirements.
F1: The role in game should be able to jump along happilyExecutorSuper Mario IIIDate23.3.2005Time used5 minEvaluationFail: White role cannot jump!Table 5
Attributes of quality requirements.
A1: The role in game should not get stuckExecutorSnurre SprettDate24.3.2005StimuliThe role should be able to move around for 10 minExpected responseSuccess in 8 of 10 executionsObserved responseSuccess in 3 of 10 executionsEvaluationFailAt the end of this phase, the students had to submit
their final delivery of their projects that included all documents, code, and
other material from all project phases. The course staff evaluated all the
groups’ deliveries and gave grades by judging document and implementation
quality, document and implementation completeness, architecture design, and
readability and structure of code and report.


## 3.4.5. The Game Project Workshop

In this workshop, selected groups had to give short
presentations about the project goal, quality attribute focus, proposed
architectural solution with some diagrams or explanations, and an evaluation of
how well did the solution worked related to functional requirements and quality
focus. Further, the selected groups ran demos of their games, and it was opened
for questions from the audience.The workshop provided an open mind environment to let
students give each other feedback, brainstorm about improvements and ideas, and
to discuss their ideas to give a better understanding of the course content and
game architecture design.


## 3.4.6. Post-Mortem Analysis

In the final task in the project, every group had to
perform a post-mortem analysis of their project. The focus of the PMA was to
analyze successes and problems of the project. The PMA was documented in a
short report that included a positive (successes) and a negative (problems)
KJ diagram (structured brainstorm map), a positive and a negative causal map (a
diagram that shows cause-effect relationships), and experiences from using PMA
[13].  The PMA made the students reflect
on their performance in the project and gave them useful feedback to improve in the
future projects and inputs for the course staff to improve the course. The main
topics analyzed in the PMA were issues related to group dynamics, time
management, technical issues, software architecture issues, project
constraints, and personal conflicts.


## 4. Experiences of Using GDF in Software Architecture

The experiences described in this section are based on
the final course evaluation, feedback from the students during the project, and
the project reports.The final
course evaluation made all students (mandatory) taking the course answer three
questions. The results reported below are a summary of the students’ responses
related to the project and the GDF.(1) What
have been good about software architecture course?(a)
About
the project itself: “Cool project”, “Really interesting project”,
“We had a lot of fun during the project”, “It is cool to make a game”,
“Fun to implement something practical such a game”, “Videogame as an
exercise is quite interesting”, “I really liked the project”, and “The game
was motivating and fun”.(b)
Project
and learning: “Good architectural discussion in the project
group I was in”, “Learned a lot about software architecture during the
project”, “The project helped to understand better the arguments explained
in the lectures, having fun at the meantime”, “Fun project where we
learned a lot”, “I think that the creation of a project from the
beginning, with the documentation until the code implementation, was very
helpful to better understand in practice the focus of the course”, “The
game project was tightly connected to the syllabus and lectures and gave
valuable experience. The main thing I learned was probably how much
simpler everything gets if you have a good architecture as a basis for
your system”, and “The interplay of game and architectural approaches”.(c)
The
project being practical work: “I think it was pretty good that you guys made
us do a lot of practical work”, and “To choose C# as a platform is a good idea
as it is used a lot in the software industry; at the same time it is very
similar to Java so it is rather easy to learn the language.”(d)
Interplay
between groups: “It was also good to see the results of the
others' projects in the final presentation”.(2) What
have been not so good about the course software architecture?(a)
XNA
support: “The way the student assistants were organized;
during the implementation periods at least they should be available in a
computer lab and not just in the classroom”, “Maybe the use of XNA
Framework XQUEST was very difficult because I never use it. Maybe some
extra lecture focus on the use of XQUEST Framework was better”, and “We did not
have lectures on XNA; could have got some more basic info…Hmm…”(b)
XNA
versus software architecture: “Took a lot of time getting to know C#, I liked
it, but I did not have the time to study architecture” and “The use of game
as a project may have removed some of the focus away from the
architecture. XNA and games in general limit the range of useful
architectures.”(3) What
would you have changed for next year's course?(a)
Project
workload: “Maybe just little more time to develop the
game” and “I would change the importance of the project. I think that the
workload of the project was very big and it can matter the 50% of the
total exam.”(b)
XNA
support: “Perhaps have some C# intro?” and “It would be
helpful to have some lab hours”.(c)
Project
constraints: “Maybe more restrictions on game type, to ensure
that the groups choose games suited for architectural experimentation.”The
responses from the students were overall very positive. In the previous years,
the students in the software architecture course had to design the architecture
and implement a robot controller for a robot simulator in Java. The feedback
from the XNA project was much more positive than the feedback from the robot
controller project. Other positive feedback we got from the students was that
they felt they learned a lot from the game project, that they liked the
practical approach of the project and having to learn C#, and the interaction
between the groups (both ATAM and the project workshop).The
negative feedback from the course evaluation was focusing on the lack of XNA
support and technical support during the project and that some student felt
that there was too much focus on C#, XNA, and games and too little on software
architecture.The
suggestions to improve the course were mainly according to the negative
feedback, namely, to improve XNA support and to adjust the workload of the
project. One student also suggested limiting the types of games to be
implemented in project to ensure more focus on software architectural
experimentation.


## 4.1. Snapshots from Some Student Projects

Figure3
shows screenshots from four student game projects. The game at the upper left
corner is a racing game, the game at the upper right corner is a platform game,
and the two games below are role-playing games (RPGs). Some of the XNA games
developed were original and interesting. Most games were entertaining but were
lacking contents and more than one level due to time constraints.Game based on XNA framework (Top left: Racing; Top right: Codename Gordon; Bottom: RPG).
(a)(b)(c)(d)


## 5. Related Work

This paper describes experiences from utilizing the
special features of a GDF in a software architecture course. The main benefits
from applying a GDF in a CS or SE course is that the students get more
motivated during the software development project. As far as we know, there are
few papers that describe the usage of a professional GDF concept applied in
universities courses that is not directly a target for learning game development,
especially no papers about usage of XNA in higher education. However, there are
some related approaches in education described in this section.El-Nasr and Smith describes how the use of modifying
ormodding existing games can be used
to learn computer science, mathematics, physics, and ascetic principles [10]. The
paper describes how they used modding of the WarCraft III engine to teach high
school students a class on game design and programming. Further, they describe
experiences from teaching university students a more advanced class on game
design and programming using the Unreal Tournament 2003 engine. Finally, they
present observations from student projects that involve modding of game
engines. Although the paper claims to teach students other things than pure
game design and programming, the GDFs were used in the context of game
development courses.The framework Minueto [28] is implemented in Java, and
it is used by students in their second year of undergraduate studies at McGill
University in Montreal, Canada. The framework encapsulates graphics, audio, and keyboard/mouse
inputs to simplify Java game development. It allows development of 2D games,
such as card games and strategy games, but it lacks in support for visual
programming and suffers from limited documentation.The Labyrinth [29] is implemented in Java, and it is a
flexible and easy-to-use computer game framework. The framework enables
instructors to expose students to very specific aspects of computer science
courses. The framework is a finished game in the Pac-Man genre, highly modular,
and it lets the students change different aspects of the game. However, it
cannot be used to develop different genres of game, and there is little room for
changing the software architecture of the framework.The JIG (Java Instructional Gaming) Project [30] is a
collaborative effort between Scott Wallace (Washington State University
Vancouver) and Andrew Nierman (University of Puget Sound) in conjunction with a
small group of dedicated students. It has three aims: (1) to build a Java
Instructional Game Engine suitable for a wide variety of students at all levels
in the curriculum; (2) to create a set of educational resources to support the
use of the game engine at small, resource-limited, schools;  (3) to develop a
community of educators that use and help improve these resources. The JIG
Project was proposed in 2006, after a survey of existing game engines revealed
a very limited supply of existing 2D Java game engines. JIG is still in
development.GarageGames [31] offers two game engines written in
C++. The Torque Game Engine targets 3D games, while the Game Builder provides a
2D API and encourages programmers to develop using a proprietary language (C++
can also be used). Both engines are aimed at a wide audience, including
students and professionals. The engines are available under separate licenses
($50 per license per year for each engine) that allow full access to the source
code. Documentation and tutorials cover topics appropriate for beginners and
advanced users.The University of Michigan’s DXFramework [32] game
engine is written in C++. The current version is targeted specifically for 2D
games, although previous versions have included a 3D API as well. This engine
is designed for game programming education and is in its third major iteration.
The DXFramework is an open source project. Compared to XNA, DXFramework has no
competitive advantage as it has limited support for visual programming, and it
is not easier than XNA to learn.The University of North Texas’s SAGE [33] game engine
is written in C++, and targets 3D games, not 2D. Like the DXFramework, SAGE is
targeted specifically for game programming educational usage. The source code
can be downloaded and is currently available without license.Marist College’s GEDI [34] game engine provides a
second alternative for 2D game design in C++, and is also designed with game
programming educational use in mind. Source code can be downloaded and is
currently available without license, but GEDI is still in the early phases of
development. Only one example game is distributed with the code, and little
documentation is available.For business teaching, Arena3D [35] is a game
visualization framework with its animated 3D representations of the work
environments; it simulates patients queuing at the front desk and interacts
with the staff. IBM has also produced a business game called INNOV8 [36] which
is “an interactive, 3-D business simulator designed to teach the fundamentals
of business process management and bridge the gap in understanding between
business leaders and IT teams in an organization”.


## 6. Conclusion and Future Work

In this paper we have presented a case study of how a
GDF was evaluated, chosen, and integrated with a software architecture course.
The main goal of introducing a GDF and a game development project in this
course was to motivate students to learn more about software architecture
during the game development project. The positive feedback from the students
indicates that this was a good choice as the students really enjoyed the project
and learned software architecture from carrying out the project.We will continue to explore the area of using games,
games concept, and game development in CS and SE education and evaluate how this
affects the students’ motivation and performance. The choice of XNA as a GDF
proved to be a good choice for our software architecture course. The main
disadvantage using XNA is the lack of support for non-Windows operating systems
like Linux and Mac OS X. Mono. XNA is a cross platform implementation of the XNA
game framework that allows XNA to run on Windows, Mac OS X, and Linux using
OpenGL [37]. The project is still in an early phase. An alternative to solve
this problem is to let the students choose between different GDFs, for example, XNA
and a Java-based GDF. The main challenge for this approach is that the course staff
needs to know all the GDFs offered to the students to give proper technical
assistance. Based on the feedback from the students; the technical support is
very important and must be considered before providing choices of more GDFs.



---

*Source: 693267-2009-05-25.xml*