Faraday Discussions
Cite this: DOI: 10.1039/c4fd00014e

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

PAPER

View Article Online
View Journal

Immune Attack players perform better on atest of cellular immunology and self
conﬁdence than their classmates who playa control video game†
Melanie Stegman*Received 17th February 2014, Accepted 5th March 2014
DOI: 10.1039/c4fd00014e

Introduction
As molecular scientists, we understand the fundamentals of molecular behavior
that underlie evolution, infection, physiology and environmental contamination.
However, the basic fundamentals of molecular behavior are considered too
abstract to teach to grade school or high school students in the context of cellularbiology. Analogies are used to describe the functions of cells and their organelles.
Concepts such as protein production and respiration are taught from a systemic,
large scale perspective, typically without presenting the role of any individual
proteins in these cellular processes. The molecular behavior of, say, ATP and its
random diﬀusion is not taught in context with mitochondrial function. The next
generation science teaching standards do not emphasize molecular behavior, andexplicitly state that assessment of students should not include any information
about individual proteins or a biochemical level of understanding of cellular
processes.7 Only students who choose to take an advanced level high school
course or who opt into college level biology courses are exposed to the fundamentals of molecular biology.
This lack of detail and lack of exposure through high school to the fundamentals of molecular behavior leave the general public without the basic
understanding required to grasp cellular biology or to understand new data.
Because this new data oen pertains to personal and public health decisions,these concepts are important for non-scientists to understand. Memorizing every
step of glycolysis is not necessary. However, a fundamental understanding of how
cellular processes are driven by concentrations of substrates, products, allostericinhibitors; how enzymes, products and substrates diﬀuse in random directions,that through random interactions specic binding occurs due to very particular
FAS Learning Technology Program, 500 Wall Street #409, Seattle, Washington, 98121, United States. E-mail:mstegman@fas.org; Tel: +1-917-886-6079† Electronic supplementary information (ESI) available: See DOI: 10.1039/c4fd00014e

This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Paper

aspects of shape, charge and other molecular forces, and that processes require aparticular enzyme for each step, would serve students as a scaﬀold for a lifetime oflearning.
Brunner's work on a game (The Ruby Realm) to teach photosynthesis to middle
school students demonstrates that trying to teach cellular processes like photosynthesis without presenting the molecular interactions leads to misconceptions.
She found that students consistently had trouble understanding that the mass ofa plant comes from the sun and air, rather than the dirt, until she introduced themolecules and their interactions in the game. Using analogies and a large scale
systematic approach (as recommended by most science teaching standards) did
not help students comprehend photosynthesis because the large scale characteristics of dirt, air, and plant material are so diﬀerent. By allowing students to“shoot” carbon dioxide molecules with sunlight and form glucose molecules withthem, Brunner was able to teach a deeper understanding of photosynthesis toyounger students.12
Klymkowsky and colleagues have demonstrated that misconceptions arecommonly held by biology undergraduates as they enter college. Additionally,
these misconceptions concern fundamental molecular behaviors, such as diﬀusion and the random motion of molecules and proteins.5 Klymkowsky reported
on a multi-year eﬀort to address misconceptions in undergraduate biology classes
and concluded that these misconceptions are diﬃcult to erase and re-teach.5
Teaching the basis of molecular behavior to younger people can help address thisproblem.
Jenkinson and McGill have shown that showing greater detail in biochemistry
videos achieves a higher level of understanding.2 Namely, they created four
versions of a receptor and its ligand interacting in space. Where typical videos
show a ligand and receptor only and place the ligand on a clear path to thereceptor, Jenkinson and McGill increased the randomness of the ligands
approach and also added water molecules that collide with the ligand. They
created four variations of the receptor and ligand video. They found that the more
complicated videos were best for evoking a deeper understanding of the processof a ligand randomly approaching and binding to its receptor. Jenkinson and
McGill found that this deeper understanding wore oﬀ aer two weeks, as thegroups that saw the more complex videos failed to answer the deeper understanding questions correctly two weeks later.3 Jenkinson and McGill state thatcomplexity may be required to properly teach molecular interactions and thatvideos with audio and interactive elements may be the most eﬀective.
What is missing is a tool to present the complex, abstract fundamentals ofmolecular behavior to younger students so that by the time we teach them thespecics of respiration or photosynthesis they can understand these processes
deeply. The tool would 1) present a great deal of detail, presenting many types ofobjects that interact and have numerous traits that dictate their activities; 2) allow
for interaction and puzzle solving: allowing students to manipulate, explore anduse the objects they nd to aﬀect the state of the molecular world; 3) be engaging,fun and interesting to non-scientists so that the general public and younger
students will take advantage of the tool and 4) attract students back to play again
and again over a long period of time so that the deeper understanding does notwear oﬀ and so fundamentals used in the tool have an aﬀect on what the students
are currently learning in school.
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Faraday Discussions

The tool we need has already been created: the video game.1,4,6,8,9 A video gamewith a story, set in a complex molecular world, in which the players not only
watch, but also interact with and listen to in-game characters, may well be the best
way to impart understanding of complex interactions among molecular entities.
Complex games with multiple ways to achieve success oen bring players back toreplay, oen over many years time. A game set in a real, complex world of
molecules and cells may be an ideal way to introduce molecular behavior.
Successful commercial video games, that keep players coming back to replay,
have a learning curve; a period of time and practice is required of the player tolearn the game mechanism and the controls. A game complex enough to engage
players in a story, with the ability to manipulate and experiment with objects will
have a lengthy learning curve, as opposed to a casual vocabulary guessing game.
Therefore, we are interested in knowing whether a complicated, third person
shooter that involves controlling a ship in 3D space as well as shooting targets will
have a learning curve that inhibits learning of the terms and concepts presented.
Additionally, we are interested in knowing whether this learning curve inhibits
players' condence with the material.
Commercially successful video games do not have a list of multiple choicequestions to answer or a vocabulary test. However, each level requires learning.
Oen some vocabulary is required to nd objects and places on a map, or tochoose weapons or to read tips about how to kill aliens. There are many potential
ways that learning may occur in a video game. Therefore, we are interested in
knowing whether we can demonstrate any retention of terms and concepts when
the game itself does not ask any explicit questions of the player. Will players learn
vocabulary that they are not forced to recall in the game? Will players learn aboutprocesses? Will they remember attributes of objects that they are not required tointeract with? Will they be able to extrapolate what they learn to the real world?
Will the game graphics, vocabulary, and situations prepare them for working withreal materials on the subject? These questions are addressed by our evaluation of
Immune Attack, and our data is presented in the results and discussion.
There is a precedent that concepts can be learned in a story-oriented,
commercially successful game that is not designed to teach explicitly. Civilization
is designed to be an engaging experience, not to teach facts about history.14
However, Kurt Squire demonstrated that middle school students gained a valuable perspective on the situations, geographies, issues and technology innovations that lie at the heart of many events we learn about in history. In summary,
Civilization players who have defended a large empire on a mountain from manysmaller invading groups can change their strategy when they start their city on alow plains area.9 In other words, the players have learned about how armies andgeography interact in general. This general understanding of the complex interactions should provide the player with a scaﬀold for a deeper understanding of
history. This type of general understanding of the interactions between moleculesand cells is what we propose that a video game can provide. This understanding ofmolecular behavior is what we think will foster a deeper understanding of ahealth and biology related curriculum.
Immune Attack is a video game that requires players to activate specicproteins to cause specic behaviors of various white blood cells to win seven gamelevels. Immune Attack is a third person shooter-styled video game. Players y a
microbot and a nanobot through veins, through connective tissue and over theThis journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Paper

surface of white blood cells, while receiving instructions to “shoot” various
proteins at appropriate times to solve various failings of a patient's immune
system. Shooting proteins with an “EM emitter” causes the proteins to become
active and perform their task.
Immune Attack is designed so that the action of the level is blended well withthe science. Through a simple game mechanism (point and shoot), Immune
Attack is able to present a wide range of molecular situations to players without
introducing a new game mechanism. And the mechanism is reasonably wellmatched with the meaning of the game.
Proteins and cells can be manipulated in Immune Attack; they have roughly
accurate sizes, shapes and clear roles to play in the action and drama of each level.
In Immune Attack, molecules and cells are not just words we need to memorize,
they are tools we need to use. Therefore, objects and concepts have scaﬀolding
upon which to learn them. This should allow Immune Attack to present moredetail and more abstract concepts to younger audiences. We also expect that theuse of these objects in the game will foster students' abilities to recall their names
and functions aer playing.
Immune Attack is unique among learning games because 1) it presents
advanced, abstract and fundamental concepts in cellular and molecular biology and2) does not explicitly teach, but rather treats the material as if it were actually part ofthe story of a typical third person shooter video game. Immune Attack is a video
game. It does not break its cover; there are no quizzes to pass, no mini games, noguarantee of formal learning. Learning happens within the context of the game:players only need to learn how to win the level in order to move on. Like a regular,commercial game, the player learns what they need to know and uses it immediately.
There is no explanation of what a protein is, instead we just get instructions toactivate these proteins and what results. We see a response by the cell to theactivation of proteins and we learn a concept. Therefore, Immune Attack presents
a unique opportunity to begin to uncover what kind of learning and what kind of
attitudes are imparted about biology through a commercial type video game.
Additionally, it should be noted that Immune Attack is designed for gamers. The
tutorial is not intuitive for anyone who has not played this kind of complex video
game already. Created in 2007–2008, Immune Attack is similar to what is referred toas a “hardcore” video game. It expects the player to devote a few minutes to listening,learning and practicing the controls. In the 6 years since Immune Attack was
released, advances in tutorial design and game design have resulted in new ways toinvolve the player in the story while giving them time to practice the controls. So thetutorial is more integrated in the game and therefore more engaging for the player.
Immune Attack, therefore, provides an excellent example of a game designed for agame playing audience. Any results we nd for non-gamers may very well be
improved upon by a more intuitive game design and more engaging tutorial.
Originally intended to teach immunology concepts to advanced high school
students and beginning college students, Immune Attack teaches much moregeneral content that immunology, rather it teaches the basics of molecularbehavior. For example, Immune Attack shows players that individual proteins
perform their own tasks and that proteins are not interchangeable. Additionally,
Immune Attack shows that cells require a certain set of proteins in order tofunction and that missing a protein's function can cause disease. Additionally,
6000 grade school and high school teachers had registered to evaluate the game in
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Faraday Discussions

their classrooms. Clearly, there was demand for the game for use with younger
students. Therefore, we decided to evaluate the eﬀectiveness of Immune Attack asa teaching tool for basic cellular and molecular biology with students in 7th–12th
grades. (We present here our data from 10–12th grade students. We will present
our data on 7th–9th grade students elsewhere.)
We address the following questions:
1. Can a video game (Immune Attack) teach concepts and terms of molecularbiology, cellular biology and immunology?
2. Can a video game benet students who do not play video games or who
perform poorly in the test game?
3. Can a video game impart condence, or in other words, a feeling of selfeﬃcacy as regards the diagrams and graphics used to represent molecular andcellular biology data?
We tested Immune Attack players using their randomly assigned classmates asa control group. We developed an assessment of terms and concepts as well as aninnovative test of condence with molecular science. We found that Immune
Attack players perform signicantly better than control students on our test ofterms and concepts. Additionally, players who reported playing 1 or more hours ofvideo games per week, that is two-thirds of the students we tested, gained asignicant amount of condence with diagrams related to molecular science. On
the other hand, students who were not gamers, who performed poorly in the gameand who claimed the game was hard to play, showed no signicant diﬀerence incondence from the control students. However, while each of these groups (nongamers, poor Immune Attack players, and those who reported that Immune
Attack is hard to play) still had an average score that was signicantly greater thanthe control on the terms and concepts test. That is, all students responded well to
our test of terms and concepts, while most gained condence with the material.
And most noticeably, this complex game did not cause any players to losecondence in their molecular biology abilities. Therefore, a complex game like
Immune Attack may well be a useful medium for molecular science education.

Experimental methods
We built an assessment tool to measure 1) knowledge gains in the area of cellularand molecular immunology, 2) understanding of the game mechanism, 3) change
in self condence regarding the subject matter and 4) voluntary self reporting ofdemographic information. A complete list of these three sets of questions isavailable as ESI.† The assessment tool is taken – all three parts – by the students
online. The online test takes about 30–40 min to complete. We created our own
tests because it was completely unknown what we might discover. We tested thegame in junior high and senior high schools across the US. We tested the game inclassrooms, not because the game is designed for use in a classroom, but because
we needed to know that the knowledge level of our test players and control players
was as similar as possible.
Terms and concepts
We developed a 27 item multiple choice test of terms and concepts to test forgains in knowledge of molecular and cellular biology. We started with basic
This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Paper

questions and received such encouraging results we spent the two years creating
harder questions. We tested hundreds of students in grades 7–12 in American
schools. Our assessment questions were tested numerous times over the course of
two years and, in collaboration with statisticians Caroline Pinkham and David
Silvernail at the University of Southern Maine, we had a multiple choice test with
27 items. The statistical report of details about the test of terms and concepts is
included in the ESI.† The statistical report on the nal iterations of this test areavailable as ESI.† The nal Cronbach alpha score for these 27 knowledge questions is 0.79.
Choosing the concepts to focus on
During the test creation phase, 2009–2010, we asked students to answer openended questions, similarly to the method of Mike Klymkowsky.5 The method of
open-ended question asking is designed to bring out students’ misconceptions.
We used the students’ misconceptions to focus our assessment questions, since
there are so many possible topics to focus on in the game. For example, when we
asked students “what is protein?” they oen answered that “protein is found inchicken and egg whites” and that protein is an “important building block of the
body.” In other words, it appeared that students knew about proteins generally,
but did not realize that each protein is unique and has a particular job to do. The
game Immune Attack requires players to locate and shoot particular proteins,“Selectin” and “ICAM” in the very rst level of the game. So we used this to design
our test questions. There are basic knowledge questions to determine whether
players remember the names of two proteins required to complete level 1.
Knowing that proteins have individual names and activities is the rst step tounderstanding the bigger concept that mutations in a single gene can lead todisease. Questions in this group we refer to as one protein/one job. Other
concepts are the process of transmigration, the process of cytokinesis and themeaning of cell diﬀerentiation. The survey questions are submitted as ESI.†
Game mechanism
An understanding of the game mechanism is expected to be a prerequisite tolearning anything from the game, and lack of game mechanism understanding
may explain a lack of learning or condence gains. Therefore, we also created a
10-item multiple choice test that is designed to test whether players know what
the goals of the rst three levels are. These questions simply ask the players whatis the name of this object and what is in this image? We included questions about
objects required for game success and objects that were not required for success.
“What color are the monocytes?” Additionally, we showed images from the gamesuch as the one in Fig. 1C and asked “What is this arrow pointing to?”Condence with cell biology/biochemical diagrams
Our test of self condence is a series of images, diagrams and photos that are
typical of the images scientists use to communicate their research results as wellas college level immunology textbooks. Rather than ask students the typicalquestions about whether they see themselves as scientists, we wanted to ask“Does this look like something you would read?” We reasoned that students do
not know what the words “molecular biology” mean, so they can't say whether
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Paper

Faraday Discussions

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

they like it or would want to practice it or learn about it. Additionally, the game
Immune Attack does not introduce a realistic science scenario and is not intended
to present the career or activities of a scientist, rather all of the game takes place at
the cellular and nanoscale levels. Immune Attack is, however, designed to present

Fig. 1 Immune Attack screen shots. (A) Level one is about the process of transmigration.
(B) The Monocyte “the big blue ball” will roll over the selectin proteins and slow down if theplayer shoots the selectin proteins. Selectin is shown in panel (C) with the white arrowdrawn in to indicate the yellow Selectin protein.
This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Paper

cell and proteins in a manner typical of scientic diagrams. Cells are drawn to
mimic schematic drawings used in journal articles and textbooks. Therefore, wereasoned that players would become more familiar with such diagrams andperhaps gain a feeling of self eﬃcacy when presented with such diagrams in the
future. So we designed questions that show the player cellular immunologyrelated diagrams and see if playing a video game about the topic would aﬀect their
feelings of self condence with the material. The ve condence questions ask
students to agree or disagree with the statement “I would be able to understandthis if I read it and thought about it.” See Fig. 5 for the images used. See Results
and discussion for more detail.
Demographic information
As with every portion of this test, reporting of demographic information was
voluntary. Students self-reported their grades in science, math and English, hours
per week spent playing video games and which kinds. Students who played
Immune Attack were also asked whether they enjoyed the game, found it “easy toplay”, would like to play such games in their science classes and whether theywould recommend Immune Attack to their friend.

ProtocolTeachers reach us though our website, ImmuneAttack.org. We sent e-mails to6000 people who registered as interested teachers prior to the game's release in
May 2008. We received responses from about 100 teachers. Upon informing those
100 teachers of our three day controlled protocol, ten became our rst in-classevaluators. Over the next 3 years, 2009 through 2012, several teachers participatedeach year.
Controlling for students’ past experiences
Teachers were instructed to divide each of their classes randomly into two halves.
Students were to play either Immune Attack or a control game for 40 min. The
next week, the same halves played the games again. Twenty-four hours aer thesecond play period, all students took our online test of cellular immunology,
game mechanisms and self condence (Fig. 2). Finally, teachers lled out another
online survey in which they described in their own words how they divided theirclasses, how long the students played, and whether any deviations from theprotocol or technical diﬃculties, etc. occurred. We were able to exclude data fromteachers who did not describe a random method of dividing their class or who
had technical diﬃculties playing the games. Teachers also distributed andcollected consent forms. Our consent form explained that the games would be
used in class and that all students can participate, but no data would be usedfrom students who did not hand in their signed parental consent/student assent
form.
The control games were related generally, but did not address any cellular or
molecular biology. The games used were CSI: The Experience15 (10–12th grades)or N-Squad16 (7–9th grades). By dividing each classroom in half, we controlled forthe background and educational experiences of the students as much as possible.
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Faraday Discussions

Fig. 2 Schedule for classroom evaluations. Our evaluation protocol consisted of two 40minute play periods and an online post test. Teachers reported to us through our online
survey, ﬁlling in their own words how they accomplished the random assignment of theirstudents to the two groups, how long the students actually spent with the games each day
and if there were any technical problems.

Students self-reported their grades in math, science and English, how many hours
per week they play video games and which kinds.
To ensure that the results were due to the game only, teachers were asked not
to introduce the game or its topics. We also did not include any lecture or notes
for teachers to read. We did, however, provide teachers with a written step by step
description of what players do in the rst three levels. This walkthrough is usefulfor teachers who wish to follow up on what students learn in the game. The
walkthrough, FAQ and curriculum guides are available at ImmuneAttack.org.

Results and discussion
A statistical report characterizing the data collected is available as ESI.† Additionally, a complete list of all the questions and their answers are available in the
ESI.†
Immune Attack players were statistically signicantly more likely to scorehigher on our test of terms and concepts than their classmates who played control
games. Fig. 3A shows that the distribution of scores on our test of terms andconcepts is shied to the right. Fig. 3B. shows that boys and girls beneted fromplaying Immune Attack to the same degree. Girls scored on average 1 point higher
than boys, but this diﬀerence is not statistically signicant. So gender and priorvideo game experience did not have an eﬀect on student scores on our test ofcellular and molecular biology terms and concepts.
We were interested in knowing whether a third-person shooter video game that
requires two hands to control with a keyboard and mouse would perhaps
intimidate or confuse non-gamers. We wanted to see if non-gamers would
perform as well as gamers on our test of terms and concepts. Fig. 3C. shows thatstudents who report playing zero hours of video games per week scored as well as
students who report playing 1–5 or 6+ hours of video games. Consistent with thisgure, the Pearson correlation and one-way ANOVA calculations in the statisticalreport also indicate no statistically signicant correlation among the high scoringgroup and the hours of games per week students reported playing (see ESI†).
This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Paper

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Fig. 3Student scores on our test of terms and concepts presented in Immune Attack.

There is no correlation with gender or with the number of hours students
report playing video games each week. However, we still want to determine
whether a video game would work for all students. So we looked at other criteria
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Faraday Discussions

we thought might aﬀect students' scores on the terms and concepts test. These
criteria are how the players reacted to the game itself.
We assumed that students who did not beat the rst level of Immune Attack
would be the group least likely to perform well on our test. We designed the test ofterms and concepts to draw from the rst three levels of Immune Attack only.
Fig. 4A. shows the scores of Immune Attack players according to the level of thegame they reported reaching. Our results show that all Immune Attack players,
regardless of how many levels they reached, scored signicantly better than thecontrol game players.
However, Immune Attack players who reached the second half of the game
scored even better on the terms and concepts than the players who only reachedthe rst half of the game. Because the test of terms and concepts only covered therst three levels of Immune Attack, this diﬀerence is not due to the fact that level
5–7 players were exposed to more of the tested material. However, students whoplayed through the second half of the game are exposed to more of the same
concepts with new cells and new protein types (Fig. 4A).
Because learning from a video game is expected to require rst an understanding of the mechanism of the game, we created a 10 question test. These
questions about the game were straightforward: for example, students wereshown Fig. 1C and asked “What is this arrow pointing to?” Control students were
not asked these questions. Scores on the 27 terms and concepts test and the 10
game mechanisms test were very well correlated. There is a strong correlation
between students' scores on the ten game mechanism questions and their scoreson the test of concepts and terms (data not shown).
Additionally, responses to post-play questions, such as “do you agree that
Immune Attack is easy to play?” were correlated with higher scores on the terms
and concepts questions. So understanding the game mechanism and understanding how to play is important for remembering terms and concepts aer thegame.
Next, we address whether the game imparts a feeling of condence with thematerial presented. We reasoned that seeing the images of cells and proteins in
the game would make players feel more familiar with these types of images. The
cells are drawn similar to typical schematics scientists use to communicate theirdata, so we were able to use images used by scientists to convey their data with thegame players in our study.
Fig. 5 shows the ve images used to test for student's self condence withimages related to cellular and molecular biology. Students were asked to agree ordisagree with statement “I would be able to understand this is I read it andthought about it.” The following answer choices were available for each question:
1. I disagree denitely.
2. I disagree somewhat.
3. I am neutral.
4. I agree somewhat.
5. I agree denitely.
We found that Immune Attack players do gain condence as regards thetransmigration and data images. In the transmigration image (5A), there is a highdegree of complexity.10 This image, from Nature Immunology Reviews, is designed
for an expert audience. The second image, “Yellow Macrophages” (5B) from
Janeway's Immunobiology textbook, is designed for undergraduate and graduate
This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Paper

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Fig. 4 Scores on test of terms and concepts by (A) level of Immune Attack reached and by(B) response to the question “Is Immune Attack easy to play?” Students were given images
to select from in order to indicate which level of Immune Attack they reached—without
needing to recall the name of the level. (See online survey, in the ESI†).

Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Faraday Discussions

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Fig. 5 Conﬁdence questions. Five images were presented to Immune Attack and control
game players. Under each picture was the statement “I would be able to understand this is I
read and thought about it.” The ﬁve answer choices were 1) I agree completely, 2) I agree
somewhat, 3) I am neutral, 4) I disagree somewhat and 5) I disagree completely.

This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Faraday Discussions

Paper

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

11level immunology and has less detail. When we consider the simpler image, we
see in Fig. 6B and 7C–D that both Immune Attack players and their control
classmates responded to the simpler image in the same way. The students'
answers form two similar curves. When we consider the more complex transmigration gure (Fig. 7B and 7A) however, we nd a clear diﬀerence between the
Immune Attack players and the control students. The control students respond
more negatively to the more complex gure. About 25% of the control students
responded “I disagree denitely,” indicating that they do not think they would be
able to understand this image. However, Immune Attack players responded to thecomplex gure almost exactly as they did to the simpler textbook image.
We wanted to know whether condence with related images would carry overto other image types. Images of neurons and of a three dimensional representation of a protein complex did not elicit a more positive response from Immune

Fig. 6 Conﬁdence data for (A) the complex transmigration diagram, (B) the simple textbook yellow macrophages diagram and (C) the data image.
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Faraday Discussions

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Fig. 7 Conﬁdence data for (A and B) the complex transmigration diagram, (C and D) the
simple text book yellow macrophages diagram and (E and F) the data images broken down
into girls’ data and boys’ data.

This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Paper

Attack players compared with the control. It may be that the positive eﬀect on
their responses is very specic to the type of image and the drawing style. Immune
Attack cells look very similar to the diagram in Fig. 1, in which H&E stainedfeatures of the white blood cells are drawn, namely the nucleus shape and color.
However, it may also be that the complex transmigration image was the only
image complex enough to elicit a negative response from the control students.
We wanted to see if Immune Attack players could extrapolate their new
understanding and condence to actual data images. Fig. 5E is from William
Muller's groundbreaking work, which showed that transmigration—the objectiveof level 1 of Immune Attack—is dependent on selectin—the protein players needto shoot to successfully transmigrate their monocyte.13 The images in the panels
will naturally look foreign, since the scanning electron microscope images are 3D,
grey and do not show any of the schematic details shown in the game, like nuclei.
However, we wrote a gure legend in simple language that uses terms from thegame and also clearly describes the experiment being presented. Immune Attack
players consistently respond more positively than control game players about
their perceived ability to understand the data images (Fig. 7E–F). For the datagure, the eﬀect size is small (0.19), because the average score does not shigreatly, while for the transmigration gure, the eﬀect size is medium (0.47).
So a diﬀerence between Immune Attack players and controls is found in theirreaction to the transmigration diagram and the data with a gure legend.
Immune Attack players respond more positively on the whole to both images. To
investigate which groups of students gained condence, we looked at boys vs.
girls in Fig. 7A–F.and then at gamers vs. non-gamers in Fig. 8A–E.
Considering rst the eﬀect of gender for the transmigration image, both female
and male Immune Attack players responded statistically signicantly more positively to the transmigration image compared to matched controls, although thediﬀerence has a smaller eﬀect size for girls than boys. For the data with gurelegend, the increase in condence is not signicant unless all 180 Immune Attack
students are compared to the control. This is primarily because the averages
between the test and control means are so small. However, in Fig. 6C and Fig. 7E
we can see that the curve of student responses is shied to the more positive aer
exposure to Immune Attack. This shi eﬀect is stronger for girls than for boys.
Considering the eﬀect of hours of game play reported per week, we nd in
Fig. 8A that non-gamers (0 h per week reported) do not diﬀer from non-gamer
control students. Students who report playing 1 or more hours per week, which is
two thirds of the students tested, show a statistically signicant shi in their
responses to the transmigration image. For the data image, results are only
signicant when all hours of games per week are considered together.
It is interesting to note, gamers (students who report playing 6 or more hours
per week) in the control group were consistently more negative about their ability
to understand these images. Perhaps video gamer players can more accurately
assess their ability to understand a new diagram (Fig. 8A–E).
Finally, we looked at whether students' scores on the terms and concepts test
are correlated with their condence scores (Fig. 9). We found that the students
who scored 9–16 and 17–27 questions correctly also responded more favorably tothe transmigration diagram and the data images. The condence measurement
for students who scored 8 or fewer correct answers on the terms and concepts test
could not be distinguished form the control response.
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Faraday Discussions

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Fig. 8 (A–E) Conﬁdence data for each of the ﬁve images, with data broken down into allstudents, students who report playing 0 h of video games per week, those who report
playing 1–5 h, and those who report playing 6 or more hours per week.

This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Paper

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Conﬁdence data for each of the ﬁve images, with data broken down by how wellstudents performed on the test of terms and concepts.

Fig. 9Conclusions
We have found that students learn molecular and cellular biology concepts andterms by playing a video game set in the molecular world. Even though the gamedoes not explicitly teach these terms and concepts, students can answer questions
on our 27 item multiple choice test signicantly better than their classmates whoplayed an unrelated video game. The higher average scores on the test of terms
and concepts was true for all Immune Attack players, regardless of how many
hours a week they play games and whether they are male or female. We found nofactor that can predict prior to playing the game whether students will perform
well on the test of terms and concepts. However, we found that students whoscored in the bottom 24% on the test of terms and concepts were more likely toreport that the Immune Attack is not easy to play. Additionally, understanding the
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Paper

Faraday Discussions

game mechanism questions was the strongest indicator of whether students
scored well on the test of terms and concepts. We conclude that a video game like
Immune Attack – a shooter-styled, self-directed game with minimal explicit
learning activities – can improve student scores on a test of terms and concepts.
Importantly, a game with a more engaging introduction to the controls and gamemechanism than Immune Attack may be even more eﬀective at helping students
remember terms and concepts.
Immune Attack also had a positive eﬀect on the condence players feel in theirabilities to understand complex diagrams related to the game. However, unlike
the test of terms and concepts, we found a diﬀerence between gamers and nongamers. In particular, we found that students who reported playing zero hours ofvideo games per week showed no diﬀerence in condence with a molecularbiology diagram as compared to the control students. So there were no condence
gains in the non-gamer group. However, Immune Attack is a complex thirdperson shooter, in which players need to learn to navigate a microbot in threedimensional space, avoid many dangerous objects and shoot a ray gun at moving
proteins. Yet, despite the complexity of the game, non-gamers do not show any
loss of condence compared to the control group regarding molecular biology.
We did not nd a diﬀerence in condence between the gender groups, however.
Boys as well as girls showed a statistically signicant shi in their response to acomplex scientic diagram.
Additionally, the majority of students who reported enjoying Immune Attack
and who played past the third level of the game showed the greatest increase incondence, regardless of whether they were gamers. Most interestingly, whenpresented with two molecular cell biology diagrams, one designed for experts andone designed for students, Immune Attack players responded to the expert
diagram as favorably as if it were the simpler one.
We have shown that a large fraction of Immune Attack players gained condence with the kinds of diagrams scientists use to communicate their data, while
not causing a loss of condence in non-gamers or students who did not perform
well in the game. Immune Attack was successful at increasing the average score of
all students, gamers and non-gamers alike, on our test of terms and concepts.
Therefore, we conclude that a complex video game like Immune Attack that 1)
requires a tutorial to teach the game mechanism similar to a hardcore video gameand 2) teaches through story-based interactions with a wide array of objects but
without explict text-based questions to answer can increase the scores of players
on a test of terms and concepts, increase the condence of gamers with thematerial and not harm the condence levels of non gamer students. We suggestthat game design improvements can positively aﬀect the condence levels of the
non-gamer students.

Acknowledgements
Many teachers participated in our controlled evaluation, provided feedback on
our assessment items, and on the game implementation and evaluation in theclassroom. Caroline Pinkham at the University of Southern Maine provided expert
statistical analysis. The Maine International Center for Digital Learning provided
outreach and communication with participating teachers. Howard Young, Ph.D.
at the National Institutes of Health has been a supporter of the Immune Attack
This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.

View Article Online

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

Faraday Discussions

Paper

project in many ways, including the assessment development. Immune Attack
was created by the Federation of American Scientists Learning Technologies
Program with funding from the National Science Foundation (USA) in collaboration with Brown University and the University of Southern California. The nal
game was developed by Escape Hatch Entertainment. Immune Attack was
released May 2008. The Windows-only game is available for free download at
ImmuneAttack.org, where there is a trailer, which previews the game action. The
evaluation work presented here was funded by the National Institutes of Health,National Institute for Allergy and Infectious Diseases (NIAID) and the Entertainment Association Foundation (EASF).

References
1 K. G. Bowling, Y. Klisch, S. Wang and M. Beier, Examining an online
microbiology game as an eﬀective tool for teaching the scientic process,
Journal of Microbiol. Biol. Educ., 2013, 14, 58.
2 J. Jenkinson and G. McGill, Visualizing protein interactions and dynamics:
evolving a visual language for molecular animation, CBE Life Sci. Educ.,
2012, 11, 103.
3 R. Joiner, J. Iacovides, M. Owen, C. Gavin, S. Clibbery, J. Darling and B. Drew,
Digital games, gender and learning in engineering: do females benet as muchas males?, J. Sci. Educ. Technol., 2011, 20, 178.
4 Y. Klisch, L. M. Miller, M. E. Beier and S. Wang, Teaching the biologicalconsequences of alcohol abuse through an online game: impacts among
secondary students, CBE Life Sci. Educ., 2012, 11, 94.
5 M. W. Klymkowsky and K. Garvin-Doxas, Recognizing Student Misconceptions
through Ed's Tools and the Biology Concept Inventory, PLoS Biol., 2008, 6, 1.
6 L. M. Miller, J. Moreno, V. Estrera and D. Lane, Eﬃcacy of MedMyst: an
Internet Teaching Tool for Middle School Microbiology, Microbiol Educ.,
2004, 5, 13.
7 NGSS Lead States, HS. Structure and Function, HS. Matter and Energy in
Organisms and Ecosystems, Achieve, Inc., 2013.
8 K. D. Squire, From content to context: Video games as designed experiences,
Educational Researcher, 2006, 35, 19.
9 K Squire, Dissertation Research, University of Wisconsin, Madison, WI, USA,2004, http://website.education.wisc.edu/~kdsquire/research.html, accessed2014.
10 K. Ley, C. Laudanna, M. I. Cybulsky and S. Nourshargh, Nat. Rev. Immunol.,
2007, 7, 678.
11 C. A. Janeway, P. Travers, M. Walport and M. Shlomshik, Immunobiology: The
Immune System in Health and Disease. 5th edition, Garland Science, New York,USA, 5th edn, 2001.
12 C. Brunner, A development process focused on pedagogy-centered design. Possible
Worlds Technical Report and Working Paper Series, Center for Children &Technology/Education Development Center, Inc., New York, USA, 2004,
http://possibleworlds.edc.org, accessed 2014.
13 W. A. Muller, S. A. Weigl, X. Deng and D. M. Phillips, J. Exp. Med., 1993, 178,
449.
Faraday Discuss.

This journal is © The Royal Society of Chemistry 2014

View Article Online

Paper

Faraday Discussions

Published on 27 June 2014. Purchased by melanie.stegman@gmail.com on 09 July 2014.

14 Sid Meier's Civilization III by Firaxis, 2001, http://www.civ3.com, accessed2014.
15 CSI The Experience: Web Adventures, 2012, http://www.civ3.com, accessed2014.
16 N-Squad Web Adventures, Rice University, Center for Technology in Teachingand Learning, 2011, http://n-squad.rice.edu, accessed 2014.

This journal is © The Royal Society of Chemistry 2014

Faraday Discuss.