"Abstract—This paper delineates the intricate process of
crafting an Augmented Reality (AR)-enriched version of the
Subway Surfers game, engineered with an emphasis on action
recognition and the leverage of Artificial Intelligence (AI)
principles, with the primary objective of boosting children's
enthusiasm towards physical activity. The gameplay,
fundamentally predicated on advanced computer vision
methodologies for discerning player kinesthetics, and reinforced
with machine learning tactics for modulating the intricacy of the
game in accordance with player capabilities, offers an immersive
and engaging interface. This innovative amalgamation serves to
not only catalyze children's interest in participating in active
exercises, but also introduces a playful aspect to it. The
procedural development of the game required the cohesive
assimilation of a diverse spectrum of technologies, encompassing
Unity for game development, TensorFlow for implementing
machine learning algorithms, and Vuforia for crafting the AR
elements. A preliminary study, conducted to assess the efficacy of
the game in fostering a pro-sport attitude in children, reported
encouraging outcomes. Given the potential of the game to incite
physical activity among young users, it could be construed as a
promising antidote to sedentarism and a potent catalyst for
endorsing a healthier lifestyle.
Keywords—Augmented reality; computer vision; game
development; action detection; action classification; machine
learning
I. INTRODUCTION
Physical inactivity has become a major health concern,
particularly among children, with global estimates suggesting
that more than 80% of adolescents worldwide are not meeting
the recommended levels of physical activity [1]. The rise of
sedentary lifestyles and the prevalence of screen-based
activities have been identified as major contributors to this
trend, with video games being a prominent example. However,
video games have the potential to be a tool for promoting
physical activity if they are designed with this goal in mind [2].
One such game is Subway Surfers, a popular endless runner
game that has been downloaded over two billion times [3]. The
game's objective is to run through a subway system, collecting
coins and avoiding obstacles. While the game has been praised
for its addictive gameplay and engaging visuals, it does not
require physical activity and could contribute to sedentary
behavior.
To address this issue, this paper presents the development
of a Subway Surfers game with augmented reality (AR) based
on action recognition and artificial intelligence (AI) to increase
children's motivation for sports. The game utilizes computer
vision algorithms to recognize the player's physical movements
and uses machine learning techniques to adjust the game's
difficulty level accordingly. The AR feature provides a more
immersive and engaging gameplay experience, encouraging
children to participate in physical activity while having fun.
The use of AR in video games has gained significant
attention in recent years due to its potential to create immersive
and interactive experiences. AR involves overlaying digital
content onto the real world, allowing users to interact with
virtual objects in real-time [4]. This technology has been used
in various applications, including education, entertainment, and
advertising [5]. In the context of video games, AR has been
used to create physically active gameplay experiences, with
some studies suggesting that AR games can increase physical
activity levels [6].
In addition to AR, the game incorporates action recognition
and AI to create a personalized gameplay experience. Action
recognition involves using computer vision algorithms to
detect and classify human actions, such as running, jumping,
and squatting [7]. By recognizing the player's movements, the
game can adjust the gameplay difficulty level to provide a
challenge that matches the player's physical abilities. This
personalized approach is important as it can help to maintain
the player's engagement and motivation, which are critical
factors in promoting physical activity [8].
The use of AI in the game also allows for real-time
adjustments to the gameplay. Machine learning techniques,
such as reinforcement learning, can be used to train the game's
AI to optimize the gameplay experience for the player [9]. By
learning from the player's actions and adjusting the gameplay
accordingly, the game can create a more challenging and
engaging experience. This approach also allows the game to
adapt to the player's progress, ensuring that the gameplay
remains challenging and motivating.
(IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 14, No. 8, 2023
683 | P a g e
www.ijacsa.thesai.org
The development of the game involved the integration of
various technologies, including Unity, TensorFlow, and
Vuforia. Unity is a game engine that provides a development
environment for creating 2D and 3D games. TensorFlow is a
popular machine learning library that was used to implement
the game's action recognition and AI algorithms. Vuforia is an
AR platform that was used to create the game's AR features
[10].
To evaluate the effectiveness of the game in increasing
children's motivation for sports, a pilot study was conducted.
The study involved 20 children aged between 7 and 12 years
old, who played the game for 30 minutes. The study found that
the game was effective in increasing the children's motivation
for physical activity, as measured by self-reported enjoyment
and willingness to engage in physical activity. The study also
found that the personalized gameplay experience created by the
game's action recognition and AI features was a key factor in
maintaining the children's engagement and motivation [11].
The potential of the Subway Surfers game with AR and AI
to promote physical activity among children suggests that it
can be a valuable tool in combating sedentary lifestyles and
promoting a healthier lifestyle. By creating an immersive and
engaging gameplay experience that encourages physical
activity, the game can provide an alternative to traditional
forms of exercise that may be less appealing to children.
Furthermore, the game's personalized approach can help to
maintain the player's engagement and motivation, which are
critical factors in promoting long-term behavior change [12].
The development of this game also highlights the potential
of technology in promoting physical activity. With the
widespread availability of smartphones and tablets, mobile
games have the potential to reach a large audience and provide
a convenient and accessible way to promote physical activity.
The integration of AR and AI technologies can further enhance
the effectiveness of these games by creating personalized and
engaging experiences.
In conclusion, this paper presented the development of a
Subway Surfers game with augmented reality based on action
recognition and artificial intelligence to increase children's
motivation for sports. The game utilizes AR to create an
immersive and engaging gameplay experience and uses action
recognition and AI to provide a personalized gameplay
experience that matches the player's physical abilities. The
pilot study conducted to evaluate the game's effectiveness in
increasing children's motivation for sports yielded promising
results, highlighting the potential of this game as a tool for
promoting physical activity among children. The development
of this game also demonstrates the potential of technology in
promoting physical activity and provides a roadmap for the
development of future mobile games with a focus on
promoting physical activity.
II. RELATED WORKS
The use of technology to promote physical activity among
children has been a topic of interest in recent years. Mobile
games, in particular, have been identified as a promising tool
for promoting physical activity, as they have the potential to
reach a large audience and provide an engaging and interactive
experience.
Several studies have explored the effectiveness of mobile
games in promoting physical activity among children [13-15].
Next study developed a mobile game that used augmented
reality to create an immersive and engaging gameplay
experience [16]. The game encouraged children to engage in
physical activity by requiring them to perform various
movements and exercises in order to progress through the
game. The study found that the game was effective in
increasing children's motivation for physical activity, with
participants reporting higher levels of enjoyment and
engagement compared to traditional forms of exercise.
Similarly, another study developed a mobile game that used
a pedometer to track children's physical activity levels [17].
The game provided feedback to the children based on their
physical activity levels, with rewards and incentives provided
for achieving certain goals. The study found that the game was
effective in increasing children's physical activity levels, with
participants reporting higher levels of engagement and
motivation compared to traditional forms of exercise.
The use of artificial intelligence in mobile games has also
been explored as a means of providing personalized and
adaptive gameplay experiences. A study by Paliokas et al.
(2020) developed a mobile game that used artificial
intelligence to adapt the gameplay experience based on the
player's physical abilities [18]. The game used motion sensors
to track the player's movements and adjust the difficulty of the
game accordingly. The study found that the personalized
gameplay experience created by the AI technology was
effective in maintaining the player's engagement and
motivation.
The use of augmented reality in mobile games has also
been identified as a promising tool for promoting physical
activity. A study by Ding & Wang et al. (2019) developed a
mobile game that used augmented reality to create an
immersive and engaging gameplay experience [19]. The game
required players to physically move around and interact with
virtual objects in order to progress through the game. The study
found that the game was effective in increasing children's
motivation for physical activity, with participants reporting
higher levels of enjoyment and engagement compared to
traditional forms of exercise.
The Subway Surfers game, which was originally developed
in 2021 by Wang and SYBO Games, has become one of the
most popular mobile games worldwide, with over two billion
downloads [20]. The game's popularity can be attributed to its
engaging gameplay, which requires players to run, jump, and
dodge obstacles in a subway environment. The game's simple
controls and colorful graphics have also contributed to its
appeal among children.
Several studies have explored the effectiveness of the
Subway Surfers game in promoting physical activity among
children. A study by Pascoal et al. (2020) found that the game
was effective in increasing children's physical activity levels,
with participants reporting higher levels of enjoyment and
engagement compared to traditional forms of exercise [21].
(IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 14, No. 8, 2023
684 | P a g e
www.ijacsa.thesai.org
Another study by Blattgerste et al. (2019) found that the game
was effective in increasing children's motivation for physical
activity, with participants reporting higher levels of enjoyment
and satisfaction compared to traditional forms of exercise [22].
However, despite the game's popularity and potential for
promoting physical activity, it has not yet been fully explored
in terms of its potential for utilizing AR and AI technologies.
This paper presents the development of a Subway Surfers game
with augmented reality based on action recognition and
artificial intelligence, which aims to further enhance the game's
potential for promoting physical activity among children [23].
The incorporation of AR technology into the Subway
Surfers game has the potential to create a more immersive and
engaging gameplay experience by allowing the player to
interact with virtual objects in the real world. This technology
can be used to encourage physical activity by requiring the
player to physically move around and interact with virtual
objects in order to progress through the game.
In addition, the use of AI technology can be used to create a
personalized and adaptive gameplay experience based on the
player's physical abilities. This technology can be used to track
the player's movements and adjust the difficulty of the game
accordingly, ensuring that the player remains engaged and
motivated throughout the gameplay experience.
Several studies have explored the use of AR and AI
technologies in promoting physical activity among children. A
study by Lam et al. (2020) developed an AR-based game that
used AI technology to adjust the difficulty of the game based
on the player's physical abilities [24]. The study found that the
personalized gameplay experience created by the AI
technology was effective in increasing children's motivation for
physical activity.
Another study by Chen (2020) developed an AR-based
game that required children to physically move around and
interact with virtual objects in order to progress through the
game [25]. The study found that the game was effective in
increasing children's physical activity levels, with participants
reporting higher levels of enjoyment and engagement
compared to traditional forms of exercise.
The development of the Subway Surfers game with AR and
AI technologies builds on these previous studies by creating a
gameplay experience that is both engaging and physically
active. By combining the popular and familiar gameplay of
Subway Surfers with the immersive and interactive experience
provided by AR and AI technologies, this game has the
potential to significantly increase children's motivation for
physical activity.
Furthermore, the use of action recognition technology in
the development of the game adds another level of interactivity
and engagement. This technology can be used to track the
player's movements and actions, allowing the game to provide
immediate feedback and rewards based on the player's
performance. This feedback can be used to encourage the
player to continue engaging in physical activity and improving
their performance.
In conclusion, the development of a Subway Surfers game
with AR based on action recognition and AI technology has the
potential to significantly increase children's motivation for
physical activity. The combination of the popular and familiar
gameplay of Subway Surfers with the immersive and
interactive experience provided by AR and AI technologies
creates a gameplay experience that is both engaging and
physically active. The use of action recognition technology
also adds another level of interactivity and engagement,
providing immediate feedback and rewards based on the
player's performance. Future studies can explore the
effectiveness of this game in promoting physical activity
among children and its potential for use in educational settings.
III. MATERIALS AND METHODS
Fig. 1 depicts the flowchart of the system's underlying
infrastructure. The first thing that must be done is the offline
camera calibration, which must be done only once. Building a
model of the pattern that is going to be monitored is the
following phase, which likewise takes place offline. This
method is sometimes referred to as model-based tracking or
tracking by detection. There is also something known as
recursive tracking, which is a kind of tracking that is dependent
on frame-to-frame tracking (for example, [26]). It never finds a
solution to the issue of errors piling up over time. Model-based
tracking, on the other hand, does not have this issue.
Descriptors of point characteristics make up the model in the
research that we have done. A picture to use as a reference for
the pattern is captured. After that, the ORB detector, which
stands for Oriented FAST and Rotated BRIEF, is used to this
reference picture in order to identify point features [27]. It is a
feature detector that is both quick and reliable. It makes use of
FAST in order to identify keypoints, and then it applies the
Harris corner detector in order to choose just the most robust
characteristics. Scale invariance is provided by the scale
pyramid, while rotation invariance is provided by the moments.
The FR AK (Fast Retina Keypoint) description is then used to
characterize the characteristics that have been discovered [28].
It does this by the rapid comparison of picture intensities
throughout a retinal sampling pattern, which results in the
computation of a cascade of binary strings. In our tests, a range
of different numbers of characteristics are investigated. The
descriptors of the retrieved features are used in the construction
and training of the classifier.
During the tracking phase, a camera image is taken, and
features are identified and characterized in the same way that
they are during the model construction stage. Using the
classifier, these characteristics are compared to the
corresponding features found in the reference picture. The
RANSAC (Random Sample Consensus) [29] method is
employed to maintain just the inliers and to reject all of the
data that is considered to be an outlier. An exact homography,
denoted by the letter H, is calculated using the excellent
matches that are still available. After that, the camera attitude
may be determined in the same way as was described before.
After determining the approximate position of the camera, the
enhanced scene is rendered. Fig. 2 demonstrates detection of
human and landmarks detection process.
(IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 14, No. 8, 2023
685 | P a g e
www.ijacsa.thesai.org
Fig. 1. Flowchart of the proposed augmented reality enabled game.
Fig. 2. Person and landmarks detection flowchart.
Fig. 3. Detection of landmarks of a human.
(IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 14, No. 8, 2023
686 | P a g e
www.ijacsa.thesai.org
Fig. 3 demonstrates the 33 landmarks of a human that the
model can detect. Further, we can use the detected keypoints in
movement detection process of the proposed system [30].
IV. RESULTS
In this part, we will provide the experiment results. Fig. 4
demonstrates recognition of different poses that are used in
gamification process. Human should stand in front of a camera.
Real-time video will be sent to the proposed system and the
received video will be processed. The proposed system
recognizes different actions that will be applied during playing
the subway surfer game.
In subway surfer game, the character can move left, right,
up or down by swiping on the screen. Swiping up allows the
character to jump, swiping down enables them to roll, and
swiping left or right changes their direction.
There are many obstacles in the game, including trains,
barriers, and walls. The player must avoid these obstacles by
jumping, rolling, or changing direction.
There are various power-ups that the player can collect,
including a magnet that attracts coins, a jetpack that allows the
player to fly, and a super sneaker that makes the character jump
higher. There are various power-ups that the player can collect,
including a magnet that attracts coins, a jetpack that allows the
player to fly, and a super sneaker that makes the character jump
higher.
The game is over when the character is caught by the
inspector or hits an obstacle. The player can then use their
coins to continue from where they left off or start over from the
beginning. The game has a variety of missions and daily
challenges that the player can complete for rewards.
Subway Surfers is a popular and entertaining mobile game
that offers endless fun and challenges to players. With its easyto-learn gameplay mechanics, addictive gameplay, and various
features, it's no surprise that it has remained popular among
gamers for many years.
a) Starting the program b) Recognition of hands c) Start the game by joining hands
d) Right shift recognition e) Stand in the center f) Left shift recognition
g) Recognition of "Crouching" h) Recognition of "Standing" i) Recognition of "Jumping"
Fig. 4. Recognition of different poses.
(IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 14, No. 8, 2023
687 | P a g e
www.ijacsa.thesai.org
Fig. 4 to 6 demonstrate the application of the proposed
system in practice. Right side of the images show a person who
plays the game, left side demonstrates game personage that is
playing the game.
Fig. 4(a) demonstrates starting the program. In the first
stage, the program recognize a person in front of the camera.
Fig. 4(b) and Fig. 4(c) demonstrate recognition of weather the
hands are joined or not. When the status hands are joined, the
game will be started. Fig. 4(d), Fig. 4(e), Fig. 4(f) recognized
movements of right shift, stand in the center, left shift,
respectively. Fig. 4(g), Fig. 4(h), and Fig. 4(i) demonstrate
recognition of three movements as Crouching, Standing, and
Jumping, respectively. All of these movements are used during
playing the game. Each movement are recognized and the
results will be sent to the game as commands like move to
right, move to left, jump, crouch, go, and run.
Fig. 5 illustrates example of crouching in playing the game.
When the program recognize a person in front of camera
crouch, the same command will be sent to the game, and game
personage crouch in order to go through an obstacle.
Fig. 6 demonstrates the case when a gamer moved to the
right and in the result the game personage moved to the right,
too. As a result, children can do different movements during
playing the game. As well as, when a gamer moves to the left,
the game personage does the movement and goes to the left
side of the road.
Fig. 7 demonstrates jumping of the gamer and at the same
time jumping of the game personage. When two points of the
shoulders go up the horizontal line, the proposed system
recognizes the jumping and send the jump command to the
game. In the result, children do different movements including
go, run, left shift, right shift, crouching and jumping during the
game playing process. Thus, we can integrate an electronic
game playing process with real sports. This approach can
motivate children to do sports while they are playing the game.
In some cases, it can be useful for adults, too.
Fig. 5. Example of crouching in playing the game.
Fig. 6. Demonstration of the case when a gamer moved to the right.
Fig. 7. Demonstration of case when gamer and game personage jumped
together.
V. DISCUSSION AND FUTURE RESEARCH
The development of the Subway Surfers game with
augmented reality (AR) based on action recognition and
artificial intelligence (AI) has the potential to increase
children's motivation for sports. However, the success of the
game depends on several future perspectives and challenges
that need to be considered.
One of the main future perspectives of this game is its
potential to revolutionize the way children interact with sports.
The integration of AR and AI technologies in the game enables
children to participate in physical activities in an interactive
and engaging way [31]. By combining physical exercise with
technology, children can have fun while improving their
physical fitness, which could ultimately lead to a more active
and healthy lifestyle. The success of this game could inspire
similar developments that use technology to motivate children
to participate in sports.
(IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 14, No. 8, 2023
688 | P a g e
www.ijacsa.thesai.org
Another future perspective of this game is its potential to
enhance the overall gaming experience. The use of AR
technology in the game provides a more immersive experience,
allowing players to interact with virtual objects in the real
world [32]. The game's AI capabilities also enable it to adapt to
the player's actions and provide personalized challenges,
making the game more engaging and challenging. This could
lead to the development of more sophisticated games that use
AI and AR technologies to create even more immersive and
engaging experiences.
However, there are several challenges that need to be
addressed in the development of this game. One of the main
challenges is the accuracy of the action recognition technology.
The game's success depends on the accurate detection of the
player's movements, which could be difficult to achieve,
especially in situations where there are multiple players or
environmental factors that could interfere with the recognition.
The development team needs to ensure that the action
recognition technology is accurate and reliable, as inaccurate
detection could negatively impact the player's experience.
Another challenge is the game's accessibility [33]. The
game's success depends on its ability to reach a wide audience,
including children from different backgrounds and abilities.
The development team needs to ensure that the game is
accessible to all children, regardless of their physical abilities
or technological literacy. This could require the development
of different game modes or features that cater to different
audiences.
Another challenge is the game's safety [34]. The game's
success could potentially lead to an increase in physical
activity, which is positive, but it could also lead to an increase
in accidents and injuries. The development team needs to
ensure that the game is designed in a way that minimizes the
risk of injury and provides clear guidelines for safe play.
Finally, the game's impact on children's motivation for
sports needs to be evaluated. While the integration of
technology in sports could be beneficial, it is important to
assess whether the game actually increases children's
motivation to participate in physical activities in the long term
[35-37]. The development team needs to conduct research to
evaluate the game's impact on children's physical activity
levels and assess whether the game actually motivates children
to engage in physical activities outside of the game.
In conclusion, the development of the Subway Surfers
game with augmented reality based on action recognition and
artificial intelligence has the potential to increase children's
motivation for sports. However, the success of the game
depends on addressing several challenges, including the
accuracy of the action recognition technology, the game's
accessibility, safety, and its impact on children's motivation for
sports. Overcoming these challenges could lead to the
development of more sophisticated games that use technology
to motivate children to participate in physical activities and
lead a more active and healthy lifestyle.
VI. CONCLUSION
In conclusion, the development of the Subway Surfers
game with augmented reality based on action recognition and
artificial intelligence has the potential to revolutionize the way
children interact with sports. By combining physical activity
with technology, the game can motivate children to participate
in sports and lead a more active and healthy lifestyle. The
integration of AR and AI technologies in the game provides an
immersive and engaging experience, allowing children to
interact with virtual objects in the real world.
However, the success of the game depends on addressing
several challenges, including the accuracy of the action
recognition technology, the game's accessibility, safety, and its
impact on children's motivation for sports. These challenges
need to be overcome to ensure that the game is accessible, safe,
and effective in increasing children's motivation for sports.
Future research is needed to evaluate the game's impact on
children's physical activity levels and assess whether the game
actually motivates children to engage in physical activities
outside of the game. The success of this game could inspire
similar developments that use technology to motivate children
to participate in sports and lead a more active and healthy
lifestyle.
Overall, the development of the Subway Surfers game with
augmented reality based on action recognition and artificial
intelligence is an exciting development that has the potential to
make a significant impact on children's health and well-being.
By combining physical activity with technology, the game
provides a fun and engaging way for children to improve their
physical fitness and lead a more active and healthy lifestyle."
Did the above paper explain why AI and machine learning are critical to AR gaming ?Refer to the sentences from the paper to answer this questions.