# Spheres and Lenses: Activity-Based Scenario / Narrative Approach for Design and Evaluation of Entertainment through Engagement

Tim Marsh, Bonnie Nardi

► To cite this version:

Tim Marsh, Bonnie Nardi. Spheres and Lenses: Activity-Based Scenario / Narrative Approach for Design and Evaluation of Entertainment through Engagement. 13th International Conference Entertainment Computing (ICEC), Oct 2014, Sydney, Australia. pp.42-51, 10.1007/978-3-662-45212-7_6 hal-01408503

HAL Id: hal-01408503

https://inria.hal.science/hal-01408503

Submitted on 5 Dec 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Spheres and Lenses: Activity-Based Scenario / Narrative Approach for Design and Evaluation of Entertainment Through Engagement

Tim Marsh<fnref n="1" /> and Bonnie Nardi<fnref n="2" />

**Abstract.** Building on A.N. Leontiev's original activity theory, we propose extensions to bridge conceptual gaps to operationalize an activity-based scenario / narrative approach leading to a universal framework to inform design and reason about the user experience of entertainment through engagement in task-based, as well as improvised, extemporaneous and serendipitous interaction and gameplay.

**Keywords:** activity theory, design, analysis, engagement, user experience, interaction, gameplay, improvisation, interactive storytelling, scenario, narrative

## 1 Introduction

Engagement infers positive characteristics, synonymous with quality of user experience in interaction and gameplay [1, 18, 19]. In this paper we focus on engagement as a means to reason about and assess quality of the user experience of entertainment, whether positive, fun and exciting, through stimulating and thought provoking, to difficult, scary, or darker experiences that are either pleasurable or a necessary part of a wider whole cultural experience [15]. While engagement typically implies flow-like interaction and gameplay with one application or game on a single platform, observation of real-world technology use also reveals additional and alternate patterns of engagement in interaction and play (figure 1). In the real-world people typically engage with a range of digital platforms appropriate to the situation, process or task at hand, shifting between applications on one or more platforms, and even pausing momentarily to reflect, while still appearing to remain engaged. This is especially so with the younger digital native generation who appear to seamlessly navigate and interweave between a variety of platforms, applications and services. While multi-platform/application use has not gone unnoticed, it is invariably described under the broad label of multitasking, for example engagement in browsing and online multitasking [10], and/or typically reflecting concurrent application/platform use and with other tasks (e.g. driving). In addition, observation in studies with games for enter-

<footnote n="1">¹Griffith Film School, Queensland College of Art, Griffith University, Australia
t.marsh@griffith.edu.au</footnote>

<footnote n="2">²Donald Bren School of Information & Computer Sciences, University of California, Irvine, US
nardi@ics.uci.edu</footnote>
tainment and games for learning in natural environments (e.g. classroom, video games clubs) have long identified behaviours that don't entirely connect with flow-like engagement between players and digital games but identify behaviours that extended beyond the game-world whereby players momentarily reflect and/or interact with other users-players out-of-game in the real-world. Rather than having a detrimental effect, extending engagement beyond the game world improved the overall experience for users-players [11, 16, 20].

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Interaction and gameplay: one application/game on one platform (left), through to more than one application on one (middle) or more platforms (right)</figcaption></figure>

In addition to multitasking and task driven interactions, we emphasize a more free-
spirited, unconstrained, non-linear, improvised and extemporaneous nature of interac-
tion and gameplay for entertainment or stimulation. Where users-players craft their
own narratives, in-game and between applications and platforms, by playing, select-
ing, searching, and creating, that is driven by and appropriate to, their own tastes,
interests, preferences, desires and individual/group cultures and sub-cultures. In this
respect, users-players have been likened to editors, curators, authors and composers
[3, 13]. A perspective that is similar in many respects to the “cut-up” technique (at-
tributed to Dadaists) practiced by artist writer Brion Gysin and William S. Burroughs
[2] and various musicians (David Bowie, Kurt Cobain) whereby narratives, storylines,
lyrics and points of view are created, cut-up into pieces and arranged “any which
way” (figure 2). In addition, in contrast to interruption/disruption as potentially nega-
tive characteristics in design and evaluation of interactive technology and media,
paradoxically interruption from email, SMS or social media, etc. are positive charac-
teristics that provide anticipation and experience that heighten engagement. In many
ways the "cut-up" and interruption are similar to techniques used in filmmaking styles
alternate to Hollywood (e.g. French New Wave, Russian montage), and Brechtian and
improvisational theatre, in which devices are used at unpredictable moments in a pro-
duction/performance to surprise, shock, startle, create juxtapositions, etc. and encour-
age reflection in an audience/participants. Our concern in this paper is not only to
inform design of interaction and gameplay from such devices per se [12], but also to
view users-players as designers, authors, curators, composers and editors creating
their own narrative, texts and experience through interaction and gameplay, within,
between and across applications/games and platforms.

In order to analyze such engagements in interaction and gameplay with one or
more applications/games on one or more platforms, we need an approach, method or
framework that can model, support and represent both in-game/application and
switching between apps and platform. While evaluation typically incorporates a
mixed methods approach with outcomes providing different but complementary re-
sults, there's little in the way of an underlying theoretical model or framework to
inform, guide and connect design, development and evaluation. Activity theory is one
such approach that has long been identified as "a powerful and clarifying descriptive
tool" [17] and widely used in analysis of work-related organizations, systems, and
human-computer interactions (HCI). However, as suggested by Kaptelinin & Nardi
(2006), activity theory needs to evolve in order to move forward towards being a
more practical and theoretical approach for design and evaluation for user experience.
In order to support more than one activity, application and platform, we revisit A. N.
Leontiev's (1981) original work on activity theory to illuminate and extend important
concepts, leading to the proposal of a universal framework for the evaluation (and
inform design) of the user experience of entertainment through engagement in interac-
tion and gameplay. This paper is structured as follows. In the next section we provide
a brief history of activity theory, describe the activity-based scenario/narrative ap-
proach, its extension to a universal framework, and describe its application to interac-
tion and gameplay with multi-platform and multi-application use.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Schematic showing (left to right) analogy of sequential ordering of activities to film frames, and through interaction, improvisation or “cut-up”, users-players can be considered similar to editors, curators, authors, designers creating their own narratives, texts & experiences</figcaption></figure>

2 Activity Theory

Originating from Soviet psychology, two versions of activity theory currently co-
exist: A. N. Leontiev's (1959/1981; 1975/1978 - Russian and English translations)
original hierarchical framework of activity and Engeström's (1987; 1990) expanded
triangle incorporating collective activity (figure 3). While essentially developing from
similar roots found in the work of Vygotsky, the two approaches are different and
even have "different views" of the same concepts (e.g. "object") - refer to Kaptelinin
& Nardi (2006) for informed discussion. While interest has primarily been on
Engeström's (1987; 1990) version (e.g. CSCW, HCI) largely because of its expansion
to analyze social/collective activities, in this paper we focus on Leontiev's (1981)
activity theory approach and in particular, on his first book publication [9]. This is
because it incorporates a hierarchical framework and powerful concepts that define
activity, and as described in the following sections, provide the means to reason about engagement in interaction and gameplay for the user experience of entertainment.

In activity theory, the smallest unit of analysis is activity. However, identification of activity and associated processes, which is crucial for analysis, has been a major hurdle and stumbling block for analysis that has arguably constrained its wider adoption and use. In his psychology and framework, Leontiev (1959/1981) elegantly captured one of the central ideas in Marxist philosophy on the alienation of worker in capitalist production, while at the same time helping to demonstrate the contented (or self-actualized) worker in the Soviet Union (USSR/CCCP) through consideration of the defining concepts of activity, "motive" and "object" (or "objective"). Motive is the intention driving an activity and objective characterizers the activity as a whole and includes all actions or processes carried out toward the fulfillment of motive. By using the original version of activity theory proposed by Leontiev (1981), the problems of identification are minimized. In addition, the same concepts to identify and understand activity provide a means to reason about engagement in interaction and gameplay. While in his second book, Leontiev (1978) joined "motive" with "object" by stating that object of activity is "its true motive", like Kaptelinin (2005), we also find the coupling of "motive" with "object" problematic and argue for considering them as separate, but related concepts. Given that "motive", one the defining concepts of activity, was joined with object (Leontiev 1978), it seems unsurprising then that dealing with, and identifying where activity starts and ends is "notoriously difficult" for researchers and academics who exclusively refer to Leontiev's (1978) second book. While Engestrom's expanded version of activity theory provides a framework to reason about people's collective activity towards an object, because there is no equivalent to the concept motive, we have no explicitly named concept or means to reason about people's level of interest or engagement in activity. Hence, using Engestrom's expanded version of activity theory may be carrying out analysis on people who may be disinterested or unengaged in the collective activity under observation. So in the words of Marx, they are alienated from their work.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Two Versions of Activity theory: Engeström's (1987; 1990) expanded triangle (left) and A. N. Leontiev's (1978; 1981) original hierarchical framework of activity (right)</figcaption></figure>

There are two ways to reason about the relationship between objective outcome and motive. The first and widely published way using Leontiev's (1981) original ac-
tivity theory is the degree to which the outcome from objective “coincides” with mo-
tive. When the outcome from objective coincides with motive, it is fulfilled or com-
plete and the activity ends, and as Leontiev (1981) states, this identifies “activity
proper”. The second alternative way to assess the relationship between objective out-
come and motive identified through re-examination and focused analysis of Leon-
tiev’s (1981) original activity theory, is the degree to which objective outcome merges
with motive which we argue also identifies “activity proper”. This reading and inter-
pretation of the relationship between objective outcome and motive has to our
knowledge received very little if any attention nor has it been previously used in the
assessment of activity. Merges implies two important aspects, firstly, the actions or
processes undertaken are heading in the right direction and secondly, merges doesn’t
necessarily suggest an end point (i.e. objective outcome coincides with motive) but
suggests that as long as actions are contributing to the merging, then motive is being
fulfilled or satisfied. So for example, if a motive to interact or play with technology is
to be entertained or stimulated and the outcome from carrying out/performing pro-
cesses provides just that, then the objective outcome merges with or towards motive.
If this condition is maintained then users-players could hypothetically continue to be
engaged in interaction or play indefinitely (or at least until some other need arises,
from disruption or fatigue, etc.).

## 2.1 Lenses: Activity-Based Scenario / Narrative Approach

An activity-based scenario/narrative approach and framework was proposed [14], building on Leontiev’s (1981) activity theory, to plan, model, describe, develop and evaluate scenarios and narrative of interaction and gameplay. In reference to figure 3, central to this is the hierarchical framework of activity composed of: activity, actions and operations and characterized respectively by objective, goals and conditions. The hierarchical structure is dynamic with shifts between activity, actions and operations orchestrated according to activity theoretical concepts and determined by situations and circumstances (of interaction and gameplay). Its power comes from its lens-like ability to focus on any level of abstraction from high-level descriptions of activities to zoom in to any level of detail/complexity. So providing a flexible and dynamic framework that supports design, development and analysis of interaction and gameplay. However, the focus of this earlier work was on individual activities dealing with one application/game on one platform [14]. In the next section we extend this to deal with one or more activities, applications and platforms.

Activity is directed towards achieving an objective (as denoted by “a”). The objec-
tive is a process characterizing the activity as a whole. When the objective is fulfilled
the activity ends. The objective is closely related to motive, and the motive is the
intention that stimulates and drives a user-player to interact / play a game. In activity
theory, the objective’s outcome and motive have to be considered in the analysis of
“activity proper” [9]. While in previous work [14] this has been used to provide a way
to frame and reason about the degree to which work/play has been successful through
the objective outcome of activity coinciding with the motive that stimulates a player
to activity (“a”). Herein we extend how we assess and design for the user experience
of entertainment through engagement in interaction and gameplay by considering
“activity proper” through how objective outcome and motive merge or are merging.
Activity is made up of a combination of actions (“b”). The action level contains the
heart of the narrative/scenario, using text, graphics, storyboards, etc. to describe the
game environment (e.g. settings, surroundings, circumstances), the game mechanics,
game rules and the gameplay. Actions are performed with conscious thought and
effort, and are planned and directed towards achieving a goal. While actions have
been considered similar to what the HCI literature refers to as tasks [17], Leontiev
(1981) also refers to actions as processes. Herein we refer to actions as processes to
provide a wider view of actions beyond tasks or task-based so that the goal of pro-
cesses can be considered as experience or entertainment. Actions/processes may
themselves be made up of sub-processes directed towards sub-goals, and sub-
processes can be made up of sub-sub-processes, and so on. This depends on the level
of complexity in a narrative, scenario of interaction/gameplay. Actions are performed
by a combination of operations. Operations are performed with little conscious
thought or effort in the use of physical interactive and virtual in-game artifacts trig-
gered by conditions of actions (“c”). Players’ shifts in focus between action and oper-
ation levels provide an indication of learning and reflection. For example, the early
phases of using an artifact will have been performed with deliberate and conscious
attention. At this point they are actions. When they become well practiced and experi-
enced, actions become routine. That is, they do not need to be planned and at such a
point are performed with little conscious thought or effort. In this way, actions be-
come operations as represented by the downward pointing vertical arrow (“d”). This
provides a way to reason about the mastery of (in-game, interface, virtual, real) arti-
facts/tools. Conversely, operations become actions when something goes wrong, im-
pedes interaction, or is associated with user-player learning represented by the upward
pointing lower vertical arrow (“e”). This provides a way to reason about “focus
shifts”, “breakdown”, learning and opportunities for design [21].

## 2.2 Sphere of Engagement Through Motive in Activity

Considering the relationship between objective and motive provides powerful ways to reason about people engaged in activity. If the outcome from objective coincides or merges with motive (that stimulated users-players to perform actions/process of an objective), then they are engaged in activity (see section 2 and 2.1). Activity should not be considered as a holder or container for action/processes and operations, but is defined by objective and motive that identifies user-player engagement in activity – we refer to this as sphere of engagement as illustrated in figures 4 to 7.

During interaction with a computer-based platform (tablet, notebook, desktop,
smart phone), activities are created, ended, fulfilled or postponed. Users-players can
become engaged in several applications (on-line, social media, game) in an interactive
session/encounter. If the motive for interacting/play with each application is different,
then activities representing each application are separate as depicted by the spheres of
engagement in figure 5. But if the overall motive that stimulated a user-player to in-
teract-play with a computer-based platform is shared between applications/activities
(e.g. entertainment, stimulation) then they may dip-in and out of, switch or shift be-
tween applications/activities while at the same time remaining engaged (irrespective of kinds of user experience or types of entertainment). Here the sphere of engagement encapsulates more than one application/activity, as in figure 6.

<figure><img src="image_5.png" /></figure>

**Clockwise from top left** - Sphere(s) of engagement: **Fig. 4.** User-player engaged in activity (A1). **Fig. 5.** Engaged in separate activities (A1-A3). **Fig. 6.** Encapsulating one or more application/platforms (A1-A3). **Fig. 7.** In-game activity (A1) and out-of-game activity (A2)

It's easier to appreciate sphere of engagement with many apps each represented by an activity that share a motive and are supported on one platform (tablet, notebook, desktop, smart phone) but what about applications (on-line, social media, games, video calls) supported on more than one platform. Likewise, we argue that if each activity share a motive such as to be entertained or stimulated, etc. then the user-player can remain engaged in interaction /gameplay switching between apps and platforms and likewise a sphere of engagement is created/develops (figure 6). As mentioned in the introduction, as observed in studies in games clubs and schools, user-players switch between in-game play to interacting with fellow user-players/students out-of-game/off-game. Rather than have a detrimental effect, this behavior appeared to heighten experience and engagement. Here the sphere of engagement encapsulates the real-world environment where the game is situated (figure 7). This perspective has implications for analysis of blended learning whereby in-game and off-game learning activities are connected through sphere of engagement. In design, learning activities can be designed so that in-game and off-game motives are connected and are encapsulated within a sphere of engagement. Activities can be the same or different for in-game and off-game. For example, while in-game play could be to learn about some topic in history, a user-player's attention could be diverted to an off-game activity to undertake a mathematical exercise. Here an action or process in the history game is either shared with/leads an activity to undertake a mathematical exercise in the real-world or an action transforms into an activity to undertake a mathematical exercise in the real-world. Similarly, the sphere of engagement illustrated in figure 7 also represents gameplay for entertainment undertaken between in-game and out-of-game activities.

## 2.3 Creation of narrative, scenario, improvised story & experience

As described in section 2.2, the activity-based narrative/scenario approach and associated concepts, provide a flexible framework for analysis and design of user-player interaction and gameplay with one or more application on one or more platforms. It's
not difficult to see how the framework can support a variety of multi-application and multi-platform use (as described in section 1) in our analogy of user-player as editor, curator, composer, director, author, etc., creating their own narrative, texts and experience through interaction and gameplay, within, between and across applications/games and platforms. For example, as discussed in section 1, user-players construct their own narratives of user experience and entertainment by shifting from app to app on one or more platforms according to tastes, interests, preferences, individual and group cultures and sub-cultures, as well as serendipitous and improvised on-the-fly interaction, the activity-based narrative/scenario approach provides a hierarchical framework and concepts to model, describe, reason about and trace such interactions and gameplay. In addition, the activity-based narrative/scenario approach can also provide support for future interactions and gameplay. For example, recently, we have become more and more aware of the increased interest from leading computing, software and social media corporations (Google, Facebook) in the once novel technological products and platforms and novel interactions that have not seen daylight outside of research labs and associated conferences for the last few decades. In particular, the focus of interest has been on emerging wearable platforms (e.g. VR headsets, iGlass, etc.) that can provide support for many apps on one platform. While the activity-based narrative/scenario approach provides a hierarchical framework and concepts to model, describe, reason about and trace user-player interactions and gameplay with such platforms (as outlined in section 2.2), it can also inform design for transitions between applications. For example, Apple is already alert to similar design opportunities as demonstrated through iPad's use of audio fades (in-and-out) and visual dissolves in response to undetermined or random user-player "cut-ups"/"mash-ups"/switches between apps and services. Extending this idea, the activity-based narrative/scenario approach could be used to inform the design of devices for orchestrating or persuading user-player "cut-ups"/"mash-ups"/switches between apps and services to heightened experiences and engagements.

# 3 Discussion and Conclusion

In this paper we have proposed an approach and framework to reason about the user experience of entertainment through engagement in task-based, as well as improvised, extemporaneous and serendipitous interaction and gameplay. Towards this we have explored the original writing of A. N. Leontev (1981) and proposed extensions to bridge conceptual gaps to operationalize an activity-based scenario/narrative approach [14] leading to a universal framework. Its power comes from two main approaches, lens and spheres: firstly, its lens-like ability provides a way to focus on any level of abstraction from high-level descriptions of activities to zoom in to any level of detail/complexity; secondly, considering activity through the relationship between "objective" and "motive", and by the degree to which they coincide and merge provides powerful ways to reason about people engaged in activity through the user experience of entertainment and captured in the term sphere of engagement provides concepts to deal with one or more activities, applications and platforms.
## References

1. Boyle, E.A., Connolly, T.M., Hainey, T., Boyle, J.M.: Engagement in digital entertainment games: A systematic review, Computer in Human Behavior, 28, 3, 771-780 (2012)

2. Burroughs, W.S. and Gysin, B.: The Third Mind (1978)

3. Danzico, L.: The Art of Editing: The New Old Skills for a Curated Life, XVII.1 Interactions, Jan/Feb (2010)

4. Engeström, Y.: Learning by Expanding: An activity-theoretical approach to developmental research. Helsinki: Orienta-Konsultit (1987)

5. Engeström, Y.: Activity Theory and Individual and Social Transformation. In: Perspectives on Activity Theory-Learning in Doing Social, Cognitive and Computational Perspectives, Part 1: Theoretical Issues, edited by Y. Engeström, R. Miettinen, and P. Punamäki. Cambridge, UK: Cambridge University Press, pp. 19-38 (1999)

6. Kaptelinin, V. & Nardi, B.: Acting with Technology: Activity Theory and Interaction Design, MIT Press (2006)

7. Kaptelinin, V. and Nardi, B.: Activity Theory in HCI Research: Fundamentals and Reflections. San Rafael: Morgan & Claypool (2012)

8. Leontiev, A. N.: Activity, Consciousness, and Personality. Prentice-Hall (1978)

9. Leontiev, A. N.: Problems of the Development of the Mind. Moscow: Progress (1981).

10. Lehmann, J., Lalmas, M., Dupret, G., Baeza-Yates, R.: Online Multitasking and User Engagement, ACM International Conference on Information and Knowledge Management (CIKM 2013), San Francisco, USA (2013)

11. Manninen T.: Virtual Team Interactions in Networked Multimedia Games - Case: "Counter-Strike" - Multi-player 3D Action Game. In Proceedings of PRESENCE2001 Conference, Philadelphia, USA, Temple University (2001)

12. Marsh, T., Nitsche, M., Wei Liu, Chung, P., Bolter, J.D., Cheok, A.D.: Film Informing Design for Contemplative Gameplay, ACM SIGGRAPH (2008)

13. Marsh, T.: Users as editors and curators: activity framework for cultural media interaction design, Cultural Computing: Art, Culture and Technology, Workshop, ICEC2010, 9th International Conference on Entertainment Computing, Seoul, South Korea (2010)

14. Marsh, T.: Activity-Based Scenario Design, Development and Assessment in Serious Games, Gaming and Cognition: Theories and Practice from the Learning Sciences, IGI Global (2010)

15. Marsh, T. & Costello, B.: Experience in serious games: between positive and serious experience, Serious Games Development & Applications, SGDA2012 (2012)

16. Marsh, T. & Sim J.J.: Entertainment and Language Learning: Voice Activated Digital Game and Interactive Storytelling Trials in Singapore Schools, ICEC2014 (2014)

17. Nardi, B.: Context and Consciousness: Activity Theory and Human-Computer Interaction. Cambridge, Massachusetts: MIT Press (1996)

18. O'Brien, H.L. & Toms, E.G.: What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science & Technology, 59(6), 938-955 (2008)

19. Schoenau-Fog, H., Louchart, S., Lim, T., & Soto-Sanfiel, M.T.: Narrative Engagement in Games – A Continuation Desire Perspective, Foundations of Digital Games (2013)

20. Seay, A.F., Jerome, W.J., Lee K.S., Kraut, R.E.: Project Massive: A Study of Online Gaming Communities, CHI 2004 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM Press (2004)

21. Winograd, T. and Flores, F.: Understanding Computers and Cognition: A New Foundation for Design. Norwood, NJ: Ablex Publishing Corporation (1986)