JMIR FORMATIVE RESEARCH

Cole-Lewis et al

Viewpoint

Understanding Health Behavior Technology Engagement: Pathway
to Measuring Digital Behavior Change Interventions
Heather Cole-Lewis*, MPH, PhD; Nnamdi Ezeanochie*, MD, DrPH; Jennifer Turgiss, MS, DrPH
Johnson and Johnson Health and Wellness Solutions, Inc, New Brunswick, NJ, United States*these authors contributed equally

Corresponding Author:
Nnamdi Ezeanochie, MD, DrPH
Johnson and Johnson Health and Wellness Solutions, Inc
1 Johnson and Johnson Plaza
New Brunswick, NJ, 08933
United States
Phone: 1 7209330786
Email: nezeanoc@its.jnj.com

Abstract
Researchers and practitioners of digital behavior change interventions (DBCI) use varying and, often, incongruent definitions of
the term “engagement,” thus leading to a lack of precision in DBCI measurement and evaluation. The objective of this paper isto propose discrete definitions for various types of user engagement and to explain why precision in the measurement of theseengagement types is integral to ensuring the intervention is effective for health behavior modulation. Additionally, this paper
presents a framework and practical steps for how engagement can be measured in practice and used to inform DBCI design andevaluation. The key purpose of a DBCI is to influence change in a target health behavior of a user, which may ultimately improvea health outcome. Using available literature and practice-based knowledge of DBCI, the framework conceptualizes two primarycategories of engagement that must be measured in DBCI. The categories are health behavior engagement, referred to as “BigE,” and DBCI engagement, referred to as “Little e.” DBCI engagement is further bifurcated into two subclasses: (1) user interactions
with features of the intervention designed to encourage frequency of use (ie, simple login, games, and social interactions) andmake the user experience appealing, and (2) user interactions with behavior change intervention components (ie, behavior change
techniques), which influence determinants of health behavior and subsequently influence health behavior. Achievement of Big
E in an intervention delivered via digital means is contingent upon Little e. If users do not interact with DBCI features and enjoy
the user experience, exposure to behavior change intervention components will be limited and less likely to influence the behavioral
determinants that lead to health behavior engagement (Big E). Big E is also dependent upon the quality and relevance of the
behavior change intervention components within the solution. Therefore, the combination of user interactions and behavior changeintervention components creates Little e, which is, in turn, designed to improve Big E. The proposed framework includes a model
to support measurement of DBCI that describes categories of engagement and details how features of Little e produce Big E.
This framework can be applied to DBCI to support various health behaviors and outcomes and can be utilized to identify gaps
in intervention efficacy and effectiveness.
(JMIR Form Res 2019;3(4):e14052) doi: 10.2196/14052
KEYWORDS
engagement; user engagement; health behavior; health determinants; digital behavior change intervention; measurements

Introduction
Background
Globally, the creative integration of technology solutions toaddress health issues is growing [1]. This integration has been
largely through the promotive uptake of healthy behaviors toachieve desired health outcomes via technology [2]. In thispaper, these technologies will be referred to as a digital behavior
https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderXchange intervention (DBCI). Recent systematic reviews
demonstrate that digital interventions supported by behavior
science evidence hold the key to driving critical behavior change
processes that lead to improved health behaviors and healthoutcomes [2]. Common examples of targeted behaviors include
smoking cessation, increase in physical activity, improvements
in dietary habits, medication adherence, and safe sexual practices[3].
JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
However, the extent to which these innovations can deliver on
the promise of demonstrable positive health outcomes depends
on the successful utilization of interventions by users and thesubsequent sustained performance of the intended healthbehaviors. This utilization has been generally referred to as“engagement” or “user-engagement.” This term is largely
recognizable, but at the same time, abstract and difficult tomeasure accurately in different settings and contexts. Thismeasurement limitation can be mitigated by applying
frameworks/models that guide the way engagement is
considered, measured, and applied in analysis within anintervention. Precise definitions and measurements forengagement will allow for better estimation of DBCI
effectiveness and provide accurate insights that inform relevantintervention recommendations [4].

Goal
The aims of this study are (1) to provide an analysis of current
engagement definitions and models in the literature, (2) topropose a new model that builds on existing frameworks but
addresses current limitations, and (3) to describe the implications
for applying the proposed framework to measurements andanalysis within DBCI.

Analysis of Current Definitions of Engagement in
Literature
It is generally accepted that user engagement with DBCI is aprecursor to improved health outcomes. Growing evidence
supports that interaction with DBCI and adherence tointervention features improve the likelihood of reaching desired
health outcomes [5]. There is currently no universally acceptedand comprehensive definition for engagement, and this has two
main implications. First, there is a limited and inconsistentunderstanding of the engagement’s specific impact, particularly
on DBCI effect size, attrition, and sustained health behaviors[4]. Second, there is also a lack of precision in DBCI
measurement and evaluation, as the term “engagement” can beinterpreted differently across industries (eg, marketing,psychology, and human-computer interaction) [6].
One definition of engagement evaluates esthetics andnavigability, particularly how users interface with digitalinterventions [7]. This type of engagement typically observes
usability, measuring interactions with features and functions of
the digital solution. This definition is important because it
provides insight into the level of use and interaction of a digitalintervention. It also informs what components of the intervention
users enjoy or use the most and opportunities to leverage these
insights to inform future interventions.
Engagement on this level is an important element for the digitalintervention scale. O’Brien and Tom [8] defined engagement
as an assessment of a user experience (interaction andinvolvement) with an innovation or technology-based
intervention [8]. They argue that this experience leads to theformation of sentiments that impact a user’s likelihood to
advocate the use of the innovation among others within theirsocial networks. Furthermore, this definition of engagement
helps researchers understand more about the type of users
involved in their intervention (ie, data from this engagement
https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

Cole-Lewis et al
addresses questions such as the emerging user-engagement
hierarchy identified via subgroups based on user interaction
with the features of the digital intervention) [9]. With anunderstanding of the baseline users of an intervention and theircharacteristics, researchers can build updates to digitalinterventions that positively support most types of users [8].
Nevertheless, there are limitations with this type of engagement
definition. Definitions based exclusively on user interactions
assume that engagement with the digital innovation ultimately
leads to the intended intervention outcome (ie, higher
engagement and adoption lead to desired health outcomes). This
assumption is problematic because of the current limited
interpretational challenges user interaction data present, suchas the difficulty in distinguishing online versus offline
interactions and the implications for intervention exposure, orthe absence of frameworks that identify unique meaningfulinteraction sequences with digital features that translate tospecific behavior change techniques (BCTs) embedded in a
DBCI [10]. Therefore, this level of engagement definition is
insufficient in meeting the need for better and more precise
measurement of engagement to inform the effectiveness of
DBCI.
Some researchers conceptualize engagement in terms of a user
performing the intended health behaviors within an intervention[11]. This type of engagement seeks to understand therelationship between using the digital intervention and behavior
change, drawing on evidence-based principles and behavioral
theories to evaluate changes in health outcomes [11,12]. The
advantage of this definition is that it is ultimately concernedwith whether an individual performed the desired healthbehavior. This definition can closely tie the effectiveness of anintervention to a measure of engagement. Definitions ofengagement based on behavior performance parameters only
do not provide details of user interactions with components ofa digital intervention. Thus, an understanding of what specificdigital features are linked with behavior engagement (ie, whatfeatures are effective) is absent. This is a significant limitation
because data needed to update features of the intervention areabsent; thus, future optimization and cost-saving processes
informed by evidence will be nearly impossible [13]. Therefore,
one is left with a potentially effective DBCI and nounderstanding of what features contributed to their efficacy,thus lacking insights to inform intervention optimization andscale.
The absence of a standardized DBCI framework for defining
and measuring engagement inhibits the ability to understandthe mechanisms of action of an intervention. Therefore, current
approaches are limited in providing accurate and detailed
measurement data necessary to demonstrate effectiveness of
DBCI.
A proposed model should include definitions of engagement
related to DBCI feature interaction as well as performance ofthe desired health behavior [14]. This approach can informwhich specific features of a DBCI influence performance of ahealth behavior. Essentially, an integrated model should be able
to address the relationship between user interaction with the
DBCI and the user health behaviors, thus answering the
JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
question, “What level of user-engagement with the DBCI, andby which users, leads to a desired health?”
Furthermore, this model should ideally delineate and identify
unique, meaningful interaction sequences that represent digitalfeatures informed by specific behavior change theory–informed
components of the intervention (eg, BCTs) [13]. Thishierarchical representation of user feature interaction andintervention exposure to BCTs informs a robust dataset that
contains explanatory variables that enable a deeper DBCI
analysis in real-world settings and an explanation of themechanism of action with regard to change processes [13]. For
this model to be complete, we need to account for behavioral
determinants that mediate or moderate the association between
digital features engagement and their impact on associatedintervention health behavior outcomes. These determinants
should include determinants related to user engagement withDBCI [15]. Examples include technology self-efficacy;satisfaction with the DBCI; intervention usability [15]; andtechnology-associated determinants informed by the technology
acceptance model (TAM), such as perceived usefulness,
perceived ease of use, and perceived compatibility [16]. In
addition, psychosocial determinants rooted in behavioral theories
relevant to the behaviors of interest such as the social ecologicalmodel [17]; Capability, Opportunity, Motivation - Behavior(COM-B) model [18]; and social cognitive theory [19] must be
included in the proposed engagement model. A model housing
both engagement definitions (engagement with DBCI anddesired health behavior, technology determinants, psychosocial
determinants, and outcomes) is robust enough to inform datasetsthat support rigorous evaluation and measurements of DBCI.

The Johnson and Johnson Approach to
Health Engagement: Definitions andFramework
Overview
Following the abovementioned recommendation outlining a
proposed model for defining and measuring engagement, we
present a definition and model for engagement. Several steps
were taken to achieve this goal.

https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

Cole-Lewis et alFirst, several definitions of engagement were reviewed fromthe literature and practice-based knowledge surrounding DBCI
[7,11-15]. These definitions were analyzed to identify current
limitations.
Second, new definitions for engagement were proposed. The
intention here was to have new sets of definitions that addressed
the observed limitations of current definitions and to standardize
terminologies around DBCI engagement with the intention ofcreating better DBCI measurements.

Defining “Big E” and “Little e” Engagement“Big E” is engagement with the targeted health behavior,
hereafter referred to as health behavior engagement, and is theprimary outcome of a DBCI. However, the goal is to attainhealth outcomes through the health behavior. Thus, it isimportant that engagement with the health behavior is achievableand measurable to determine if a DBCI is successful.
“Little e” is engagement with the digital behavior changeintervention. This is sometimes referred to as user interaction
with the digital solution. Hereafter, the term DBCI engagement
will be used to represent engagement with the digital solution.
DBCI engagement is comprised of two types of interactions:••User interactions with the DBCI features and the context
in which those interactions happen (little eUI)Interactions with behavior change intervention
components/active ingredients specifically designed toinfluence the behavioral determinants which in turninfluence the health behaviors (little eBCT)The success of health behavior engagement is dependent upon
DBCI engagement. If users do not interact with DBCI features,
exposure to the behavior change intervention components willbe less likely to influence behavioral determinants and lead tohealth behavior engagement across a broad population. Healthbehavior engagement is also dependent upon the quality andrelevance of the behavior change intervention components
within the solution. Therefore, user interactions and behavior
change intervention components combine to create Little e; thisin turn influences Big E. Figure 1 illustrates how thesedefinitions of engagement inform the overall intervention
measurement.

JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Cole-Lewis et al

Figure 1. Diagram illustrating the Johnson and Johnson approach to health engagement. UI: user interaction; Ux: user experience; BCT: behavior
change technique.

Figure 1 indicates that a sedentary individual will need tointeract with the appropriate level of the DBCI features (little
eUI) to be exposed to the right and effective level of behavior
change intervention components (little eBCT). This should ideallylead to changes in determinants that support engagement withthe health behavior. Inadequate levels of little eUI interaction
or ineffective exposure to appropriate BCTs will not lead tohealth behavior engagement (Figure 2). In some cases,
appropriate exposure to a BCT is based on what is clinically
relevant to influence a health behavior and by extension, a healthoutcome. For example, physical activity (steps) tracking is a
BCT (self-monitoring of a behavior), but attaining x number ofsteps will be needed to begin seeing a decrease in y amount ofweight (health outcome).

Figure 2. Diagram illustrating relationship engagement measurements. UI: user interaction; BCT: behavior change technique.

Principles from program planning and intervention design
literature [20] were leveraged to develop a framework model
unique for building DBCI. This model is a hybrid that uses thelayout of a theory of change conceptual model and a traditional
logic model (Table 1) to provide the structure upon which we

https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

systematically embedded the new engagement definitions andthen tested the robustness of these definitions in order to produce
the right amount of data to inform complete and accurate
measurements of DBCI effectiveness.

JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Cole-Lewis et al

Table 1. The Johnson and Johnson approach to health proposed engagement definitions and framework. As with a traditional logic model, this tableshould be read from left to right. The underlying assumption is that exposure to an intervention should lead to changes in determinants that in turninfluences health behaviors, health outcomes, and organizational outcomes. For each phase of the logic model, the table illustrates examples of theengagement category, measurement category, sample metrics, and the metrics/data source.
Logic model
category

Exposure to intervention

Determinants

Health behavior

Engagement
category

Little e engagement with the DBCIa—b

Big E engagement Outcome
in health behaviors

Measurement
category

UIc interactions

BCTd intervention Capability
Opportunity motivation
Contextual informa- components
tion

Health behaviors

Health outcomes at Organizational
individual level
outcomes

Sample metrics

ClicksGoal setting

Change in:

Restructuring
thoughts

SkillsSufficient level ofphysical activity

Improved A1C

SwipesSelf-efficacy to perform
physical activity

Targeted nutrition
behaviors

Location data

Health outcomes

Organizational
outcomes
Outcome

Maintain ideal
weightDescrease demandof health care system

In-app/out of app

In-app/out of app

Access to food store withhealth meal options
Metric/datasourceaIn-appIn-appIn-app/out of app

In-app/out of app

DBCI: digital behavior change interaction.

b

Not applicable.

c

UI: user interface.

d

BCT: behavior change technique.

This framework provides an engagement category that details
how features of Little e inform Big E and builds upon
components that align with an intervention logic model, suchas exposure to the intervention, determinants, health behaviors,
health outcomes, and organizational outcomes. There were five
steps that informed the process of building the model and its
components:
1.

2.
3.

4.

5.

Utilization of market research, economic evaluation, andsubject matter expertise in therapeutic areas to inform ourchoice of the organizational, individual health, and behavior
outcomes (ie, we knew what specific behaviors needed tobe performed to lead to the outcomes of interest and by
what effect size).
Review of the behaviors and created subbehaviors andspecific performance indicators associated with them.
Examination of the behavior science literature to deduce
the factors (mediator or moderators) that influence theperformance of the behaviors among individuals in thetarget population. These factors were grouped using various
frameworks such as (a) enabler and barriers; (b) mediators
or moderators; (c) factors personally and environmentally
informed by the socioecological model [17]; and (d)capability, motivational, or opportunity factors [18].
Use of the understanding and learnings from several
theories, such as the classical and operant processes, social
cognitive theory, social learning theory, andself-determination theory [19,21-23], to determine which
behaviors fall within the involuntary and voluntary process.
Thereafter, we used this guidance to determine antecedentand consequence behavioral determinants (ie, BCTs thatmake up components of the intervention).
Use of the understanding from the intervention mapping
literature [20] to determine specific change processes thathttps://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

illustrated our best-informed assumptions about which
specific BCTs drive factors influence performance of the
behaviors. This helped define and guide the digital productstrategy where each behavior change process is represented
by a set of user experience/user interaction (UI) and content
features.
The measurement category provides additional detail of what
factors should be measured in each engagement category.
Finally, the example metric category provides examples of
metrics used during evaluation.

Implications of the Engagement
Framework for Interventions
Measurements
Overview
The framework provides a structure to discuss implications forthe measurement and analysis of DBCI. These implications will
be discussed under the categories of the engagement framework.
They include exposure to the intervention, determinants,
behavior, and outcomes. These categories are organized in amanner that provides information on the potential causal
pathway of change between intervention exposure and outcomes,that is, exposure to the intervention (Little e) influences
determinants that influences behaviors and ultimately leads tohealth outcomes. Insights on approaches to DCBI analysis using
engagement data will also be discussed in this paper.

Exposure to the Intervention (Little e)
This category is comprised of two engagement types: (1) userinteraction with the DBCI features (little eUI) and (2) interactions
with the behavior change interventions components (little eBCT).
JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Cole-Lewis et al

User Interaction With the Digital Behavior ChangeInterventions

components of behavior change theory that are used to influencechanges in determinants, and ultimately, health behaviors.

This refers to user interaction with the technology. Exemplary
categories of digital solution interactions that could be measured
include number of logins, clicks, swipes, time spent interactingwith each feature, order of interactions (path taken), andreal-time feedback/assessment of user experience. These
interactions can be captured within the solution at the backend
by using combinations of product analytics data, representing
user-level individual interactions data over time (ie,longitudinally). It is critical to collect individual level,time-series data and specify this request upon the developmentof a DBCI solution. Often, however, out-of-box analyticalprograms do not have this capability as the default setting.

Behavior change interventions are most effective whendeveloped using behavior change theory [2]. Therefore, we haveto understand exposure to behavior change techniques todetermine specifically which behavior change intervention
components are important to design and then measure in the
DBCI. Behavior change techniques are the smallest component
parts of widely used theories of health behavior, sourced fromvarious fields and through consensus of experts [10]. BCTs areused because they enable researchers to focus on specificcomponents of an intervention in order to determine how mucheach aspect of the intervention is contributing to the desired
outcome. In other words, use of behavior change techniques,
the smallest component part of a behavior change theory, allows
us to parse out the effect of each behavior change technique,thus developing an understanding of which technique works
for which population and outcome. When these insights aredeveloped in a real-world setting across a range of users, they
can be translated for a larger group of users to obtain better
outcomes. These BCTs are often experienced as features in theuser experiences (Figure 3) and, in some cases, inform how thetechnology solutions may function.

Interactions With Behavior Change Intervention
Components
Interaction with, and subsequent exposure to, the behavior
change intervention components (little eBCT) is designed
specifically to influence the determinants of behavior [15].
Using theory-informed models [17,19-23], we can detail whichuser little eBCT interactions are associated with specific healthbehaviors. In traditional intervention design, these are the key

Figure 3. Diagram illustrating unique constellation of user interaction/user interface features that represent behavior change techniques. UI: userinteraction; BCT: behavior change technique.

Further, BCTs arranged in a taxonomy, or ontology, allow fororganization of the components in technology design by making
behavior change theory concepts machine readable (eg,hierarchically categorized and mutually exclusive), whichenables behavior science to converge with data science [24].
Examples of BCTs include goal setting, action planning,
feedback, monitoring, and problem solving. A full list of
https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

behavior change techniques commonly used in various fields
of behavior science can be found in The Behavior Change
Technique Taxonomy (v1) of 93 Hierarchically ClusteredTechniques developed by Michie et al [10]. This approachenables the identification of active ingredients of the DBCI (ie,the behavior change intervention components that are mostcritical to improve behavior) and ultimately determine a
JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
dose-response relationship between the DBCI and the desired
health behavior and health outcome.
Measurement of behavior change intervention components (little
eBCT) can be represented by a combination of user interface
interactions. Some of the same interactions captured forunderstanding user interaction with the system (little eUI) can
be combined to create measures for DBCI components/active
ingredients (Figure 3).
In this illustration, the app user is exposed to an “Action
Planning” BCT (little eBCT) via a combination of calendar text
features and a select button (little eUI) to schedule an exercise
routine.

Determinants
Determinants refers to the influence of factors that shapeindividual decisions and actual performance of the behaviors.
Prior work on digital health technology engagement models
advocate for examination of two main level of determinants:(1) determinants of user engagement with the intervention (eg,
DBCI) and (2) psychosocial determinants [15]. Determinantsof user engagement focus on the influences that support userinteraction with a DBCI. For example, technology self-efficacy,satisfaction with the DBCI, intervention usability, andnavigability and technology perceived compatibility and easeof use [15,16]. Psychosocial determinants refer to influence offactors (usually social) on individual/group decision andperformance of behaviors (desirable or not) [25]. These factors
are informed by various behavior science theories such as the
COM-B model, socioecological model, the classical and operant
processes, social cognitive theory/social learning theory, andself-determination theory [17-19,22,23].
Both types of determinants can be grouped as mediators ormoderators and help tie both definitions of engagement (Big E
and Little e) together. According to the COM-B model, behavior
change occurs when motivation meets capability andopportunity. Specifically, for a behavior to happen, the person
must have the capability, opportunity, and motivation to performthe behavior [18]. In our Johnson and Johnson approach tohealth engagement framework, behavior is defined as anobservable action that a person does, capability is a physicaland psychological ability to perform the behavior, opportunity
is an immediate environmental factor(s) that promotes or enables
a behavior, and motivation refers to the mental resolve to directa behavior [18].
Within the DBCI, determinants can be captured in app throughassessments, although the measurement of determinant is notcritical. These assessments could be periodic or based on apredefined logic (for example, business rules informing a
DBCI). In some instances, this predefined logic is built around
specific ecological and environment contexts that are timed or
random. These types of assessments are generally referred toas an ecological momentary assessment (EMA) [26]. In thecontext of a study or research, determinants could be measured
using a survey. Determinants are important to measure theeffectiveness of intervention components on behavior andoutcomes. Therefore, it is important to ensure they can bemeasured within DBCI during product testing to understand thehttps://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

Cole-Lewis et alfull mechanism of action. Once a DBCI is released, testing ofdeterminants is unnecessary if no new audiences or population
subgroups are introduced. When choosing a data capture
method, it is critical to weigh the value of the data to be obtainedwith user burden expected. This is because a tedious user
experience on data capture (eg, more burden on the user toactively track data) can, in turn, influence user engagement
determinants negatively [27].

Health Behavior (Big E)
In this model, we categorize health behaviors as lifestyle
behaviors, health care–related behaviors, and one-off behaviors.
Lifestyle behaviors refer to activities performed very frequently,
such as physical activity, sleeping, and eating. Examples ofhealth care–related behaviors include adherence to medicationand treatment plans, while getting a flu shot is an example of aone-off health behavior. Health behaviors can be measured
though a variety of ways (in app, EMAs, surveys, etc). However,when possible, behaviors should be captured passively using
in-app or out-of-app sensors to ensure accuracy of the data andreduce user burden on the people to track and monitor their
behavior (except in cases when the intervention specifically
calls for it).

Analysis of Digital Behavior ChangeInterventions Using Engagement Data
Overview
Data from the engagement framework ultimately aims todescribe specific relations between Little e and Big E andexplains the pathway of these relationships through the influenceof determinants. In other words, DBCI analysis seeks tounderstand which BCT exposures lead to which health behavior
and through the effects of which determinants. This framework
informs the development of a specific dataset that addresses
these questions. There are two broad approaches to consider
when analyzing such data: an a priori theory approach and agrounded theory approach.

A Priori Theory Approach
In this approach, DBCI interactions and health behavior
engagement can be identified or assigned as the solution has
been designed. This is largely informed by theory and learnings
from behavior-based intervention implementation literature,such as intervention mapping [20]. A priori definition helps
determine if specific assumptions or theoretical concepts utilized
in the design of the DBCI solution hold true. For example, if a
DBCI feature was designed to help users address a specificmotivational determinant of behavior, and the feature was neverutilized, you would not expect the motivational determinant to
be influenced through that feature/conceptual approach.
Similarly, if the feature was utilized, but the person’s motivation
did not change, you may question whether this component wasan effective way to address this motivational factor. The criticallimitation of this approach, however, is that researchers are
restricted to the number of behavior change pathways (ie, fromLittle e to Big E) that were originally accounted for. Emerging
new change processes could be potentially missed if they were
not initially built in (ie, the analysis of the intervention is only
JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
as good as the change processes and pathways originallycaptured).

A Grounded Theory Approach
In this approach, DBCI interactions and health behaviors areinterpreted in real time or after the solution has been used todevelop new learnings in order to make the DCBI more
effective, if needed. In this instance, methods such as cluster
analysis or path analysis can help understand how users
experience the intervention, identify useful features, anddetermine various user pattern groups (that might indicate whichpath is most successful for specific types of users) [28,29]. Thisapproach helps identify patterns and interactions that the user
takes. In some cases, the emerging user engagement patterns
and change processes were not intuitive or originally intended.
These findings can be used for further exploration and to inform
other types of analyses. However, there could be a challenge in
capturing and interpreting the data and thus running the risk ofspurious associations and findings if they are not vetted byevidence or do not fit the schema of an existing theory [28].
In summary, there is potential value in each approach. However,the pros and cons of both approaches should be carefully
considered when making analytic decisions.

Conclusion and the Promise of
Engagement Data to Inform Measurementof Digital Behavior Change Interventions
Measurement of both little eUI and intervention BCT components(little eBCT) are necessary to understand the effectiveness of
DBCI on Big E (health behavior). DBCI guided by the Johnson
and Johnson approach to health engagement framework helps
clarify exactly which actions a user takes (path) and the context
in which such interaction and health behavior occur. Due toadvancements in technology, it is possible to collect data overtime regarding Little e and Big E and to consider additional
determinants’ data on dynamic contextual information (eg,location, time of day, and biometric characteristics like heartrate). Other determinant information (including demographics)
are also important and should be leveraged to build user profiles
that are robust and describe behavior change processes fromLittle e to Big E and influence our understanding of behavior
change theories [15].
The full potential of digital health solutions can be achieved
when insights from human behaviors can be observed andmeasured discretely to enhance and improve our existing models
of behavior change theory, which were developed mostly using

Cole-Lewis et al
static measures of behaviors. Given the power of technology,
we are now able to capture fluid behaviors of an individual andare better equipped to address the question of which intervention
works best for whom and for what outcome.
It is also possible to measure mechanisms that influence
adoption and maintenance of health behaviors to build productefficiencies and effectiveness (relevant to the greatest audience
subgroup as feasible). Such mechanisms include Little e(interaction with UI + interaction with BCT components=healthbehavior via a determinant). Equipped with data about userinteractions with technology, exposure to intervention BCT
components, and execution of health behavior, we can answer
questions of not only if the intervention, but also how theintervention worked and for whom.
We can identify and capitalize on efficiencies and observed
insights to optimize a user’s path to the health behavior andincrease the likelihood of sustaining that health behavior.
Evidence has shown that tailored and individualizedinterventions are successful for improving health behavior andhealth outcomes [30]. Utilizing all data available (little eUI, little
eBCT, Big E), statistical and computational analyses can beutilized to understand patterns of human behavior, identify
determinants of human behavior, and personalize interventions
specifically for the person receiving the intervention [29,30].
By studying little eUI, little eBCT, and health behaviors in tandem
and separately, we can address questions such as the following:•••

What is the best user experience for users with particularcharacteristics?
What is (are) the most appropriate set of behavior change
technique(s)/intervention components, given a user has acertain set of characteristics?
What is the next appropriate intervention component toprovide a user, given they have had positive technology
interactions with these components previously?

Essentially, through these data, we become equipped to tailor
and personalize interventions without requiring a static batteryof psychosocial and preference questions in the initialinteraction.
Finally, to reach the full potential of technology applied tobehavior change (DBCI), there is a need to draw from diverse
populations and observe and learn how DBCI works for different
people. Implementing an engagement framework–guided DBCI
at scale enables the gathering of a diverse set of data while
simultaneously diversifying the intervention appropriately at
scale, rather than starting in small pockets and building healthtechnologies that are inflexible.

Conflicts of Interest
HC-L, NE, and JT are full-time employees of Johnson & Johnson Health and Wellness Solutions, a center of excellence (COE)group within Johnson & Johnson.

References
1.

Yamey G. Scaling up global health interventions: a proposed framework for success. PLoS Med 2011 Jun;8(6):e1001049[FREE Full text] [doi: 10.1371/journal.pmed.1001049] [Medline: 21738450]

https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
2.
3.

4.

5.

6.
7.
8.
9.
10.

11.

12.

13.
14.

15.
16.
17.
18.
19.
20.
21.
22.
23.
24.

25.
26.
27.

Bull S, Ezeanochie N. From Foucault to Freire Through Facebook. Health Educ Behav 2015 Sep 17;43(4):399-411. [doi:10.1177/1090198115605310]
Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobileinterventions: are our theories up to the task? Transl Behav Med 2011 Mar;1(1):53-71 [FREE Full text] [doi:10.1007/s13142-011-0021-7] [Medline: 21796270]
Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related
to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res
2011 Jan 06;13(1):e2 [FREE Full text] [doi: 10.2196/jmir.1639] [Medline: 21212045]
Lustria MLA, Noar SM, Cortese J, Van Stee SK, Glueckauf RL, Lee J. A meta-analysis of web-delivered tailored healthbehavior change interventions. J Health Commun 2013;18(9):1039-1069. [doi: 10.1080/10810730.2013.768727] [Medline:
23750972]
Bouvier P, Lavoué E, Sehaba K. Defining Engagement and Characterizing Engaged-Behaviors in Digital Gaming. Simulation& Gaming 2014 Nov 06;45(4-5):491-507. [doi: 10.1177/1046878114553571]
O'Brien HL, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology.
J Am Soc Inf Sci 2008 Apr;59(6):938-955. [doi: 10.1002/asi.20801]
O'Brien HL, Toms EG. The development and evaluation of a survey to measure user engagement. J Am Soc Inf Sci 2009
Oct 19;61(1):50-69. [doi: 10.1002/asi.21229]
Evans L. Search Engine Watch. 2009. Can social media be measured? URL: http://searchenginewatch.com/sew/news/
2064649/can-social-media-be-measured [accessed 2019-04-15] [WebCite Cache ID 76M8Inzlm]
Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy(v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior changeinterventions. Ann Behav Med 2013 Aug;46(1):81-95. [doi: 10.1007/s12160-013-9486-6] [Medline: 23512568]
Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and Promoting Effective Engagement
With Digital Behavior Change Interventions. Am J Prev Med 2016 Nov;51(5):833-842. [doi: 10.1016/j.amepre.2016.06.015][Medline: 27745683]
Gruman J, Rovner MH, French ME, Jeffress D, Sofaer S, Shaller D, et al. From patient education to patient engagement:
implications for the field of patient education. Patient Educ Couns 2010 Mar;78(3):350-356. [doi: 10.1016/j.pec.2010.02.002][Medline: 20202780]
West R, Michie S. A guide to development and evaluation of digital behavior interventions in healthcare. Surrey, United
Kingdom: Silverback Publishing; 2016.
Mohr DC, Schueller SM, Montague E, Burns MN, Rashidi P. The behavioral intervention technology model: an integrated
conceptual and technological framework for eHealth and mHealth interventions. J Med Internet Res 2014 Jun;16(6):e146[FREE Full text] [doi: 10.2196/jmir.3077] [Medline: 24905070]
Short C, Rebar A, Plotnikoff R, Vandelanotte C. Designing engaging online behavior change interventions: A proposedmodel of user engagement. The European Health Psychologist 2015;17(1):32-38.
Holden RJ, Karsh B. The technology acceptance model: its past and its future in health care. J Biomed Inform 2010
Feb;43(1):159-172 [FREE Full text] [doi: 10.1016/j.jbi.2009.07.002] [Medline: 19615467]
Sallis J, Owen N, Fisher E. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Healthbehavior: Theory, research, and practice. San Francisco, CA: Jossey-Bass; 2015:43-64.
Jackson C, Eliasson L, Barber N, Weinman J. Applying COM-B to medication adherence: a suggested framework forresearch and interventions. European Health Psychologist 2014;16(1):7-17.
Bandura A. Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall; 1986.
Eldredge L, Markham C, Ruiter R, Kok G, Fernandez M, Parcel G. Planning health promotion programs: an intervention
mapping approach. Hoboken, NJ: John Wiley & Sons; 2016.
Guerra LGGC, Silva MTA. Learning processes and the neural analysis of conditioning. Psychology & Neuroscience 2010
Jul;3(2):195-208. [doi: 10.3922/j.psns.2010.2.009]
Bandura A, Walters R. Social learning theory. Englewood Cliffs, NJ: Prentice-hall; 1977.
Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.
American Psychologist 2000;55(1):68-78. [doi: 10.1037/0003-066X.55.1.68]
Larsen KR, Michie S, Hekler EB, Gibson B, Spruijt-Metz D, Ahern D, et al. Behavior change interventions: the potentialof ontologies for advancing science and practice. J Behav Med 2016 Aug 01. [doi: 10.1007/s10865-016-9768-0] [Medline:
27481101]
Bolívar J, Daponte A, Rodríguez M, Sánchez JJ. The Influence of Individual, Social and Physical Environment Factors on
Physical Activity in the Adult Population in Andalusia, Spain. IJERPH 2010 Jan 05;7(1):60-77. [doi: 10.3390/ijerph7010060]
Shiffman S, Stone AA, Hufford MR. Ecological Momentary Assessment. Annu Rev Clin Psychol 2008 Apr;4(1):1-32.
[doi: 10.1146/annurev.clinpsy.3.022806.091415]
Suh H, Shahriaree N, Hekler E, Kientz J. Developing and validating the user burden scale: A tool for assessing user burdenin computing systems. 2016 May 12 Presented at: Proceedings of the CHI conference on human factors in computing
systems; 2016; San Jose, CA p. A-3999. [doi: 10.1145/2858036.2858448]

https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

Cole-Lewis et al

JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
28.
29.

30.

Cole-Lewis et al

Nunnally J. Psychometric Theory. New York City, NY: McGraw-Hill; 1978.
Anderson JC, Gerbing DW. Predicting the performance of measures in a confirmatory factor analysis with a pretest
assessment of their substantive validities. Journal of Applied Psychology 1991 Oct;76(5):732-740. [doi:10.1037/0021-9010.76.5.732]
Teyhen DS, Aldag M, Centola D, Edinborough E, Ghannadian JD, Haught A, et al. Incentives to Create and Sustain Healthy
Behaviors: Technology Solutions and Research Needs. Military Medicine 2014 Dec;179(12):1419-1431. [doi:10.7205/MILMED-D-14-00111]

Abbreviations
BCT: behavior change technique
COM-B: Capability, Opportunity, Motivation - Behavior
DBCI: digital behavior change intervention
EMA: ecological momentary assessment
TAM: technology acceptance model
UI: user interaction

Edited by P Santana-Mancilla; submitted 19.03.19; peer-reviewed by V Guzman, J Contreras-Castillo; comments to author 18.04.19;
revised version received 23.07.19; accepted 14.08.19; published 10.10.19
Please cite as:
Cole-Lewis H, Ezeanochie N, Turgiss J
Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions
JMIR Form Res 2019;3(4):e14052
URL: https://formative.jmir.org/2019/4/e14052
doi: 10.2196/14052
PMID: 31603427©Heather Cole-Lewis, Nnamdi Ezeanochie, Jennifer Turgiss. Originally published in JMIR Formative Research(http://formative.jmir.org), 10.10.2019. This is an open-access article distributed under the terms of the Creative Commons
Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction
in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete
bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license
information must be included.

https://formative.jmir.org/2019/4/e14052

XSL• FO
RenderX

JMIR Form Res 2019 | vol. 3 | iss. 4 | e14052 | p. 10(page number not for citation purposes)