PINPOINT: Gaming Technology to Engage Adolescent Sickle Cell Patients in Precision Pain Phase II
Phase: NA
Conditions: Sickle Cell Disease, Sickle Cell Anemia in Children, Sickle Cell Thalassemia, Sickle Cell SC Disease
Sponsor: Klein Buendel, Inc.

 
^ƚƵĚǇWƌŽƚŽĐŽůĂŶĚ^ƚĂƟƐƟĐĂůŶĂůǇƐŝƐWůĂŶ 
 
PINPOINT: GAMING TECHNOLOGY TO ENGAGE 
ADOLESCENT SICKLE CELL PATIENTS IN  
PRECISION PAIN PHASE II  
(5R44MD010746) 
 
NCT04579926 
 
09/17/2018 
 
 
 
 
 
 
 
 

Study Protocol 
2͘^W/&//D^ 
Sickle cell disease (SCD) is the most common inherited blood disorder in the U.S.,1 and affects 70,000 to 
100,000 individuals primarily African Americans and Hispanics.2-4 SCD is characterized by red blood cells (RBCs) 
which contain an abnormal type of hemoglobin, hemoglobin S (HbS). This abnormality causes RBCs to become 
sickle-shaped which results in the RBCs having difficulty passing through small blood vessels. This blockage 
reduces blood flow (i.e., vaso-occlusion) to tissues and vital organs3,5,6 causing serious and life-threatening 
comorbidities such as organ failure, acute chest syndrome, and stroke.7,8 Pain is the hallmark symptom.9 Acute 
painful crises are the most common cause of healthcare utilization for people with SCD, and pain severity and 
frequency vary widely by individual.10 SCD painful crises can last anywhere from a couple of hours to a couple of 
weeks and should be treated according to the severity of pain that is being experienced.5 
Research suggests that painful episodes have phases, and can be categorized into different or specific types 
of pain.11-13 Treating a crisis in the early stages can avoid peak pain when it is more difficult to treat,14,15 and 
reduce the onset of irreversible tissue damage.11 Thus, pain assessment methods which can recognize pain early 
as well as the type of pain experienced could potentially improve treatment outcomes. In previous studies, pain 
assessment has included self-reported diaries16,17 and a variety of pain-scoring questionnaires18-22, and more 
recently a mobile application (“app”) to record and assess SCD-related symptoms in adults.23 However, there 
remains a lack of valid and systematic methods of pain assessment for children resulting in a significant barrier 
to effective treatment of SCD pain.24 Data suggests that children can be accurate self-reporters of pain as long as 
the measures are designed for children’s developmental stage and cognitive abilities and can be rapidly 
completed with high engagement.4 Therefore, novel approaches that are age-appropriate, and are easy and 
appealing to children may result in reduced barriers to pain assessment and better overall health outcomes. 
Children are “growing up digital”. Eighty-four (84%) of adolescents, ages 12-17, own a smartphone.25 Over 
half of teens (58%) have access to a tablet device.26 Data show that interactive technology can enhance 
children’s learning and cognitive and social abilities,27 and have sustainable effects.28-30 “Gamification” is the use 
of game thinking and mechanics in non-game contexts.31-33 Gamified applications use core game aspects such as 
fun, play, transparency, design and challenge, and apply them to real-world objectives.34 Few products have 
been created applying gamification to clinical healthcare despite experts’ suggestions that it would be highly 
innovative and effective.35   
 
In our Phase I SBIR project, Pinpoint: Gaming Technology to Engage Adolescent Sickle Cell Patients in 
Precision Pain Management, we created and usability tested a prototype pain identification tablet app for teens 
with SCD. We created the Pain Assessment Tool (PAT) which consisted of transforming the pediatric Patient 
Reported Outcomes Measurement Information System (PROMIS) measures from a conventional paper format 
into a media-rich, interactive game response format. In addition, interactive modules focused on identifying, 
understanding, and communicating pain symptoms were initially prototyped. The technology applied in the 
Phase I study was specifically selected for fit with the target audience (i.e., teens); app-based and use of 
gamification approaches. The additional technological innovation proposed for the Phase II Pinpoint project is 
also well-suited to the targeted market (i.e., tablet and smartphone mobile-responsiveness). We have 
demonstrated the technical merit, feasibility, and market potential of the Pinpoint app in our Phase I project. 
The purpose of this Phase II application is to (1) continue research and development of the Pinpoint program 
with an emphasis on developing a scalable product for commercialization and (2) testing improvement in 
communicating about pain by the target users (teens with SCD). Specifically, this project will develop an app for 
use on smartphones and tablets for 13-17 year olds with SCD that will describe, categorize, and report pain 
experienced by adolescents with SCD, educate teens with SCD on effective strategies for communicating their 
pain to their adult care team (e.g., parents, family members, care providers), and educate teens on SCD disease 
care management. All aspects of the app will incorporate gaming principles which will engage adolescents and 
improve pain specification. The following ^ƉĞĐŝĨŝĐŝŵƐ will be achieved: 

 
/Dϭ͗ Develop a fully programmed, interactive Pinpoint app consisting of 5 modules addressing pain 
identification and communication. 
 
/DϮ͗ Conduct usability testing of the Pinpoint app to evaluate the user interface, ease of use, and 
perceived barriers in order to optimize the app prior to large scale evaluation (n=14).  
 
/Dϯ͗ Test the full Pinpoint app with 13-17 year olds with SCD (n=100) using a within-subjects design to 
evaluate changes in (1) knowledge acquisition for communicating about pain and types of SCD pain; (2) the Pain 
Assessment Tool (gamified adaptation of the PROMIS measures)36; (3) SCD general knowledge and self-efficacy; 
(4) family cohesion; and (5) app usage. 
Significance and innovation are high. Pinpoint will be the first app to identify and translate specific pain 
types for SCD into a gamified app using applied gamification principles while also instructing teens on how to 
effectively communicate their pain to others. Pinpoint also has strong clinical significance. It has the potential to 
be utilized in tandem with any pediatric care setting and can potentially reduce barriers to obtaining children’s 
self-report by using a medium that they are comfortable with and enjoy, while providing important pain 
information that affect treatment choices and overall health.  
ϯ͘Z^Z,^dZd'z 
ϯ͘^ŝŐŶŝĨŝĐĂŶĐĞ 
ϯ͘ϭ͘Sickle cell disease is a prevalent disorder with significance public health impact. Sickle cell disease 
(SCD) is a genetic disorder that affects red blood cells (RBCs), and is characterized by hemolysis,7 vaso-
occlusion,7,8 and inflammation.9 It is the most common genetic disorder in the U.S.3 occurring in approximately 1 
out of 365 African Americans and 1 out of 16,300 Hispanic Americans.37  In SCD, hemoglobin molecules become 
dehydrated, deoxygenated, and “sticky” causing the molecules to clump and form long fibers.3,5,6 This results in 
the RBCs changing into a crescent (“sickle”) shape making the cells less flexible.2,6 When the sickled cells block 
the flow of blood (vaso-occlusion8), a person with SCD may experience painful episodes (vaso-occlusive painful 
episode [VOEs]), and can experience permanent damage to tissue and vital organs,3,5,6 acute chest syndrome,5,36 
and stroke.1,2,5,6,8,9,38 SCD complications can be serious and have a significant impact upon well-being and quality 
of life.5  
ϯ͘Ϯ͘Sickle cell disease pain is the hallmark symptom of SCD, yet approaches to assess specific pain types 
are lacking. SCD pain is multifactorial,16,38 and can be triggered by identifiable (e.g., fatigue/overexertion,9,14 cold 
temperatures,14 dehydration,14 emotional stress39) or unidentifiable factors.8,11,40 In children with SCD, VOEs are 
the most common clinical problem5 and the leading cause of hospital admissions.3,8,11,39-41 Children are more 
likely to have longer hospital stays41,42 and report significantly more severe pain.42 41VOEs become more frequent 
as children age,18,43-45 and if left untreated, can result in morbidity and mortality.46 The literature supports the 
concept that VOEs have different phases (e.g., pre-pain, pain start, pain acceleration, pain decline, 
etc.).12,14,15,44,47 In addition, there are also three types of SCD pain: (1) acute recurrent painful crises; (2) chronic 
pain syndromes (i.e., pain lasting 3+ months); and (3) neuropathic pain.11 The National Heart, Lung and Blood 
Institute (NHLBI) also identifies three types of SCD pain; acute, chronic, and mixed.13  Effective SCD pain 
management requires both rapid and comprehensive assessments.13,39 Rapid assessment focuses on the isolated 
pain event, pain intensity, and relief. Comprehensive assessment is used for chronic pain or as a follow-up to an 
acute pain episode and yields a multidimensional treatment plan involving the patient, family, and the 
healthcare team.13 41Research suggests that a simple measure which takes into account the child’s 
developmental stage and cognitive abilities should be used when assessing pain.4 Previous studies have assessed 
pain using self-reported diaries16,1748,49and validated measures.18-22,18-2241,50 
Barriers to effective care for SCD pain in children include: clinicians who are concerned about the validity of 
a child’s pain score; fear of negative effects from analgesic use; mistaking pain relief-seeking behaviors for drug-
seeking behaviors51; and conflicting perceptions between patients, families, and medical professionals.4 A lack of 
valid, consistent and systematic methods for child pain assessment is also a barrier.24,46While the validity of child 
pain self-report is a commonly reported barrier, most studies involving children and adolescents have shown 
that they are able to sufficiently report and describe their pain.4,52,53   
ϯ͘ϯ͘Effective communication can improve psychosocial and health outcomes, yet child-centered 
communication training is remarkably uncommon. There are a number of sequelae such as negative self-

esteem, mood and anxiety disturbances, academic and peer relationship difficulties, and worry about the future 
which can impact a child with SCD.22,54-56 Research suggests that cognitive-behavioral interventions can improve 
psychosocial and behavioral outcomes including communication skill building, yet the majority of interventions 
are parent-centered and do not focus on communication and social skills development with the child.57,58 
Children are capable communicators; by age 8, children can conceptualize complex health terms if asked in a 
format that boosts comprehension.59-61 However children are typically relegated to the role of being the receiver 
rather than the expressor (i.e., sender) of information. A child’s participation in health decisions can increase 
internal locus of control, decision-making ability, and adherence while lack of involvement can have adverse 
consequences (e.g., increased fear and anxiety, poor self-esteem, etc.62-64). Evidence suggests that regardless of 
age, the child’s own perspective should be directly solicited and be given equal attention and merit as the 
parent’s report, and that it is important for a child to participate in their health communication.65-67 Thus, 
creating educational and skill-building interventions that use age-specific formats and designs (e.g., 
gamification) while reducing resource barriers (e.g., technology use) could improve skill acquisition for effective 
communication between SCD youths, their parents, and medical team.  
ϯ͘4. Children are high consumers of technology, but innovative approaches for assessing SCD pain are 
lacking. Children prefer using technology to learn and “gaming” is a popular approach. 28,68-70 Among 12-17 year 
olds, 97% (99% of boys, 94% of girls) play computer, web, portable, or console games.71 The gaming industry is 
growing, and its principles are being applied to education and health. “Gamification” is a concept that translates 
games into a mechanism to promote desired behaviors; using game design in non-game contexts.31,33,72-74 
Despite its popularity, gamified approaches to assess child health are lacking especially for pain assessment. A 
literature review revealed only seven studies that have examined mHealth-based pain management approaches 
for adolescents with chronic pain.17,75-79 One pilot study examined change in pain with the use of an app and 
found that pain-related outcomes appeared to improve; still, the remaining studies do suggest mHealth tools for 
assessing pain are feasible76,80,81 and can yield high compliance.75,80 Notably, only one study utilized a SCD 
population80 and none included medical professionals within the context of program delivery. While there is lack 
of existent literature for mHealth and gaming formats for pain management among adolescents, interactive 
games can be especially effective in delivering health-related messages to youth.82 The interactive games for 
health literature among youths has shown video games can improve self-efficacy; stimulate health discussions 
with friends, family and clinical team; encourage seeking support and advice; and can emphasize behavior 
acquisition via experiential learning rather than focusing on the learning of facts.82 Interactive games can 
provide information about causes, treatments, and self-care options,82 and can improve self-care and reduced 
emergency clinical utilization.82  
ϯ͘ϱ. Use of well-established theories elevates significance. User-centered Design (UCD)83,84; Self-
Determination Theory (SDT)85,86; and Social Cognitive Theory (SCT)87 will be used. UCD solicits user input through 
iterative cycles, with adjustments made based on feedback,88 resulting in greater user experiences and more 
reliable and effective results.31,89 UCD will be used for the development of the Pinpoint games and preferred 
user-interface (UI). SDT involves factors that either facilitate or impede motivation and is a commonly used 
approach in health promotion.90-94 In SCT, behavior is determined by environmental (social support, modeling) 
and cognitive factors (self-efficacy, self-concept). SCT has shown effectiveness in assessing and increasing 
health, as well as using virtual models.95-99 
 
ϯ͘/ŶŶŽǀĂƚŝŽŶ 
Pinpoint is a highly innovative project using gamification to create an adolescent SCD pain assessment tool 
based on the Patient Reported Outcomes Measurement Information System (PROMIS) measures.36 Pinpoint will 
use theoretically-driven approaches, content from a best-selling book for teens with SCD,100 and expert opinion 
and apply them to emerging technology to produce an industry-leading, innovative app that can impact the 
standards for SCD pain assessment and disease education. Pinpoint will be the first app to develop an 
assessment of specific SCD pain types into a gaming format and instruct children on how to effectively 
communicate about their pain to their adult care team (e.g., parents, teachers, healthcare professionals). It will 
also be the first translation of the pediatric PROMIS measures into a gaming format. Gamification for healthcare 

is an underutilized approach with high market potential. Games have been employed successfully for learning 
and with health promotion, yet gamified assessments and educational tools remain relatively undeveloped.  
Pinpoint applies “gaming” to pain assessment and disease education. Another innovation will be the potential 
clinical relevance of Pinpoint. Child pain assessment is complex and challenging, and there is data to indicate 
that VOE pain is often undertreated. 101 Research suggests discrepancies between SCD patient behavior and pain 
score which can lead to misunderstanding between providers and patients.102-104 Pain severity is often verbally 
reported, yet evidence suggests that the majority of health professionals (up to 85% of doctors and 60% of 
nurses) believe verbal report is not a reliable indicator of pain intensity and inadequate pain assessment tools 
are one of the greatest barriers to SCD care.102-104 This suggests that doctors are interested and may find clinical 
utility for a SCD pain tool that can be used by patients in their daily lives. Use of a SCD pain tool that bridges the 
gap between daily disease management and ongoing clinical care is likely to be utilized if it is (a) easy to use and 
interpret by all target audiences (patient, parent, provider); and (b) increases patient knowledge.105,106 Patients 
require a tool that is easy-to-use and relatable; while medical staff require measures that are low burden, easily 
integrated into their practice, and facilitate care. The primary focus of Pinpoint in Phase II will be to educate the 
patient on how to better understand and communicate their pain to others, not create an app to be integrated 
into existing Electronic Health Records (EHR). Yet, the app will address many of the barriers of clinical uptake, 
because it can be easily incorporated into healthcare practice. Specifically, (1) gamification can increase 
engagement and autonomy of the patient to be more proactive in their care. Pinpoint will be built “by kids and 
for kids” serving as formative research for further integrating user-centered health assessments into clinical 
practice; and (2) the app will provide a feature that allows feedback from the patient to the provider to improve 
patient/provider communication. 
ϯ͘ϭ͘DĂƌŬĞƚWŽƚĞŶƚŝĂů͘Pinpoint will have strong market potential because it (a) reaches an untouched and 
underserved population (SCD teens) with a program that will not only match their technology preferences but 
also provide them with critical health information using an interactive educational technology design; (b) has 
strong market potential with over 250 SCD centers in the U.S. who could purchase and provide the app to 
patients107; (c) can improve patient quality of care and information sharing; (d) is being developed with the 
leading SCD book publisher; and (e) will be a resource to physicians who act as proactive pain managers for 
patients.  A multi-pronged marketing strategy is being proposed (see ŽŵŵĞƌĐŝĂůŝǌĂƚŝŽŶWůĂŶ).  
 
ϯ͘ƉƉƌŽĂĐŚ 
 
ϯ͘ϭ͘WŚĂƐĞ/KǀĞƌǀŝĞǁĂŶĚKƵƚĐŽŵĞƐ. The Phase I project examined the feasibility and acceptability of a 
gamified tablet application (Pinpoint) intended to encourage teens (aged 13-17) to talk about and assess their 
SCD pain. The prototype was programmed for use on iOS and Android tablets. The Phase I project had three 
main goals: (1) consult with SCD experts to establish the core components of an app that could be used with 
children, parents, and providers to establish better pain identification and communication; (2) develop a 
functional prototype app with target audience input and feedback; and (3) assess the feasibility and 
acceptability of the app to guide future Phase II development. Phase I used multi-method formative research to 
guide app design and content prior to conducting lab usability testing of the app prototype to determine if it 
would be (1) feasible and (2) a product that teens with SCD would use. Phase I had 8 specific aims: (1) work with 
an Expert Advisory Board (EAB) of SCD experts to develop a new pain assessment tool (PAT) to engage 
adolescent SCD patients, improve pain specification by patients, and improve pain management by clinicians; (2) 
conduct cognitive interviews with 13-17 year old SCD patients to guide and refine the PAT and app concepts; (3) 
conduct focus groups with 13-17 year old SCD patients to guide and refine development of app content, design, 
and aesthetics to fully develop a functioning prototype; (4) program the PAT and UI elements into a tablet app 
prototype; (5) conduct usability testing with 13-17 year old SCD patients to assess functionality, navigation, and 
satisfaction; (6) conduct cognitive interviews with medical providers to provide input on app content, perceived 
barriers to use, and its potential for clinical use and integration to inform future implementation; (7) author a 
comprehensive Specifications Document to guide content development, UI design, and programming of the full-
scale Pinpoint app in a future Phase II project; and (8) analyze data from cognitive interviews, focus groups, and 
usability testing sessions for the EAB to use along with the Specification Document to determine the feasibility 

of producing and evaluating a full-scale Pinpoint app in a future Phase II project. Deliverables were met and 
feasibility was confirmed by the EAB after a review of the Phase I findings. Phase I outcomes provided the 
necessary framework and data to guide Phase II goals: full-scale development of the app and testing the app in 
an experimental trial in Phase II.  
 
ǆĞĐƵƚŝǀĞ^ƵŵŵĂƌǇŽĨWŚĂƐĞ/&ŝŶĚŝŶŐƐ͗ The prototype consisted of a gamified Pain Assessment Tool (PAT), 
gamified vocabulary game, interactive body scanner reflection activity, interactive educational self-disclosure 
activity, and excerpts from the Hope and Destiny Jr. book. Teens participated in cognitive interviews, focus 
groups, and usability testing (N=23). Of these participants, the average age was 14.7 + 1.2 years, 43.48% were 
female, and 75% were diagnosed with Hb SS (n=23). The System Usability Scale (SUS),108 a validated tool for 
assessing the usability and acceptability of technological products, served as the primary outcome.  The SUS 
score (n=12) was 86.6 (68% is “above average”), suggesting a very high level of acceptability and usability among 
users. Healthcare providers were interviewed on the app’s acceptability and potential function within the clinical 
practice (n=5). Providers gave the Pinpoint favorable scores with all responses yielding above average scores on 
a five-point Likert scale.  
 
ϯ͘ϭĂ͘WŚĂƐĞ/ŝŵϭ͘'ĂƚŚĞƌŝŶƉƵƚĂŶĚĞǀĂůƵĂƚŝǀĞĨĞĞĚďĂĐŬĨƌŽŵĂŶǆƉĞƌƚĚǀŝƐŽƌǇŽĂƌĚ;). The EAB 
included experts in SCD, psychometrics, and gamification principles: (1) Deepika Darbari, MD, Children’s 
National Medical Center in Washington, DC (CNMC-DC); (2) Allan Platt, PA-C, MMSc, author of Hope and Destiny 
and former Program Coordinator at the Georgia Comprehensive Sickle Cell Center at Grady Health System; (3) 
Lewis Li-Yen Hsu, MD, PhD, author of Hope and Destiny, Jr. and Professor and Director of Pediatric Sickle Cell at 
the Children’s Hospital at the University of Illinois; (4) Julie Panepinto, MD, MSPH, Professor and Director of the 
Center for Clinical Effectiveness Research for the Children’s Research Institute, Medical College of Wisconsin. 
She is a psychometrics expert, and has developed and translated several measures among SCD youth; (5) Talal 
Ziad Ali, PhD, MSN, RN (Wayne University) has studied the use of video games as a tool for decreasing pain in 
adolescents with SCD; and (6) Lauren Patrick, M.S., President of NetHealth, an expert in game theory application 
to healthcare.  
 
Dr. Hilton Hudson (Co-I, and President of Hilton Publishing Company) facilitated the EAB which met formally 
four times. During the initial meetings, the EAB vetted the merit of establishing a new assessment tool versus 
using existing validated pain measures. The EAB unanimously agreed to use the PROMIS measures (see ϯ͘ϳ͘Ϯ͘) 
due to the clinical utility of the measures and that the pediatric PROMIS measures were currently being used in 
two clinical studies with pediatric SCD patients. The EAB agreed that additional pain identification and qualifiers 
were still needed since the PROMIS measures did not have targeted specificity for SCD pain. Thus, an interactive 
vocabulary game was brainstormed as an app activity. The objective of this activity was to have the user 
recognize and define vocabulary related to pain and its descriptors which could then be used to better 
categorize and specify pain type. Using a modified Delphi technique with 3 rounds and a card sorting series to 
finalize the grouping of the words, the EAB established a list of adjectives that were included in a word bank 
within the app that would allow app users to best describe the pain they are experiencing within particular areas 
of their bodies. Using the modified Delphi method, the EAB narrowed down the list from 137 words to a final 
word bank of 115 which were programmed as content in the interactive vocabulary game. The EAB also rank 
ordered Hope and Destiny, Jr. book content (see ϯ͘ϰ͘ϯĂ) based on Phase I learning objectives. Twenty-two 
educational topics were identified, and 3 were programmed. The EAB provided valuable feedback in the 
formative stages of the app’s development and concluded that Phase I results supported developing an 
enhanced version of the app and evaluating it for effectiveness in Phase II research.                                                                           
 
ϯ͘ϭď͘WŚĂƐĞ/ŝŵϮ͘CŽŐŶŝƚŝǀĞIŶƚĞƌǀŝĞǁƐ ǁŝƚŚdĞĞŶƐ;ƉƉĞŶĚŝǆ). Formative research approaches 
suggest that a low number of participants can be sufficient to guide formative development (i.e., 4-8 
participants) if data saturation is present.109  Our cognitive interviews (n=4) revealed a high level of agreement 
and thematic saturation was met. Teens were asked about their pain and previous assessments used, their 
communication strategies for discussing pain, use of apps and games for learning about SCD including preferred 
components and tracking features, and educational content that would be beneficial. Interviews revealed that 
teens experienced pain on average 2.5 times/week; an average of 1.5 times being “very bad”. All teens reported 
telling their parents about pain and thought it was extremely important to discuss pain with their provider (avg. 

score 9.25 out of 10). Desired app features were: (1) accurate pain scale; (2) information on new therapies and 
mediations; (3) how to stay healthy; and (4) general SCD information. Desired SCD-specific topics for educational 
content were: (1) how to stay healthy; (2) SCD studies and progress on cure; (3) gender-specific coping 
strategies; and (4) information on the different types of SCD.   
ϯ͘ϭĐ͘WŚĂƐĞ/ŝŵϯ. &ŽĐƵƐ'ƌŽƵƉƐ ;ƉƉĞŶĚŝǆ). Two focus groups of 13-17 year olds with SCD informed 
prototype development. Focus groups ascertained whether the Pinpoint concept was easily understood and 
interesting, and identified whether the initially programmed features would be essential for a successful 
prototype. Focus groups were conducted in Charlotte, North Carolina as to broaden the geographical catchment 
of potential app users (per suggestion of the EAB). A total of 7 teens participated (3 females; 4 males). Formative 
data approaches suggest that 5-8 participants is sufficient to guide technology 
development,109 and FG 
studies among SCD 
teens have successfully used 
comparable sample 
sizes.110 Feedback influenced 
app modifications 
that were programmed into 
Prototype 2 and 
evaluated in lab usability. 
The focus groups 
evaluated the app’s design, 
content and 
features, and overall 
engagement. Five 
activities (see ϯ͘ϭĚ) were 
reviewed by the 
teens: (1) the Pain 
Assessment Tool; 
(2) the Self-Disclosure 
Activity; (3) 
Vocabulary Game; (4) Body 
Scanner; and (5) 
Pain Drawing Tool. The 
prototype was very 
well received; they thought it was understandable, logical, 
and easy to use. 
Participants reported high likeability for the gaming 
aspects including 
the sounds and visual effects, interactivity, and clarity of 
what was expected 
of the user. They found the content educational, useful, 
and fun. Average 
rating of the app was 7.8 out of 10. Input on what to modify or include for a fully programmed app included: 
optional narration, reducing the number of assessment questions, the option to show healthcare providers 
some of their responses but not all (non-pain items), daily push notifications to use the app, be able to send pain 
records to healthcare provider, include an eating and drinking log, blog and/or group chat feature, and 
interactive avatar character to guide the user through the app.  
ϯ͘ϭĚ͘WŚĂƐĞ/ŝŵϰ͘ƉƉWƌŽŐƌĂŵŵŝŶŐ ĂŶĚŽŵƉŽŶĞŶƚƐ. The prototype was built on a full stack web 
application using HTML/JavaScript as its main interface. AngularJS v1.6.3 was used as the front-end framework. 
Animations were performed using GreenSock Animation Platform and CSS 3 transitions. Look and feel was 
designed with Adobe Illustrator and developed using SASS and CSS 3.0 with HTML elements plus SVG, PNG, and 
JPG images and graphics. PHP was utilized to create spreadsheets populated with data collected from the front 
end. The application is hosted on a remote server provided by KB with the 
following infrastructure profile: Internet information must travel through KB’s 
Sonicwall TZ600 firewall. Traffic then move to KB’s web server, a Dell Power Edge 
T-430 with 128 GB of RAM, 4.5 TB available disk space on a RAID 5 redundant 
drive system using Windows HyperV Operating System. The virtual web server 
uses Windows 2012 IIS Web server software. Servers are kept in a locked room 
on site.  
There are 3 app modules with 8 activities in the Phase I prototype: (Module 
1) >ĞĂƌŶďŽƵƚŽŵŵƵŶŝĐĂƚŝŽŶ which includes (a) Body Scanner Reflection 
activity, (b) Self-Disclosure activity, and (c) Vocabulary Game; (Module 2) WĂŝŶ 
ƐƐĞƐƐŵĞŶƚdŽŽů which includes (a) interactive Pain Survey and (b) Draw My 
Pain activity; and (Module 3) Hope and Destiny Jr. content which includes (a) 
Lifestyle Tips, (b) “Stories”, and (c) Glossary. The Body Scanner Reflection is a 
guided meditation that segments the body into 6 sections and encourages 

reflection of how the user’s body feels. The Vocabulary Game has users match as many SCD pain related words 
with their definitions in a 2-minute timed game. The Self-Disclosure Activity teaches the different types of 
communication levels using a Johari window and allows users to test their knowledge with example characters.  
The Pain Assessment Tool is an interactive survey containing 56 questions about user pain. The Pain Drawing 
Tool requires users to draw their pain on a full body diagram. Lifestyle Tips provide educational content to help 
prevent pain episodes. The “Stories” provide motivational excerpts from teens with SCD, and the Glossary is a 
list of medical terms and definitions. 
 
ϯ͘ϭe͘WŚĂƐĞ/ŝŵϱ. >Ăď hƐĂďŝůŝƚǇdĞƐƚ;ƉƉĞŶĚŝx C). Remote usability testing was conducted with 12 users 
recruited from Sickle Cell Disease Association of America (SCDAA) chapters across the U.S. to test navigability, 
detect problems, identify problem severity, and develop recovery strategies. In remote usability testing, 
researchers are in a different location than the users, but both parties are on a web conference call and are 
sharing screens. Researchers can see what participants are doing, and they can communicate with each other in 
real time. The researcher provides test participants with activities to complete while using a design or interface, 
and users typically think out loud as they work on the tasks. The researcher observes the user as they work 
through the tasks, and ask questions for clarification or to gather more data. Two key benefits of remote 
usability testing are that the participant is in their natural environment and gives more realistic insight than 
when in the lab, and it allows feedback from participants in geographically dispersed areas.111-113 Evidence 
supports the use of remote usability testing as a viable approach to gather high-quality user experience 
feedback.114,115 The Pinpoint sessions occurred over the web using Zoom videoconferencing and were 
approximately 1 hour in length. Participants were asked to complete defined tasks within the app, including 
completion of the PAT. Participants were also asked to complete the System Usability Scale (SUS).108 The 
sessions were audio recorded via Zoom videoconferencing software. Similar to the cognitive interviews and 
focus groups, participants had to be between the age of 13 and 17 years, be diagnosed with SCD, able to speak 
and read English, and have parental consent to participate and assent to participate. Compensation was $30. 
Five participants were female (42%), and the average age was 14.6 + 1.0. Severity in usability concern/errors are 
categorized into levels: Level 1 is the most severe and represents issues that make it impossible for the user to 
finish a task; Level 2 are programming issues that do not function ideally, but do not impede users from finishing 
a task; Level 3 are programming issues or tasks that do not cause a high level of concern. Level 1 severities were 
noted with the first usability tester, however, they were quickly debugged and did not impede with future 
usability testers. Level 2 and 3 issues can be easily addressed in future iterations of Pinpoint development. In 
addition to navigation and technical items, most testers reported high acceptability of visual design, 
functionality, and content. Overall, users thought the program would be helpful in identifying and reporting 
their pain. After usability testing, the testers completed the System Usability Scale (SUS),108 a validated tool for 
assessing the usability and acceptability of technology-based products.116,117 A SUS score of 68% is considered 
above average108 and served as the criterion for measuring Pinpoint’s usability. The total SUS score for the 
program was 86.6% suggesting a high level of acceptability and usability.  
 
ϯ͘ϭĨ͘WŚĂƐĞ/ŝŵϲ͘ŽŐŶŝƚŝǀĞ/ŶƚĞƌǀŝĞǁƐǁŝƚŚWƌŽǀŝĚĞƌƐ;ƉƉĞŶĚŝx D). Interviews were conducted with 
medical providers of SCD patients (n=5) to review the app to determine whether they found it appropriate, 
acceptable, useful, and the content appropriate. They were also asked to respond to their interest in using 
Pinpoint, and its clinic meaningfulness. Eligibility criteria were: medical provider to SCD patients, over 18, able to 
speak and read English, and consent to participate. Compensation was $50. Similar procedures for remote app 
evaluation as presented above in the Lab Usability Testing section were used. The providers reported that the 
body scanner was an effective tool to guide patients through body self-reflection (avg. score of 4 out of 5); 
patients would learn to better self-disclose (avg. score of 4 out of 5); patients would learn new words to better 
describe their emotions and pain (avg. score of 4.25 out of 5); and patients would be more likely to assess their 
pain because of the app (avg. score of 4.66 out of 5). They thought clinic implementation could be accomplished, 
and liked the idea of summary pain reports provided via email, patient portal, or app. 

ϯ͘ϭŐ͘WŚĂƐĞ/ŝŵϳ. 
SƉĞĐŝĨŝĐĂƚŝŽŶƐ
ŽĐƵŵĞŶƚ. A site 
map is a visual 
representation of the 
content areas of the 
technology program 
and how they relate 
to one another. The 
Specifications 
Document provides 
general program 
information, 
program progression, 
system architecture, 
and user dialogues. A 
site map (&ŝŐƵƌĞ4) 
and Specification 
Document has been completed.  
    ϯ͘ϭŚ͘WŚĂƐĞ/ŝŵϴ͘/ŵƉŽƌƚĂŶĐĞŽĨ ZĞƐƵůƚƐ͘Phase I findings show: (1) SCD teens are interested in an app 
that helps them communicate about their pain; (2) KB can create an app with gamification principles for SCD and 
communication content; and (3) participants approved of the app design and concept. The Specification 
Document and EAB feedback indicate it is possible to develop a full-scale app in Phase II and market potential 
exists.  
 
ϯ͘ϭŝ͘ǀŝĚĞŶĐĞĨŽƌ^ĐŝĞŶƚŝĨŝĐWƌĞŵŝƐĞ͘SCD is a prevalent genetic disorder with significant public health 
impact. Pain is its hallmark symptom and accurate pain reporting is critical to effective care. High quality 
research has demonstrated that children can be accurate self-reporters of pain. Yet, barriers to effective 
communication between SCD youths and their care team (parents, providers, etc.) still exist. Children are high 
utilizers of technology; gaming approaches are not only popular but have demonstrated positive health behavior 
effects in the mHealth literature. Despite the encouraging evidence of “games for health”, the area of 
gamification for assessing pain, especially SCD pain, is remarkably underdeveloped despite promising results. 
Our proposal addresses gaps in the current literature, but is founded in strong preliminary evidence. Our Phase I 
research used a multi-method approach with input from key stakeholders and experts in the field. Using a user-
centered design approach and social cognitive theory, we produced a functional prototype that was rated high 
for aesthetics and usability suggesting acceptability and feasibility of the product. Our Phase I outcomes provide 
strong preliminary evidence for moving forward with a full-scale production of the app.   
 
ϯ͘Ϯ͘WƌŽũĞĐƚKǀĞƌǀŝĞǁ͘The Phase II project will be 
conducted over 24 months (&ŝŐƵƌĞϱ). Pinpoint will be fully 
developed in Months 1-9 and evaluated for behavior 
change (i.e., increased communication skills) in Months 10-
24 among a sample of teens with SCD aged 13-17 years. 
Pinpoint will be a web-based app that provides pain 
identification and assessment, and disease education. 
Using the user-centered design (UCD) approach from Phase 
I, usability testing will be conducted on newly programmed 
features (n=14) and will guide final app development. A within-subjects design will enroll teens with SCD using a 
time series approach. Teens will be recruited from across the U.S. through SCD support groups and networks 
(see >ĞƚƚĞƌƐŽĨ^ƵƉƉŽƌƚ), and will use the app for 12 weeks. The primary outcome will be improvements in 

communication skills regarding SCD. Teens (n=100) will complete baseline, 6-week, and 12-week assessments 
(&ŝŐƵƌĞϳ). 
 
ϯ͘ϯ͘/ŶǀĞƐƚŝŐĂƚŝǀĞdĞĂŵ͘This project will be led by Valerie Myers, Ph.D. Dr. Myers is a Senior Scientist and 
licensed psychologist. She studies technological approaches to promote health among at-risk populations in 
real-world settings. Dr. Myers has served as a PI or Co-I on several NIH- and state-funded grants. Formerly, she 
was a Research Associate at Children’s Hospital of Philadelphia (CHOP) and conducted psychosocial assessments 
and treatment with patients and families with SCD through the Sickle Cell Disease Study. Dr. Myers will be joined 
by Hilton Hudson, MD (Co-I) and Ms. Mary Buller, MA (Co-I). Dr. Hudson specializes in general cardiac and 
thoracic surgery, and has extensive medical and administrative experience at a number of agencies. In addition 
to practicing medicine, Dr. Hudson is President and CEO of Hilton Publishing Co. and EMIC LLC (Evidence-based 
Medical Information Company). He will Chair the EAB. Dr. Hudson is well-suited for his position as EAB Chair due 
to his SCD experience, and his experience with disseminating point-of-care tools within the medical domain. Ms. 
Buller has a master’s degree in communication and will help guide instructional design and creation of new app 
content. Ms. Buller has been PI or Co-I on a number of SBIR and STTR Phase I and II projects including projects 
targeting under-served populations and utilizing gamification techniques. KB’s Creative Team will lead the user-
interface design and programming of the app. KB has extensive experience with formative research, prototype 
testing, and full-scale production on multiple technology interfaces and has developed a successful iterative, 
user-centered design process (based on the ADDIE Model,118-120 MDA Framework,121,122 and AGILE programming 
techniques123,124). KB has developed health behavior technologies including app and game development for 
children and teens, and successfully recruited and implemented technologies in pediatric clinical settings on a 
number of federally-funded projects.  
 
ϯ͘4. dĞĐŚŶŽůŽŐǇĞƐŝŐŶĂŶĚĞǀĞůŽƉŵĞŶƚ;/DϭͿ͗The AIM 1 goal is to develop a fully programmed, 
interactive Pinpoint app consisting of 5 modules addressing pain identification and communication.  
ϯ͘ϰ͘ϭǆƉĞƌƚĚǀŝƐŽƌǇŽĂƌĚ;Ϳ'ƵŝĚĂŶĐĞ͘ The EAB was instrumental in the Phase I project, and will 
continue to provide guidance on newly developed modules for Phase II (see ϯ͘ϰ͘ϯĐ). Drs. Darbari and Hsu will 
continue to serve as well as Mr. Platt. A new member (Ms. Lisa Rose, M.Ed) will join the EAB. Ms. Rose is the 
Executive Director at HOPE for SCD, an organization that provides medical education to families and partners 
with local SCD community organizations. HOPE for SCD has published a book, Sickle What?,125 which is a part of 
the Hope and Destiny Sickle Cell Patient Education Series. In the event that an area of expertise is needed, we 
will recruit an additional member. The EAB will meet a total of 6 times during the project to provide guidance on 
content development and recruitment (see >ĞƚƚĞƌƐŽĨ^ƵƉƉŽƌƚ). Dr. Hsu and Mr. Platt (both authors of Hope and 
Destiny) will also serve as Other Significant Contributors (OSCs) on the project to provide more in-depth and 
time-intensive content development and recruitment assistance. 
ϯ͘ϰ͘ϮWĂƌĞŶƚĚǀŝƐŽƌǇŽĂƌĚ;WͿ'ƵŝĚĂŶĐĞ. During our Phase I focus group and usability testing sessions, 
parents expressed a strong interest in being a part of the app development as well as providing a parent 
perspective on communication strategies with their children. In response, a Parent Advisory Board (PAB) will be 
established. The PAB will provide early and frequent user-focused feedback on app design, and will meet at total 
of 6 times during the project period. Five parents have provided >ĞƚƚĞƌƐŽĨ^ƵƉƉŽƌƚ, and another five have 
expressed interest in participating. The inclusion of teens, SCD experts, and parents assures that we will have 
multiple stakeholder feedback across all phases of our project.  
 
ϯ͘ϰ͘ϯ WƌŽƚŽƚǇƉĞWƌŽŐƌĂŵŵŝŶŐ͘  
 
ϯ͘ϰ͘ϯa dŚĞŽƌĞƚŝĐĂů&ŽƵŶĚĂƚŝŽŶ͕ŽŶƚĞŶƚdƌĂŶƐůĂƚŝŽŶ͕ĂŶĚ'ĂŵŝĨŝĐĂƚŝŽŶ͘  Pinpoint content development 
and instructional design will be led by Dr. Myers with assistance from Ms. Buller, Dr. Hudson, EAB, and PAB. 
Pinpoint content will be drawn from the best-selling book, Hope and Destiny, Jr.100 This book is the only 
comprehensive, culturally-sensitive book on SCD for children available on the market. Translation of content will 
be guided by social cognitive theory (SCT).87 Two primary book content areas will be targeted for translation: (1) 
the educational content which describes SCD and its co-occurring conditions geared towards the child’s 
perspective; and (2) the “Stories from Kids Like You” sections. The “Stories” are testimonials from youth with 

SCD that are presented throughout the book and are real-life narratives regarding the different educational 
components presented. Book content will be categorized into SCT theoretical components, e.g., Modeling; 
Outcome Expectancies; Self-efficacy; and Identification. Gaming theory elements and components will be then 
be applied to target the SCT assumptions of: (1) people learn from others (e.g., cognitive and observational 
modeling presented through direct, symbolic, and synthesized models); (2) self-regulated behavior is essential to 
the learning process (e.g., goal-setting, self-observation, self-assessment, self-reinforcement); and (3) learning 
involves the interaction and reciprocal causation of multiple factors (e.g., beliefs and expectations; 
reinforcement). In Phase I, a sampling of the educational content and “Stories” were programmed into the app. 
Phase II will program additional book content.   
 
Gamification has three primary elements: (1) ǇŶĂŵŝĐƐ(“the grammar of the game”) is the game content 
which can include such things as emotion and progression through levels to improve learning (this targets the 
SCT components of  modeling, reciprocal causation, and outcome expectancies); (2) DĞĐŚĂŶŝĐƐ (“the verbs of 
the game”) drives the actions of the game forward and include such 
things as rewards, feedback, and challenges (this targets the SCT 
components of self-regulation, outcome expectancies, self-efficacy, and 
identification); and (3) ŽŵƉŽŶĞŶƚƐ (“the nouns of the game”) are the 
real instances of game dynamics and mechanics and can include a 
number of options such as badges, leaderboards, levels, points, 
achievements, avatars, content unlocking, quests, social recognition, 
teams, and virtual goods (this targets the SCT components of modeling, 
outcome expectancies, self-efficacy, self-regulation, identification, and 
reciprocity). &ŝŐƵƌĞϲ provides an example of the use of an avatar and 
how it could be applied to SCD knowledge acquisition using SCT 
components.  An avatar (i.e., “learning buddy”) was a requested addition 
for Phase II by the EAB and the teens.         
 
ϯ͘ϰ͘ϯďƉƉĞǀĞůŽƉŵĞŶƚ. The design process will be informed by: (1) the ADDIE Model,118-120 a user 
interface and instructional design approach; (2) the Mechanics, Dynamics, and Aesthetics (MDA) 
framework,121,122 a dynamic game development process that incorporates the perspective of the researcher, 
developer, and user; and (3) AGILE programming, a collaborative and incremental programming 
methodology.123,124 This multi-model approach is systematic, flexible, and cost-effective. It ensures that all 
design and programming elements are aligned and stakeholders’ inputs are incorporated throughout the multi-
staged development. Design elements such as ƐƉĂĐĞ(colors, sounds, visual space), ĐŽŵƉŽŶĞŶƚƐ (characters, 
objects), and ŵĞĐŚĂŶŝĐƐ (actions) will be determined for Pinpoint’s features (i.e., assessment, activities, games, 
tools). App user-interface (UI) design and programming will be conducted by the KB Creative Team using similar 
application approaches used in Phase I (see ϯ͘ϭĚ).  
ϯ͘ϰ͘ϯĐ͘ ĞǀĞůŽƉŵĞŶƚ ŽĨ EĞǁ CŽŵƉŽŶĞŶƚƐ͘ The Pinpoint app will focus on helping SCD teens build practical 
and constructive communication skills and providing them with a platform for practice and feedback. Examples 
of communication content areas include nonverbal communication, self-disclosure, expression of emotions, 
trust, mutual respect, empathy, conflict management, and coping strategies. Consistent with research on 
mHealth strategies for adolescents,126-130 the proposed design emphasizes gaming as a teaching strategy. These 
interactive components will be the centerpiece of the app. Outcomes of each game will be used to ensure 
uptake of information (e.g., users must achieve a targeted score or level to demonstrate comprehension to 
receive a reward), thus providing feedback to participants early and often. To reinforce learning and bolster 
adherence, “badges” (i.e., visual icons), will be awarded intermittently. Badges are highly effective gaming tools 
used to encourage user engagement and build game “loyalty”.131-134  
Details of components programmed in Phase I were provided in ^ĞĐƚŝŽŶ ϯ͘ϭĚ and &ŝŐƵƌĞ 4. Three modules 
were programmed in Phase I and will be refined and expanded in Phase II. Two additional modules will be built 
from scratch: Daily Pain Journaling and Resources for Parents, Providers, and Teens; each module will contain 
multiple sections. Based on Phase I feedback, the app will use an avatar to help guide and provide modeling and 
instruction to the user. Teens will be asked to set pain management goals (e.g., methods to maintain health, 

commitment to reporting pain episodes, etc.). The Learn ĂďŽƵƚ ŽŵŵƵŶŝĐĂƚŝŽŶ module will be expanded: 
additional Johari window games for self-disclosure will be created and a new section (Practicing Disclosure) will 
include an interactive game to promote expressive communication. Additional games targeted towards 
improving expression of emotions and increasing trust will be included in this section. ĂŝůǇ WĂŝŶ Journal will be 
a new module which focuses on daily self-monitoring of pain. This module will include interactions with the 
avatar to build skills for mutual respect, empathy, conflict management, and coping strategies. The WĂŝŶ 
ƐƐĞƐƐŵĞŶƚ dŽŽů module will be expanded to include a historical log of the pain assessments which will be 
housed in the app database and can be “pushed” to others (e.g., parents, providers) in an email format. The 
Hope & Destiny, Jr. module will include additional book content. The Extra ZĞƐŽƵƌĐĞƐ module will contain 
parent, provider, and teen information such as current trends and research in SCD. For all modules, Pinpoint will 
use media-rich activities and games with videos and other engaging tools.   
ϯ͘ϰ͘ϰ hƐĂďŝůŝƚǇdĞƐƚŝŶŐ;/DϮͿ͗ The AIM 2 goal is to conduct usability testing of the Pinpoint app to 
evaluate the user interface, ease of use, and perceived barriers in order to optimize the app prior to large scale 
evaluation (n=14).  The purpose of usability testing is to validate the app’s design and to identify and correct 
problems prior to testing the app in a more systematic trial.  Research suggests that at least 10 users are needed 
to capture 95% of most usability problems.135-137 To identify any problems that impact >15% of  users at a 90% 
chance of detection (log(1-.90)/log(1-.15)), 14 teens are needed.138,139 A parent will give consent; teens will give 
assent. Consent and assent forms and procedures will be IRB-approved. Eligibility criteria are: a) age 13-17 
years; b) diagnosed with SCD; c) own a smartphone or tablet, and d) read and speak English. Compensation will 
be $25. We will attempt to recruit equal numbers of males and females for diversity of gender perspectives. 
Sessions will be conducted remotely in a procedure similar to Phase I (see ϯ͘ϭĞ). The users will complete 
defined tasks while being observed and complete a usability survey. They will rate their satisfaction with the app 
with regards to design, appeal, functionality, and navigation using Likert-type scales. This approach for assessing 
feasibility and appeal of technology learning has been used successfully with youths.140 Usability will be assessed 
using the System Usability Scale (SUS),141 a validated tool for assessing the usability of technology-based 
products.142,143 Project staff will make notes of issues encountered during testing.  
 
  
ϯ͘ϱ͘ǆƉĞƌŝŵĞŶƚĂůĞƐŝŐŶ;/Dϯ). AIM 3 will test the full Pinpoint app with 13-17 year olds with SCD (n=100) 
using a within-subjects times series design over 12 weeks to evaluate changes in (1) knowledge acquisition for 
communicating about pain and types of SCD pain; (2) the Pain Assessment Tool; (3) SCD general knowledge and 
self-efficacy, (4) family cohesion, and (5) app usage. The primary outcome measure (knowledge acquisition for 
communicating about pain and types of pain) will be developed. Change in knowledge acquisition for 
communication relative to baseline was chosen as the primary outcome because Phase I participants strongly 
expressed that improving communication skills and understanding their pain were the primary reasons why 
teens would use the app. In addition to teens being recruited, their parent will also be recruited to complete a 
parent proxy version of the primary outcome measure. 
 
ϯϱ͘ϭ͘dƌŝĂůĞƐŝŐŶ ;ŶсϭϬϬͿ͘ A within-subjects (repeated measures) design will be used. A number of designs 
were considered. A comparative-effectiveness trial was rejected since there is no existing app geared for SCD 
teens that could act as an appropriate comparator. A randomized controlled design was contemplated, and we 
considered a number of possible control conditions (e.g., no treatment control, waitlist control, etc.). However, 
we rejected the RCT concept due to resource demands for obtaining a large sample size for sufficient power 
(i.e., time and cost relative to a 24-month Phase II timeline), and input from the EAB and parents. Specifically, 
the EAB and parents believe parents would not be interested in participating if the possibility of a control 
condition was present. The purpose of a Phase II grant is to continue the research and development efforts 
initiated in Phase I and prepare it for commercialization. Our primary goals are to fully build the app and assess 
for individual improvements in communication and pain identification over time. Thus, a within-subjects design 
was chosen. The advantages of the within-subjects design are increased power with smaller sample size, and a 
reduction in error variance associated with individual differences. While the lack of a control group is a 
limitation, it was necessitated to overcome challenges in recruiting SCD teens experienced in Phase I. We will 
devote project resources to recruiting and retaining a sample in the treated group (i.e., receives Pinpoint app) 

that is sufficient to detect pre-post changes in primary and secondary outcomes rather spend resources on 
securing a sample of which half will not be treated. We will also avoid having parents decline to participate due 
to the chance their teen will not receive the app. We have included two design aspects to control threats to 
validity144: (1) we have a 6-week interim measure to create a time-series on the assessment of change; and (2) 
we will include a measure of communication and knowledge related to influenza as a nonequivalent dependent 
variable, which we do not expect will change pre to post as a result of the app despite it being subject to the 
same history, maturation, or regression threats as our primary and secondary measures. We will also include 
demographics, especially age of the teen and parental age and education, as covariates in the analysis as 
statistical controls against threats such as maturation. 
 
A description of the app was provided in ^ĞĐƚŝŽŶƐϯ͘ϭĚ and ϯ͘ϰ͘ϯĐ. Participants will download the app 
following the procedures outlined in ^ĞĐƚŝŽŶϯ͘ϳ͘ϭ. Project staff will provide users with a one-on-one app 
orientation and how to use its unique features via videoconferencing. Daily notifications will engage users in 
daily app use and provide up-to-date information on users’ progress. Participant responses will also be used to 
evaluate additional or on-going support of users’ needs. To protect privacy and to ensure that the participant is 
the person completing the app activities, KB will uniquely identify the user’s smartphone/tablet based off the 
device’s “hardware footprint”. The app will upload all app activity data to the KB secured web server database. 
 
ϯ͘ϲ͘dĂƌŐĞƚWŽƉƵůĂƚŝŽŶ͘ There are two populations: teens with SCD and their parent. Inclusion criteria for teens 
are: (a) age 13-17 years; (b) diagnosed with SCD; (c) own a smartphone or tablet, and (d) read and speak English. 
Parents will be recruited to complete a parent proxy version of the primary outcome measure (child’s 
knowledge acquisition; see ϯ͘ϳ͘Ϯ), but will not use the app. Parent inclusion criteria are: (a) parent/guardian of 
eligible child and (b) able to speak and read English.  
 
ϯ͘ϳ. dƌŝĂůŽŶĚƵĐƚ͘ 
 
ϯ͘ϳ͘ϭ͘ZĞĐƌƵŝƚŵĞŶƚĂŶĚsŝƐŝƚWƌŽĐĞĚƵƌĞƐ͘ 
Recruitment and Screening: Potential study participants will be recruited through nationwide SCD support 
groups, SCD networks, social media ads, and existing research participant databases. Recruitment was a 
challenge during Phase I due to IRB delays at the study hospital. Thus, we reached out to multiple national 
organizations to assist with Phase I recruitment. During Phase I, a number of organizations expressed interest in 
being a recruitment center for Phase II. We have four committed organizations and four others have expressed 
commitment via phone or email communication (see >ĞƚƚĞƌƐŽĨ^ƵƉƉŽƌƚ). In addition, we have the commitment 
of the EAB and PAB to assist with recruitment.  
Participants will be sent an email describing the study and a link to an eligibility screening survey via KB’s 
secure server. The online screener will take 5 minutes. Phone screening will be used as needed. Participants will 
receive an email informing them of eligibility. Eligible participants will be scheduled for an orientation/baseline 
visit which will be conducted via videoconferencing; a reminder email and text message will be sent 3 days prior 
to the assessment (similar reminders will be sent for 6- and 12-week visits).  
 
Orientation and baseline/pre-test assessment: In a one-on-one orientation session to be conducted via 
videoconferencing, participants will have the study explained and provide written consent (parent) and assent 
(child). Immediately following, the teen will complete pre-test surveys online using their personal smartphone or 
tablet device (see ϯ͘ϴ) which should take about 30 minutes to complete. Parents will complete an online survey 
(proxy measure of the primary outcome). After the surveys, teens will be assisted with app download, provided 
instructions on how to use the app, and encouraged to use the app daily.  
 
6-week interim assessment: Participants will complete interim assessments of initial changes (i.e., teens to 
complete all measures; parent to complete the single proxy measure assessing their child’s knowledge 
acquisition). An invitation email and text messages containing a link to the surveys along with their unique 
password (PIN number) will be sent. Participants will complete the web-based surveys using either their tablet 
or smartphone; no video meeting will be required. Participants who do not complete the surveys within 3 days 
will receive an email and text reminder (and 3 more reminders at 3-day intervals if still not completed).  

 
12-week post-test assessment: Participants will complete a post-test assessment via videoconference. They 
will meet individually with a study staff to complete the online surveys and a brief post-test interview.  
ϯ͘ϳ͘Ϯ͘KƵƚĐŽŵĞƐ͗ĂƐĞůŝŶĞ͕/ŶƚĞƌŝŵ͕ĂŶĚWŽƐƚ-ƚĞƐƚDĞĂƐƵƌĞƐ͘The baseline, 6-week interim, and 12-week 
post-test surveys will use a measure created by the investigative team and EAB as the primary outcome (i.e., 
knowledge acquisition for communicating about pain and types of pain). Secondary outcomes will be the app 
Pain Assessment Tool (PAT) and validated measures of SCD knowledge, disease self-efficacy, and family 
cohesion. Surveys should take 30 minutes to complete (see &ŝŐƵƌĞ ϳ and ƉƉĞŶĚŝǆE).  
 
KŶŽǁůĞĚŐĞĐƋƵŝƐŝƚŝŽŶ;ŽŵŵƵŶŝĐĂƚŝŶŐĂďŽƵƚƉĂŝŶĂŶĚ^ƉĂŝŶŬŶŽǁůĞĚŐĞ)͗ The primary outcome 
measure will be developed for this project in collaboration with the EAB, and psychometric properties for will be 
evaluated (e.g., content-validity, criterion-related validity, construct validity, and internal consistency) using a 3-
step scale development process: (1) domain identification and item generation; (2) content expert validation, 
and (3) pilot test.145 The investigative team will generate items using a sorting process that encompasses 
themes/construct elements noted in the communication (nonverbal communication, self-disclosure, expression 
of emotions, trust, mutual respect, empathy, conflict management, and coping strategies) and pain 
identification literature (Step 1). Items will then be subjected to expert validation by the EAB (Step 2),145 who will 
identify and delete theoretically incoherent items thus ensuring that the items in the instrument demonstrate 
content validity. Specifically, the EAB will sort the items by “how much” each item measures its targeted 
construct. The comprehensiveness of the entire instrument will be evaluated by identifying items which the EAB 
perceives to be incongruent with its nominated domain. The EAB will then be asked to identify the clarity of 
each item’s construction and wording to ensure that there are no ambiguous and poorly written items. Lawshe’s 
Content Validity Ratio (CVR)146 will be utilized to assess the content expert judgment; a minimum CVR value of 
0.49 will be required for retention in the scale.146 For Step 3, usability testers (see AIM 2) will be asked to 
complete the measure to guide preliminary psychometric evidence and allow for revision of the instrument prior 
to launching in the full study. Reporting of internal consistency reliability is a necessary part of scale 
development.147 Cronbach’s coefficient alpha of .70 is the suggested minimum.148 All the factors’ 
intercorrelations will be calculated and should be less than 1.00 to be conceptually distinct. To allow precision in 
evaluating the new measure, confirmatory factor analyses (CFA) will be conducted. Exploratory factor analysis 
(EFA) may be required depending on the results of the CFA. The overall purpose of exploratory and confirmatory 
factor analyses will be to ensure the stability of the factor structure. 
 
WĂŝŶƐƐĞƐƐŵĞŶƚdŽŽů;WdͿ͗ The PROMIS® (Patient-Reported Outcomes Measurement Information System) 
is a set of self-report measures that evaluates and monitors physical, mental, and social health in adults and 
children. It can be used with the general population and individuals living with chronic conditions.36 PROMIS was 
designed to enhance communication between clinicians and patients in diverse research and clinical settings. It 
was also created to be relevant across multiple health conditions for the assessment of symptoms and functions. 
The PROMIS Pediatric multisite initiative created pediatric self-report scales measuring the unidimensional 
health attributes of depressive symptoms, anxiety, anger, pain interference, peer relationships, fatigue, physical 
functioning - mobility, physical functioning - upper extremity, and asthma.149 The PROMIS pediatric measures 
have been validated in a variety of illnesses experienced by children including cancer, kidney disease, and 
obesity.150 Only one study has examined PROMIS in SCD children and preliminary results show that the measures 
are feasible and valid.50 To date, there have been no studies that have gamified the PROMIS measures. In Phase 
I, the following PROMIS questionnaire banks were gamified and usability tested: Family Relationships, Physical 
Activity, Physical Stress Experiences, Strength Impact, Pain Behavior, Pain Quality-Sensory, and Pain Quality-
Affective. These same domains will be refined in Phase II and serve as secondary outcomes in the trial. In 
addition, we will gamify the Pain Interference, Psychological Stress Experiences, and Peer Relationships domains 
(per EAB recommendation in Phase I).  
 
^<ŶŽǁůĞĚŐĞ͗ SCD knowledge will be measured using two scales. The SCD Transition Program 
questionnaire151 is a reliable 10-item measure developed from the Stepping Up to Adult Care Program.152 
Content areas covered include the etiology, clinical manifestations, and medical management of SCD. Higher 
scores indicate better knowledge of SCD and its treatment. The SCD Transition Knowledge Questionnaire153 is a 
25-item multiple-choice measure that assesses knowledge of SCD relevant to preparation for transition to adult 

services. It is designed to assess SCD knowledge in seven areas: (1) pathophysiology, (2) genetics, (3) physical 
symptoms, (4) treatment, (5) self-care, (6) psychosocial and developmental issues, and (7) health care delivery. 
It has good psychometric properties.153,154 Higher scores represent greater knowledge.  
 
Sickle Cell Self-efficacy Scale: This instrument, used to assess self-efficacy in adolescents with SCD,155 is 
comprised of nine questions measuring participants' perceptions of their ability to function on a day-to-day basis 
and to manage SCD symptoms (e.g., pain). The instrument is reliable and valid for assessing adolescents' self-
efficacy for engaging successfully in day-to-day activities despite having SCD.  
 
&ĂŵŝůǇŽŚĞƐŝŽŶ͗ Family cohesion will be assessed by the single item Family Cohesion scale from the Child 
Health Questionnaire (CHQ)156, a pediatric quality of life (QOL) survey that has been normed for children age 5 to 
18 years including those with chronic diseases. The CHQ consists of 28 items that assess 14 different concepts 
regarding child and family physical and psychosocial well-being. There is evidence of good reliability and validity 
of this measure156 including the single-item Family Cohesion 
question.157 This scale, which rates a family’s ability to get 
along with one another, uses a single item (“In general, how 
would you rate your family's ability to get along with one 
another?”) rated on a 5-point scale. The PAT is assessing QOL, 
therefore to reduce duplication of information and study 
burden only the Family Cohesion item will be administered.  
 
&ůƵ/YŵĞĂƐƵƌĞ͗ This is a 10-item quiz developed by 
the Centers for Disease Control to assess general knowledge 
about influenza and vaccination. It will serve as a 
nonequivalent dependent variable. 
 
ƉƉhƐĂŐĞ͗ App use will be monitored using Web Trends software; tracking macro and micro level data. It 
will record how users move through the program including the time spent on each activity, response latency to 
individual items, and scores achieved on games. Summaries of these data (means, ranges) will be reviewed to 
understand usage patterns. Teens will be sent “push notifications” to encourage app use. Users will evaluate the 
app on the ^ǇƐƚĞŵhƐĂďŝůŝƚǇ^ĐĂůĞ at the 6- and 12-week assessment.142  
 
ϯ͘ϳ͘ϯ͘ĚŚĞƌĞŶĐĞ͕ƌŽƉŽƵƚƐ͕ĂŶĚWĂƌƚŝĐŝƉĂŶƚŽŵƉĞŶƐĂƚŝŽŶ͘ 
 
Retention, Tracking, and Attrition: Although participants can withdraw at any time, steps will be used to 
minimize dropouts to ensure high internal validity. Participant progression will be tracked. Staff will use 
behavioral reinforcement strategies known to be effective in the behavior change literature158-160 and be trained 
to develop rapport with participants by communicating expectations for participants and staff. Personalizing this 
relationship will enable participants to be more direct about problems, providing a strong foundation for 
problem-solving. While drop-out rates vary among youths with SCD,57,161 we conservatively planned for 20% 
attrition. Our method of tracking and multiple contacts will help us retain some of these participants for intent-
to-treat purposes, despite their discontinued app use. Participants will provide contact information for one 
individual who can always contact them and update contact information at each assessment. If they do not 
respond to email/text reminders, we will follow-up by phone. Finally, we will create a dummy coded variable for 
each respondent where 1=completers and 2= drop outs and test for associations with key variables in our 
analytic models. If bias is identified, we will control for it by adding it as a covariate. 
 
Compensation: All participants (n=100) will be compensated to incentivize and improve study retention. A 
$20 gift card will be given to the teens after each study assessment (baseline, 6-week, and 12-week) for a total 
compensation of $60. Parents will be provided a $10 gift card after each study assessment for a total of $30.  
 
App Use:  Implementation of technology-based interventions can be challenging due to participants 
forgetting, fatiguing, or believing they have learned all they need to know. In order to bolster compliance with 
using either of the assigned app, weekly reminders to use the app will be sent to participants by text message. 
  
ϯ͘ϳ͘4͘^ĂĨĞƚǇŽŶƐŝĚĞƌĂƚŝŽŶƐ͘Participant safety is a priority. This study will include a Data and Safety 
Monitoring Plan (DSMP). The team will monitor progress, discuss concerns, and take corrective actions. Dr. 
Myers (PI) is a licensed psychologist and will assemble a group of experts including at least one licensed mental 
health professional, physician, and biostatistician. Participants who express emotional or physical distress will be 

referred to their treating physician immediately. Based on our experience and demonstrated ability to conduct 
this type of research, we are confident we can achieve safe intervention delivery.  
 
ϯ͘ϴ͘ĂƚĂŽůůĞĐƚŝŽŶ͕DĂŶĂŐĞŵĞŶƚ͕ĂŶĚ^ĞĐƵƌŝƚǇ͘KB has used online data collection successfully in a number 
of studies. Study consents and pretests and posttests will be collected online via participants’ personal 
smartphone or tablet. Surveys will be programmed using QuestionPro software on KB’s secure web server to 
ensure a structured approach and reduce errors/missing values. All procedures will be approved by KB’s IRB.  
KB has considerable experience maintaining large databases and ensuring high-quality data. Ms. Lucia Liu, 
KB’s Biostatistical Manager, will supervise data management. All data will be uploaded in real-time using KB’s 
secured network; data will be de-identified and stored in KB’s secured database. Established KB protocols for 
quality assurance (valid ranges; internal consistency checks, etc.) will ensure high quality data. The KB data team 
will compile master data files. Survey and app usage data will be linked. Missing data should be limited since 
data is recorded electronically. Staff will identify missing responses and check that entries were not intentionally 
skipped. Analyses will be done on data collected. While unlikely, data may not be missing at random (NMAR). KB 
will assign all lost responses to extreme category and perform multiple imputation procedures that can handle 
various amounts of missing data and use covariates and propensity scores.  
Data security: KB’s server farm has 5 Dell PowerEdge servers with two 3.2 Ghz Xeon processors, four with 32 
gigabytes of RAM and one with 64 gigabytes, and three 1 Terabyte hard drives that operate off a hardware 
RAID5 system, connected to a local area network (LAN) running the Windows 2008 or 2012 operating system 
and to the Internet through a 20-megabyte Fiber connection. KB programmers monitor and maintain all 
programs and databases. Sensitive information is protected by a hardware firewall (Cisco ASA 5505) and each 
server has its own native Windows security software. All KB servers are connected via 1 Gigabit high-speed 
switched network, ensuring high-speed transfer between machines. All networked computers are protected 
from viruses by Sunbelt Vipre Enterprise. Nightly backup of each computer protects against data loss. 
 
͘ϭϬ͘ǀŝĚĞŶĐĞĨŽƌ^ĐŝĞŶƚŝĨŝĐZŝŐŽƌ and WŽƚĞŶƚŝĂů>ŝŵŝƚĂƚŝŽŶƐ͘The study has numerous strengths, few limitations, 
and is scientifically rigorous. The study is theory-driven (i.e., User-centered Design and Social Cognitive theories). 
Formative data will incorporate multiple stakeholder input. The programming approach will be multi-model and 
evidence-based (i.e., ADDIE Model, MDA framework, AGILE programming) resulting in a systematic, flexible, and 
cost-effective approach. Sample sizes are sufficient and supported by the literature. Effectiveness will be 
evaluated in a within-subjects design which allows for sufficient power (i.e., statistical inference with fewer 
subjects), efficiency (i.e., quicker trial completion), and longitudinal analysis (i.e., individual changes over time). 
The study team and environment are strong. Potential limitations are: (1) difficulty recruiting; (2) sources of 
bias; and (3) lack of control group. We have tried to reduce their impact. We will work closely with the EAB and 
PAB to maximize recruitment. Our primary outcome measure is new, but we are using a rigorous approach to 
scale development. Self-report instruments are open to bias, however the use of validated measures are a 
strength. Limitations of no control group are lessened via the use of a time-series design for outcome changes 
and use of a nonequivalent dependent variable.  
 
ϱ͘WZKdd/KEK&,hDE^h:d^ 
 
ϱ͘ZŝƐŬƐƚŽ^ƵďũĞĐƚƐ 
 
ϱ͘ϭ͘,ƵŵĂŶ ^ƵďũĞĐƚƐ/ŶǀŽůǀĞŵĞŶƚ͕ŚĂƌĂĐƚĞƌŝƐƚŝĐƐ͕ĂŶĚĞƐŝŐŶ͘The proposed research involving 
human subjects will be conducted by Klein Buendel, Inc. (KB: FWA No. 00003715).  
The subject population for this study will be teens with Sickle cell disease (SCD) and their parent. 
Participants will consist of both males and females and participants will be of African American, Hispanic, 
and white race/ethnicity. All participants will be asked to complete a demographics form. The 
demographics are detailed in Section 7 and the Targeted Enrollment Table.    
Human Subjects will include: 
1. Usability Testing Participants (n = 14) will evaluate the user interface, ease of use, and perceived 
barriers in order to optimize the app prior to large scale evaluation. The purpose of usability 

testing is to validate the app’s design and to identify and correct problems prior to the 
evaluation period.  A parent will give consent; teens will give assent. Consent and assent forms 
and procedures will be IRB-approved. Eligibility criteria are: a) age 13-17 years; b) diagnosed 
with SCD; c) own a smartphone or tablet, and d) read and speak English. Compensation will be 
$25.  
2. Evaluation Period Participants (n=100) will test the full Pinpoint app using a within-subjects pre-
post design over 12 weeks to evaluate changes in (1) knowledge acquisition for communicating 
about pain and types of SCD pain; (2) the Pain Assessment Tool; (3) SCD general knowledge and 
self-efficacy; (4) family cohesion; and (5) app usage. A parent will give consent; teens will give 
assent. Consent and assent forms and procedures will be IRB-approved. Eligibility criteria are: a) 
age 13-17 years; b) diagnosed with SCD; c) own a smartphone or tablet, and d) read and speak 
English. Participants will use the app for 12 weeks after a one-on-one orientation to the app 
where teens will be assisted with app download, provided instructions on how to use the app, 
and encouraged to use the app daily. Participants will complete a baseline, a 6-week, and a 12-
week assessment. Participants will be compensated to incentivize and improve study retention. 
A $20 gift card will be given to the teens after each study assessment (baseline, 6-week, and 12-
week) for a total compensation of $60. 
3. Parent Online Survey participants (n=100) will complete an online survey at the baseline, 6-week, 
and 12-week assessment. Parent inclusion criteria are: (a) parent/guardian of eligible child and 
(b) able to speak and read English. Parents will be provided a $10 gift card after each study 
assessment for a total of $30. 
 
ϱ͘Ϯ͘^ŽƵƌĐĞƐŽĨZĞƐĞĂƌĐŚDĂƚĞƌŝĂů͘Data from the usability testing will be collected in the form of 
handwritten notes, audio tapes, computer files, transcriptions and questionnaires. Transcripts of audiotaped 
discussions will be analyzed via Atlas-ti© content and thematic analysis software to identify consistent and 
recurrent content and themes in response to discussions. Data gathered from the usability testing will be 
summarized in notes by research staff to identify usability problems. 
 
Participants (teens and parents) will complete a baseline, a 6-week, and a 12-week assessment during the 
evaluation period. Data gathered from the questionnaires and surveys will be collected online, using KB’s online 
survey software, QuestionPro®. App usage will be monitored using Web Trends software. All data will be 
obtained specifically for research purposes. Access to all data will be limited to the Principal Investigator (PI), Co-
Investigators, specific project staff, and consultants. Data will be kept confidential and anonymous and will be 
kept in locked files and secure servers.  
 
ϱ͘ϯ͘WŽƚĞŶƚŝĂů ZŝƐŬƐ͘There are minimal risks associated with participation in this study. The potential risks in 
this project are limited to minimal psychological risks. Participation in the usability testing, which requires children 
to answer general usability questions about the Pinpoint tablet app, has slight risks.  Participants may be hesitant 
or uncomfortable responding to questions about their opinions regarding the Pinpoint tablet app.  
 
Participation in the evaluation period and the parent online surveys has minimal risks.  Participants may be 
hesitant to answer questions related to (1) knowledge acquisition for communicating about pain and types of 
SCD pain; (2) the Pain Assessment Tool; (3) SCD general knowledge and self-efficacy, (4) family cohesion, and (5) 
app usage. All responses collected in this project will not be shared with anyone except project staff. All 
participants will be allowed to withdraw from the project at any time. 
ϱ͘ĚĞƋƵĂĐǇŽĨWƌŽƚĞĐƚŝŽŶŐĂŝŶƐƚZŝƐŬƐ 
ϱ͘ϭ͘ZĞĐƌƵŝƚŵĞŶƚĂŶĚ/ŶĨŽƌŵĞĚŽŶƐĞŶƚ͘Human Subjects oversight will be conducted by Western 
Institutional Review Board (WIRB; DHHS IRB Reg. No. 00000533).  WIRB will review and approve subject 
materials (including recruitment materials, consent forms, and survey instruments) and data management 
procedures prior to study implementation. 
Potential study participants will be recruited primarily through nationwide SCD support groups, SCD 
networks, social media ads, and existing research participant databases. In addition, we have the commitment 
of the EAB and PAB to assist with recruitment. 
For the usability testing, children participants will view the app and provide feedback about its user interface, 

ease of use, and perceived barriers. Usability testing sessions will be conducted remotely. In remote usability 
testing, researchers are in a different location than the users, but both parties are on a web conference call and 
are sharing screens. Researchers can see what participants are doing, and they can communicate with each 
other in real time. The researcher provides test participants with activities to complete while using a design or 
interface, and users typically think out loud as they work on the tasks. The researcher observes the user as they 
work through the tasks, and ask questions for clarification or to gather more data.  The users will complete 
defined tasks while being observed and complete a usability survey. They will rate their satisfaction with the app 
with regards to design, appeal, functionality, and navigation using Likert-type scales. Project staff will make 
notes of issues encountered during testing. Usability testing participants will be asked for assent and parental 
consent to participate in the usability testing. Consent/assent will be obtained using the procedures below.  
For evaluation period, participants will be sent an email describing the study and a link to a screening survey 
administered on KB’s secure server that assesses inclusion criteria and demographic characteristics. The online 
screener will take about five minutes to complete. If needed, phone recruitment and screening will also be used. 
After being screened, participants will receive an email to inform them of their eligibility status. Eligible 
participants will be scheduled for an orientation and baseline assessment, which will be conducted via 
videoconferencing. Participants will have the study explained and provide online consent (parent) and assent 
(child). After providing consent and assent, participants will attend a one-on-one orientation. During the 
orientation, teens will be assisted with app download, provided instructions on how to use the app, and 
encouraged to use the app daily. Then the baseline assessment session will be conducted via videoconferencing 
Participants will complete the (online) baseline survey(s) using their personal smartphone or tablet device, 
which should take about 30 minutes to complete.  
Participants will complete a 6-week assessment of initial changes. An invitation email and text messages 
containing a link to the survey(s) along with their unique password (PIN number) will be sent. Participants will 
complete the online survey(s) using either their tablet or smartphone; no videoconferencing will be required for 
the 6-week assessment.  
Participants will complete a 12-week post-test assessment. Participants will meet individually with study 
staff via videoconference to complete the online survey(s) and a brief post-test interview.  
Consenting/Assent procedures are as follows: 
 A parent will be present and give consent for the child during the videoconference.  Children will provide 
assent.  Eligible children will be invited to participate, consented/assented using the following procedures: 
1. Research staff will send an introductory email inviting eligible parents and children to participate and 
follow-up with emails and telephone calls to recruit them.   
2. Interested parents will go to the study website to register and complete an online enrollment form, 
providing contact information (i.e., name, address, telephone numbers [home and mobile phones], and 
email address[es]) for themselves, their child, and people who can always locate them and their child 
(i.e., a locator).   
3. Each parent and child will sign online consent form/assent form at the study website. If a parent or child 
does not consent/assent the child will not be enrolled.  Participants will give their consent/assent 
electronically and be able to download and save (or print) a copy of the consent/assent form.  The IRB-
approved consent/assent forms will describe the purpose of the project, risks and benefits, and 
selection criteria. Participants will be informed that they have a right to withdraw from the study at any 
time.  No additional data will be collected from participants who decide to withdraw; however, the 
information collected prior to withdrawal will remain in the study.  All correspondence will contain the 
PIs’ email addresses and phone numbers for invited participants to use if they have questions. 
 
Participation in all stages of this Phase II application is entirely voluntary and subjects are free to withdraw from 
the project at any time without negative consequences.  Furthermore, withdrawing from the study will not affect 
the participants’ benefits.  No additional data will be collected from participants who decide to withdraw; however, 
the information collected prior to withdrawal will remain in the study. 
 
ϱ͘Ϯ͘WƌŽƚĞĐƚŝŽŶŐĂŝŶƐƚZŝƐŬ͘To minimize potential psychological risk, usability testing facilitators/ 
interviewers will establish ground rules for discussion that includes an open, non-evaluative exchange of 

comments and ideas. IRB-approved discussion procedures will be carried out under the supervision of senior 
project staff. Participants will be told that they do not have to answer any questions that make them 
uncomfortable and can terminate their participation at any time they wish. Notes, surveys, and data will be 
accessible only by research personnel. Data forms and computer records will be labeled only with participant 
identification number. Participant names and study identification numbers will be recorded and stored 
separately from the research data files. 
 
Any survey questions and discussion transcripts will be de-identified before being submitted for analysis and 
the Research Assistant will remove any references to individual names or identifier data. There are no clinical 
data to be captured. Responses to survey questions and participation in discussions will not reasonably place the 
subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or 
reputation. The data collected on this project will be obtained with the "use of educational tests, survey 
questions and interview procedures." 
 
To protect privacy and to ensure that the participant is the person completing the app activities, KB will 
uniquely identify the user’s smartphone/tablet based off the device’s “hardware footprint”. The app will upload 
all app activity data to the KB secured web server database. 
 
All surveys and data collection forms will be IRB-approved and administered under the supervision of senior 
project staff. Participants will be told that they do not have to answer any questions which make them 
uncomfortable and can terminate their participation at any time they wish. Notes, surveys, and data will be 
accessible only by research personnel. All notes will be stored in locked cabinets; all electronic data files will be 
stored on KB’s secure network servers behind computer firewalls, with routine backup. All identifiers will be 
stored in secure files and behind computer firewalls and results will be shared in aggregate form only.   
 
The Pinpoint app and databases will be hosted on KB’s state-of-the-art web server farm. Its Live and Test 
web servers consist of Windows 2003, Internet Information Server 6.0, dual 2.8 Ghz Processors, 4 Ghz Ram, and 
120 Gb Hard Drive space. The SQL Server consists of Windows 2003, Microsoft SQL 2000, 2.8 Ghz Processor, 3 
Gb Ram and 240 Gb of hard drive space configured with RAID 5. File sharing and print serving is accomplished 
through a Windows 2003 dual 3.2 Ghz processor server with 4 Gb RAM and 750 Gb of hard drive space 
configured with RAID 5. All computers are protected by Norton Corporate Edition Antivirus software. Nightly 
backup protects against loss of data. Three T1 lines provide Internet access. Usage is monitored with Web 
Trends to ensure that the server hardware and T1 Internet service is sufficient for the number of hits (and 
upgraded as necessary). Information stored on KB’s web server is protected by a hardware firewall (Cisco PIX); 
each server has native Windows security software. All the KB servers are connected via 1 Gigabit high speed 
switched network. 
 
ϱ͘WŽƚĞŶƚŝĂůĞŶĞĨŝƚƐŽĨƚŚĞWƌŽƉŽƐĞĚZĞƐĞĂƌĐŚƚŽWĂƌƚŝĐŝƉĂŶƚƐĂŶĚKƚŚĞƌƐ 
 
The benefits of this research are substantially greater than any risks. Potential benefits for all participants 
will be the knowledge that one has helped to evaluate a web based application that engages adolescent sickle 
cell disease patients in precision pain management. 
 
 
ϱ͘/ŵƉŽƌƚĂŶĐĞŽĨƚŚĞ<ŶŽǁůĞĚŐĞƚŽďĞ'ĂŝŶĞĚ 
 
SCD is the most common inherited blood disorder in the U.S.,1 and affects 70,000 to 100,000 individuals, 
primarily African Americans and Hispanics.2-4 SCD causes serious and life-threatening comordities such as organ 
failure, acute chest syndrome, and stroke.7,8 Pain is the hallmark symptom.9  
 
Pinpoint will be the first app to identify and translate specific pain types for SCD into a gamified app using 
applied gamification principles while also instructing teens on how to effectively communicate their pain to 
others. Pinpoint also has strong clinical significance. It has the potential to be utilized in tandem with any 
pediatric care setting and can potentially reduce barriers to obtaining children’s self-report by using a medium 
that they are comfortable with and enjoy, while providing important pain information that affect treatment 
choices and overall health.  
 
ϱ&͘dƌĂŝŶŝŶŐŝŶƚŚĞƚŚŝĐĂůŽŶĚƵĐƚŽĨZĞƐĞĂƌĐŚǁŝƚŚ,ƵŵĂŶ^ƵďũĞĐƚƐ 

 
The Principal Investigator, Co-Investigators, and all study personnel have completed the required training in 
the Ethical Conduct of Research with Human Subjects. Necessary documentation will be provided to NIH prior to 
funds being awarded. 
 
ϱ'͘,/WZĞƐĞĂƌĐŚƵƚŚŽƌŝǌĂƚŝŽŶ 
      Although we will not be collecting medical information or Protected Health Information, we may email the 
app content to physicians. 
 
ϲ͘dE^&dzDKE/dKZ/E'W>E  
  
A data safety and monitoring plan will be implemented for this project, following well-established 
procedures within KB. We will not set up an independent Data Safety and Monitoring Board because this trial is 
very low risk; does not include use of therapeutics; and physical risks are not anticipated. However, the progress 
of the research will be monitored monthly by the PI. All critical events will be reviewed by the Investigator to 
ensure that project methods have not been intrusive or disruptive.  
 
The team will monitor progress, discuss concerns, problem-solve, and take corrective actions. Dr. Myers (PI) 
is a licensed psychologist in Colorado (#3630) and Louisiana (#1197) and will assemble a group of experts 
including at least one licensed mental health professional, one physician, and one biostatistician. Participants 
who express emotional or physical distress will be referred to their treating physician immediately. Based on our 
experience and demonstrated ability to conduct this type of research, we are confident we can achieve safe 
intervention delivery.  
 
The data management staff will attend regular project meetings with the PI and other project staff and 
routinely report on the quality of the data and other outcomes of this monitoring process. Annual reports on 
data safety and monitoring are submitted to WIRB.  
 
Compliance with data collection protocols will be monitored throughout data collection by the Investigators. 
The Investigators are responsible for reporting all adverse events to the IRBs. Only Grade 1 events are expected 
on this trial. Thus, adverse events will be reported annually, as required by DHHS. This report will be reviewed by 
WIRB. Any action taken by the IRBs or project investigators resulting in a temporary or permanent suspension of 
the trial will be communicated immediately by the PI to the grant program officer. 
 
ϳ͘/E>h^/KEK&tKDEED/EKZ/d/^ 
Females and minorities (African Americans and Hispanics) will be included in this research as part of the 
sample of patient participants.   
SCD rates in African Americans and Hispanics are far higher than in Whites. It is estimated that SCD affects 
to 100,000 Americans. Sickle cell disease occurs in approximately 1 in 365 African Americans; 1 in 16,300 
Hispanics, and nearly 1 in 400,000 Whites. Thus, the sample of participants in the proposed research will contain 
the following 3 racial/ethnic groups: Black, Hispanic, and White. We will attempt to recruit: 97% African 
American; 2% Hispanic (any race); and 1% White.  
 
During the recruitment process, the Project Coordinator will screen for participants using IRB-approved 
protocols for screening based on gender, race, and ethnicity. If the Project Coordinator finds that we are over-
recruiting a particular gender or racial or ethnic group, we will place them on a waiting list until we have recruited an 
adequate number of targeted participants. See Targeted Enrollment Table. 
 
ϴ͘/E>h^/KEK&,/>ZE 
 
All usability testing participants (n=14) and the evaluation period participants (n=100) will be children between 
the ages of 13 and 17.   Thus, this study is in compliance with NIH Policy for the inclusion of children. Both parental 
consent and child assent will be required. 
 
 

 
References 
 
1. 
Sickle cell disease. Genetics Home Reference Web site. Available at: 
http://ghr.nlm.nih.gov/condition/sickle-cell-disease. Published 2012.  Updated August 2012. Accessed 
March 5, 2015, 2015. 
2. 
Sickle cell disease (SCD): facts about sickle cell disease. Centers for Disease Control and Prevention Web 
site. Available at: http://www.cdc.gov/ncbddd/sicklecell/facts.html. Published 2014.  Updated January 
16, 2014. Accessed March 5, 2015, 2015. 
3. 
Brawley OW, Cornelius LJ, Edwards LR, et al. National Institutes of Health Consensus Development 
Conference Statement: hydroxyurea treatment for sickle cell disease. Annals of Internal Medicine. 
2008;148(12):932-938. 
4. 
Stinson J, Naser B. Pain management in children with sickle cell disease. Paediatr Drugs. 2003;5(4):229-
241. 
5. 
Cope A, Darbyshire PJ. Sickle cell disease, update on management. Paediatrics and Child Health. 
2013;23(11):480-485. 
6. 
Mukerji I. About sickle cell disease. Sicklecellinfo.net Web site. Available at: 
http://www.sicklecellinfo.net/index.htm. Published 2004.  Updated March 5, 2004. Accessed March 5, 
2015, 2015. 
7. 
Bhagat VM, Baviskar SR, Mudey AB, Goyal RC. Poor health related quality of life among patients of sickle 
cell disease. Indian Journal of Palliative Care. 2014;20(2):107-111.Pmc4129996 
8. 
Schnog JB, Duits AJ, Muskiet FA, ten Cate H, Rojer RA, Brandjes DP. Sickle cell disease; a general 
overview. Neth J Med. 2004;62(10):364-374. 
9. 
Ameringer S, Elswick RK, Jr., Smith W. Fatigue in adolescents and young adults with sickle cell disease: 
biological and behavioral correlates and health-related quality of life. Journal of Pediatric Oncology 
Nursing. 2014;31(1):6-17.Pmc3982311 
10. 
Bakshi N, Stinson J, Ross D, et al. Development, content validity and user review of a web-based multi-
dimensional pain diary for adolescent and young adults with sickle cell disease [published online ahead 
of print 2015/01/08]. Clinical Journal of Pain. 2015 doi: doi: 10.1097/ajp.0000000000000195 
11. 
Ballas SK, Gupta K, Adams-Graves P. Sickle cell pain: a critical reappraisal. Blood. 2012;120(18):3647-
3656. 
12. 
Ballas SK, Smith ED. Red blood cell changes during the evolution of the sickle cell painful crisis. Blood. 
1992;79(8):2154-2163. 
13. 
National Heart Lung and Blood Institute; National Institutes of Health. The management of sickle cell 
disease. 2002. 
14. 
Beyer JE, Simmons LE, Woods GM, Woods PM. A chronology of pain and comfort in children with sickle 
cell disease. Archives of Pediatrics and Adolescent Medicine. 1999;153(9):913-920. 
15. 
Jenerette CM, Brewer CA, Ataga KI. Care seeking for pain in young adults with sickle cell disease. Pain 
Management Nursing. 2014;15(1):324-330. 
16. 
Dampier C, Ely B, Brodecki D, O'Neal P. Characteristics of pain managed at home in children and 
adolescents with sickle cell disease by using diary self-reports. The Journal of Pain. 2002;3(6):461-470. 
17. 
Schatz J, Schlenz A, McClellan CB, et al. Changes in coping, pain and activity following cognitive-
behavioral training: a randomized clinical trial for pediatric sickle cell disease using smartphones. Clinical 
Journal of Pain. 2015;31(6):536-547. 
18. 
Dampier C, Lieff S, LeBeau P, et al. Health-related quality of life in children with sickle cell disease: a 
report from the Comprehensive Sickle Cell Centers Clinical Trial Consortium. Pediatric Blood & Cancer. 
2010;55(3):485-494.Pmc2911637 

19. 
Crandall M, Savedra M. Multidimensional assessment using the adolescent pediatric pain tool: a case 
report [published online ahead of print 2005/08/09]. J Spec Pediatr Nurs. 2005;10(3):115-123. doi: 
10.1111/j.1744-6155.2005.00023.x 
20. 
Franck LS, Treadwell M, Jacob E, Vichinsky E. Assessment of sickle cell pain in children and young adults 
using the Adolescent Pediatric Pain Tool. Journal of Pain and Symptom Management. 2002;23(2):114-
120. 
21. 
Lopez G, Liles DK, Knupp CL. Edmonton symptom assessment system for outpatient symptom 
monitoring of sickle cell disease. Southern Medical Journal. 2014;107(12):768-772. 
22. 
Panepinto JA, O'Mahar KM, DeBaun MR, Loberiza FR, Scott JP. Health-related quality of life in children 
with sickle cell disease: child and parent perception. British Journal of Haematology. 2005;130(3):437-
444. 
23. 
Shah N, Jonassaint J, De Castro L. Patients welcome the sickle cell disease mobile application to record 
symptoms via technology (SMART) [published online ahead of print 2014/02/12]. Hemoglobin. 
2014;38(2):99-103. doi: doi: 10.3109/03630269.2014.880716 
24. 
Beyer JE. Judging the effectiveness of analgesia for children and adolescents during vaso-occlusive 
events of sickle cell disease. Journal of Pain and Symptom Management. 2000;19(1):63-72. 
25. 
eMarketer. Teens' ownership of smartphones has surged. July 5, 2016. 
https://www.emarketer.com/Article/Teens-Ownership-of-Smartphones-Has-Surged/1014161. Accessed 
January 2, 2017. 
26. 
Growing Wireless. Kids wireless use facts. Growing Wireless Web site. Available at: 
http://www.growingwireless.com/get-the-facts/quick-facts. Accessed December 31, 2017. 
27. 
Salen K. Gaming literacies: a game design study in action. Journal of Educational Multimedia and 
Hypermedia. 2007;16(3):301-322. 
28. 
National Association for the Education of Young Children, Fred Rogers Center for Early Learning, 
Children's Media. Technology and interactive media as tools in early childhood programs serving 
children from birth through age 8. Available at: 
http://issuu.com/naeyc/docs/ps_technology_issuu_may2012?e=2112065/2087657. Published 2012. 
29. 
Moeller B, Reitzes T. Integrating technology with student-centered learning. A report to the Nellie Mae 
Education Foundation. 2011 
30. 
Noeth RJ, Volkov BB. Evaluating the effectiveness of technology in our shools: ACT policy report. ACT 
Website. 2004 
31. 
Nicholson S. A user-centered theoretical framework for meaningful gamification. Paper presented at: 
Games+Learning+Society 8.0; June 2012; Madison, WI.  
http://scottnicholson.com/pubs/meaningfulframework.pdf.  Accessed March 31, 2015.  
32. 
Deterding S, Sicart M, Nacke L, O'Hara K, Dixon D. Gamification: using game-design elements in non-
gaming contexts. Paper presented at: ACM CHI Conference on Human Factors in Computing Systems; 
May 7-12 2011.  
33. 
Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining 
gamification. 2011.  Accessed 2011/09/28/.  
34. 
Palmer D, Lunceford S, Patton AJ. The engagement economy: how gamification is reshaping businesses. 
Deloitte Review. 2014 
35. 
Herger M. An overview of gamification in healthcare and fitness. Enterprise Gamification Web site. 2013 
36. 
HealthMeasures. PROMIS. HealthMeasures Web site. Available at: 
http://www.healthmeasures.net/explore-measurement-systems/promis. Accessed December 28, 2017. 
37. 
Sickle cell disease (SCD): data and statistics. Centers for Disease Control and Prevention Website. 
Available at: http://www.cdc.gov/NCBDDD/sicklecell/data.html.  Updated August 31, 2016. Accessed 
November 20, 2017. 

38. 
Sickle-cell disease and other haemoglobin disorders. Fact Sheets. World Health Organization Web site. 
Available at: http://www.who.int/mediacentre/factsheets/fs308/en/. Published 2011.  Updated January 
2011. Accessed March 6, 2015, 2015. 
39. 
Ballas SK. Current issues in sickle cell pain and its management. Hematology: The Education Program of 
the American Society of Hematology. 2007:97-105. 
40. 
Sickle cell disease (SCD): complications and treatments. Centers for Disease Control and Prevention Web 
site. Available at: http://www.cdc.gov/ncbddd/sicklecell/treatments.html. Published 2015.  Updated 
February 18, 2015. Accessed March 5, 2015, 2015. 
41. 
Fosdal MB. Perception of pain among pediatric patients with sickle cell pain crisis. Journal of Pediatric 
Oncology Nursing. 2015;32(1):5-20. 
42. 
Fosdal MB, Wojner-Alexandrov AW. Events of hospitalization among children with sickle cell disease. 
Journal of Pediatric Nursing. 2007;22(4):342-346. 
43. 
Shapiro BS, Dinges DF, Orne EC, et al. Home management of sickle cell-related pain in children and 
adolescents: natural history and impact on school attendance. Pain. 1995;61(1):139-144. 
44. 
Ballas SK. The sickle cell painful crisis in adults: phases and objective signs. Hemoglobin. 1995;19(6):323-
333. 
45. 
Dobson CE, Byrne MW. Original research: using guided imagery to manage pain in young children with 
sickle cell disease. American Journal of Nursing. 2014;114(4):26-36; test 37, 47. 
46. 
Wilson BH, Nelson J. Sickle cell disease pain management in adolescents: a literature review. Pain 
Management Nursing. 2015;16(2) 
47. 
Akinola NO, Stevens SM, Franklin IM, Nash GB, Stuart J. Rheological changes in the prodromal and 
established phases of sickle cell vaso-occlusive crisis. British Journal of Haematology. 1992;81(4):598-
602. 
48. 
Heath LE, Heeney MM, Hoppe CC, et al. Successful utilization of an electronic pain diary in a 
multinational phase 3 interventional study of pediatric sickle cell anemia. Clinical Trials. 2017;14(6):563-
571. doi: 10.1177/1740774517723307 
49. 
Bakshi N, Smith ME, Ross D, Krishnamurti L. Novel metrics in the longitudinal evaluation of pain data in 
sickle cell disease. Clinical Journal of Pain 2017;33(6):517-527. doi: 10.1097/ajp.0000000000000431 
50. 
Dampier C, Barry V, Gross HE, et al. Initial evaluation of the pediatric PROMIS(R) health domains in 
children and adolescents with sickle cell disease. Pediatric Blood and Cancer. 2016;63(6):1031-1037. doi: 
10.1002/pbc.25944.PMC4833539 
51. 
Haywood C, Jr., Tanabe P, Naik R, Beach MC, Lanzkron S. The impact of race and disease on sickle cell 
patient wait times in the emergency department. American Journal of Emergency Medicine. 
2013;31(4):651-656. doi: 10.1016/j.ajem.2012.11.005.PMC3608692 
52. 
Sporrer KA, Jackson SM, Agner S, Laver J, Abboud MR. Pain in children and adolescents with sickle cell 
anemia: a prospective study utilizing self-reporting. American Journal of Pediatric 
Hematology/Oncology. 1994;16(3):219-224. 
53. 
Dampier C, Ely E, Brodecki D, O'Neal P. Home management of pain in sickle cell disease: a daily diary 
study in children and adolescents. Journal of Pediatric Hematology/Oncology. 2002;24(8):643-647. 
54. 
Thompson RJ, Jr., Gil KM, Burbach DJ, Keith BR, Kinney TR. Psychological adjustment of mothers of 
children and adolescents with sickle cell disease: the role of stress, coping methods, and family 
functioning. Journal of Pediatric Psychology. 1993;18(5):549-559. 
55. 
Pinquart M, Shen Y. Behavior problems in children and adolescents with chronic physical illness: a meta-
analysis. Journal of Pediatric Psychology. 2011;36(9):1003-1016. doi: 10.1093/jpepsy/jsr042 
56. 
Unal S, Toros F, Kutuk MO, Uyaniker MG. Evaluation of the psychological problems in children with sickle 
cell anemia and their families. Pediatric Hematology and Oncology. 2011;28(4):321-328. doi: 
10.3109/08880018.2010.540735 

57. 
Anie KA, Green J. Psychological therapies for sickle cell disease and pain (Review). Cochrane Database of 
Systematic Reviews. 2015(5):34. doi: 10.1002/14651858.CD001916.pub3 
58. 
Edwards LY, Edwards CL. Psychosocial treatments in pain management of sickle cell disease. Journal of 
the National Medical Association. 2010;102(11):1084-1094. 
59. 
Rebok G, Riley A, Forrest C, et al. Elementary school-aged children's reports of their health: a cognitive 
interviewing study. Quality of Life Research. 2001;10(1):59-70. 
60. 
Varni JW, Limbers CA, Burwinkle TM. How young can children reliably and validly self-report their 
health-related quality of life?: an analysis of 8,591 children across age subgroups with the PedsQL 4.0 
Generic Core Scales. Health and Quality of Life Outcomes. 2007;5(1):1. 
61. 
Petersson C, Simeonsson RJ, Enskar K, Huus K. Comparing children's self-report instruments for health-
related quality of life using the International Classification of Functioning, Disability and Health for 
Children and Youth (ICF-CY). Health and Quality of Life Outcomes. 2013;11(1):75. 
62. 
Tieffenberg JA, Wood EI, Alonso A, Tossutti MS, Vicente MF. A randomized field trial of acindes: a child-
centered training model for children with chronic illnesses (asthma and epilepsy). Journal of Urban 
Health. 2000;77(2):280-297. doi: 10.1007/BF02390539 
63. 
Runeson I, Hallstrom I, Elander G, Hermeren G. Children's participation in the decision-making process 
during hospitalization: an observational study. Nursing Ethics. 2002;9(6):583-598. 
64. 
de Winter M, Baerveldt C, Kooistra J. Enabling children: participation as a new perspective on child-
health promotion. Child: Care, Health and Development. 1999;25(1):15-23. 
65. 
Vetter TR, Bridgewater CL, McGwin G, Jr. An observational study of patient versus parental perceptions 
of health-related quality of life in children and adolescents with a chronic pain condition: who should 
the clinician believe? Health and Quality of Life Outcomes. 2012;10:85. 
66. 
Cremeens J, Eiser C, Blades M. Factors influencing agreement between child self-report and parent 
proxy-reports on the Pediatric Quality of Life InventoryT 4.0 (PedsQLT) generic core scales. Health and 
Quality of Life Outcomes. 2006;4:58. 
67. 
Cremeens J, Eiser C, Blades M. Characteristics of health-related self-reported measure for children aged 
three to eight years: a review of the literature. Quality of Life Research. 2006;15(4):739-754. 
68. 
Brown JS. Growing up digital: how the web changes work, education, and the ways people learn. 
Change. 2000;32(2):11-20. 
69. 
Oblinger D, Oblinger JL, Lippincott JK. Educating the Net Generation. Boulder, CO: EDUCAUSE; 2005. 
70. 
Bates AW, Poole G. Effective Teaching with Technology in Higher Education: Foundations for Success. 
Indianapolis, IN: Jossey-Bass, An Imprint of Wiley; 2003. 
71. 
Lenhart A, Kahne J, Middaugh E, Macgill A, Evans C, Vitak J; Pew Research Center. Teens, Video Games 
and Civics.  Published September 16 2008. 
72. 
Lee JJ, Hammer J. Gamification in education: what, how, why bother? Academic Exchange Quarterly. 
2011;15(2):146. 
73. 
King D, Greaves F, Exeter C, Darzi A. "Gamification": influencing health behaviours with games. Journal 
of the Royal Society of Medicine. 2013;106(3):76-78. 
74. 
McCallum S. Gamification and serious games for personalized health. Stud Health Technol Inform. 
2012;177:85-96. 
75. 
Stinson JN, Jibb LA, Nguyen C, et al. Development and testing of a multidimensional iPhone pain 
assessment application for adolescents with cancer. Journal of Medical Internet Research. 
2013;15(3):e51. doi: 10.2196/jmir.2350.Pmc3636147 
76. 
Jibb LA, Stevens BJ, Nathan PC, Seto E, Cafazzo JA, Stinson JN. A smartphone-based pain management 
app for adolescents with cancer: establishing system requirements and a pain care algorithm based on 
literature review, interviews, and consensus. JMIR Research Protocols. 2014;3(1):e15.Pmc3978558 

77. 
Fortier MA, Chung WW, Martinez A, Gago-Masague S, Sender L. Pain buddy: a novel use of m-health in 
the management of children's cancer pain. Computers in Biology and Medicine. 2016;76:202-214. doi: 
10.1016/j.compbiomed.2016.07.012.PMC5639256 
78. 
Gary KA, Rallabhandi P, Walek E, et al. An mHealth hybrid app for self-reporting pain measures for sickle 
cell disease. 2016 IEEE Healthcare Innovation Point-Of-Care Technologies Conference (HI-POCT); 9-11 
Nov; Cancun, Mexico. 
79. 
Stinson JN, Jibb LA, Nguyen C, et al. Construct validity and reliability of a real-time multidimensional 
smartphone app to assess pain in children and adolescents with cancer. Pain. 2015;156(12):2607-2615. 
doi: 10.1097/j.pain.0000000000000385 
80. 
Jonassaint CR, Shah N, Jonassaint J, De Castro L. Usability and feasibility of an mHealth intervention for 
monitoring and managing pain symptoms in sickle cell disease: the Sickle Cell Disease Mobile Application 
to Record Symptoms via Technology (SMART). Hemoglobin. 2015;39(3):162-168. 
81. 
Jibb LA, Stevens BJ, Nathan PC, et al. Implementation and preliminary effectiveness of a real-time pain 
management smartphone app for adolescents with cancer: A multicenter pilot clinical study. Pediatr 
Blood and Cancer. 2017;64(10) doi: 10.1002/pbc.26554 
82. 
Lieberman DA. Management of chronic pediatric diseases with interactive health games: theory and 
research findings. Journal of Ambulatory Care Management. 2001;24(1):26-38. 
83. 
Abras C, Maloney-Krichmar D, Preece J. User-centered design.  In: Bainbridge, W, ed. Encyclopedia of 
Human-Computer Interaction. Vol 37. Thousand Oaks: Sage Publications; 2004:445-456. 
84. 
Norman DA, Draper SW. User Centered System Design: New Perspectives on Human-Computer 
Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates Inc.; 1986. 
85. 
Deci EL. Effects of externally mediated rewards on intrinsic motivation. Journal of Personality and Social 
Psychology. 1971;18:105-115. 
86. 
Deci EL, Ryan RM. Handbook of Self-Determination Research. University Rochester Press; 2004. 
87. 
Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs NJ: 
Prentice Hall; 1986. 
88. 
Mazzara ML; IBM. Application of theory: minimalism and user centered design. Available at: 
http://www.gotomedia.com/goto/web2expo/workshop/resources/minimialism_ucd.pdf. 2001.  
Accessed December 20, 2017. 
89. 
Pagulayan RJ, Keeker K, Wixon D, Romero RL, Fuller T. User-centered design in games. In: Sears, A, Jacko, 
JA, eds. Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging 
Applications. Mahwah, NJ: Lawrence Erlbaum Associaties, Inc.; 2003:883-906. 
90. 
Silva MN, Markland D, Minderico CuS, et al. A randomized controlled trial to evaluate self-determination 
theory for exercise adherence and weight control: rationale and intervention description. BMC Public 
Health. 2008;8(1):234. 
91. 
Silva MN, Vieira PN, Coutinho SlR, et al. Using self-determination theory to promote physical activity and 
weight control: a randomized controlled trial in women. J Behav Med. 2010;33(2):110-122. 
92. 
Ryan RM, Patrick H, Deci EL, Williams GC. Facilitating health behaviour change and its maintenance: 
interventions based on Self-Determination Theory. The European Health Psychologist. 2008;10(1):2-5. 
93. 
Vansteenkiste M, Soenens B, Vandereycken W. Motivation to change in eating disorder patients: a 
conceptual clarification on the basis of self-determination theory. International Journal of Eating 
Disorders. 2005;37(3):207-219. 
94. 
Markland D, Ryan RM, Tobin VJ, Rollnick S. Motivational interviewing and self-determination theory. 
Journal of Social and Clinical Psychology. 2005;24(6):811-831. 
95. 
Rogers ER, King SR. Intervention based on social cognitive theory: evaluating adolescents' knowledge of 
OTC pain medications. Journal of the American Pharmaceutical Association (American Pharmaceutical 
Association). 2013;53(1):30-38. doi: 10.1331/JAPhA.2013.11231 

96. 
Edwards LY, Edwards CL. Psychosocial treatments in pain management of sickle cell disease. Journal of 
the National Medical Association. 2010;102:1084-1094. 
97. 
Lewis Hobart M. Learning from myself: avatars and educational video games. Current Issues in 
Education. 2012;15(3) 
98. 
Gazzard A. The avatar and the player: understanding the relationship beyond the screen. 2009.  
Accessed 2013/03/23/.  
99. 
Kaufman GF, Libby LK. Changing beliefs and behavior through experience-taking. Journal of Personality 
and Social Psychology. 2012;103(1):1. 
100. 
Hope & Destiny Jr. - The Adolescents Guide to Sickle Cell Diseases book information. Hilton Publishing 
Company Website. Available at: http://www.hiltonpub.com/bookstore/products/205314-hope-destiny-
jr-the-adolescents-guide-to-sickle-cell-diseases.aspx. Accessed April 5, 2015. 
101. 
Zempsky WT. Evaluation and treatment of sickle cell pain in the emergency department: paths to a 
better future. Clinical Pediatric Emergency Medicine. 2010;11(4):265-273.Pmc3076949 
102. 
Pack-Mabien A, Labbe E, Herbert D, Haynes J, Jr. Nurses' attitudes and practices in sickle cell pain 
management. Applied Nursing Research. 2001;14(4):187-192. 
103. 
Labbe E, Herbert D, Haynes J. Physicians' attitude and practices in sickle cell disease pain management. 
Journal of Palliative Care. 2005;21(4):246-251. 
104. 
Shapiro BS, Benjamin LJ, Payne R, Heidrich G. Sickle cell-related pain: perceptions of medical 
practitioners. Journal of Pain and Symptom Management. 1997;14(3):168-174. 
105. 
Greenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine 
practice in improving the process and outcomes of patient care: a literature review. Journal of 
Evaluation in Clinical Practice. 1999;5(4):401-416. 
106. 
Varni JW, Burwinkle TM, Lane MM. Health-related quality of life measurement in pediatric clinical 
practice: an appraisal and precept for future research and application. Health and Quality of Life 
Outcomes. 2005;3(1):34. 
107. 
Sickle cell disease: national resource directory. Centers for Disease Control and Prevention Web site. 
Available at: https://www.cdc.gov/ncbddd/sicklecell/map/map-nationalresourcedirectory.html. 
Published 2011.  Updated June 19, 2017. Accessed December 28, 2017. 
108. 
Brooke J. SUS-A quick and dirty usability scale. In: Jordan, PW, Thomas, B, Weerdmeester, BA, 
McClelland, IL, eds. Usability evaluation in industry. London: Taylor & Francis; 1996:189-194. 
109. 
Krueger RA, Casey MA. Focus Groups: A Practical Guide for Applied Research. Sage; 2009. 
110. 
Jones FC, Broome ME. Focus groups with African American adolescents: enhancing recruitment and 
retention in intervention studies. Journal of Pediatric Nursing. 2001;16(2):88-96. 
111. 
UserTesting. What is remote usability testing? User Testing Blog. Vol 2017: User Testing Web site.  
Available at: https://www.usertesting.com/blog/2015/11/30/what-is-remote-usability-testing/. 
112. 
UserTesting. The UX Research Methodology guidebook. UserTesting Web site. Available at: 
https://info.usertesting.com/UX-Research-Methodology-
Guidebook.html?utm_source=blog&utm_campaign=ps. Accessed December 29, 2017. 
113. 
Kanneganti S. 5 reason for going remote with mobile website usability testing. UserZoom Blog. Vol 2017.  
Available at: https://www.userzoom.com/mobile-usability-testing/5-reasons-for-going-remote-with-
mobile-website-usability-testing/. 
114. 
Remote testing. usability.gov Web site. Available at: https://www.usability.gov/how-to-and-
tools/methods/remote-testing.html. Accessed December 29, 2017. 
115. 
Martin R, Al Shamari M, Seliaman M, Mayhew PJ. Remote asynchronous testing: a cost-effective 
alternative for website usability evaluation. International Journal of Computer and Information 
Technology. 2014;3(1):99-104. 
116. 
Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: Adding an adjective rating 
scale. J Usability Stud. 2009;4(3):114-123. 

117. 
Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput 
Interact. 2008;24(6):574-594. 
118. 
Peterson C. Bringing ADDIE to life: instructional design at its best. Journal of Educational Multimedia and 
Hypermedia. 2003;12(3):227-241. 
119. 
Gustafson KL, Branch RM. What is instructional design? In: Reiser, RA, Dempsey, JV, eds. Trends and 
Issues in Instructional Design and Technology. Columbus, OH: Merril/Prentice Hall; 2002. 
120. 
Molenda M. In search of the elusive ADDIE model. Performance Improvement. 2015;54(2):40-42. 
121. 
Hunicke R, LeBlanc M, Zubek R. MDA: a formal approach to game design and game research. 2004.  
122. 
Zichermann G, Cunningham C. Gamification by Design: Implementing Game Mechanics in Web and 
Mobile Apps. O'Reilly Media, Inc.; 2011. 
123. 
Dattatreya V, Chalapati Rao KV. Agile programming and design patterns in web development - a case 
study. International Journal of Software Engineering & Applications. 2012;3(1):37-45. 
124. 
Khalid A, Zahra S, Khan MF. Suitability and contribution of agile methods in mobile software 
development. International Journal of Modern Education and Comupter Science. 2014;2:56-62. 
125. 
Hope for SCD, Rose L. Sickle What? 1 ed. Munster, IN: Hilton Publishing; 2016. 
126. 
Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-
management of adolescent type 1 diabetes: a pilot study. Journal of medical Internet research. 
2012;14(3):e70. 
127. 
Graafland M, Dankbaar M, Mert A, et al. How to systematically assess serious games applied to health 
care. JMIR serious games. 2014;2(2) 
128. 
Kulyk O, den Daas C, David S, van Gemert-Pijnen L. How Persuasive are Serious Games, Social Media and 
mHealth Technologies for Vulnerable Young Adults?  
129. 
Hieftje K, Duncan L, Sawyer B, Haskell S, Fiellin L. A Tablet Game for Risk Reduction and HIV Prevention 
in Adolescents. Journal of Mobile Technology in Medicine. 2012;1(4S):8-8. 
130. 
Shegog R, Brown K, Bull S, et al. Serious games for sexual health. Games for Health Journal. 
2015;4(2):69-77. 
131. 
Nah FFH, Telaprolu VR, Rallapalli S, Venkata PR. Gamification of education using computer games. 
Human Interface and the Management of Information. Information and Interaction for Learning, Culture, 
Collaboration and Business: Springer; 2013:99-107. 
132. 
Initiative EL. 7 things you should know about BADGES. 2012. 
https://net.educause.edu/ir/library/pdf/ELI7085.pdf. 
133. 
Easley D, Ghosh A. Incentives, gamification, and game theory: an economic approach to badge design. 
Paper presented at: Proceedings of the fourteenth ACM conference on Electronic commerce2013.  
134. 
Miller AS, Cafazzo JA, Seto E. A game plan: Gamification design principles in mHealth applications for 
chronic disease management. Health informatics journal. 2014:1460458214537511. 
135. 
Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. 
Behavior Research Methods, Instruments, & Computers. 2003;35(3) 
136. 
Macefield R. How to specify the participant group size for usability studies: a practitioner's guide. 
Journal of Usability Studies. 2009;5(1) 
137. 
Turner CW, Lewis JR, Nielsen J. Determining usability test sample size. In: Karwowski, W, ed. 
International Encyclopedia of Ergonomics and Human Factors. Vol 2nd. Boca Raton, FL: CRC Press; 
2006:3084-3088. 
138. 
Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. Morgan 
Kaufmann; 2012. 
139. 
Sauro J. A Practical Guide to Measuring Usability: 72 Answers to the Most Common Questions About 
Quantifying the Usability of Websites and Software. CreateSpace Independent Publishing Platform; 
2010. 

140. 
Sim G, MacFarlane S, Read J. All work and no play: measuring fun, usability, and learning in software for 
children. Computers & Education. 2006;46:235-248. 
141. 
Brooke J. SUS - A quick and dirty usability scale. In: Jordan, PW, Thomas, B, Weerdmeester, BA, 
McClelland, IL, eds. Usability Evaluation in Industry. London: Taylor & Francis; 1996:189-194. 
142. 
Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating 
scale. Journal of Usability Studies. 2009;4(3):114-123. 
143. 
Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. International 
Journal of Human-Computer Interaction. 2008;24(6):574-594. 
144. 
Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized 
Causal Inference. Boston, MA: Houghton Mifflin Company; 2002. 
145. 
Germain ML. Stages of psychometric measure development: the example of the generalized expertise 
measure (GEM). 2006 
146. 
Lawshe CH. A quantitative approach to content validity1. Pers Psychol. 1975;28(4):563-575. 
147. 
Hinkin TR. A review of scale development practices in the study of organizations. J Manage. 
1995;21(5):967-988. 
148. 
Nunnally JC. Psychometric Theory. New York, NY: McGraw-Hill; 1978. 
149. 
Varni JW, Magnus B, Stucky BD, et al. Psychometric properties of the PROMIS (R) pediatric scales: 
precision, stability, and comparison of different scoring and administration options. Quality of Life 
Research. 2014;23(4):1233-1243. doi: 10.1007/s11136-013-0544-0.PMC4312615 
150. 
DeWalt DA, Gross HE, Gipson DS, et al. PROMIS(R) pediatric self-report scales distinguish subgroups of 
children within and across six common pediatric chronic health conditions. Quality of Life Research. 
2015;24(9):2195-2208. doi: 10.1007/s11136-015-0953-3.PMC4531096 
151. 
Treadwell M, Johnson S, Sisler I, et al. Development of a sickle cell disease readiness for transition 
assessment. International Journal of Adolescent Medicine and Health. 2016;28(2):193-201. doi: 
10.1515/ijamh-2015-0010 
152. 
Johnson A, Waters P, Griffin A, et al. Stepping Up To Adult Care: Developing a Healthcare Transition 
Program For Pediatric Sickle Cell Disease Populations. 
153. 
Newland JA. Factors influencing independence in adolescents with sickle cell disease. Journal of Child 
and Adolescent Psychiatric Nursing. 2008;21(3):177-185. doi: 10.1111/j.1744-6171.2008.00149.x 
154. 
Barakat LP, Schwartz LA, Salamon KS, Radcliffe J. A family-based randomized controlled trial of pain 
intervention for adolescents with sickle cell disease. Journal of Pediatric Hhematology/Oncology. 
2010;32(7):540-547. doi: 10.1097/MPH.0b013e3181e793f9 
155. 
Clay OJ, Telfair J. Evaluation of a disease-specific self-efficacy instrument in adolescents with sickle cell 
disease and its relationship to adjustment. Child Neuropsychology. 2007;13(2):188-203. doi: 
10.1080/09297040600770746 
156. 
Landgraf JM, Abetz L, Ware JE. The CHQ User's Manual. Boston: The Health Institute, New England 
Medical Center; 1996. 
157. 
Raat H, Botterweck AM, Landgraf JM, Hoogeveen WC, Essink-Bot M-L. Reliability and validity of the short 
form of the child health questionnaire for parents (CHQ-PF28) in large random school based and general 
population samples. Journal of Epidemiology and Community Health. 2005;59(1):75. 
158. 
Robinson KA, Dennison CR, Wayman DM, Pronovost PJ, Needham DM. Systematic review identifies 
number of strategies important for retaining study participants. Journal of Clinical Epidemiology. 
2007;60(8):757-765. doi: 10.1016/j.jclinepi.2006.11.023.PMC1997303 
159. 
Coday M, Boutin-Foster C, Goldman Sher T, et al. Strategies for retaining study participants in behavioral 
intervention trials: retention experiences of the NIH Behavior Change Consortium. Annals of Behavioral 
Medicine. 2005;29 Suppl:55-65. doi: 10.1207/s15324796abm2902s_9 

160. 
Nicholson LM, Schwirian PM, Groner JA. Recruitment and retention strategies in clinical studies with 
low-income and minority populations: Progress from 2004-2014. Contemporary Clinical Trials. 
2015;45(Pt A):34-40. doi: 10.1016/j.cct.2015.07.008 
161. 
Karlson CW, Rapoff MA. Attrition in randomized controlled trials for pediatric chronic conditions. Journal 
of Pediatric Psychology. 2009;34(7):782-793. doi: 10.1093/jpepsy/jsn122 
162. 
Krueger RA. Focus groups: A practical guide for applied research. Sage; 2009. 
163. 
Tucker JA, Roth DL, Huang J, Crawford MS, Simpson CA. Effects of interactive voice response self-
monitoring on natural resolution of drinking problems: utilization and behavioral economic factors. 
Journal of Studies on Alcohol and Drugs. 2012;73(4):686. 
164. 
Fritz MS, MacKinnon DP. Required sample size to detect the mediated effect. Psychological Science. 
2007;18(3):233-239. 
165. 
MacKinnon DP, Lockwood CM, Williams J. Confidence limits for the indirect effect: distribution of the 
product and resampling methods. Multivariate Behavioral Research. 2004;39(1):99. 
 
 
 
 

^ƚĂƟƐƟĐĂůŶĂůǇƐŝƐWůĂŶ  
 
^ƚĂƟƐƟĐĂů WŽǁĞƌĂůĐƵůĂƟŽŶƐĂŶĚŶĂůǇƐŝƐ͘  
^ĂŵƉůĞƐŝǌĞ͗ The primary outcome is detecting a positive change on the knowledge acquisition questionnaire 
(communication skill and pain identification).  dŽďĞĐŽŶƐĞƌǀĂƟǀĞ͕ǁĞĂƐƐƵŵĞd ƚŚĂƚϴϬйŽĨƚŚŽƐĞŽīĞƌĞĚƚŚĞĂƉƉ
will have baseline and post-test results. Our sample size of 100 allows for up to 20% dropouts. We expect results 
similar to those found by Barakat et.al.159 on our secondary outcome measures (which were similar to ours). We 
used their results to derive our nonparametric sample sizes on SCD disease-ƐƉĞĐŝĮĐĂŶĚƚƌĂŶƐŝƟŽŶŬŶŽǁůĞĚŐĞ͕
and family cohesion͘&ŽƌƚŚĞƚƌĂŶƐŝƟŽŶŬnowledge construct͕ĂƐƐƵŵŝŶŐƚŚĞĐŽƌƌĞůĂƟŽŶďĞĨŽƌĞ-to-ĂŌĞƌŝƐϬ͘ϱ͕ƚŚĞ
sample size is N=22, using power of 80% and a Bonferroni corrected Type I error of 0.0167. For disease-ƐƉĞĐŝĮĐ 
ŬŶŽǁůĞĚŐĞ͕ƚŚĞƐĂŵƉůĞƐŝǌĞŶĞĐĞƐƐĂƌǇƚŽĂĐŚŝĞǀĞϴϬйƉŽǁĞƌŝƐϳϲƉĂƌƟĐŝƉĂŶts, ƵƐŝŶŐƚŚĞƐĂŵĞĂƐƐƵŵƉƟŽŶƐ
(Type I error of 0.0167). For the third outcome (i.e., family cohesion), if we assume 80 pĂƌƟĐŝƉĂŶƚƐĂƚall ƟŵĞ 
points, then the power is somewhat lower since the sample size required to achieve 80% power is 132 (mean 
ĚŝīĞƌĞŶĐĞďĞƚǁĞĞŶpost-test and bĂƐĞůŝŶĞǁĂƐϲ͘ϱϭǁŝƚŚĂƉŽŽůĞĚƐƚĂŶĚĂƌĚĚĞǀŝĂƟŽŶŽĨϮϮ͘ϴͿ͘For this outcome, 
tŚĞƉŽǁĞƌǁŝƚŚϴϬƉĂƌƟĐŝƉĂŶƚƐŝƐϱϰй͘We weighed the ŝŵƉŽƌƚĂŶĐĞŽĨƚŚĞĮƌƐƚƚǁŽŽƵƚĐŽŵĞƐ (disease-ƐƉĞĐŝĮĐ
and tƌĂŶƐŝƟŽŶŬŶŽǁůĞĚŐĞͿ ƌĞůĂƟǀĞƚŽƚŚĞŝŶĐƌĞĂƐĞŝŶĐŽƐƚĂŶĚĚƵƌĂƟŽŶƚŽŝŶĐƌĞĂƐĞƚŚĞƐĂŵƉůĞƚŽ165 (i.e., 132 
plus dropouts up to 20%) to account for the single-item family cohesion variable. We felt that having greater 
than 50% power for family cohesion and ƐƵĸĐŝĞŶƚƉŽǁĞƌĨŽƌƚŚĞƉĂƌƟĐŝƉĂŶƚ-related measures (80% for SCD 
disease-ƐƉĞĐŝĮĐĂŶĚƚƌĂŶƐŝƟŽŶŬŶŽǁůĞĚŐĞ), we would not increase the sample size by 65% (i.e., from 100 to 165) 
in order to achieve 80% power for family cohesion͘tĞŚĂǀĞŶĞǀĞƌƚŚĞůĞƐƐůĞŌŝƚŝŶ, because we respect its 
importance. As a Phase II study, we believe that we can accept less power on that single measure given 
ĐŽŶƐŝƐƚĞŶĐǇŽĨŽƵƚĐŽŵĞƐǁŝůůĚƌŝǀĞĨƵƌƚŚĞƌĞǀŽůƵƟŽŶŽĨƚŚŝƐƉƌŽĚƵĐƚ͘ 
ŶĂůǇƐĞƐ͗ There are 2 aims that will be evaluated (AIMs 2 and 3). As a Phase II SBIR, the analyses are heavily 
ĚĞƐĐƌŝƉƟǀĞŝŶŶĂƚƵƌĞĨŽƌƚŚŝƐƚƌŝĂů͘ Means (ƐƚĂŶĚĂƌĚĚĞǀŝĂƟŽŶƐͿ͕ŵĞĚŝĂŶƐ;ŝŶƚĞƌƋƵĂƌƟůĞƌĂŶŐĞͿ͕ĚŝƐƚƌŝďƵƟŽŶƐ͕
ĨƌĞƋƵĞŶĐŝĞƐĂŶĚƉĞƌĐĞŶƟůĞƐ͕ĞƚĐ͘ǁŝůůďĞƵƐĞĚƚŽĐŚĂƌĂĐƚĞƌŝǌĞƚŚĞŽƵƚĐŽŵĞŵĞĂƐƵƌĞƐ͘ 
For ŝŵϮ ;hƐĂďŝůŝƚǇͿ, 14 users are needed to ŝĚĞŶƟĨǇĂŶǇƉƌŽďůĞŵƐƚŚĂƚŝŵƉĂĐƚ>15% of users at a 90% 
ĐŚĂŶĐĞŽĨĚĞƚĞĐƟŽŶ;ůŽŐ;ϭ-.90)/log(1-.15)). DĂƚĂŽŶĂƉƉůŝĞĚĨŽƌŵĂƟǀĞƌĞƐĞĂƌĐŚƐƵŐŐĞƐƚƐƚŚĂƚϱƉĂƌƟĐŝƉĂŶƚƐŝƐ
ƐƵĸĐŝĞŶƚƚŽŝĚĞŶƟĨǇůŝŬĞƐͬĚŝƐůŝŬĞƐĂŶĚƉŽƚĞŶƟĂůĞƌƌŽƌƐ͖ƚŚƵƐ͕ƚŚĞƉŽǁĞƌƚŽdetect Level 1 severity usability errors 
ƐŚŽƵůĚďĞŵŽƌĞƚŚĂŶƐƵĸĐŝĞŶƚ with n=14.167 Similar to Phase I, usability outcomes will also include program 
ƐĂƟƐĨĂĐƟŽŶƌĂƟŶŐƐǁŝƚŚƌĞŐĂƌĚƐƚŽĚĞƐŝŐŶ͕ĂƉƉĞĂů͕ĂŶĚĨƵŶĐƟŽŶĂůŝƚǇ͘ĂĐŚĐŽŶĐĞƉƚǁŝůůďĞƌĂƚĞĚŽŶĂϱ-point 
Likert scale. Means will be calculated and an average of 67% or higher will be considered acceptable. We will 
ĞǆĂŵŝŶĞŵĞĂŶƐĐŽƌĞƐ͕ĐŚĂŶŐĞƐŝŶŵĞĂŶƐĐŽƌĞƐ͕ĂŶĚǀĂƌŝĂƟŽŶǁŝƚŚŝŶŐĞŽŐƌĂƉŚŝĐĂůůŽĐĂƟŽŶ. The SUS will serve as 
the primary criterion for measuring usability.113,121,122 Further, we will look for outliers to assess if there may be 
ƐƉĞĐŝĮĐĐŚĂƌĂĐƚĞƌŝƐƟĐƐŽĨƵƐĞƌƐǁŚŽŚĂǀĞŽƌĚĞǀĞůŽƉƉƌŽďůĞŵƐĂƐƚŚĞƉƌŽĐĞƐƐĞƐĞǀŽůǀĞ͘  
 
īĞĐƟǀĞŶĞƐƐ ;/Dϯ) ŝƐĐƌŝƟĐĂůƚŽƉƌŽĚƵĐƚĚĞǀĞůŽƉŵĞŶƚĂŶĚǁŝůůĞǀĂůƵĂƚĞǁŚĞƚŚĞƌ Pinpoint increases 
ĐŽŵŵƵŶŝĐĂƟŽŶĂŶĚƉĂŝŶŝĚĞŶƟĮĐĂƟŽŶ knowledge. DĞĂŶƐ;ƐƚĂŶĚĂƌĚĚĞǀŝĂƟŽŶƐͿ͕ŵĞĚŝĂŶƐ;iŶƚĞƌƋƵĂƌƟůĞƌĂŶŐĞͿ͕
ĚŝƐƚƌŝďƵƟŽŶƐ͕ĨƌĞƋƵĞŶĐŝĞƐĂŶĚƉĞƌĐĞŶƟůĞƐǁŝůůďĞƵƐĞĚƚŽĐŚĂƌĂĐƚĞƌŝǌĞƚŚĞprimary outcome measure. We will 
ƚĞƐƚƚŚĞǁŝƚŚŝŶƉĞƌƐŽŶĐŚĂŶŐĞŽǀĞƌƟŵĞfor knowledge aĐƋƵŝƐŝƟŽŶ͕SCD disease-ƐƉĞĐŝĮĐĂŶĚƚƌĂŶƐŝƟŽŶ
knowledge, SCD self-ĞĸĐĂĐǇ, and family cohesion. Linear regression will ĂƐƐĞƐƐƚŚĞĞīĞĐƚƐŽĨĂŐĞ͕ďŝŽůŽŐŝĐƐĞǆ, 
ĂŶĚĞĚƵĐĂƟŽŶ͘ Parents will complete a proxy version of the primary outcome; child and parent-report 
agreement will be assessed using intra-ĐůĂƐƐĐŽƌƌĞůĂƟŽŶƐĂŶĚŵĞĚŝĂŶĚŝīĞƌĞŶĐĞƚĞƐƚed using Wilcoxon 
ƐŝŐŶŝĮĐĂŶĐĞƚĞƐƚƐ͘<ŽůŵŽŐŽƌŽǀ-^ŵŝƌŶŽǀƚĞƐƚƐǁŝůůĂƐƐĞƐƐĨŽƌŶŽƌŵĂůƐĐŽƌĞĚŝƐƚƌŝďƵƟŽŶ͘ 
 
ŚĂƌĂĐƚĞƌŝƐƟĐƐŽĨĚƌŽƉŽƵƚƐĂŶĚͬŽƌŶŽŶĐŽŵƉůŝĂŶƚŝŶĚŝǀŝĚƵĂůƐǁŝůůďĞĞǆĂŵŝŶĞĚƚŽƐŚĞĚůŝŐŚƚŽŶƉŽƚĞŶƟĂů
ǁĞĂŬŶĞƐƐĞƐŽĨƚŚĞĂƉƉ;ĚŝīĞƌĞŶƟĂůĚƌŽƉŽƵƚŽǀĞƌĂůůŽƌǁŝƚŚŝŶĐĞƌƚĂŝŶƐƵďŐƌŽƵƉƐͿ͘hƟůŝǌĂƟŽŶĚĂƚĂǁŝůůďĞ
examined to understand the acceptability and ŝŶƚĞŶƐŝƚǇŽĨŝŶƚĞƌĂĐƟŽŶƐǁŝƚŚƚŚĞĂƉƉ͘dŚĞƉĞŶĞƚƌĂƟŽŶŽĨƚŚĞĂƉƉ
into this at-ƌŝƐŬƉŽƉƵůĂƟŽŶŝƐǀĞƌǇŝŵƉŽƌƚĂŶƚŝŶƵŶĚĞƌƐƚĂŶĚŝŶŐ ƉŽƚĞŶƟĂůŝŵƉĂĐƚ͘ŽŵƉůŝĞƌĂǀĞƌĂŐĞĐĂƵƐĂůĞīĞĐƚ
models168 ǁŝůůďĞƵƐĞĚƚŽĞǆĂŵŝŶĞƚŚĞŝŶƚĞŶƐŝƚǇŽĨƚŚĞĂƉƉ͛ƐƵƟůŝǌĂƟŽŶ͘DĞĚŝĂƟŽŶŵŽĚĞůƐƵƐŝŶŐƚŚĞůĂƚĞƐƚ

bootstrapping methods169,170 will be examined to provide evidence that the changes induced by the treatment 
ĞīĞĐƚƐĂƌĞĐŽŶƐŝƐƚĞŶƚǁŝƚŚƚŚĞĐŚĂŶŐĞƐďĞŝŶŐŵĞĚŝĂƚĞĚǀŝĂƚŚĞĂƉƉ͘ 
 
ƐĂƐĞŶƐŝƟǀŝƚǇĂŶĂůǇƐŝƐƚŽĚĞŵŽŶƐƚƌĂƚĞĚŝƐĐŽƌĚĂŶƚǀĂůŝĚŝƚǇ͕ǁĞǁŝůůƉĞƌĨŽƌŵƚŚĞƐĂŵĞĂŶĂůǇƐŝƐĂƐŽŶƚŚĞ
primary outcome, using the number of correct answers in the CDC Flu IQ (i.e., nonequivalent dependent 
variable)͘dŚŝƐŽƵƚĐŽŵĞƐŚŽƵůĚŶŽƚďĞŝŵƉĂĐƚĞĚďǇƚŚĞŝŶƚĞƌǀĞŶƟŽŶĂŶĚǁĞĞǆƉĞĐƚƚŽƐĞĞŶŽĐŚĂŶŐĞŝŶƚŚĞŵĞĂŶ
ŶƵŵďĞƌĐŽƌƌĞĐƚ͘ůƚŚŽƵŐŚĨĂŝůƵƌĞƚŽƐŚŽǁĂĚŝīĞƌĞŶĐĞŝƐŶŽƚĞǀŝĚĞŶĐĞĨŽƌŶŽĚŝīĞƌĞŶĐĞ͕ƚŚĞĚŝƐĐŽƌĚĂŶƚĞīĞĐƚ
ƐŝǌĞƐŚŽƵůĚƉƌŽǀŝĚĞĂĚĚŝƟŽŶĂůŝŶĨŽƌŵĂƟŽŶthat the primary results are consistenƚǁŝƚŚĂŶŝŶƚĞƌǀĞŶƟŽŶĞīĞĐƚ͘ 
 
ǀŝĚĞŶĐĞĨŽƌ^ĐŝĞŶƟĮĐZŝŐŽƌ and WŽƚĞŶƟĂů>ŝŵŝƚĂƟŽŶƐ͘The study has numerous strengths, ĨĞǁůŝŵŝƚĂƟŽŶƐ, and is 
ƐĐŝĞŶƟĮĐĂůůǇƌŝŐŽƌŽƵƐ͘The study is theory-driven (i.e., User-centered Design and SociaůŽŐŶŝƟǀĞƚŚĞŽƌŝĞƐͿ. 
&ŽƌŵĂƟǀĞĚĂƚĂǁŝůůŝŶĐŽƌƉŽƌĂƚĞŵƵůƟƉůĞƐƚĂŬĞŚŽůĚĞƌinput. The programming ĂƉƉƌŽĂĐŚǁŝůůďĞŵƵůƟ-model and 
evidence-ďĂƐĞĚ;ŝ͘Ğ͕͘/DŽĚĞů͕DĨƌĂŵĞǁŽƌŬ͕'/>ƉƌŽŐƌĂŵŵŝŶŐͿƌĞƐƵůƟŶŐŝŶĂƐǇƐƚĞŵĂƟĐ͕ŇĞǆŝďůĞ͕ĂŶĚ
cost-ĞīĞĐƟǀĞĂƉƉƌŽĂĐŚ͘^ĂŵƉůĞƐŝǌĞƐĂƌĞƐƵĸĐŝĞŶƚĂŶĚƐƵƉƉŽƌƚĞĚďǇƚŚĞůŝƚĞƌĂƚƵƌĞ͘EīĞĐƟǀĞŶĞƐƐǁŝůůďĞ
evaluated in a within-subjects design which ĂůůŽǁƐĨŽƌƐƵĸĐŝĞŶƚƉŽǁĞƌ;ŝ͘Ğ͕͘ƐƚĂƟƐƟĐĂůŝŶĨĞƌĞŶĐĞǁŝƚŚĨĞǁĞƌ
subjects), ĞĸĐŝĞŶĐǇ;i.e., quicker trial ĐŽŵƉůĞƟŽŶ), and longitudinaůĂŶĂůǇƐŝƐ;ŝ͘Ğ͕͘ŝŶĚŝǀŝĚƵĂůĐŚĂŶŐĞƐŽǀĞƌƟŵĞͿ͘ 
The study team and environment are strong͘WŽƚĞŶƟĂůůŝŵŝƚĂƟŽŶƐĂƌĞ͗;1) ĚŝĸĐƵůƚǇƌĞĐƌƵŝƟŶŐ; (2) sources of bias; 
and (3) lack of control group. We have tried to reduce their impact. We will work closely with the EAB and PAB to 
maximize recruitment. Our primary outcome measure is new, but we are using a rigorous approach to scale 
development. Self-report instruments are open to bias, however the use of validated measures is a strength. 
>ŝŵŝƚĂƟŽŶƐŽĨŶŽĐŽŶƚƌŽůŐƌŽƵƉĂƌĞůĞƐƐĞŶed ǀŝĂƚŚĞƵƐĞŽĨĂƟŵĞ-series design for outcome changes and ƚĞƐƟŶŐ 
of a nonequivalent dependent variable.