Development of a Mobile App for an Executive Functioning Intervention for Adolescents
Phase: NA
Conditions: Attention Deficit Hyperactivity Disorder Symptoms
Sponsor: University of California, San Francisco

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Study Protocol and Statistical Analysis Plan 
 
Title: Development of a Mobile App for an Executive Functioning Intervention for Adolescents 
NCT04018794 
 
Date: September 9, 2020 
 
 
 
 

Development of a Mobile App for an Executive Functioning Intervention for Adolescents 
NCT04018794 
 
Study Protocol: 
 
Study Start: September 30, 2019  
Study Completion: September 1, 2020    
 
Brief Summary: This study develops an initial prototype of a mobile tool that will support 
clinician-directed behavioral/organizational skills treatment for adolescents with Attention Deficit 
Hyperactivity Disorder (ADHD) with input guided from key stakeholders. 
 
Abstract/Summary: Attention-Deficit/Hyperactivity Disorder (ADHD) is one of the most 
common childhood mental health disorders, affecting 7-9% of children and adolescents, and 
leading to substantial impairment in adolescence. Despite evidence suggesting that behavioral 
interventions are efficacious, approximately 40-60% of adolescents receiving behavioral 
treatment show little to no improvement and skills are rarely generalized beyond treatment 
sessions. Lack of adolescent motivation and engagement, between-session skills use, reward 
saliency, and family involvement are key contributors to these limited effects. Mobile digital 
health (dHealth) strategies and gamification techniques, offer unique opportunities for 
overcoming the barriers of treatments specific to ADHD by using interactive tools to reinforce in-
vivo skill practice, providing opportunities for immediate reinforcement, and motivating 
adolescents with digital rewards. The primary goal of this study is to develop and preliminarily 
test the integration of a digital health tool into organizational/behavioral skills treatment for 
adolescents with ADHD by improving executive functioning skills, providing in-vivo skills 
reinforcement, and monitoring adolescents' skill utilization. This research will use an iterative 
stakeholder-centered design to develop, refine, and preliminarily test a novel digital health tool, 
applied as an adjunct to behavioral treatment for adolescents with ADHD (ages 11-15). This 
includes focus groups with key stakeholders and an open preliminary feasibility trial and 
usability testing. Data collected from focus groups will inform what content and features could 
be developed to overcome challenges to adolescent engagement and parent involvement. 
During the open trial (N=20) we will assess intervention feasibility, usability, and acceptability. 
During and after the clinical trial, we will collect continuous feedback from users on the usability 
and utility of the tool. At the end of this study we will complete debugging and programming to 
maximize usability before a future larger clinical trial. 
 
Study Objectives: The primary objective of the proposed study is to develop and preliminarily 
test the integration of a digital health tool into organizational/behavioral skills treatment for 
adolescents with ADHD to support clinician-directed behavioral treatment for adolescents with 
ADHD by promoting organizational/EF skills utilization, providing in-vivo skills reinforcement, 
and monitoring adolescents’ skill utilization. A two-phase user experience design cycle will be 
used consisting of initial design and focus groups (Phase 1: Define, 6 months), followed by an 
open trial with extended formative usability testing and software refinement/debugging (Phase 
2: Refine, 6 months). During this open trial (N=20) we will assess intervention feasibility, 
usability, and acceptability.  
 
Aim 1: Design, build, and refine a mobile app for improving EF skills engagement in behavioral 
organizational skills treatment for adolescents. 
Hypothesis 1: Adolescents, their parents, and SMHPs will rate the intervention as 
highly feasible, easy to use, acceptable, and responsive to their needs. 
 

Aim 2: Determine the feasibility, acceptability, and usability of the ATOM app. 
Hypothesis: Given this is an uncontrolled preliminary feasibility study, descriptive data 
will be examined for measures of feasibility, usability, user engagement, adherence/ 
retention, acceptability, and therapeutic process. 
 
Design: Aim 1 (Discovery and Design) will entail iterative software development (6 months) and 
two focus groups. Facilitating a user-centered design process, we will obtain direct input from a 
team of end users (8 adolescents, 8 parents, 8 providers) regarding design, content, and 
functionality. This design will be used to test the central hypothesis that adolescents who use 
the dHealth tool in treatment will consistently utilize skills between sessions. Following 
established guidelines for conducting focus groups particularly with adolescents we will include 
different activity-oriented questions (sticky notes exercises, gameful personas, prototype 
screenshots, and interactive testing with a tablet) in each group to enrich the data collected and 
reduce lapses in attention, which is especially relevant for the target population. Qualitative data 
collected during these sessions including video recordings, written materials (drawings, sticky 
notes, written ideas), and facilitator notes will be used for transcription, coding, and data 
analysis. Data from focus groups will inform what content and features could be developed to 
overcome challenges to adolescent engagement and parent involvement.  
Aim 2 (Test) will conduct extended formative usability testing during an open trial with 
behavioral organizational skills treatment (2 months). We will assess the usability, feasibility, and 
acceptability of the tool in conjunction with their participation in the HOPS intervention at their 
school. Inclusion of these stakeholders will allow for an assessment of the tool components 
during treatment. Adolescents will access the application from their personal device or will be 
provided with a study tablet if they do not have one (although we expect this to be low based on 
current participant usage, high national rates (91%), and that the school district has a program 
that provides tablets to all middle school students. Adolescents will be advised to use the tool 
daily to practice skills and to access information as needed. Consistent with usability 
methodology, passive usage analytics will be collected and adolescents, parents, and SMHPs 
will complete the System Usability Scale at mid-intervention (4-weeks after start) and post-
intervention. Participants will be interviewed to provide detailed feedback during and after the 
open trial to assess reactions to the tool (usability, feasibility, acceptability) to inform refinements 
and identify barriers and facilitators of use. The end of Aim 2 will also entail debugging and 
programming to maximize usability before a larger clinical randomized control trial. 
 
Eligibility Criteria: Participants will include 20 young adolescents (ages 11 to 15) with ADHD, 
and their parents recruited from local schools and the university/medical center clinic.  8 
adolescent subjects and their parents will be enrolled to participate in focus group discussions 
as part of Aim 1 and then 12 additional adolescents and their families will be recruited (for a 
total of N=20) to participate in the feasibility trial. Participants will also consist of 8 mental health 
providers (n=8 during Aim 1, of which n=5 will participate in the trial during Aim 2). The primary 
teachers of the N=20 adolescents participating in the trial will also participate during the trial 
phase by completing measures about the adolescent before and after the intervention. The 
entire participant count is based on plans to include 8 adolescents, 8 parents, and 8 mental 
health providers during the discovery/design phase (Aim 1), and 20 adolescents and their 
parents plus 5 school mental health providers (SMHPs) schools during the pilot study (Aim 2). 
Adolescents’ eligibility for the study is based on meeting following inclusion and exclusion 
criteria: a) Inclusion criteria: 1) Youth ages of 11-14 years (6-8th grade) that are attending a 
participating school2) referred by SMHP as a youth with apparent ADHD-related problems, 3) 
≥6 symptoms (item score ≥2) of Inattention or Hyperactivity-Impulsivity on the pooled parent and 
teacher Vanderbilt ADHD Rating Scale 4) ≥3 on the Impairment Rating Scale by parent and 
teacher (cross-situational impairment)5) Parent consent and adolescent assent must be 

provided; b) Exclusion criteria:1) No presence of conditions that are incompatible with this 
study’s treatment including: Parent or adolescent report of a prior diagnosis of either Autism 
Spectrum Disorder, Bipolar Disorder, a Dissociative Disorder, Severe visual or hearing 
impairment, severe language delay or intellectual impairment, or a Psychotic Disorder will be 
excluded. Rationale: Individuals with these disorders often have very dysregulated behavior and 
impairments that deviate from the focus of this study. 2) Adolescent is in all-day special 
education classes or if core classes not in regular education classrooms. Rationale: The vast 
majority of adolescent with ADHD are served in regular education classrooms and students in 
full-day self-contained classrooms often have different challenges than students in regular 
education. 3) Adolescent planning to change (start or stop) psychotropic medication. Note: 
Adolescents taking medication will be required to meet all entry criteria, including impairment 
criteria, thus indicating a need for the intervention. Adolescent taking medication for attention or 
behavior are eligible as long as their medication regimens are stable. Participating parents will 
also need to be able to read/speak English because all measures are in English, and the 
intervention will be conducted in English. 
 
Statistical Analysis Plan (SAP):  
Analyses Aim 1: Focus group and interview data will be analyzed thematically, 
following established guidelines for qualitative data analysis and representation procedures. 
Recordings will be transcribed and open-coded using Transana 3.30 software. Using a content 
analysis approach, we will condense the data into a thematic framework to generate an 
independent list of thematic categories and subcategories based on the study team’s review of 
the data. Key themes will inform iterative refinements to the features and functions of the app. 
Responses will be coded by two independent coders (a third coder is consulted if needed to 
resolve discrepancies). The codebook will be updated iteratively as new themes emerge. 
Qualitative data (focus group and interview themes) and quantitative data (ratings) will be used 
to guide the initial design, development, and refinements. 
Analyses Aim 2: For feasibility and acceptability, descriptive statistics of adolescent-
reported, parent-reported, and provider-reported satisfaction with intervention structure, length, 
convenience, and perceived impact on outcomes will be assessed. We will examine means and 
standard deviations of ratings completed by adolescents, parents, and SMHPs on the 
Feasibility/Acceptability Questionnaire (ratings of 4 or above are considered good) and System 
Usability Scale. For usability, the statistical test of choice will be ones ample t-test tested at the 
nominal alpha 2-tailed = 0.05 level.  Usage analytics descriptives (i.e., how often users respond 
to prompts, access components) will also be examined as an indicator of usability.  
Representativeness of the sample and enrollment rate of underserved individuals will be 
assessed to inform potential reach.  
Power Analysis: Given the primary goals are to establish feasibility and usability of a 
dHealth tool for enhancing skills utilization in adolescents, the sample size was set for practical 
reasons based on our experiences and the resources/timeline allowable with the funding.  
 
Results: Thematic qualitative analyses found students, parents, and school mental health 
providers prioritized a streamlined interface design for the tool, the use of personalized 
reminders, progress monitoring graphing tools, visual rewards, and an electronic calendar with 
prioritized events. Key focus group/qualitative themes from included the stakeholders’ 
prioritization of tools to engage users, personalization capabilities, reminders, and ease of use 
for all stakeholders. Quantitative usability testing results collected from the open feasibility trial 
found high overall usability of the intervention digital health tool. Participants rated the tool as 
highly acceptable, usable, and feasible to implement (System Usability Scale, M= 71.7). Overall 
usability ratings were high across components (M>=4.3, on a scale of 1=low to 5=high). 

Informed Consent Form (ICF): 
 
 
CONSENT TO PARTICIPATE IN A RESEARCH STUDY 
 
Study Title: Development of a Mobile App for an Executive Functioning Intervention for Adolescents  
 
Research Project 
Director: 
  [REDACTED] 
 
Study Coordinator: 
    [REDACTED] 
 
This is a research study about developing and evaluating a mobile digital application to enhance 
executive functioning skills including organization, planning, and time management for 
adolescents participating in intervention for attention and/or executive functioning concerns. The 
study researchers, [REDACTED], the study project coordinator, or a study clinician from the  
[REDACTED]will explain this study to you.  
 
Research studies include only people who choose to take part. Please take your time to make 
your decision about participating and discuss your decision with your family or friends if you 
wish. If you have any questions, you may ask the researchers. 
 
You and your child are being asked to participate because your child is having attentional, 
behavioral, and/or academic concerns at school and/or home. Please note that students and 
families can participate in the study only if their teacher also agrees to participate 
 
Why is this study being done? 
 
The purpose of this phase of the study is to evaluate a mobile digital health tool for supporting 
intervention focused on improving organization and time management skills for adolescents 
ages 11 to 15 with difficulties with attention and/or organization. Knowledge from this study will 
further research on improving interventions for youth with attentional difficulties. This study is 
sponsored by  [REDACTED].  
 
How many people will take part in this study? 
 
A total of 20 families, and participating adolescents’ teachers will participate in this study.   
 
What will happen if I take part in this research study? 
 
If you agree to participate, and your child agrees to participate and agrees to the 
recording of program meetings, the following procedures will occur: 
1. First, we will conduct screening procedures to find out if you and your child can participate in 
the program. You provided your consent for your school and our program staff to share 
confidential education, psychoeducational and other relevant information about your child 
and your consent for your child’s teacher to complete assessments for the program by 

signing the Consent for Release and Exchange of Information form. Screening, which is 
required for the study, will consist of the following:  
• 
You and your child’s teacher will complete ratings of your child’s behavior.  
• 
If, based on those ratings, it seems as though your child meets criteria for our study, you 
and your child will come to your child’s school, the  [REDACTED] for an evaluation.  
• 
This evaluation will take approximately 30 minutes to complete and will include:  
o Completion of a medical and developmental history and previous service use.  
o Questionnaires for you and your child to complete about your child’s functioning. 
• 
You will receive results of your child’s screening evaluation, which may be released to 
your health care provider at your request, but otherwise will not be released.  
• 
If all study criteria are met, your child will be eligible to continue in the study; if study 
criteria are not met, you will be referred back to your school for other referral options. 
Information already gathered will be kept by the researchers so that they can report on 
characteristics of families who consented but did not participate in the study.  
 
2. During this study, approximately 20 children between the ages of 11 to 15 with attention 
and/or organization concerns will receive an organizational skills intervention program 
augmented with a mobile application tool. A school counselor, school social worker, or other 
school mental health provider at your child’s school will deliver the intervention to your child. 
The organizational skills interventions to be delivered to your child will include:  
• 
Developing a system for organization of bookbag, binder and locker and implementing 
strategies for homework and time management.  
• 
The school counselor/school mental health provider will set-up a meeting schedule with 
your child and will determine when and where he/she will meet with your child based 
upon your child’s needs. It is likely that the school counselor/school mental health 
provider will meet with your child during school hours twice per week for at least 16 
meetings. Each meeting will last no longer than 30 minutes. 
• 
You (parent) will be asked to attend two sessions with the school counselor and your 
child. These sessions will be spent updating you on your child’s progress and teaching 
you how to implement the strategies at home. 
• 
You and your child will be provided access to a mobile application to download on your 
personal mobile devices (such as smartphones, tablets, laptops) or other computers. 
The application is designed to augment the meetings and serve as a reference tool for 
practicing the skills between meetings. You and your child will be asked to use the 
application tool while participating in the program. 
• 
When you and your child are using the app during this program, the app will collect data 
on how you navigate through the features and time spent using the app. The app will 
also collect data on the points your child earns for practicing organization skills using the 
skills checklists, responses to the practice plan questions, responses to questions about 
child's overall progress, as well as information entered into the calendar feature including 
assignment information and dates. Parents, adolescents, and providers will be able to 
view a progress monitoring graph that displays the points earned by the adolescent.  
 
3. All sessions will be video recorded for training and research purposes. An expert trainer 
from  [REDACTED]will provide training and consultation to the school counselor during the 
program. The expert trainer may also observe and provide live support during the sessions 
remotely via videoconference software. All videoconferencing will take place over a secure 
internet connection that meets the  [REDACTED]HIPAA requirements.  
 

4. All families will participate in a follow-up assessment, about 10-12 weeks after the first 
assessment. During this assessment, all parents, teachers, and children will complete 
interviews and questionnaires like those completed at the screening/eligibility assessment.  
 
5. Study location: All study procedures will take place either at your child’s school or  
[REDACTED]. 
 
Will any video or audiotaping be done during this study?  
 
All sessions will be video recorded and will be secured on password-protected database servers 
for research and training purposes. The recordings will be identified only by numbers, and none of 
your names will appear on the electronic file labels. The recordings will be kept indefinitely for 
research, training, and development purposes. The recording is required for future trainings, 
analyses, and development of our research. If you or your child decide to withdraw from 
participating and no longer wish to have your video recordings for research purposes in this study 
at any point in the future, then his/her image will be blurred from recordings.  
 
What if my child is taking medication for ADHD? 
 
If your child is taking medication, he or she still is eligible for the current project as long as he or 
she has been taking the same type or dose of medication for the past month (immediately prior 
to the screening). Your child can continue to take his/her medication as usual throughout his/her 
participation in the current project.  
 
How long will I be in the study? 
 
The total time commitment for all participating families includes completion of rating scales and 
participation in two assessments (once at the beginning of this phase of the study and once at 
the end of the program). The assessments take about 30 minutes for parents and children to 
complete. Participation in the program involves your child attending up to 16 twice/weekly 
intervention meetings (up to 30 minutes each session) for eight weeks and two 30-minute 
meetings between parents, children, and school counselors/mental health providers.  
 
Can I stop being in the study? 
 
Yes. You can decide to stop at any time. Just tell the study researcher or staff person right away 
if you wish to stop being in the study. Also, the study researcher may stop you from taking part 
in this study at any time if he or she believes it is in your best interest, if you do not follow the 
study rules, or if the study is stopped. 
 
What risks can I expect from being in the study? 
 
 
Potential Risks for Completing Evaluation: Potential risks include discomfort that some 
people may feel when being interviewed, observed, or when completing questionnaires 
about personal matters. Protection against these risks includes highly-trained staff, including 
licensed child psychologists. The assessments and intervention will be conducted by 
experienced mental health professionals who are trained to be sensitive to the feelings and 
well-being of the participants.  
 
 
Potential Risks for Psychosocial Interventions: As part of participation in this program, 
participants will be exposed to individual intervention meetings with the school counselor at 

their school site. This may be embarrassing or stigmatizing for some families. However, 
consumer satisfaction from studies of the kind of intervention program we are providing 
have been positive. All participating parents, children, and school counselors will be 
instructed about confidentiality and requested that they not reveal the identity of any 
participants or repeat any personal information provided during these meetings. Protection 
against these risks also includes use of highly trained staff and availability of child 
psychologists; all staff will be trained in protecting the confidentiality of patient information.  
 
 
Potential Risks for Using the App: As part of participation in this program, participants will be 
asked to use the app to record points from the organizational skills list and answer questions 
about their progress. Potential risks include experiencing frustration or discomfort with not 
being able to adhere to the program, experiencing low motivation to use the app, or 
boredom from using the app. As with any digital application, there is risk of loss of 
privacy/confidentiality of the child's identity or information collected from the app if the device 
with the app or the digital copy of the data stored from the app are lost, misplaced, or stolen 
or if the child's identity and information recorded in the app is otherwise accidentally 
discovered by anyone else. All participants will be given a username login and password to 
use to enter the app. Additional protection against these risks include all digital copies of the 
data will be kept on a secure password-protected server during the study and after its 
conclusion. The application and all digital copies of the data will not include labels with 
names, but will be labeled with code numbers only.  
 
 
Potential Risks for audio and video recordings: Potential risks include discomfort that some 
people may feel when being recorded. Loss of confidentiality is the major risk if the child’s 
identity accidentally is discovered by anyone viewing or hearing the recordings. Video and 
audio recordings made during the intervention sessions will be used solely for research and 
training purposes and will be kept in a secure password-protected website or secure video-
streaming program. All digital copies will be kept on a secure password- protected server 
during the study and after its conclusion. Video and audio recordings will be labeled with 
code numbers only; labels will not include names. 
 
 
For more information about risks and side effects, ask one of the researchers. 
 
Are there benefits to taking part in the study? 
 
There may be no direct benefits to you or your child from participating in this study. However, 
the information that you provide may help health professionals to design better programs for 
children, parents, and schools. We cannot and do not promise that you or your child will receive 
any personal benefit from participating in this study. Potential benefits include possible reduction 
in attention, behavioral, and/or organizational problems, and improvement in academic 
functioning as a result of participating in the intervention program. This study will provide careful 
monitoring of your child’s behavior and you may also gain insight into your child through the 
evaluation procedure. 
 
What other choices do I have if I do not take part in this study? 
 
Your other choices may include not getting the intervention, getting standard intervention for 
attention and/or organizational problems without being in a study, or taking part in another 
study. If you decide not to take part in this study, there will be no penalty to you. Your choice will 
not affect your or your child’s care or how you are treated at  [REDACTED]. There may be non-
study school interventions available to you or your child.  

 
How will my information be used?  
 
Researchers will use your information to conduct this study. Once the study is done using your 
information, we may share them with other researchers so they can use them for other studies in 
the future. We will not share your name or any other personal information that would let the 
researchers know who you are. If information from this study is published or presented at 
scientific meetings, your name and other personal information will not be used. You will not need 
to provide additional permission for us to share this de-identified information with other 
researchers. 
 
Will information about my child and me be kept private? 
 
We will do our best to make sure that the personal information gathered for this study is kept 
private. However, we cannot guarantee total privacy. Participation in research may involve a 
loss of privacy, but information about you and your child will be kept as confidential as possible. 
Neither your name nor your child’s name will be used in any published reports about this study. 
All records will be identified by code number only, and the records will be kept in a locked file 
cabinet. The key to this file cabinet will be stored in a secure location. You will provide your 
consent for your school and our program staff to share confidential education, 
psychoeducational and other relevant information by signing a Consent for Release and 
Exchange of Information form. Your personal information may be given out if required by law.  
 
Authorized representatives from the following organizations may review your research data for 
the purpose of monitoring or managing the conduct of this study:  
 
Representatives of the  [REDACTED] 
 
Representatives of the National Institutes of Health  
 
Representatives of the Sponsor, NIH National Center for Advancing Translational Science 
 
Representatives of the Food and Drug Administration (FDA) 
There are some special circumstances that may invoke breaking confidentiality. Such special 
circumstances consist of our becoming aware of (a) suicidal or homicidal intent or planning on 
the part of any participant or family member, or (b) child abuse. All members of the research 
staff will be trained to immediately page or notify the Principal Investigator (PI) upon discovery 
of either (a) or (b), and to stay with any participant who is in danger until appropriate care can 
be provided. The PI will evaluate the situation and take the appropriate steps to resolve the risk, 
including arranging for any emergency care, warning any potential victim, providing referral to 
crisis intervention, or making an abuse report to child protective services.  
 
Are there any costs to me for taking part in this study?  
 
No. This research is supported by the  [REDACTED]. There will be no cost to you or to your 
insurance carrier(s) for any of the research procedures described here. 
 
Will I be paid for taking part in this study? 
 
In return for your time, effort and travel expenses, you will be paid $20, in cash, for completing 
the initial evaluation measures and $30, in cash, for completing the post intervention measures 
(total $50). You will be paid at the evaluation, and your will be required to sign a receipt for cash 
received. You child will have the opportunity to earn up to $50 in gift cards based on their work 
during the intervention program. Your child’s teachers will be paid $25 in cash, for completing 
measures at each of the two evaluations (total $50).  

 
What are my rights if I take part in this study? 
 
Taking part in this study is your choice. You may choose either to take part or not to take part in 
the study.  If you decide to take part in this study, you may leave the study at any time. No 
matter what decision you make, there will be no penalty to you in any way. You will not lose any 
of your regular benefits, and you can still get your care from our institution the way you usually 
do. 
 
Who can answer my questions about the study? 
 
You can talk to the researcher(s) about any questions, concerns, or complaints you have about 
this study. You can contact [REDACTED]. If you still have questions or concerns after talking 
to  [REDACTED]or if you wish to ask questions about the study or your rights as a research 
participant to someone other than the researchers or if you wish to voice any problems or 
concerns you may have about the study, please call the Institutional Review Board at  
[REDACTED].   
 
CONSENT 
 
You have been given a copy of this consent form to keep. The person who signed below has 
explained this information to you, and your questions were answered. 
 
PARTICIPATION IN RESEARCH IS VOLUNTARY.  You or your child have the right to decline 
to be in this study, or to withdraw from it at any point without penalty or loss of benefits to which 
you are otherwise entitled. 
 
If you wish to participate and have your child participate in this study, please sign below. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Date 
 
 
Parent or Legal Guardian Signature for Consent 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Name of Parent or Legal Guardian (please print) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Name of Child (please print) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Date 
 
 
Person Obtaining Consent