Modifying (Phase I) and Evaluating (Phase II) Virtual Reality Job Interview Training for Youth in Transition
Phase: NA
Conditions: Autism Spectrum Disorder
Sponsor: University of Michigan

Running head: MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH 
1 of 24 
 
PROTOCOL TITLE: 
Modifying (Phase I) and Evaluating (Phase II) Virtual Reality Job Interview Training for Youth in 
Transition  
NCT03514134 
 
PRINCIPAL INVESTIGATOR: 
Name: Matthew J. Smith, PhD, MSW, MPE, LCSW 
Department: School of Social Work 
Telephone Number:  (734) 764-3309 
Email Address: mattjsmi@umich.edu 
 
VERSION DATE: May 26, 2020 
 
***Phase I data is completing collection at Northwestern University where the PI is adjunct 
faculty until 07/31/2018. 
 
***The UM research team will be responsible for analyzing the Phase I data and executing the 
Phase II Randomized Controlled Trial. 
 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
2 of 24 
 
1.0 Purpose of the Study:  
The prevalence of autism spectrum disorders (ASD) in the U.S. is estimated at 1%1, with 50,000 youth 
with ASD turning 18 annually2. These youth as well as others with developmental disabilities have a 
high unemployment rate (50-75%),3-5 which could be explained by the limited availability of services to 
support their transition to employment6, 7.  Age of transition ranges from 16 years old through the early 
adult years (e.g., age 21) after high school may have been completed. The high unemployment rate 
demonstrates a clear need to develop evidence-based services to support transition-age youth with 
ASD to prepare for employment before they face additional barriers to employment in adulthood8. We 
propose to modify and test the acceptability and effectiveness of an existing virtual reality training 
program to improve job interview skills for these youth with ASD. Current services for these youth 
minimally focus on practicing interview skills, which were identified as a potential mechanism to 
increase the likelihood to obtain work in recent studies of adults with neuropsychiatric disorders9, 10, 
including adults with ASD11 and by community stakeholders (i.e., individuals with ASD, parents, 
educators, employers)12.  
The social communication symptoms of ASD and other developmental disabilities make it difficult to 
socially interact and relate to the experiences of others13-15. These symptoms present a major barrier for 
youth with ASD to develop effective job interview skills, which are critical for obtaining employment12, 16-
18. Current interventions for individuals with ASD largely focus on sustaining active employment19-25 and 
provide minimal job interview training3, 6. Thus, developing a novel intervention to target interview skills 
among youth with ASD in the safe learning environment of their high school could be a highly effective 
approach to their vocational outcomes. The successful deployment of such an intervention could help 
fill a critical gap in services for this population. 
A long-term goal of our research program is to increase employment for individuals with ASD 
and other educational disabilities. Our collaborative team received two NIMH SBIR grants to develop 
and evaluate “Virtual Reality Job Interview Training” (VR), which improved interview skills and 
vocational outcomes for adults with severe mental illness (SMI)9, 26-29. Although the learning objectives 
that guide VR were adopted to support adults with SMI, the objectives were not well-aligned with “best 
practices” for youth with ASD. However, we pilot tested VR in young adults with ASD, which 
demonstrated efficacy at improving interview skills and vocational outcomes11, 28. Thus, we will build 
significantly on this work to invite students, parents, employers and vocational teachers to use VR and 
provide constructive feedback to guide the development of a modified VR (VR-M) system that targets 
students with educationally-defined disabilities  
Phase I: To create VR-M to focus on virtual reality job interview training for transition-age youth 
with ASD receiving vocational services. We will solicit feedback on the training tool using mixed-
method semi-structured feedback forms with high school (HS) youth with ASD (n=20) or other 
educationally defined disabilities (n=10) as well as employed adults with ASD or other prior educational 
disability as well as parents, employers, and vocational counselors that support youth with ASD, 
educational disability, or adjudication (n=30) to systematically evaluate the core components of VR 
(e.g., interview content, virtual character, VR interface, and didactic content). Qualitative interview data 
will be analyzed for themes regarding technical content-based limitations of VR. SIMmersion will then 
integrate the qualitative results to create a tailored learning system.  
Phase II: To evaluate the feasibility and preliminary effectiveness of VR-M in a randomized pilot 
trial comparing vocational-services-as-usual with VR-M added on (VSU+VR-M) to VSU only. 
Transition-age youth with ASD (n=90) will be enrolled in a 6-month trial to test all study protocol 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
3 of 24 
 
aspects including VR-M training effectiveness, recruitment procedures, and assessment tools. VR-M 
will be administered at educational programs supporting rural, suburban, and urban youth to capture 
greater variation in programming and students. We will monitor VR-M treatment fidelity, acceptability, 
usability, feasibility, and tolerability.  
Phase IIa: To assess whether VR-M is associated with enhanced job interview skills (primary 
target) and interviewing self-confidence, and reduced interviewing anxiety. Compared to VSU, 
the VSU+VR-M group will improve interview skills (Hyp 1), increase self-confidence (Hyp 2), increase 
motivation (Hyp 3) and reduce anxiety (Hyp 4). 
Phase IIb: To assess whether VR-M is associated with higher employment rates and attaining 
employment sooner at 6-month follow-up. Compared to VSU, the VSU+VR-M group will: a) have a 
greater likelihood of attaining employment (Hyp 5) and b) attain employment sooner at 6-month follow-
up (Hyp 6).  
Exploratory Aim 1: To develop hypotheses for future trials, we will explore mechanisms for 
improving interview skills and employment outcomes (mediators) or differential response to VR-
M (moderators) as well as conduct a preliminary evaluation of VR-M delivery within a school 
setting. Mediators will include interview skills (for evaluating outcomes), self-confidence, and anxiety. 
Moderators will include VR-M performance scores and ASD heterogeneity variables (i.e., ASD-
symptoms (e.g., social communication), depressive symptoms, and cognition). We will also explore 
ASD heterogeneity as a mechanism for differential response to VR-M, and collect exploratory 
implementation data. 
Pilot Implementation Aim: To conduct a preliminary implementation evaluation of delivering the 
new VR-M to students in educational programming. We will ask teachers (n= 30) to complete 
Installation Cost Capture forms, Delivery Cost Capture forms, Teacher Context Form and a brief survey 
regarding their use of evidence based practice and whether they observed any factors that became 
barriers to delivering VR-M or factors that helped facilitate the delivery and feasibility of VR-M. 
Teachers may also be asked to participate in an audio recorded semi-structured interview.  We will 
provide up to $45.00 in gift cards for participation in total. 
 
2.0 Background / Literature Review / Rationale for the study: 
The rate of employment among individuals with ASD is 25-50%2, 4, 30 and is much lower (6-12%) among 
recent high school (HS) graduates5, 30. Transition-age youth with ASD and without an intellectual 
disability (ASD-ID) were more likely to be competitively employed than those with an ID (ASD+ID) 
(11.8% vs. 4.1%, respectively). However, these rates were similar between these groups when enrolled 
in SE (11.8 vs. 12.2%, respectively)30. More than 50% of these youth with ASD attempt to pursue work 
or education immediately after HS2, 31, 32, while others delay their engagement in these activities for two 
years after HS graduation2. In part, the struggle to find a job could be explained by inadequate 
transition planning during HS, and gaps between vocational needs and available services6, 7. The ASD-
ID group has limited access to transition planning and SE services3, 8 and a much lower rate for 
attaining any type of job than the ASD+ID group (who has greater access to services) (18% vs. 86%, 
respectively)30. Thus, novel interventions are needed to enhance the vocational services for this 
population4, 30.  

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
4 of 24 
 
Evidence suggests the job interview is a major barrier to employment for individuals with ASD33, 34. 
Moreover, Autism Speaks hosted a roundtable discussion with community stakeholders that concluded 
interviewing is a barrier to employment for the 500,000 teens and young adults with ASD who will begin 
looking for work over the next decade12. Specifically, the social communication deficits that characterize 
ASD are both verbal (e.g., difficulty with small talk) and nonverbal (e.g., poor eye contact) and interfere 
with the reciprocity and flow of conversation during an interview35. Moreover, individuals with ASD have 
deficits in adaptive social skills even in the presence of typical intellectual ability36. These 
characteristics suggest that youth with ASD may struggle with using and interpreting nonverbal 
communication, conveying their qualifications for a job, and engaging in small talk during an interview. 
Adults with ASD from a small study reported that they feel uncomfortable and lack confidence at 
making a good first impression during a job interview28. As such, these individuals may have difficulty 
successfully navigating the job interview. The interview process is also a barrier to employment for 
adults with severe mental illness (SMI)26 whose social deficits converge with those observed in ASD37. 
A promising new intervention developed by our group suggests that using virtual reality to train 
interview skills increases the likelihood for both adults with SMI9 or ASD11 to obtain work. Although 
youth with ASD typically access jobs via vocational counselors, family referrals, or community 
resources, enhancing their interview skills will open up additional opportunities as they pursue 
competitive employment in the community. 
Schools are critical settings in which educational professionals can address the vocational needs of 
students with ASD who will be transitioning to the workforce. Students with ASD often view their adult 
educators as reliable, helpful, and supportive39, which helps facilitate learning sustainable behavior40, 41. 
Also, the school is an important setting to develop novel services as it provides a unique opportunity for 
researchers to access a large number of students and stakeholders whose involvement will help drive 
the development of evidence-based interventions. Overall, school-based learning is an ideal 
mechanism for delivering VR to students with ASD as many of them will need to develop these skills to 
prepare for their transition into post-secondary school career development.  
 
3.0 Inclusion and exclusion criteria: 
Phase II: To evaluate the feasibility and preliminary effectiveness of VR-M in a randomized 
controlled trial (RCT) comparing vocational-services-as-usual with VR-M added on (VSU+VR-M) 
to VSU only 
All study procedures for students will take place at educational programs where students receive 
services. 
The inclusion criteria for students are as follows:  
1. 16-26 years old and enrolled in high school or post-high school transition programming;  
2. Educational Diagnosis of autism spectrum disorder using the Social Responsivity Scale 2nd 
Edition28, or a diagnosis of cognitive impairment, emotional disability, educational impairment or 
other health impairment according to the students Individualized Education Plan (IEP). They 
may have diagnosis of autism spectrum disorder as indicated in their IEP/school plan, or as 
indicated by a parent; 
3. Fluency in English with at least a 3rd grade reading level (confirmed with the Wide Range 
Achievement Test);  

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
5 of 24 
 
4. Currently receiving transitional services as specified in IEP or other; 
5. Willing to be video-recorded; and 
6. Willing and able to provide informed consent. 
The exclusion criteria for students are as follows:  
1. Has an uncorrected hearing or visual problem that prevents him or her from using the training. 
2. Has a medical illness that compromises their cognition (for example, moderate to severe 
traumatic brain injury) 
Inclusion and exclusion criteria is subject to PI discretion. Some students may still be enrolled after 
discussion with the teacher if it is determined that they still might be able to participate despite matching 
exclusion criteria, or not meeting all of the inclusion criteria. These cases will be documented in 
REDCap. We will also report to IRB as ORIO. 
The inclusion criteria for teachers/educational professionals are as follows:  
1. 18 years or older;  
2. Provides transitional services to students at the participating educational partners: Ann Arbor 
Academy, Beaumont Hospital, Troy Project Search Program, Southfield Public Schools, and 
Lincoln Park Avenues Mixter Institute for Transition. 
 
4.0 Procedures Involved: 
A. Study Settings 
This study will collect human subject data from students in these primary locations: Ann Arbor Academy 
(1153 Oak Valley Dr., Ann Arbor, MI 48108) Beaumont Hospital, Troy Project Search (44201 Dequindre 
Rd, Troy, MI 48085), Southfield Public Schools (16299 Mt Vernon St, Southfield, MI 48075), Mixter 
Institute of Transition (2201 Electric, Lincoln Park, MI 48146), and Autism Model School (3020 
Tremainsville Rd, Toledo, OH 43613).  Other locations to be identified at a later point. Personal 
information about schools or districts that participate will not be shared without their expressed consent. 
Each study site is the location of an educational partner where high school students or transition-age 
youth with autism or other ED receive vocational services or a community program serving transition 
age youth with ED. Each location has classrooms where program evaluation will take place. All 
students will provide written informed consent (if 18-26) or written assent (if 16-17 years old; with 
parental permission). Students who are 18-26 and whose parents have educational rights will require 
the parent to sign. 
B. Study Design 
We will conduct a single-blind randomized controlled trial to evaluate the effectiveness of VR-M by 
comparing a vocational services-as-usual group to a group actively using the intervention. We will then 
conduct a 6 month follow up interview with the participants to evaluate their community outcomes. 
C. Study Intervention 
SIMmersion, LLC designed VR to enhance job interview skills for adults with SMI. The new tool to be 
used in Phase II was designed for transition-age youth based on feedback from Phase I. The job-
related interview learning goals are: 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
6 of 24 
 
1. Are You someone people want to work with? 
a. Confident 
b. Positive 
c. Professional 
d. Interested 
2. Are You a Good Worker? 
a. Honest 
b. Dependable 
c. Teamwork 
3. Are You Good for the Job? 
a. Strength and skills 
b. Past experiences 
c. Overcoming limitations 
SIMersion designed VR-M with the following components to help learning: 
C.1a. Electronic Learning (e-learning): Interactive e-learning screens display critical information needed 
to prepare for a job interview such as creating a resume, researching a position, types of questions to 
ask, advice on how to disclose a disability, and an emphasis on effective skills for interacting with the 
interviewer (e.g., dealing with emotionally provocative questions). 
C.1.b. VR-M Interface and “Travis/Rita”: After using the e-learning module, trainees will navigate the 
program to begin the simulated interviews with either Mr. Travis Bishop or Ms. Rita Muniz, virtual 
human resource staff members at a department store named “Wondersmart”. Travis/Rita start of 
seated in an office where the trainee has joined them for an interview. When Travis/Rita speaks, the 
program enters Full Screen mode so that they are speaking directly to the trainee without any 
distractions on screen. When they finish asking a question, the program returns to the interface where 
trainees speak a prescript response of their choosing. 
There are three difficulty levels: easy (e.g. friendly), medium (e.g. direct), and hard (e.g. stern or asks 
illegal questions). VR-M uses “memory” and the interviewer behavior is driven by an advanced 
“emotional model”. For example, if a trainee responses inappropriately to several questions, 
“Travis/Rita” may become dismissive and end the interview, but if the trainee continually answers 
appropriately, they may become more friendly and encouraging. Based on the trainee’s statement, the 
software must then choose a reply for Travis/Rita. Selection is based on three factors: 1) difficulty, 2) 
the history of the conversation, and 3) Travis/Rita’s evolving relationship with the trainee, driven by 
trainee responses. Each factor will be used to computer conditional probabilities associated with each 
possible reply and one is selected. Travis/Rita stay true to his or her character and the emotional state 
created during the play, and therefore may behave differently each time the system is used. 
Buttons at the top of the screen control the interface. The text displays various statements that can be 
spoken to respond to the interviewer’s questions. The interview transcript can be accessed through a 
tab in the middle of the screen. The trainee may change the topic during the conversation using buttons 
to the left of the yellow text. An on-screen coach appears in the bottom right of the interface. The 
trainees can take notes and review prior questions (white and gray boxes).. 
C.1.c. Video Clips: Professional actors/actresses portrayed Rita/Travis during the video-recorded 
sessions. The videos were separated into >1,000 question and response clips that are played during 
the simulated interview. Trainees talk with Travis/Rita using a microphone and voice recognition 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
7 of 24 
 
software. This method exposes trainees to an interactive environment and helps them learn to react to 
an interviewer’s social cues.  
C.1.d Pre-scripted Statements and Voice Recognition: A panel of vocational experts supervised 
hundreds of responses to standard interview questions written by professional scriptwriters during the 
initial SBIR grant award. The scriptwriters worked with the scientific team to compose realistic dialogue 
that provides trainees with opportunities to practice the skills identified in the learning objectives. 
Trainee statements included a wide variety of natural choices with 5-15 potential responses that vary in 
appropriateness of their content. This method allows trainees to choose and learn from their responses. 
VR-M uses voice recognition technology so trainees can practice speaking the pre-scripted responses 
to difficult questions in a stress-free environment. Then, trainees can use the rehearsed answers in real 
interviews. 
C.1.e Non-branching Logic: SIMmersion’s PeopleSIMTM technology uses non-branching logic; which 
allows trainees to behave and speak freely within the confines of a safe simulation. Most social 
simulations, in contrast, use branching logic where trainees select a response from a list of options, 
which terminate when all options are exhausted. This approach minimizes repeated use to a few trials. 
SIMmersion’s technology integrates the video clips and non-branching logic to enable Travis/Rita to 
vary his/her memory, emotion, and personality. This variation supports hours of unique repeated 
practice and naturalistic conversations with VR-M. 
C.1.f Job Coach and Help Buttons: Trainees receive in-the-moment feedback from an on-screen coach 
named Kendra who provides nonverbal cues regarding the trainee’s choice of questions and 
statements. If further clarification is needed, the trainee can click “help” buttons that provide additional 
detail to clarify the interview question or the trainee’s response statement. For example, the coach 
gives the trainee a “thumbs down” sign if an inappropriate response is selected. If the trainee is unclear 
about the negative feedback, he or she can click the help button to get a more detailed written and 
verbal explanation about why the statement was inappropriate (e.g., “This statement focuses on a 
negative character trait; try focusing on your strengths”).  
C.1.g Individualized Customization: Prior to starting VR interviews, trainees complete an online job 
application, which includes questions about employment history and work skills. Trainees will specify 
one of fourteen different jobs (auto repair, cashier, child care, clerical, customer service, food service, 
greeter, inventory, janitorial, maintenance/grounds, security, stock clerk, tech support, web developer). 
This “on-line” application procedure is based on applications currently in use by employers such as 
Target and Home Depot. This practice prepares trainees to accurately complete online applications. 
Additionally, the application data will populate the list of questions from which VR-M will draw.  For 
example, a trainee may apply for a customer service position on the application, yet identify previous 
experience in inventory; Travis/Rita may ask, “I see from your resume that you have experience in 
inventory and are applying for a customer service position. Why are you looking to make that change?” 
This innovative feature allows trainees to customize their interview experiences to better prepare 
themselves for future interviews. 
C.1.h VR Transcripts: During or after the interview, trainees can view a transcript that replays individual 
exchanges or the entire conversation. If trainees are using the speech recognition feature, the transcript 
will also replay his or her recorded voice. This feature lets the trainee hear (and reflect upon) the tone 
of voice used to ask questions and make statements. The transcripts are color coded to reflect helpful, 
unhelpful, and neutral responses. Participants can click on the response to receive detailed information 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
8 of 24 
 
about why a response was helpful for hurtful to their job interview and learn how Travis/Rita understood 
their responses. 
C.1.i. Scoring and Summary Feedback: After each virtual interview, trainees receive scores in 
categories based on the learning goals. The scores are scaled form 0-100 and are computed via an 
algorithm, which accounts for the types of responses provided by the trainee throughout the course of 
the interview. Participants view this scoring feedback and if they score 80 or higher they are informed if 
they got the job or not. Trainees also receive feedback summaries of what they did well and where they 
need improvement. This feedback helps trainees understand the subtleties of their interview skills. 
D. Study Visit Procedures  
The research team will evaluate the initial effectiveness of the training by conducting a randomized 
controlled trial (RCT) at Ann Arbor Academy and Troy Beaumont Hospital Project Search.   
For Phase II, school vocational teachers will review school records to screen which of their students 
meet eligibility criteria in terms of age, diagnosis, and reading level. Teachers will keep this list 
internally and will not share with the study team. Once students are identified, the teachers will send a 
prepared email to the parents of the study with a copy of the consent form, and invite them sign the 
form and return it to study team via email or fax if they wish to participate and allow their child to 
participate. If a family does not have access to email/fax, the parents can return the signed copy of the 
permission form to the student's teacher at school. They will also be provided contact information to the 
study team so they can ask them any questions. This is not a recruitment email and the teachers will be 
acting in their paid capacity as a teacher and not a researcher. All parents have given consent to be 
emailed or called by the teacher for matters related to events at the school. They may also be invited to 
attend an information meeting hosted by the PI and/or study team.  During the informational meeting, 
the research team will provide a power point presentation to educate parents about the interview 
training and the study. After the meeting, parents will be invited to take home a parental permission 
form. They will be asked to sign and return this form to the research team via a scanned copy through 
email or fax. If a family does not have access to email/fax, the parents can return the signed copy of the 
permission form to the student's teacher at school. The teacher will place this form in a folder (kept in a 
locked cabinet in a locked office) and will give the hard copy to the research team. We will emphasize 
that we want parent permission forms returned and not student consent/assents, as those will be 
obtained during the study visit. 
The PI and or study team will be available to answer any questions the parents may have while 
reviewing the permission form. After the parents return the form, students under 18 years of age will be 
approached at school during Study Visit 1 to provide assent. Parents and students will be informed that 
choosing not to participate will not affect their or the student’s relationship with the school or teachers. 
For students 18 years or older, the research team will send an informational letter home informing the 
parents of the study and then the research team will approach the students 18-26 years old in order to 
obtain written informed consent. 
Also, teachers from all schools/programs will be recruited to participate in this study as part of an initial 
implementation. The teachers will be recruited beforehand during a phone call, or through the same 
educational meeting hosted at the school to inform parents and recruit students. 
D.1. Pre-Study Visit:  

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
9 of 24 
 
Parents and teachers will be asked to complete surveys prior to Study Visit 1. Surveys will be 
administered electronically, or by pen and paper. Completers will receive $15 (parents and/or 
teachers). 
Parents will be asked to complete: 
1. Student Background Form and Contact Info Form (age, gender, race, ethnicity, etc.) 
2. Behavior Checklist (ABCL or CBCL) (Achenbrock & Edelbrock, 2001). These surveys assess 
behavioral problems and social competencies as reported by parents and/or teachers. They are 
standardized in the field and have a long history of validity and reliability. If parents do not 
complete this form, then we will ask the teachers to do so (ABCL/CBCL/TRF). 
3. Social Responsiveness Scale 2nd Edition (SRS-2) (Constantine & Gruber, 2005), which is a 
teacher/parent based rating scale that measures the severity of autism spectrum symptoms 
Teachers will be asked to complete: 
1. Teacher Demographic Form 
2. Social Responsiveness Scale 2nd Edition (SRS-2) (Constantine & Gruber, 2005), which is a 
teacher/parent based rating scale that measures the severity of autism spectrum symptoms 
3. Behavior Checklist (ABCL/CBCL/TRF) if parents do not complete the form 
4. Student Background Form and Contact Info if parents do not complete the form 
D.2. Study Visit 1 & 2: 
These visits should last approximately 60 minutes each. Students will receive a $25 gift card for each 
visit. The consent and reading test will always be first, but the following activities may be done in a 
different order depending on the student availability. We aim to work best within the school and their 
availability. Study staff will complete the following tasks with student: 
• 
Assent/Consent 
• 
Reading Level: Wide Range Achievement Test (4th Edition) to assess current reading level. 
(If students do not pass a 3rd grade reading level, participation will end and they will be 
compensated $10). 
• 
Cognitive Assessments on an iPad using the National Institute of Health Tool Box 
Application 
• 
Baseline Employment Survey 
• 
Mood Survey 
• 
Job practice interview Role-Play (audio and video recorded) 
• 
Self-Report Surveys about: 
a. Anxiety 
b. Motivation 
c. Self-confidence 
 
D.4. Randomization: 
Students will be randomized into the VR-M group or the control group. This will be done at a ratio of 2:1 
(VR-M:control). Family and teachers will be informed by research team if student is active in the 
Intervention group or the Control group. The Control group will continue their educational services as 
usual and skip “visits” 3-16. They will return for the last in person Study Visit 17. Block randomization 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
10 of 24 
 
tables will be created prior to the study based on computer generated blocks of four. Blocks of four will 
ensure nearly equal groups. Envelopes or paper containing the randomization by sequential subject 
entry number will be created and stored, and then opened when a participant has completed pre-
randomization procedures. 
D.5. Study Visit 3: 
Students randomized to the VR-M group will receive a 45-60 minute orientation to use the Virtual 
Interview Training program. 
D.6. Study Visit 4: 
Students will train with the educational content. Study staff will be on site to help troubleshoot and 
answer questions if needed.  
D.7. Study Visit 5: 
Students will use one class period for orientation to learn to navigate and participant in the virtual 
interview. 
D.8. Study Visit 6-16: 
Students will complete up to 3 virtual job interviews per class period. Teachers will help implement this 
in their classrooms and may decide to utilize homework for the students where they are assigned to 
use the tool at home as well as in the school. 
D.9. Study Visit 17: 
This visit should last approximately 60 minutes. Students will receive a $25 gift card for this visit. If the 
student does not complete this study visit on the scheduled dates, the study team will continue to follow 
up with the student via phone or email five to six times during the follow up phase. If the student has not 
responded after three months, the study team will consider the student lost to follow up. If it is more 
convenient for the student, the study team will make arrangements to complete the study visit at the 
University of Michigan School of Social Work. Study staff will complete the following tasks with student: 
1. Employment Follow Up Survey 
2. Mood Survey 
3. Job practice interview Role-Play (audio and video recorded) 
4. Self-Report Surveys about: 
a. Anxiety 
b. Motivation 
c. Self-confidence 
5. Acceptability/Usability of VR-M 
 
D.10. 6 Month Follow Up: 
STUDY VISIT 
17:
Employment Follow Up 
Survey
(5 minutes)
Mood Survey
(5 minutes)
Job Practice Role Play
(25 minutes)
Self-Report Surveys
(15 minutes)
Acceptability/Usability 
Survey
(5 minutes)

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
11 of 24 
 
All students will complete a follow-up survey by phone or over email. We will send a letter reminding 
them of their 6-month follow up and asking them to give us a call. In this letter it will specify that the 
survey can be completed by students, parents, or students and parents together. We will also follow up 
with them by phone or email after the letter is sent by using an IRB approved script. This script will be 
similar to the IRB approved letter. Survey will include follow up questions about their employment. 
Students will receive $10 gift cards for this survey. 
 
D11. 9 Month Follow Up: (For COVID-impacted Autism Model School students only) 
Students and/or parents of students who participated from Autism Model School will be asked to 
complete a 9 month follow up survey by phone or email. The call will be optional and students/parents 
are allowed to opt out if they choose. We will read a consent/assent addendum script at the 9-month 
follow up call. We are adding the 9 month follow up to account for COVID-related issues in finding 
employment during the pandemic. 
E. Study Procedures for Phase II Pilot Implementation Evaluation and Teacher Involvement 
Approximately n=30 teachers will be involved in the delivery of VR-M and will be asked to complete a 
series of online surveys about VR-M including: Installation, Delivery, and Implementation and Feasibility 
regarding the potential barriers, and costs of delivery of the intervention to their students. These 
surveys will require approximately 30 minutes to complete electronically and include: 
1. Installation (5 minutes) 
2. Delivery (5 minutes) 
3. Adaptation (5 minutes) 
4. Cost Capture (5 minutes) 
5. Implementation (10 minutes) 
Teachers will receive a $25 gift card at the end of the study with the completion of these surveys. 
Teachers may also be asked to participate in an audio recorded semi-structured interview at the end of 
Implementation and will be compensated $20 gift card if they participate. 
Teacher involvement throughout the study will include reviewing records so they may inform families 
with students 16 years and older (via email or phone) that the school is partnering with the University of 
Michigan to evaluate the job interview tool. The teachers will share the research team's contact 
information with the families so they may contact the research team directly if they are interested in 
participating in the study. Teachers may also complete surveys on students who have consented or 
assented to participate. Parents will be asked to complete the following surveys first, and the teachers 
will serve as a backup to complete teacher-report versions if the parents do not participate. These 
surveys may include, for example, the ABCL/CBCL/TRF surveys, and the SRS survey, which are 
surveys that contain questions on student demographic information as well as behavioral information. 
They may also be asked to complete the Student Background survey if parents do not complete them. 
If teachers are unsure or do not know the answers to some of the questions regarding a student’s 
background, they will leave that question blank and will allow research staff to follow up with the 
students if needed. 
 
Also, the University of Michigan research team will train teachers how to teach students to use the job 
interview tool to the students who are randomly assigned to receive the virtual interview training. As 
transition teachers, the teachers are already providing vocational training to the students and so this 
type of training fits within the normal scope of their teaching efforts. The teachers are showing students 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
12 of 24 
 
how to log into the website, then the teachers do a demonstration of how students can use the tool via 
point and click within the interface. Then the teacher does a brief interview with the virtual hiring 
manager so that the students can see/learn how to interact with the virtual character (pointing and 
clicking as well as speaking to the virtual hiring manager). Then the students are expected to use the 
tool on their own in an independent fashion. That is the extent that we are training teachers to 'teach' 
the tool. Where the naturalistic setting comes into play is that part of our study is an implementation 
evaluation....beyond what we are training teachers to do......what happens during the natural course of 
delivery in the school setting. How do the students respond to the tool? Do the students find it usable 
and acceptable? Does the delivery go as planned or do teachers need to adapt the strategy for 
delivery? For example, we recommend students engage with the tool for 3 hours per week; however, 
some programs may only have 2 hours per week to give....or teachers planned to deliver the tool within 
the school's computer lab but perhaps that didn't work as well as they thought and they are now 
delivering the tool with chromebooks in the classroom. Any questions about technology-related glitches 
and such are covered by the research team who are onsite.  
 
F. Participant Timelines 
Phase II: Students enrolled in the Phase II randomized controlled trial will complete study visits 1 -16 
within approximately 5 weeks of enrolling in the study. Their participation will be complete after 
completing the 6 month follow-up.  Educational professionals and leadership enrolled in the Phase II 
pilot implementation will complete surveys for pilot implementation related to installation costs; delivery 
costs; delivery feasibility; barriers and facilitators to delivery; and a semi-structured interview around 
potential adaptations to delivery.   
G. Site Specific Regulations 
Ann Arbor Academy, Beaumont Hospital Troy Project Search, Southfield Public Schools, Lincoln Park 
Avenues Mixter Institute of Transition, and Autism Model School will not have any local scientific or 
ethical review structure that is required to approve the study and will use the University of Michigan’s 
IRB as the IRB of record. 
H. Vulnerable Population Safeguards 
Minors will participate in this study. A legal guardian will be required to provide parental consent and if 
the child participates in the study, they will be assented using the most appropriate form for their 
language and cognitive level. Pregnant women will not be excluded from the study. Their participation 
will not differ from participants who are not pregnant. 
 
5.0 Multiple Sites: 
This is a multi-site project funded by the National Institute of Mental Health and discretionary funding 
where the Principal Investigator is Dr. Matthew Smith located at the University of Michigan. He has 
previously worked directly with teachers at each site where the teachers provided information to 
families and then directed them to Dr. Smith so that he can facilitate the recruitment of students and 
families into the study to complete the feedback forms, and train local school staff at each study site to 
administer the intervention.  
A. University of Michigan School of Social Work is the coordinating site where all data will be 
entered, cleaned, stored, and analyzed 
 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
13 of 24 
 
B. Ann Arbor Academy is research site #1. Ann Arbor Academy has students from grade 4-12 with 
a variety of learning issues including students with autism spectrum disorder as well as other 
educational/ learning impairments.  Their college prep program prepares students for higher 
education and the workplace.  At this site, we will recruit research participants (both minors and 
adults) to participate in this research study. Although data will be collected at this site, research 
staff will transfer the collected data in a locked box to the UM research site for processing.   
 
C. Troy Beaumont Project Search is research site #2. Troy Beaumont Project Search is a 
business-led program for post high school age students with intellectual and developmental 
impairments between the ages of 18 and 26. The program is conducted entirely at the 
workplace where students receive instruction and training in a workplace setting. Although data 
will be collected at this site, research staff will transfer the collected data in a locked box to the 
UM research site for processing. 
 
D. Southfield Public School is research site #4. Data collection will primarily take place at the 
Kennedy Learning Center. Their goal is to provide their student population with a supportive and 
caring environment as well as to teach them skills to achieve their best potential. They work with 
a variety of students with disabilities. Although data will be collected at this site, research staff 
will transfer the collected data in a locked box to the UM research site for processing. 
 
E. Lincoln Park Avenues Mixter Institute of Transition is research site #5. Data collection will 
primarily take place at Mixter Institute of Transition. Their goal is to help students with autism 
improve their abilities to learn, function independently, and socialize so that they may realize 
their full potential to pursue post-secondary education goals such as joining the workforce 
and/or participating in their communities. Although data will be collected at this site, research 
staff will transfer the collected data in a locked box to the UM research site for processing. 
 
F. Autism Model School is research site #6. Data collection will primarily take place at Autism 
Model School. The school is a nonprofit community school that serves over 100 students ages 5 
to 22 years old with the goal to provide a nurturing environment and assist with developing the 
full potential of differently-abled students who have Autism Spectrum Disorder. 
 
G. When new research sites become available, the IRB will be notified and appropriate IRB 
documentation will be submitted. 
 
H. The school will disseminate an approval letter for the research team to conduct the study once 
the UM IRB has approved the protocols for the study and submitted the approval letter and a 
copy of the study protocol to each school for their records. 
 
I. The research coordinator will provide the PI with an electronic copy of the approved protocol 
and consent forms. The PI and research coordinator will be responsible to provide the Principals 
and lead educational staff/teachers at each study site with an updated version of the protocol 
and consent forms as well as a summary of changes that change the study procedures or 
change the potential risk of study participation (and any other related documentation) after 
receiving the IRB approval letter from the research coordinator. 
 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
14 of 24 
 
J. There are no local site investigators for this study. Based on the preliminary natural of this 
investigation, UM research personnel will supervise teachers facilitating the program evaluation 
completed by the students. UM research personnel will train on-site teachers to orient students 
on how to navigate the intervention as it is web-based. 
 
6.0 Incomplete Disclosure or Deception: 
This study will not be using deception. 
 
7.0 Recruitment: 
Phase I data collection is being completed at Northwestern University and will be sent to UM for 
analysis. Thus, there will be no Phase I data collection. 
 
For Phase II, school vocational teachers will review school records to screen which of their students 
meet eligibility criteria in terms of age, diagnosis, and reading level. Teachers will keep this list 
internally and will not share with the study team. Once students are identified, the teachers will send a 
prepared email to the parents of the study with a copy of the consent form, and invite them sign the 
form and return it to study team via email or fax if they wish to participate and allow their child to 
participate. If a family does not have access to email/fax, the parents can return the signed copy of the 
permission form to the student's teacher at school. They will also be provided contact information to the 
study team so they can ask them any questions. This is not a recruitment email and the teachers will be 
acting in their paid capacity as a teacher and not a researcher. All parents have given consent to be 
emailed or called by the teacher for matters related to events at the school. They may also be invited to 
attend an information meeting hosted by the PI and/or study team.  During the informational meeting, 
the research team will provide a power point presentation to educate parents about the interview 
training and the study. After the meeting, parents will be invited to take home a parental permission 
form. They will be asked to sign and return this form to the research team via a scanned copy through 
email or fax. If a family does not have access to email/fax, the parents can return the signed copy of the 
permission form to the student's teacher at school. The teacher will place this form in a folder (kept in a 
locked cabinet in a locked office) and will give the hard copy to the research team. We will emphasize 
that we want parent permission forms returned and not student consent/assents, as those will be 
obtained during the study visit. 
The PI and or study team will be available to answer any questions the parents may have while 
reviewing the permission form. After the parents return the form, students under 18 years of age will be 
approached at school during Study Visit 1 to provide assent. Parents and students will be informed that 
choosing not to participate will not affect their or the student’s relationship with the school or teachers. 
For students 18 years or older, the research team will send an informational letter home informing the 
parents of the study and then the research team will approach the students 18-26 years old in order to 
obtain written informed consent. 
Also, teachers from all schools/programs will be recruited to participate in this study as part of an initial 
implementation. The teachers will be recruited beforehand during a phone call, or through the same 
educational meeting hosted at the school to inform parents and recruit students. 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
15 of 24 
 
 
8.0 Consent Process: 
A. Consent/Assent Procedures: Parents will receive an email from teachers with information about 
the study and a copy of the parent consent form, along with contact information for study team 
staff. They are allowed to sign and email the consent form back to the study team and give 
permission for their child to participate. They may also return the signed form to the school, 
where study staff will pick it up. Alternatively, the research team will provide a permission form 
to all parents who attend an informational meeting, and provide a consent form to the teachers 
to hand out to any parents that may have been missing. These forms can be reviewed at home, 
signed, and returned to the PI via email or as a hard copy. 
 
After receiving permission from parents, the research team will obtain written assent/consent 
form students. This will be done during Study Visit 1. The teacher will escort the student to the 
private study visit room for the research staff to complete the assent/consent procedures. These 
procedures will take place in a private office or room to maintain student privacy and 
confidentiality. Students will then have an opportunity to review the form with the research staff 
prior to signing their consent/assent. 
 
B. Consent Team: The consent process will be conducted by the study team staff who have up to 
date PEERS training for the protection of human subject research. 
 
C. Consent Timeline: The assent/consent process will require approximately 10 minutes to read 
and up to 5-10 minutes to discuss to make sure that the participants fully understand the extent 
of their involvement in the study. 
 
D. Consent Process: Informed consent will be obtained in full accordance with the University of 
Michigan IRB guidelines.  
 
E. Re-consenting: Re-consenting will occur only if the study protocol involves changes in the data 
collection process, if there is a change in risk level for participation, or other changes the 
participant should be aware of. 
It is necessary that one parent or legal guardian give permission for the participation of a minor that is 
under his/her care. Written assent will be obtained from all participating minors under 18 years of age. 
Adult students will provide informed consent while meeting with the research team in a private room 
onsite at the school. All participants will be given the opportunity to ask questions regarding the study 
and their consent/assent prior to providing consent/assent. Participants will be informed of their right to 
withdraw from the research at any time without consequence. Informed consent is then documented as 
“obtained” in the study’s database. Adults have an option to email or provide a hard copy of the signed 
consent over to the research team. 
It is possible that students will turn 18 during the research. We will request a waiver of informed 
consent as these students provided their assent to participate based upon the same information that 
was provided in the adult consent form. There would be no change in protocol or in the follow up survey 
due to their change in age, and the consent form is the same as the assent form. 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
16 of 24 
 
 
9.0 Risks to Participants: 
This study involves students completing a battery of clinical, cognitive, and role-play assessments.  
There is no risk associated with participation in this study for students (both minor and adult) beyond 
potential boredom from navigating and using the computer program. All students will complete the 
studying within their high school setting.  
Teacher participants face even fewer risks as they will provide their feedback in the context of an 
internet-based survey.  
Confidentiality and safety procedures will be in full compliance with the Institutional Review Board of 
University of Michigan. 
When obtaining consent, participants are notified that they may stop participating at any time without 
repercussion. If consent is revoked in writing, the consent specifies the right to use or share information 
previously obtained as needed for the purpose of the study. 
 
10.0 Potential Benefits to Participants: 
Participants completing the intervention could possibly learn some new strategies to be more 
successful at Job Interviewing.  
Results from this study may lead to a better understanding of helping improve job interview 
skills for youth with disabilities. It is therefore possible that results of this study may aid in 
intervention strategies for youth with disabilities. 
 
11.0 Financial Compensation: 
Students will be paid up to $85 in gift cards for completion of assessments, self-report 
measures, role-play interviews, and a 6-month follow up. Parents can receive $15 in gift cards 
for completion of surveys. Teachers can receive up to $45, with an optional $15 if parents do 
not complete pre-visit surveys. 
A. Students: 
a. Study Visit 1:  
 
 
 
 
$25 gift card 
b. If ineligible after WRAT (Study Visit 1) 
 
$10 gift card 
c. Study Visit 2:  
 
 
 
 
$25 gift card 
d. Study Visit 17: 
 
 
 
 
$25 gift card 
e. 6 Month Follow Up:  
 
 
 
$10 gift card 
B. Parents: 
a. If they complete pre-visit surveys  
 
$15 gift card 
C. Teachers: 
a. If parents do not complete pre-visit surveys 
$15 gift card 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
17 of 24 
 
b. Teacher implementation/evaluation surveys 
$25 gift card 
c. Semi-structured Interview  
 
 
$20 gift card 
 
12.0 Provisions to Protect the Privacy Interests of Participants: 
To minimize stress and fatigue during testing, participants will be assessed in an encouraging 
and accepting atmosphere. They can take breaks during the assessment process and may 
discontinue the assessment if they do not wish to continue. The research team have been 
trained in the administration of the research measures and have undergone training 
procedures to help minimize the stress this may cause.  
The research assessments and intervention use will be conducted in a private or semi-private 
computer lab or educational room setting for the students. For teachers, the feedback form will 
be conducted in a setting of the participant’s choosing. All methods of correspondence will be 
secure, including telephone, voicemails, text messages, or email. Participants will provide 
informed consent with respect to their preferred method of communication. In addition, we will 
use a personal identification number (PIN) to protect participant privacy by labeling all study 
documents with the PIN rather than any identifying information. The use of the PIN will help 
keep participant enrollment confidential.  
Access to protected information will be limited to the PI and research team members. There 
will be a document located on the UM servers (password protected behind firewalls) that links 
PINs to study participant names. The PI and research team will have access to this file in order 
to coordinate study visits and track participant progress to complete the study. REDCap, a 
HIPAA compliant and encrypted database may also be used to host protected participant 
information. 
While testing participants at the schools, we will record data on either secure electronic 
equipment (i.e., laptop or tablet) or paper forms. The information linking the identifiable 
information with the testing forms will never be stored on the portable equipment. Data will be 
uploaded to the server and removed from the device as soon as possible. All precautions will 
be taken to protect the electronic devices from theft or loss. If a device is lost, this will not pose 
any risk to the participant since no data can be linked to their identifiable information. 
All forms and materials will be kept in a locked file in a locked room accessible only to research 
personnel. Confidentiality procedures will be in full compliance with the Institutional Review 
Board of University of Michigan.  Any confidential data obtained at the study site will be 
transferred back to the School of Social Work via a lock box to be secured into a locked file 
and in a locked room. 
 
13.0 Confidentiality and Data Management: 
 
Description of Security measure to Protect Data Sources: If REDCap is not being used, paper research 
records will be created at the schools and labeled with the PIN, where participants may provide consent 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
18 of 24 
 
and complete study measures. All study data will be stored in a locked cabinet within a locked office at 
the school. These records will serve as the source for electronic data entry. Once per week, the 
research records will be transferred via a locked box to the UM research team where the data will be 
cleaned, re-entered into the electronic database, and stored in a locked drawer in a locked room. All 
records will be de-identified after study completion. All paper research data will be stored on password 
protected servers for a period of 7 years. 
 
All participant documents at UM will be labeled with the participant’s PIN. Each page of each document 
will be labeled with the PIN. Documents will be organized into a study binder that is also labeled with 
the participant PIN. All Binders and Study Documents will be stored in locked file drawers within a 
locked room. 
 
Quality Assurance Measures: All participants will be recruited using strategies, documents, and text 
approved by the University of Michigan Institutional Review Board (IRB). The research team will 
regularly hold meetings to discuss the effectiveness of approved recruitment strategies and if new 
strategies should be reviewed by the IRB and then implemented. The study coordinator will screen 
participants for eligibility using formal study forms. The PI will regularly audit accrual to ensure that 
participants meet eligibility criteria and that the study enrollment is consistent with the projected 
enrollment targets agreed upon with NIMH. In addition, the study coordinator will audit all study files to 
ensure that all required study data is completed on each form, and that there is no missing data. Data 
will be double entered on electronic REDCap data entry forms to maintain the validity and integrity of 
the collected data. The REDCap system is accessible via a virtual private network that is password 
protected and behind firewalls. The data will be transported from the schools to UM in a locked storage 
box. To protect confidentiality, all data will be numerically coded using a personal identification number 
(PIN), and information linking the PIN to the subject’s name will be kept in a secured file cabinet and 
office. All video files will be encrypted prior to being stored electronically. In addition, computer data 
files will be stored on password-protected computers and communication among the staff will use PINs, 
not names. No information concerning data will be presented with participant names. The biostatistician 
and project coordinator will also perform all necessary checks and controls to ensure the reliability and 
validity of the data, including monitoring data collection and collection procedures, data storage, data 
management, and data analysis.   
 
Data Sharing: This study involves modifying an intervention to help adolescents with autism spectrum 
disorder. We will share de-identified data with a neuropsychologist in order to have reports written up 
and provided to participants and/or participant parents. This is stated in the consent form, and 
participants have the option to share the information with educational staff or to not share with 
educational staff. If younger than 18, both parent and student must agree to share data in order for it to 
be shared; if one person says no, then the reports will not be shared with educational staff or schools. 
Also, a data sharing plan is currently in place with NIH to provide all clinical data collected in our 
autism-related research to the National Database for Autism Research (NDAR). This includes 
submission of de-identified descriptive data from the samples proposed for collection on a semi-annual 
basis (January 15th and July 15th) and submission of all other experimental data upon completion of 
the three year award period. The goal of this data sharing policy is to facilitate autism spectrum disorder 
research by giving the autism research community access to publicly made data.  Existing mechanisms 
and protocols for data submission already established by NDAR will be employed in the dissemination 
of our data, to include a computerized database with de-identified data, and strict quality control, 
described as follows: 
 
A. Computerized database – With assistance from the study’s biostatistician, we will develop an 
enrollment strategy to obtain the information necessary to generate a Global Unique Identifier 
(GUID) for each participant and an electronic database in REDCap to manage this information 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
19 of 24 
 
and all data derived from this project. The data entered on each subject will include results from 
diagnostic evaluations for autism on all subjects (using SRS-2), neuropsychological data from 
the NIH Toolbox, role-play scores, 6-month employment data, and vocational training data. All 
data in this project will be verified and provided for release in .csv (comma separated values) 
submission files to NDAR. 
B.  Quality control - We currently have in place a variety of measures to ensure and check the 
quality of clinical data. All interviewers participate in a formal training course and all clinical data 
are verified through evaluations by an editor and diagnostician. Data collectors also receive 
extensive training in the administration of all measures and assessments.  
C. Human Subjects Issues - We will use consent language that will clearly stipulate: (1) disclosure 
that clinical research data will be deposited to this NIH autism database; (2) assurance that 
such data will be provided without personal identifiers; (3) disclosure that analyses of these 
data will be conducted by other scientists not included within the current research team; and (4) 
disclosure that there is no plan to provide subjects with any financial benefits from commercial 
products derived from the data. All consent materials will be in full accordance with IRBs of 
participating institutions. 
 
14.0 Data Analysis: 
Phase I Data Analysis. The qualitative interview data will be analyzed inductively and iteratively using 
thematic analysis and the constant comparative approach43, 44 to identify emergent themes regarding 
potential modifications to VR. The research team will analyze a subset of transcripts from  the 
qualitative data to iteratively develop codes until a final codebook is agreed upon by the team and 
intercoder reliability is achieved; codes will then be applied to transcripts45, 46. 
Phase II Data Analysis. For Phase II, between-group differences in the characteristics of the VSU+VR-
M and VSU groups will be examined using Chi-Square analysis (categorical variables) or an analysis of 
variance (ANOVA) (scaled variables). Variables with significant between-group differences will be 
examined as covariates in all subsequent analyses. Measures of dispersion (mean, standard deviation, 
variance, range) and distribution (kurtosis, skewness) will be computed. Scatterplots will be generated 
to review bivariate relationships. 
Phase II emphasizes the evaluation of VR-M training on interviewing skills and attaining employment. 
We recognize that the heterogeneity of ASD may affect these outcomes. We will assess the effect of 
VR-M on changes in job interview skills, self-confidence, motivation, and anxiety between pretest and 
postest. We will determine the likelihood of attaining a job for the two conditions by 6 months, a “real-
world” outcome and the issue of greatest concern. The mean number of interviews per week and time-
to-employment for the two groups at 6-month follow-up will be examined. Additional analyses will 
evaluate the mechanisms of VR-M that lead to improved interview performance and positive 
employment outcomes via mediators (improved self-confidence, improved motivation, reduced anxiety) 
and moderators (e.g., ASD-related symptoms, depressive symptoms, and neurocognitive function) that 
might affect responsiveness to VR-M. This data will inform our understanding of how interventions like 
VR-M affect outcomes. We will use repeated-measures ANCOVAs with ASD-related symptoms, 
depressive symptoms, neurocognitive function, and social cognitive function (now listed as 
heterogeneity variables) as covariates. We will also use ANCOVA to evaluate the change over time, 
with posttest scores as the dependent variables while covarying for pretest variables. Cohen’s d effect 
sizes will characterize the observed differences. We will use a log-rank test comparing time-to-position 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
20 of 24 
 
in the two groups to determine a hazard ratio to test if time-to-position differs between groups. The 
effects of covariates, including blocking factors, on time to employment will be assessed using Cox 
proportional hazards regression47. 
Exploratory Data 
We will explore potential mechanisms for VR-M effectiveness (i.e., improved interview skills, attain a 
job), by first analyzing mediators (i.e., interview skills, self-confidence, and anxiety) and moderators 
(e.g., ASD heterogeneity variables) using boot strapping (for mediators)48 and linear regression (for 
moderators)49. Second, we will explore potential mechanisms for a lack of improvement in interview 
skills and failure to attain a job) by examining group differences and associations with ASD 
heterogeneity between groups who did or did not have poor outcomes. Third, we will collect exploratory 
implementation data by interviewing vocational counselors and school administrators (n=10) about: 1) 
potential barriers and facilitators for VR-M implementation and scalability; and 2) how VR-M might 
reduce staff time required to deliver services. We will analyze this exploratory implementation data 
iteratively using thematic analysis and the constant comparative approach43, 44 to identify emergent 
themes regarding the barriers and facilitators of implementing VR-M. 
 
14.0 Data Monitoring Plan to Ensure the Safety of Participants: 
This is a clinical trial and will have a data safety and monitoring plan. It will be strictly monitored by the 
PI who regularly reviews testing and data management procedures to ensure protection of human 
subjects. We follow IRB guidelines to minimize risk and report problems. We plan to establish a Data 
Safety and Monitoring Board to meet annually during the active phase of the RCT. The DSMB will 
review the research protocol and plans for data safety and monitoring, adverse events, risk 
management policies, and preliminary data analysis. We plan to invite University of Michigan faculty 
members: Joseph Himle, Addie Weaver, University of Wisconsin: Lauren Bishop-Fitzpatrick, and 
community partner: Joselyn Reese. We are required by our funding agency to report results annually.  
All data reported is de-identified prior to entry and analysis. We initiate protocol revisions with the IRB 
when necessary. All electronic data is stored on a secure server managed by University of Michigan 
School of Social Work, and access to the server requires a password and netID, and is fire walled, 
requiring VPN connection if off campus (per IT security protocols). REDCap will be another storage 
site. An additional password is needed to access identifiable data stored on this secure server and on 
REDCap. Further, all data collected in the field is stored on encrypted drives and also identified only by 
ID number. All paper data is stored in locked filing cabinets in our lab at UM. Data analysis is completed 
by approved project personnel.  
 
15.0 Qualifications to Conduct Research and Resources Available: 
Overview and anticipated contribution to success of the proposed work: The University of Michigan 
(UM) is one of the most resource intensive universities in the world, with state-of-the-art computing, 
statistical, library, and research consultation resources readily available to faculty researchers. The 
facilities and resources necessary for the proposed work are all currently available to Dr. Smith and his 
team. Below, the key units and facilities are described in detail to provide information about the specific 
contribution of each.    

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
21 of 24 
 
University of Michigan 
Office: The facilities and resources necessary for the proposed work are all currently available to Dr. 
Smith and his team. Dr. Smith and the study staff have offices at the School of Social Work. The study 
staff and Dr. Smith have meeting and office space within the Curtis Research Center where Dr. Smith is 
the contributing faculty.  
Computer: The P.I. and the research staff have computer resources for data analysis and word 
processing for manuscript preparation.  
Scientific Environment:  
A. School of Social Work: The UM is a leader in social work education and research and is ranked 
first in the U.S. News and World Report rankings of social work programs.  The interdisciplinary 
Joint Doctoral Program in Social Work and Social Science stands alone in the nation. This 
distinctive program trains academics, researchers, and practitioners to meet the challenges of 
society by utilizing theories and research from anthropology, economics, political science, 
psychology and sociology. There are approximately 600 MSW and approximately 75 Ph.D. 
students enrolled each term in the School of Social Work programs.  For this project, Dr. Smith 
will regularly involve graduate students as work-study research assistants and they will facilitate 
data cleaning and double data entry in addition to other project supports. 
 
B. The Vivian A. and James L. Curtis Research and Training Center: Through the generous 
contribution of the Curtis family, The Vivian A. and James L. Curtis Research and Training 
Center is funded to foster interdisciplinary externally funded research and UM-community 
evaluation research partnerships to advance knowledge in the areas of mental health, 
substance abuse, and health. The Curtis Center’s research agenda focuses in studying health 
and mental health disparities and on the conduct of translational research to ameliorate these 
disparities. Center support includes study design consultation, advanced statistical consultation, 
grant and manuscript writing assistance, research space, and several other services.  Expert 
consultation is provided by a multidisciplinary group of experienced, federally funded 
researchers. For this project, the Curtis Center will serve as the hub for the project staff and Dr. 
Smith. We will utilize its research consultation and research support resources to increase 
chances of success in completing the proposed project. 
 
C. CTSA – (Clinical Translational Science Awards) and Michigan Institute for Clinical and Health 
Research (MICHR):  The University of Michigan is the recipient of an NIH CTSA award, which 
has its home in MICHR (http://www.michr.umich.edu). MICHR provides a comprehensive 
institutional source of support and infrastructure for clinical and translational researchers at the 
University of Michigan. It is designed to serve any investigator, in any school of the University, 
performing clinical or translational research. It serves to connect and integrate education, career 
development, infrastructure, and to catalyze research that spans the laboratory, the clinic, and 
the community. MICHR is organized in fourteen unique resource programs, focusing on the 
following domains: novel methods, technical core, Clinical Research Unit (MCRU), 
education/career development, biomedical informatics, regulatory support, pilot grants, 
biostatistics, research ethics, pediatric research, community engagement, clinical translation 
sciences, and health disparities research.  Research Support Services include research 
development, data management and project management and monitoring (PMM) of federally 
funded, multi-center trials. MICHR offers all investigators expert consultation, budgeting and 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
22 of 24 
 
administrative support for project development, funding for pilot studies, the Engage participant 
recruitment portal, and support for clinical research management system, among other research 
development services. For this project, MICHR will provide assistance in data management, 
based on the REDCap platforms.  REDCap is an electronic data capture (EDC) system that is 
secure, HIPAA compliant, and web-based. This easy-to-use, no-cost tool for the University of 
Michigan Clinical Researchers is intended to replace Microsoft Excel and Access. Using 
REDCap's stream-lined process for rapidly developing databases, an investigator may create, 
design, and manage clinical research data online. Further, REDCap provides automated export  
procedures  for  seamless data downloads to common statistical packages (SPSS, SAS, Stata, 
R), as well as a built-in scheduling calendar, ad hoc reporting tools, and advanced electronic 
data capture form features, such as branching logic, file uploading, and calculated fields. 
 
D. Center for Statistical Consultation and Research (CSCAR): CSCAR is a service and research 
unit under the administrative oversight of the VPR that provides integrated, comprehensive 
statistical consulting services covering all aspects of research design and analysis from initial 
study design through presentation of research findings.  CSCAR provides free statistical support 
to UM faculty and research staff and is located approximately three blocks from the School of 
Social Work.  Among the CSCAR services provided are those related to proposal presentation 
and study design (e.g., power and sample size calculations), dataset consulting (e.g., database 
design, transferring datasets across platforms and different software packages), choice of 
statistical methods, use of statistical software (e.g., SAS, SPSS, SYSTAT, S-Plus, BMDP, JMP, 
LISREL, and AMOS on Windows, UNIX, and Macintosh platforms), interpretation of and 
presentation of results, collaborative research, workshops, remote consultation using 
“whiteboard” software, and geographic information systems consultation. With regard to 
computing resources, CSCAR provides PCs and Mac computers with the latest versions of a 
large number of software programs such as STATA, MPLUS, AMOS, MATLAB, R, SAS, SPSS, 
ArcGIS, and ArcView GIS, among others. The CSCAR Library PC also has a variety of 
statistical software packages available for use including a sample size and power analysis 
package. For this project, CSCAR will provide any statistical consultation needed to increase 
the chances of success of this project. 
 
Dr. Smith’s Autism Research Recruitment Pipeline 
Dr. Smith developed a recruitment network across the State of Illinois while he was a faculty member at 
Northwestern University. These programs consist of vocational service providers that include: Alexian 
Brothers Autism Spectrum & Developmental Disorders Resource Center, Anixter Center, Aspiritech, 
Chicagoland Autism Connection, Easter Seals Metropolitan Chicago, Have Dreams, New Connections 
Academy & South Campus, and No Boundaries. In addition, the Illinois Department of Human Services, 
Division of Rehabilitation Services (DRS) oversees 70+ standardized vocational training program 
across the State of Illinois.  
As part of the proposed project, we will continue our collaborative work with our community partners in 
order to disseminate research findings to families and the broader autism community. We have several 
methods to share research findings with the autism community and general public (e.g., presentations 
at community meetings and family conferences; television interviews). For instance, in addition to 
presenting at conferences for research professionals, we will share findings with stakeholders by giving 
presentations at the Autism Society of Illinois, Chicagoland Autism Connection, and during quarterly 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
23 of 24 
 
calls for the Illinois DRS. Additionally, family reports will be distributed to all study participants and 
made publicly available on our partner’s and UM websites. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
24 of 24 
 
References 
1. 
CDC. Prevalence of autism spectrum disorders: Autism and Developmental Disabilities 
Network. In: Prevention CfDCa, ed. Vol 61: Morbidity and Mortality Weekly; 2008:1-19. 
2. 
Shattuck PT, Narendorf SC, Cooper B, Sterzing PR, Wagner M, Taylor JL. Postsecondary 
education and employment among youth with an autism spectrum disorder. Pediatrics. 
2012;129(6):1042-1049. 
3. 
Liptak GS, Kennedy JA, Dosa NP. Social participation in a nationally representative sample of 
older youth and  young adults with autism. Jouranl of Developmental & Behavioral Pediatrics. 
2011;32(3):1-7. 
4. 
Hendricks D. Employment and adults with autism spectrum disorders: challenges and strategies 
for success. J Vocat Rehabil. 2010;32(2):125-134. 
5. 
Volkmar FR, State M, Klin A. Autism and autism spectrum disorders: diagnostic issues for the 
coming decade. J Child Psychol Psychiatry. 2009;50(1-2):108-115. 
6. 
Gerhardt PF, Lainer I. Addressing the needs of adolescents and adults with autism: a crisis on 
the horizon. J Contemp Psychother. 2011;41:37-45. 
7. 
Higgins KK, Koch LC, Boughfman EM, Vierstra C. School-to-work transition and Asperger 
Syndrome. Work. 2008;31(3):291-298. 
8. 
Shattuck PT, Roux AM, Hudson LE, Taylor JL, Maenner MJ, Trani JF. Services for adults with 
an autism spectrum disorder. Can J Psychiatry. 2012;57(5):284-291. 
9. 
Smith MJ, Fleming MF, Wright MA, Jordan N, Humm LB, Olsen D, Bell MD. Job Offers to 
Individuals With Severe Mental Illness After Participation in Virtual Reality Job Interview 
Training. Psychiatr Serv. 2015;66(11):1173-1179. 
10. 
Smith MJ, Fleming MF, Wright MA, Roberts AG, Humm LB, Olsen D, Bell MD. Virtual reality job 
interview training and 6-month employment outcomes for individuals with schizophrenia seeking 
employment. Schizophr Res. 2015;166(1-3):86-91. 
11. 
Smith MJ, Fleming MF, Wright MA, Losh M, Humm LB, Olsen D, Bell MD. Brief report: 
vocational outcomes for young adults with autism spectrum disorders at six months after virtual 
reality job interview training. J Autism Dev Disord. 2015;45(10):3364-3369. 
12. 
AutismSpeaks. Employment Think Tank Report (Internet).  
http://www.autismspeaks.org/sites/default/files/as_think_tank_exec_summary_web1.pdf. 
Accessed 07/31/2014. 
13. 
Baron-Cohen S, Wheelwright S. The empathy quotient: an investigation of adults with Asperger 
syndrome or high functioning autism, and normal sex differences. J Autism Dev Disord. 
2004;34(2):163-175. 
14. 
Baron-Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I. The "Reading the Mind in the Eyes" 
Test revised version: a study with normal adults, and adults with Asperger syndrome or high-
functioning autism. J Child Psychol Psychiatry. 2001;42(2):241-251. 
15. 
Paul R, Orlovski SM, Marcinko HC, Volkmar F. Conversational behaviors in youth with high-
functioning ASD and Asperger syndrome. J Autism Dev Disord. 2009;39(1):115-125. 
16. 
Geller L, Greenberg M. Managing the transition process from high school to college and 
beyond: challenges for individuals, families, and society. Social Work in Mental Health. 
2010;8(1):92-116. 
17. 
Berney T. Asperger syndrome from childhood into adulthood. Advances in Psychiatric 
Treatment. 2004;10:341-351. 
18. 
Sasson NJ, Nowlin RB, Pinkham AE. Social cognition, social skill, and the broad autism 
phenotype. Autism. 2012. 
19. 
Garcia-Villamisar D, Hughes C. Supported employment improves cognitive performance in 
adults with Autism. J Intellect Disabil Res. 2007;51(Pt 2):142-150. 
20. 
Garcia-Villamisar D, Ross D, Wehman P. Clinical differential analysis of persons with autism in 
a work setting: a follow-up study. J Vocat Rehabil. 2000;14(3):183-185. 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
25 of 24 
 
21. 
Garcia-Villamisar D, Wehman P, Navarro M. Changes in the quality of autistic people's life that 
work in supported and sheltered employment. A 5-year follow-up study. J Vocat Rehabil. 
2002;17(4):309-312. 
22. 
Howlin P, Alcock J, Burkin C. An 8 year follow-up of a specialist supported employment service 
for high-ability adults with autism or Asperger syndrome. Autism. 2005;9(5):533-549. 
23. 
Lawer L, Brusilovskiy E, Salzer MS, Mandell DS. Use of vocational rehabilitative services 
among adults with autism. J Autism Dev Disord. 2009;39(3):487-494. 
24. 
Mawhood L, Howlin P. The outcome of a supported employment scheme for high-functioning 
adults with autism or Asperger syndrome. Autism. 1999;3(3):229-254. 
25. 
Taylor JL, McPheeters ML, Sathe NA, Dove D, Veenstra-Vanderweele J, Warren Z. A 
systematic review of vocational interventions for young adults with autism spectrum disorders. 
Pediatrics. 2012;130(3):531-538. 
26. 
Bell MD, Weinstein A. Simulated job interview skill training for people with psychiatric disability: 
feasibility and tolerability of virtual reality training. Schizophrenia Bulletin. 2011;37 Suppl 2:S91-
97. 
27. 
Smith MJ, Boteler Humm L, Fleming MF, Jordan N, Wright MA, Ginger EJ, Wright K, Olsen D, 
Bell MD. Virtual Reality Job Interview Training For Veterans with Posttraumatic Stress Disorder. 
Journal of Vocational Rehabilitation. 2015;42:271-279. 
28. 
Smith MJ, Ginger EJ, Wright K, Wright MA, Taylor JL, Humm LB, Olsen DE, Bell MD, Fleming 
MF. Virtual reality job interview training in adults with autism spectrum disorder. J Autism Dev 
Disord. 2014;44(10):2450-2463. 
29. 
Smith MJ, Ginger EJ, Wright M, Wright K, Boteler Humm L, Olsen D, Bell MD, Fleming MF. 
Virtual reality job interview training for individuals with psychiatric disabilities. J Nerv Ment Dis. 
2014;202(9):659-667. 
30. 
Taylor JL, Seltzer MM. Employment and post-secondary educational activities for young adults 
with autism spectrum disorders during the transition to adulthood. J Autism Dev Disord. 
2011;41(5):566-574. 
31. 
Cederlund M, Hagberg B, Billstedt E, Gillberg IC, Gillberg C. Asperger syndrome and autism: a 
comparative longitudinal follow-up study more than 5 years after original diagnosis. J Autism 
Dev Disord. 2008;38(1):72-85. 
32. 
Eaves LC, Ho HH. Young adult outcome of autism spectrum disorders. J Autism Dev Disord. 
2008;38(4):739-747. 
33. 
Hagner D, Cooney BF. "I do that for everybody': Supervising employees with autism. Focus on 
Autism and Other Developmental Disabilities. 2005;20(2):91-97. 
34. 
Romoser M. Malemployment in autism. Focus on Autism and Other Developmental Disabilities. 
2000;15(4):246-247. 
35. 
Morgan L, Leatzow A, Clark S, Siller M. Interview Skills for Adults with Autism Spectrum 
Disorder: A Pilot Randomized Controlled Trial. J Autism Dev Disord. 2014. 
36. 
Klin A, Saulnier CA, Sparrow SS, Cicchetti DV, Volkmar FR, Lord C. Social and communication 
abilities and disabilities in higher functioning individuals with autism spectrum disorders: the 
Vineland and the ADOS. J Autism Dev Disord. 2007;37(4):748-759. 
37. 
Couture SM, Penn DL, Losh M, Adolphs R, Hurley R, Piven J. Comparison of social cognitive 
functioning in schizophrenia and high functioning autism: more convergence than divergence. 
Psychol Med. 2010;40(4):569-579. 
38. 
Kindermann SS, Brown GG, Zorrilla LE, Olsen RK, Jeste DV. Spatial working memory among 
middle-aged and older patients with schizophrenia and volunteers using fMRI. Schizophr Res. 
2004;68(2-3):203-216. 
39. 
Humphrey N, Symes W. Responses to bullying and use of social support among pupils with 
autism spectrum disorders (autism) in mainstream schools: a qualitative study. Journal of 
Research in Special Education Needs. 2010;10(2):82-90. 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
26 of 24 
 
40. 
Koegel RL, Koegel LK, Carter CM. Pivotal teaching interactions for children with autism. School 
Psychology Review. 1999;28(4):576-594. 
41. 
Lovannone R, Dunlap G, Huber H, Kincaid D. Effective educational practics for students with 
autism spectrum disorders. Focus on Autism and Other Developmental Disabilities. 
2003;18(3):150165. 
42. 
Council NR. Educating children with autism. In: autism Coeifcw, ed. Washington, D.C.: National 
Academy Press; 2001. 
43. 
Guest G, MacQueen KM, Namey EE. Applied Thematic Analysis. Thousand Oaks, CA: Sage 
Publications, Inc.; 2012. 
44. 
Lincoln Y, Guba E. Processing Naturally Obtained Data. Naturalistic Inquiry. London: Sage 
Publications; 1985:332-356. 
45. 
Bernard HR, Ryan GW. Text analysis: qualitative and quantitative methods. In: Bernard HR, ed. 
Handbook of Methods in Cultural Anthropology. Walnut Creek, CA: Altamira Press; 1998. 
46. 
Kurasaki K. Intercoder reliability for validating conclusions drawn from open-ended interview 
data. Field Methods. 2000;12:179-194. 
47. 
Cox DR, Oakes D. Analysis of Survival Data. New York: Chapman & Hall; 1984. 
48. 
Preacher KJ, Hayes AF. SPSS and SAS procedures for estimating indirect effects in simple 
mediation models. Behav Res Methods Instrum Comput. 2004;36(4):717-731. 
49. 
Hayes AF. Introduction to Mediation, Modration, and Conditional Process Analysis: A 
Regression-Based Approach. New York: Guilford Press; 2013. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

MODIFYING (PHASE I) AND EVALUATING (PHASE II) VR FOR YOUTH  
27 of 24