Enhancing Mental Health Care by Scientifically Matching Patients to Providers' Strengths
Phase: NA
Conditions: Mental Illness
Sponsor: University of Massachusetts, Amherst

 
1 
Enhancing Mental Health Care by Scientifically Matching Patients to Providers’ Strengths 
Patient-Centered Outcomes Research Institute (PCORI) Contract #IHS-1503-28573 
ClinicalTrials.gov #NCT02990000 
The Study Protocol 
Document Date: October 10, 2019 
 
Table of Contents 
 
A. Full original study protocol submitted to, and approved by, the University of Massachusetts 
Amherst Institutional Review Board (November 2016); pp. 2-35 
 
B. Summary of the 1st formal modification to the PCORI contract (July 2017), and the full 
revised study protocol (1st revision) submitted to, and approved by, the University of 
Massachusetts Amherst Institutional Review Board (November 2017); pp. 36-69 
 
C. Summary of the 2nd formal modification to the PCORI contract (September 2018), and the 
full revised study protocol (2nd revision) submitted to, and approved by, the University of 
Massachusetts Amherst Institutional Review Board (March 2018); pp. 70-106 
 
D. Summary of the 3rd formal modification to the PCORI contract (January 2018); no revision to 
the study protocol; p. 107 
 
E. Final study consent form; p. 108-112 
 
 
 

 
2 
A. Full original study protocol submitted to, and approved by, the University of 
Massachusetts Amherst Institutional Review Board (November 2016) 
 
PROTOCOL 
APPLICATION FORM 
SOCIAL, BEHAVIORAL, AND EDUCATIONAL FULL BOARD 
HUMAN SUBJECTS IN SOCIAL, BEHAVIORAL, AND EDUCATIONAL 
RESEARCH 
 
University of Massachusetts Amherst (UMass) 
Institutional Review Board (IRB) 
 
Protocol ID: 2016-3401 
Title: Enhancing Mental Health Care 
 
Protocol Director: Michael J. Constantino 
Degree: PhD 
Title: Professor 
Department Name: Psychological & Brain Sciences 
Mailing Address: 612 Tobin Hall, 135 Hicks Way 
Phone: 5-1388; Fax: 5-0996 
E-mail: mconstantino@psych.umass.edu 
Human Subjects Training Completed? yes 
 
Administrative Contact: Aggie Mitchkoski 
Degree:  
Title: Clerk III 
Department Name: Psychological & Brain Sciences 
Mailing Address:  
Phone: 5-2383; Fax:  
E-mail: aggie@psych.umass.edu 
 
Subject Populations(s) Checklist 
Yes/No 
 
Minors (under 18) 
Pregnant Women 
Cognitively Impaired or Decisionally Challenged 
Older individuals (75 and over) 
Healthy Volunteers 
Students/Employees 
International Populations 
Prisoners 
Other (i.e., any population that is not specified above) 
 
N 
N 
N 
N 
N 
N 
N 
N 
Y 
 

 
3 
Other: Subjects will include two mental health care stakeholder groups: (1) therapists 
affiliated with Atrius Health’s Behavioral Health Department, or Atrius-affiliated partners, who 
are providing outpatient psychotherapy, and (2) adult patients receiving psychotherapy for 
varied mental health complaints from the participating therapists. Atrius Health, a formal 
subcontract to UMass on this project, is an innovative health care organization serving patients 
in eastern and central Massachusetts. 
 
Study Location(s) Checklist 
Yes/No 
 
University of Massachusetts Amherst 
Baystate Medical 
University Health Services 
Hartford Hospital 
Other (Specify other Study Locations) 
 
Y 
N 
N 
N 
Y 
 
Other: All study operations will be coordinated through Dr. Michael Constantino’s (PI) 
Psychotherapy Research Lab at UMass Amherst. Subject data will be collected in multiple 
outpatient mental health clinics that are within the Atrius Health network proper, or a 
preferred partner of it. Atrius Health is a Pioneer Accountable Care Organization, and one of the 
largest multi-specialty practices in the US. The organization treats more than 32,000 patients a 
year in its behavioral health division. All treatment sites are in eastern and central 
Massachusetts. 
 
General Checklist 
Yes/No 
 
Training Grant? 
Funded Study (or proposal submitted to sponsor)? 
Cooperating Institution(s)? 
Federally Sponsored Project? 
Human blood, cells, tissues, or body fluids (tissues)? 
Subjects will be paid for participations? 
 
N 
Y 
Y 
Y 
N 
Y 
 
Cooperating Institution(s): (1) University at Albany, SUNY (Dr. James Boswell; Co-PI and 
subcontract); (2) Outcome Referrals Institute, Inc. (ORI; Dr. David Kraus; Co-PI and subcontract); 
and (3) Atrius Health (Dr. Samuel Nordberg; Co-PI and subcontract). As part of this protocol, I 
would like to initiate UMass entering into an Institutional Authorized Agreement (IAA) with 
each of the three subcontracts. A previous IAA (for a separate project) is in place with my 
collaborator, Dr. Boswell, at University at Albany, SUNY. He and his research assistants have 
already affiliated with UMass through the Collaborative IRB Training Initiative (CITI) program. As 
additional personnel across the three subcontracts complete their CITI training (as part of the 
project start-up period), I will submit their names and CITI certificates as formal changes to this 
protocol. 
 
Funding Checklist 

 
4 
Grants/Contracts: 
 
Funding Administered By: UNIVERSITY 
PGCA#: 1503-28753 
GAID#:  
Funded By: Patient-Centered Outcomes Research Institute 
1828 L Street, NW, Suite 900 
Washington, DC 20036 
Phone: (202) 827-7700 | Fax: (202) 355-9558 
info@pcori.org 
Principle Investigator: Michael J. Constantino 
Grant/Contract Title: Enhancing Mental Health Care by Scientifically Matching Patients to 
Providers’ Strengths 
 
Are the contents of this protocol the same as described in grant/contract proposal? Y 
Is this a training grant? N 
Are any subcontracts issues under this grant? Y 
 
Fellowships – None 
Gift Funding – None 
Dept. Funding – None 
Other Funding – None 
 
1. Purpose of the study 
 
a. Provide a brief lay summary of the purpose of the study.  
Research has shown that mental health care (MHC) providers differ significantly in 
their ability to help patients. In addition, providers demonstrate different patterns of 
effectiveness across symptom and functioning domains. For example, some 
providers are reliably effective in treating numerous patients and problem domains, 
others are reliably effective in some domains (e.g., depression, substance abuse) yet 
appear to struggle in others (e.g., anxiety, social functioning), and some are reliably 
ineffective, or even harmful, across patients and domains. Knowledge of these 
provider differences is based largely on patient-reported outcomes collected in 
routine MHC settings. 
 
Unfortunately, provider performance information is not systematically used to refer 
or assign a particular patient to a scientifically based best-matched provider. MHC 
systems continue to rely on random or purely pragmatic case assignment and 
referral, which significantly “waters down” the odds of a patient being 
assigned/referred to a high performing provider in the patient’s area(s) of need, and 
increases the risk of being assigned/referred to a provider who may have a track 
record of ineffectiveness. This research aims to solve the existing non-patient-
centered provider-matching problem. 

 
5 
 
Specifically, we aim to demonstrate the comparative effectiveness of a scientifically-
based patient-provider match system compared to status quo pragmatic case 
assignment. We expect in the scientific match group significantly better treatment 
outcomes (e.g., symptoms, quality of life) and higher patient satisfaction with 
treatment. We also expect to demonstrate feasibility of implementing a scientific 
match process in a community MHC system and broad dissemination of the easily 
replicated scientific match technology in diverse health care settings. The 
importance of this work for patients cannot be understated. Far too many patients 
struggle to find the right provider, which unnecessarily prolongs suffering and 
promotes health care system inefficiency. A scientific match system based on 
routine outcome data uses patient-generated information to direct this patient to 
this provider in this setting. In addition, when based on multidimensional 
assessment, it allows a wide variety of patient-centered outcomes to be represented 
(e.g., symptom domains, functioning domains, quality of life). 
 
b. What does the Investigator(s) hope to learn from the study?  
The goal of this project is to test the effectiveness of an innovative, scientifically-
informed patient-therapist referral match algorithm based on MHC provider 
outcome data. We will employ a randomized controlled trial (RCT) to compare the 
match algorithm with commonplace pragmatic referral matching (based on provider 
availability, convenience, or self-reported specialty). Psychosocial treatment will 
remain naturalistically administered by varied providers (e.g., psychologists, social 
workers) to patients with mental health concerns. We hypothesize that the scientific 
match group will outperform the pragmatic match group in decreasing patient 
symptoms and treatment dropout, and in promoting patient functional outcomes, 
perceived treatment credibility, outcome expectation, and care satisfaction, as well 
as therapeutic alliance quality. Doing so will establish the match algorithm as a 
mechanism of effective patient-centered MHC, and will suggest that this 
scientifically derived patient-provider matching intervention can be integrated into 
MHC systems to aid in treatment decision making, as well as increase 
personalization. 
 
2. Study Procedures 
 
a. Describe all study procedures. 
We will compare the efficacy of naturalistic treatment either with or without the aid 
of scientific matching to a provider with a double-blind RCT. The project will involve 
two main phases. First, we will conduct a baseline assessment of consenting 
therapists’ performance to determine their relative strengths and weaknesses in 
treating the problem domains measured by a multidimensional outcome tool. This 
period will establish our therapist sample pool and inform the match manipulation 
(a match will represent a patient being assigned to a therapist who has empirically 

 
6 
demonstrated during the baseline phase that he or she is stably effective at treating 
patients with the same presenting complaint). 
 
Second, and after the baseline period, new consenting outpatients will be randomly 
assigned to the match (experimental) or no match (control) condition. Participating 
clinic administrators will collaborate with the research team to apply the 
randomization protocol. Treatment outcome will be assessed through the patient’s 
actual termination point or 16 weeks, whichever comes sooner (we will also conduct 
a follow-up outcome assessment at 1 year after the point of termination on a 
randomly selected subsample). Outside of being matched to a therapist from a 
short-list of providers who have demonstrated (during the phase 1 baseline) reliable 
success in treating the patient’s primary problem area, and completing study-
specific measures for which participants will receive monetary compensation, 
treatment will be delivered as usual (the short list still allows for pragmatic 
considerations like availability and administrator assignment options). Additional 
methodological details follow. 
 
Phase 1: To inform the match condition, we will first conduct a 6-month baseline 
assessment of participating therapists’ performance (across 15 new cases each) to 
determine their strengths in treating behavioral health domains measured by the 
primary outcome measure on which the match algorithm is based – the 
Treatment Outcome Package (TOP; Kraus, Seligman, & Jordan, 2005), described 
below in the listing of relevant phase 1 attachments to this protocol. Developed and 
processed by our Co-PI (Dr. Kraus) and his subcontractor company, Outcome 
Referrals, Inc. (ORI), the TOP will be administered across the Atrius provider network 
and its preferred partners; as Atrius’ care model already incorporates routine 
outcomes monitoring, we can leverage the existing infrastructure within this 
practice-based research network to support this study with little to no extra burden 
on administrators, providers, and patients. Being assigned 15 new cases in a 6-
month baseline period is readily achievable for the fulltime therapists in the 
network. 
 
Our target sample is 44 therapist participants and each therapist’s next 15 patients 
assigned after consenting to participate in the study. Therapists will be 
psychologists, social workers, and licensed mental health counselors. Recruitment 
will be coordinated among our UMass-employed project coordinator (PC), clinic staff 
members, and clinic administrators, and will involve presenting information about 
the study (both phases 1 and 2) to providers through flyers, verbal script, or email 
(included as phase 1 attachments to this protocol). Interested therapist participants 
will meet or speak via teleconference with the PC to learn about the study 
details/procedures and to provide formal consent through an online baseline survey 
to which they will be directed (this all occurs prior to the baseline period for 
establishing TOP “report cards” that will inform the match in phase 2). Therapists 
will be told that the study is examining various referral processes that will not affect 

 
7 
their delivery of treatment-as-usual. They will be informed that they will be blind to 
the specific nature of the referral manipulation in phase 2, but will be fully debriefed 
following the entire study and offered an opportunity to provide feedback on their 
experience. Therapists will also be informed of the study-specific assessments in 
which their study patients will engage in both phase 1 and phase 2 (though they will 
not have access to these research data at any time). Therapists will also need to 
consent to completing a few study-specific measures at phase 1 baseline and 
throughout treatment with each patient during the phase 2 RCT (these are described 
below in the listing of relevant phase 2 attachments to this protocol) For phase 1, 
therapists will be compensated with a $20 Amazon gift card for a one-time 
completion of the phase 1 baseline forms. For phase 2, they will be compensated 
$50 per patient (again in the form of Amazon gift cards) for completing several 
measures of therapy process at multiple points throughout each patient’s 
treatment. 
 
During phase 1, patients will only be engaged in the study by virtue of registering 
into the TOP system at their treatment baseline, and completing the TOP at 
baseline, week 8, and their termination point or week 16, whichever comes sooner 
(to mimic the definition of treatment outcome in the RCT phase discussed below). 
Given their involvement in this one element of the research protocol, patients will 
need to give the staff at their clinic verbal consent to be contacted by the PC or 
research assistant (RA). The staff members will collaborate with the research team 
to track these patients being treated by study-consenting therapists during this 
phase 1 part of the entire study. Once contacted via telephone, the PC or RA will 
explain the nature of the patient’s involvement, which will simply involve registering 
on the TOP system, and completing the TOP main outcome measures at baseline, 
week 8, and termination. The patient will then be directed to a secure online system 
to complete their consent form and their baseline TOP forms. For the week 8 and 
the posttreatment TOP administrations, the PC/RA will email specific links that will 
direct patients to the measure (The PC/RA will also follow-up with phone call 
reminders if needed). The online system was developed and is maintained by our 
subcontractor, ORI. For this minimal time burden, patients will be entered into a 
raffle to win one of ten $25 Amazon gift cards. These 660 patients will be involved in 
phase 1 only. Thus, they will be debriefed by the PC or RA after they terminate their 
treatment-as-usual. 
 
Relevant phase 1 attachments to this protocol:  
 
(1) Therapist recruitment materials: flyers, verbal script, or email 
(2) Therapist consent form and baseline phase 1 measures packet: 
 
Provider Characteristics Form (PCF). This measure was developed by the research 
team to assess therapist demographic information, clinical experience, degree type, 
percent time seeing various patient types/diagnoses, any specialty training they 

 
8 
have received, and dimensional ratings of the influence of various theoretical 
orientations on their treatment approach. 
 
Therapist Perceived Strengths (TPS). This measure was developed by the research 
team to assess therapists’ beliefs about their effectiveness in treating the various 
TOP domains when uninformed of their data-driven TOP track record. This measure 
will allow us to examine how accurate therapists are in perceiving their own 
strengths and weaknesses. 
 
(3) Patient phase 1 recruitment verbal script 
(4) Patient phase 1 consent form and baseline phase 1 measures packet: 
 
TOP-Consumer Registration Form (TOP-CR; Kraus et al., 2005). The TOP-CR will be 
used routinely during the phase 1 baseline (and the phase 2 RCT) to assess patient 
demographics. On this form, patients indicate their age, gender, ethnicity, marital 
status, income level, employment status, religious identification, education level, 
general health status, and medical and mental health treatment history. 
 
TOP-Clinical Scales and Case Mix (TOP-CS & TOP-CM; Kraus et al., 2005). This is the 
primary measure in our study; it will be used to establish the therapist report cards 
during the baseline phase to inform the match manipulation in phase 2. It also tracks 
patient outcomes. The TOP-CS consists of 58 items assessing 12 symptom and 
functional (including strengths) domains (risk-adjusted for case mix variables 
assessed via 37 items on the companion TOP-CM, such as divorce, job loss, 
comorbidity): work functioning, sexual functioning, social conflict, depression, panic 
(somatic anxiety), psychosis, suicidal ideation, violence, mania, sleep, substance 
abuse, and quality of life. Global symptom severity is assessed by summing all items 
or by averaging the z-scores (i.e., standard deviation units relative to the general 
population mean) across each of the 12 clinical scales. Domain-specific symptom 
severity is quantified as the individual z-scores for each clinical scale using general 
population means and standard deviations for the conversion. The TOP-CS has been 
shown to have excellent factorial structure, as well as good test-retest reliability 
across all scales. It is sensitive to change while possessing limited floor and ceiling 
effects (Kraus et al., 2005). The TOP also has demonstrated good convergent validity, 
with scales like the Beck Depression Inventory (Beck, Steer, & Brown, 1996) and the 
Brief Symptom Inventory (Derogatis, 1975). 
 
**Note: The TOP-CS and TOP-CM are the same measures that patients will complete 
at treatment week 8 and their termination point or week 16, whichever comes 
sooner. 
 
(5) Patient phase 1 debriefing form: 
 

 
9 
This form will be appended at the end of the final of the three patient online 
assessments. It will provide additional information about the study and repeat 
contact information should patients wish to discuss the study further. 
 
Phase 2: At the conclusion of baseline phase 1, participating therapists will each 
have had TOP-based report cards generated to inform matching in phase 2. The 
therapists themselves will not see their report cards; rather, this information will be 
used by the research team with regard to the match manipulation. Phase 2 marks 
the beginning of new patient recruitment into the RCT. The patient population will 
be adult men and women (age 18-65) largely referred by Atrius primary care and 
obstetrics/gynecology for triage and treatment through Atrius’ behavioral health 
specialty practice. Our study target sample size is 264 patients (6 per therapist). The 
only non-age exclusion criterion for this study will be patients who are not the 
primary, informed decision-maker for their care. 
 
Once referred into the Atrius behavioral health system per usual care process, the 
front desk staff at the participating clinics will ask potential patient subjects if they 
are interested in hearing more about the study were they to be treated within Atrius 
with individual outpatient psychotherapy (versus being referred out or receiving 
care that does not involve individual outpatient psychotherapy). Next, a triage 
clinician at the clinics will determine that patients make their own treatment 
decisions and will receive individual therapy as part of their “in-house” treatment 
plan; if so, the patient is study-eligible. Atrius clinic/triage staff will provide the 
research team with a daily list of referrals (via direct telephone contact or through a 
secure server) of patients who have provided verbal consent to be contacted about 
the study and who have been determined to be appropriate for relevant treatment 
within the Atrius system. The PC will then reach out to patients who provided verbal 
consent to be contacted. 
 
If a patient agrees to engage in a consent/baseline assessment session, the PC will 
schedule a telephone diagnostic interview with a trained graduate clinical 
psychology RA (employed at either UMass or University at Albany). The RA will first 
review the study details/procedures and obtain consent through an online baseline 
survey to which the patient will be directed (again administered through the ORI 
platform). Patients will be informed that their participation in the trial will largely 
mimic the same treatment that they would receive if they were not participating. All 
patients will also be told that the study is examining various referral processes that 
will not affect their treatment; they will be kept unaware of the specific nature of 
the referral manipulation, but will be told that they will be fully debriefed following 
the study and offered an opportunity to provide feedback on their experience. 
Patients will be asked to remain with the same therapist through at least 16 weeks 
of treatment; however, if they do request a transfer earlier, their data will not be 
analyzed for the study given the confound of multiple outpatient psychotherapy 

 
10 
providers. This will be treated as a dropout point for the sake of the trial. Patients 
will also be informed that they will complete all assessments that are part of their 
routine clinical care, as well as several study-specific measures, for which they will 
be compensated $50 total. 
 
Once consent is obtained, the RAs will then administer the DSM-5 updated M.I.N.I. 
7.0.2 International Neuropsychiatric Interview (M.I.N.I.; Sheehan et 
al., 1988), described below in the listing of relevant phase 2 attachments to this 
protocol. The interviews will be audio-recorded through a secure web-conferencing 
platform. This will allow for subsequent review by an independent graduate RA to 
determine diagnostic reliability. Following the interview, patients will complete the 
self-report measures remaining in the online baseline survey (i.e., the TOP-CR, TOP-
CS, and TOP-CM, a brief measure of global distress, a measure of existential 
isolation, and a measure of interpersonal problems, all described below in the listing 
of relevant phase 2 attachments to this protocol). 
 
After the full baseline assessment, patients will be randomly assigned to condition 
(scientifically informed matched vs. pragmatic match) with a participating Atrius 
provider. The PC, unaware of therapist baseline performance, will generate the 
randomization sequences using an online random generator. Understanding 
variability in site size/traffic, we will strive to recruit patients and providers from as 
many sites as possible in order to increase generalizability. Within condition, 
patients will be assigned sequentially to the site-specific therapists until they reach 
their study quota. Patients in the match condition will be assigned to therapists who 
have a demonstrated strength (derived from the baseline period) in treating, at a 
minimum, the patient’s highest self-reported distress domain on the TOP-CS. 
Beyond the minimal match on the most elevated TOP-CS domain, our match 
algorithm will attempt to match patients to therapists on as many TOP-CS 
dimensions as possible, ultimately providing the clinics with at least several well-
matched choices for assignment within the match condition. In order to preserve 
this level of choice for the clinics, there will be natural variability in the number 
of well-matched domains (some patients matched only on the minimum 1 TOP-CS 
domain, others matched on 2 or more domains). The match variability across both 
conditions will allow us to measure degree of match dimensionally as a moderator 
variable of our main treatment effect. Therapists will also be unaware of their 
patient’s treatment condition (double blind), and they will treat both matched and 
non-matched patients (i.e., they will be crossed over the two conditions to minimize 
administrative disruptions). In the low-probability event that there is no therapist 
meeting minimal match criteria for a patient in the match condition, that patient will 
be removed from the primary study analyses (though will, of course, still be offered 
treatment-as-usual at the clinic) and replaced with the next patient where a match 
does exist. As described in our power analysis below, we are oversampling in order 
to account for these “dropouts,” or removed data points. 
 

 
11 
In addition to the baseline assessments already described, patients will be assessed 
via online surveys at regular intervals during treatment (RAs will email hyperlinks to 
these surveys with reminders to complete them at the appropriate time intervals; 
RAs will also follow up with phone calls if needed). These during-treatment 
assessments will include the TOP-CS and measures of existential isolation and 
interpersonal problems at every odd-numbered week after the start of treatment, as 
well as global distress, therapeutic alliance quality, perceived treatment credibility, 
and outcome expectation after every even-numbered week (all measures of these 
constructs are described below in the listing of relevant phase 2 attachments to this 
protocol). During treatment, therapists will also be asked to complete their 
respective versions of the alliance and credibility/expectation measures (also at 
even-numbered weeks). RAs will email hyperlinks to these online surveys 
with reminders to complete them at the appropriate time intervals; RAs will also 
follow up with phone calls if needed. 
 
As a reminder, in both conditions, the providers will deliver treatment 
naturalistically (i.e., with no manipulation or influence from the research team). For 
the sake of the RCT, “treatment outcome” will be considered the point at which 
treatment terminates, or 16 weeks, whichever comes sooner. For some patients, 
mutual termination will occur in response to outcome data-informed clinically 
significant improvement. For others, treatment will terminate at the Atrius-defined 
endpoint for a given individual treatment track (e.g., 16 sessions for major 
depression or generalized anxiety). After the 16th week, or the termination session if 
it comes sooner, patients will complete posttreatment measures: the TOP-CS and 
TOP-CM, a measure of treatment satisfaction, a brief measure of global distress, a 
measure of existential isolation, and a measure of interpersonal problems (all 
described below in the listing of relevant phase 2 attachments to this protocol). 
Therapists will also document the nature of termination (measure described below 
in the listing of relevant phase 2 attachments to this protocol). Also at 
posttreatment, as defined by the trial, patients will undergo a repeat diagnostic 
telephone assessment (i.e., a RA-administered M.I.N.I., as described above). 
 
We will also conduct a follow-up outcome assessment at 1 year after the patient’s 
own termination on a randomly selected subsample of 40 patients. Patients can 
easily be tracked in coordination with Atrius’ health portal; further, patients will 
have provided consent for this follow-up contact (should they be randomly chosen 
for it). At this assessment point, patients will again complete online the TOP-CS and 
TOP-CM, the brief measure of global distress, the measure of existential isolation, 
and the measure of interpersonal problems. 
 
Note that all self-report measures (for both patients and therapists) at all timepoints 
will be completed on Wi-Fi-connected tablets, or on home computers, through ORI’s 

 
12 
secure web-based platform. The TOP has its own dedicated website and HIPAA-
compliant, secure server, and all other study-specific measures will be integrated 
into the TOP administration process. 
 
We predict that the scientific match group will outperform the no-match group to a 
clinically significant degree on TOP outcomes, global symptomatology, and 
interpersonal problems. We also expect that the match group will be more effective 
in promoting alliance quality and fostering more positive patient perceptions of 
treatment credibility and outcome expectation, all of which are established 
correlates (and candidate mechanisms) of positive treatment outcomes. Finally, we 
expect there to be less patient dropout in the match condition, and higher patient 
treatment satisfaction. Secondarily, we will examine 4 potential moderators of the 
expected between-group treatment effects on the primary TOP outcomes: (a) 
patient race (as it may be that the match algorithm is particularly 
potent, and an important responsiveness tool, for historically understudied or 
underrepresented patients), (b) degree of match of therapist strengths to patient 
problems (rated dimensionally as a ratio given that therapists can be matched on 
more than just the minimum 1 domain, and the elimination of harmful matches for 
any distressed domain reported by the patient), (c) patient distress severity, and (d) 
complexity of patient presenting problem. Thus, we will test if matching is only, or 
particularly, effective under the conditions of a central patient characteristic, a 
multiple domain match, and/or for patients with the most severe or complex 
pathology. As noted, we will also assess therapists’ self-perceived strengths on the 
TOP domains. We expect to replicate previous literature showing that therapists are 
poor judges of their own efficacy, tending to underestimate negative effects and 
overestimate positive effects with their patients (Lambert, 2011), which would 
further underscore the importance of a data-driven match process. 
 
Finally, for a subsample of stakeholders, we will conduct posttrial exit interviews (Ns 
= 5 patients, 5 therapists) to gather invaluable input on how to be responsive to the 
study findings in terms of dissemination, implementation, and policymaking, 
including the potential importance of integrating diagnosis, provider age, race, or 
gender into subsequent matching approaches. We will recruit stakeholders in order 
of completion until we reach our target Ns (therapists can only be involved once 
they have treated all 6 of their study patients). There are no other 
inclusion/exclusion criteria for the exit interviews; we will simply stop asking if 
participants are interested once we have reached our target Ns. This is consistent 
with the study consent forms, which clearly state that interested participants may 
be selected to engage in the interview. 
 
Fully reflecting stakeholder engagement, and to eliminate any biases or power 
dynamics introduced by the PIs or their research staff, Advisory Board 
members (with appropriate credentialing for working with human subjects) will 
conduct the individual interviews. The PIs (Constantino & Boswell) will train 3 

 
13 
Advisory Board members on qualitative interviewing, and each will administer 1-2 
pilot interviews as part of the training, plus 5 study interviews. The interviews will be 
conducted and audio-recorded via a secure webconferencing service and will last 
approximately 45-60 minutes. Participants will be compensated a $100 gift card for 
their time. RAs will transcribe the interviews, removing any identifying patient 
information. These RAs will also conduct qualitative analysis of these text-based 
data. 
 
Relevant phase 2 attachments to this protocol: 
 
(1) Patient phase 2 recruitment verbal script 
(2) Patient phase 2 consent form and phase 2 baseline measures packet: 
 
TOP-CRF, TOP-CS, and TOP-CM. All described previously. 
 
Symptom Checklist-10 (SCL-10; Rosen, Drescher, Moos, & Gusman, 1999). To 
evaluate outcome with an index separate from the TOP (to test convergence and 
enhance the validity of any between-condition effects), we will also assess global 
distress with the SCL-10, a 10- item, well-validated and widely used self-report 
inventory that assesses psychological wellbeing. 
 
Existential Isolation Scale (EIS; Pinel et al., 2014). To assess this isolation subtype, 
participants will complete the EIS, a 6-item scale that requires participants to rate 
the extent to which they agree with items such as “I often have the same reactions 
to things as other people around me do” (reverse-coded) and “Other people usually 
do not understand my experiences” and “People often have the same ‘take’ or 
perspective on things that I do” (reverse-coded). Participants respond using a 7-
point scale. The EIS has high internal consistency, and has been validated extensively 
(Pinel et al., 2014). 
 
Inventory of Interpersonal Problems-32 (IIP-32; Horowitz, Alden, Wiggins, & Pincus, 
2000). To assess interpersonal problems, participants will complete the 32-item 
circumplex version of the IIP. This widely used instrument reflects interpersonal 
inhibitions and excesses, with each item rated on a 5-point scale. Higher total scores 
indicate more interpersonal problems. The IIP-32 also has 8 subscales (Domineering, 
Vindictive, Intrusive, Cold, Socially Inhibited, Nonassertive, Overly Accommodating, 
and Self-Sacrificing) that comprise a circumplex of problematic interpersonal 
behavior around the main interpersonal dimensions of affiliation and control. Like 
the original measure (Horowitz, Rosenberg, Baer, Ureno, & Villansenor, 1988), the 
IIP-32 has evidenced good psychometric properties. 
 
(3) RA-administered diagnostic assessment (baseline and posttreatment): 
 
M.I.N.I. 7.0.2 International Neuropsychiatric Interview (M.I.N.I.; Sheehan et al., 

 
14 
2016). The M.I.N.I. is a brief, structured diagnostic interview for DSM-5 and 
International Classification of Diseases (ICD; World Health Organization, 2008) 
psychiatric disorder classification. With its administration time of approximately 15 
minutes, the M.I.N.I. is the psychiatric interview of choice in clinical trials and 
epidemiological studies. Despite its brevity, its psychometric properties compare 
favorably to longer instruments like the Structured Clinical Interview for DSM (SCID; 
First, Spitzer, Gibbon, & Williams, 1996). As part of the diagnostic evaluation, the 
RAs will complete the Clinical Global Impression (CGI), a widely used observer-rated 
scale that includes a 0-7 judgment of illness severity for which higher scores indicate 
more extreme illness. 
 
(4) Patient phase 2 during-treatment measures: 
 
TOP-CS, SCL-10, EIS, IIP-32. All described previously. 
 
Working Alliance Inventory-Short Form, patient version (WAI-SF-P; Tracey, 
& Kokotovic, 1989). The WAI is the most widely used alliance measure, assessing 
patient-therapist agreement on the goals and tasks of treatment, and the quality of 
their relational bond. This 12-item short form, assessing these dimensions from the 
patient’s perspective, has demonstrated sound psychometric properties. 
 
Credibility/Expectancy Scale, patient version (CEQ; Devilly & Borkovec, 2000). The 
CEQ is the most widely used and psychometrically sound measure of the patient’s 
perceived logicalness of a given treatment and expectation for the personal efficacy 
of that treatment. 
 
(5) Therapist phase 2 during-treatment measures: 
 
Working Alliance Inventory-Short Form, therapist version (WAI-SF-T; Tracey & 
Kokotovic, 1989). This is the parallel version of the WAI-SF described above, though 
now as rated from the therapist’s perspective. 
 
Credibility/Expectancy Scale, therapist version (CEQ; Devilly & Borkovec, 2000). This 
is the parallel version of the CEQ described above, though now as rated from the 
therapist’s perspective (i.e., the therapist’s sense of how logical the patient sees the 
treatment and how optimistic the patient is about receiving benefit from it). 
 
(6) Patient phase 2 posttreatment measures: 
 
TOP-CS, TOP-CM, SCL-10, EIS, IIP-32. All described previously. 
 
TOP-Satisfaction with the Treatment Process (TOP-STP; Kraus et al., 2005). This 32-
item measure assesses patient’s satisfaction with their provider, the treatment they 
received, and the treatment milieu (e.g., staff, other patients, etc.). 

 
15 
 
(7) Therapist phase 2 posttreatment measure: 
 
Nature of Termination Form (NTF). This measure was developed by the research 
team to assess the nature of patients’ termination from the provider’s open-ended 
perspective, as well as through a choice format of unilateral/patient-generated, 
unilateral/therapist-generated, or mutual. Therapists can also describe in an open-
ended format any unusual or noteworthy circumstances that may have led to the 
termination of therapy with this client (e.g., transfer of client to another therapist). 
 
(8) Patient phase 2 subsample follow-up measures: 
 
TOP-CS, TOP-CM, SCL-10, EIS, IIP-32. All described previously. 
 
(9) Stakeholder exit interview protocols (patient and therapist versions) 
 
(10) Patient phase 2 debriefing form 
 
(11) Therapist phase 1 and 2 debriefing form 
 
b. State if audio or video taping will occur. Describe what will become of the tapes 
after use, e.g., shown at scientific meetings, erased. Describe the final disposition 
of the tapes. 
For the baseline patient assessment during phase 2, RAs will administer via 
telephone the M.I.N.I., which will be audio-recorded. This will allow a different RA to 
review the recording and to make independent diagnostic and symptom severity 
determinations. With these two sets of ratings, we can then calculate interrater 
reliability on baseline diagnosis. 
 
Audio recordings from the baseline diagnostic assessments will be digitally stored 
through the secure web-conferencing service. All data will be encrypted and 
password protected. Only the necessary research team members will know the login 
and password information and have the capacity to access the recordings. When it is 
time to analyze the recordings for reliability coding, designated, trained RAs will also 
have access to the recordings. The RAs, of course, will have completed the 
mandatory ethics training in human subjects’ research, data management, and 
HIPAA compliance. These RAs will be independent evaluators who will not have 
access to other therapist or patient data. The recordings themselves will not be 
labeled with any identifiable information. The PI will routinely monitor the collection 
and analysis of recorded data. 
 
After the recordings have been assessed for diagnostic reliability, the files will be 
securely deleted by the sponsored project contract term date of 7/14/20. No audio 
data or identifiable text data stemming from the recordings will be presented at 

 
16 
meetings or in published articles. Only the reliability coefficients will be 
disseminated with the results of the full trial. 
 
c. State if deception will be used. If so, provide a rationale and describe debriefing 
procedures. Submit a debriefing script in Section #11 (Attachments). 
Although the protocol does not involve deception, it does involve incomplete 
disclosure in Phase 2 given that participants are not given all of the information 
about the study until debriefing. Thus, in the debriefing form, we now provide 
participants the opportunity to withdraw their data upon learning the full scope of 
the research. 
 
3. Background 
 
a. Describe past findings leading to the formulation of the study. 
Research has consistently identified significant variability in skill and outcomes 
between therapists (Baldwin & Imel, 2013; Boswell et al., 2013; Westra, 
Constantino, Arkowitz, & Dozois, 2011), even when therapists utilize an empirically 
supported treatment (EST). In fact, differences between treatment providers 
account for a greater portion of treatment outcome variance than the specific 
interventions delivered in controlled trials (Krause, Lutz, & Saunders, 2007; 
Wampold & Imel, 2015). Thus, improvements in MHC can occur by identifying 
effective providers in addition to promoting ESTs (Kraus et al., 2007). 
 
In the largest study to date on this topic, our team investigated therapists’ 
naturalistic treatment outcomes over many different problem domains (e.g., 
depression, anxiety, substance use, mania, sleep) in a sample of 6,960 patients and 
nearly 700 providers (Kraus, Castonguay, Boswell, Nordberg, & Hayes, 2011). The 
majority of therapists demonstrated a differential pattern of effectiveness 
depending on the problem domain, and therapist domain-specific effectiveness 
correlated poorly across domains suggesting that therapist competencies may be 
domain-specific, rather than reflecting a core attribute or general underlying 
therapeutic skill. Importantly, although some therapists demonstrated effectiveness 
over multiple problem domains, no therapists demonstrated reliable effectiveness 
across all domains. Further, a small, but notable 4% of the therapists did not 
demonstrate effective outcomes on any domain. These data suggest that in any 
population of therapists (payer network, hospital, or community mental health 
system), there is an opportunity for behavioral health to do what medicine did 
decades ago – encourage provider specialization. Virtually every clinician has an area 
where they are above average (82-96%; Kraus et al., 2011, 2016), and our research 
suggests that if they specialize to their unique skills, population-level outcomes (i.e., 
symptom reduction, behavior change, increased functionality) will improve 
dramatically. This would reflect a major, and likely highly impactful shift to current 
MHC systems. 
 

 
17 
However, patients and referrers are typically unaware of the unique track record 
(“report cards”) of local-area providers, which represents a critical gap in knowledge 
transfer within the MHC system. Without systematically collecting and disseminating 
performance report cards, stakeholders (e.g., patients, therapists, administrators 
responsible for case assignment, primary care physicians) lack vital information on 
which to base MHC choices and referral decisions, and that can inform personalized 
treatment (Boswell, Constantino, Kraus, Bugatti, & Oswald, 2015). Conversely, there 
is potentially immense advantage to matching patients to providers based on 
scientific outcome data (Constantino, Boswell, Bernecker, & Castonguay, 2013). 
 
Consistent with this notion, the Institute of Medicine (IOM) has 
made recommendations to: (a) customize care based on the patient’s needs, (b) 
share knowledge, (c) engage in data-driven decision-making, (d) promote 
transparency (including information on performance and patient satisfaction; Kohn, 
Saxena, Levav, & Saraceno, 2004), and (e) use valid and reliable assessment 
instruments to assess progress and to aid decision-making. The IOM has also 
recommended that MHC patients be provided with information on the quality of 
practitioner care (e.g., provider report cards) and use this information when making 
treatment decisions. Importantly, we have survey data that point to MHC patients, 
therapists, and administrators endorsing such applied knowledge transfer as a high 
priority (Boswell et al., 2015). Provider track record report cards are meaningful data 
to the MHC patient population, as are the mental health benefits that could stem 
from being well matched to provider. 
 
We have developed over the past 20 years an innovative, technology-based 
mechanism/intervention to deliver report cards and drive this match concept within 
a patient-centered MHC model (Kraus et al., 2011). Our longitudinal data suggest 
that our match algorithm, based on our multidimensional outcome tool (the TOP) is 
efficacious for MHC outcomes. In addition to our study highlighted above (Kraus et 
al., 2011), a more recent prospective study of 59 therapists and 3,540 patients 
resulted in a between-treatment controlled Cohen’s d effect size of .80 (Kraus et 
al., 2016). Each therapist’s first 30 patients were used to classify a therapist’s skills in 
the 12 domains of symptoms and functioning as either statistically above average, 
average, or below average. The best matching algorithm functioned as follows: for 
each new, successive patient, he or she was classified as well-matched if the risk of 
harm was eliminated (i.e., the therapist was not below average when treating any 
elevated domain) and the therapist was above average in treating the patient’s 
three most out-of-the-norm domains (e.g., depression, suicidality, and panic). Poorly 
matched patients had below average outcomes, with small effect sizes (d = .30) 
Well-matched patients, by contrast, achieved very large pre- vs. posttreatment 
effect sizes of d = 1.19. These data lend strong support that the proposed 
comparative effective research (CER) will yield similar results (i.e., increased efficacy 
and reduced harm) in realigning the skills of a large population of therapists in one 
of the forerunner Accountable Care Organizations (our partner Atrius) when 

 
18 
matching empirically derived therapist skills with patient need. The 
technology/intervention is well established, it has demonstrated efficacy, and awaits 
investigation in a well-powered randomized controlled trial (RCT). 
 
4. Subject Population 
 
a. State how many subjects you propose to use and state the rationale for the 
proposed number. 
For the primary 3-level hierarchical model assessing treatment condition effects at 
the patient level on linear change rates within patients, we used Raudenbush and 
Liu’s (2001) formula as incorporated in the Optimal Design program to determine 
the minimum numbers of therapists and patients needed to detect a moderate 
effect of condition (standardized difference between change rates = .50). With a 
minimum of 6 measurements spaced over the maximum 16 treatment weeks and 
assuming 5 patients per therapist, an intra-class correlation of .15, and an alpha of 
.05, we will need a total of 44 therapists and 220 patients to achieve a power of .80 
to detect moderate condition effects on linear change rates. Factoring a 20% 
dropout rate at the patient level, running our experiment on 264 patients (6 per 
therapist) should provide sufficient statistical power to detect group differences on 
our primary outcome variables. To summarize, based on this power analysis, we will 
recruit 44 therapists (tracking the routinely administered outcomes for 15 of their 
patients each) for phase 1 (total phase 1 patient n = 660). We will then recruit 264 
patients for the phase 2 trial, assigning patients to the same 44 therapists who 
participated in phase 1 (they will see 6 cases each during the trial). 
 
b. Describe the subject population, including the age range, gender, ethnic 
background, and type of subjects (e.g. students, professors, subjects with learning 
disabilities, mental health disorders, etc.). Please incorporate specific 
inclusion/exclusion criteria (e.g. physical and psychological health, demographic 
information, or other unique characteristics). 
Therapist participants: As noted, our target sample is 44 therapist participants. 
Reflecting Atrius Health’s psychotherapist demographics, we anticipate that our 
sample will roughly breakdown as follows: 64% social workers and 36% 
psychologists. Demographically, the mean age of therapists is 54 years (range = 28-
72). The majority of the eligible sample is female (61.5%) and White (92.8%; 6% 
Hispanic). Based on these projections and our power analysis, our targeted/planned 
therapist enrollment is indicated in an attached Targeted/Planned Enrollment Table 
(Therapists). 
 
Patient participants: Patient participants will be adult men and women (age 18-65) 
largely referred by Atrius primary care and obstetrics/gynecology for triage and 
treatment through Atrius’ behavioral health specialty practice. All patient 
participants, including those self-referred, will have an Atrius primary care physician 
(PCP). Recruitment to the study simply means a willingness to be randomized to 

 
19 
condition and to complete a few study-specific measures. As this is an effectiveness 
design with a premium on ecological validity and scalability, virtually all patients in 
the network will be eligible. Atrius has implemented a care-model for mental health 
to support PCPs working with patients who have the most pressing behavioral health 
issues. Therefore, it is most likely that the sample will be predominated by the 
following diagnoses: complex mood, trauma, and anxiety disorders, eating disorders, 
simple schizophrenia, borderline personality disorder, substance abuse, and 
insomnia. 
 
The composition of our patient sample will roughly match the average Atrius 
utilization data for age, gender, race, and ethnicity. We expect the average age to be 
40 years (SD = 18.79 years). We expect that 62% will be female (38% male). For 
race/ethnicity, based on recent Atrius outpatient data, we expect the following: 76% 
White, 10% Black/African American, 5% Asian, 4.5% multiracial, 4.5% Hispanic, and 
less than 1% Native American and Hawaiian/Pacific Islander. These figures are 
consistent with county census data in the greater Boston area (including North and 
South Shores). The only exclusion criteria will be patients who are not the primary, 
informed decision-maker for their care and adults over age 65 years. The latter is 
because older adults (a) represent a small portion of patients at Atrius, and (b) their 
mental health treatment is complicated by aging issues for which specialized care 
may be required. Based on these projections and our power analysis, our 
targeted/planned therapist enrollment is attached in an Estimated Final 
Racial/Ethnic and Gender Enrollment Table. 
 
c. State the number and rationale for involvement of potentially vulnerable subjects 
to be entered into the study, including minors, pregnant women, prisoners, 
economically and educationally disadvantaged, decisionally challenged, and 
homeless people. 
We are not specifically targeting these specific vulnerable populations, and our 
design and/or the Atrius system will specifically exclude minors and prisoners. 
However, given the effectiveness design focused on maximizing ecological validity, 
some of our patients are sure to have economic and educational vulnerabilities, 
which are risk factors for mental health issues. Some women might also be 
pregnant. 
 
d. If women, minorities, or minors are not included, a clear compelling rationale must 
be provided. 
Minors will be excluded because they are typically not solely responsible for their 
own treatment decisions, and the outcome measure used in this study, and on 
which the match manipulation is based, focuses on adults. 
 
e. State the number, if any, of subjects who are laboratory personnel, employees, 
and/or students. They should be presented with the same written informed 
consent. If compensation is allowed, they should also receive it. 

 
20 
N/A 
 
f. State the number, if any, of subjects who are involved in research conducted 
abroad and describe any unique cultural, economic or political conditions. 
N/A 
 
g. Describe your procedures for recruiting subjects, including how potential subjects 
will be identified for recruitment. Attach advertisements, flyers, etc. in Section #11 
(Attachments). Note: Potential subjects may not be contacted before IRB approval. 
Therapist participants:  
 
Recruitment will be coordinated among our UMass-employed PC, clinic staff 
members, and clinic administrators, and will involve presenting information 
about the study (both phases 1 and 2) to providers through flyers, verbal script, or 
email. Interested therapist participants will meet or speak via teleconference with 
the PC to learn about the study details/procedures and to provide formal consent 
through an online baseline survey to which they will be directed (this all occurs prior 
to the baseline period for establishing TOP “report cards” that will inform the match 
in phase 2). Therapists will be told that the study is examining various referral 
processes that will not affect their delivery of treatment-as-usual. They will be 
informed that they will be blind to the specific nature of the referral manipulation in 
phase 2, but will be fully debriefed following the study and offered an opportunity to 
provide feedback on their experience. Therapists will also be informed of the study-
specific assessments in which their study patients will engage in both phase 1 and 
phase 2 (though they will not have access to these research data at any time). 
Therapists will also need to consent to completing a few study-specific measures at 
phase 1 baseline and throughout treatment with each patient during the phase 2 
RCT (these are described below in the listing of relevant phase 2 attachments to this 
protocol). For phase 1, therapists will be compensated with a $20 Amazon gift card 
for a one-time completion of the phase 1 baseline forms. For phase 2, they will be 
compensated $50 per patient (again in the form of Amazon gift cards) for 
completing several measures of therapy process at multiple points throughout each 
patient’s treatment. 
 
Patient participants: 
 
Phase 1 - During phase 1, patients will only be engaged in the study by virtue of 
registering into the TOP system at their treatment baseline, and completing the TOP 
at baseline, week 8, and their termination point or week 16, whichever comes 
sooner (to mimic the definition of treatment outcome in the RCT phase discussed 
below). Given their involvement in this one element of the research protocol, 
patients will need to give the staff at their clinic verbal consent to be contacted by 
the PC or an RA. The staff members will collaborate with the research team to track 
these patients being treated by a study-consenting therapist during this phase 1 part 

 
21 
of the study. Once contacted via telephone, the PC or RA will explain the nature of 
the patient’s involvement, which will simply involve registering on the TOP system, 
and completing the TOP main outcome measures at baseline, week 8, and 
termination. The patient will then be directed to a secure online system to complete 
their consent form and their baseline TOP forms. For the week 8 and the 
posttreatment TOP administrations, the PC will send specific links that will direct 
patients to the measure. The online system was developed and is maintained by our 
subcontractor, ORI. For this minimal time burden, patients will be entered into a 
raffle to win one of ten $25 Amazon gift cards. These 660 patients will be involved in 
phase 1 only. Thus, they will be debriefed by the PC or RA after they terminate their 
treatment-as-usual. 
 
Phase 2 - Once referred into the Atrius behavioral health system per usual care 
process, the front desk staff at the participating clinics will ask potential patient 
subjects if they are interested in hearing more about the study were they to be 
treated within Atrius with individual outpatient psychotherapy (versus being 
referred out or receiving care that does not involve individual outpatient 
psychotherapy). Next, a triage clinician at the clinics will determine that patients 
make their own treatment decisions and will receive individual therapy as part of 
their “in-house” treatment plan; if so, the patient is study-eligible. Atrius clinic/triage 
staff will provide the research team with a daily list of referrals (via direct telephone 
contact or through a secure server) of patients who have provided verbal consent to 
be contacted about the study and who have been determined to be appropriate for 
relevant treatment within the Atrius system. The PC will then reach out to patients 
who provided verbal consent to be contacted. 
 
If a patient agrees to engage in a consent/baseline assessment session, the PC will 
schedule a telephone diagnostic interview with a trained graduate clinical 
psychology RA (employed at either UMass or University at Albany). The RA will first 
review the study details/procedures and obtain consent through an online baseline 
survey to which the patient will be directed. Patients will be informed that their 
participation in the trial will largely mimic the same treatment that they would 
receive if they were not participating. All patients will also be told that the study is 
examining various referral processes that will not affect their treatment; they will be 
kept unaware of the specific nature of the referral manipulation, but will be told that 
they will be fully debriefed following the study and offered an opportunity to 
provide feedback on their experience. Patients will be asked to remain with the 
same therapist through at least 16 weeks of treatment; however, if they do request 
a transfer earlier, this will be treated as a dropout point for the sake of the trial (i.e., 
their data will not be analyzed in the primary analyses). Patients will also 
be informed that they will complete all assessments that are part of their routine 
clinical care, as well as several study-specific measures, for which they will be 
compensated $50 total. 
 

 
22 
h. Compensation. Explain the amount and type of compensation (payment, 
experimental credit, gift card, etc.), if any, that will be given for participation in the 
study. Include a schedule for compensation and provisions for prorating. 
Therapist participants:  
 
Therapists will complete a few study-specific measures at phase 1 baseline for which 
they will be compensated $20 in total (in the form of an Amazon gift card). 
 
During Phase 2, therapists will also complete a few study-specific measures 
throughout treatment with each of the 6 patients seen during the phase 2 RCT; they 
will be compensated $50 per patient for this additional, but minimal, time burden. 
The compensation will again be in the form of Amazon gift cards. 
 
If therapists complete their measurement schedule through all possible contact 
points for a given participating patient (i.e., baseline + 16 treatment weeks = 17 
weeks), or complete their measurement schedule through a planned termination for 
a participating patient that occurs prior to week 16 of treatment, they will receive 
full compensation (i.e., a $50 gift card for that patient). However, if a therapist 
withdraws from the study, they will have the option to be compensated on a 
prorated basis for the measures that they have already completed regarding each of 
their participating patients. This proration works out to approximately $3 per week 
for a participating patient, which will be deducted for the number of weeks 
“missing” from therapists’ assessment schedule (i.e., based on the point at which 
the therapist withdrew from the study). For example, if a therapist completes the 
measurement schedule for a given patient through week 8 (9 weeks, including 
baseline) and then withdraws from the study, they will have “missed” 8 weeks of 
data collection for that participating patient. Their compensation for this 
participating patient will be adjusted as follows: $50 - $24 ($3 x 8 weeks) = $26. This 
adjustment will be completed for any and all relevant participating patients. To 
summarize, therapists who withdraw from the study will have the option either to 
(a) receive their relevant prorated compensation, or (b) to forgo prorated 
compensation in order to no longer be contacted by the research team. 
 
Patient participants:  
 
Patients in phase 1 will register into the TOP system at their treatment baseline and 
complete the TOP at baseline, week 8, and their termination point or week 16, 
whichever comes sooner. For this minimal time burden, patients will be entered into 
a raffle to win one of ten $25 Amazon gift cards. These 660 patients will be involved 
in phase 1 only. Provided that patients complete the baseline assessment, they will 
be entered into the raffle. And given the small amount of compensation and the low 
odds of winning the raffle, they will also remain in the raffle even if their treatment 
ends prior to the second assessment point (week 8), and even if they decide not to 
complete the treatment termination assessment. Patients will be withdrawn from 

 
23 
the raffle if they withdraw from the study protocol (thereby withdrawing their data 
and indicating that they no longer wish to be contacted by the research team). 
 
Patients in Phase 2 will undergo a semi-structured diagnostic interview at both 
baseline and posttreatment, as well as complete several study specific measures 
throughout treatment (and, if randomly selected, at a follow up); they will be 
compensated $50 total for these non-routine aspects of their care. The 
compensation will be in the form of an Amazon gift card. If patients complete their 
measurement schedule through all possible contact points (i.e., baseline + 16 
treatment weeks = 17 weeks), or complete their measurement schedule through a 
planned termination that occurs prior to week 16 of treatment, they will receive full 
compensation. However, if they drop out of treatment prior to week 16, and their 
end point was not a planned termination that can be considered posttreatment for 
the purpose of the study, compensation will occur on a prorated schedule. This 
works out to approximately $3 per week, which will be deducted for the number of 
weeks “missing” from the schedule. For example, if a patient completes the 
measurement schedule through week 8 (9 weeks, including baseline), and they did 
not engage in a planned termination, they will have “missed” 8 weeks of data 
collection. Their compensation will be adjusted as follows: $50 - $24 ($3 x 8 weeks) = 
$26. 
 
Patients who withdraw from the study (which is distinct from simply dropping out of 
treatment) will be given the option to (a) receive prorated compensation for the 
completion of measures up until the point of withdrawal (following the proration 
schedule outline above), or (b) to forgo prorated compensation in order to no longer 
be contacted by the research team. 
 
If the patient participates in an exit interview, he or she will receive full 
compensation in the form of an additional $100 Amazon gift card. 
 
Finally, note that in the event that a participant (either a patient or therapist) 
withdraws from the study during phase 2, the other dyad member (either the 
patients seen by a withdrawing therapist or the therapist treating a withdrawing 
patient) will not be penalized; that is, as long as they have already consented to the 
study, they will receive the full amount of reimbursement (i.e., a $50 gift card) 
regardless of the point at which their patient/therapist withdraws. However, note 
that consistent with the wishes of the participant, we will, of course, stop collecting 
data at the point of withdrawal (i.e., if therapists withdraw, we will stop collecting 
data from their patients who will be compensated fully; if patients withdraw, we will 
stop collecting data from their therapist regarding that patient and the therapist will 
be compensated fully for that patient). 
 
i. Please state: A: The total expected duration of the study, including the time 
expected for data analysis (e.g., This study is expected to last 1 year) AND B: How 

 
24 
much time each subject is expected to be involved in the study (e.g., The 
involvement of each subject will be 1-session for a total of 90 minutes). 
A) The project is funded in the form a cost-reimbursement contract for which a 
specific milestone schedule exists. The contract start date is 9/15/16 and the 
contract term date is 7/14/20. All analyses will be completed by the term date. 
Details are available in the attached milestone schedule. 
 
B) Therapist subjects will be involved for 6 months in phase 1, as well as through the 
phase 2 trial (approximately 2 years, though with variability depending on when 
they have been assigned and have treated their 6 study cases). Patient subjects in 
phase 1 only will be involved in the study protocol through their actual termination 
point or 16 weeks, whichever comes sooner. Patients in phase 2 only will be 
involved in the study protocol through their actual termination point or 16 
weeks, whichever comes sooner (we will also conduct a follow-up outcome 
assessment at 1 year after termination on a randomly-selected subsample of 40 
patients). 
 
5. Risks 
 
HHS Regulations define a subject at risk as follows: “...any individual who may be exposed to 
the possibility of injury, including physical, psychological, or social injury, as a consequence of 
participation as a subject in any research...” This also includes risks to subject confidentiality 
and any discomforts, hazards, or inconveniences. 
 
For the categories below, include a description of risks. 
 
a. Describe the risks related to: 
Physical well-being 
None anticipated. 
 
Psychological well-being 
Regardless of condition, all participating clinics will employ their usual triage 
assessments, therapists will employ their usual treatments, and patients will be 
receiving their usual treatments. Consequently, there are no risks from our research 
protocol over and above what would normally be expected in routine assessment 
and psychotherapy, and the clinics all have their clinical and safety protocols in place 
(and clinical personnel to execute them). 
 
In treatment, some individuals may experience emotional upset during sessions. 
Additionally, some participants may experience disappointment with their rate of 
progress or setbacks. The risk associated with such reactions will be addressed 
clinically by the therapists who are treating these issues and who have peer and 
administrative support. To reiterate, these treatment risks would occur in the course 
of treatment-as-usual. These are not additional risks stemming from the research 

 
25 
protocol. Further, outcome monitoring systems, which the basis of our protocol, are 
already being used by Atrius providers without incident. 
 
As is typical in psychological research, some of the assessment questions from the 
research measures may be experienced as intrusive and/or may cause anxiety. The 
risk from such increased anxiety, however, is minimized by the use of skilled and 
extensively trained assessors who are aware that such reactions may be related to a 
person’s presenting problems, or simply a function of the intimate and emotionally 
intense nature of psychological services. In addition, the PIs, PC, and Atrius site staff 
and administrators will be available to meet with any participant who may be unduly 
disturbed due to the few research tasks. Because the pre- and posttreatment 
diagnostic interviews will be conducted via telephone, the graduate RA (being 
trained as a clinician and supervised by their site PI, Dr. Constantino or Boswell, both 
of whom are licensed clinical psychologists and mental health care providers) will 
have the patient’s contact information (phone number and current address) on 
hand. If the patient reveals clinically elevated suicidality or homicidality, the RA will 
contact 9-1-1 and report the patient’s contact information and location address for 
emergency response. The RA, if applicable, will also execute any duty to warn to the 
best of their ability (in addition to contacting the local authorities). 
 
There is also the possibility that the research team will become aware of critical 
items being endorsed on the TOP (e.g., suicidal ideation). In this case, the PI or PC 
(who will be responsible for regularly reviewing TOP data as it is collected) will 
inform the Atrius Co-PI (Dr. Nordberg), who will determine if the therapist should be 
notified of these critical item responses. However, as Atrius therapists will 
constantly be engaging in psychological and risk assessments as part of routine care, 
this information will likely be known already. To be sure, though, Atrius will have a 
protocol in place to share critical item data without having to share the full TOP 
report (which, as per the research design, is not being shared with clinicians). 
 
Economic well-being 
Given that therapist performance data are being collected, it is reasonable to be 
concerned about possible employment implications were an employer (i.e., clinic 
administrator) to attempt to interpret study information incompletely (i.e., infer lack 
of therapist effectiveness to the point of questioning employability). This risk, 
however, is extremely minimal for the following reasons: 
 
(1) As a condition of being involved in the study, clinic administrators will be 
required to agree that therapists’ participation or non-participation in this research 
will in no way affect their standing/employment at their community mental health 
clinic. 
 
(2) The research team will not reveal therapist performance data to clinic 
administrators or staff members; that is, the study could be considered “triple-

 
26 
blind.” Neither patients nor therapists will know when they are in an experimentally-
matched vs. typically-matched dyad, and administrators/staff members will not have 
access to the therapists’ report cards. 
 
(3) However, administrators and staff members are required to be in the know 
about well-matched therapist “short-lists,” as this is essential to the research design; 
that is, when patients are randomized to a well-matched therapist, those potential 
therapists need to be identifiable. It is possible that administrators or staff members 
might misinterpret these data to suggest that a given therapist is ineffective (if he or 
she is never or rarely showing up on a shortlist). However, we will guard against this 
misinterpretation by educating administrators and staff members that the shortlist 
only represents, in a small cross-section of time, therapists that have been shown to 
be effective on at least 1 of 14 domains, which represents a given patient’s most 
severe problem at that time (the match criterion). We will stress that this does not 
mean that a therapist is globally ineffective. It may just be that patients randomly 
assigned to the match group are tending not to have the types of problems for 
which a given therapist is relatively effective. That therapist, though, could be highly 
effective at treating one or even many other domains. 
 
(4) Finally, administrators and staff members will not be told which therapists are or 
are not participating in the study. Thus, lack of being on a shortlist, for all that they 
will know (unless a therapist openly reveals that he or she is participating in the 
study), could simply connote a choice to not participate in the project. 
 
Social well-being 
None anticipated. 
 
Breach of confidentiality (including audio/video taping) 
A breach of confidentiality represents a risk, but every step will be taken to minimize 
this risk. Atrius clinics and ORI routinely handle private health information and are in 
compliance with HIPAA regulations. Any “hard” materials (e.g., diagnostic 
assessment summaries) that are collected will be stored in a locked cabinet in the 
PI’s Psychotherapy Research Lab. There will be no hard copy data collected at the 
Atrius clinic sites. Virtually all of the data collected in this study (including consent) 
will be through a secure, web-based platform using a tablet or computer. This 
method offers greater protection because it guards against human error and 
negates the need for long-term storage of paper forms. In addition, digital 
recordings of diagnostic assessments will be stored in a secure, password protected 
website. The recordings themselves will be encrypted. 
 
b. For research conducted internationally, describe any political or sociocultural 
considerations that may affect your research design (for example, in some 
communities it may not be customary to sign documents, etc.) 
N/A 

 
27 
 
c. Discuss plans for ensuring necessary medical or professional intervention in the 
event of a distressed subject. 
The Co-PIs, project coordinator, Atrius site staff members, and Atrius administrators 
will monitor the treatments and data collection; thus, they can assist in regularly 
monitoring any adverse events. Such negative occurrences are unlikely to be trial-
related, as all patients will be receiving treatment-as-usual. Therefore, any adverse 
event will be addressed with the Atrius clinics well-established procedures for 
monitoring services and managing treatment-related disturbances. Nevertheless, 
any adverse event will be recorded and immediately reported to the IRB (UMass), 
PCORI (funder), and the project’s Data Safety and Monitoring Board (DSMB). 
 
Should, during the course of the study, a patient show evidence of psychological or 
physical deterioration, the patient will be assessed comprehensively in the domains 
of concern (except in the case of a life-threatening physical emergency, such as the 
emergence of acute chest pain, in which case 9-1-1 will be called immediately). If the 
therapist deems that the patient meets criteria for a psychiatric hold (e.g., patient is 
an imminent danger to self or others), the therapist will arrange for the patient to be 
brought to the emergency department and will contact his/her Atrius administrator 
and the PI to debrief. If a patient is not meeting criteria for a psychiatric hold, but is 
showing clear signs of decreased mental status, the therapist will continue to meet 
with the patient, as well as – in consultation with the Atrius administrator – make 
arrangements for the most appropriate level of care (e.g., day treatment). 
 
As noted, because the pre- and posttreatment diagnostic interviews will be 
conducted via telephone, the graduate RA (being trained as a clinician and 
supervised by their site PI, Dr. Constantino or Boswell, both of whom are licensed 
clinical psychologists and mental health care providers) will have the patient’s 
contact information (phone number and current address) on hand. If the patient 
reveals clinically elevated suicidality or homicidality, the RA will contact 9-1-1 and 
report the patient’s contact information and location address for emergency 
response. The RA, if applicable, will also execute any duty to warn to the best of 
their ability (in addition to contacting the local authorities). 
 
Also as noted, there is also the possibility that the research team will become aware 
of critical items being endorsed on the TOP (e.g., suicidal ideation). In this case, the 
PI or PC (who will be responsible for regularly reviewing TOP data as it is collected) 
will inform the Atrius Co-PI (Dr. Nordberg), who will determine if the therapist 
should be notified of these critical item responses. However, as Atrius therapists will 
constantly be engaging in psychological and risk assessments as part of routine care, 
this information will likely be known already. To be sure, though, Atrius will have a 
protocol in place to share critical item data without having to share the full TOP 
report (which, as per the research design, is not being shared with clinicians). 
 

 
28 
6. Benefits 
 
a. Describe the potential benefit(s) to be gained by the subjects or by the acquisition 
of important knowledge which may benefit future subjects, etc. (This DOES NOT 
include compensation or extra credit). 
The most direct benefit a participant in this study may receive is the reduction of 
symptom-related distress and improved functioning. In addition, patients (especially 
those in the match condition) will receive more personalized MHC. Psychotherapists 
(especially those in the match condition) may experience a greater level of positive 
impact across their caseloads. Given that the actual treatments being provided will 
not be manipulated, the benefits of participation are judged to far outweigh the 
potential study-specific risks. 
 
There is immense potential for future therapists and patients to benefit from the 
results of this study; if the hypotheses are supported, there will be cause for 
substantial revamping of MHC systems to capitalize on matching patients to 
therapists who have an empirically demonstrable track record of strength in treating 
patients with similar presenting problems. 
 
7. Procedures to Maintain Confidentiality 
 
a. Describe the procedures in place which protect the privacy of the subjects and 
maintain the confidentiality of the data, as required by the federal regulations, if 
applicable. 
Multiple steps will be taken to protect confidentiality. As mentioned, minimal paper 
forms (e.g., diagnostic summary forms) will be kept in a locked cabinet in the PI’s 
locked Psychotherapy Research Lab. There will be no hard copy data collected at the 
Atrius clinic sites. Virtually all of the data collected in this study (including consent) 
will be through a secure, web-based platform using a tablet or computer. This 
method offers greater protection because it guards against human error and 
negates the need for long-term storage of paper forms. In addition, digital 
recordings of diagnostic assessments will be stored in a secure, password protected 
website. The recordings themselves will be encrypted. 
 
Only designated study personnel will have access to identifiable, study specific, 
private information about human subjects. When registering on the TOP system, 
both patients and therapists are assigned a random number code that links all 
subsequent assessments and is separated from identifiable information. This 
random number code will function as each participant’s study code and will be used 
to link participants’ data. As noted, all therapist and patient data (outside of 
diagnostic assessment summaries) will be collected through a web-based platform. 
The assigned participant code will be used to link/aggregate information, so private 
information will not be requested after the baseline assessment/consent process. 
Only the PI and essential research staff will have access to the list that links 

 
29 
identifiable information with the participant’s study code. Any audio recordings will 
be encrypted and password protected. Only the Co-PIs will know this password and 
have the capacity to access the recordings. When it is time to analyze the recordings 
for reliability coding, designated, trained RAs will also have access to the recordings; 
however, they will not have access to additional identifiable information (only the 
information required to complete the analysis). For any data used for research and 
publication purposes, the confidentiality of participant information will be ensured. 
 
b. If information derived from the study will be provided to a government agency, or 
any other person or group, describe to whom the information will be given and the 
nature of the information. 
The PI is required to submit information (i.e., contractual “deliverables”) on a regular 
basis to PCORI (the study sponsor), including IRB protocols, interim progress reports, 
advisory board meeting minutes, engagement plan updates, evidence of diagnostic 
criterion reliability from training cases, interim data reports, presentation abstracts 
and documentation of acceptance, manuscript copies, letters of endorsement from 
scientific and consumer groups, final data analysis summary, and final research 
report. Details on deliverables are available in the aforementioned (and attached) 
milestone schedule. No protected health information will be transmitted to PCORI. 
 
c. Specify where and under what conditions study data will be kept, how specimens 
will be labeled and stored (if applicable), who has access to the data and 
specimens, and what will be available to whom. 
As noted, minimal paper forms (e.g., diagnostic summary forms) will be kept in a 
locked cabinet in the PI’s locked Psychotherapy Research Lab. There will be no hard 
copy data collected at the Atrius clinic sites. Virtually all of the data collected in this 
study (including consent) will be through a secure, web-based platform using a 
tablet or computer. Digital recordings of diagnostic assessments will be stored in 
password protected website, and securely deleted by the project contract’s term 
date. Only the relevant members of the research team will have access to the 
participants’ data and only the PI will have long-term access to identifiable 
information. As noted, all assessments will be linked with a de-identified participant 
code. Any records linking the code to the participant’s name or voice recording will 
be kept in a separate locked file cabinet in the PI’s office. These records will be 
destroyed 5 years after the contract term date. 
 
8. Potential Conflict of Interest 
 
a. Do any of the involved investigators or their immediate family (as described 
below) have consulting arrangements, management responsibilities or equity 
holdings in the Sponsoring company, vendor(s), provider(s) of goods, or 
subcontractor(s)? Y 
 

 
30 
b. Do any investigators or their immediate family have any financial relationship with 
the Sponsoring company, including the receipt of honoraria, income, or 
stock/stock options as payment? N 
 
c. Is any Investigator(s) a member of an advisory board with the Sponsoring 
company? N 
 
d. Do any investigators receive gift funds from the Sponsoring company? N 
 
e. Do any investigators or their immediate family have an ownership or royalty 
interest in any intellectual property utilized in this protocol? Y 
 
“Immediate family” means a spouse, dependent children as defined by the IRS, or a 
domestic partner. If one or more of the above relationships exist, please include a 
statement in the consent form to disclose this relationship. i.e., a paid consultant, a paid 
member of the Scientific Advisory Board, has stock or stock options, or receives 
payment for lectures given on behalf of the sponsor. The consent form should disclose 
what institution(s) or companies are involved in the study through funding, cooperative 
research, or by providing study drugs or equipment. If you answer yes to any of the 
questions above, please go to the policies for more information. 
 
9. Informed Consent 
 
You can add different Consent Forms, Alteration Forms, and Waivers. Provide consent 
process background information, in the table below, for each Consent Form(s), Alteration 
Form(s), and Waiver(s). 
 
9.1. Consent Form – therapist phase 1 & 2 consent forms 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
PC or an RA. 
 
How is consent being obtained? 
Therapists will meet or speak via teleconference with the PC or an RA to learn about the 
study details/procedures and to provide formal consent through an online baseline 
survey to which they will be directed. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
The PI and his collaborators will provide close oversight of the entire protocol. 
 
9.2. Consent Form – patient phase 1 consent form 
 

 
31 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
Atrius clinic staff will obtain verbal consent from patients about their willingness to be 
contacted about the study. Patients will then meet or speak via teleconference with the 
PC or an RA to learn about the study details/procedures and to provide formal consent 
through an online baseline survey to which they will be directed (this is the case for 
both phase 1 and phase 2). 
 
How is consent being obtained? 
If a patient agrees to engage in a consent/baseline assessment session, the PC will 
schedule a telephone diagnostic interview with a trained graduate clinical psychology 
RA (employed at either UMass or University at Albany). The RAs will first review the 
study details/procedures and obtain consent through an online baseline survey to which 
the patient will be directed. Coercion will be minimized by clearly stating that 
participation is voluntary and will in no way impact the patient’s treatment. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
Competency for making one’s own treatment decisions will be an inclusion criterion for 
the study. Moreover, if a clinic staff member, the PC, or an RA interacts with a therapist 
or patient who appears to have competency issues in the decision-making process for 
engaging in the study, they will immediate bring this concern to the PI or a Co-PI before 
enrolling them. The team will them make an informed decision as to whether to include 
that person in the study. The DSMB will be consulted if appropriate. 
 
9.3. Consent Form – patient phase 2 consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
Atrius clinic staff will obtain verbal consent from patients about their willingness to be 
contacted about the study. Patients will then meet or speak via teleconference with the 
project coordinator or a research assistant to learn about the study details/procedures 
and to provide formal consent through an online baseline survey to which they will be 
directed (this is the case for both phase 1 and phase 2). 
 
How is consent being obtained? 
If a patient agrees to engage in a consent/baseline assessment session, the PC will 
schedule a telephone diagnostic interview with a trained graduate clinical psychology 
RA (employed at either UMass or University at Albany). The RAs will first review the 
study details/procedures and obtain consent through an online baseline survey to which 
the patient will be directed. Coercion will be minimized by clearly stating that 
participation is voluntary and will in no way impact the patient’s treatment. 
 

 
32 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
Competency for making one’s own treatment decisions will be an inclusion criterion for 
the study. Moreover, if a clinic staff member, the PC, or an RA interacts with a therapist 
or patient who appears to have competency issues in the decision-making process for 
engaging in the study, they will immediate bring this concern to the PI or a Co-PI before 
enrolling them. The team will them make an informed decision as to whether to include 
that person in the study. The DSMB will be consulted if appropriate. 
 
9.4. Consent Form – patient exit interview supplemental consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
The exit interviewer (i.e., RA or advisory board member). 
 
How is consent being obtained? 
If a patient agrees to engage in an exit interview, the interviewer will review the study 
details/procedures and obtain supplemental consent through an online link to which the 
patient will be directed. Coercion will be minimized by clearly stating that participation 
is voluntary and will in no way impact the patient’s treatment. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
Competency for making one’s own treatment decisions will have been an inclusion 
criterion for the main study. Moreover, if the interviewer interacts with a patient who 
appears to have competency issues in the decision-making process for engaging in the 
exit interview, they will immediate bring this concern to the PI or a Co-PI before 
enrolling them. The team will them make an informed decision as to whether to include 
that person in the interview protocol. The DSMB will be consulted if appropriate. 
 
9.5. Consent Form – therapist exit interview supplemental consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
The exit interviewer (i.e., RA or advisory board member). 
 
How is consent being obtained? 
If a therapist agrees to engage in an exit interview, the interviewer will review the study 
details/procedures and obtain supplemental consent through an online link to which the 
therapist will be directed. Coercion will be minimized by clearly stating that participation 
is voluntary. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 

 
33 
If the interviewer interacts with a therapist who appears to have competency issues in 
the decision-making process for engaging in the exit interview, they will immediate 
bring this concern to the PI or a Co-PI before enrolling them. The team will them make 
an informed decision as to whether to include that person in the interview protocol. 
 
10. Assent Background 
 
All minors must provide an affirmative consent to participate by signing a simplified assent 
form, unless the Investigator(s) provides evidence to the IRB that the minor subjects are not 
capable of assenting because of age, maturity, psychological state, or other factors. 
 
11. Attachments 
 
Document Type 
Document Name 
Attached Date 
Questionnaires 
Patient Phase 2 
During-Treatment Measures 
Packet 
10/04/2016 
Questionnaires 
Therapist Phase 2 
During-Treatment Measures 
Packet 
10/04/2016 
Questionnaires 
TOP-STP 
10/04/2016 
Questionnaires 
PCORI_Nature of Termination 
Form 
10/04/2016 
Questionnaires 
Stakeholder Exit Interview 
Protocols 
10/04/2016 
Other 
PCORI Milestone Schedule 
10/04/2016 
Other 
PCORI_Targeted Enrollment 
Tables 
10/04/2016 
Other 
Participant Flow 
10/04/2016 
Other 
Data Collection Schedule 
10/04/2016 
Federal Grant/Sub-contract 
PCORI IHS-1503-
28573_Constantino_executed 
contract 
10/04/2016 
Federal Grant/Sub-contract 
PCORI_IHS_Research 
Plan_final update 9-12-16 
10/04/2016 
Federal Grant/Sub-contract 
PCORI Original Contract 
Proposal_all sections 
10/04/2016 
Other 
PCORI_Phase 1_Patient 
Debriefing Form 
10/04/2016 
Other 
Constantino Lab Personnel 
Link- Google Docs 
10/04/2016 
Advertisements 
PCORI_Phase 1_Clinician 
Recruitment_All Forms 
11/13/2016 

 
34 
Combined_R1 
Advertisements 
PCORI_Phase 1_Patient 
Recruitment_Verbal 
Script_R1 
11/13/2016 
Advertisements 
PCORI_Phase 2_Patient 
Recruitment_Verbal 
Script_R1 
11/13/2016 
Other 
PCORI_Phase 1 & 
2_Therapist Debriefing Form 
11/13/2016 
Other 
PCORI_Phase 2_Patient 
Debriefing Form_R1 
11/13/2016 
Other 
PCORI_Phase 2_Patient Data 
Collection Email Template 
11/13/2016 
Other 
PCORI_Phase 2_Patient Data 
Collection Reminder Call 
Script 
11/13/2016 
Other 
PCORI_Phase 2_Therapist 
Data Collection Email 
Template 
11/13/2016 
Other 
PCORI_Phase 2_Therapist 
Data Collection Reminder Call 
Script 
11/13/2016 
Other 
PCORI_Phase 1_Patient Data 
Collection Email Template 
11/13/2016 
Other 
PCORI_Phase 1_Patient Data 
Collection Reminder Call 
Script 
11/13/2016 
Other 
PCORI_Phase 1_Therapist 
Data Collection Email 
Template 
11/13/2016 
Other 
PCORI_Phase 1_Therapist 
Data Collection Reminder Call 
Script 
11/13/2016 
Questionnaires 
CGI 
11/13/2016 
Questionnaires 
MINI 7.0.2 Standard 
11/13/2016 
Questionnaires 
Patient Phase 1 Consent & 
Baseline Measures Packet_R1 
11/13/2016 
Questionnaires 
Patient Phase 2 Consent & 
Baseline Measures Packet_R1 
11/13/2016 
Questionnaires 
Therapist Consent & Baseline 
Measures Packet_R1 
11/13/2016 
 
Obligations 

 
35 
 
Obligations of the Principal Investigator are: Modifications - Changes in any aspect of the study 
(for example, project design, procedures, consent forms, advertising materials, additional key 
personnel or subject population) will be submitted to the IRB for approval before instituting the 
changes; Consent Forms - All subjects will be given a copy of the signed consent form. 
Investigators will be required to retain signed consent documents for six (6) years after close of 
the grant or three (3) years if unfunded; Training - Human subject training certificates, including 
those for any newly added personnel, will be provided for all key personnel; Adverse Events - 
All adverse events occurring in the course of the protocol will be reported to the IRB as soon as 
possible, but not later than ten (10) working days; Continuing Review – IRB Protocol Report 
Forms will be submitted annually at least two weeks prior to expiration, six weeks for protocols 
that require full review; Completion Report - The IRB will be notified when the study is 
complete. To do this, complete the IRB Protocol Report Form and select “Final Report.” Training 
- Human subject training certificates, including those for any newly added personnel, will be 
provided for all key personnel; Adverse Events/Unanticipated Problems - All events occurring in 
the course of the protocol will be reported to the IRB as soon as possible, but not later than five 
(5) working days; Continuing Review - IRB Protocol Report Forms will be submitted annually at 
least two weeks prior to expiration, six weeks for protocols that require full review; Completion 
Report - The IRB will be notified when the study is complete. To do this, complete the IRB 
Protocol Report Form and select “Final Report.” 
 
The Principal Investigator has read and agrees to abide by the above obligations. Y 
 
 

 
36 
B. Summary of the 1st formal modification to the PCORI contract (July 2017), and the full 
revised study protocol (1st revision) submitted to, and approved by, the University of 
Massachusetts Amherst Institutional Review Board (November 2017) 
 
CONTRACT MODIFICATION SUMMARY 
 
On July 10, 2017, PCORI approved the 1st contract modification (IHS-1503-28573 Modification 
001), which included the following changes: 
 
• Effective June 7, 2017, Atrius Health was removed as our clinical partner and 
subcontract site, and Dr. Samuel Nordberg was removed as site Co-PI. 
 
• Effective August 1, 2017, Psychological and Behavioral Consultants (PsycBC) was added 
as our clinical partner and subcontract site, and Don Sykes was added as site Co-PI. 
 
• The project’s milestone schedule was revised in light of the subcontract site change. 
 
In concert with this modification, we submitted the following 1st revision of our study protocol 
to the UMass IRB, which was approved in November 2017. As outlined in the revised protocol, 
the major changes included: 
 
• Phase 1 of trial no longer required recruiting and consenting a subsample of patients. 
Rather, because PsycBC already routinely used our multidimensional outcome tool (the 
TOP) to track patient improvement, we had de-facto baseline data on therapists’ 
historical strengths and weaknesses (across a minimum of 15 cases) in treating the 12 
mental health domains that the TOP assesses and on which our match algorithm is 
based. Thus, Phase I now simply involved recruiting and consenting therapists to the 
study for whom we already possessed (or would soon possess based on standard 
operating procedures) the requisite cases and performance data, which we could 
securely access via the previously executed business agreement between ORI and 
PsycBC. 
 
• Phase 1 therapist recruitment procedures were adjusted to align with our new clinical 
partner’s (PsycBC) communications with their providers and their daily operations. 
 
• Phase 2 patient recruitment procedures were adjusted to align with PsycBC’s 
recruitment flow and daily operations. 
 
PROTOCOL 
APPLICATION FORM 
SOCIAL, BEHAVIORAL, AND EDUCATIONAL FULL BOARD 
HUMAN SUBJECTS IN SOCIAL, BEHAVIORAL, AND EDUCATIONAL 
RESEARCH 

 
37 
 
University of Massachusetts Amherst (UMass) 
Institutional Review Board (IRB) 
 
Protocol ID: 2016-3401 
Title: Enhancing Mental Health Care 
 
Revision Form 
1. Summarize the proposed changes to the protocol in lay terms (including details of ALL 
changes proposed AND modify all relevant protocol sections and attachments accordingly). 
As recently and extensively discussed with Margaret Burggren and Gaurav Dhawan, we submit 
here a revised protocol based on a contract modification for our PCORI-funded research 
project. The revisions are included in all relevant sections of this protocol; however, for ease of 
review, we have also attached a Word document that tracks all changes (in the “Other” section 
of the attachments page). The title of the document is: “PCORI IRB Proposal_R1_for 
PsycBC_FINAL submitted.docx”  
 
PsycBC is our new clinical subcontractor (replacing Atrius Health). All revisions in the protocol 
itself, and in all attachment attachments, reflect this new partnership. 
 
In the aforementioned Word document, we also note with comment bubbles when an 
attachment to this protocol has been revised, has stayed the same, or has been deleted 
because it is no longer relevant. Again, we hope that such use of tracked changes/comments is 
helpful to the review team. Of course, we can also answer any remaining questions. 
 
Thank you for your time and efforts in reviewing this protocol revision. 
 
2. Indicate Level of Risk involved with the changes proposed. 
No change. 
 
3. Describe any Other Changes. 
None 
 
Protocol Director: Michael J. Constantino 
Degree: PhD 
Title: Professor 
Department Name: Psychological & Brain Sciences 
Mailing Address: 612 Tobin Hall, 135 Hicks Way 
Phone: 5-1388; Fax: 5-0996 
E-mail: mconstantino@psych.umass.edu 
Human Subjects Training Completed? yes 
 
Subject Populations(s) Checklist 
Yes/No 
 
 

 
38 
Minors (under 18) 
Pregnant Women 
Cognitively Impaired or Decisionally Challenged 
Older individuals (75 and over) 
Healthy Volunteers 
Students/Employees 
International Populations 
Prisoners 
Other (i.e., any population that is not specified above) 
N 
N 
N 
N 
N 
N 
N 
N 
Y 
 
Other: Subjects will include two mental health care stakeholder groups: (1) therapists affiliated 
with PsycBC who are providing outpatient psychotherapy, and (2) adult patients receiving 
psychotherapy for varied mental health complaints from the participating therapists. PsycBC, a 
formal subcontract to UMass on this project, is an innovative health care organization and one 
of the largest providers of outpatient mental healthcare services in Ohio. PsycBC’s role on this 
project is restricted to providing the research team access to these two subject populations, 
and assisting the team in recruitment. Thus, PsycBC is not engaged in human subjects’ research. 
 
Study Location(s) Checklist 
Yes/No 
 
University of Massachusetts Amherst 
Baystate Medical 
University Health Services 
Hartford Hospital 
Other (Specify other Study Locations) 
 
Y 
N 
N 
N 
Y 
 
Other: All study operations will be coordinated through Dr. Michael Constantino’s (PI) 
Psychotherapy Research Lab at UMass Amherst. Subject data will be collected through our 
clinical partner, PsycBC, which employs a large team of psychiatrists, advanced practice nurses, 
psychologists, clinical counselors, and social workers serving children, adolescents, adults, and 
families in locations throughout Ohio and northern Kentucky. PsycBC’s experienced specialists 
provide therapy for a wide range of mental health issues. PsycBC includes multiple treatment 
sites in Ohio that will contribute to data collection. 
 
General Checklist 
Yes/No 
 
Training Grant? 
Funded Study (or proposal submitted to sponsor)? 
Cooperating Institution(s)? 
Federally Sponsored Project? 
Human blood, cells, tissues, or body fluids (tissues)? 
Subjects will be paid for participations? 
 
N 
Y 
Y 
Y 
N 
Y 
 

 
39 
Cooperating Institution(s): (1) University at Albany, SUNY (Dr. James Boswell; Co-PI and 
subcontract); (2) Outcome Referrals Institute, Inc. (ORI; Dr. David Kraus; Co-PI and subcontract); 
and (3) PsycBC (Dr. Tom Swales; subcontract director). Note: At the time of this revision, an IAA 
has already been established for the approved original protocol with SUNY Albany and ORI. 
After consulting with UMass IRB staff, it is now clear that our new subcontract, PsycBC, is not 
engaged in human subjects’ research; thus, no IAA is required/requested. 
 
Funding Checklist 
Grants/Contracts: 
 
Funding Administered By: UNIVERSITY 
PGCA#: 1503-28753 
GAID#:  
Funded By: Patient-Centered Outcomes Research Institute 
1828 L Street, NW, Suite 900 
Washington, DC 20036 
Phone: (202) 827-7700 | Fax: (202) 355-9558 
info@pcori.org 
Principle Investigator: Michael J. Constantino 
Grant/Contract Title: Enhancing Mental Health Care by Scientifically Matching Patients to 
Providers’ Strengths 
 
Are the contents of this protocol the same as described in grant/contract proposal? Y 
Is this a training grant? N 
Are any subcontracts issues under this grant? Y 
 
Fellowships – None 
Gift Funding – None 
Dept. Funding – None 
Other Funding – None 
 
1. Purpose of the study 
 
a. Provide a brief lay summary of the purpose of the study.  
Research has shown that mental health care (MHC) providers differ significantly in 
their ability to help patients. In addition, providers demonstrate different patterns of 
effectiveness across symptom and functioning domains. For example, some 
providers are reliably effective in treating numerous patients and problem domains, 
others are reliably effective in some domains (e.g., depression, substance abuse) yet 
appear to struggle in others (e.g., anxiety, social functioning), and some are reliably 
ineffective, or even harmful, across patients and domains. Knowledge of these 
provider differences is based largely on patient-reported outcomes collected in 
routine MHC settings. 
 

 
40 
Unfortunately, provider performance information is not systematically used to refer 
or assign a particular patient to a scientifically based best-matched provider. MHC 
systems continue to rely on random or purely pragmatic case assignment and 
referral, which significantly “waters down” the odds of a patient being 
assigned/referred to a high performing provider in the patient’s area(s) of need, and 
increases the risk of being assigned/referred to a provider who may have a track 
record of ineffectiveness. This research aims to solve the existing non-patient-
centered provider-matching problem. 
 
Specifically, we aim to demonstrate the comparative effectiveness of a scientifically-
based patient-provider match system compared to status quo pragmatic case 
assignment. We expect in the scientific match group significantly better treatment 
outcomes (e.g., symptoms, quality of life) and higher patient satisfaction with 
treatment. We also expect to demonstrate feasibility of implementing a scientific 
match process in a community MHC system and broad dissemination of the easily 
replicated scientific match technology in diverse health care settings. The 
importance of this work for patients cannot be understated. Far too many patients 
struggle to find the right provider, which unnecessarily prolongs suffering and 
promotes health care system inefficiency. A scientific match system based on 
routine outcome data uses patient-generated information to direct this patient to 
this provider in this setting. In addition, when based on multidimensional 
assessment, it allows a wide variety of patient-centered outcomes to be represented 
(e.g., symptom domains, functioning domains, quality of life). 
 
b. What does the Investigator(s) hope to learn from the study?  
The goal of this project is to test the effectiveness of an innovative, scientifically-
informed patient-therapist referral match algorithm based on MHC provider 
outcome data. We will employ a randomized controlled trial (RCT) to compare the 
match algorithm with commonplace pragmatic referral matching (based on provider 
availability, convenience, or self-reported specialty). Psychosocial treatment will 
remain naturalistically administered by varied providers (e.g., psychologists, social 
workers) to patients with mental health concerns. We hypothesize that the scientific 
match group will outperform the pragmatic match group in decreasing patient 
symptoms and treatment dropout, and in promoting patient functional outcomes, 
perceived treatment credibility, outcome expectation, and care satisfaction, as well 
as therapeutic alliance quality. Doing so will establish the match algorithm as a 
mechanism of effective patient-centered MHC, and will suggest that this 
scientifically derived patient-provider matching intervention can be integrated into 
MHC systems to aid in treatment decision making, as well as increase 
personalization. 
 
2. Study Procedures 
 
a. Describe all study procedures. 

 
41 
We will compare the efficacy of naturalistic treatment either with or without the aid 
of scientific matching to a provider with a double-blind RCT. The project will involve 
two main phases. First, we will access a naturalistic baseline assessment of 
consenting PsycBC therapists’ performance to determine their relative strengths and 
weaknesses in treating the problem domains measured by a multidimensional 
outcome tool. This period will establish our therapist sample pool and inform the 
RCT match manipulation (a match will represent a patient being assigned to a 
therapist who has empirically demonstrated during the baseline phase that he or 
she is stably effective at treating patients with the same type of presenting 
complaint). 
 
Second, and after the baseline period, new consenting outpatients will be randomly 
assigned to the match (experimental) or no match (control) condition. The PsycBC 
administrators and their project-specific coordinator will collaborate with the 
research team to apply the randomization protocol. Treatment outcome will be 
assessed through the patient’s actual termination point or 16 weeks, whichever 
comes sooner (we will also conduct a follow-up outcome assessment at 1 year after 
the point of termination on a randomly selected subsample). Outside of being 
matched to a therapist from a short-list of providers who have demonstrated (during 
the phase 1 baseline) reliable success in treating the patient’s primary problem area, 
and completing study-specific measures for which participants will receive monetary 
compensation, treatment will be delivered as usual (the short list still allows for 
pragmatic considerations like availability and administrator assignment options). 
 
Additional methodological details by study phase follow. 
 
Phase 1: The most significant revision to the research protocol is that we no longer 
need to recruit/enroll patients for phase 1. Rather, phase 1 now focuses solely on 
PsycBC clinicians as our research participants. To inform the match condition, we will 
first establish the baseline track record of participating therapists’ performance 
(across a minimum of 15 adult psychotherapy cases each) to determine their 
strengths in treating behavioral health domains measured by the primary outcome 
measure on which the match algorithm is based – the Treatment Outcome Package 
(TOP; Kraus, Seligman, & Jordan, 2005), which is described below in the listing of 
relevant phase 1 attachments to this protocol. Developed and processed by our Co- 
PI (Dr. Kraus) and his subcontractor company, Outcome Referrals, Inc. (ORI), the TO 
is administered routinely as a core element of the PsycBC care model. That is, 
PsycBC already has an executed business agreement with ORI to have their patients 
complete the TOP as part of their standard clinical routine. Thus, we can leverage 
the existing PsycBC infrastructure to support the present study with little to no extra 
burden on administrators, providers, and patients. Moreover, although patient data 
are part of this baseline phase, they are protected within the business agreement 
between ORI and PsycBC, and the agreement allows for these coded data to be used 
to establish therapists’ performance “report cards.” So, to reiterate, patient TOP 

 
42 
data are collected as part of standard operating procedure for PsycBC. At this stage, 
we are not collecting these patient data as a research protocol; rather, these coded 
patient data points (i.e., clinical care data points) inform our match intervention (by 
establishing therapist performance report cards across at least 15 cases) that is at 
the heart of phase 2 (described below). In phase 1, we are only actively recruiting 
provider participants; thus, no patient protected health information (PHI) is 
transmitted to the research team. 
 
Importantly, at the time of this proposed IRB revision, most PsycBC clinicians who 
will choose to participate in the study will already have baseline data on the 
minimum 15 adult cases (through the patient’s actual termination point or 16 
weeks, whichever comes sooner) to establish their track record. In these cases, we 
simply need to enroll the therapist in the study (as discussed next). For therapists 
who wish to participate, but have yet to accumulate baseline performance data on 
the minimum 15 cases, we will track their performance (as per the TOP) on new, 
consecutive referrals until 15 total cases have been established for which the patient 
has either terminated or has been seen for at least 16 weeks. Few therapists will fall 
in this second category, and even if they do, they will generally only need a few 
cases to reach 15. Thus, we expect no issues completing the phase 1 performance 
baseline and finalizing the match algorithm for the phase 2 RCT by the established 
contractual milestone of 10/1/17. 
 
Our minimum target therapist sample is 44 PsycBC providers (all of whom will be 
over the age of 18 themselves, and treating patients within the age range of 18-65). 
Therapists will be psychologists, clinical counselors, and social workers. Recruitment 
will be coordinated among our UMass-employed project coordinator (PC), the 
PsycBC-employed PC, clinic staff members, and the Co-PIs. Specifically, the PsycBC 
team will verbally present information about the study (both phases 1 and 2) to their 
providers during staff meetings. Alternatively, this information can be presented 
through email. At this preliminary recruitment stage, this information will be used to 
heighten awareness about the study and to garner interest in participating. (The 
verbal script for staff meetings and the email text are included as phase 1 
attachments to this protocol.) The PsycBC PC will then provide the UMass PC (via 
email) the names of providers who expressed interest in learning more about the 
study. 
 
The UMass PC will subsequently contact interested therapist participants via email 
or teleconference (whichever is more convenient for the provider) to provide more 
study details/procedures and to direct the provider to an online consent form and 
survey. Providers remaining interested will access the secure study website to 
provide formal consent and to complete the baseline survey to which they will be 
directed after consenting. Therapists will be told that the study is examining various 
referral processes that will not affect their delivery of treatment-as-usual. They will 
be informed that they will be blind to the specific nature of the referral 

 
43 
manipulation in phase 2, but will be fully debriefed following the entire study and 
offered an opportunity to provide feedback on their experience. Therapists will also 
be informed of the assessments in which their study patients will engage in both 
phase 1 (which is standard practice) and phase 2 (though they will not have access 
to the phase 2 research data at any time). Therapists will also need to consent to 
completing the aforementioned baseline survey prior to phase 1, as well as a few 
study-specific measures for each patient during the phase 2 RCT (the baseline survey 
and the phase 2 attachments are described in the relevant sections below and are 
included as phase 1 and 2 attachments, respectively, to this protocol). Relevant to 
phase 1, therapists will be compensated with a $20 Amazon gift card for the one-
time completion of the online baseline survey, which will take no longer than 25 
minutes to complete. Non-consenting therapists will receive case assignments as per 
standard care protocol and will simply not be included in the study (though we will 
analyze consenting and non-consenting therapists on demographic differences to 
see if any systematic sample bias exists). 
 
Once therapists are enrolled in the study, the research team will access their 
naturalistically-collected TOP data to establish their performance across the 
minimum 15 cases to determine their personal strengths in treating patients across 
the risk-adjusted mental health problem domains measured by the TOP (recall that 
nothing changes in the therapist’s service operation during this phase and, in fact, 
most of these TOP data points will have already been processed through ORI for 
cases seen by the providers in the past). Specifically, to establish therapists’ 
performance track records, we will draw on each relevant patient’s coded TOP data 
from baseline, week 8, and their termination point or week 16, whichever comes 
sooner (to mimic the definition of treatment outcome in the RCT phase discussed 
below). To reiterate, the research team is not formally enrolling patients into phase 
1 of the study; rather, their coded data are simply processed by ORI, through its 
business agreement with PsycBC and its subcontractor role in the current project, to 
inform participating therapist report cards and the match algorithm).  
 
Note that enrolled therapists will have an already-established TOP ID. This will allow 
the research team to link therapists? baseline survey data to their RCT data (i.e., 
responses to their own measures and their participating patients’ measures) without 
use of any identifying information. As per customary precautions described below, a 
key that links therapist names and contract information with their data code will be 
kept in a separate, secure file that only trained research personnel can access. 
 
Relevant phase 1 attachments to this protocol:  
 
(1) Therapist recruitment materials: verbal script; email 
(2) Therapist consent form and baseline phase 1 survey measures: 
 

 
44 
Provider Characteristics Form (PCF). This measure was developed by the research 
team to assess therapist demographic information, clinical experience, degree type, 
percent time seeing various patient types/diagnoses, any specialty training they 
have received, and dimensional ratings of the influence of various theoretical 
orientations on their treatment approach. 
 
Therapist Perceived Strengths (TPS). This measure was developed by the research 
team to assess therapists’ beliefs about their effectiveness in treating the various 
TOP domains when uninformed of their data-driven TOP track record. This measure 
will allow us to examine how accurate therapists are in perceiving their own 
strengths and weaknesses. 
 
Phase 2: At this phase, the RCT will commence. The therapists will have already 
consented prior to phase 1 to be involved in the entire study, and they will know 
that patient data from their naturalistic baseline cases will have been used to create 
a personalized performance report card that will inform a prospective match with 
new patients they will treat in the trial. The therapists themselves will not see their 
report cards (as they will have been informed at the time of consent); rather, this 
information will be used by the research team with regard to the match 
manipulation.  
 
Phase 2 marks the beginning of patient recruitment into the RCT. The patient 
population will be adult men and women (age 18-65) in PsycBC’s referral stream 
(largely Cleveland clinic and primary care [PCP] practice). Recruitment to the study 
simply means a willingness to be randomized to condition based on TOP-derived 
presenting problem and to complete supplemental assessments (for monetary 
compensation, as per below) at baseline, at regular intervals during treatment, and 
at posttreatment. As this is an effectiveness design with a premium on ecological 
validity and scalability, virtually all patients in the PsycBC network will be eligible. It 
is most likely that the sample will be predominated by the following problem 
domains: depression, panic, substance abuse, and poor quality of life. The only 
study-related patient-level exclusion criterion will be patients who are not the 
primary, informed decision-maker for their care. Thus, patients will present with a 
multitude of presenting problems across a spectrum of Diagnostic and Statistical 
Manual of Mental Disorders (5th ed.; DSM-548) diagnoses. Our minimum study 
target sample size is 264 patients (6 per therapist).  
 
We do not anticipate problems meeting our recruitment numbers in the project 
time frame, as PsycBC schedules approximately 950 new patients per month. 
Moreover, their care model already uses the TOP to screen patients for appropriate 
level of care, and, as a formal subcontract on the project, they are willing to use a 
patient-level-best-matched clinician list that is generated in real time (based on the 
predictive validity of our match algorithm). Including the randomization protocol 
into the treatment delivery model will not create any systemic barriers. 

 
45 
 
Patients will flow into PsycBC via electronic or self-referrals. At initial contact, the 
PsycBC PC will ask patients for permission to be contacted by study personnel (i.e., 
the UMass PC) if they are interested in learning more about participation (this verbal 
script remains included as a phase 2 attachment to this protocol). If they are, they 
will be asked by the PsycBC PC to sign an authorization agreement (included in the 
phase 2 consent form) to allow their contact information (name, email address, and 
phone number) to be shared with the research team. The PsycBC PC’s role is 
restricted to this recruitment task and administration of authorization to release the 
limited PHI; thus, no PsycBC personnel will be engaged in human subjects’ research. 
 
The PsycBC PC will provide the UMass PC with a daily list of referrals who have 
provided signed authorization to be contacted about the study. The UMass PC will 
then contact eligible patients to schedule a baseline consent/assessment. If a patient 
agrees to engage in a consent/baseline assessment session, the PC will schedule a 
teleconference diagnostic interview via a secure platform with a trained graduate 
clinical psychology research assistant (RA). During this session, the RA will first 
review the study details/procedures and respond to any questions. Patients will be 
told that the study is examining various referral processes that will not affect their 
treatment; they will be kept unaware of the specific nature of the referral 
manipulation, but will be told that they will be fully debriefed following the study 
and offered an opportunity to provide feedback on their experience (via an exit 
interview). They will also be told that although their participation in the trial will 
largely mimic the same treatment that they would receive if they were not 
participating, they must consent to be randomized, complete extra study-specific 
measures (before, during, and after treatment), complete an audio-recorded 
diagnostic interview (before and after treatment), and accept assignment to a 
clinician who will deliver individual psychotherapy. Patients will also be asked to 
remain with the same therapist through at least 16 weeks of treatment; however, if 
they request a transfer earlier, this will be treated as a dropout point for the sake of 
the trial. 
 
If a patient consents to be enrolled, they will sign the consent form and complete a 
baseline survey of measures (i.e., the TOP-CR, TOP-CS, and TOP-CM, a brief measure 
of global distress, a measure of existential isolation, and a measure of interpersonal 
problems, all described below in the listing of relevant phase 2 attachments to this 
protocol) through a secure online platform linked to their typical TOP 
administration. Next, the trained research assistant (RA) will administer (on the 
same individual teleconference) the M.I.N.I. 7.0.2 International Neuropsychiatric 
Interview (described below in the listing of relevant phase 2 attachments to this 
protocol). Following PsycBC’s standard intake process and this research-focused 
baseline consent/assessment session, patients will be randomized to condition and 
assigned to a provider based on the experimental parameters of that condition (i.e., 
scientific match vs. pragmatic match). For their involvement in the additional 

 
46 
diagnostic assessments and the additional measures that they will complete during 
the active treatment phase, patients will be compensated with a $50 Amazon gift 
card (on a prorated schedule for any missed assessments). 
 
After the full baseline assessment, patients will be randomly assigned to condition 
(scientifically informed matched vs. pragmatic match) with a participating PsycBC 
provider. The UMass PC will generate the randomization sequences using an online 
random generator. Within condition, patients will be assigned sequentially to the 
therapists until they reach their study quota of 6 patients. Patients in the match 
condition will be assigned to therapists who have a demonstrated strength (derived 
from the baseline period) in treating, at a minimum, the patient’s highest self-
reported distress domain on the TOP-CS. Beyond the minimal match on the most 
elevated TOP-CS domain, our match algorithm will attempt to match patients to 
therapists on as many TOP-CS dimensions as possible, ultimately providing PsycBC 
with at least several well-matched choices for assignment within the match 
condition. In order to preserve this level of choice, there will be natural variability in 
the number of well-matched domains (some patients matched only on the minimum 
1 TOP-CS domain, others matched on 2 or more domains). The match variability 
across both conditions will allow us to measure degree of match dimensionally as a 
moderator variable of our main treatment effect. Therapists will also be unaware of 
their patient’s treatment condition (double blind), and they will treat both matched 
and non-matched patients (i.e., they will be crossed over the two conditions to 
minimize administrative disruptions). In the low probability event that there is no 
therapist meeting minimal match criteria for a patient in the match condition, that 
patient will be removed from the primary study analyses (though will, of course, still 
be offered treatment-as-usual at the clinic) and replaced with the next patient 
where a match does exist. As described in our power analysis below, we are 
oversampling in order to account for these “dropouts,” or removed data points. 
 
In addition to the baseline assessments already described, patients will be assessed 
via online surveys at regular intervals during treatment (the secure ORI platform will 
email hyperlinks to these surveys with reminders to complete them at the 
appropriate time intervals; the UMass PC can also follow-up with phone calls if 
needed). These during-treatment assessments will include the TOP-CS and measures 
of existential isolation and interpersonal problems at every odd-numbered week 
after the start of treatment, as well as global distress, therapeutic alliance quality, 
perceived treatment credibility, and outcome expectation after every even-
numbered session (all measures of these constructs are described below in the 
listing of relevant phase 2 attachments to this protocol). During treatment, 
participating therapists will also be asked to complete their respective versions of 
the alliance and credibility/expectation measures (also at even-numbered weeks; 
the UMass PC will email hyperlinks to these online surveys with reminders to 
complete them at the appropriate time intervals; the PC will also follow-up with 
phone calls if needed). For completing these measures, therapists will be 

 
47 
compensated $50 per patient (again in the form of Amazon gift cards). All data 
collection will be coordinated through ORI, for which patients and therapists are 
assigned unique codes. Through their business agreement, ORI has direct access to 
PBC medical records; thus, it can push the relevant measures and track 
patient/therapist progress throughout the study. 
 
As reminder, in both conditions, the providers will deliver treatment naturalistically 
(i.e., with no manipulation or influence from the research team). For the sake of the 
RCT, “treatment outcome” will be considered the point at which treatment 
terminates, or 16 weeks, whichever comes sooner. After the 16th week, or the 
termination session if it comes sooner, patients will complete posttreatment 
measures: the TOP-CS and TOP-CM, a measure of treatment satisfaction, a brief 
measure of global distress, a measure of existential isolation, and a measure of 
interpersonal problems (all described below in the listing of relevant phase 2 
attachments to this protocol). Therapists will also document the nature of 
termination (measure described below in the listing of relevant phase 2 attachments 
to this protocol). Also at posttreatment, as defined by the trial, patients will undergo 
a repeat diagnostic telephone assessment (i.e., an RA-administered M.I.N.I., as 
described above). 
 
We will also conduct a follow-up outcome assessment at 1 year after the patient’s 
own termination on a randomly-selected subsample of 40 patients. Patients can 
easily be tracked in coordination with ORI and PsycBC; further, patients will have 
provided consent for this follow-up contact (should they be randomly chosen for it). 
At this assessment point, patients will again complete online the TOP-CS and TOP-
CM, the brief measure of global distress, the measure of existential isolation, and 
the measure of interpersonal problems. 
 
Note that all self-report measures (for both patients and therapists) at all time-
points will be completed on Wi-Fi-connected tablets, or on home computers, 
through ORI’s secure web-based platform. The TOP has its own dedicated website 
and HIPAA-compliant, secure server, and all other study-specific measures will be 
integrated into the TOP administration process. 
 
We predict that the scientific match group will outperform the no match group to a 
clinically significant degree on TOP outcomes, global symptomatology, and 
interpersonal problems. We also expect that the match group will be more effective 
in promoting alliance quality and fostering more positive patient perceptions of 
treatment credibility and outcome expectation, all of which are established 
correlates (and candidate mechanisms) of positive treatment outcomes. Finally, we 
expect there to be less patient dropout in the match condition, and higher patient 
treatment satisfaction. Secondarily, we will examine 4 potential moderators of the 
expected between-group treatment effects on the primary TOP outcomes: (a) 
patient race (as it may be that the match algorithm is particularly potent, and an 

 
48 
important responsiveness tool, for historically understudied or underrepresented 
patients), (b) degree of match of therapist strengths to patient problems (rated 
dimensionally as a ratio given that therapists can be matched on more than just the 
minimum 1 domain, and the elimination of harmful matches for any distressed 
domain reported by the patient), (c) patient distress severity, and (d) complexity of 
patient presenting problem. Thus, we will test if matching is only, or particularly, 
effective under the conditions of a central patient characteristic, a multiple domain 
match, and/or for patients with the most severe or complex pathology. As noted, we 
will also assess therapists’ self-perceived strengths on the TOP domains. We expect 
to replicate previous literature showing that therapists are poor judges of their own 
efficacy, tending to underestimate negative effects and overestimate positive effects 
with their patients (Lambert, 2011), which would further underscore the importance 
of a data-driven match process. 
 
Finally, for a subsample of stakeholders, we will conduct post-trial exit interviews 
(Ns = 5 patients, 5 therapists) to gather invaluable input on how to be responsive to 
the study findings in terms of dissemination, implementation, and policymaking, 
including the potential importance of integrating diagnosis, provider age, race, or 
gender into subsequent matching approaches. We will recruit stakeholders in order 
of completion until we reach our target Ns (therapists can only be involved once 
they have treated all 6 of their study patients). There are no other 
inclusion/exclusion criteria for the exit interviews; we will simply stop asking if 
participants are interested once we have reached our target Ns. This is consistent 
with the study consent forms, which clearly state that interested participants may 
be selected to engage in the interview. 
 
Fully reflecting stakeholder engagement, and to eliminate any biases or power 
dynamics introduced by the PIs or their research staff, Advisory Board members 
(with appropriate credentialing for working with human subjects) will conduct the 
individual interviews. The PIs (Constantino & Boswell) will train 3 Advisory Board 
members on qualitative interviewing, and each will administer 1-2 pilot interviews as 
part of the training, plus 5 study interviews. The interviews will be conducted and 
audiorecorded via a secure webconferencing service and will last approximately 45-
60 minutes. Participants will be compensated with a $100 Amazon gift card for their 
time. RAs will transcribe the interviews, removing any identifying patient 
information. These RAs will also conduct a qualitative analysis of these text-based 
data. 
 
Relevant phase 2 attachments to this protocol: 
 
(1) Patient phase 2 recruitment verbal script 
(2) Patient phase 2 consent form and phase 2 baseline measures packet: 
 

 
49 
TOP-Consumer Registration Form (TOP-CR; Kraus et al., 2005). The TOP-CR will be 
used routinely during the phase 1 baseline (and the phase 2 RCT) to assess patient 
demographics. On this form, patients indicate their age, gender, ethnicity, marital 
status, income level, employment status, religious identification, education level, 
general health status, and medical and mental health treatment history. 
 
TOP-Clinical Scales and Case Mix (TOP-CS & TOP-CM; Kraus et al., 2005). This is the 
primary measure in our study; it will be used to establish the therapist report cards 
during the baseline phase to inform the match manipulation in phase 2. It also tracks 
patient outcomes. The TOP-CS consists of 58 items assessing 12 symptom and 
functional (including strengths) domains (risk-adjusted for case mix variables 
assessed via 37 items on the companion TOP-CM, such as divorce, job loss, 
comorbidity): work functioning, sexual functioning, social conflict, depression, panic 
(somatic anxiety), psychosis, suicidal ideation, violence, mania, sleep, substance 
abuse, and quality of life. Global symptom severity is assessed by summing all items 
or by averaging the z-scores (i.e., standard deviation units relative to the general 
population mean) across each of the 12 clinical scales. Domain-specific symptom 
severity is quantified as the individual z-scores for each clinical scale using general 
population means and standard deviations for the conversion. The TOP-CS has been 
shown to have excellent factorial structure, as well as good test-retest reliability 
across all scales. It is sensitive to change while possessing limited floor and ceiling 
effects (Kraus et al., 2005). The TOP also has demonstrated good convergent validity 
with scales like the Beck Depression Inventory (Beck, Steer, & Brown, 1996) and the 
Brief Symptom Inventory (Derogatis, 1975). 
 
Symptom Checklist-10 (SCL-10; Rosen, Drescher, Moos, & Gusman, 1999). To 
evaluate outcome with an index separate from the TOP (to test convergence and 
enhance the validity of any between condition effects), we will also assess global 
distress with the SCL-10, a 10-item, well-validated and widely used self-report 
inventory that assesses psychological wellbeing. 
 
Existential Isolation Scale (EIS; Pinel et al., 2014). To assess this isolation subtype, 
participants will complete the EIS, a six-item scale that requires participants to rate 
the extent to which they agree with items such as “I often have the same reactions 
to things as other people around me do” (reverse-coded) and “Other people usually 
do not understand my experiences” and “People often have the same ‘take’ or 
perspective on things that I do” (reverse-coded). Participants respond using a 7-
point scale. The EIS has high internal consistency, and has been validated extensively 
(Pinel et al., 2014). 
 
Inventory of Interpersonal Problems-32 (IIP-32; Horowitz, Alden, Wiggins, & Pincus, 
2000). To assess interpersonal problems, participants will complete the 32-item 
circumplex version of the IIP. This widely used instrument reflects interpersonal 
inhibitions and excesses, with each item rated on a 5-point scale. Higher total scores 

 
50 
indicate more interpersonal problems. The IIP-32 also has 8 subscales (Domineering, 
Vindictive, Intrusive, Cold, Socially Inhibited, Nonassertive, Overly Accommodating, 
and Self-Sacrificing) that comprise a circumplex of problematic interpersonal 
behavior around the main interpersonal dimensions of affiliation and control. Like 
the original measure (Horowitz, Rosenberg, Baer, Ureno, & Villansenor, 1988), the 
IIP-32 has evidenced good psychometric properties. 
 
(3) RA administered diagnostic assessment (baseline and posttreatment): 
 
M.I.N.I. 7.0.2 International Neuropsychiatric Interview (M.I.N.I.; Sheehan et al., 
2016). The M.I.N.I. is a brief, structured diagnostic interview for DSM-5 and 
International Classification of Diseases (ICD; World Health Organization, 2008) 
psychiatric disorder classification. With its administration time of approximately 15 
minutes, the M.I.N.I. is the psychiatric interview of choice in clinical trials and 
epidemiological studies. Despite its brevity, its psychometric properties compare 
favorably to longer instruments like the Structured Clinical Interview for DSM (SCID; 
First, Spitzer, Gibbon, & Williams, 1996). As part of the diagnostic evaluation, the 
RAs will complete the Clinical Global Impression (CGI), a widely used observer-rated 
scale that includes a 0-7 judgment of illness severity for which higher scores indicate 
more extreme illness. 
 
(4) Patient phase 2 during-treatment measures: 
 
TOP-CS, SCL-10, EIS, IIP-32. All described previously. 
 
Working Alliance Inventory-Short Form, patient version (WAI-SF-P; Tracey, 
& Kokotovic, 1989). The WAI is the most widely used alliance measure, assessing 
patient-therapist agreement on the goals and tasks of treatment, and the quality of 
their relational bond. This 12-item short form, assessing these dimensions from the 
patient’s perspective, has demonstrated sound psychometric properties. 
 
Credibility/Expectancy Scale, patient version (CEQ; Devilly & Borkovec, 2000). The 
CEQ is the most widely used and psychometrically sound measure of the patient’s 
perceived logicalness of a given treatment and expectation for the personal efficacy 
of that treatment. 
 
(5) Therapist phase 2 during-treatment measures: 
 
Working Alliance Inventory-Short Form, therapist version (WAI-SF-T; Tracey & 
Kokotovic, 1989). This is the parallel version of the WAI-SF described above, though 
now as rated from the therapist’s perspective. 
 
Credibility/Expectancy Scale, therapist version (CEQ; Devilly & Borkovec, 2000). This 
is the parallel version of the CEQ described above, though now as rated from the 

 
51 
therapist’s perspective (i.e., the therapist’s sense of how logical the patient sees the 
treatment and how optimistic the patient is about receiving benefit from it). 
 
(6) Patient phase 2 posttreatment measures: 
 
TOP-CS, TOP-CM, SCL-10, EIS, IIP-32. All described previously. 
 
TOP-Satisfaction with the Treatment Process (TOP-STP; Kraus et al., 2005). This 32-
item measure assesses patient’s satisfaction with their provider, the treatment they 
received, and the treatment milieu (e.g., staff, other patients, etc.). 
 
(7) Therapist phase 2 posttreatment measure: 
 
Nature of Termination Form (NTF). This measure was developed by the research 
team to assess the nature of patients’ termination from the provider’s open-ended 
perspective, as well as through a choice format of unilateral/patient-generated, 
unilateral/therapist-generated, or mutual. Therapists can also describe in an open-
ended format any unusual or noteworthy circumstances that may have led to the 
termination of therapy with this client (e.g., transfer of client to another therapist). 
 
(8) Patient phase 2 subsample follow-up measures: 
 
TOP-CS, TOP-CM, SCL-10, EIS, IIP-32. All described previously. 
 
(9) Stakeholder exit interview protocols (patient and therapist versions) 
 
b. State if audio or video taping will occur. Describe what will become of the tapes 
after use, e.g., shown at scientific meetings, erased. Describe the final disposition 
of the tapes. 
For the baseline and posttreatment patient assessments during phase 2, RAs will 
administer via teleconference the semi-structured diagnostic interview (M.I.N.I.), 
which will be audiorecorded. This will allow a different RA to review the recording 
and to make independent diagnostic and symptom severity determinations. With 
these two sets of ratings, we can then calculate interrater reliability on baseline and 
posttreatment diagnosis. 
 
Audio recordings from the baseline diagnostic assessments will be digitally stored 
through the secure web-conferencing service. All data will be encrypted and 
password protected. Only the necessary research team members will know the login 
and password information and have the capacity to access the recordings. When it is 
time to analyze the recordings for reliability coding, designated, trained RAs will also 
have access to the recordings. The RAs, of course, will have completed the 
mandatory ethics training in human subjects’ research, data management, and 
HIPAA compliance. These RAs will be independent evaluators who will not have 

 
52 
access to other therapist or patient data. The recordings themselves will not be 
labeled with any identifiable information. The PI will routinely monitor the collection 
and analysis of recorded data. 
 
After the recordings have been assessed for diagnostic reliability, the files will be 
securely deleted by the sponsored project contract term date of 6/16/20. No audio 
data or identifiable text data stemming from the recordings will be presented at 
meetings or in published articles. Only the reliability coefficients will be 
disseminated with the results of the full trial. 
 
c. State if deception will be used. If so, provide a rationale and describe debriefing 
procedures. Submit a debriefing script in Section #11 (Attachments). 
Although the protocol does not involve deception, it does involve incomplete 
disclosure in Phase 2 given that participants are not given all of the information 
about the study until debriefing. Thus, in the debriefing form, we now provide 
participants the opportunity to withdraw their data upon learning the full scope of 
the research. 
 
3. Background 
 
a. Describe past findings leading to the formulation of the study. 
Research has consistently identified significant variability in skill and outcomes 
between therapists (Baldwin & Imel, 2013; Boswell et al., 2013; Westra, 
Constantino, Arkowitz, & Dozois, 2011), even when therapists utilize an empirically 
supported treatment (EST). In fact, differences between treatment providers 
account for a greater portion of treatment outcome variance than the specific 
interventions delivered in controlled trials (Krause, Lutz, & Saunders, 2007; 
Wampold & Imel, 2015). Thus, improvements in MHC can occur by identifying 
effective providers in addition to promoting ESTs (Kraus et al., 2007). 
 
In the largest study to date on this topic, our team investigated therapists’ 
naturalistic treatment outcomes over many different problem domains (e.g., 
depression, anxiety, substance use, mania, sleep) in a sample of 6,960 patients and 
nearly 700 providers (Kraus, Castonguay, Boswell, Nordberg, & Hayes, 2011). The 
majority of therapists demonstrated a differential pattern of effectiveness 
depending on the problem domain, and therapist domain-specific effectiveness 
correlated poorly across domains suggesting that therapist competencies may be 
domain-specific, rather than reflecting a core attribute or general underlying 
therapeutic skill. Importantly, although some therapists demonstrated effectiveness 
over multiple problem domains, no therapists demonstrated reliable effectiveness 
across all domains. Further, a small, but notable 4% of the therapists did not 
demonstrate effective outcomes on any domain. These data suggest that in any 
population of therapists (payer network, hospital, or community mental health 

 
53 
system), there is an opportunity for behavioral health to do what medicine did 
decades ago – encourage provider specialization. Virtually every clinician has an area 
where they are above average (82-96%; Kraus et al., 2011, 2016), and our research 
suggests that if they specialize to their unique skills, population-level outcomes (i.e., 
symptom reduction, behavior change, increased functionality) will improve 
dramatically. This would reflect a major, and likely highly impactful shift to current 
MHC systems. 
 
However, patients and referrers are typically unaware of the unique track record 
(“report cards”) of local-area providers, which represents a critical gap in knowledge 
transfer within the MHC system. Without systematically collecting and disseminating 
performance report cards, stakeholders (e.g., patients, therapists, administrators 
responsible for case assignment, primary care physicians) lack vital information on 
which to base MHC choices and referral decisions, and that can inform personalized 
treatment (Boswell, Constantino, Kraus, Bugatti, & Oswald, 2015). Conversely, there 
is potentially immense advantage to matching patients to providers based on 
scientific outcome data (Constantino, Boswell, Bernecker, & Castonguay, 2013). 
 
Consistent with this notion, the Institute of Medicine (IOM) has 
made recommendations to: (a) customize care based on the patient’s needs, (b) 
share knowledge, (c) engage in data-driven decision-making, (d) promote 
transparency (including information on performance and patient satisfaction; Kohn, 
Saxena, Levav, & Saraceno, 2004), and (e) use valid and reliable assessment 
instruments to assess progress and to aid decision-making. The IOM has also 
recommended that MHC patients be provided with information on the quality of 
practitioner care (e.g., provider report cards) and use this information when making 
treatment decisions. Importantly, we have survey data that point to MHC patients, 
therapists, and administrators endorsing such applied knowledge transfer as a high 
priority (Boswell et al., 2015). Provider track record report cards are meaningful data 
to the MHC patient population, as are the mental health benefits that could stem 
from being well matched to provider. 
 
We have developed over the past 20 years an innovative, technology-based 
mechanism/intervention to deliver report cards and drive this match concept within 
a patient-centered MHC model (Kraus et al., 2011). Our longitudinal data suggest 
that our match algorithm, based on our multidimensional outcome tool (the TOP) is 
efficacious for MHC outcomes. In addition to our study highlighted above (Kraus et 
al., 2011), a more recent prospective study of 59 therapists and 3,540 patients 
resulted in a between-treatment controlled Cohen’s d effect size of .80 (Kraus et 
al., 2016). Each therapist’s first 30 patients were used to classify a therapist’s skills in 
the 12 domains of symptoms and functioning as either statistically above average, 
average, or below average. The best matching algorithm functioned as follows: for 
each new, successive patient, he or she was classified as well-matched if the risk of 
harm was eliminated (i.e., the therapist was not below average when treating any 

 
54 
elevated domain) and the therapist was above average in treating the patient’s 
three most out-of-the-norm domains (e.g., depression, suicidality, and panic). Poorly 
matched patients had below average outcomes, with small effect sizes (d = .30) 
Well-matched patients, by contrast, achieved very large pre- vs. posttreatment 
effect sizes of d = 1.19. These data lend strong support that the proposed 
comparative effective research (CER) will yield similar results (i.e., increased efficacy 
and reduced harm) in realigning the skills of a large population of therapists in one 
of the forerunner Accountable Care Organizations (our partner Atrius) when 
matching empirically derived therapist skills with patient need. The 
technology/intervention is well established, it has demonstrated efficacy, and awaits 
investigation in a well-powered RCT. 
 
4. Subject Population 
 
a. State how many subjects you propose to use and state the rationale for the 
proposed number. 
For the primary 3-level hierarchical model assessing treatment condition effects at 
the patient level on linear change rates within patients, we used Raudenbush and 
Liu’s (2001) formula as incorporated in the Optimal Design program to determine 
the minimum numbers of therapists and patients needed to detect a moderate 
effect of condition (standardized difference between change rates = .50). With a 
minimum of 6 measurements spaced over the maximum 16 treatment weeks and 
assuming 5 patients per therapist, an intra-class correlation of .15, and an alpha of 
.05, we will need a total of 44 therapists and 220 patients to achieve a power of .80 
to detect moderate condition effects on linear change rates. Factoring a 20% 
dropout rate at the patient level, running our experiment on 264 patients (6 per 
therapist) should provide sufficient statistical power to detect group differences on 
our primary outcome variables.  
 
To summarize, based on this power analysis, we will for phase 1 access a naturalistic 
baseline assessment of a minimum of 44 consenting therapists’ performance across 
a minimum of 15 cases to determine their strengths in treating the risk-adjusted 
domains measured by the TOP. We will then recruit a minimum of 264 patients for 
the phase 2 trial, assigning patients to the same 44 therapists who participated in 
phase 1 (they will see 6 cases each during the trial). 
 
b. Describe the subject population, including the age range, gender, ethnic 
background, and type of subjects (e.g. students, professors, subjects with learning 
disabilities, mental health disorders, etc.). Please incorporate specific 
inclusion/exclusion criteria (e.g. physical and psychological health, demographic 
information, or other unique characteristics). 
Therapist participants: As noted, our target sample is 44 therapist participants (age 
range = 30-65 years) who will be social workers, psychologists, and licensed clinical 
counselors. Reflecting PsycBC’s therapist pool demographics, we anticipate that our 

 
55 
provider sample will break down as follows: approximately 70% will be female; 88% 
will be white/non-Hispanic, 3% Black, 2% Hispanic, 2% “Other/mixed,” and 5% Asian. 
Based on these projections and our power analysis, our targeted/planned therapist 
enrollment is indicated in an attached Targeted/Planned Enrollment Table 
(Therapists). 
 
Patient participants: Patient participants will be 264 adult men and women (age 18-
65) in PsycBC’s referral stream (largely Cleveland clinic and primary care [PCP] 
practice). Recruitment to the study simply means a willingness to be randomized to 
condition and to complete supplemental assessments (for monetary compensation) 
at baseline, at regular intervals during treatment, and at posttreatment. As this is an 
effectiveness design with a premium on ecological validity and scalability, virtually 
all patients in the PsycBC network will be eligible. It is most likely that the sample 
will be predominated by the following problem domains: depression, panic, 
substance abuse, and poor quality of life. The only study-related, patient-level 
exclusion criterion will be patients who are not the primary, informed decision-
maker for their care. Thus, patients will present with a multitude of presenting 
problems across a spectrum of Diagnostic and Statistical Manual of Mental 
Disorders (5th ed.; DSM-5) diagnoses. The composition of our sample will roughly 
match the average utilization data for age, gender, and race/ethnicity at PsycBC. 
Based on these projections and our power analysis, our targeted/planned patient 
enrollment is attached in an Estimated Final Racial/Ethnic and Gender Enrollment 
Table (Patients). 
 
c. State the number and rationale for involvement of potentially vulnerable subjects 
to be entered into the study, including minors, pregnant women, prisoners, 
economically and educationally disadvantaged, decisionally challenged, and 
homeless people. 
We are not specifically targeting these specific vulnerable populations, and our 
research design and/or the PsycBC care system will specifically exclude minors and 
prisoners. However, given the effectiveness design focused on maximizing ecological 
validity, some of our patients are sure to have economic and educational 
vulnerabilities, which are risk factors for mental health issues. Some women might 
also be pregnant. 
 
d. If women, minorities, or minors are not included, a clear compelling rationale must 
be provided. 
Minors will be excluded because they are typically not solely responsible for their 
own treatment decisions, and the outcome measure used in this study, and on 
which the match manipulation is based, focuses on adults. 
 
e. State the number, if any, of subjects who are laboratory personnel, employees, 
and/or students. They should be presented with the same written informed 
consent. If compensation is allowed, they should also receive it. 

 
56 
N/A 
 
f. State the number, if any, of subjects who are involved in research conducted 
abroad and describe any unique cultural, economic or political conditions. 
N/A 
 
g. Describe your procedures for recruiting subjects, including how potential subjects 
will be identified for recruitment. Attach advertisements, flyers, etc. in Section #11 
(Attachments). Note: Potential subjects may not be contacted before IRB approval. 
Therapist participants:  
 
Recruitment will be coordinated among our UMass-employed PC, the PsycBC-
employed PC, clinic staff members, and the Co-PIs, and will involve presenting 
information about the study (both phases 1 and 2) to providers through verbal script 
at staff meetings or by email. At this preliminary recruitment stage, this information 
will be used to heighten awareness about the study and to garner interest in 
participating. The PsycBC PC will then provide the UMass PC (via email) the names of 
providers who expressed interest in learning more about the study. The UMass PC 
will subsequently contact interested therapist participants via email or 
teleconference (whichever is more convenient for the provider) to provide more 
study details/procedures and to direct the provider to an online consent form and 
survey. Providers remaining interested will access the secure study website to 
provide formal consent and to complete the baseline survey to which they will be 
directed after consenting. Therapists will be told that the study is examining various 
referral processes that will not affect their delivery of treatment-as-usual. They will 
be informed that they will be blind to the specific nature of the referral 
manipulation in phase 2, but will be fully debriefed following the entire study and 
offered an opportunity to provide feedback on their experience. Therapists will also 
be informed of the assessments in which their study patients will engage in both 
phase 1 (which is standard practice) and phase 2 (though they will not have access 
to the phase 2 research data at any time). Therapists will also need to consent to 
completing the aforementioned baseline survey prior to phase 1, as well as a few 
study-specific measures for each patient during the phase 2 RCT. 
 
Patient participants: 
 
Phase 2 marks the beginning of patient recruitment into the RCT. Recruitment to the 
study simply means a willingness to be randomized to condition and to complete 
supplemental assessments (for monetary compensation). Patients will flow into 
PsycBC via electronic or self-referrals. At initial contact, the PsycBC PC will ask 
patients for permission to be contacted by study personnel (i.e., the UMass PC) if 
they are interested in learning more about participation. If they are, they will be 
asked by the PsycBC PC to sign an authorization agreement (included in the consent 
form) to allow their contact information to be shared with the research team. The 

 
57 
PsycBC PC’s role is restricted to this recruitment task and administration of 
authorization to release the limited PHI; thus, no PsycBC personnel will be engaged 
in human subjects’ research.  
 
The PsycBC PC will provide the UMass PC with a daily list of referrals who have 
provided signed authorization to be contacted about the study. The UMass PC will 
then contact eligible patients to schedule a baseline consent/assessment. If a patient 
agrees to engage in a consent/baseline assessment session, the PC will schedule a 
teleconference diagnostic interview via a secure platform with a trained graduate 
clinical psychology research assistant (RA). During this session, the RA will first 
review the study details/procedures and respond to any questions. Patients will be 
told that the study is examining various referral processes that will not affect their 
treatment; they will be kept unaware of the specific nature of the referral 
manipulation, but will be told that they will be fully debriefed following the study 
and offered an opportunity to provide feedback on their experience (via an exit 
interview). They will also be told that although their participation in the trial will 
largely mimic the same treatment that they would receive if they were not 
participating, they must consent to be randomized, complete extra study-specific 
measures (before, during, and after treatment), complete an audio-recorded 
diagnostic interview (before and after treatment), and accept assignment to a 
clinician who will deliver individual psychotherapy. Patients will also be asked to 
remain with the same therapist through at least 16 weeks of treatment; however, if 
they request a transfer earlier, this will be treated as a dropout point for the sake of 
the trial. If the patient consents to be enrolled, they will sign the consent form and 
complete a baseline survey of measures (i.e., the TOP-CR, TOP-CS, and TOP-CM, a 
brief measure of global distress, a measure of existential isolation, and a measure of 
interpersonal problems) through a secure online platform linked to their typical TOP 
administration. Next, the trained RA will administer (on the same individual 
teleconference) the M.I.N.I. Following PsycBC’s standard intake process and this 
research-focused baseline consent/assessment session, patients will be randomized 
to condition and assigned to a provider based on the experimental parameters of 
that condition (i.e., scientific match vs. pragmatic match). 
 
h. Compensation. Explain the amount and type of compensation (payment, 
experimental credit, gift card, etc.), if any, that will be given for participation in the 
study. Include a schedule for compensation and provisions for prorating. 
Therapist participants:  
 
Therapists will complete, in no longer than 25 minutes, a few study-specific 
measures as part of a phase 1 baseline survey for which they will be compensated 
$20 in total (in the form of an Amazon gift card).  
 
During Phase 2, therapists will also complete a few study-specific measures 
throughout treatment with each of the 6 participating patients treated during the 

 
58 
phase 2 RCT; they will be compensated $50 per patient for this additional, but 
minimal, time burden. The compensation will again be in the form of an Amazon gift 
card. 
 
If therapists complete their measurement schedule through all possible contact 
points for a given participating patient (i.e., baseline + 16 treatment weeks = 17 
weeks), or complete their measurement schedule through a planned termination for 
a participating patient that occurs prior to week 16 of treatment, they will receive 
full compensation (i.e., a $50 gift card for that patient).  
 
However, if a therapist withdraws from the study, they will have the option to be 
compensated on a prorated basis for the measures that they have already 
completed regarding each of their participating patients. This proration works out to 
approximately $3 per week for a participating patient, which will be deducted for 
the number of weeks “missing” from therapists’ assessment schedule (i.e., based on 
the point at which the therapist withdrew from the study). For example, if a 
therapist completes the measurement schedule for a given patient through week 8 
(9 weeks, including baseline) and then withdraws from the study, they will have 
“missed” 8 weeks of data collection for that participating patient. Their 
compensation for this participating patient will be adjusted as follows: $50 - $24 ($3 
x 8 weeks) = $26. This adjustment will be completed for any and all relevant 
participating patients. To summarize, therapists who withdraw from the study will 
have the option either to (a) receive their relevant prorated compensation, or (b) to 
forgo prorated compensation in order to no longer be contacted by the research 
team.  
 
If the therapist participates in an exit interview, he or she will receive full 
compensation in the form of an additional $100 Amazon gift card. 
 
Patient participants:  
 
Patients in Phase 2 will undergo a semi-structured diagnostic interview at both 
baseline and posttreatment, as well as complete several study specific measures 
throughout treatment (and, if randomly selected, at a follow up); they will be 
compensated $50 total for these non-routine aspects of their care. The 
compensation will be in the form of an Amazon gift card. If patients complete their 
measurement schedule through all possible contact points (i.e., baseline + 16 
treatment weeks = 17 weeks), or complete their measurement schedule through a 
planned termination that occurs prior to week 16 of treatment, they will receive full 
compensation. However, if they drop out of treatment prior to week 16, and their 
end point was not a planned termination that can be considered posttreatment for 
the purpose of the study, compensation will occur on prorated schedule. This works 
out to approximately $3 per week, which will be deducted for the number of weeks 
“missing” from the schedule. For example, if a patient completes the measurement 

 
59 
schedule through week 8 (9 weeks, including baseline), and they did not engage in a 
planned termination, they will have “missed” 8 weeks of data collection. Their 
compensation will be adjusted as follows: $50 - $24 ($3 x 8 weeks) = $26. 
 
Patients who withdraw from the study (which is distinct from simply dropping out of 
treatment) will be given the option to (a) receive prorated compensation for the 
completion of measures up until the point of withdrawal (following the proration 
schedule outline above), or (b) to forgo prorated compensation in order to no longer 
be contacted by the research team. 
 
If the patient participates in an exit interview, he or she will receive full 
compensation in the form of an additional $100 Amazon gift card. 
 
Finally, note that in the event that a participant (either a patient or therapist) 
withdraws from the study during phase 2, the other dyad member (either the 
patients seen by a withdrawing therapist or the therapist treating a withdrawing 
patient) will not be penalized; that is, as long as they have already consented to the 
study, they will receive the full amount of reimbursement (i.e., a $50 gift card) 
regardless of the point at which their patient/therapist withdraws. However, note 
that consistent with the wishes of the participant, we will, of course, stop collecting 
data at the point of withdrawal (i.e., if therapists withdraw, we will stop collecting 
data from their patients who will be compensated fully; if patients withdraw, we will 
stop collecting data from their therapist regarding that patient and the therapist will 
be compensated fully for that patient). 
 
i. Please state: A: The total expected duration of the study, including the time 
expected for data analysis (e.g., This study is expected to last 1 year) AND B: How 
much time each subject is expected to be involved in the study (e.g., The 
involvement of each subject will be 1-session for a total of 90 minutes). 
A) The project is funded in the form a cost-reimbursement contract for which a 
specific milestone schedule exists. The contract start date is 9/15/16 and the 
contract term date is 6/15/20. All analyses will be completed by the term date. 
Details are available in the attached updated milestone schedule. 
 
B) Therapist subjects will be involved for 2 months in phase 1, as well as through the 
phase 2 trial (approximately 2 years, though with variability depending on when 
they have been assigned and have treated their 6 study cases). Patients in phase 2 
only will be involved in the study protocol through their actual termination point or 
16 weeks, whichever comes sooner (we will also conduct a follow-up outcome 
assessment at 1 year after termination on a randomly-selected subsample of 40 
patients). 
 
5. Risks 
 

 
60 
HHS Regulations define a subject at risk as follows: “...any individual who may be exposed 
to the possibility of injury, including physical, psychological, or social injury, as a 
consequence of participation as a subject in any research...” This also includes risks to 
subject confidentiality and any discomforts, hazards, or inconveniences. 
 
For the categories below, include a description of risks. 
 
a. Describe the risks related to: 
Physical well-being 
None anticipated. 
 
Psychological well-being 
Regardless of condition, PsycBC will employ its usual triage assessments, therapists 
will employ their usual treatments, and patients will be receiving their usual care. 
Consequently, there are no risks from our research protocol over and above what 
would normally be expected in routine assessment and psychotherapy, and PsycBC 
has its usual clinical and safety protocols in place (and the clinical personnel to 
execute them). 
 
In treatment, some individuals may experience emotional upset during sessions. 
Additionally, some participants may experience disappointment with their rate of 
progress or setbacks. The risk associated with such reactions will be addressed 
clinically by the therapists who are treating these issues and who have peer and 
administrative support. To reiterate, these treatment risks would occur in the course 
of treatment-as-usual. These are not additional risks stemming from the research 
protocol. Further, the TOP outcome monitoring system, which is at the center of our 
research project, is already being used by PsycBC providers without incident. 
 
As is typical in psychological research, some of the assessment questions from the 
research measures may be experienced as intrusive and/or may cause anxiety. The 
risk from such increased anxiety, however, is mitigated by the use of skilled and 
extensively trained assessors who are aware that such reactions may be related to a 
person’s presenting problems, or simply a function of the intimate and emotionally 
intense nature of psychological services. In addition, the PIs, PCs, and/or PsycBC 
staff and administrators will be available to meet with any participant who may be 
unduly disturbed due to the few research tasks. Because the pre- and posttreatment 
diagnostic interviews will be conducted via telephone, the graduate RA (being 
trained as a clinician and supervised by their site PI, Dr. Constantino or Boswell, both 
of whom are licensed clinical psychologists and mental health care providers) will 
have the patient’s contact information (phone number and email address) on hand. 
If the patient reveals clinically elevated suicidality or homicidality, the RA will 
contact 9-1-1 and report the patient’s contact information and location address 
(which they will request verbally, if necessary) for emergency response. The RA, if 

 
61 
applicable, will also execute any duty to warn to the best of their ability (in addition 
to contacting the local authorities). 
 
Economic well-being 
Given that therapist performance data are being collected, it is reasonable to be 
concerned about possible employment implications were an employer (i.e., clinic 
administrator) to attempt to interpret study information incompletely (i.e., infer lack 
of therapist effectiveness to the point of questioning employability). This risk, 
however, is extremely minimal for the following reasons: 
 
(1) As a condition of being involved in the study, clinic administrators will be 
required to agree that therapists’ participation or non-participation in this research 
will in no way affect their standing/employment at their community mental health 
clinic. 
 
(2) The research team will not reveal therapist performance data to clinic 
administrators or staff members; that is, the study could be considered “triple-
blind.” Neither patients nor therapists will know when they are in an experimentally-
matched vs. typically-matched dyad, and administrators/staff members will not have 
access to the therapists’ report cards. 
 
(3) However, administrators and staff members are required to be in the know 
about well-matched therapist “short-lists,” as this is essential to the research design; 
that is, when patients are randomized to a well-matched therapist, those potential 
therapists need to be identifiable. It is possible that administrators or staff members 
might misinterpret these data to suggest that a given therapist is ineffective (if he or 
she is never or rarely showing up on a shortlist). However, we will guard against this 
misinterpretation by educating administrators and staff members that the shortlist 
only represents, in a small cross-section of time, therapists that have been shown to 
be effective on at least 1 of 14 domains, which represents a given patient’s most 
severe problem at that time (the match criterion). We will stress that this does not 
mean that a therapist is globally ineffective. It may just be that patients randomly 
assigned to the match group are tending not to have the types of problems for 
which a given therapist is relatively effective. That therapist, though, could be highly 
effective at treating one or even many other domains. 
 
(4) Finally, administrators and staff members will not be told which therapists are or 
are not participating in the study. Thus, lack of being on a shortlist, for all that they 
will know (unless a therapist openly reveals that he or she is participating in the 
study), could simply connote a choice to not participate in the project. 
 
Social well-being 
None anticipated. 
 

 
62 
Breach of confidentiality (including audio/video taping) 
A breach of confidentiality represents a risk, but every step will be taken to minimize 
this risk. PsycBC and ORI routinely handle PHI and are in compliance with HIPAA 
regulations. Any “hard” materials (e.g., diagnostic assessment summaries) that are 
collected for research purposes only will be stored in a locked cabinet in the PI’s 
Psychotherapy Research Lab. There will be no hard copy data collected at the 
PsycBC clinic sites. Most of the data collected in this study (including consent) will be 
through a secure, web-based platform using a tablet or computer. This method 
offers greater protection because it guards against human error and negates the 
need for long-term storage of paper forms. Finally, digital recordings of diagnostic 
assessments will be stored in a secure, password protected website. The recordings 
themselves will be encrypted 
 
b. For research conducted internationally, describe any political or sociocultural 
considerations that may affect your research design (for example, in some 
communities it may not be customary to sign documents, etc.) 
N/A 
 
c. Discuss plans for ensuring necessary medical or professional intervention in the 
event of a distressed subject. 
The Co-PIs, project coordinator, PsycBC staff members, and PsycBC administrators 
will monitor the treatments and data collection; thus, they can assist in regularly 
monitoring any adverse events. Such negative occurrences are unlikely to be trial-
related, as all patients will be receiving treatment-as-usual. Therefore, any adverse 
event will be addressed with PsycBC’s well-established procedures for monitoring 
services and managing treatment-related disturbances. Nevertheless, any adverse 
event will be recorded and immediately reported to the IRB (UMass), PCORI 
(funder), and the project’s Data Safety and Monitoring Board (DSMB). 
 
Should, during the course of the study, a patient show evidence of psychological or 
physical deterioration, the patient will be assessed comprehensively in the domains 
of concern (except in the case of a life-threatening physical emergency, such as the 
emergence of acute chest pain, in which case 9-1-1 will be called immediately). If the 
therapist deems that the patient meets criteria for a psychiatric hold (e.g., patient is 
an imminent danger to self or others), the therapist will arrange for the patient to be 
brought to the emergency department and will contact his/her PsycBC administrator 
and the PI to debrief. If a patient is not meeting criteria for a psychiatric hold, but is 
showing clear signs of decreased mental status, the therapist will continue to meet 
with the patient, as well as - in consultation with the PsycBC administrator - make 
arrangements for the most appropriate level of care. 
 
As noted, because the pre- and posttreatment diagnostic interviews will be 
conducted via telephone, the graduate RA (being trained as a clinician and 
supervised by their site PI, Dr. Constantino or Boswell, both of whom are licensed 

 
63 
clinical psychologists and mental health care providers) will have the patient’s 
contact information (phone number and email address) on hand. If the patient 
reveals clinically elevated suicidality or homicidality, the RA will contact 9-1-1 and 
report the patient’s contact information and location address (which they will 
request verbally, if necessary) for emergency response. The RA, if applicable, will 
also execute any duty to warn to the best of their ability (in addition to contacting 
the local authorities). 
 
6. Benefits 
 
a. Describe the potential benefit(s) to be gained by the subjects or by the acquisition 
of important knowledge which may benefit future subjects, etc. (This DOES NOT 
include compensation or extra credit). 
The most direct benefit a participant in this study may receive is the reduction of 
symptom-related distress and improved functioning. In addition, patients (especially 
those in the match condition) will receive more personalized MHC. Psychotherapists 
(especially those in the match condition) may experience a greater level of positive 
impact across their caseloads. Given that the actual treatments being provided will 
not be manipulated, the benefits of participation are judged to far outweigh the 
potential study-specific risks. 
 
There is immense potential for future therapists and patients to benefit from the 
results of this study; if the hypotheses are supported, there will be cause for 
substantial revamping of MHC systems to capitalize on matching patients to 
therapists who have an empirically demonstrable track record of strength in treating 
patients with similar presenting problems. 
 
7. Procedures to Maintain Confidentiality 
 
a. Describe the procedures in place which protect the privacy of the subjects and 
maintain the confidentiality of the data, as required by the federal regulations, if 
applicable. 
Multiple steps will be taken to protect confidentiality. As mentioned, minimal paper 
forms (e.g., diagnostic summary forms) will be kept in a locked cabinet in the PI’s 
locked Psychotherapy Research Lab. There will be no hard copy data collected at the 
PsychBC sites. Virtually all of the data collected in this study (including consent) will 
be through a secure, web-based platform using a tablet or computer. This method 
offers greater protection because it guards against human error and negates the 
need for long-term storage of paper forms. Digital recordings of diagnostic 
assessments will be stored in a secure, password protected website. The recordings 
themselves will be encrypted. 
 
Only designated study personnel will have access to identifiable, study specific, 
private information about human subjects. When registering on the TOP system, as 

 
64 
required by PsychBC’s standard operating procedures, both patients and therapists 
are assigned a random number code that links all subsequent assessments and is 
separated from identifiable information. This random number code will function as 
each participant’s study code and will be used to link participants’ data. As noted, all 
therapist and patient data (outside of diagnostic assessment summaries and the TOP 
administrations) will be collected through a web-based platform. The assigned 
participant code will be used to link/aggregate information, so private information 
will not be requested after the baseline assessment/consent process. Only the PI 
and essential research staff will have access to the list that links identifiable 
information with the participant’s study code. Any audio recordings will be 
encrypted and password protected. Only the Co-PIs will know this password and 
have the capacity to access the recordings. When it is time to analyze the recordings 
for reliability coding, designated, trained RAs will also have access to the recordings; 
however, they will not have access to additional identifiable information (only the 
information required to complete the analysis). For any data used for research and 
publication purposes, the confidentiality of participant information will be ensured. 
 
b. If information derived from the study will be provided to a government agency, or 
any other person or group, describe to whom the information will be given and the 
nature of the information. 
The PI is required to submit information (i.e., contractual “deliverables”) on a regular 
basis to PCORI (the study sponsor), including IRB protocols, interim progress reports, 
advisory board meeting minutes, engagement plan updates, evidence of diagnostic 
criterion reliability from training cases, interim data reports, presentation abstracts 
and documentation of acceptance, manuscript copies, letters of endorsement from 
scientific and consumer groups, final data analysis summary, and final research 
report. Details on deliverables are available in the aforementioned (and attached 
and updated) milestone schedule. No PHI will be transmitted to PCORI. 
 
c. Specify where and under what conditions study data will be kept, how specimens 
will be labeled and stored (if applicable), who has access to the data and 
specimens, and what will be available to whom. 
As noted, minimal paper forms (e.g., diagnostic summary forms) will be kept in a 
locked cabinet in the PI’s locked Psychotherapy Research Lab. There will be no hard 
copy research-only data collected at the PsycBC sites. Virtually all of the data 
collected in this study (including consent) will be through a secure, web-based 
platform using a tablet or computer. Digital recordings of diagnostic assessments 
will be stored in a password protected website, and securely deleted by the project 
contract’s term date. Only the relevant members of the research team will have 
access to the participants’ data and only the PI will have long-term access to 
identifiable information. As noted, all assessments will be linked with a participant 
code. Any records linking the code to the participant’s name or voice recording will 
be kept in a separate locked file cabinet in the PI’s office. These records will be 
destroyed 5 years after the contract term date. 

 
65 
 
8. Potential Conflict of Interest 
 
a. Do any of the involved investigators or their immediate family (as described 
below) have consulting arrangements, management responsibilities or equity 
holdings in the Sponsoring company, vendor(s), provider(s) of goods, or 
subcontractor(s)? Y 
 
b. Do any investigators or their immediate family have any financial relationship with 
the Sponsoring company, including the receipt of honoraria, income, or 
stock/stock options as payment? N 
 
c. Is any Investigator(s) a member of an advisory board with the Sponsoring 
company? N 
 
d. Do any investigators receive gift funds from the Sponsoring company? N 
 
e. Do any investigators or their immediate family have an ownership or royalty 
interest in any intellectual property utilized in this protocol? Y 
 
“Immediate family” means a spouse, dependent children as defined by the IRS, or a 
domestic partner. If one or more of the above relationships exist, please include a 
statement in the consent form to disclose this relationship. i.e., a paid consultant, a paid 
member of the Scientific Advisory Board, has stock or stock options, or receives 
payment for lectures given on behalf of the sponsor. The consent form should disclose 
what institution(s) or companies are involved in the study through funding, cooperative 
research, or by providing study drugs or equipment. If you answer yes to any of the 
questions above, please go to the policies for more information. 
 
9. Informed Consent 
 
You can add different Consent Forms, Alteration Forms, and Waivers. Provide consent 
process background information, in the table below, for each Consent Form(s), Alteration 
Form(s), and Waiver(s). 
 
9.1. Consent Form – therapist consent form revised 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
UMass personnel only: either the PC or an RA. 
 
How is consent being obtained? 

 
66 
Therapists will meet or speak via teleconference with the UMass PC or an RA to learn 
about the study details/procedures and to provide formal consent through an online 
baseline survey to which they will be directed. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
The PI and his collaborators will provide close oversight of the entire protocol, including 
regular consultations with a study Advisory Board and the DSMB. 
 
9.2. Consent Form – therapist exit interview supplemental consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
The exit interviewer (i.e., RA or advisory board member). 
 
How is consent being obtained? 
If a therapist agrees to engage in an exit interview, the interviewer will review the study 
details/procedures and obtain supplemental consent through an online link to which the 
therapist will be directed. Coercion will be minimized by clearly stating that participation 
is voluntary. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
If the interviewer interacts with a therapist who appears to have competency issues in 
the decision-making process for engaging in the exit interview, they will immediate 
bring this concern to the PI or a Co-PI before enrolling them. The team will them make 
an informed decision as to whether to include that person in the interview protocol. 
 
9.3. Consent Form – patient consent form revised 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
UMass personnel only: either the PC or an RA. 
 
How is consent being obtained? 
If a patient agrees to engage in a consent/baseline assessment session, the PC will 
schedule a telephone diagnostic interview with a trained graduate clinical psychology 
RA (employed at either UMass or University at Albany). The RAs will first review the 
study details/procedures and obtain consent through an online baseline survey to which 
the patient will be directed. Coercion will be minimized by clearly stating that 
participation is voluntary and will in no way impact the patient’s treatment. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 

 
67 
Competency for making one’s own treatment decisions will be an inclusion criterion for 
the study. Moreover, if a clinic staff member, the PC, or an RA interacts with a patient 
who appears to have competency issues in the decision-making process for engaging in 
the study, they will immediate bring this concern to the PI or a Co-PI before enrolling 
them. The team will them make an informed decision as to whether to include that 
person in the study. The DSMB will be consulted if appropriate. 
 
9.4. Consent Form – patient exit interview supplemental consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
The exit interviewer (i.e., RA or advisory board member). 
 
How is consent being obtained? 
If a patient agrees to engage in an exit interview, the interviewer will review the study 
details/procedures and obtain supplemental consent through an online link to which the 
patient will be directed. Coercion will be minimized by clearly stating that participation 
is voluntary and will in no way impact the patient’s treatment. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
Competency for making one’s own treatment decisions will have been an inclusion 
criterion for the main study. Moreover, if the interviewer interacts with a patient who 
appears to have competency issues in the decision-making process for engaging in the 
exit interview, they will immediate bring this concern to the PI or a Co-PI before 
enrolling them. The team will them make an informed decision as to whether to include 
that person in the interview protocol. The DSMB will be consulted if appropriate. 
 
10. Assent Background 
 
All minors must provide an affirmative consent to participate by signing a simplified assent 
form, unless the Investigator(s) provides evidence to the IRB that the minor subjects are not 
capable of assenting because of age, maturity, psychological state, or other factors. 
 
11. Attachments 
 
Document Type 
Document Name 
Attached Date 
Questionnaires 
Patient Phase 2 
During-Treatment Measures 
Packet 
10/04/2016 
Questionnaires 
Therapist Phase 2 
During-Treatment Measures 
Packet 
10/04/2016 
Questionnaires 
TOP-STP 
10/04/2016 

 
68 
Questionnaires 
Stakeholder Exit Interview 
Protocols 
10/04/2016 
Federal Grant/Sub-
contract 
PCORI IHS-1503-
28573_Constantino_executed 
contract 
10/04/2016 
Federal Grant/Sub-
contract 
PCORI Original Contract 
Proposal_all sections 
10/04/2016 
Other 
Constantino Lab Personnel 
Link- Google Docs 
10/04/2016 
Other 
PCORI_Phase 2_Patient Data 
Collection Email Template 
11/13/2016 
Other 
PCORI_Phase 2_Patient Data 
Collection Reminder Call 
Script 
11/13/2016 
Other 
PCORI_Phase 2_Therapist 
Data Collection Email 
Template 
11/13/2016 
Other 
PCORI_Phase 2_Therapist 
Data Collection Reminder Call 
Script 
11/13/2016 
Questionnaires 
MINI 7.0.2 Standard 
11/13/2016 
Advertisements 
PCORI_Clinician 
Recruitment_Verbal 
Script_REVISED_clean 
08/13/2017 
Advertisements 
PCORI_Clinician 
Recruitment_Email_REVISED_clean 
08/13/2017 
Advertisements 
PCORI_Patient 
Recruitment_Verbal 
Script_REVISED_clean 
08/13/2017 
Questionnaires 
PCORI_Clinician Consent & 
Baseline Measures 
Packet_REVISED 
08/13/2017 
Questionnaires 
PCORI_Patient Consent & 
Baseline Measures 
Packet_REVISED 
08/13/2017 
Questionnaires 
PCORI_Patient Posttreatment 
Measures Packet_REVISED 
w. debriefing form 
08/13/2017 
Questionnaires 
PCORI_Clinician 
Posttreatment Measures 
Packet_REVISED w. 
debriefing form 
08/13/2017 
Questionnaires 
TOP-CS & TOP-CM 
08/13/2017 

 
69 
Other 
PCORI_Targeted Enrollment 
Tables_REVISED_clean 
08/13/2017 
Other 
PCORI Milestone 
Schedule_REVISED 
08/13/2017 
Other 
Participant Flow_REVISED 
08/13/2017 
Other 
Data Collection Schedule 
Revised 
08/13/2017 
Federal Grant/Sub-
contract 
Constantino_IHS1503-28573_Mod 
001 SUB_FE 20170808_FINAL 
EXECUTED MOD 
08/13/2017 
Other 
PCORI IRB Proposal_R1_for 
PsycBC_FINAL submitted 
08/13/2017 
Other 
ORI-PBC_Business Associate 
Agreement 
08/13/2017 
 
Obligations 
 
Obligations of the Principal Investigator are: Modifications - Changes in any aspect of the study 
(for example, project design, procedures, consent forms, advertising materials, additional key 
personnel or subject population) will be submitted to the IRB for approval before instituting the 
changes; Consent Forms - All subjects will be given a copy of the signed consent form. 
Investigators will be required to retain signed consent documents for six (6) years after close of 
the grant or three (3) years if unfunded; Training - Human subject training certificates, including 
those for any newly added personnel, will be provided for all key personnel; Adverse Events - 
All adverse events occurring in the course of the protocol will be reported to the IRB as soon as 
possible, but not later than ten (10) working days; Continuing Review – IRB Protocol Report 
Forms will be submitted annually at least two weeks prior to expiration, six weeks for protocols 
that require full review; Completion Report - The IRB will be notified when the study is 
complete. To do this, complete the IRB Protocol Report Form and select “Final Report.” Training 
- Human subject training certificates, including those for any newly added personnel, will be 
provided for all key personnel; Adverse Events/Unanticipated Problems - All events occurring in 
the course of the protocol will be reported to the IRB as soon as possible, but not later than five 
(5) working days; Continuing Review - IRB Protocol Report Forms will be submitted annually at 
least two weeks prior to expiration, six weeks for protocols that require full review; Completion 
Report - The IRB will be notified when the study is complete. To do this, complete the IRB 
Protocol Report Form and select “Final Report.” 
 
The Principal Investigator has read and agrees to abide by the above obligations. Y 
 
 

 
70 
C. Summary of the 2nd formal modification to the PCORI contract (September 2018), and the 
full revised study protocol (2nd revision) submitted to, and approved by, the University of 
Massachusetts Amherst Institutional Review Board (March 2018) 
 
CONTRACT MODIFICATION SUMMARY 
 
On September 14, 2018, PCORI approved the 2nd contract modification (IHS-1503-28573 
Modification 002), which included the following changes – all in the service of boosting patient 
recruitment and retention: 
 
• We increased the number of PsycBC telephone intake specialists who were trained to 
recruit and assign study patients. This increased the number of potential patients who 
could be recruited to the study on any given day. 
 
• We revised the language of the study script/pitch that the PsycBC telephone intake 
specialists used to recruit patients. This revision better emphasized how completing a 
routine outcome measure was already standard practice at PsycBC, and how completing 
such a measure at intake could improve a patient’s quality of care through personalized 
matching to providers. This immediately normalized the measurement process as part 
of usual care, and it highlighted personalized care over participant burden. In fact, the 
entire first part of the recruitment pitch had to do with clinical care; the introduction of 
the study came after, and it was billed as an opportunity to be part of an ongoing 
project on this personalized care notion and to earn financial compensation for doing 
so. 
 
• PsycBC began offering periodic incentives (in the form of a payment bonus or tickets to 
local events) to the intake specialist who successfully directed the most patients to our 
online study consent form in a given period of time (e.g., a 1-week competition). This 
bonus was completely unrelated to the project budget; it was a motivational strategy 
within their own payroll system. 
 
• We started offering a $15 recruitment incentive. This compensation incentivized 
patients’ willingness to leave the initial intake call to review the study consent form. We 
felt that this would be useful given that a high percentage of people were enrolling if 
they reviewed the materials. However, getting potential participants to agree to review 
the form was an early challenge. 
 
• We eliminated the diagnostic interview calls (using the M.I.N.I.), as these assessments 
were providing little yield and may have been perceived as off-putting and burdensome. 
In practice, many early patients who enrolled in the study were failing to keep their 
baseline telephone assessment appointment with our research assistants. Although we 
regularly followed up to reschedule, we feared that these potentially burdensome 
assessments were posing a risk to retention. Moreover, even if it was not leading to a 
patient dropping out of the study, many enrolled patients were completing all 

 
71 
assessments other than the M.I.N.I., which was resulting in missing data regardless. We 
also wondered whether when people read the consent form and saw that we were 
asking them to engage in two 30-minute phone calls in addition to completing 
measures, this may have deterred them from enrolling. Thus, given the limited yield (at 
best) and overt disruption (at worst) of the diagnostic interview, we dropped it from our 
protocol. Fortunately, the data were never intended to be primary, and we could still 
characterize our sample with the TOP data (our primary match and outcome measure). 
Further, the diagnostic assessments were not included in PsycBC’s standard intake 
process (as they were with our former clinical partner). Thus, using them actually 
rendered our study less naturalistic vis-à-vis the system that we were trying to 
affect/improve with our intervention. 
 
• PsycBC hired a full-time employee whose sole job description was to recruit patients to 
the trial. 
 
• Our PsycBC collaborator, Tom Swales, who holds a significant amount of regional 
credibility, agreed to liaise with community physicians to market the study, which they 
could then mention to their patients prior to them contacting PsycBC. We suspected 
that the more patients were in the know about the project prior to calling or arriving for 
care, the more likely they would agree to take part vs. view it as an unexpected 
inconvenience. 
 
• We posted a special announcement about the study on PsycBC’s website, with the idea 
that it might predispose patients to participate if they read about the potential personal 
benefits before the intake call. 
 
• We raised the upper age limit of study-eligible patients to 70 instead of 65. The 
literature does not demarcate older adulthood at 65, so we felt that raising the age 
would have no untoward effect on response to treatment. 
 
• We continued to monitor PsycBC therapists who were not enrolled in the study, but 
now had the requisite number of baseline cases with TOP data for which we could 
establish a baseline report card. This would allow us to recruit from this pool if there 
was employment turnover from study-enrolled clinicians, or if other strategic needs 
arose (e.g., if having more therapists at a particular site could also positively affect 
patient recruitment and retention). 
 
• The project’s milestone schedule was unrevised for this contract modification. 
 
For this modification, the only major changes to the protocol included the recruitment 
incentive and increase in the upper patient age limit, which our funder, PCORI, had already 
suggested and verbally approved in March 2018. Thus, at that time, we submitted the following 
2nd (and minor) revision of our study protocol to the UMass IRB, which was approved in March 

 
72 
2018. This was the final protocol in place for the remainder, and majority, of the study. (Note 
that the protocol still references the diagnostic interviewing component, as that study element 
was not jettisoned until September 2018, as per the contract modification details noted above.) 
 
PROTOCOL 
APPLICATION FORM 
SOCIAL, BEHAVIORAL, AND EDUCATIONAL FULL BOARD 
HUMAN SUBJECTS IN SOCIAL, BEHAVIORAL, AND EDUCATIONAL 
RESEARCH 
 
University of Massachusetts Amherst (UMass) 
Institutional Review Board (IRB) 
 
Protocol ID: 2016-3401 
Title: Enhancing Mental Health Care 
 
Revision Form 
1. Summarize the proposed changes to the protocol in lay terms (including details of ALL 
changes proposed AND modify all relevant protocol sections and attachments accordingly). 
By way of a brief reminder, subjects in the current study include two mental health care 
stakeholder groups: (1) therapists affiliated with Psychological and Behavioral Health 
Consultants (PsycBC) who are providing outpatient psychotherapy, and (2) adult patients 
receiving psychotherapy (for varied mental health complaints) from the participating PsycBC 
therapists. PsycBC is a formal subcontract to UMass on this project, and their role on this 
project is restricted to providing the research team access to these two subject populations, 
and assisting the team in recruitment. Thus, PsycBC is not engaged in human subjects’ research. 
The amendment proposed here deals solely with patient recruitment procedures. There are no 
changes to the research protocol itself. 
 
Specifically, we are behind in our recruitment milestones, and our funder, PCORI, has asked us 
to consider strategies for increasing recruitment. In response, and internal to their business, 
PsycBC has provided additional resources to help boost recruitment. For example, they have 
devoted more intake staff to accept calls and to pitch the study via our verbal recruitment 
scripts. We initially rolled out the recruitment via just one of several intake call lines in order to 
work closely with just one PsycBC staff member. This, however, limited the number of potential 
patients to be recruited during a given day; opening up multiple lines for recruitment should 
help increase our numbers. PsycBC has also implemented a financial bonus for the intake 
worker who successfully recruits the most patients. We appreciate PsycBC’s active role in 
attempting to bump recruitment; however, our PCORI Program Officer is concerned that these 
internal resources changes might not be sufficient on their own. Hence the present proposed 
amendment. 
 
For context, at present, more PsycBC patients than we anticipated simply decline to learn more 
about the study during their initial intake call, presumably because they are eager to be 

 
73 
assigned to a clinician immediately on that initial call vs. going to a website to read about the 
study, to consent, to complete baseline measures, and then to return to a second intake call to 
be assigned to their clinician. Although we have successfully recruited 40 patients at present, 
with data to support that people are generally willing to participate if they agree to access the 
study information/consent form, we need to increase the number of people agreeing to access 
our online study consent form in order to catch up to our recruitment milestone projections 
(currently set at 66 patients recruited by March 1, and 112 by April 15). 
 
The proposed strategy, which was recommended by our Program Officer, is to provide a 
monetary incentive for patients to agree to review our study materials online vs. declining 
outright on the initial intake call. PCORI has worked with research teams in the past who have 
used this strategy to successful effect, and they are willing to help us re-work our budget if the 
IRB approves this recruitment incentive. Our rationale is that a small monetary incentive may 
have a big impact in getting people to agree to pause momentarily their intake process to learn 
about, and consent to, our study (as noted, once patients get to the consent form, they often 
agree to participate). Given that reading the consent form takes several minutes, and that the 
person has to be willing to have their intake process span two different calls (which can delay 
by minutes to hours their assignment to a PsycBC therapist), we think that it is reasonable to 
compensate potential participants $15 for this time added to the intake process. As noted, 
because this money is tied to recruitment only, not participation, it is squarely a recruitment 
incentive, not a participant compensation/payment (for which a compensation schedule 
already exists in the current protocol). Although PCORI originally suggested offering $25, as did 
our DSMB and Advisory Board when consulting them about our recruitment issue, we feel that 
this might end up being a disincentive to actually participate (i.e., a person may be content with 
earning $25 simply to read a consent form, but then say “no thanks” to participating). Instead, 
we think that offering enough to be an incentive, but an amount that is more proportional to 
the time ask and to the compensation being offered for engaging in the full study protocol (i.e., 
$50), is likely to be more effective. 
 
In sum, we are asking for approval to offer a $15 recruitment incentive to access our study 
consent form. We are also increasing the upper age limit of patients from 65-70. As the 
literature does not demarcate older adulthood at 65, this change is very minor, but might allow 
us to recruit a few extra patients who are interested in participating. There are no new 
attachments or other revisions to the study protocol language for this proposed amendment, 
and this recruitment incentive incurs no additional risk to potential participants. 
 
2. Indicate Level of Risk involved with the changes proposed. 
No change. 
 
3. Describe any Other Changes. 
As our funder is eager to learn if we can implement this recruitment incentive, I can be 
available to talk during your meeting on 3/7 if questions arise. My cell phone is 413-320-5752. 
Thank you! 
 

 
74 
Protocol Director: Michael J. Constantino 
Degree: PhD 
Title: Professor 
Department Name: Psychological & Brain Sciences 
Mailing Address: 612 Tobin Hall, 135 Hicks Way 
Phone: 5-1388; Fax: 5-0996 
E-mail: mconstantino@psych.umass.edu 
Human Subjects Training Completed? yes 
 
Subject Populations(s) Checklist 
Yes/No 
 
Minors (under 18) 
Pregnant Women 
Cognitively Impaired or Decisionally Challenged 
Older individuals (75 and over) 
Healthy Volunteers 
Students/Employees 
International Populations 
Prisoners 
Other (i.e., any population that is not specified above) 
 
N 
N 
N 
N 
N 
N 
N 
N 
Y 
 
Other: Subjects will include two mental health care stakeholder groups: (1) therapists affiliated 
with PsycBC who are providing outpatient psychotherapy, and (2) adult patients receiving 
psychotherapy for varied mental health complaints from the participating therapists. PsycBC, a 
formal subcontract to UMass on this project, is an innovative health care organization and one 
of the largest providers of outpatient mental healthcare services in Ohio. PsycBC’s role on this 
project is restricted to providing the research team access to these two subject populations, 
and assisting the team in recruitment. Thus, PsycBC is not engaged in human subjects’ research. 
 
Study Location(s) Checklist 
Yes/No 
 
University of Massachusetts Amherst 
Baystate Medical 
University Health Services 
Hartford Hospital 
Other (Specify other Study Locations) 
 
Y 
N 
N 
N 
Y 
 
Other: All study operations will be coordinated through Dr. Michael Constantino’s (PI) 
Psychotherapy Research Lab at UMass Amherst. Subject data will be collected through our 
clinical partner, PsycBC, which employs a large team of psychiatrists, advanced practice nurses, 
psychologists, clinical counselors, and social workers serving children, adolescents, adults, and 
families in locations throughout Ohio and northern Kentucky. PsycBC’s experienced specialists 
provide therapy for a wide range of mental health issues. PsycBC includes multiple treatment 
sites in Ohio that will contribute to data collection. 

 
75 
 
General Checklist 
Yes/No 
 
Training Grant? 
Funded Study (or proposal submitted to sponsor)? 
Cooperating Institution(s)? 
Federally Sponsored Project? 
Human blood, cells, tissues, or body fluids (tissues)? 
Subjects will be paid for participations? 
 
N 
Y 
Y 
Y 
N 
Y 
 
Cooperating Institution(s): (1) University at Albany, SUNY (Dr. James Boswell; Co-PI and 
subcontract); (2) Outcome Referrals Institute, Inc. (ORI; Dr. David Kraus; Co-PI and subcontract); 
and (3) PsycBC (Dr. Tom Swales; subcontract director). Note: At the time of this revision, an IAA 
has already been established for the approved original protocol with SUNY Albany and ORI. 
After consulting with UMass IRB staff, it is now clear that our new subcontract, PsycBC, is not 
engaged in human subjects’ research; thus, no IAA is required/requested. 
 
Funding Checklist 
Grants/Contracts: 
 
Funding Administered By: UNIVERSITY 
PGCA#: 1503-28753 
GAID#:  
Funded By: Patient-Centered Outcomes Research Institute 
1828 L Street, NW, Suite 900 
Washington, DC 20036 
Phone: (202) 827-7700 | Fax: (202) 355-9558 
info@pcori.org 
Principle Investigator: Michael J. Constantino 
Grant/Contract Title: Enhancing Mental Health Care by Scientifically Matching Patients to 
Providers’ Strengths 
 
Are the contents of this protocol the same as described in grant/contract proposal? Y 
Is this a training grant? N 
Are any subcontracts issues under this grant? Y 
 
Fellowships – None 
Gift Funding – None 
Dept. Funding – None 
Other Funding – None 
 
1. Purpose of the study 
 
a. Provide a brief lay summary of the purpose of the study.  

 
76 
Research has shown that mental health care (MHC) providers differ significantly in 
their ability to help patients. In addition, providers demonstrate different patterns of 
effectiveness across symptom and functioning domains. For example, some 
providers are reliably effective in treating numerous patients and problem domains, 
others are reliably effective in some domains (e.g., depression, substance abuse) yet 
appear to struggle in others (e.g., anxiety, social functioning), and some are reliably 
ineffective, or even harmful, across patients and domains. Knowledge of these 
provider differences is based largely on patient-reported outcomes collected in 
routine MHC settings. 
 
Unfortunately, provider performance information is not systematically used to refer 
or assign a particular patient to a scientifically based best-matched provider. MHC 
systems continue to rely on random or purely pragmatic case assignment and 
referral, which significantly “waters down” the odds of a patient being 
assigned/referred to a high performing provider in the patient’s area(s) of need, and 
increases the risk of being assigned/referred to a provider who may have a track 
record of ineffectiveness. This research aims to solve the existing non-patient-
centered provider-matching problem. 
 
Specifically, we aim to demonstrate the comparative effectiveness of a scientifically-
based patient-provider match system compared to status quo pragmatic case 
assignment. We expect in the scientific match group significantly better treatment 
outcomes (e.g., symptoms, quality of life) and higher patient satisfaction with 
treatment. We also expect to demonstrate feasibility of implementing a scientific 
match process in a community MHC system and broad dissemination of the easily 
replicated scientific match technology in diverse health care settings. The 
importance of this work for patients cannot be understated. Far too many patients 
struggle to find the right provider, which unnecessarily prolongs suffering and 
promotes health care system inefficiency. A scientific match system based on 
routine outcome data uses patient-generated information to direct this patient to 
this provider in this setting. In addition, when based on multidimensional 
assessment, it allows a wide variety of patient-centered outcomes to be represented 
(e.g., symptom domains, functioning domains, quality of life). 
 
b. What does the Investigator(s) hope to learn from the study?  
The goal of this project is to test the effectiveness of an innovative, scientifically-
informed patient-therapist referral match algorithm based on MHC provider 
outcome data. We will employ a randomized controlled trial (RCT) to compare the 
match algorithm with commonplace pragmatic referral matching (based on provider 
availability, convenience, or self-reported specialty). Psychosocial treatment will 
remain naturalistically administered by varied providers (e.g., psychologists, social 
workers) to patients with mental health concerns. We hypothesize that the scientific 
match group will outperform the pragmatic match group in decreasing patient 
symptoms and treatment dropout, and in promoting patient functional outcomes, 

 
77 
perceived treatment credibility, outcome expectation, and care satisfaction, as well 
as therapeutic alliance quality. Doing so will establish the match algorithm as a 
mechanism of effective patient-centered MHC, and will suggest that this 
scientifically derived patient-provider matching intervention can be integrated into 
MHC systems to aid in treatment decision making, as well as increase 
personalization. 
 
2. Study Procedures 
 
a. Describe all study procedures. 
We will compare the efficacy of naturalistic treatment either with or without the aid 
of scientific matching to a provider with a double-blind RCT. The project will involve 
two main phases. First, we will access a naturalistic baseline assessment of 
consenting PsycBC therapists’ performance to determine their relative strengths and 
weaknesses in treating the problem domains measured by a multidimensional 
outcome tool. This period will establish our therapist sample pool and inform the 
RCT match manipulation (a match will represent a patient being assigned to a 
therapist who has empirically demonstrated during the baseline phase that he or 
she is stably effective at treating patients with the same type of presenting 
complaint). 
 
Second, and after the baseline period, new consenting outpatients will be randomly 
assigned to the match (experimental) or no match (control) condition. The PsycBC 
administrators and their project-specific coordinator will collaborate with the 
research team to apply the randomization protocol. Treatment outcome will be 
assessed through the patient’s actual termination point or 16 weeks, whichever 
comes sooner (we will also conduct a follow-up outcome assessment at 1 year after 
the point of termination on a randomly selected subsample). Outside of being 
matched to a therapist from a short-list of providers who have demonstrated (during 
the phase 1 baseline) reliable success in treating the patient’s primary problem area, 
and completing study-specific measures for which participants will receive monetary 
compensation, treatment will be delivered as usual (the short list still allows for 
pragmatic considerations like availability and administrator assignment options). 
 
Additional methodological details by study phase follow. 
 
Phase 1: The most significant revision to the research protocol is that we no longer 
need to recruit/enroll patients for phase 1. Rather, phase 1 now focuses solely on 
PsycBC clinicians as our research participants. To inform the match condition, we will 
first establish the baseline track record of participating therapists’ performance 
(across a minimum of 15 adult psychotherapy cases each) to determine their 
strengths in treating behavioral health domains measured by the primary outcome 
measure on which the match algorithm is based – the Treatment Outcome Package 
(TOP; Kraus, Seligman, & Jordan, 2005), which is described below in the listing of 

 
78 
relevant phase 1 attachments to this protocol. Developed and processed by our Co- 
PI (Dr. Kraus) and his subcontractor company, Outcome Referrals, Inc. (ORI), the TO 
is administered routinely as a core element of the PsycBC care model. That is, 
PsycBC already has an executed business agreement with ORI to have their patients 
complete the TOP as part of their standard clinical routine. Thus, we can leverage 
the existing PsycBC infrastructure to support the present study with little to no extra 
burden on administrators, providers, and patients. Moreover, although patient data 
are part of this baseline phase, they are protected within the business agreement 
between ORI and PsycBC, and the agreement allows for these coded data to be used 
to establish therapists’ performance “report cards.” So, to reiterate, patient TOP 
data are collected as part of standard operating procedure for PsycBC. At this stage, 
we are not collecting these patient data as a research protocol; rather, these coded 
patient data points (i.e., clinical care data points) inform our match intervention (by 
establishing therapist performance report cards across at least 15 cases) that is at 
the heart of phase 2 (described below). In phase 1, we are only actively recruiting 
provider participants; thus, no patient protected health information (PHI) is 
transmitted to the research team. 
 
Importantly, at the time of this proposed IRB revision, most PsycBC clinicians who 
will choose to participate in the study will already have baseline data on the 
minimum 15 adult cases (through the patient’s actual termination point or 16 
weeks, whichever comes sooner) to establish their track record. In these cases, we 
simply need to enroll the therapist in the study (as discussed next). For therapists 
who wish to participate, but have yet to accumulate baseline performance data on 
the minimum 15 cases, we will track their performance (as per the TOP) on new, 
consecutive referrals until 15 total cases have been established for which the patient 
has either terminated or has been seen for at least 16 weeks. Few therapists will fall 
in this second category, and even if they do, they will generally only need a few 
cases to reach 15. Thus, we expect no issues completing the phase 1 performance 
baseline and finalizing the match algorithm for the phase 2 RCT by the established 
contractual milestone of 10/1/17. 
 
Our minimum target therapist sample is 44 PsycBC providers (all of whom will be 
over the age of 18 themselves, and treating patients within the age range of 18-65). 
Therapists will be psychologists, clinical counselors, and social workers. Recruitment 
will be coordinated among our UMass-employed project coordinator (PC), the 
PsycBC-employed PC, clinic staff members, and the Co-PIs. Specifically, the PsycBC 
team will verbally present information about the study (both phases 1 and 2) to their 
providers during staff meetings. Alternatively, this information can be presented 
through email. At this preliminary recruitment stage, this information will be used to 
heighten awareness about the study and to garner interest in participating. (The 
verbal script for staff meetings and the email text are included as phase 1 
attachments to this protocol.) The PsycBC PC will then provide the UMass PC (via 

 
79 
email) the names of providers who expressed interest in learning more about the 
study. 
 
The UMass PC will subsequently contact interested therapist participants via email 
or teleconference (whichever is more convenient for the provider) to provide more 
study details/procedures and to direct the provider to an online consent form and 
survey. Providers remaining interested will access the secure study website to 
provide formal consent and to complete the baseline survey to which they will be 
directed after consenting. Therapists will be told that the study is examining various 
referral processes that will not affect their delivery of treatment-as-usual. They will 
be informed that they will be blind to the specific nature of the referral 
manipulation in phase 2, but will be fully debriefed following the entire study and 
offered an opportunity to provide feedback on their experience. Therapists will also 
be informed of the assessments in which their study patients will engage in both 
phase 1 (which is standard practice) and phase 2 (though they will not have access 
to the phase 2 research data at any time). Therapists will also need to consent to 
completing the aforementioned baseline survey prior to phase 1, as well as a few 
study-specific measures for each patient during the phase 2 RCT (the baseline survey 
and the phase 2 attachments are described in the relevant sections below and are 
included as phase 1 and 2 attachments, respectively, to this protocol). Relevant to 
phase 1, therapists will be compensated with a $20 Amazon gift card for the one-
time completion of the online baseline survey, which will take no longer than 25 
minutes to complete. Non-consenting therapists will receive case assignments as per 
standard care protocol and will simply not be included in the study (though we will 
analyze consenting and non-consenting therapists on demographic differences to 
see if any systematic sample bias exists). 
 
Once therapists are enrolled in the study, the research team will access their 
naturalistically-collected TOP data to establish their performance across the 
minimum 15 cases to determine their personal strengths in treating patients across 
the risk-adjusted mental health problem domains measured by the TOP (recall that 
nothing changes in the therapist’s service operation during this phase and, in fact, 
most of these TOP data points will have already been processed through ORI for 
cases seen by the providers in the past). Specifically, to establish therapists’ 
performance track records, we will draw on each relevant patient’s coded TOP data 
from baseline, week 8, and their termination point or week 16, whichever comes 
sooner (to mimic the definition of treatment outcome in the RCT phase discussed 
below). To reiterate, the research team is not formally enrolling patients into phase 
1 of the study; rather, their coded data are simply processed by ORI, through its 
business agreement with PsycBC and its subcontractor role in the current project, to 
inform participating therapist report cards and the match algorithm).  
 
Note that enrolled therapists will have an already-established TOP ID. This will allow 
the research team to link therapists? baseline survey data to their RCT data (i.e., 

 
80 
responses to their own measures and their participating patients’ measures) without 
use of any identifying information. As per customary precautions described below, a 
key that links therapist names and contract information with their data code will be 
kept in a separate, secure file that only trained research personnel can access. 
 
Relevant phase 1 attachments to this protocol:  
 
(1) Therapist recruitment materials: verbal script; email 
(2) Therapist consent form and baseline phase 1 survey measures: 
 
Provider Characteristics Form (PCF). This measure was developed by the research 
team to assess therapist demographic information, clinical experience, degree type, 
percent time seeing various patient types/diagnoses, any specialty training they 
have received, and dimensional ratings of the influence of various theoretical 
orientations on their treatment approach. 
 
Therapist Perceived Strengths (TPS). This measure was developed by the research 
team to assess therapists’ beliefs about their effectiveness in treating the various 
TOP domains when uninformed of their data-driven TOP track record. This measure 
will allow us to examine how accurate therapists are in perceiving their own 
strengths and weaknesses. 
 
Phase 2: At this phase, the RCT will commence. The therapists will have already 
consented prior to phase 1 to be involved in the entire study, and they will know 
that patient data from their naturalistic baseline cases will have been used to create 
a personalized performance report card that will inform a prospective match with 
new patients they will treat in the trial. The therapists themselves will not see their 
report cards (as they will have been informed at the time of consent); rather, this 
information will be used by the research team with regard to the match 
manipulation.  
 
Phase 2 marks the beginning of patient recruitment into the RCT. The patient 
population will be adult men and women (age 18-70) in PsycBC’s referral stream 
(largely Cleveland clinic and primary care [PCP] practice). Recruitment to the study 
simply means a willingness to be randomized to condition based on TOP-derived 
presenting problem and to complete supplemental assessments (for monetary 
compensation, as per below) at baseline, at regular intervals during treatment, and 
at posttreatment. As this is an effectiveness design with a premium on ecological 
validity and scalability, virtually all patients in the PsycBC network will be eligible. It 
is most likely that the sample will be predominated by the following problem 
domains: depression, panic, substance abuse, and poor quality of life. The only 
study-related patient-level exclusion criterion will be patients who are not the 
primary, informed decision-maker for their care. Thus, patients will present with a 
multitude of presenting problems across a spectrum of Diagnostic and Statistical 

 
81 
Manual of Mental Disorders (5th ed.; DSM-548) diagnoses. Our minimum study 
target sample size is 264 patients (6 per therapist).  
 
We do not anticipate problems meeting our recruitment numbers in the project 
time frame, as PsycBC schedules approximately 950 new patients per month. 
Moreover, their care model already uses the TOP to screen patients for appropriate 
level of care, and, as a formal subcontract on the project, they are willing to use a 
patient-level-best-matched clinician list that is generated in real time (based on the 
predictive validity of our match algorithm). Including the randomization protocol 
into the treatment delivery model will not create any systemic barriers. 
 
Patients will flow into PsycBC via electronic or self-referrals. At initial contact, the 
PsycBC PC will ask patients for permission to be contacted by study personnel (i.e., 
the UMass PC) if they are interested in learning more about participation (this verbal 
script remains included as a phase 2 attachment to this protocol). If they are, they 
will be asked by the PsycBC PC to sign an authorization agreement (included in the 
phase 2 consent form) to allow their contact information (name, email address, and 
phone number) to be shared with the research team. The PsycBC PC’s role is 
restricted to this recruitment task and administration of authorization to release the 
limited PHI; thus, no PsycBC personnel will be engaged in human subjects’ research. 
 
The PsycBC PC will provide the UMass PC with a daily list of referrals who have 
provided signed authorization to be contacted about the study. The UMass PC will 
then contact eligible patients to schedule a baseline consent/assessment. If a patient 
agrees to engage in a consent/baseline assessment session, the PC will schedule a 
teleconference diagnostic interview via a secure platform with a trained graduate 
clinical psychology research assistant (RA). During this session, the RA will first 
review the study details/procedures and respond to any questions. Patients will be 
told that the study is examining various referral processes that will not affect their 
treatment; they will be kept unaware of the specific nature of the referral 
manipulation, but will be told that they will be fully debriefed following the study 
and offered an opportunity to provide feedback on their experience (via an exit 
interview). They will also be told that although their participation in the trial will 
largely mimic the same treatment that they would receive if they were not 
participating, they must consent to be randomized, complete extra study-specific 
measures (before, during, and after treatment), complete an audio-recorded 
diagnostic interview (before and after treatment), and accept assignment to a 
clinician who will deliver individual psychotherapy. Patients will also be asked to 
remain with the same therapist through at least 16 weeks of treatment; however, if 
they request a transfer earlier, this will be treated as a dropout point for the sake of 
the trial. 
 
If a patient consents to be enrolled, they will sign the consent form and complete a 
baseline survey of measures (i.e., the TOP-CR, TOP-CS, and TOP-CM, a brief measure 

 
82 
of global distress, a measure of existential isolation, and a measure of interpersonal 
problems, all described below in the listing of relevant phase 2 attachments to this 
protocol) through a secure online platform linked to their typical TOP 
administration. Next, the trained research assistant (RA) will administer (on the 
same individual teleconference) the M.I.N.I. 7.0.2 International Neuropsychiatric 
Interview (described below in the listing of relevant phase 2 attachments to this 
protocol). Following PsycBC’s standard intake process and this research-focused 
baseline consent/assessment session, patients will be randomized to condition and 
assigned to a provider based on the experimental parameters of that condition (i.e., 
scientific match vs. pragmatic match). For their involvement in the additional 
diagnostic assessments and the additional measures that they will complete during 
the active treatment phase, patients will be compensated with a $50 Amazon gift 
card (on a prorated schedule for any missed assessments). 
 
After the full baseline assessment, patients will be randomly assigned to condition 
(scientifically informed matched vs. pragmatic match) with a participating PsycBC 
provider. The UMass PC will generate the randomization sequences using an online 
random generator. Within condition, patients will be assigned sequentially to the 
therapists until they reach their study quota of 6 patients. Patients in the match 
condition will be assigned to therapists who have a demonstrated strength (derived 
from the baseline period) in treating, at a minimum, the patient’s highest self-
reported distress domain on the TOP-CS. Beyond the minimal match on the most 
elevated TOP-CS domain, our match algorithm will attempt to match patients to 
therapists on as many TOP-CS dimensions as possible, ultimately providing PsycBC 
with at least several well-matched choices for assignment within the match 
condition. In order to preserve this level of choice, there will be natural variability in 
the number of well-matched domains (some patients matched only on the minimum 
1 TOP-CS domain, others matched on 2 or more domains). The match variability 
across both conditions will allow us to measure degree of match dimensionally as a 
moderator variable of our main treatment effect. Therapists will also be unaware of 
their patient’s treatment condition (double blind), and they will treat both matched 
and non-matched patients (i.e., they will be crossed over the two conditions to 
minimize administrative disruptions). In the low probability event that there is no 
therapist meeting minimal match criteria for a patient in the match condition, that 
patient will be removed from the primary study analyses (though will, of course, still 
be offered treatment-as-usual at the clinic) and replaced with the next patient 
where a match does exist. As described in our power analysis below, we are 
oversampling in order to account for these “dropouts,” or removed data points. 
 
In addition to the baseline assessments already described, patients will be assessed 
via online surveys at regular intervals during treatment (the secure ORI platform will 
email hyperlinks to these surveys with reminders to complete them at the 
appropriate time intervals; the UMass PC can also follow-up with phone calls if 
needed). These during-treatment assessments will include the TOP-CS and measures 

 
83 
of existential isolation and interpersonal problems at every odd-numbered week 
after the start of treatment, as well as global distress, therapeutic alliance quality, 
perceived treatment credibility, and outcome expectation after every even-
numbered session (all measures of these constructs are described below in the 
listing of relevant phase 2 attachments to this protocol). During treatment, 
participating therapists will also be asked to complete their respective versions of 
the alliance and credibility/expectation measures (also at even-numbered weeks; 
the UMass PC will email hyperlinks to these online surveys with reminders to 
complete them at the appropriate time intervals; the PC will also follow-up with 
phone calls if needed). For completing these measures, therapists will be 
compensated $50 per patient (again in the form of Amazon gift cards). All data 
collection will be coordinated through ORI, for which patients and therapists are 
assigned unique codes. Through their business agreement, ORI has direct access to 
PBC medical records; thus, it can push the relevant measures and track 
patient/therapist progress throughout the study. 
 
As reminder, in both conditions, the providers will deliver treatment naturalistically 
(i.e., with no manipulation or influence from the research team). For the sake of the 
RCT, “treatment outcome” will be considered the point at which treatment 
terminates, or 16 weeks, whichever comes sooner. After the 16th week, or the 
termination session if it comes sooner, patients will complete posttreatment 
measures: the TOP-CS and TOP-CM, a measure of treatment satisfaction, a brief 
measure of global distress, a measure of existential isolation, and a measure of 
interpersonal problems (all described below in the listing of relevant phase 2 
attachments to this protocol). Therapists will also document the nature of 
termination (measure described below in the listing of relevant phase 2 attachments 
to this protocol). Also at posttreatment, as defined by the trial, patients will undergo 
a repeat diagnostic telephone assessment (i.e., an RA-administered M.I.N.I., as 
described above). 
 
We will also conduct a follow-up outcome assessment at 1 year after the patient’s 
own termination on a randomly-selected subsample of 40 patients. Patients can 
easily be tracked in coordination with ORI and PsycBC; further, patients will have 
provided consent for this follow-up contact (should they be randomly chosen for it). 
At this assessment point, patients will again complete online the TOP-CS and TOP-
CM, the brief measure of global distress, the measure of existential isolation, and 
the measure of interpersonal problems. 
 
Note that all self-report measures (for both patients and therapists) at all time-
points will be completed on Wi-Fi-connected tablets, or on home computers, 
through ORI’s secure web-based platform. The TOP has its own dedicated website 
and HIPAA-compliant, secure server, and all other study-specific measures will be 
integrated into the TOP administration process. 
 

 
84 
We predict that the scientific match group will outperform the no match group to a 
clinically significant degree on TOP outcomes, global symptomatology, and 
interpersonal problems. We also expect that the match group will be more effective 
in promoting alliance quality and fostering more positive patient perceptions of 
treatment credibility and outcome expectation, all of which are established 
correlates (and candidate mechanisms) of positive treatment outcomes. Finally, we 
expect there to be less patient dropout in the match condition, and higher patient 
treatment satisfaction. Secondarily, we will examine 4 potential moderators of the 
expected between-group treatment effects on the primary TOP outcomes: (a) 
patient race (as it may be that the match algorithm is particularly potent, and an 
important responsiveness tool, for historically understudied or underrepresented 
patients), (b) degree of match of therapist strengths to patient problems (rated 
dimensionally as a ratio given that therapists can be matched on more than just the 
minimum 1 domain, and the elimination of harmful matches for any distressed 
domain reported by the patient), (c) patient distress severity, and (d) complexity of 
patient presenting problem. Thus, we will test if matching is only, or particularly, 
effective under the conditions of a central patient characteristic, a multiple domain 
match, and/or for patients with the most severe or complex pathology. As noted, we 
will also assess therapists’ self-perceived strengths on the TOP domains. We expect 
to replicate previous literature showing that therapists are poor judges of their own 
efficacy, tending to underestimate negative effects and overestimate positive effects 
with their patients (Lambert, 2011), which would further underscore the importance 
of a data-driven match process. 
 
Finally, for a subsample of stakeholders, we will conduct post-trial exit interviews 
(Ns = 5 patients, 5 therapists) to gather invaluable input on how to be responsive to 
the study findings in terms of dissemination, implementation, and policymaking, 
including the potential importance of integrating diagnosis, provider age, race, or 
gender into subsequent matching approaches. We will recruit stakeholders in order 
of completion until we reach our target Ns (therapists can only be involved once 
they have treated all 6 of their study patients). There are no other 
inclusion/exclusion criteria for the exit interviews; we will simply stop asking if 
participants are interested once we have reached our target Ns. This is consistent 
with the study consent forms, which clearly state that interested participants may 
be selected to engage in the interview. 
 
Fully reflecting stakeholder engagement, and to eliminate any biases or power 
dynamics introduced by the PIs or their research staff, Advisory Board members 
(with appropriate credentialing for working with human subjects) will conduct the 
individual interviews. The PIs (Constantino & Boswell) will train 3 Advisory Board 
members on qualitative interviewing, and each will administer 1-2 pilot interviews as 
part of the training, plus 5 study interviews. The interviews will be conducted and 
audiorecorded via a secure webconferencing service and will last approximately 45-
60 minutes. Participants will be compensated with a $100 Amazon gift card for their 

 
85 
time. RAs will transcribe the interviews, removing any identifying patient 
information. These RAs will also conduct a qualitative analysis of these text-based 
data. 
 
Relevant phase 2 attachments to this protocol: 
 
(1) Patient phase 2 recruitment verbal script 
(2) Patient phase 2 consent form and phase 2 baseline measures packet: 
 
TOP-Consumer Registration Form (TOP-CR; Kraus et al., 2005). The TOP-CR will be 
used routinely during the phase 1 baseline (and the phase 2 RCT) to assess patient 
demographics. On this form, patients indicate their age, gender, ethnicity, marital 
status, income level, employment status, religious identification, education level, 
general health status, and medical and mental health treatment history. 
 
TOP-Clinical Scales and Case Mix (TOP-CS & TOP-CM; Kraus et al., 2005). This is the 
primary measure in our study; it will be used to establish the therapist report cards 
during the baseline phase to inform the match manipulation in phase 2. It also tracks 
patient outcomes. The TOP-CS consists of 58 items assessing 12 symptom and 
functional (including strengths) domains (risk-adjusted for case mix variables 
assessed via 37 items on the companion TOP-CM, such as divorce, job loss, 
comorbidity): work functioning, sexual functioning, social conflict, depression, panic 
(somatic anxiety), psychosis, suicidal ideation, violence, mania, sleep, substance 
abuse, and quality of life. Global symptom severity is assessed by summing all items 
or by averaging the z-scores (i.e., standard deviation units relative to the general 
population mean) across each of the 12 clinical scales. Domain-specific symptom 
severity is quantified as the individual z-scores for each clinical scale using general 
population means and standard deviations for the conversion. The TOP-CS has been 
shown to have excellent factorial structure, as well as good test-retest reliability 
across all scales. It is sensitive to change while possessing limited floor and ceiling 
effects (Kraus et al., 2005). The TOP also has demonstrated good convergent validity 
with scales like the Beck Depression Inventory (Beck, Steer, & Brown, 1996) and the 
Brief Symptom Inventory (Derogatis, 1975). 
 
Symptom Checklist-10 (SCL-10; Rosen, Drescher, Moos, & Gusman, 1999). To 
evaluate outcome with an index separate from the TOP (to test convergence and 
enhance the validity of any between condition effects), we will also assess global 
distress with the SCL-10, a 10-item, well-validated and widely used self-report 
inventory that assesses psychological wellbeing. 
 
Existential Isolation Scale (EIS; Pinel et al., 2014). To assess this isolation subtype, 
participants will complete the EIS, a six-item scale that requires participants to rate 
the extent to which they agree with items such as “I often have the same reactions 
to things as other people around me do” (reverse-coded) and “Other people usually 

 
86 
do not understand my experiences” and “People often have the same ‘take’ or 
perspective on things that I do” (reverse-coded). Participants respond using a 7-
point scale. The EIS has high internal consistency, and has been validated extensively 
(Pinel et al., 2014). 
 
Inventory of Interpersonal Problems-32 (IIP-32; Horowitz, Alden, Wiggins, & Pincus, 
2000). To assess interpersonal problems, participants will complete the 32-item 
circumplex version of the IIP. This widely used instrument reflects interpersonal 
inhibitions and excesses, with each item rated on a 5-point scale. Higher total scores 
indicate more interpersonal problems. The IIP-32 also has 8 subscales (Domineering, 
Vindictive, Intrusive, Cold, Socially Inhibited, Nonassertive, Overly Accommodating, 
and Self-Sacrificing) that comprise a circumplex of problematic interpersonal 
behavior around the main interpersonal dimensions of affiliation and control. Like 
the original measure (Horowitz, Rosenberg, Baer, Ureno, & Villansenor, 1988), the 
IIP-32 has evidenced good psychometric properties. 
 
(3) RA administered diagnostic assessment (baseline and posttreatment): 
 
M.I.N.I. 7.0.2 International Neuropsychiatric Interview (M.I.N.I.; Sheehan et al., 
2016). The M.I.N.I. is a brief, structured diagnostic interview for DSM-5 and 
International Classification of Diseases (ICD; World Health Organization, 2008) 
psychiatric disorder classification. With its administration time of approximately 15 
minutes, the M.I.N.I. is the psychiatric interview of choice in clinical trials and 
epidemiological studies. Despite its brevity, its psychometric properties compare 
favorably to longer instruments like the Structured Clinical Interview for DSM (SCID; 
First, Spitzer, Gibbon, & Williams, 1996). As part of the diagnostic evaluation, the 
RAs will complete the Clinical Global Impression (CGI), a widely used observer-rated 
scale that includes a 0-7 judgment of illness severity for which higher scores indicate 
more extreme illness. 
 
(4) Patient phase 2 during-treatment measures: 
 
TOP-CS, SCL-10, EIS, IIP-32. All described previously. 
 
Working Alliance Inventory-Short Form, patient version (WAI-SF-P; Tracey, 
& Kokotovic, 1989). The WAI is the most widely used alliance measure, assessing 
patient-therapist agreement on the goals and tasks of treatment, and the quality of 
their relational bond. This 12-item short form, assessing these dimensions from the 
patient’s perspective, has demonstrated sound psychometric properties. 
 
Credibility/Expectancy Scale, patient version (CEQ; Devilly & Borkovec, 2000). The 
CEQ is the most widely used and psychometrically sound measure of the patient’s 
perceived logicalness of a given treatment and expectation for the personal efficacy 
of that treatment. 

 
87 
 
(5) Therapist phase 2 during-treatment measures: 
 
Working Alliance Inventory-Short Form, therapist version (WAI-SF-T; Tracey & 
Kokotovic, 1989). This is the parallel version of the WAI-SF described above, though 
now as rated from the therapist’s perspective. 
 
Credibility/Expectancy Scale, therapist version (CEQ; Devilly & Borkovec, 2000). This 
is the parallel version of the CEQ described above, though now as rated from the 
therapist’s perspective (i.e., the therapist’s sense of how logical the patient sees the 
treatment and how optimistic the patient is about receiving benefit from it). 
 
(6) Patient phase 2 posttreatment measures: 
 
TOP-CS, TOP-CM, SCL-10, EIS, IIP-32. All described previously. 
 
TOP-Satisfaction with the Treatment Process (TOP-STP; Kraus et al., 2005). This 32-
item measure assesses patient’s satisfaction with their provider, the treatment they 
received, and the treatment milieu (e.g., staff, other patients, etc.). 
 
(7) Therapist phase 2 posttreatment measure: 
 
Nature of Termination Form (NTF). This measure was developed by the research 
team to assess the nature of patients’ termination from the provider’s open-ended 
perspective, as well as through a choice format of unilateral/patient-generated, 
unilateral/therapist-generated, or mutual. Therapists can also describe in an open-
ended format any unusual or noteworthy circumstances that may have led to the 
termination of therapy with this client (e.g., transfer of client to another therapist). 
 
(8) Patient phase 2 subsample follow-up measures: 
 
TOP-CS, TOP-CM, SCL-10, EIS, IIP-32. All described previously. 
 
(9) Stakeholder exit interview protocols (patient and therapist versions) 
 
b. State if audio or video taping will occur. Describe what will become of the tapes 
after use, e.g., shown at scientific meetings, erased. Describe the final disposition 
of the tapes. 
For the baseline and posttreatment patient assessments during phase 2, RAs will 
administer via teleconference the semi-structured diagnostic interview (M.I.N.I.), 
which will be audiorecorded. This will allow a different RA to review the recording 
and to make independent diagnostic and symptom severity determinations. With 
these two sets of ratings, we can then calculate interrater reliability on baseline and 
posttreatment diagnosis. 

 
88 
 
Audio recordings from the baseline diagnostic assessments will be digitally stored 
through the secure web-conferencing service. All data will be encrypted and 
password protected. Only the necessary research team members will know the login 
and password information and have the capacity to access the recordings. When it is 
time to analyze the recordings for reliability coding, designated, trained RAs will also 
have access to the recordings. The RAs, of course, will have completed the 
mandatory ethics training in human subjects’ research, data management, and 
HIPAA compliance. These RAs will be independent evaluators who will not have 
access to other therapist or patient data. The recordings themselves will not be 
labeled with any identifiable information. The PI will routinely monitor the collection 
and analysis of recorded data. 
 
After the recordings have been assessed for diagnostic reliability, the files will be 
securely deleted by the sponsored project contract term date of 6/16/20. No audio 
data or identifiable text data stemming from the recordings will be presented at 
meetings or in published articles. Only the reliability coefficients will be 
disseminated with the results of the full trial. 
 
c. State if deception will be used. If so, provide a rationale and describe debriefing 
procedures. Submit a debriefing script in Section #11 (Attachments). 
Although the protocol does not involve deception, it does involve incomplete 
disclosure in Phase 2 given that participants are not given all of the information 
about the study until debriefing. Thus, in the debriefing form, we now provide 
participants the opportunity to withdraw their data upon learning the full scope of 
the research. 
 
3. Background 
 
a. Describe past findings leading to the formulation of the study. 
Research has consistently identified significant variability in skill and outcomes 
between therapists (Baldwin & Imel, 2013; Boswell et al., 2013; Westra, 
Constantino, Arkowitz, & Dozois, 2011), even when therapists utilize an empirically 
supported treatment (EST). In fact, differences between treatment providers 
account for a greater portion of treatment outcome variance than the specific 
interventions delivered in controlled trials (Krause, Lutz, & Saunders, 2007; 
Wampold & Imel, 2015). Thus, improvements in MHC can occur by identifying 
effective providers in addition to promoting ESTs (Kraus et al., 2007). 
 
In the largest study to date on this topic, our team investigated therapists’ 
naturalistic treatment outcomes over many different problem domains (e.g., 
depression, anxiety, substance use, mania, sleep) in a sample of 6,960 patients and 
nearly 700 providers (Kraus, Castonguay, Boswell, Nordberg, & Hayes, 2011). The 
majority of therapists demonstrated a differential pattern of effectiveness 

 
89 
depending on the problem domain, and therapist domain-specific effectiveness 
correlated poorly across domains suggesting that therapist competencies may be 
domain-specific, rather than reflecting a core attribute or general underlying 
therapeutic skill. Importantly, although some therapists demonstrated effectiveness 
over multiple problem domains, no therapists demonstrated reliable effectiveness 
across all domains. Further, a small, but notable 4% of the therapists did not 
demonstrate effective outcomes on any domain. These data suggest that in any 
population of therapists (payer network, hospital, or community mental health 
system), there is an opportunity for behavioral health to do what medicine did 
decades ago – encourage provider specialization. Virtually every clinician has an area 
where they are above average (82-96%; Kraus et al., 2011, 2016), and our research 
suggests that if they specialize to their unique skills, population-level outcomes (i.e., 
symptom reduction, behavior change, increased functionality) will improve 
dramatically. This would reflect a major, and likely highly impactful shift to current 
MHC systems. 
 
However, patients and referrers are typically unaware of the unique track record 
(“report cards”) of local-area providers, which represents a critical gap in knowledge 
transfer within the MHC system. Without systematically collecting and disseminating 
performance report cards, stakeholders (e.g., patients, therapists, administrators 
responsible for case assignment, primary care physicians) lack vital information on 
which to base MHC choices and referral decisions, and that can inform personalized 
treatment (Boswell, Constantino, Kraus, Bugatti, & Oswald, 2015). Conversely, there 
is potentially immense advantage to matching patients to providers based on 
scientific outcome data (Constantino, Boswell, Bernecker, & Castonguay, 2013). 
 
Consistent with this notion, the Institute of Medicine (IOM) has 
made recommendations to: (a) customize care based on the patient’s needs, (b) 
share knowledge, (c) engage in data-driven decision-making, (d) promote 
transparency (including information on performance and patient satisfaction; Kohn, 
Saxena, Levav, & Saraceno, 2004), and (e) use valid and reliable assessment 
instruments to assess progress and to aid decision-making. The IOM has also 
recommended that MHC patients be provided with information on the quality of 
practitioner care (e.g., provider report cards) and use this information when making 
treatment decisions. Importantly, we have survey data that point to MHC patients, 
therapists, and administrators endorsing such applied knowledge transfer as a high 
priority (Boswell et al., 2015). Provider track record report cards are meaningful data 
to the MHC patient population, as are the mental health benefits that could stem 
from being well matched to provider. 
 
We have developed over the past 20 years an innovative, technology-based 
mechanism/intervention to deliver report cards and drive this match concept within 
a patient-centered MHC model (Kraus et al., 2011). Our longitudinal data suggest 
that our match algorithm, based on our multidimensional outcome tool (the TOP) is 

 
90 
efficacious for MHC outcomes. In addition to our study highlighted above (Kraus et 
al., 2011), a more recent prospective study of 59 therapists and 3,540 patients 
resulted in a between-treatment controlled Cohen’s d effect size of .80 (Kraus et 
al., 2016). Each therapist’s first 30 patients were used to classify a therapist’s skills in 
the 12 domains of symptoms and functioning as either statistically above average, 
average, or below average. The best matching algorithm functioned as follows: for 
each new, successive patient, he or she was classified as well-matched if the risk of 
harm was eliminated (i.e., the therapist was not below average when treating any 
elevated domain) and the therapist was above average in treating the patient’s 
three most out-of-the-norm domains (e.g., depression, suicidality, and panic). Poorly 
matched patients had below average outcomes, with small effect sizes (d = .30) 
Well-matched patients, by contrast, achieved very large pre- vs. posttreatment 
effect sizes of d = 1.19. These data lend strong support that the proposed 
comparative effective research (CER) will yield similar results (i.e., increased efficacy 
and reduced harm) in realigning the skills of a large population of therapists in one 
of the forerunner Accountable Care Organizations (our partner Atrius) when 
matching empirically derived therapist skills with patient need. The 
technology/intervention is well established, it has demonstrated efficacy, and awaits 
investigation in a well-powered RCT. 
 
 
4. Subject Population 
 
a. State how many subjects you propose to use and state the rationale for the 
proposed number. 
For the primary 3-level hierarchical model assessing treatment condition effects at 
the patient level on linear change rates within patients, we used Raudenbush and 
Liu’s (2001) formula as incorporated in the Optimal Design program to determine 
the minimum numbers of therapists and patients needed to detect a moderate 
effect of condition (standardized difference between change rates = .50). With a 
minimum of 6 measurements spaced over the maximum 16 treatment weeks and 
assuming 5 patients per therapist, an intra-class correlation of .15, and an alpha of 
.05, we will need a total of 44 therapists and 220 patients to achieve a power of .80 
to detect moderate condition effects on linear change rates. Factoring a 20% 
dropout rate at the patient level, running our experiment on 264 patients (6 per 
therapist) should provide sufficient statistical power to detect group differences on 
our primary outcome variables.  
 
To summarize, based on this power analysis, we will for phase 1 access a naturalistic 
baseline assessment of a minimum of 44 consenting therapists’ performance across 
a minimum of 15 cases to determine their strengths in treating the risk-adjusted 
domains measured by the TOP. We will then recruit a minimum of 264 patients for 
the phase 2 trial, assigning patients to the same 44 therapists who participated in 
phase 1 (they will see 6 cases each during the trial). 

 
91 
 
b. Describe the subject population, including the age range, gender, ethnic 
background, and type of subjects (e.g. students, professors, subjects with learning 
disabilities, mental health disorders, etc.). Please incorporate specific 
inclusion/exclusion criteria (e.g. physical and psychological health, demographic 
information, or other unique characteristics). 
Therapist participants: As noted, our target sample is 44 therapist participants (age 
range = 30-65 years) who will be social workers, psychologists, and licensed clinical 
counselors. Reflecting PsycBC’s therapist pool demographics, we anticipate that our 
provider sample will break down as follows: approximately 70% will be female; 88% 
will be white/non-Hispanic, 3% Black, 2% Hispanic, 2% “Other/mixed,” and 5% Asian. 
Based on these projections and our power analysis, our targeted/planned therapist 
enrollment is indicated in an attached Targeted/Planned Enrollment Table 
(Therapists). 
 
Patient participants: Patient participants will be 264 adult men and women (age 18-
70) in PsycBC’s referral stream (largely Cleveland clinic and primary care [PCP] 
practice). Recruitment to the study simply means a willingness to be randomized to 
condition and to complete supplemental assessments (for monetary compensation) 
at baseline, at regular intervals during treatment, and at posttreatment. As this is an 
effectiveness design with a premium on ecological validity and scalability, virtually 
all patients in the PsycBC network will be eligible. It is most likely that the sample 
will be predominated by the following problem domains: depression, panic, 
substance abuse, and poor quality of life. The only study-related, patient-level 
exclusion criterion will be patients who are not the primary, informed decision-
maker for their care. Thus, patients will present with a multitude of presenting 
problems across a spectrum of Diagnostic and Statistical Manual of Mental 
Disorders (5th ed.; DSM-5) diagnoses. The composition of our sample will roughly 
match the average utilization data for age, gender, and race/ethnicity at PsycBC. 
Based on these projections and our power analysis, our targeted/planned patient 
enrollment is attached in an Estimated Final Racial/Ethnic and Gender Enrollment 
Table (Patients). 
 
c. State the number and rationale for involvement of potentially vulnerable subjects 
to be entered into the study, including minors, pregnant women, prisoners, 
economically and educationally disadvantaged, decisionally challenged, and 
homeless people. 
We are not specifically targeting these specific vulnerable populations, and our 
research design and/or the PsycBC care system will specifically exclude minors and 
prisoners. However, given the effectiveness design focused on maximizing ecological 
validity, some of our patients are sure to have economic and educational 
vulnerabilities, which are risk factors for mental health issues. Some women might 
also be pregnant. 
 

 
92 
d. If women, minorities, or minors are not included, a clear compelling rationale must 
be provided. 
Minors will be excluded because they are typically not solely responsible for their 
own treatment decisions, and the outcome measure used in this study, and on 
which the match manipulation is based, focuses on adults. 
 
e. State the number, if any, of subjects who are laboratory personnel, employees, 
and/or students. They should be presented with the same written informed 
consent. If compensation is allowed, they should also receive it. 
N/A 
 
f. State the number, if any, of subjects who are involved in research conducted 
abroad and describe any unique cultural, economic or political conditions. 
N/A 
 
g. Describe your procedures for recruiting subjects, including how potential subjects 
will be identified for recruitment. Attach advertisements, flyers, etc. in Section #11 
(Attachments). Note: Potential subjects may not be contacted before IRB approval. 
Therapist participants:  
 
Recruitment will be coordinated among our UMass-employed PC, the PsycBC-
employed PC, clinic staff members, and the Co-PIs, and will involve presenting 
information about the study (both phases 1 and 2) to providers through verbal script 
at staff meetings or by email. At this preliminary recruitment stage, this information 
will be used to heighten awareness about the study and to garner interest in 
participating. The PsycBC PC will then provide the UMass PC (via email) the names of 
providers who expressed interest in learning more about the study. The UMass PC 
will subsequently contact interested therapist participants via email or 
teleconference (whichever is more convenient for the provider) to provide more 
study details/procedures and to direct the provider to an online consent form and 
survey. Providers remaining interested will access the secure study website to 
provide formal consent and to complete the baseline survey to which they will be 
directed after consenting. Therapists will be told that the study is examining various 
referral processes that will not affect their delivery of treatment-as-usual. They will 
be informed that they will be blind to the specific nature of the referral 
manipulation in phase 2, but will be fully debriefed following the entire study and 
offered an opportunity to provide feedback on their experience. Therapists will also 
be informed of the assessments in which their study patients will engage in both 
phase 1 (which is standard practice) and phase 2 (though they will not have access 
to the phase 2 research data at any time). Therapists will also need to consent to 
completing the aforementioned baseline survey prior to phase 1, as well as a few 
study-specific measures for each patient during the phase 2 RCT. 
 
Patient participants: 

 
93 
 
Phase 2 marks the beginning of patient recruitment into the RCT. Recruitment to the 
study simply means a willingness to be randomized to condition and to complete 
supplemental assessments (for monetary compensation). Patients will flow into 
PsycBC via electronic or self-referrals. At initial contact, the PsycBC PC will ask 
patients for permission to be contacted by study personnel (i.e., the UMass PC) if 
they are interested in learning more about participation. If they are, they will be 
asked by the PsycBC PC to sign an authorization agreement (included in the consent 
form) to allow their contact information to be shared with the research team. The 
PsycBC PC’s role is restricted to this recruitment task and administration of 
authorization to release the limited PHI; thus, no PsycBC personnel will be engaged 
in human subjects’ research.  
 
The PsycBC PC will provide the UMass PC with a daily list of referrals who have 
provided signed authorization to be contacted about the study. The UMass PC will 
then contact eligible patients to schedule a baseline consent/assessment. If a patient 
agrees to engage in a consent/baseline assessment session, the PC will schedule a 
teleconference diagnostic interview via a secure platform with a trained graduate 
clinical psychology research assistant (RA). During this session, the RA will first 
review the study details/procedures and respond to any questions. Patients will be 
told that the study is examining various referral processes that will not affect their 
treatment; they will be kept unaware of the specific nature of the referral 
manipulation, but will be told that they will be fully debriefed following the study 
and offered an opportunity to provide feedback on their experience (via an exit 
interview). They will also be told that although their participation in the trial will 
largely mimic the same treatment that they would receive if they were not 
participating, they must consent to be randomized, complete extra study-specific 
measures (before, during, and after treatment), complete an audio-recorded 
diagnostic interview (before and after treatment), and accept assignment to a 
clinician who will deliver individual psychotherapy. Patients will also be asked to 
remain with the same therapist through at least 16 weeks of treatment; however, if 
they request a transfer earlier, this will be treated as a dropout point for the sake of 
the trial. If the patient consents to be enrolled, they will sign the consent form and 
complete a baseline survey of measures (i.e., the TOP-CR, TOP-CS, and TOP-CM, a 
brief measure of global distress, a measure of existential isolation, and a measure of 
interpersonal problems) through a secure online platform linked to their typical TOP 
administration. Next, the trained RA will administer (on the same individual 
teleconference) the M.I.N.I. Following PsycBC’s standard intake process and this 
research-focused baseline consent/assessment session, patients will be randomized 
to condition and assigned to a provider based on the experimental parameters of 
that condition (i.e., scientific match vs. pragmatic match). 
 

 
94 
h. Compensation. Explain the amount and type of compensation (payment, 
experimental credit, gift card, etc.), if any, that will be given for participation in the 
study. Include a schedule for compensation and provisions for prorating. 
Therapist participants:  
 
Therapists will complete, in no longer than 25 minutes, a few study-specific 
measures as part of a phase 1 baseline survey for which they will be compensated 
$20 in total (in the form of an Amazon gift card).  
 
During Phase 2, therapists will also complete a few study-specific measures 
throughout treatment with each of the 6 participating patients treated during the 
phase 2 RCT; they will be compensated $50 per patient for this additional, but 
minimal, time burden. The compensation will again be in the form of an Amazon gift 
card. 
 
If therapists complete their measurement schedule through all possible contact 
points for a given participating patient (i.e., baseline + 16 treatment weeks = 17 
weeks), or complete their measurement schedule through a planned termination for 
a participating patient that occurs prior to week 16 of treatment, they will receive 
full compensation (i.e., a $50 gift card for that patient).  
 
However, if a therapist withdraws from the study, they will have the option to be 
compensated on a prorated basis for the measures that they have already 
completed regarding each of their participating patients. This proration works out to 
approximately $3 per week for a participating patient, which will be deducted for 
the number of weeks “missing” from therapists’ assessment schedule (i.e., based on 
the point at which the therapist withdrew from the study). For example, if a 
therapist completes the measurement schedule for a given patient through week 8 
(9 weeks, including baseline) and then withdraws from the study, they will have 
“missed” 8 weeks of data collection for that participating patient. Their 
compensation for this participating patient will be adjusted as follows: $50 - $24 ($3 
x 8 weeks) = $26. This adjustment will be completed for any and all relevant 
participating patients. To summarize, therapists who withdraw from the study will 
have the option either to (a) receive their relevant prorated compensation, or (b) to 
forgo prorated compensation in order to no longer be contacted by the research 
team.  
 
If the therapist participates in an exit interview, he or she will receive full 
compensation in the form of an additional $100 Amazon gift card. 
 
Patient participants:  
 
Patients in Phase 2 will undergo a semi-structured diagnostic interview at both 
baseline and posttreatment, as well as complete several study specific measures 

 
95 
throughout treatment (and, if randomly selected, at a follow up); they will be 
compensated $50 total for these non-routine aspects of their care. The 
compensation will be in the form of an Amazon gift card. If patients complete their 
measurement schedule through all possible contact points (i.e., baseline + 16 
treatment weeks = 17 weeks), or complete their measurement schedule through a 
planned termination that occurs prior to week 16 of treatment, they will receive full 
compensation. However, if they drop out of treatment prior to week 16, and their 
end point was not a planned termination that can be considered posttreatment for 
the purpose of the study, compensation will occur on prorated schedule. This works 
out to approximately $3 per week, which will be deducted for the number of weeks 
“missing” from the schedule. For example, if a patient completes the measurement 
schedule through week 8 (9 weeks, including baseline), and they did not engage in a 
planned termination, they will have “missed” 8 weeks of data collection. Their 
compensation will be adjusted as follows: $50 - $24 ($3 x 8 weeks) = $26. 
 
Patients who withdraw from the study (which is distinct from simply dropping out of 
treatment) will be given the option to (a) receive prorated compensation for the 
completion of measures up until the point of withdrawal (following the proration 
schedule outline above), or (b) to forgo prorated compensation in order to no longer 
be contacted by the research team. 
 
If the patient participates in an exit interview, he or she will receive full 
compensation in the form of an additional $100 Amazon gift card. 
 
Finally, note that in the event that a participant (either a patient or therapist) 
withdraws from the study during phase 2, the other dyad member (either the 
patients seen by a withdrawing therapist or the therapist treating a withdrawing 
patient) will not be penalized; that is, as long as they have already consented to the 
study, they will receive the full amount of reimbursement (i.e., a $50 gift card) 
regardless of the point at which their patient/therapist withdraws. However, note 
that consistent with the wishes of the participant, we will, of course, stop collecting 
data at the point of withdrawal (i.e., if therapists withdraw, we will stop collecting 
data from their patients who will be compensated fully; if patients withdraw, we will 
stop collecting data from their therapist regarding that patient and the therapist will 
be compensated fully for that patient). 
 
i. Please state: A: The total expected duration of the study, including the time 
expected for data analysis (e.g., This study is expected to last 1 year) AND B: How 
much time each subject is expected to be involved in the study (e.g., The 
involvement of each subject will be 1-session for a total of 90 minutes). 
A) The project is funded in the form a cost-reimbursement contract for which a 
specific milestone schedule exists. The contract start date is 9/15/16 and the 
contract term date is 6/15/20. All analyses will be completed by the term date. 
Details are available in the attached updated milestone schedule. 

 
96 
 
B) Therapist subjects will be involved for 2 months in phase 1, as well as through the 
phase 2 trial (approximately 2 years, though with variability depending on when 
they have been assigned and have treated their 6 study cases). Patients in phase 2 
only will be involved in the study protocol through their actual termination point or 
16 weeks, whichever comes sooner (we will also conduct a follow-up outcome 
assessment at 1 year after termination on a randomly-selected subsample of 40 
patients). 
 
5. Risks 
 
HHS Regulations define a subject at risk as follows: “...any individual who may be exposed 
to the possibility of injury, including physical, psychological, or social injury, as a 
consequence of participation as a subject in any research...” This also includes risks to 
subject confidentiality and any discomforts, hazards, or inconveniences. 
 
For the categories below, include a description of risks. 
 
a. Describe the risks related to: 
Physical well-being 
None anticipated. 
 
Psychological well-being 
Regardless of condition, PsycBC will employ its usual triage assessments, therapists 
will employ their usual treatments, and patients will be receiving their usual care. 
Consequently, there are no risks from our research protocol over and above what 
would normally be expected in routine assessment and psychotherapy, and PsycBC 
has its usual clinical and safety protocols in place (and the clinical personnel to 
execute them). 
 
In treatment, some individuals may experience emotional upset during sessions. 
Additionally, some participants may experience disappointment with their rate of 
progress or setbacks. The risk associated with such reactions will be addressed 
clinically by the therapists who are treating these issues and who have peer and 
administrative support. To reiterate, these treatment risks would occur in the course 
of treatment-as-usual. These are not additional risks stemming from the research 
protocol. Further, the TOP outcome monitoring system, which is at the center of our 
research project, is already being used by PsycBC providers without incident. 
 
As is typical in psychological research, some of the assessment questions from the 
research measures may be experienced as intrusive and/or may cause anxiety. The 
risk from such increased anxiety, however, is mitigated by the use of skilled and 
extensively trained assessors who are aware that such reactions may be related to a 
person’s presenting problems, or simply a function of the intimate and emotionally 

 
97 
intense nature of psychological services. In addition, the PIs, PCs, and/or PsycBC 
staff and administrators will be available to meet with any participant who may be 
unduly disturbed due to the few research tasks. Because the pre- and posttreatment 
diagnostic interviews will be conducted via telephone, the graduate RA (being 
trained as a clinician and supervised by their site PI, Dr. Constantino or Boswell, both 
of whom are licensed clinical psychologists and mental health care providers) will 
have the patient’s contact information (phone number and email address) on hand. 
If the patient reveals clinically elevated suicidality or homicidality, the RA will 
contact 9-1-1 and report the patient’s contact information and location address 
(which they will request verbally, if necessary) for emergency response. The RA, if 
applicable, will also execute any duty to warn to the best of their ability (in addition 
to contacting the local authorities). 
 
Economic well-being 
Given that therapist performance data are being collected, it is reasonable to be 
concerned about possible employment implications were an employer (i.e., clinic 
administrator) to attempt to interpret study information incompletely (i.e., infer lack 
of therapist effectiveness to the point of questioning employability). This risk, 
however, is extremely minimal for the following reasons: 
 
(1) As a condition of being involved in the study, clinic administrators will be 
required to agree that therapists’ participation or non-participation in this research 
will in no way affect their standing/employment at their community mental health 
clinic. 
 
(2) The research team will not reveal therapist performance data to clinic 
administrators or staff members; that is, the study could be considered “triple-
blind.” Neither patients nor therapists will know when they are in an experimentally-
matched vs. typically-matched dyad, and administrators/staff members will not have 
access to the therapists’ report cards. 
 
(3) However, administrators and staff members are required to be in the know 
about well-matched therapist “short-lists,” as this is essential to the research design; 
that is, when patients are randomized to a well-matched therapist, those potential 
therapists need to be identifiable. It is possible that administrators or staff members 
might misinterpret these data to suggest that a given therapist is ineffective (if he or 
she is never or rarely showing up on a shortlist). However, we will guard against this 
misinterpretation by educating administrators and staff members that the shortlist 
only represents, in a small cross-section of time, therapists that have been shown to 
be effective on at least 1 of 14 domains, which represents a given patient’s most 
severe problem at that time (the match criterion). We will stress that this does not 
mean that a therapist is globally ineffective. It may just be that patients randomly 
assigned to the match group are tending not to have the types of problems for 

 
98 
which a given therapist is relatively effective. That therapist, though, could be highly 
effective at treating one or even many other domains. 
 
(4) Finally, administrators and staff members will not be told which therapists are or 
are not participating in the study. Thus, lack of being on a shortlist, for all that they 
will know (unless a therapist openly reveals that he or she is participating in the 
study), could simply connote a choice to not participate in the project. 
 
Social well-being 
None anticipated. 
 
Breach of confidentiality (including audio/video taping) 
A breach of confidentiality represents a risk, but every step will be taken to minimize 
this risk. PsycBC and ORI routinely handle PHI and are in compliance with HIPAA 
regulations. Any “hard” materials (e.g., diagnostic assessment summaries) that are 
collected for research purposes only will be stored in a locked cabinet in the PI’s 
Psychotherapy Research Lab. There will be no hard copy data collected at the 
PsycBC clinic sites. Most of the data collected in this study (including consent) will be 
through a secure, web-based platform using a tablet or computer. This method 
offers greater protection because it guards against human error and negates the 
need for long-term storage of paper forms. Finally, digital recordings of diagnostic 
assessments will be stored in a secure, password protected website. The recordings 
themselves will be encrypted 
 
b. For research conducted internationally, describe any political or sociocultural 
considerations that may affect your research design (for example, in some 
communities it may not be customary to sign documents, etc.) 
N/A 
 
c. Discuss plans for ensuring necessary medical or professional intervention in the 
event of a distressed subject. 
The Co-PIs, project coordinator, PsycBC staff members, and PsycBC administrators 
will monitor the treatments and data collection; thus, they can assist in regularly 
monitoring any adverse events. Such negative occurrences are unlikely to be trial-
related, as all patients will be receiving treatment-as-usual. Therefore, any adverse 
event will be addressed with PsycBC’s well-established procedures for monitoring 
services and managing treatment-related disturbances. Nevertheless, any adverse 
event will be recorded and immediately reported to the IRB (UMass), PCORI 
(funder), and the project’s Data Safety and Monitoring Board (DSMB). 
 
Should, during the course of the study, a patient show evidence of psychological or 
physical deterioration, the patient will be assessed comprehensively in the domains 
of concern (except in the case of a life-threatening physical emergency, such as the 
emergence of acute chest pain, in which case 9-1-1 will be called immediately). If the 

 
99 
therapist deems that the patient meets criteria for a psychiatric hold (e.g., patient is 
an imminent danger to self or others), the therapist will arrange for the patient to be 
brought to the emergency department and will contact his/her PsycBC administrator 
and the PI to debrief. If a patient is not meeting criteria for a psychiatric hold, but is 
showing clear signs of decreased mental status, the therapist will continue to meet 
with the patient, as well as - in consultation with the PsycBC administrator - make 
arrangements for the most appropriate level of care. 
 
As noted, because the pre- and posttreatment diagnostic interviews will be 
conducted via telephone, the graduate RA (being trained as a clinician and 
supervised by their site PI, Dr. Constantino or Boswell, both of whom are licensed 
clinical psychologists and mental health care providers) will have the patient’s 
contact information (phone number and email address) on hand. If the patient 
reveals clinically elevated suicidality or homicidality, the RA will contact 9-1-1 and 
report the patient’s contact information and location address (which they will 
request verbally, if necessary) for emergency response. The RA, if applicable, will 
also execute any duty to warn to the best of their ability (in addition to contacting 
the local authorities). 
 
6. Benefits 
 
a. Describe the potential benefit(s) to be gained by the subjects or by the acquisition 
of important knowledge which may benefit future subjects, etc. (This DOES NOT 
include compensation or extra credit). 
The most direct benefit a participant in this study may receive is the reduction of 
symptom-related distress and improved functioning. In addition, patients (especially 
those in the match condition) will receive more personalized MHC. Psychotherapists 
(especially those in the match condition) may experience a greater level of positive 
impact across their caseloads. Given that the actual treatments being provided will 
not be manipulated, the benefits of participation are judged to far outweigh the 
potential study-specific risks. 
 
There is immense potential for future therapists and patients to benefit from the 
results of this study; if the hypotheses are supported, there will be cause for 
substantial revamping of MHC systems to capitalize on matching patients to 
therapists who have an empirically demonstrable track record of strength in treating 
patients with similar presenting problems. 
 
7. Procedures to Maintain Confidentiality 
 
a. Describe the procedures in place which protect the privacy of the subjects and 
maintain the confidentiality of the data, as required by the federal regulations, if 
applicable. 

 
100 
Multiple steps will be taken to protect confidentiality. As mentioned, minimal paper 
forms (e.g., diagnostic summary forms) will be kept in a locked cabinet in the PI’s 
locked Psychotherapy Research Lab. There will be no hard copy data collected at the 
PsychBC sites. Virtually all of the data collected in this study (including consent) will 
be through a secure, web-based platform using a tablet or computer. This method 
offers greater protection because it guards against human error and negates the 
need for long-term storage of paper forms. Digital recordings of diagnostic 
assessments will be stored in a secure, password protected website. The recordings 
themselves will be encrypted. 
 
Only designated study personnel will have access to identifiable, study specific, 
private information about human subjects. When registering on the TOP system, as 
required by PsychBC’s standard operating procedures, both patients and therapists 
are assigned a random number code that links all subsequent assessments and is 
separated from identifiable information. This random number code will function as 
each participant’s study code and will be used to link participants’ data. As noted, all 
therapist and patient data (outside of diagnostic assessment summaries and the TOP 
administrations) will be collected through a web-based platform. The assigned 
participant code will be used to link/aggregate information, so private information 
will not be requested after the baseline assessment/consent process. Only the PI 
and essential research staff will have access to the list that links identifiable 
information with the participant’s study code. Any audio recordings will be 
encrypted and password protected. Only the Co-PIs will know this password and 
have the capacity to access the recordings. When it is time to analyze the recordings 
for reliability coding, designated, trained RAs will also have access to the recordings; 
however, they will not have access to additional identifiable information (only the 
information required to complete the analysis). For any data used for research and 
publication purposes, the confidentiality of participant information will be ensured. 
 
b. If information derived from the study will be provided to a government agency, or 
any other person or group, describe to whom the information will be given and the 
nature of the information. 
The PI is required to submit information (i.e., contractual “deliverables”) on a regular 
basis to PCORI (the study sponsor), including IRB protocols, interim progress reports, 
advisory board meeting minutes, engagement plan updates, evidence of diagnostic 
criterion reliability from training cases, interim data reports, presentation abstracts 
and documentation of acceptance, manuscript copies, letters of endorsement from 
scientific and consumer groups, final data analysis summary, and final research 
report. Details on deliverables are available in the aforementioned (and attached 
and updated) milestone schedule. No PHI will be transmitted to PCORI. 
 
c. Specify where and under what conditions study data will be kept, how specimens 
will be labeled and stored (if applicable), who has access to the data and 
specimens, and what will be available to whom. 

 
101 
As noted, minimal paper forms (e.g., diagnostic summary forms) will be kept in a 
locked cabinet in the PI’s locked Psychotherapy Research Lab. There will be no hard 
copy research-only data collected at the PsycBC sites. Virtually all of the data 
collected in this study (including consent) will be through a secure, web-based 
platform using a tablet or computer. Digital recordings of diagnostic assessments 
will be stored in a password protected website, and securely deleted by the project 
contract’s term date. Only the relevant members of the research team will have 
access to the participants’ data and only the PI will have long-term access to 
identifiable information. As noted, all assessments will be linked with a participant 
code. Any records linking the code to the participant’s name or voice recording will 
be kept in a separate locked file cabinet in the PI’s office. These records will be 
destroyed 5 years after the contract term date. 
 
8. Potential Conflict of Interest 
 
a. Do any of the involved investigators or their immediate family (as described 
below) have consulting arrangements, management responsibilities or equity 
holdings in the Sponsoring company, vendor(s), provider(s) of goods, or 
subcontractor(s)? Y 
 
b. Do any investigators or their immediate family have any financial relationship with 
the Sponsoring company, including the receipt of honoraria, income, or 
stock/stock options as payment? N 
 
c. Is any Investigator(s) a member of an advisory board with the Sponsoring 
company? N 
 
d. Do any investigators receive gift funds from the Sponsoring company? N 
 
e. Do any investigators or their immediate family have an ownership or royalty 
interest in any intellectual property utilized in this protocol? Y 
 
“Immediate family” means a spouse, dependent children as defined by the IRS, or a 
domestic partner. If one or more of the above relationships exist, please include a 
statement in the consent form to disclose this relationship. i.e., a paid consultant, a paid 
member of the Scientific Advisory Board, has stock or stock options, or receives 
payment for lectures given on behalf of the sponsor. The consent form should disclose 
what institution(s) or companies are involved in the study through funding, cooperative 
research, or by providing study drugs or equipment. If you answer yes to any of the 
questions above, please go to the policies for more information. 
 
9. Informed Consent 
 

 
102 
You can add different Consent Forms, Alteration Forms, and Waivers. Provide consent 
process background information, in the table below, for each Consent Form(s), Alteration 
Form(s), and Waiver(s). 
 
9.1. Consent Form – therapist consent form revised 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
UMass personnel only: either the PC or an RA. 
 
How is consent being obtained? 
Therapists will meet or speak via teleconference with the UMass PC or an RA to learn 
about the study details/procedures and to provide formal consent through an online 
baseline survey to which they will be directed. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
The PI and his collaborators will provide close oversight of the entire protocol, including 
regular consultations with a study Advisory Board and the DSMB. 
 
9.2. Consent Form – therapist exit interview supplemental consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
The exit interviewer (i.e., RA or advisory board member). 
 
How is consent being obtained? 
If a therapist agrees to engage in an exit interview, the interviewer will review the study 
details/procedures and obtain supplemental consent through an online link to which the 
therapist will be directed. Coercion will be minimized by clearly stating that participation 
is voluntary. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
If the interviewer interacts with a therapist who appears to have competency issues in 
the decision-making process for engaging in the exit interview, they will immediate 
bring this concern to the PI or a Co-PI before enrolling them. The team will them make 
an informed decision as to whether to include that person in the interview protocol. 
 
9.3. Consent Form – patient consent form revised 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
UMass personnel only: either the PC or an RA. 

 
103 
 
How is consent being obtained? 
If a patient agrees to engage in a consent/baseline assessment session, the PC will 
schedule a telephone diagnostic interview with a trained graduate clinical psychology 
RA (employed at either UMass or University at Albany). The RAs will first review the 
study details/procedures and obtain consent through an online baseline survey to which 
the patient will be directed. Coercion will be minimized by clearly stating that 
participation is voluntary and will in no way impact the patient’s treatment. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
Competency for making one’s own treatment decisions will be an inclusion criterion for 
the study. Moreover, if a clinic staff member, the PC, or an RA interacts with a patient 
who appears to have competency issues in the decision-making process for engaging in 
the study, they will immediate bring this concern to the PI or a Co-PI before enrolling 
them. The team will them make an informed decision as to whether to include that 
person in the study. The DSMB will be consulted if appropriate. 
 
9.4. Consent Form – patient exit interview supplemental consent form 
 
Who is obtaining consent? The person obtaining consent must be knowledgeable 
about the study and authorized by the PI to consent human subjects. 
The exit interviewer (i.e., RA or advisory board member). 
 
How is consent being obtained? 
If a patient agrees to engage in an exit interview, the interviewer will review the study 
details/procedures and obtain supplemental consent through an online link to which the 
patient will be directed. Coercion will be minimized by clearly stating that participation 
is voluntary and will in no way impact the patient’s treatment. 
 
What steps are you taking to determine that potential subjects are competent to 
participate in the decision-making process? 
Competency for making one’s own treatment decisions will have been an inclusion 
criterion for the main study. Moreover, if the interviewer interacts with a patient who 
appears to have competency issues in the decision-making process for engaging in the 
exit interview, they will immediate bring this concern to the PI or a Co-PI before 
enrolling them. The team will them make an informed decision as to whether to include 
that person in the interview protocol. The DSMB will be consulted if appropriate. 
 
10. Assent Background 
 
All minors must provide an affirmative consent to participate by signing a simplified assent 
form, unless the Investigator(s) provides evidence to the IRB that the minor subjects are not 
capable of assenting because of age, maturity, psychological state, or other factors. 

 
104 
 
11. Attachments 
 
Document Type 
Document Name 
Attached Date 
Questionnaires 
Patient Phase 2 
During-Treatment Measures 
Packet 
10/04/2016 
Questionnaires 
Therapist Phase 2 
During-Treatment Measures 
Packet 
10/04/2016 
Questionnaires 
TOP-STP 
10/04/2016 
Questionnaires 
Stakeholder Exit Interview 
Protocols 
10/04/2016 
Federal Grant/Sub-
contract 
PCORI IHS-1503-
28573_Constantino_executed 
contract 
10/04/2016 
Federal Grant/Sub-
contract 
PCORI Original Contract 
Proposal_all sections 
10/04/2016 
Other 
Constantino Lab Personnel 
Link- Google Docs 
10/04/2016 
Other 
PCORI_Phase 2_Patient Data 
Collection Email Template 
11/13/2016 
Other 
PCORI_Phase 2_Patient Data 
Collection Reminder Call 
Script 
11/13/2016 
Other 
PCORI_Phase 2_Therapist 
Data Collection Email 
Template 
11/13/2016 
Other 
PCORI_Phase 2_Therapist 
Data Collection Reminder Call 
Script 
11/13/2016 
Questionnaires 
MINI 7.0.2 Standard 
11/13/2016 
Advertisements 
PCORI_Clinician 
Recruitment_Verbal 
Script_REVISED_clean 
08/13/2017 
Advertisements 
PCORI_Clinician 
Recruitment_Email_REVISED_clean 
08/13/2017 
Advertisements 
PCORI_Patient 
Recruitment_Verbal 
Script_REVISED_clean 
08/13/2017 
Questionnaires 
PCORI_Clinician Consent & 
Baseline Measures 
Packet_REVISED 
08/13/2017 

 
105 
Questionnaires 
PCORI_Patient Consent & 
Baseline Measures 
Packet_REVISED 
08/13/2017 
Questionnaires 
PCORI_Patient Posttreatment 
Measures Packet_REVISED 
w. debriefing form 
08/13/2017 
Questionnaires 
PCORI_Clinician 
Posttreatment Measures 
Packet_REVISED w. 
debriefing form 
08/13/2017 
Questionnaires 
TOP-CS & TOP-CM 
08/13/2017 
Other 
PCORI_Targeted Enrollment 
Tables_REVISED_clean 
08/13/2017 
Other 
PCORI Milestone 
Schedule_REVISED 
08/13/2017 
Other 
Participant Flow_REVISED 
08/13/2017 
Other 
Data Collection Schedule 
Revised 
08/13/2017 
Federal Grant/Sub-
contract 
Constantino_IHS1503-28573_Mod 
001 SUB_FE 20170808_FINAL 
EXECUTED MOD 
08/13/2017 
Other 
PCORI IRB Proposal_R1_for 
PsycBC_FINAL submitted 
08/13/2017 
Other 
ORI-PBC_Business Associate 
Agreement 
08/13/2017 
 
Obligations 
 
Obligations of the Principal Investigator are: Modifications - Changes in any aspect of the study 
(for example, project design, procedures, consent forms, advertising materials, additional key 
personnel or subject population) will be submitted to the IRB for approval before instituting the 
changes; Consent Forms - All subjects will be given a copy of the signed consent form. 
Investigators will be required to retain signed consent documents for six (6) years after close of 
the grant or three (3) years if unfunded; Training - Human subject training certificates, including 
those for any newly added personnel, will be provided for all key personnel; Adverse Events - 
All adverse events occurring in the course of the protocol will be reported to the IRB as soon as 
possible, but not later than ten (10) working days; Continuing Review – IRB Protocol Report 
Forms will be submitted annually at least two weeks prior to expiration, six weeks for protocols 
that require full review; Completion Report - The IRB will be notified when the study is 
complete. To do this, complete the IRB Protocol Report Form and select “Final Report.” Training 
- Human subject training certificates, including those for any newly added personnel, will be 
provided for all key personnel; Adverse Events/Unanticipated Problems - All events occurring in 
the course of the protocol will be reported to the IRB as soon as possible, but not later than five 

 
106 
(5) working days; Continuing Review - IRB Protocol Report Forms will be submitted annually at 
least two weeks prior to expiration, six weeks for protocols that require full review; Completion 
Report - The IRB will be notified when the study is complete. To do this, complete the IRB 
Protocol Report Form and select “Final Report.” 
 
The Principal Investigator has read and agrees to abide by the above obligations. Y 
 
 

 
107 
D. Summary of the 3rd formal modification to the PCORI contract (January 2018); no revision 
to the study protocol 
 
CONTRACT MODIFICATION SUMMARY 
 
On January 8, 2018, PCORI approved the 3rd contract modification (IHS-1503-28573 
Modification 003), which included the following changes: 
 
• We updated our patient enrollment target from 264 to 281 based on a more 
conservative attrition rate of 25% (up from our original, less conservative estimate of 
20%). With this modification, enrolling 281 patients into the trial allowed us to meet our 
target of 211 usable case for final data analysis (281 * .75 = 211). Based on this 
adjustment, we also updated accordingly our Estimated Final Racial/Ethnic and Gender 
Enrollment Table. 
 
• We updated our timeline and milestone schedule based on the PCORI-approved 6-
month extension to the contract end date. This no-cost extension was precipitated by 
delays in patient recruitment, which we successfully addressed via our formal project 
remediation plan. 
 
For this modification, there were no changes to the study protocol. 
 
 

 
108 
Online Survey Consent Form – Patients 
 
Title of Research: “Enhancing Mental Health Care” Study 
IRB Number: 2016-3401 
Researcher: Michael J. Constantino, Ph.D. 
 
Description of the Study 
 
You are being asked to participate in this research study because you are an adult (age 18-70) 
who makes your own treatment decisions and will receive outpatient psychotherapy through a 
PsychBC therapist. 
 
PsychBC is collaborating with researchers at the University of Massachusetts, University at 
Albany, and Outcome Referrals, Inc. on a study aimed at learning more about ways to improve 
mental health care. The study is being funded by the Patient-Centered Outcomes Research 
Institute (PCORI; IHS-1503-28673). 
 
Your participation involves the following activities: 
 
• After reading and signing this consent document, you will be directed to a brief survey, 
which will take no more than 30 minutes to complete. When done, you will speak with your 
intake worker who will randomly assign you to receive treatment from a PsychBC provider 
in one of two patient-provider match conditions that we are testing. Neither you nor your 
therapist will know the basis of the match, though you will be debriefed after treatment 
ends. Importantly, outside of your assignment to therapist, your treatment will not be 
affected. Your therapist (who has already agreed to participate in this study) will conduct 
treatment-as-usual with no influence from the research team. We will, though, ask for you 
to remain with the same therapist throughout your treatment, though you are certainly free 
to request a transfer if you prefer. 
 
• Soon after your assignment to a therapist, a research team member will email you to orient 
you to the few remaining study procedures. During treatment, you will be asked to 
complete several study-specific measures online at regular intervals (the project 
coordinator will email links to these surveys with reminders to complete them at the 
appropriate times). These measures will take no more than 15 minutes to complete each 
time. 
 
• After your treatment ends or at week 16, whichever comes sooner, you will again be asked 
to complete several study-specific measures online (taking no more than 15 minutes to 
complete). 
 
• If you give consent, you can also be randomly selected to complete measures at 1 year 
following your final session. This will give us a sense of your longer-term mental health 
functioning. 

 
109 
 
• Finally, you can express interest in taking part in a separate brief exit telephone interview. If 
you are selected, the interview will involve answering a series of questions about how to be 
responsive to the study findings in terms of dissemination, implementation, and 
policymaking. This phone interview will take 45-60 minutes. 
 
Your participation is completely voluntary and you can stop at any time. Your responses will 
remain confidential, including from your therapist. Also, your participation or non-
participation in this research will in no way effect your standing or treatment at your 
community mental health clinic. 
 
Benefits 
 
You may experience a direct benefit from your participation in this study. Your consideration of 
the questionnaire items might provide additional insight into your functioning and how your 
therapy is going, or went. 
 
Moreover, your information will help us test whether different patient-clinician match 
scenarios differentially affect patients’ mental health symptom and functioning. Thus, society 
and future mental health treatment may benefit from your participation. 
 
Risks and Protections 
 
Your participation in this study is unlikely to involve significant risk. You will receive treatment-
as-usual. The only differences are that you will (1) be randomly assigned to receive treatment 
from a PsychBC provider in one of several match conditions that we are testing, and (2) you will 
complete the measures discussed above that assess your mental health symptoms and 
functioning. The items may cause some slight emotional discomfort; however, this will not be 
over and above what you routinely discuss during your therapy sessions. The additional time 
that it will take to complete the study questionnaires may also be a slight inconvenience to you. 
However, you will be paid for your time according to the compensation schedule below. 
 
Your responses to the measures will be protected, including from your therapist. However, if 
you endorse items that seem critical for your therapist to know, a clinic administrator may 
choose to inform the therapist of just these relevant responses. This could influence what you 
discuss in subsequent sessions, though it would be in the service of improving your care. 
 
Risk of loss of private information/confidentiality is considered minimal given that relevant 
study information and online forms will be linked with a study code, rather than identifying 
information. Moreover, virtually all of the data collected in this study (including consent) will be 
through a secure, web-based platform using a tablet or computer. This method offers greater 
protection because it guards against human error and negates the need for long-term storage 
of paper forms. Summary documentation might take hard copy form, in which case the forms 

 
110 
would be stored in a locked cabinet in a locked psychotherapy research lab. These forms will be 
destroyed after the study is complete. 
 
Assessment results can only be accessed by research personnel through a protected username 
and password. Only your name, phone number, and email address will be collected during the 
consent process to facilitate your payment for participation. You will be paid via email with 
Amazon.com gift cards, though your phone number will be a backup contact means should 
there be technical issues with email. Your contact information will not be connected to any 
other personal information, and it will be kept in a locked cabinet in a locked research office. All 
information obtained in this study is strictly confidential unless disclosure is required by law or 
deemed clinically necessary (as discussed above). In addition, the Institutional Review Board, 
the sponsor of the study (PCORI), and University or government officials responsible for 
monitoring this study may inspect these records. For any data used for research and publication 
purposes, the confidentiality of participant information will be ensured. 
 
Your participation in this research is entirely voluntary. Even after you agree to participate in 
this research, you may decide to leave the study at any time without penalty or loss of benefits 
to which you may otherwise have been entitled. You should also be aware that the investigator 
may withdraw you from participation at his/her professional discretion. 
 
Compensation 
 
In exchange for your participation in the study through the full 16 weeks of treatment, or a 
planned termination prior to 16 weeks, you will be compensated in full with a $50.00 
Amazon.com gift card. However, if you drop out of treatment prior to week 16, or your last 
session was not a planned termination, your compensation will be prorated. Specifically, we will 
deduct $3.00 per “missing” week of data collection. For example, if you complete the 
measurement schedule through week 8 (9 weeks, including baseline), and you did not engage 
in a planned termination, you will have “missed” 8 weeks of data collection. Your compensation 
would be adjusted as follows: $50 - $24 ($3 x 8 weeks) = $26.00. Note that you would also have 
the option to forgo this compensation if you no longer wished to be contacted by the research 
team in any way. 
 
If you engage in the exit interview, you will be compensated an additional $100 Amazon.com 
gift card. 
 
Potential Conflict of Interest Disclosure 
  
Dr. David Kraus (a co-investigator) is a stockholder of Outcome Referrals, Inc. (a subcontractor 
to UMass on this grant). Dr. Kraus is also the inventor of the scientific referral patent (US Patent 
No. 7,873,525) that is being tested through this grant. 
 
Contact Information 
 

 
111 
If you have questions about this project or if you have a research-related problem, you may 
contact the project coordinator, Felicia Romano (508-331-3905, fromano@umass.edu), or the 
principal researcher, Dr. Michael Constantino (413-545-1388; constanm@umass.edu). 
 
If you want to contact someone not directly involved in this study, you can reach the 
psychology Department Chair through Laura Wildman-Hanlon (413-545-2378). 
 
If you have any questions concerning your rights as a research subject, you may contact the 
University of Massachusetts Amherst Human Research Protection Office (HRPO) at (413) 545-
3428 or humansubjects@ora.umass.edu.  
 
Agreement to Participate 
 
By clicking the first “I agree,” you affirm that (1) you are at least 18 years of age, which is the 
minimum age to participate in this study, and (2) the purpose and nature of this research have 
been sufficiently explained, that you have read and understood this consent form, and that you 
agree to participate in this research study. You are free to withdraw at any time simply by 
closing this browser window (prior to submission of your responses). Please print a copy of this 
page for your records. 
 
[I agree/I do not agree] 
 
By clicking the second “I agree,” you affirm that you will not discuss the contents of this study 
with anyone other than the researchers (unless you are harmed due to participation).   
 
[I agree/I do not agree] 
 
 
Patient Authorization to Release Information for Research 
 
If you have consented to participate in the study, we also need written authorization to share 
limited information about you with the research team. Specifically, we ask for your name, 
preferred telephone number, and preferred email address. This will allow the research team to 
contact you about study procedures and to compensate you for your time.  
 
You are in no way obligated to release this information. Your treatment at PsychBC will not 
be affected in any way whether you are or are not you engage in the study. 
 
I have read and understood the present request, and I authorize PsychBC to share my contact 
information with the research staff. I understand that no other protected health information 
will be shared with the research team prior to your consent to participate in the study. 
 
[I agree/I do not agree] 
 

 
112 
By clicking the second “I agree,” you authorize PsychBC to share the following contact 
information: 
 
Name:  
Preferred telephone number: 
Preferred Email address: