Collaborative Care for Chronic Pain in Primary Care
Phase: NA
Conditions: Chronic Non-malignant Pain
Sponsor: Kaiser Permanente

 
 
 
 
 
 
 
 
Collaborative Care for Chronic Pain in 
Primary Care (PPACT)  
 
PROTOCOL 
NCT02113592 
 
07/07/2015 
 
 
Sponsored by  
The National Institutes of Health (NIH) Health Care Systems Research Collaboratory by the 
NIH Common Fund through cooperative agreement U24AT009676 from the Office of 
Strategic Coordination within the Office of the NIH Director and cooperative agreements 
UH2AT007788 and UH3NS088731 from the National Institute of Neurological Disorders and 
Stroke (NINDS)  
 
 
 
 
 
 

 
 
For publication 
 
 
2 
Table of Contents 
1. OVERVIEW ................................................................................................................................................. 4 
2. STUDY AIMS .............................................................................................................................................. 5 
3. STUDY DESIGN ........................................................................................................................................... 6 
3.1. Cluster Randomized Design ............................................................................................................................. 6 
3.2. Eligibility .......................................................................................................................................................... 6 
3.2.1. Primary Care Providers ............................................................................................................................. 6 
3.2.2. Patients ..................................................................................................................................................... 6 
3.3. Recruitment and Retention Strategies ............................................................................................................ 6 
3.3.1. Recruitment .............................................................................................................................................. 6 
3.3.2. Retention .................................................................................................................................................. 7 
3.4. Informed Consent and HIPAA Compliance ...................................................................................................... 8 
3.4.1. Patients ..................................................................................................................................................... 8 
3.4.2. Primary Care Providers ............................................................................................................................. 8 
3.5. Treatment Arms .............................................................................................................................................. 9 
3.5.1. Usual Care ................................................................................................................................................. 9 
3.5.2. PPACT Intervention .................................................................................................................................. 9 
4. PPACT INTEGRATED INTERVENTION COMPONENTS .................................................................................. 10 
4.1. Comprehensive Intake Evaluation and Re-Assessment ................................................................................ 11 
4.1.1. Overview of Goals and Strategies .......................................................................................................... 11 
4.2. Core Groups for Coping Skills Training / Yoga-Based Adapted Movement Practice ..................................... 14 
4.2.1. Overview of Goals and Strategies .......................................................................................................... 14 
4.2.2. Schedule and Delivery ............................................................................................................................ 15 
4.3. Individualized Care Management.................................................................................................................. 16 
4.3.1. Overview of Goals and Strategies .......................................................................................................... 16 
4.4. Pharmacist Chart Review............................................................................................................................... 16 
4.5. Physical Therapy Consultation ...................................................................................................................... 16 
5. MEASUREMENTS AND EVALUATIONS ....................................................................................................... 16 
5.2. Primary and Secondary Patient Outcome Measures .................................................................................... 16 
5.2.1. Pain Severity and Functioning ................................................................................................................ 16 
5.2.2. Use of Opioids and Other Medications .................................................................................................. 17 
5.2.3. Utilization of Health Care Services ......................................................................................................... 17 
5.2.4. Other Secondary Outcomes ................................................................................................................... 17 
5.2.5. Clinical Covariates................................................................................................................................... 18 
5.3. Quantitative Data Collection Schedule ......................................................................................................... 18 
5.4. Supporting the Clinical Collection of Patient Reported Outcomes ............................................................... 19 
5.5. Process Evaluation ......................................................................................................................................... 19 
6. STUDY INTERVENTION STAFF TRAINING AND SUPERVISION ...................................................................... 21 
7. SAFETY REPORTING AND MONITORING .................................................................................................... 22 
7.1. Adverse Events and Serious Adverse Events ................................................................................................. 22 
7.2. Data Safety Monitoring Plan (DSMP) ............................................................................................................ 22 
8. STATISTICAL METHODS ............................................................................................................................ 23 
8.1. Sample Size and Power.................................................................................................................................. 23 
8.2. Randomization............................................................................................................................................... 23 
8.3. Dropout and Withdrawal............................................................................................................................... 23 
8.4. Quantitative Data Methods and Analysis ...................................................................................................... 24 
8.5. Economic Data Methods and Analysis .......................................................................................................... 25 
9. LITERATURE CITED ................................................................................................................................... 26 

 
 
For publication 
 
 
3 
 
STUDY PROTOCOL MODIFICATIONS 
04/21/2014: Originally, primary care providers could opt out after clustering and randomization occurred. 
However, with this approach we could not include an additional PCP to take the place of the PCP who had opted 
out. Therefore, we would be left with fewer than the optimal number of PCPs in each cluster wave. We changed 
the protocol to have PCPs opt out prior to clustering and randomization.  
 
07/07/2015: The target number for enrollment was changed from “up to 160 primary care providers (PCPs) and 
1,200 patients with chronic pain on long term opioid treatment (CP-LOT)” to “up to 375 PCPs and 1,000 patients 
with chronic pain on long term opioid treatment (CP-LOT).” In addition, the number of PCP clusters changed from 
120 PCP clusters to 106 clusters, with 53 receiving the intervention and 53 receiving usual care. 
The description of PCP recruitment was changed to reflect that PCPs are provided an in-person presentation to 
the study rather than an informational letter. The detailed PowerPoint presentation covers all elements of 
informed consent. PCPs that are unable to attend the in-person presentation are sent an email with an overview 
of the study and a copy of the PowerPoint presentation.  

 
 
For publication 
 
 
4 
1. OVERVIEW 
Chronic pain affects at least 116 million adults in the United States and exacts a tremendous cost in suffering and lost 
productivity. While health systems offer specialized pain services, the primary care setting is where most patients 
seek and receive care for pain. Patients with chronic pain are seen primarily by primary care providers (PCPs) in 
settings where medications have become the mainstay of pain treatment for many PCPs with few other treatment 
options available to them. Yet rising concerns about the safety and efficacy of opioid medications for the treatment 
of chronic pain have heightened awareness of the need for better treatment options. Primary care-based treatment 
of chronic pain by interdisciplinary teams (including behavioral specialists, nurse care managers, physical therapists, 
and pharmacists) is one of the most promising approaches for improving outcomes and managing costs. Recent 
national health policy changes, in addition to the increasing recognition of the high prevalence and cost of chronic 
pain conditions present a unique opportunity to shift the care paradigm for patients with chronic pain.  
The overarching goal of the Pain Program for Active Coping and Training (PPACT) is to test the effectiveness of 
integrating an evidence-based, interdisciplinary pain management intervention within a primary care environment. 
The trial is conducted in three health care delivery systems (Kaiser Permanente Georgia, Hawaii, and Northwest) and 
is anticipated to involve up to 375 PCPs and 1,000 patients with chronic pain on long term opioid treatment (CP-LOT).  
The PPACT trial compares usual care services to an interdisciplinary, primary care-based approach to treating CP-LOT 
patients. The primary outcome for the trial is a composite of pain severity and interference as measured by the 4-
item version of the Brief Pain Inventory – Short Form, known as the PEGS,1 and administered routinely in clinical care. 
Secondary outcomes include patient satisfaction, potential reductions in dispensed opioid medication (measured as 
morphine equivalent dose), and other health service use variables extractable through patients’ electronic health 
records (EHR).  

 
 
For publication 
 
 
5 
2. STUDY AIMS 
Aim 1: Conduct a cluster randomized pragmatic clinical trial in 106 PCP clusters across the three KP health plan 
settings (Hawaii, Northwest, Georgia) to compare the effects of the PPACT intervention to usual care on:  
• 
Patients’ pain symptoms, pain-related functioning, and satisfaction with health care services; 
• 
Patients’ use of health care services, including receipt of opioid medication; and, 
• 
The cost of the program and economic impact of the intervention.  
Aim 2:  Conduct process and formative evaluations to understand, describe, explain, and enhance intervention 
Reach, Effectiveness, Adoption, Implementation, and Maintenance. 

 
 
For publication 
 
 
6 
3. STUDY DESIGN 
3.1. Cluster Randomized Design 
The trial compares usual care services to a multidisciplinary, primary care-based approach to treating CP-LOT 
patients. The study team has planned for up to 1,000 patients from 106 clusters of PCPs (6-12 patients per cluster) 
across three KP health plans (KPNW, KPGA, and KPH) to participate in the trial, with 53 PCP clusters randomized to 
the PPACT intervention and 53 PCP clusters to usual care representing 9-30 PCP clusters in each of the three 
participating KP regions.  
3.2. Eligibility 
 
3.2.1. Primary Care Providers 
Primary care provider participants (PCPs). All PCPs (internal medicine and family practice providers [medical doctors 
– MD, doctors of osteopathic medicine – DO, physician assistants – PA, and nurse practitioners – NP with established 
panels of patients]) in the three participating KP regions are eligible to participate in the trial.  
 
3.2.2. Patients 
Patient inclusionary eligibility criteria includes the following: 
1. Adult (18 years of age or older) health plan members from KPNW, KPGA, and KPH who receive their primary 
care services from participating PCPs 
2. Health plan membership of at least 180 days/6 months duration  
3. Long term opioid use defined by: 90+ day supply of short acting opioid spanning at least 120 days or 2 or 
more long acting opioid dispense in the past 180 days 
4. Pain diagnosis within the past year 
Patient exclusionary criteria are limited to the following:  
1. Patients currently enrolled in intensive addiction medicine services or with evidence of active substance 
dependence of sufficient severity to interfere with their ability to actively participate in the 
behavioral/lifestyle change program 
2. Patients with cognitive impairment severe enough to preclude their participation in a behavioral/lifestyle 
change program 
3. Patients with current malignant cancer diagnosis 
4. Any evidence of  patient having received hospice or other end-of-life palliative care within past year 
5. Patients unwilling to participate in the skills training/lifestyle change elements of the program (PCPs cannot 
mandate that their referred patients participate in the program).  
Exclusionary criteria are purposively kept to a minimum to ensure that study participants resemble those most in 
need of these services in the broader population.  
3.3. Recruitment and Retention Strategies 
 
3.3.1. Recruitment  
Based on the previously stated eligibility criteria, one database for each regional healthplan compiles the health 
record numbers (HRNs) of all PPACT eligible patients who are paneled to all participating PCPs. This database is 
refreshed just prior to beginning recruitment of any given cluster to ensure the most accurate PCP paneling of 
patients. PCPs in each clinic are invited to participate in an informational meeting held in their primary care clinic 
prior to recruitment efforts in that clinic. During this meeting, PCPs are informed of their option to opt out of 

 
 
For publication 
 
 
7 
participation in the study. The PCP’s receipt of the study overview, expectations of participating, and absence of 
opting out of the study constitutes that PCP’s informed consent.  
For half of the participating PCPs their patients are randomized to participate in the intervention while for the 
otherhalf their patients are randomized to the usual care condition. PCPs, other clinic, and study staff are blinded to 
their randomization status until all of their patients have been recruited. The number of PPACT eligible patients on a 
given PCP’s panel ranges from 1 to 146 in the three KP healthcare systems. One goal of this project is to target the 
more complex patients with chronic pain; those which often place the greatest strain on PCPs and the regions as a 
whole. In an effort to accomplish this goal the study team 1) prioritizes recruitment of patients on ≥ 120 daily 
morphine equivalent dose of opioids, patients concomitantly prescribed benzodiazepines, and patients with high 
primary care service utilization (defined as 12 or more contacts in the prior 3 months) and 2) asks that PCPs review a 
list of their prioritized patients to ensure that all patients on the list are appropriate candidates for the program.  
All PCP approved PPACT eligible patients are mailed a recruitment brochure summarizing the study and providing an 
opt out contact number.. Additionally, it states that patients might be contacted within a week to participate in the 
study if they have not called to opt-out. The brochure specifies that there are limited spaces in the program such that 
not everyone receiving a brochure will be contacted to participate. 
Consistent with the pragmatic nature of this trial and the full partnership of the three KP healthcare systems in 
incorporating this intervention reorganizing pain-related services and enhancing pain-related assessments, patients 
are informed that the study represents a partnership between their KP providers and the research team to evaluate 
existing pain-related services within the health plan and to evaluate the impact of providing a closer coordination of 
pain-related services available within the primary care clinics. KP staff working in coordination with the health plan 
and the research center then attempt to contact by phone those patients who do not opt out. Recruitment callers 
invite patients to enroll in the clinic summarizing the intervention as one that provides closer coordination of pain-
related services within the primary care clinics, verbally reviewing the elements of consent (waived written consent), 
and obtain baseline patient reported outcomes. The study’s target cluster size is 10-12 patients per cluster, thus, 
recruitment for any given cluster stops when 10-12 patients in a given cluster have been consented. Patients on the 
PCP approved list who have not been contacted are not recruited for the study. When recruitment for each PCP 
cluster within the cluster wave has been finalized, interventionists (staff not involved in the initial recruitment calls of 
patients) are unblinded to the study condition of each PCP cluster and their associated patients. These 
interventionists contact patients in the usual carecondition to inform them of their next assessment study contact in 
3 months. Patients in the intervention condition are contacted to schedule their intake evaluation sessions.  
 
3.3.2. Retention 
Intervention retention.There is a recognition that the target patient population for the PPACT trial is one for whom 
motivation and adherence to recommended treatment and self-care practices is frequently an issue.2-4  Accordingly, 
everyday clinical interventions frequently embed elements of motivational enhancement techniques to encourage 
patient adherence and retention as does the PPACT intervention.5 A rescue session protocol details the process 
interventionists undergo when individuals do not attend sessions regularly or unable to attend due to various 
circumstances (e.g., surgery, transportation issues). While attempts are made by interventionists to engage all 
individuals with the highest “dose” of intervention that is feasible, as a pragmatic trial, attention to retention is 
informed by what would be realistic for included healthcare systems to sustain in everyday clinical care.  
Data collection retention. The study employs a tiered system (patient health record sent to patient through health 
plan secured e-mail system, automated telephone calls using the KP Message Center, live telephone calls by medical 
assistant) to enhance collection of the primary study outcome, the PEGS.6 This approach builds off of the current 
clinical data collection processes available in the health care delivery systems in which the trial is conducted and, as 
such, represents a modest enhancement of existing clinical systems. Importantly, all participants may refuse to 
provide specific responses or measures or may withdraw from enhanced data collection procedures or the 
intervention at any time. 

 
 
For publication 
 
 
8 
3.4. Informed Consent and HIPAA Compliance 
 
3.4.1. Patients 
The study team obtains oral consent and oral review of HIPAA elements from all participants enrolled in the study, 
accordingly the  IRBs in the three included KP healthcare organizations have granted a waiver of signed informed 
consent and an alteration of privacy rule authorization (HIPAA) (no signature). When study interviewers obtain oral 
consent from a prospective study participant over the telephone, the interviewer indicates that each element of 
informed consent and HIPAA privacy guidelines/study use of health data have been reviewed with the KP member by 
checking the requisite element within the patient record in the study electronic tracking system. This helps ensure 
that each element of informed consent and HIPAA privacy guidelines/study use of health data have been thoroughly 
reviewed with all prospective participants.  Following screening and oral consent, an informational letter including all 
elements of informed consent, is sent to all participants recently enrolled in both intervention and usual care. For 
individuals randomized to the intervention arm, the letter includes details of study activities and expectations. For 
this study, obtaining oral rather than written consent is an appropriate consent procedure because intervention 
activities involve the coordination of clinical care services already available to most KP members (e.g., physical 
therapy, behavioral services, nurse care management, and pharmacy) and therefore the intervention is expected to 
cause no more risk of harm than what already exists for patients undergoing usual care treatment for chronic pain. 
Some patients participating in the intervention are expected to have worsening pain and/or other physical or 
emotional problems during the study period. However, these are risks inherent in the population and would occur 
whether or not they were enrolled in the study and  the risk of adverse outcomes should not be heightened as a 
function of being enrolled in the study. Further, because the intervention is embedded directly in the primary care 
clinics and conducted in partnership with participating patients’ PCPs, in the event that a patient’s symptoms 
significantly worsen during the intervention, their PCP will be immediately contacted by a PPACT intervention team 
member and their PCP will work with the patient to identify and provide appropriate care. This is consistent with the 
standard of care provided at KP.  
For the pragmatic trial, data is assessed and recorded in accordance with regular clinical care (either in the 
intervention arm or the comparison usual care condition), and subsequently extracted by our research staff from EHR 
and administrative databases in each of the KP health plans participating in the study. As such the IRBs have granted 
a waiver of consent to use computerized records to collect assessment data for the trial. The study team believes 
that the assessment portion of the pragmatic trial clearly satisfies the criteria of 45 CFR 46:116 for waiver of informed 
consent. Those criteria are: 
• 
“The research involves no more than minimal risks to the subjects” – The only risk to participants from 
this procedure is violation of confidentiality, which the study protects against. 
• 
“The waiver or alteration will not adversely affect the rights or welfare of the subjects” – Research use of 
records will have no effect on insurance coverage, access to care, or eligibility for any benefit from 
participating health systems. The various HMO regions already routinely permit the use of EHR records 
for research purposes without member consent.  
• 
“The research could not practically be carried out without the waiver or alteration” – It would not be 
possible to meet in person with all PPACT participants to obtain written consent. 
Finally, patients in the intervention arm of the study are informed that group sessions may be recorded and shared 
with supervising staff to evaluate the quality of services the patient is receiving and to help the PPACT providers and 
the health plan understand how to best improve services for health plan members. A signed release of information is 
obtained for the potential recording of groups. Should a patient not be willing to sign the release of information, the 
group they are assigned to is not audio recorded.  
 
3.4.2. Primary Care Providers 

 
 
For publication 
 
 
9 
PCPs are provided a thorough presentation at their regular primary care meeting (or presentation materials when not 
in attendance), which includes all elements of informed consent. The PCPs have a one week opt-out opportunity 
after the meeting or receipt of materials. The sIRB deems this is an appropriate consent procedure given the minimal 
risk to the PCP posed by the study. Benefits to the PCPs who are randomized to the intervention arm of the study 
include assistance managing their patients with chronic pain who are on opioids, patients who often utilize primary 
care services at a greater capacity. The PCPs in the intervention arm are provided with a comprehensive assessment 
of their study patients, the intervention team’s pain management recommendations, and templates for 
communicating with those patients who often present with challenging communication styles. All PCPs participating 
in the trial regardless of which study arm they (and their patients) are randomized have the benefit of the study 
supporting quarterly collection of patient reports of pain and pain-related functioning using the standard monitoring 
tool adopted in all KP region to monitor patients on long term opioid therapy. Participating providers, regardless of 
study arm, are free to refer their patients to any pain-related services they deem warranted.  
3.5. Treatment Arms 
 
3.5.1. Usual Care  
As noted in the earlier section 3.3.1 on “Recruitment,” all potential patient participants are approached about their 
willingness to participate in regular quarterly assessment of their pain and related functioning and status as well as 
their availability and willingness to participate in the PPACT intervention if offered and oral consent is obtained 
before they are enrolled in the study. For those patients whose PCPs are randomized to the usual care condition, 
further study-related contact is limited to quarterly study data collection over the following year.  
Those patients in PCP clusters randomized to either the PPACT intervention or the usual care arm of the study are 
able to utilize all diagnostic and treatment health plan services available to them for pain and related conditions.  
 
3.5.2. PPACT Intervention 
The intervention involves (1) a comprehensive intake evaluation with periodic reevaluation (evaluations performed 
by a behavioral health specialist or nurse care manager, physical therapist (PT), and a chart based medication review 
by a pharmacist), (2) group coaching sessions (co-led by a behavioral specialist and nurse care manager, with 
consultation from the PT to support the adapted movement activities), and (3) interim care management contacts 
(conducted by the behavioral health specialist or the nurse care manager). Section 4 below describes the PPACT 
intervention in more detail. 
 
 

 
 
For publication 
 
 
10 
4. PPACT INTEGRATED INTERVENTION COMPONENTS 
The intervention consists of (1) a comprehensive intake evaluation with, (2) group coaching sessions, and (3) interim 
care management contacts by intervention team members as needed. The interdisciplinary intervention team 
includes a behavioral health specialist (most often a social worker or master’s-level counselor), a nurse care manager, 
and a PT with additional consultation from a pharmacist.  
The intervention approach integrates ancillary services—behavioral services, nursing care management, PT, and 
pharmacy consultation—into the primary care environment with the goal of helping patients develop the skills to 
increasingly self-manage their condition. Many CP-LOT patients may have received brief trials of one or more of 
these services, albeit in a fragmented fashion. By coordinating such services within the primary care setting and 
providing services consistent with evidence-based treatment protocols, CP-LOT patients receiving the intervention 
are expected to be better able to manage their chronic pain and reduce their reliance on opioid medication. This 
approach is consistent with chronic care models of care,7-11 previous collaborative care and multidisciplinary 
approaches to the management of chronic pain,12-16 and chronic pain treatment guideline criteria.17-19 A visual 
depiction of participant flow through the intervention is shown in Figure 1: PPACT Intervention Description. Each 
segment of the intervention is described in more detail below.  
 
 

 
 
For publication 
 
 
11 
4.1. Comprehensive Intake Evaluation and Re-Assessment 
 
4.1.1. Overview of Goals and Strategies 
An important part of the PPACT program involves a comprehensive up-front assessment of the patients’ pain and 
functional status to elucidate possible contributing factors to their pain and impaired functioning. The evaluation is 
designed to orient patients to the program approach, address any questions or concerns they have about 
participating in the program, and use assessment findings to develop individualized treatment plans that can then be 
added to the patients’ EHR to guide PPACT program staff, PCPs, and other health plan providers. The “persistent pain 
cycle” depicted in Figure 2: Reversing the Persistent Pain Cycle provides the framework that the PPACT team uses to 
orient patients, PCPs, and other health care providers to the potential contributors to chronic pain and those most 
pertinent for a given patient. This framework is also used as a means of helping patients understand the role of pain 
management skills training and utilizing movement and body awareness strategies in reversing the cycle of persistent 
pain. Further, as the framework orients both patients and clinical providers to the many factors (the pain amplifiers 
in Figure 2) that may exacerbate pain and contribute to pain management difficulties, it helps motivate patients and 
providers alike to begin to address these 
factors. 
Although some of these “pain amplifiers” 
are addressed directly within the PPACT 
program or use CBT techniques useful for 
treating such conditions (e.g., stress, 
depression), other problems are outside the 
purview of the program but instructional 
materials have been developed to help 
patients understand the link between 
persistent pain and these “amplifiers” 
(smoking and pain, sleep problems and 
pain) or intervention staff work with 
patients and their PCPs to identify health 
plan and community resources for 
addressing their needs. Throughout the 
PPACT program, patients are encouraged to 
focus on the pain management pratices and 
treatments from the upper quadrants of the 
figure labeled as “First Line 
Recommendations/Active Approaches” 
(green and blue domains summarizing “Pain 
Management Skills” and “Movement & 
Body Awareness Strategies”)  to reinforce 
the active role patients can take in 
managing their pain and beginning to 
reverse the cycle depicted in Figure 2. Our 
clinical and research experience suggests 
that many patients focus most of their 
energy on trying to identify causes and 
contributors to pain and seeking out 
“Common Conventional Medical Treatment 
Options” (depicted in the lower right orange 
quadrant); patients are encouraged to view 
the more passive approaches to pain 
Figure 2. Reversing the Persistent Pain Cycle 
 

 
 
For publication 
 
 
12 
management summarized in the lower quadrants of the figure as second line treatments to be used more judiciously 
in augmenting the day-to-day lifestyle changes more consistent with pain management approaches summarized in 
the top of the figure.  
With regard to the PPACT intake evaluation, all assessment instruments have been chosen for their clinical utility to 
help PPACT program staff, patients, and PCPs best identify targets for skills training and ancillary treatment needs 
and to gauge patients’ functional progress in the program. Intervention assessment tools include those designed to 
address the following: 
• Pain-related symptoms and disability. Although administered routinely through every day clinical care, the 
intervention team administers the complete BPI20,21 to gauge changes in pain severity and impairment related 
to pain (pain interference) and to give patients and their PCPs information about patient functioning during 
the course of the intervention. The intervention team also administers the Oswestry Disability Index,22-24 a 
psychometrically validated assessment for disability due to musculoskeletal pain conditions that is widely used 
by KP PTs and thus provides a metric for them to gauge PPACT patient functioning and change. Finally, a few 
questions are included to get a sense of the duration of participants’ chronic pain, their attribution regarding 
the cause, and the impact of their condition on employment and/or disability status. 
• Identifying treatable comorbidities and pain generators. Several clinical issues can exacerbate pain severity 
and contribute to functional impairment. Accordingly, the intervention team screens for depression using the 
Personal Health Questionnaire,25 anxiety symptoms using the Generalized Anxiety Disorders Scale,26 post-
traumatic stress disorder using the Primary Care PTSD screen,27,28 adverse childhood experiences using an 
adaptation of the ACE questions,29 alcohol abuse using the Audit-C,30,31 and sleep problems and sleep apnea 
using the Epworth Sleepiness Scale.32-36 All selected screening measures included in our assessment battery 
have demonstrated strong psychometric properties and are regularly used in everyday clinical care settings, 
including the health plans in which the study is conducted.  
• Helping patients and PCPs identify potential opioid problems and concerns. Because a primary focus of the 
proposed study is to help patients in the PPACT intervention identify non-opioid alternatives for pain 
management, the intervention team uses the Prescribed Opioids Difficulties Scale37 to help patients identify 
consequences of their opioid treatment (e.g., difficulty concentrating, excess sedation or fatigue, 
constipation). The scale also provides an entry point for the PPACT interventionists to initiate a dialogue with 
participating patients about the pros and cons of using opioid medications for managing chronic pain.  
• Identifying modifiable psychosocial and behavioral mediators of pain. The intervention team includes 
questionnaires that help underscore for patients their coping approaches to pain and their emotional and 
attitudinal reactions to pain, factors that can either exacerbate or help reverse the chronic pain cycle. 
Exacerbating factors include fear of movement (measured using an abbreviated version of the Tampa Scale for 
Kinesiophobia38) and pain catastrophizing.39-41 Helpful factors include patients’ ability to cope with their pain; 
e.g., by increasing physical and social activities (measured using a modified version of the CHAMPS Physical 
Activity Questionnaire42,43), increasing levels of adaptive coping (as measured using an adaptation of the 
Coping Strategies Questionnaire44,45), and improving their perceived self-efficacy beliefs about their ability to 
cope with the consequences of chronic pain (as measured by the chronic pain self-efficacy scale).46  

 
 
For publication 
 
 
13 
Although data collected through the PPACT intake and follow-up evaluations (including scale scores from the 
instruments described above) are included in the 
treatment plan in the EHR and are extractable by 
research staff as a means to gauge patients’ success 
in the program, these are not primary research data 
for the study (see Figure 3 for a fictional example of 
a completed patient feedback form).  
These feedback forms are provided to participating 
patients at the third intake visit and attached to an 
encounter in the EHR for PCP and other KP provider 
access. Patients are asked to complete the 
evaluation utilized to produce and update these 
feedback forms upon entry into the program and  
brief meetings are held with patients’ PCPs to review 
the information. This allows the PCP and nurse care 
manager or behavioral health specialist to discuss 
areas of strength for the patient, areas of need 
targeted by the program, and strategic opportunities 
for the PCP and the program to partner in delivering 
a consistent message to the patient. Following this 
collaboration between intervention team and PCP, 
the PCP is encouraged to hold a scheduled telephone 
appointment with the participating patient to 
commend their participation in the program, recognize their identified strengths, and encourage their work on 
targeted opportunities for enhancement.  
This outreach by the PCP serves to highlight for the patient the collaboration between the program and PCP and 
demonstrates the coordination occurring among the patient’s overall team. The PCP is provided with an optional 
script template to use during this call. This script template and proactive nature of the PCP outreach is designed to 
promote enhanced collaboration between the PCP and patient.  
4.1.2. Schedule and Delivery 
Each patient in the intervention arm is scheduled for three 60-minute intake evaluation visits in the first month:  
1) Intake #1 – behavioral specialist or nurse care manager “new” visit 
2) Intake #2 – PT evaluation 
3) Intake #3 – behavioral specialist or nurse care manager: 60 minute “follow-up” visit 
The pharmacist completes a medication chart review after the in-person medication review done during intake visit 
#1 or 3 and charted by the behavioral specialist or nurse care manager. 
PT visits are also schedule mid-treatment to allow the PT to refine physical activity goals and to help patients utilize 
more advanced portions of the adapted movement yoga DVD (e.g., floor-based exercises) if warranted. Post-
treatment assessment/maintenance planning with the nurse care manager or behavioral specialist is to occur shortly 
after the last group session (approximately four months after enrollment). 
Figure 3. Example Completed Patient Feedback Form 
 

 
 
For publication 
 
 
14 
4.2. Core Groups for Coping Skills Training / Yoga-Based Adapted Movement Practice 
 
4.2.1. Overview of Goals and Strategies 
Group coaching sessions. The group coaching sessions are based upon the approach utilized in the study team’s 
previous studies of persistent pain47-56 and are designed to modify psychosocial variables this team and others have 
found to be related to pain and functional impairment.39,40,57-62 The variables targeted for change are (1) self-efficacy 
for pain and pain catastrophizing, (2) fear of movement, and (3) physical deconditioning. The group coaching portion 
of the intervention is designed to (1) enhance patients' self-efficacy in using coping skills to control pain, (2) decrease 
maladaptive pain catastrophizing, (3) decrease fear of movement, and (4) increase social and physical activities. The 
group sessions occur every week over 3 months for a total of 12 groups. The groups are co-led by a behavioral 
specialist and/or nurse care manager, with consultation from a PT to support the adapted movement activities and 
physical activity-relevant portions of the intervention.  
Coping skills training modules. Table 1 provides an overview of our coping skills modules, which are summarized 
below. 
• 
Session 1: Understanding pain/pain education and role of 
pain coping skills. Simple diagrams, including the 
neuromatrix and persistent pain cycle, are used to illustrate 
the pain cycle along with the role of the brain and other 
parts of the central nervous system in influencing the pain 
experience. The group explores pain’s effect on patients’ 
activities, feelings, and thoughts and how these changes 
similarly impact the pain they experience. The menu of 
coping skills modules is discussed, as well as the fact that 
these skills can be used not only for managing pain, but 
also for managing stressors related to pain. Patients are 
taught how to use a brief relaxation method (progressive 
muscle relaxation) that enables them to apply relaxation 
during daily activities that may increase their pain (e.g., 
walking, transferring from one position to another, 
prolonged sitting). 
• 
Session 2: Applying progressive muscle relaxation (PMR) 
and adaptation model. Using the information presented in 
Session 1, experiential activities encourage the group to 
envision how application of the program’s coping skills can 
change their pain, stress, and adaptation to challenging 
situations. Time is spent breaking down the PMR activity to 
promote a successful experience of this important skill and an overall understanding of how this directly 
impacts the perception of pain and stress in the brain. 
• 
Session 3: Activity-rest cycle. Patients are taught to use a quota system to pace their activities and increase 
activity level. The quota system involves targeting a daily activity that the patient tends to overdo and 
learning to split this activity into periods of moderate activity (e.g., 10 minutes walking) followed by limited 
rest (e.g., 5 minutes rest). The patient will build up the activity quota over time. A range of activity options 
are discussed, along with benefits of gradually increasing activity. Barriers and obstacles to using this quota 
system are identified and solutions for overcoming them are formulated. 
• 
Session 4: Pleasant activity scheduling. Pleasant activity scheduling is used to help patients identify and 
incorporate a variety of enjoyable and realistic activities in their day-to-day life that help them overcome the 
Table 1. Coping Skills Training 
 

 
 
For publication 
 
 
15 
deactivation common for pain patients and to address mood-related impairments common among patients 
with chronic pain. 
• 
Session 5: Relaxation mini-practices. Patients are taught to use, and then practice as a group, these brief 
relaxation techniques that are designed for use in the midst of various daily activities. These mini-practices 
provide an alternative to longer relaxation methods, such as the full PMR, but still provide the mental and 
physiological benefits necessary to overcome instances of pain, tension, and stress. 
• 
Session 6: Pleasant imagery. Patients are assisted in identifying an imaginary, personal scene and are then 
guided through pleasant imagery sessions that focus attention on pleasant experiences in the midst of pain, 
stress, or negative thoughts. The group then strategizes about building these imagery sessions into their day 
to promote relaxation. 
• 
Session 7: Emotional regulation: leaning in. Mood modulation skills, mindfulness, and the role of acceptance 
are taught and practiced to assist patients in working with strong emotions.63  
• 
Session 8: Emotional regulation: leaning out. In working with patients to counterbalance leaning into and 
away from challenging emotions, distraction techniques using physical or auditory stimuli are discussed and 
practiced as helpful tools in managing pain.63  
• 
Session 9: Cognitive restructuring. Cognitive restructuring is used to help patients recognize overly negative 
thoughts that occur in response to pain. Effects of such thoughts on feelings and behaviors are discussed. 
• 
Session 10: Use of calming self-statements. Patients develop alternative, calming/coping thoughts and self-
statements that are more helpful/useful in coping with pain. 
• 
Session 11: Problem-solving/reinforcing the application of learned skills. Following patient-stated reviews of 
the coping skills used throughout the program, the group works through several problem-solving scenarios to 
gain experience applying learned coping skills in the context of challenges faced. 
• 
Session 12: Relapse prevention and maintenance enhancement training. Patients are taught strategies to 
enhance maintenance of learned coping skills. In order to pinpoint situational factors affecting maintenance, 
each patient is taught to identify high-risk situations that are likely to interfere with coping efforts. A 
rationale for anticipating and coping with setbacks is discussed. Cognitive strategies for recognizing early 
warning signs of pain and symptom flares and coping with setbacks are emphasized.  
Adapted movement component of group. During the adapted movement component of the group, patients are 
instructed in yoga-based movement (stretching and strengthening). This approach to practice utilizes the “Relax into 
Yoga” DVD (based on  the Yoga of Awareness research trials64-67) which offers a gentle  yoga practice tailored to 
encourage patients to participate daily in gentle and accessible movements with the intention that these skills will 
begin to generalize to everyday activities requiring physical movement. In-session, yoga-based adapted movement is 
limited to seated and supported standing poses because the degree of deconditioning expected for this target 
population suggests that these practices are best suited to their current functional limitations. The “Relax Into Yoga” 
DVD  that is used in group sessions and given to patients to support them in adopting a regular stretching/yoga 
practice outside of class does contain floor-based routines. Participants able to get up and down from the floor may 
augment their practice with the use of these routines and work with the PPACT PT to ensure that they know and 
have practiced safe ways of getting down to and up from the floor. 
 
4.2.2. Schedule and Delivery 
Following the completion of the Comprehensive Intake Evaluation for each patient in a given cluster (estimated to 
take 4 weeks), all patients in the cluster begin the Core Group Series. This series of group sessions consists of one 2-
hour session per week for a total of 12 weeks. Patients in a given cluster attend these 12 group sessions together. 
Group sessions are led by the behavioral specialist and/or nurse care manager, with consultation from the 
Intervention Team Pharmacist and PT. These groups are held at the patients’ primary care clinic or clinic hub.  

 
 
For publication 
 
 
16 
4.3. Individualized Care Management 
 
4.3.1. Overview of Goals and Strategies 
Incorporating interim care management contacts. The nurse care manager or behavioral specialist also works with 
individual participants in person or by telephone on an as-needed basis. Although the formal individual evaluation 
and group sessions constitute a complete dose of the PPACT intervention, the study team recognizes there are 
instances when individuals are unavailable for group sessions and brief in-person or telephone sessions are indicated. 
Such contacts are focused on brief review of coping skills strategies, monitoring participants’ progress in meeting 
their individualized goals, and helping address barriers or obstacles to meeting these goals. Such calls last no more 
than 10-15 minutes and are an efficient means of ensuring that any difficulties implementing planned behavioral 
changes are addressed quickly. Importantly, the interventionists are trained to help patients initiate their own self-
care and problem-solving, rather than attempting to solve the difficulties for the patient. This is consistent with a 
motivational enhancement approach that the team has used successfully in other interventions.68-71  
4.4. Pharmacist Chart Review 
During the intake process, a pharmacist conducts a chart review targeting the following areas of the patient’s current 
medication regimen: (1) drug therapy gaps or redundancies; (2) drug–drug or drug–disease interactions; (3) adequacy 
of current doses; (4) adherence concerns (based on participant’s response to intake questions about pattern and 
frequency of medication use); and (5) adverse events. The patient’s medication history is also reviewed in order to 
assess the success of previously tried medications and adequacy of those trials. The outcome of this review is to 
advise the referring PCP on potential adjustments to the patients’ opioid treatment approach, provide feedback on 
other current pain medications (and possibly psychotropic medications) being prescribed to help the PCP identify 
therapeutic alternatives to opioid treatment that may best meet the patients’ needs, and provide information that 
can assist the treating PCP in beginning to taper the patient’s opioid medication. 
4.5. Physical Therapy Consultation 
During the intake process, patients meet with the PPACT PT to assess current functioning and identify movement 
adaptations that will help them most realistically begin to increase activity and thus most fully participate in the 
group coaching sessions. The PT evaluation targets the following areas: (1) history and physical exam, which inform 
the remaining evaluation components; (2) education regarding biomechanics to assist patients’ movement during 
day-to-day activities; (3) development of a plan for graduated aerobic physical activity; and (4) modifications 
necessary for the yoga-based adapted movement component of program.  
Follow-up evaluations with the PT occur at mid-program and focus on extending patients’ progress with graduated 
aerobic physical activity and yoga-based adapted movement. Progress meeting patients’ goals is evaluated, with a 
focus on helping them reach the next step toward those goals. The PT works with patients who are ready to progress 
to more advanced forms of the yoga-based adapted movement in order to review body mechanics that ensure safe 
use of the DVD guided poses.  
5. MEASUREMENTS AND EVALUATIONS 
5.2. Primary and Secondary Patient Outcome Measures 
This section includes a description of primary and secondary outcomes with appropriate supportive evidence, 
justification, and validation. Primary and secondary outcomes are measured at the patient level unless otherwise 
indicated. Table 3 summarizes primary and secondary outcome measures for the study, their source, and designated 
analytic purpose.  
 
5.2.1. Pain Severity and Functioning 
The primary outcome measure for the study is the PEGS,1 a psychometrically validated 4-item version of the short 
form of the Brief Pain Inventory (BPI-SF).72 The 4-item PEGS asks patients to report on their average pain severity as 
well as report on pain-related impairments in functioning in key life domains (general activity, enjoyment of life, and 

 
 
For publication 
 
 
17 
quality of sleep) on an 11-point Likert scale [0  to 10]. The 3-item PEG has been widely adopted for clinical pain 
assessment, epidemiological studies, and studies of pain treatment effectiveness72,73 and found to be more 
acceptable for use by PCPs and their support staff in the busy everyday clinical practice setting. We used the 4-item 
version as PCPs in the pilot were interested in to what extent patients’ pain was interfering with their sleep..  
Distribution of PEGS and PEGS slope. The PEGS questions are measured on a 11-point Likert scale. Question 1 
measures pain severity, while the remaining three PEGS questions measure the impact of pain on functioning 
(general activity, sleep, and enjoyment of life). The sum of these four questions constitutes the study’s primary 
measure of pain (PEGS-overall), while the pain severity item and the sum of three items measuring the impact of pain 
on functioning form subscales of severity (PEGS-severity) and impact (PEGS-impact). Based on data for KPNW 
members, the distributions of these scales are unimodal, centered at about the midpoint of the scale, and reasonably 
symmetric. The primary outcome variable for PPACT will be the slope of PEGS-overall on time (measured over the 12 
months following study enrollment). Table 2 provides summary information for these measures for KPNW patients. 
The distributions for all three slopes are symmetrically distributed about 0. We also used these slopes to calculate 
intra-class correlation (ICC) estimates using the patient’s paneled PCP as the cluster-level variable (mimicking the 
planned cluster structure for the main study). The resulting ICC was 0.0013. 
 
5.2.2. Use of Opioids and Other Medications 
An important secondary outcome for this study is the level of opioid medication used by participants as measured by 
their daily morphine equivalents of short-acting (U.S. Drug Enforcement Agency Schedule II or Non-Schedule II) or 
long-acting opioids. Morphine equivalents per dispensing (MEDs) is calculated by first multiplying the quantity 
dispensed times the milligram strength per dosage unit dispensed, times the opioid-specific morphine equivalents 
conversion factor.74,75 Next, total daily morphine equivalents are calculated by dividing the total morphine 
equivalents per dispensing by the days’ supply dispensed. Finally, daily morphine equivalents are applied across the 
corresponding days. If an individual had opioids available from multiple dispensings on the same day, morphine 
equivalents are summed for that day. This method of calculating MED has been widely used and applied to EHR data. 
181;182  
Benzodiazepine and opioid polydrug use is common and of increasing concern to our clinical delivery systems 
because of potential adverse events. Research suggests that benzodiazepines and opioids alter the pharmacokinetic 
effects of one another and that benzodiazepines may increase the rewarding and reinforcing effects of opioids, 
thereby placing patients at an increased risk for abuse.76-78 Because our health plans have prioritized reducing 
benzodiazepine and opioid polydrug prescribing and identifying appropriate nonpharmacotherapy treatment to 
support patients with such use, receipt of benzodiazepines is a secondary outcome we will track in the study.  
 
5.2.3. Utilization of Health Care Services 
Planned secondary outcome analyses include examination of the utilization of health care services of specific 
relevance for the PPACT target patient population that are hypothesized to be reduced for those randomized to the 
intervention when compared to their usual care counterparts. These utilization variables include both aggregated 
and disaggregated primary care contacts (outpatient visits, e-mail contacts, telephone contacts), use of specialty pain 
services (including physiatry, pain medicine, physical therapy, and occupational therapy services), inpatient services 
related to the participant’s pain condition (e.g., surgeries, implementation of pain-related devices),  and overall 
outpatient utilization. Utilization of other health services for which directional hypotheses are not indicated but for 
which data will be collected include the receipt of acupuncture or chiropractic care for pain reimbursed by the health 
plan. Resource and time permitting, analyses will be conducted with these additional secondary health service 
utilization variables. Validation of these utilization variables is reviewed in section 7.1 above. 
 
5.2.4. Other Secondary Outcomes 
While patient satisfaction information is routinely collected by the health plan in each of the KP regions participating 
in PPACT, each region uses its own distinct survey. That information is not sampled consistently and frequently 
among our target population, nor is it necessarily available in a manner that can be linked to a KP member’s record, 

 
 
For publication 
 
 
18 
given the anonymous format used to collect the information in both KPGA and KPH. Accordingly, we worked with our 
regional stakeholders to identify the critical patient satisfaction questions that they believed to be of importance in 
evaluating the intervention. These included two questions assessing patients’ satisfaction with their primary care 
services as well as their satisfaction with overall pain-related services provided by the health plan. The questions 
assess satisfaction over the past three months on a 5-point Likert scale (“very dissatisfied” to “very satisfied”). 
Because satisfaction measures are not routinely collected in the EHR, these data will be collected directly by study 
personnel and stored apart from the patients’ EHR. The data will be collected twice: at the time the patient initially 
enrolls in the study and at 6 months after enrollment, when those randomized to the intervention condition will have 
completed the study intervention. 
 
5.2.5. Clinical Covariates 
Patient-level clinical covariates that are extracted from either the EHR or other existing clinical information systems 
include demographics (age, gender, race/ethnicity) and diagnostic variables that are important in categorizing the 
patient population, including the presence of concomitant psychiatric diagnoses, evidence of substance use disorder 
history, and number of pain disorder conditions. These variables are be extracted from the EHR for the six months 
preceding study enrollment for each participant to better characterize the patient as he/she enters the study.  
5.3. Quantitative Data Collection Schedule 
Table 2 lists quantitative data collection measures and procedures and indicates the schedule for data collection. 
Table 2. PPACT Outcome Variables  
Measure                                                                       Source 
Schedule of Assessment 
Up to 12 months 
preceding patient 
enrollment 
Study Month 
0 
3 
6 
9 
12 
Patient-Reported Outcomes 
PEGS 
Primary outcome 
Study 
survey 
 
 
 
 
 
 
Roland Morris Disability 
Questionnaire  
Secondary outcome 
Study 
survey 
 
 
 
 
 
 
Patient Satisfaction Survey 
Secondary outcome 
Study 
survey 
 
 
 
 
 
 
Medication-Related Outcomes 
Opioids dispensed 
 
Secondary outcome 
EHR 
 
Benzodiazepines dispensed  
Secondary outcome 
EHR 
 
Health Service Utilization 
Primary care utilization 
(outpatient visits, emails, 
telephone contacts and total)  
Secondary outcome 
EHR 
 
Emergency and urgent care 
services 
Secondary outcome 
EHR 
 
Use of specialty pain services 
(physiatry, pain clinic, physical 
and occupational therapy)  
Secondary outcome 
EHR 
 
Overall outpatient service 
utilization 
Secondary outcome 
EHR 
 
Inpatient services related to 
pain condition 
Secondary outcome 
EHR 
 
 

 
 
For publication 
 
 
19 
5.4. Supporting the Clinical Collection of Patient Reported Outcomes 
To ensure adequate PEGS data for primary outcome analyses, additional processes are in place to ensure quarterly 
PEGS data availability for all enrolled patients. This approach extends the current clinical data collection processes. 
Every quarter, a message is sent to PPACT patients asking them to complete the PEGS via the kp.org web-based 
patient health record system. One week after the message is sent to the PPACT patients, local analysts extract a data 
file of all patients’ PEGS data.  The lead analyst uploads the non-completers to the KP Message Center, an automated 
calling vendor service that the study uses to conduct the PEGS over the telephone. This service also allows for a 
message to be left; the patient can call back at a later date and still do the brief automated interview. Five days after 
the KP Message Center pushes out their automated phone calls, the results from those interviews is extracted and a 
list of PEGS non-completers is again generated and uploaded into the tracking system. In the last step of this three-
phase process, a medical assistant will call the non-completers to attempt to conduct the brief interview with a live 
person. This multi-step process takes advantage of what data is already captured via the individual regions’ clinical 
processes while still trying to get as much data as possible in a cost-effective manner. 
5.5. Process Evaluation 
The PPACT process evaluation assesses fidelity of intervention delivery (the extent to which the intervention is 
delivered as intended), the intervention dose (how much of the intended intervention is delivered), and the reach to 
the groups targeted by the intervention (the proportion of intended recipients who actually participate in an 
intervention) using the RE-AIM framework as a guide.79,80  
For the study’s formative evaluation framework, we use PRISM,81 created to complement RE-AIM and focused on 
delineating criteria for successful implementation of interventions in health systems. Further, the structure, staffing, 
and analysis of formative evaluation data is guided by the Rapid Assessment Process (RAP),82,83 which employs 
ethnographic assessment by teams to gather and analyze information quickly to build an evolving understanding of 
conditions related to a planned or existing intervention. Data used for the process evaluation include: journal entries 
compiled by the study team (to document the conversations, current practices, and PPACT-related concerns that 
arise in the course of their interactions with stakeholders, project staff and teams) as well as patient surveys and 
telephone interviews with patients, clinicians and operational leaders. As part of RAP, the qualitative team meets 
regularly to review data collection and incremental data analyses to compile an emerging picture of the progress of 
the intervention, and the results of these analyses become part of debriefing meetings and progress reports to the 
larger research team.  
Reach is an individual-level measure reflecting the percentage and characteristics of persons who receive or are 
affected by a program—in this case, patients and PCPs. For patients, the project uses EHR data to examine: 
1. The percentage of patients excluded from the trial and the rationale for exclusion (diagnostic criteria, patient 
availability, level of pain) 
2. The percentage of patients who participate in the program based on the denominator of all patients who were 
approached for participation in each health plan, as well as all potentially eligible patients in the health plan 
regardless of whether or not they were approached for participation 
3. The characteristics of participating patients compared to non-participating patients in the health plan (both those 
refusing participation, and those never approached for recruitment) 
To describe the reach as it applies to participating PCPs, the project uses health plan administrative data to examine: 
1. The percentage of PCPs who participate in the program based on the denominator of all PCPs approached for 
participation in the health plan, as well as all potentially eligible PCPs in the health plan regardless of whether or 
not they were approach for participation. 
2. The characteristics of participating PCPs compared to those of non-participating PCPs (both those declining 
participation as well as those never approached for participation).  

 
 
For publication 
 
 
20 
3. In addition to the quantitative data described above, the qualitative data collected as part of the formative 
evaluation is critical for understanding the reach and recruitment findings. 
Effectiveness comprises individual-level measures focused on the impact of the intervention on important outcomes. 
This includes the following: 
1. Broader outcomes of importance include patient satisfaction, shifts in patient utilization of health services 
and medication use, and overall intervention cost and potential cost offset associated with the program. 
2. Resource and time permitting, the team may conduct exploratory analyses examining the robustness of the 
intervention across patient subgroups (e.g., gender, age, ethnicities, pain type(s), comorbid conditions). 
3. Short-term attrition in the intervention will be examined as well as differential rates by patient characteristics 
such as those described above. 
In addition to the quantitative data described above, the qualitative data collected as part of the formative 
evaluation will better allow the study team to best understand the reach and recruitment findings. 
Adoption. No primary care clinics in any of the participating health plan regions have declined to participate, nor has 
the study team identified conditions that would restrict a given clinic from participating. However, the team will 
continue to monitor adoption as the intervention is more broadly rolled out in each of the regions, identifying clinics 
who do not participate and the reason for non-participation. The study team will also compare the characteristics 
(e.g., size, location type, demographics of patients served) of those clinics who participate compared to those who 
don’t.  
Implementation is assessed both at the individual and organization level. Each is described in turn below. 
Individual implementation adherence. Individual implementation measures the adherence of patients to the 
intended intervention level as measured through attendance at intervention-related sessions (assessment intake 
evaluations and reassessments, group sessions, scheduled telephone contacts) and completion of intended home 
practice. As each intervention visit is scheduled using a PPACT program-specific identity code and visit type, this can 
be easily extracted from the EHR to evaluate degree of implementation.  
Organization implementation adherence. Organization implementation measures how closely the intervention staff 
follow the intended intervention program as well as the consistency of the program over time. To examine 
intervention staff’s adherence to the intervention protocol, all intervention individual and group sessions are digitally 
recorded, and sections of these tapes are reviewed in the supervision sessions (see section 8.1) to ensure that study 
procedures are closely followed. Remedial training is provided for any clinicians who deviate from the established 
protocol. Treatment adherence and therapist competence ratings are obtained as part of the supervision process. 
Treatment adherence refers to the extent to which a therapist uses the interventions prescribed by the protocol. 
Protocol adherence criteria are used for each session, with satisfactory adherence defined as 90% or more of the 
maximum possible score on the adherence rating scale. Ratings of therapists' competence in delivering the 
interventions is used to evaluate digital recordings of the sessions reviewed in supervision sessions.84  
Maintenance is measured at the organization level, evaluating the extent to which the intervention program 
becomes part of the routine organizational practices and policies in a given primary care clinic and across 
participating regions. The team will interview operational leaders in each region regarding the sustainability of the 
intervention and fit with organizational priorities.  

 
 
For publication 
 
 
21 
6. STUDY INTERVENTION STAFF TRAINING AND SUPERVISION 
All core PPACT interventionists (behavioral specialists, nurse care managers, physical therapists, and pharmacists) 
receive training prior to conducting treatment sessions with study participants. The initial training consists of a 3-day 
didactic and experiential course conducted by Drs. Keefe, DeBar, and Benes with participation from the KPNW PT 
(Gabriel), and pharmacist (Thorsness).  
Interventionists are provided with a detailed outline of the intake and reassessment process as well as detailed 
outlines for each group treatment session, and the treatment strategies taught through didactic instruction, taped 
illustrations of techniques from model cases, and role-play of common scenarios. All instruction sessions are 
videotaped for reference and/or education of new interventionists and retraining as need. 
Procedures to ensure consistency of treatment. To ensure that the interventionists consistently follow the 
appropriate treatment protocol, (1) the interventionists follow a detailed intervention manual, (2) telephone-based 
supervision sessions are conducted with all interventionists, (3) each treatment session is audiotaped to provide 
opportunities for review during the weekly supervision meetings, where study investigators give feedback on 
interventionists’ performance, and (4) ratings of treatment adherence are conducted. Protocol adherence criteria 
have been developed for each session with satisfactory adherence defined as 90% or more of the maximum possible 
score on the adherence rating scale. Ratings of interventionists' competence in delivering the intervention85 are used 
to evaluate 10% of the sessions. Sessions to be evaluated are randomly selected. However, the intensity of 
supervision will be decreased over the course of the trial as PPACT providers gain more experience delivering the 
intervention mimicking the way supervision is often provided in everyday care settings. We anticipate supervision 
occurring weekly for the first four months, bi-weekly for the next four months, and monthly thereafter. In addition, 
we have planned for annual “booster” training to ensure that any new staff are fully trained and to refresh skills 
among interventionists. We considered carefully the intensity and frequency of training and supervision that is most 
appropriate for this intervention. While some simple and most systems-level interventions in pragmatic trials may 
call for little in the way of specific training for implementing the protocol nor systematic review of practitioner 
efforts,86 the complicated problems of CP-LOT patients call for more systematic training and clinical supervision. This 
level of oversight is consistent with what regularly occurs in clinical settings. Importantly, many CP-LOT patients have 
had many treatment failures, due in part to the fragmented nature of their care; our approach is designed to address 
this with strong initial support and training for the interventionists working with these patients. While the increasing 
reliance in health care on less highly specialized and trained providers (e.g., nurse care managers, masters-level 
behavioral specialists) represents an exciting new direction for behavioral science in ensuring the sustainability of 
evidence-based interventions in everyday practice settings, it is imperative to determine the level of training and 
supervision necessary to ensure that the treatment is both effective for patients and feasible for providers.  

 
 
For publication 
 
 
22 
7. SAFETY REPORTING AND MONITORING  
7.1. Adverse Events and Serious Adverse Events 
NIH guidelines indicate that an adverse event is any untoward medical occurrence in a study participant. We have 
operationally defined a serious adverse event as a death or hospitalization during a patient’s participation in the trial. 
Because patients with chronic pain are anticipated to have fluctuating physical and emotional symptoms as part of 
the natural course of their condition (and would be expected to occur regardless of patients’ enrollment in the trial), 
such symptoms will not be systematically monitored as part of the trial. However, because the intervention is 
embedded directly in the primary care clinics and conducted in partnership with participating patients’ PCPs, in the 
event that a patient’s symptoms significantly worsen during the intervention, their PCP will be immediately 
contacted by a PPACT intervention team member and their PCP will work with the patient to identify and provide 
appropriate care. This is consistent with the standard of care provided at KP. 
7.2. Data Safety Monitoring Plan (DSMP) 
As the intervention aspect of this study is based on best available evidence and constitutes a reorganization of 
currently available clinical services, we do not foresee any new risks above and beyond standard clinical care. 
Nonetheless, patients with complex chronic pain conditions are vulnerable to clinical outcomes that constitute 
serious adverse events, and while such events are unlikely to occur as a consequence of study participation, we are 
obliged to investigate and respond appropriately to these events. Consequently, we will implement a safety 
monitoring plan based on those used successfully in other, similar interventions. This plan involves EHR monitoring of 
all study participants every 6 months to assess the rate of death and hospitalization. Given the minimal risk posed by 
the study, however, we do not propose formal safety stopping rules and hence do not plan to conduct formal 
statistical analyses comparing these rates between treatment arms. In addition to calculating overall rates of 
occurrence of these events, an independent KP clinician in each region will conduct chart reviews of all deaths of 
intervention participants to identify any connection to study participation. All potential study-related deaths will be 
promptly reported to each of our IRBs and to our NINDS Project Officer. Because the number of hospitalizations in 
this group may be high and the study poses only minimal risk, we do not plan to chart-review hospitalizations as a 
matter of course. However, if our reports suggest a possible increased risk of hospitalizations associated with the 
intervention, we will work with our monitoring groups to develop a plan to do chart reviews on all or a subset of 
hospitalizations. All of this information will be reviewed by experienced clinicians on the investigative team and by an 
NINDS-appointed independent monitor. 

 
 
For publication 
 
 
23 
8. STATISTICAL METHODS 
8.1. Sample Size and Power 
We calculated power using the PASS software program, which applies the formulas from Donner and Klar87 and 
assumes a simple ANOVA framework with no covariate adjustment. Based on direct estimates of the intraclass 
correlation coefficient (ICC) of PEG slopes clustered within provider groups that we derived from historical data from 
the KPNW region, we estimate the ICC to be .0013. In the calculations presented below we conservatively use ICCs of 
.002, .005 and .01.  From the literature, we also expect standardized effect sizes to range from .022 to 0.54,88-92 and 
therefore conservatively calculated power for effect sizes ranging from .16 to .24 standard deviation units (SDUs).  
Our initial study design nominally called for 120 total clusters of 10 patients each. In practice, however, we 
randomized 106 PCP clusters, and cluster sizes varied from 3-13, with a mean of 8 and interquartile range of 6-10. As 
seen in Table 3, we constructed our power calculations to accommodate the possibility of such smaller cluster sizes. 
With the likely ICC of .002 and 106 clusters with an average cluster size of 8, we should have 93% to detect a 
standard effect size of 0.24 and 88% power to detect an effect size of 0.22. 
Table 3.  Power for detecting given effect sizes under various design scenarios 
  
Number of 
Clusters 
  
Patients 
per Cluster 
ICC=.002 
ICC=.005 
ICC=.01 
Effect Size (in SDUs) 
Effect Size (in SDUs) 
Effect Size (in SDUs) 
.16 
.18 
.20 
.22 
.24 
.16 
.18 
.20 
.22 
.24 
.16 
.18 
.20 
.22 
.24 
120 
8 
68% 
78% 
86% 
92% 
96% 
68% 
78% 
86% 
91% 
95% 
66% 
76% 
84% 
90% 
95% 
106 
8 
63% 
73% 
82% 
88% 
93% 
62% 
72% 
81% 
88% 
93% 
61% 
71% 
80% 
87% 
92% 
 
8.2. Randomization 
Given the lagged nature of the intervention rollout, even within a given clinic, it is necessary to randomize all 
providers in a given clinic at the outset of intervention activities in that clinic. However, the assignments are not 
revealed to either patients or providers until all of the patients for a given PCP have been recruited. To preserve 
blinding, recruitment staff are totally distinct from the intervention staff and remain blinded during the entire course 
of the study as they collect follow-up assessments. 
8.3. Dropout and Withdrawal 
Primary Care Provider. If an enrolled PCP leaves their KP practice, changes clinics, or asks to withdraw from the study, 
every effort will be made to collect process data from the PCP before their departure and to document the reason for 
leaving. We will continue to collect data on the provider’s enrolled patients and will analyze them according to the 
group to which they were originally assigned (i.e., per intention to treat, ITT).  
Patients. If an enrolled patient discontinues KP coverage or changes providers during the study, we will continue to 
collect data on them and will analyze them according to the group to which they were originally assigned. Such 
patients who are in clusters randomized to the intervention arm of the study will continue to be offered individual or 
telephone contact with the study interventionists throughout the time that their enrolled cohort is in the active 
phase of the intervention so as to provide as much therapeutic benefit to these patients as is possible. 
We will document the extent to which either of the above events occurs and will compare the frequency of such 
occurrences between intervention and control participants.  

 
 
For publication 
 
 
24 
8.4. Quantitative Data Methods and Analysis 
The following analytic framework will be used for our primary and secondary outcome analyses. All analyses will be 
performed using an intention-to-treat framework, and tests will be evaluated at a two-tailed alpha level of .05. 
Because of the nested structure of the data (observations nested within patients nested within provider groups), we 
will use a three-level hierarchical linear model (HLM: mixed models, random effects regression, and multilevel 
models) to account for the intraclass correlation that results from the nesting.93-95 An advantage of multilevel 
modeling is that unlike repeated measures analysis of variance, it does not require the same number of data points 
from all patients, thus all patients with at least a baseline measure can be included in the analysis. The first level of 
the model will include time as a predictor (five timepoints, representing the number of weeks since baseline), thus 
modeling the within-person trajectories across time. We will use two parameters (linear and quadratic slope) to 
characterize change across time, with linear slope capturing initial rate of change and quadratic slope reflecting the 
degree to which the change slowed (or increased) over time. The second level of the model may include patient-level 
covariates as predictors of the baseline PEGS score and the slope parameters for time. Randomization is expected to 
balance most potential patient-level covariates, however, in the case of remaining residual imbalances, covariates 
will be included in the model. These may include variables such as substance use problems/history, number of pain 
conditions and type, and other comorbid medical and mental health conditions. The third level will include a dummy 
variable for arm as the predictor of the patient-level intercept and slope parameters for time.  A significant 
coefficient for arm on the slope(s) of time would indicate that there are different trajectories across time for each 
arm. A pattern in which those in PPACT demonstrate a greater reduction in pain impact over time than those in the 
usual care arm would provide support for the effectiveness of PPACT. We will use the same analytical framework for 
the RMDQ. Because there are only two timepoints available for satisfaction, we will be limited to a two-level model 
of the difference scores between 6 months and baseline of patients nested within provider groups.  
Level-1 Model 
    Ytij = π0ij + π1ij*(LIN_TIMEtij) + π2ij*(QUAD_TIMEtij) + etij 
 
Level-2 Model 
    π0ij = β00j + β01j*(Patient_covariatesij) + r0ij 
    π1ij = β10j + β11j*(Patient_covariatesij) + r1ij 
    π2ij = β20j + β21j*(Patient_covariatesij) + r2ij 
Level-3 Model 
    β00j = γ000 + γ001(ARMj) + u00j 
    β01j = γ010  
    β10j = γ100 + γ101(ARMj) + u10j 
    β11j = γ110  
    β20j = γ200 + γ201(ARMj) + u20j 
    β21j = γ210  
where: Ytij is the outcome for person i under provider j at time t, π are level 1 (occasion) regression coefficients, etij is 
the random error associated with person i under provider cluster j at time t, Lin_Time is the number of weeks since 
baseline and Quad_Time is the number of weeks since baseline squared, β are level 2 (patient) regression 
coefficients, r are level 2 random effects, γ are level 3 (provider cluster) regression coefficients, u are level 3 random 
effects, and arm is an indicator variable.   
Hierarchical generalized linear modeling (HGLM) will be used to test the secondary outcomes: opioids dispensed, 
pain treatment and diagnostic procedures, emergency/urgent care visits, primary care visits, and specialty care visits 
over 12 months. These will be two-level models, with patients forming the first level of the model and clinics the 
second level. Patient-level covariates will be included in the first level, and PPACT versus a usual-care dummy variable 
will form the second level of the model. This will allow us to test whether the secondary outcomes differ for the two 
groups, controlling for differences in patient characteristics. Because the utilization variables are likely to follow non-

 
 
For publication 
 
 
25 
normal distributions, we will use Poisson, Negative Binomial, or Gamma distributions as appropriate for the 
distribution of each secondary outcome variable. 
Level-1 Model 
    ηij = β0j + β1j*(Patient_covariatesij) 
Level-2 Model 
    β0j = γ00 + γ01*(ARMj) + u0j 
    β1j = γ10  
where: ηij is the outcome defined by the identity link (log) and distribution (gamma, Poisson, or Negative Binomial), β 
are level 1 (person) regression coefficients, γ are level 2 (provider cluster) regression coefficients, u is the level 2 
random effect for the level 1 intercept, and arm is an indicator variable.  
8.5. Economic Data Methods and Analysis 
In our economic analysis of the PPACT intervention, we will assess the resources and costs necessary to deliver the 
PPACT intervention in routine clinical practice, and the cost-effectiveness of the PPACT intervention compared with 
usual care. Costs will be reported at three levels: 1) costs related to the intervention delivery, 2) medical care costs 
related to pain control, and 3) the total cost of medical care. Intervention costs will be estimated using EHR data, 
supplemented with data collected directly from intervention team staff. A sampling of clinical visits and interviews 
with intervention delivery staff will be used to determine the time needed to deliver the intervention. We will also 
consider costs related to the administration of the PPACT intervention in practice, including project management, 
training, and additional team meetings.  
Using EHR data, we will aggregate medical care events related to pain control and total costs at meaningful levels in 
order to demonstrate how the intervention impacts medical care resource use, specifically inpatient stays, outpatient 
procedures, clinic visits, and pharmacy dispenses. We will identify medical care utilization events that are related to 
pain control and the intervention using ICD-9CM, ICD-10CM, and CPT codes. To facilitate costing, we will examine the 
number and type of health care encounters participants receive over the course of their 12-month participation in 
the study. We will capture inpatient stays by extracting the information in the discharge abstract, including ICD-9 and 
ICD-10 codes and procedures and length-of-stay information necessary to cost the event.  
Medical care utilization events will be analyzed using mixed effects Poisson regression analysis, with primary care 
provider cluster as a random effects factor and follow-up time as an offset variable. We will estimate quantities of 
medical care events (i.e., inpatient stays, outpatient procedures, clinic visits, and pharmacy dispenses) using separate 
regression models.  
 

 
 
For publication 
 
 
26 
9. LITERATURE CITED 
1. 
Krebs EE, Lorenz KA, Bair MJ, et al. Development and initial validation of the PEG, a three-item scale assessing 
pain intensity and interference. J Gen Intern Med. 2009;24(6):733-738. 
2. 
Jensen MP, Nielson WR, Kerns RD. Toward the development of a motivational model of pain self-
management. The Journal of Pain. 2003;4(9):477-492. 
3. 
Jensen MP, Nielson WR, Turner JA, Romano JM, Hill ML. Changes in readiness to self-manage pain are 
associated with improvement in multidisciplinary pain treatment and pain coping. Pain. 2004;111(1-2):84-95. 
4. 
Vong SK, Cheing GL, Chan F, So EM, Chan CC. Motivational enhancement therapy in addition to physical 
therapy improves motivational factors and treatment outcomes in people with low back pain: a randomized 
controlled trial. Archives of physical medicine and rehabilitation. 2011;92(2):176-183. 
5. 
Rollnick S, Miller WR, Butler C. Motivational interviewing in health care: helping patients change behavior. 
Guilford Press; 2008. 
6. 
Owen-Smith A, Mayhew M, Leo MC, et al. Automating Collection of Pain-Related Patient-Reported Outcomes 
to Enhance Clinical Care and Research. J Gen Intern Med. 2018;33(Suppl 1):31-37. 
7. 
Wagner EH. The Chronic Care Model. 2006; http://www.improvingchroniccare.org/. 
8. 
Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: 
translating evidence into action. Health affairs. 2001;20(6):64-78. 
9. 
Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the chronic care model in the new millennium. 
Health affairs. 2009;28(1):75-85. 
10. 
Wagner EH. Chronic disease management: what will it take to improve care for chronic illness? Effective 
clinical practice: ECP. 1998;1(1):2. 
11. 
Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. The Milbank Quarterly. 
1996:511-544. 
12. 
Dobscha SK, Corson K, Perrin NA, et al. Collaborative care for chronic pain in primary care: a cluster 
randomized trial. Jama. 2009;301(12):1242-1252. 
13. 
Kroenke K, Bair M, Damush T, et al. Stepped Care for Affective Disorders and Musculoskeletal Pain (SCAMP) 
study: design and practical implications of an intervention for comorbid pain and depression. General 
hospital psychiatry. 2007;29(6):506-517. 
14. 
Loeser JD, Turk DC. Multidisciplinary pain management. Paper presented at: Seminars in Neurosurgery2004. 
15. 
Turk DC. 2 Efficacy and Cost-Effectiveness Treatment for Chronic Pain: An Analysis and Evidence-Based 
Synthesis. Chronic pain management: Guidelines for multidisciplinary program development. 2007:15. 
16. 
Von Korff M, Moore JE, Lorig K, et al. A randomized trial of a lay person-led self-management group 
intervention for back pain patients in primary care. Spine. 1998;23(23):2608-2615. 
17. 
Chou R, Huffman LH. Nonpharmacologic therapies for acute and chronic low back pain: a review of the 
evidence for an American Pain Society/American College of Physicians clinical practice guideline. Annals of 
internal medicine. 2007;147(7):492-504. 
18. 
Chou R, Loeser JD, Owens DK, et al. Interventional therapies, surgery, and interdisciplinary rehabilitation for 
low back pain: an evidence-based clinical practice guideline from the American Pain Society. Spine. 
2009;34(10):1066-1077. 
19. 
Management ASoATFoCP. Practice guidelines for chronic pain management: an updated report by the 
American Society of Anesthesiologists Task Force on Chronic Pain Management and the American Society of 
Regional Anesthesia and Pain Medicine. Anesthesiology. 2010;112(4):810. 
20. 
Cleeland C, Ryan K. Pain assessment: global use of the Brief Pain Inventory. Annals, Academy of Medicine, 
Singapore. 1994. 
21. 
Keller S, Bann CM, Dodd SL, Schein J, Mendoza TR, Cleeland CS. Validity of the brief pain inventory for use in 
documenting the outcomes of patients with noncancer pain. The Clinical journal of pain. 2004;20(5):309-318. 
22. 
Roland M, Fairbank J. The Roland–Morris disability questionnaire and the Oswestry disability questionnaire. 
Spine. 2000;25(24):3115-3124. 

 
 
For publication 
 
 
27 
23. 
Vianin M. Psychometric properties and clinical usefulness of the Oswestry Disability Index. Journal of 
chiropractic medicine. 2008;7(4):161-163. 
24. 
Wittink H, Turk DC, Carr DB, Sukiennik A, Rogers W. Comparison of the redundancy, reliability, and 
responsiveness to change among SF-36, Oswestry Disability Index, and Multidimensional Pain Inventory. The 
Clinical journal of pain. 2004;20(3):133-142. 
25. 
Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current 
depression in the general population. Journal of affective disorders. 2009;114(1-3):163-173. 
26. 
Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the 
GAD-7. Archives of internal medicine. 2006;166(10):1092-1097. 
27. 
Kimerling R, Ouimette P, Prins A, et al. Brief report: Utility of a short screening scale for DSM-IV PTSD in 
primary care. Journal of general internal medicine. 2006;21(1):65-67. 
28. 
Ouimette P, Wade M, Prins A, Schohn M. Identifying PTSD in primary care: Comparison of the Primary Care-
PTSD screen (PC-PTSD) and the General Health Questionnaire-12 (GHQ). Journal of Anxiety disorders. 
2008;22(2):337-343. 
29. 
Felitti VJ, Anda RF, Nordenberg D, et al. Relationship of childhood abuse and household dysfunction to many 
of the leading causes of death in adults: The Adverse Childhood Experiences (ACE) Study. American journal of 
preventive medicine. 2019;56(6):774-786. 
30. 
Berner MM, Kriston L, Bentele M, Härter M. The alcohol use disorders identification test for detecting at-risk 
drinking: a systematic review and meta-analysis. Journal of studies on alcohol and drugs. 2007;68(3):461-473. 
31. 
Kriston L, Hölzel L, Weiser A-K, Berner MM, Härter M. Meta-analysis: are 3 questions enough to detect 
unhealthy alcohol use? Annals of internal medicine. 2008;149(12):879-888. 
32. 
Beaudreau SA, Spira AP, Stewart A, et al. Validation of the Pittsburgh Sleep Quality Index and the Epworth 
Sleepiness Scale in older black and white women. Sleep medicine. 2012;13(1):36-42. 
33. 
Hayes AL, Spilsbury JC, Patel SR. The Epworth score in African American populations. Journal of Clinical Sleep 
Medicine. 2009;5(04):344-348. 
34. 
Heaton K. A psychometric analysis of the Epworth Sleepiness Scale. Journal of nursing measurement. 
2007;15(3). 
35. 
Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. sleep. 
1991;14(6):540-545. 
36. 
Nguyen ATD, Baltzan MA, Small D, Wolkove N, Guillon S, Palayew M. Clinical reproducibility of the Epworth 
sleepiness scale. Journal of Clinical Sleep Medicine. 2006;2(02):170-174. 
37. 
Banta-Green CJ, Von Korff M, Sullivan MD, Merrill JO, Doyle SR, Saunders K. The prescribed opioids 
difficulties scale: a patient-centered assessment of problems and concerns. The Clinical journal of pain. 
2010;26(6):489. 
38. 
Shelby RA, Somers TJ, Keefe FJ, et al. Brief fear of movement scale for osteoarthritis. Arthritis care & 
research. 2012;64(6):862-871. 
39. 
Keefe FJ, Brown GK, Wallston KA, Caldwell DS. Coping with rheumatoid arthritis pain: catastrophizing as a 
maladaptive strategy. Pain. 1989;37(1):51-56. 
40. 
Keefe FJ, Lefebvre JC, Egert JR, Affleck G, Sullivan MJ, Caldwell DS. The relationship of gender to pain, pain 
behavior, and disability in osteoarthritis patients: the role of catastrophizing. Pain. 2000;87(3):325-334. 
41. 
Sullivan MJ, Bishop SR, Pivik J. The pain catastrophizing scale: development and validation. Psychological 
assessment. 1995;7(4):524. 
42. 
Mills KM, Stewart AL, King AC, et al. Factors associated with enrollment of older adults into a physical activity 
promotion program. Journal of Aging and Health. 1996;8(1):96-113. 
43. 
Sepsis P, Stewart A, McLellan B, et al. Seniors' ratings of the helpfulness of health promotion program 
features in starting and maintaining physical activity. Journal of Aging and Physical Activity. 1995;3(2):193-
207. 
44. 
Jensen MP, Keefe FJ, Lefebvre JC, Romano JM, Turner JA. One-and two-item measures of pain beliefs and 
coping strategies. Pain. 2003;104(3):453-469. 

 
 
For publication 
 
 
28 
45. 
Rosenstiel AK, Keefe FJ. The use of coping strategies in chronic low back pain patients: relationship to patient 
characteristics and current adjustment. Pain. 1983;17(1):33-44. 
46. 
Anderson KO, Dowds BN, Pelletz RE, Edwards WT, Peeters-Asdourian C. Development and initial validation of 
a scale to measure self-efficacy beliefs in patients with chronic pain. Pain. 1995;63(1):77-83. 
47. 
Emery CF, Keefe FJ, France CR, et al. Effects of a brief coping skills training intervention on nociceptive flexion 
reflex threshold in patients having osteoarthritic knee pain: a preliminary laboratory study of sex differences. 
Journal of pain and symptom management. 2006;31(3):262-269. 
48. 
Keefe FJ, Ahles TA, Sutton L, et al. Partner-guided cancer pain management at the end of life: a preliminary 
study. Journal of pain and symptom management. 2005;29(3):263-272. 
49. 
Keefe FJ, Blumenthal J, Baucom D, et al. Effects of spouse-assisted coping skills training and exercise training 
in patients with osteoarthritic knee pain: a randomized controlled study. Pain. 2004;110(3):539-549. 
50. 
Keefe FJ, Caldwell DS, Baucom D, et al. Spouse‐assisted coping skills training in the management of 
osteoarthritic knee pain. Arthritis & Rheumatism: Official Journal of the American College of Rheumatology. 
1996;9(4):279-291. 
51. 
Keefe FJ, Caldwell DS, Baucom D, et al. Spouse‐assisted coping skills training in the management of knee pain 
in osteoarthritis: Long‐term followup results. Arthritis Care & Research. 1999;12(2):101-111. 
52. 
Keefe FJ, Caldwell DS, Williams DA, et al. Pain coping skills training in the management of osteoarthritic knee 
pain: a comparative study. Behavior Therapy. 1990;21(1):49-62. 
53. 
Keefe FJ, Caldwell DS, Williams DA, et al. Pain coping skills training in the management of osteoarthritic knee 
pain-II: follow-up results. Behavior Therapy. 1990;21(4):435-447. 
54. 
Keefe FJ, Shelby RA, Somers TJ, et al. Effects of coping skills training and sertraline in patients with non-
cardiac chest pain: a randomized controlled study. PAIN®. 2011;152(4):730-741. 
55. 
Naylor MR, Helzer JE, Naud S, Keefe FJ. Automated telephone as an adjunct for the treatment of chronic 
pain: a pilot study. The Journal of Pain. 2002;3(6):429-438. 
56. 
Naylor MR, Naud S, Keefe FJ, Helzer JE. Therapeutic Interactive Voice Response (TIVR) to reduce analgesic 
medication use for chronic pain management. The Journal of Pain. 2010;11(12):1410-1419. 
57. 
Buckelew SP, Parker JC, Keefe FJ, et al. Self-efficacy and pain behavior among subjects with fibromyalgia. 
Pain. 1994;59(3):377-384. 
58. 
Geisser ME, Robinson ME, Keefe FJ, Weiner ML. Catastrophizing, depression and the sensory, affective and 
evaluative aspects of chronic pain. Pain. 1994;59(1):79-83. 
59. 
Keefe FJ, Kashikar-Zuck S, Robinson E, et al. Pain coping strategies that predict patients' and spouses' ratings 
of patients' self-efficacy. Pain. 1997;73(2):191-199. 
60. 
Keefe FJ, Lipkus I, Lefebvre JC, et al. The social context of gastrointestinal cancer pain: a preliminary study 
examining the relation of patient pain catastrophizing to patient perceptions of social support and caregiver 
stress and negative responses. PAIN®. 2003;103(1-2):151-156. 
61. 
Somers TJ, Keefe FJ, Carson JW, Pells JJ, LaCaille L. Pain catastrophizing in borderline morbidly obese and 
morbidly obese individuals with osteoarthritic knee pain. Pain Research and Management. 2008;13(5):401-
406. 
62. 
Sullivan MJ, Thorn B, Haythornthwaite JA, et al. Theoretical perspectives on the relation between 
catastrophizing and pain. The Clinical journal of pain. 2001;17(1):52-64. 
63. 
Turk DC, Meichenbaum D, Genest M. Pain and behavioral medicine: A cognitive-behavioral perspective. Vol 1: 
Guilford Press; 1983. 
64. 
Carson JW, Carson KM, Jones KD, Bennett RM, Wright CL, Mist SD. A pilot randomized controlled trial of the 
Yoga of Awareness program in the management of fibromyalgia. PAIN®. 2010;151(2):530-539. 
65. 
Carson JW, Carson KM, Porter LS, Keefe FJ, Seewaldt VL. Yoga of Awareness program for menopausal 
symptoms in breast cancer survivors: results from a randomized trial. Supportive care in cancer. 
2009;17(10):1301-1309. 
66. 
Carson JW, Carson KM, Porter LS, Keefe FJ, Shaw H, Miller JM. Yoga for women with metastatic breast 
cancer: results from a pilot study. Journal of pain and symptom management. 2007;33(3):331-341. 

 
 
For publication 
 
 
29 
67. 
Wren AA, Wright MA, Carson JW, Keefe FJ. Yoga for persistent pain: new findings and directions for an 
ancient practice. Pain. 2011;152(3):477. 
68. 
DeBar LL, Ritenbaugh C, Aickin M, et al. YOUTH: a health plan–based lifestyle intervention increases bone 
mineral density in adolescent girls. Archives of pediatrics & adolescent medicine. 2006;160(12):1269-1276. 
69. 
DeBar LL, Stevens VJ, Perrin N, et al. A primary care–based, multicomponent lifestyle intervention for 
overweight adolescent females. Pediatrics. 2012;129(3):e611-e620. 
70. 
DeBar LL, Striegel-Moore RH, Wilson GT, et al. Guided self-help treatment for recurrent binge eating: 
Replication and extension. Psychiatric Services. 2011;62(4):367-373. 
71. 
DeBar LL, Wilson GT, Yarborough BJ, et al. Cognitive behavioral treatment for recurrent binge eating in 
adolescent girls: A pilot trial. Cognitive and behavioral practice. 2013;20(2):147-161. 
72. 
Kroenke K, Theobald D, Wu J, Tu W, Krebs EE. Comparative responsiveness of pain measures in cancer 
patients. The Journal of Pain. 2012;13(8):764-772. 
73. 
Krebs EE, Bair MJ, Damush TM, Tu W, Wu J, Kroenke K. Comparative responsiveness of pain outcome 
measures among primary care patients with musculoskeletal pain. Medical care. 2010;48(11):1007. 
74. 
Boudreau D, Von Korff M, Rutter CM, et al. Trends in long‐term opioid therapy for chronic non‐cancer pain. 
Pharmacoepidemiology and drug safety. 2009;18(12):1166-1175. 
75. 
Von Korff M, Saunders K, Ray GT, et al. Defacto long-term opioid therapy for non-cancer pain. The Clinical 
journal of pain. 2008;24(6):521. 
76. 
Gudin JA, Mogali S, Jones JD, Comer SD. Risks, management, and monitoring of combination opioid, 
benzodiazepines, and/or alcohol use. Postgraduate medicine. 2013;125(4):115-130. 
77. 
Jones JD, Mogali S, Comer SD. Polydrug abuse: a review of opioid and benzodiazepine combination use. Drug 
and alcohol dependence. 2012;125(1-2):8-18. 
78. 
Webster LR. Considering the risks of benzodiazepines and opioids together. Blackwell Publishing Inc Malden, 
USA; 2010. 
79. 
RJ B-W, PJ C, CL T, BD L. The use of the eating disorder examination with children: a pilot study. Int J Eat 
Disord. 1996;19(4):391-397. 
80. 
Campbell CI, Weisner C, LeResche L, et al. Age and gender trends in long-term opioid analgesic use for 
noncancer pain. American journal of public health. 2010;100(12):2541-2547. 
81. 
Dunn KM, Saunders KW, Rutter CM, et al. Opioid prescriptions for chronic pain and overdose: a cohort study. 
Annals of internal medicine. 2010;152(2):85-92. 
82. 
Kuehn BM. Opioid prescriptions soar. Jama. 2007;297(3):249-251. 
83. 
Sullivan MD, Von Korff M, Banta-Green C, Merrill JO, Saunders K. Problems and concerns of patients receiving 
chronic opioid therapy for chronic non-cancer pain. PAIN®. 2010;149(2):345-353. 
84. 
Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the integrity of a psychotherapy protocol: assessment of 
adherence and competence. Journal of consulting and clinical psychology. 1993;61(4):620. 
85. 
Borkovec TD, Nau SD. Credibility of analogue therapy rationales. Journal of behavior therapy and 
experimental psychiatry. 1972;3(4):257-260. 
86. 
Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic–explanatory continuum indicator summary 
(PRECIS): a tool to help trial designers. Journal of clinical epidemiology. 2009;62(5):464-475. 
87. 
Von Korff M, Kolodny A, Deyo RA, Chou R. Long-term opioid therapy reconsidered. Annals of internal 
medicine. 2011;155(5):325-328. 
88. 
Dixon KE, Keefe FJ, Scipio CD, Perri LM, Abernethy AP. Psychological interventions for arthritis pain 
management in adults: a meta-analysis. Health Psychology. 2007;26(3):241. 
89. 
Goldenberg DL. Multidisciplinary modalities in the treatment of fibromyalgia. Journal of Clinical Psychiatry. 
2008;69(Suppl 2):30-34. 
90. 
Sarzi-Puttini P, Atzeni F, Salaffi F, Cazzola M, Benucci M, Mease PJ. Multidisciplinary approach to 
fibromyalgia: what is the teaching? Best Practice & Research Clinical Rheumatology. 2011;25(2):311-319. 
91. 
Scascighini L, Toma V, Dober-Spielmann S, Sprott H. Multidisciplinary treatment for chronic pain: a systematic 
review of interventions and outcomes. Rheumatology. 2008;47(5):670-678. 

 
 
For publication 
 
 
30 
92. 
Smith BH, Torrance N. Management of chronic pain in primary care. Current Opinion in Supportive and 
Palliative Care. 2011;5(2):137-142. 
93. 
Hox JJ. Multilevel analysis: techniques and applications. Routledge; 2010. 
94. 
Raudenbush SW, Bryk AS. Hierarchical linear models:  Applications and data analysis methods. 2nd ed. 
Thousand Oaks, CA: Sage Publications Inc; 2002. 
95. 
Snijders T, Bosker R. Multilevel analysis: An introduction to basic and advanced multilevel modeling. 
Thousand Oaks, CA: Sage; 1999.