Effective Screening for Pain Study
Phase: NA
Conditions: Pain, Pain Management
Sponsor: VA Office of Research and Development

 
 
  
  
  
  
  
  
  
Effective Screening for Pain Study  
NCT01816763  
July 2018  
  
  
 

 
   
 
Page 2 of 25  
  
  
  
  
Abstract  
 Objectives: The overall objective of our mixed method study is to evaluate a strategy to improve the quality and clinical 
usefulness of ‘the 5th Vital Sign’ in VA primary care. We will formatively evaluate experiences with and implementation 
of the enhanced pain screening approaches (patient self administered computer aided) including clinician perspectives 
on the use of pain screening to guide management. Our specific aims are to:   
Aim 1. Characterize primary care team and Veteran patient perspectives on enhanced pain screening approaches 
compared with usual pain screening re., clinical usefulness and how to use skills, roles, and tasks of multidisciplinary 
clinicians and consultants to optimize pain care. Interviews and focus groups will inform development of the screening 
interface. Aim 2. Evaluate in a three-arm RCT the effects of enhanced pain screening approaches using the PEG (Arm 1), 
the ‘NRS pain now (Arm 2), compared with DVPRS (Arm 3) followed by usual clinician-assessed pain screening, on the 
detection of patient pain as well as  feasibility, acceptability, and experiences with (e.g., pain missingness), patient-
reported tablet-based pain assessment using quantitative patient screening outcomes, and interviews with patients, 
clinic, and facility staff  Research Plan: At three sites (VA GLA, Minneapolis, and Portland) we will conduct interviews and 
focus groups with primary care staff about their experiences with pain screening and how to link screening to 
management in order to inform development of an integrated enhanced pain screening approach (using PEG and NRS 
now and DVPRS) on a touchscreen together with survey questions on risk for under-assessed or under-treated pain. The 
integrated protocol will be installed on a touchscreen tablet to use in a 3 arm RCT to evaluate the optimal approach to 
pain screening, along with an evaluation of the implementation experience.   
Methodology: Clinicians, clinic staff, and facility staff will be interviewed and multidisciplinary focus groups conducted at 
3 sites to characterize the range of themes related to staff perspectives on patient reported versus clinician documented 
pain. Development of the enhanced pain screening protocol will account for clinician perspectives informed by 
qualitative input, as well as subsequent patient usability testing, and then the protocol integrated into a tablet interface 
will be subjected to an RCT that will be placed in primary care for selected Veterans to compare with staff administered 
pain screening. Veterans will be randomly allocated in a 1:1:1 ratio to the three arms and analyses will evaluate a variety 
of quantitative and qualitative outcomes. Veteran responses to the brief tablet based pain screening protocol and 
additional tablet based items will be linked to Veterans’ health data to evaluate feasibility and the association with social 
and clinical vulnerability (e.g., perhaps minority or cognitively impaired Veterans are particularly disadvantaged by the 
computer vs. in person pain assessments?). Linking to existing health data allows us to minimize the length and 
intrusiveness of the tablet based survey, while still assessing how social and clinical vulnerability is associated with 
screening measure and mode. Experience during the trial will inform implementation and be gathered through 
interviews with both patients and providers and clinic staff.   
Results: This is a new project and results have not yet been obtained.   
Significance: We will characterize how to best conduct pain screening including a direct comparison of patient reported 
to clinician documented pain, and selection of an optimal measure to improve clinician and patient centeredness of pain 
measurement (e.g., by considering the incorporation of function). This project will inform national VA implementation of 
a preferred measure and method.  
  
  
  
Contents:  

 
   
 
Page 3 of 25  
  
Protocol Title: Effective Screening for Pain Protocol……………………………………………………………..…………………………………………3   
1.0  
Study Personnel……………………………………………………………………………………………………………………………..……………………4  
2.0  
Introduction…………………………………………………………………………………………………………………………………………………………7  
3.0  
Objectives………………………………………………………………………………………………………………………………………..………………….9  
4.0  
Resources and Personnel……………………………………………………………………………………………………………………………….…10  
5.0  
Study Procedures………………………………………………………………………………………………………………………………………………11  
5.1 
Study 
Design…………………………………………………………………………………………………………………………………………11 
Phase 1 Overview…………………………………………………………………………………..…………………………………………….11 
Phase 1 procedures……………………………………….………………………………………………………………………….…………..11  
a. Assess the usability of patient-self-administered pain assessment via tablet………………………………..…13  
Phase 2 Overview……………………………………………………………………………………………………………….…………………14  
A. Multisite RCT to test pain screening approaches via tablet:………………………………..………….…………14  
1. Recruitment and Consenting………………………………………………………………………………………….…..14   
a. Participant Eligibility……………………………………………………………….……………………………..14  
b. Recruitment .................................................................................................................. 15 
2. Consent Process ........................................................................................................................ 16 
3. Data Collection Procedures ....................................................................................................... 16 
4. Data Security ............................................................................................................................. 16 
5. Tablet based survey content ..................................................................................................... 17 
B. Patient Follow up Interviews ............................................................................................................................... 18 
1. Recruitment and Consenting ..................................................................................................... 18 
a. Participant Eligibility ..................................................................................................... 18 
b. Recruitment .................................................................................................................. 18 
2. Consent Process ........................................................................................................................ 19 
3. Data Collection Procedures ....................................................................................................... 19 
4. Compensation ........................................................................................................................... 19 
5. Data Security ............................................................................................................................. 19 
C. Provider interviews .............................................................................................................................................. 19 
1. Recruitment and Consenting ..................................................................................................... 19 
a. Participant Eligibility ..................................................................................................... 20 
b. Recruitment .................................................................................................................. 20 
2. Consent Process ........................................................................................................................ 20 
3. Data Collection Procedures ....................................................................................................... 20 
4. Data Security ............................................................................................................................. 20 
5.2  Informed Consent Procedures .......................................................................................................................... 21 
5.3  Study Evaluations/Data Analysis ...................................................................................................................... 21 
5.4  Withdrawal of Subjects ..................................................................................................................................... 22 
6.0  Reporting ....................................................................................................................................................................... 22 
7.0  Privacy and Confidentiality ............................................................................................................................................ 22 
8.0  Communication Plan ..................................................................................................................................................... 23 
9.0  References ..................................................................................................................................................................... 23 

 
   
 
Page 4 of 25  
  
 
Protocol Title:  Effective Screening for Pain Protocol  
  
1.0  
Study Personnel  
 Principal Investigator: Karl Lorenz MD,MSHS Investigator (15% effort all 3 years, contributed) is a primary care and 
palliative care physician, a two-time VA HSR&D Career Development Awardee,  and Director of the Palliative Care 
Quality Improvement Resource Center (QuIRC) and Research Core for the VA Center of Excellence  (COE) for the Study of 
Healthcare Provider Behavior. QuIRC’s mission is to promote provider-facing informatics quality improvement tools, 
with a focus on the VA’s electronic medical record and data resources to drive performance measurement and 
improvement. Dr. Lorenz was the PI of the HELP-Vets Study that characterized gaps in the implementation and 
effectiveness of the ‘5th Vital Sign.’  Dr. Lorenz has collaborated with Dr. Krebs and Dobscha through the VA Pain 
Research Working Group,. QuIRC’s ties to Indianapolis include a history of working with the Human Computer 
Interaction laboratory to characterize practice tool usability. Dr. Lorenz has close ties to the informatics and clinical 
teams at Portland where he has collaborated with Dr. Lesselroth and conducted evaluation activities in the recent past. 
QuIRC and the Center for Implementation Practice and Research Support are connected through Dr. Mittman who sits 
on the Advisory Board for QuIRC. Dr. Ahluwahlia is a joint mentee of Drs. Lorenz and Mittman. Dr. Lorenz has assembled 
a highly experienced and dedicated team of experts, most of whom have successfully worked together and published 
extensively with him on closely related research in the past.   
Co- Investigator: Erin Krebs, MD, MPH (10%,all three years, contributed) is a primary care physician at the Minneapolis 
VA Health Care System, medical director of the Minneapolis Women Veterans Comprehensive Health Center, and core 
investigator at the Minneapolis VA Center for Chronic Disease Outcomes Research (CCDOR). She is a recipient of a VA 
HSR&D Career Development Award (CDA-2) focused on the safe and effective management of opioid analgesics and PI 
on the VA HSR&Dfunded SPACE trial, which is comparing opioid-intensive versus opioid-avoidant prescribing strategies 
for chronic musculoskeletal pain. She is an active member of the VA Pain Research Working Group and an editorial 
board member for Pain Medicine’s (official journal of the American Academy of Pain Medicine) primary care and health 
services section. She has published with Dr. Lorenz on pain screening and validation of the PEG ultra-brief measure for 
chronic pain assessment, which is being evaluated in the ESP trial. Dr. Krebs will be Minneapolis site-PI. She will oversee 
conduct of the qualitative and clinical trial phases at that site. She will oversee all site specific activities (e.g., IRB and 
data collection) and she will contribute to the analyses of qualitative and quantitative data, including all publication 
activities.  
 
Co-Investigator: Steven K. Dobscha M.D., (10% effort all 3 years, contributed) is Director of the HSR&D Center of 
Innovation located at the Portland VAMC. Dr. Dobscha is also a health services investigator working in the areas of 
chronic pain, psychiatric disorders comorbid with general medical illness, and suicide prevention and has contributed to 
knowledge of the effectiveness of collaborative approaches to chronic pain and depression, barriers to providing optimal 
care for pain and other chronic conditions in primary care settings, and opioid use patterns among veterans with chronic 
pain. Previous VA HSR&D funding has supported randomized trials of care management and collaborative care for 
depression and chronic pain in primary care, respectively, and he is currently PI of several VA HSR&D studies related to 
suicide prevention of Veterans. For the proposed study, Dr. Dobscha will serve as Site Investigator at the Portland 
VAMC, and he will contribute to the analyses of qualitative and quantitative data including all publication activities.  
Co-Investigator: Peter Glassman M.B.B.S., M.Sc., FACP (5% effort for years 2-3 contributed): Dr. Glassman is the 
CoDirector of the VA Center for Medication Safety (based in Hines, Illinois) and a member of the Medical Advisory Panel 

 
   
 
Page 5 of 25  
  
for the Veterans Heath Administration’s Pharmacy Benefits Management Service. He is also an alternate member on the 
Drug Safety Oversight Board of the Center for Drug Evaluation and Research at the Food and Drug Administration. He is 
board certified in both Internal Medicine and in Hospice and Palliative Medicine and has substantial administrative and/or 
research experience with numerous publications in pharmacy benefits management, electronic prescribing and drug 
alerts, quality of pharmaceutical-related care, and medication safety.  When Dr. Lorenz transitions to Palo Alto in May of 
2015, Dr. Glassman will assume the role of Site-PI at GLA. He will serve as a Co-Investigator and contribute to research 
products. He will work closely with project director Dr. Giannitrapani to oversee the qualitative data collection that will 
occur in GLA during wave 2. His time commitment is less than the other site PIs because the RCT will not collect survey 
data at GLA.   
  
Co-Investigator: Brian Mittman: PhD, (5% effort all 3 years) is a health services researcher who is an international leader 
in implementation science, quality improvement research and healthcare management.  He directs the QUERI 
implementation resource center, Center for Implementation Practice and Research Support (CIPRS), and as Senior Social 
Scientist at the Sepulveda HSR&D Center of Excellence.  He serves as co-editor in chief of the journal Implementation 
Science and on various advisory committees in the fields of implementation science, HSR and comparative effectiveness 
research, including the Patient Centered Outcomes Research Initiative (PCORI) Methodology Committee,  American 
Association of Medical Colleges (AAMC) Advisory Panel on Research and International Scientific Advisory Committee for 
Knowledge Translation Canada.  He served as interim Associate Director of HSR&D for QUERI from 2002-2004.  He and 
Dr. Lorenz co-mentor Dr. Sangeeta Ahluwahlia who will lead qualitative data collection and analysis  with a focus on 
evaluating implementation relevant concerns for routine pain screening. He will provide advice on the development of 
the qualitative interview and focus group guides for Aim 1 and Aim 2, as well as all publication activities related to those 
Aims and shape the relevance and findings of the ESP project and so that it can robustly inform subsequent intervention 
and research. He will contribute to the analyses of qualitative and quantitative data including all publication activities  
  
CO-Investigator: Robert Kerns, Ph.D., (5% effort all 3 years, contributed) is a clinical health psychologist, National  
Program Director for Pain Management for the Veterans Health Administration (VHA), Director of the Pain Research,  
Informatics, Medical comorbidities, and Education (PRIME) Center at the VA Connecticut Healthcare System, and 
Professor of Psychiatry, Neurology, and Psychology at Yale University.  He will participate in this project as an 
internationally recognized pain expert and has contributed substantially to the conceptual and empirical literatures of 
direct relevance to this application, especially in the areas of pain screening and measurement.  Of particular relevance, 
he was the chair of the “Pain as the 5th Vital Sign” initiative that launched the VHA’s strategy for routine screening for 
the presence and intensity of pain using the numeric pain rating scale, and he chaired a work group that drafted the 
“Pain as the 5th Vital Sign Toolkit” that continues to serve as the source document for the initiative.  He will collaborate 
with the ESP team in the overall scientific direction of the project and help them to troubleshoot any problems that 
arise.  In addition, he will help with interpretation of data and preparation of manuscripts that arise from this research.  
He will offer his assistance as National Program Director for Pain Management in the dissemination of the findings of 
this project, including how it informs next steps to improve the quality of pain measurement and  
management in VA. He will contribute to the analyses of qualitative and quantitative data including all publication 
activities  
  
Consultant: Sangeeta Ahluwalia, PhD, MA, MPH: (25% effort x 9 months in Year 1; 25% x 12 months in Year 2, 25% x 6 
months in Year 3) Dr. Ahluwalia serves as an Investigator at the Sepulveda Center for the Study of Healthcare Provider 
Behavior, and a Program Member of the VA palliative care Quality Improvement Resource Center (QuIRC). Her research 
centers on improving palliative care for older adults with serious progressive illness through the development of quality 
improvement interventions. She has formal training in and extensive experience with qualitative research methods and 
analyses, including focus groups, structured, semi-structured and key informant interviews, and direct non-participant 
observation (including workflow observation). She has additional expertise in implementation research methods, 
particularly in structuring and conducting formative and process evaluations. She has worked closely with Dr. Lorenz 
over the past 3 years with the VA palliative care Quality Improvement Resource Center which he directs. In that role, she 

 
   
 
Page 6 of 25  
  
helped to develop and evaluate the Palliative Care National Template (PC-NCT), an EMR-(CPRS) based template designed 
to increase documentation and measurement of key symptom screening outcomes. She conducted semi-structured 
interviews and provider focus groups to characterize provider approaches to and preferences for symptom 
management. She will oversee the development of the qualitative sampling approaches and the interview and focus 
group guides, the qualitative data collection phase, and participate in and supervise all qualitative analyses including 
manuscript development and writing. After the first year Dr. Aluwalia transitioned to RAND as is no longer a GLA 
employee. When the grant was originally written, she intended to be involved over the course of the study. She however 
received a National Palliative Care Research Center CDA which provided her with full salary support so she did not take 
funds from the project. She did contribute time in years 0 and years 1 for the collection and Analysis of the wave 1 focus 
groups. In years 2 and 3 she has an appointment at RAND and is not officially involved with the project. She does, 
however, continue to be a collaborator on products from the wave 1 data and she will continue to be involved in an ‘in 
kind’ role contributing to the development of all qualitative tools for Aim 2.  
  
OTHER Personnel  
Statistical Analyst: TBD, (25% Year 2, 50% Year 3) They will carry out key database management tasks necessary for the 
successful conduct of the study's data analysis activities. The analyst will work closely with Dr. Lorenz and other project 
team members in accessing and using data sources that contribute to the operationalization of all patient variables. In 
year 2, he will prepare a database for the integrated pain screening and research survey and evaluate the quality of data 
on initial patients including skip patterns, completeness, and data quality to assure appropriate function of the tablets 
and data and uniform interpretation of the abstraction guideline by all abstractors. In year 3, the analyst will work with 
Drs Lorenz, Giannitrapani and the analysis team to lead the final preparation of the analytic dataset including data 
cleaning, checks, and final operationalization of all independent and dependent variables, and support the statistical 
programming required for all main and secondary aims analyses. The analyst will work closely with the Project Director 
and the Data Security and Privacy manager to assure compliance of the study protocols, data storage, and data handling 
procedures with human subjects and privacy requirements. Dr. Lorenz is working to identify an analyst based in Palo 
Alto.   
  
Statistician: Martin Lee, PhD (10% Year 2, 10% Year 3) Dr. Martin Lee, PhD (Sr. biostatistician) is a core faculty member 
of the Center for the Study of Healthcare Provider Behavior and an Adjunct Professor at the UCLA School of Public 
Health, with expertise in a wide variety of statistical issues including the treatment selection models and other 
approaches being considered by the proposed study. Dr. Lee will provide expert guidance for the data analysis including 
variable derivation and statistical modeling. He will work closely with Dr. Lorenz and the rest of the ESP project team to 
facilitate all project aims and oversee the accuracy and quality of all statistical reports by the project. He provided 
statistical support to the HELP-Vets project (Lorenz PI) which evaluated the ‘5th vital sign’ and its accuracy and 
usefulness in primary care.  
  
Project Director: Karleen Giannitrapani, PhD, MPH (GLAPalo Alto) BD: GS 12/1 (0% Year 1, 75% Year 2, 80% Year 3): 
The Project Director will work closely with Dr. Lorenz to oversee and implement all phases and activities of the ESP 
Project. The Project Director will oversee and prepare budgets, personnel actions, and project management oversight.  
The Project Director will work to ensure high quality interactions with the UCLA subcontractor and coordination with VA 
OI&T in tablet rollout at each of the study sites. The PD will act as a liaison between VA Palo Alto and each of the sites 
including working closely with each Site PI and the site staff, for IRB, for data collection, for supervision and coordination 
of the preparatory marketing activities with clinicians and Veterans, for oversight of the privacy and security issues 
related to data transfer, uses of the data to coordinate follow-up telephone surveys, conduct of followup surveys 
(working closely with the team to develop and oversee research assistants and site coordinators  in conducting the 
follow-ups remotely), and in coordinating and participating in the analyses both qualitative and quantitative, as well as 
in producing manuscripts for publication, presentations for VA national leadership and for research venues, and in 
working the team to apply ESP lessons to operational tablet rollout and modification of the VA’s pain screening policies, 

 
   
 
Page 7 of 25  
  
processes, and monitoring. The Project Director will participate fully in all intellectual activities of the grant including 
publication.  
  
Research Assistant/ Site coordinator GS 11/1 (GLA): (20% year 1, 5% year 2, 10% year 3) Hannah Schreibeis-Baum, 
MPH: The project associate will provide administrative support including generation and documentation of 
correspondence related to project staffing and initial and annual human subjects and research training, human subjects 
and information security protocols, coordination of team communication (i.e. telephone and videoconferencing, 
broadcast emails, faxes, letters) between all sites and collaborators.  Additionally the project associate will help with 
consenting and interviewing participants in the primary care clinics. The project associate will work with Dr. Ahluwalia 
(at GLA) and assisting with planning, preparing for, and running qualitative data collection including attending focus 
groups, and  participating in the analysis of qualitative data as a member of that team. She will act as the site coordinator 
and maintain site IRB documents once the Project Director is based in Palo Alto.  
  
Research Assistant/ Site coordinator GS 11/1 (Portland): Anne Kovas, MPH (40% FTE all 3 years): Anne Kovas is a 
Research Health Science Specialist at the Portland VAMC.  Ms. Kovas holds a MPH in epidemiology and biostatistics, has 
worked in healthcare research for over a decade, and has worked with Dr. Dobscha as a research associate on several 
studies in the areas of chronic pain and primary care health services. Ms. Kovas will be responsible for working with Dr. 
Dobscha, Portland Site PI, to initiate, undertake, and supervise all site related grant activities including IRB, data security 
and privacy, consents, marketing the project to patients and providers, maintaining communications with the facility, 
research, and clinical staff during the project, as well as collecting qualitative site data (for Aim 1 and Aim 2) with the 
remote input of Dr. Ahluwalia and the direct supervision of the site PI.  Ms. Kovas will also provide all administrative 
support to local sites in coordinating and communicating with collaborators. With the site PI, Ms. Kovas will provide 
support for site specific personnel, budget, and IRB activities.    
  
Research Assistant/ Site coordinator GS 11/4 (Minneapolis): Agnes Jenson, BA (10% FTE all 3 years): Agnes Jenson is a 
Health Science Specialist and an experienced project coordinator who has previously managed studies involving pain, 
PTSD, and meditation interventions.  Ms. Jenson will be responsible for working with Dr. Krebs, Minneapolis Site PI, to 
initiate, undertake, and supervise all site related grant activities including IRB, data security and privacy, consents, 
marketing the project to patients and providers, maintaining communications with the facility, research, and clinical staff 
during the project under the direct supervision of the site PI.  Ms. Jenson will also provide all administrative support to 
local sites in coordinating and communicating with collaborators. With the site PI, Ms. Jenson will provide support for 
site specific personnel, budget, and IRB activities.    
  
Research Assistants Minneapolis (30% Year 1-FY14, David Leverty, BS, GS-7; 50% Year 3-FY16, TBD, GS-9):  The 
Research Assistant works with the Site PI to initiate, undertake, and coordinate all site-related project activities. The 
Research Assistant is responsible for site IRB administration including developing the initial IRB application, requesting 
any protocol modifications, and submitting continuing review requests. The Research Assistant provides all 
administrative support for the project, maintains all study-related files, participates on all project conference calls, and 
follows-up on all questions and feasibility assessments related to project implementation at the site.  For the qualitative 
aspects of the study the Research Assistant will participated in identifying an approach to market the project to patients 
and providers, recruit and consent subjects, and collect data.  
  
Research Assistants Portland We are engaging other part time research assistants to help with in-clinic data collection 
and administrative support at Portland. Their time is covered by the Portland COIN. For wave one Portland used Maura 
Pisiotta BA  (GS 9/1 20% year 2) and Risa Comer BA (GS 11/1 20% year 1). For Wave two Portland is using Holly Williams 
BA (GS 9/1 15% years 2 and 3) and Stephanie Veazie (GS 7/1 25% years 2 and 3).  
  
Research Assistant/ Site coordinator GS 9/2 (Palo Alto): Matthew McCaa, BA (50% 3m year 2, 50% 9m year 3): 
Matthew  McCaa will work under the direction of Project Director Dr. Giannitrapani to undertake the in clinic data 
collection for the RCT. He also has experiences with qualitative coding and will work with Dr. Giannitrapani collect and 

 
   
 
Page 8 of 25  
  
analyze the wave to patient and provider interviews. Dr. Giannitrapani will provide training to all research assistants and 
ensure all study and field procedures are followed. Mr. McCaa will also assume cross site coordination duties and 
maintain meeting agendas, minutes, and action items.   
  
Research Assistant/ Administrative Support  GS 9/1 (Palo Alto): Roger Day, BA, BS: Roger Day will work under the 
direction of Project Director Dr. Giannitrapani to undertake the in clinic data collection for the RCT. He will also handle 
travel for all project staff; provide support with all logistics.   
Administrative support GS 7/1 (GLA): Ernest Raye 30% year 2:  Ernest Raye BA provided administrative support at GLA 
in year 2. His role is to schedule meetings, meeting reminders, agendas and minutes; facilitate cross-site collaboration; 
handle travel for all project staff; provide support with all logistics. These functions will handled by Matthew McCaa in 
the later part of year 2 and year 3.   
  
2.0  
Introduction  
Despite the importance of pain and widespread interest in patient-centeredness, the VA remains exceptional in 
emphasizing and successfully making pain and its management a routine feature of the health record and a focus of 
care.  Awareness of pain and efforts to improve pain management rest on the VA’s ‘5th Vital Sign’ – a policy and practice 
of nursing staff routinely screening for ‘pain now’ at every health encounter using a 0-10 Numeric Rating Scale (NRS) (1).   
Our team’s previous research on the VA’s ‘5th Vital Sign’ informs the specific design of the ESP study as well as the 
proposed research products. (5-7)  Our evaluation of routine pain screening practice in the VA identified some important 
concerns that keep the VA’s ‘5th Vital Sign’ from living up to its full promise:  
• 
The “fifth vital sign” measure of current pain intensity, demonstrates limited accuracy leading to only moderate 
correspondence (e.g., ROC of 0.76-0.8 in two studies) for chronic disabling pain measured with the Brief Pain 
Inventory (BPI) (5,10), and   
• 
Nursing staff who deliver pain assessment in about 50% of cases use informal, unstructured pain queries (e.g., ‘Is 
your knee good today?’) rather than adhering to formal use of the 0-10 NRS leading to pain underestimation, 
despite efforts to standardize assessment (5), and  
• 
Linkage is missing between the current pain screening approach and improved care—pain screening information is 
passively available but seldom used by busy providers who act on moderate to severe pain with augmented 
management in only about 15% of cases. (6,7)  
Several alternatives to the current pain screening approach (e.g., nurse administered ‘NRS pain now’ improve the 
sensitivity and specificity of screening for chronic pain – the optimal measure being a three item scale incorporating 
intensity and emotional and physical interference (e.g., the PEG). (6,7) The PEG includes items that assess physical and 
emotional interference, and it is very similar to the gold standard BPI from which it is derived in sensitivity, specificity, 
and sensitivity to change in detecting clinically important, functionally impairing pain. (6)  
A patient-reported approach (e.g., PRO) might be preferable to clinician screening, but these alternatives should be 
formally compared.  On the one hand, nursing staff responsible for screening for the ‘Fifth Vital Sign” screen informally 
which reduces screening sensitivity. (5)  We found that nursing staff members’ work experience, confidence in pain 
management, and use of informal rating contribute to pain screening variation. (11) Thus, by removing nursing staff 
rater variation, A PRO approach might improve standardization of the report. Other benefits could include scalability and 
sustainability over time including evolution consistent with automating aspects of care.  
Although a PRO approach might reduce variation due to nursing staff rating factors, it could also increase variation 
by introducing unique issues including the specific items, mode, and context. For example, we also found in HELP-Vets 
that environmental context (e.g., a distracting or busy environment) contributed to pain screening variation, (11) and 
those culd be worse depending on the physical environment of a tablet! Further underscoring the need to compare 
approaches, clinician documented pain could be less sensitive but more actionable (e.g., clinicians document the pain 

 
   
 
Page 9 of 25  
  
they can do the most about), and some patients’ who are challenged to self report (e.g., those with profound cognitive 
or visual impairment) might be best evaluated by a clinician.   
In considering an enhanced pain screening solution, we will conduct a formative evaluation for more widespread 
implementation including consideration of the role of informatics and how to link screening-to management.  (8,12) In 
the longer term, implementation success has to be framed in how screening improves patient outcomes  – in the case of 
pain, through the use of meaningful metrics (e.g., functional), actionable treatment strategies, and multidisciplinary and 
generalist-specialist collaboration.  Although it’s unclear exactly how to link screening to management, clinicians need to 
be engaged to determine how screening information should be presented and integrated in workflow to facilitate 
patient outcomes.    
Informatics offers an important focus in how to link screening to management. (12) Informatics leverages the 
electronic availability of the ‘5th Vital Sign’ embodied in VA data systems. Although they have not been standardized or 
distributed in VA practices, informatics features of the VA-based trial on which the VA’s national pain policy of ‘stepped 
pain management’  is based included treatment recommendations communicated through electronic alerts and email, 
‘order sets’ based on treatment recommendations, and triggered consultations. (12,13) No specific research illuminates 
informatics design for pain management although features of effective alerts, clinical reminders, order support, and 
analogous applications are well described. (14-16) The ESP project will conduct preliminary work to characterize 
informatics solutions to link screening to management that can be experimentally evaluated in a subsequent study.   
 The ESP project aims to conduct a naturalistic experiment of the use of pain screening and context that as closely as 
possible replicates daily VA primary care practice. The project aims to have minimal drop out because it will directly 
inform national implementation of a new routine pain screening process (a process that is used in approximately 30 
million + health encounters annually). Representing the general population of veterans is particularly crucial for those 
Veterans who because of vulnerability might be both more likely to decline to participate, but are also those whose 
reports of pain might be most jeopardized by a move to computer based screening. (such as those with mental health 
conditions). The project is focused on implementation or how to roll out tablets without special research support; thus a 
broad variety of clinical and non-clinical staff will provide important input on how to make rollout of the tablets 
successful.   
  
3.0  
Objectives  
The overall objective of our mixed method study is to evaluate a strategy to improve the quality and clinical usefulness 
of ‘the 5th Vital Sign’ in VA primary care. We will first develop a patient-self administered computer-aided pain 
assessment, then evaluate and refine the assessment using human-computer interaction laboratory usability testing. 
Next we will conduct a three site randomized controlled trial (RCT) to test a computer-based pain assessment protocol 
compared with usual pain screening. In the RCT we will evaluate three arms – one arm will use the PEG, a more sensitive 
screener including pain functional impairment, and a second arm will replicate the ‘NRS now’ allowing a direct 
comparison of patient-administered assessment vs. clinician assessed pain and a third arm will test the NRS at one week. 
All subjects will be screened with the ‘5th Vital Sign’. (allowing a three arm comparison of each of two computer based 
approaches vs. usual practice) Our specific aims are to:  
Aim 1. Characterize primary care team perspectives on enhanced pain screening approaches compared with usual pain 
screening, including clinical usefulness and how to use skills, roles, and tasks of multidisciplinary primary care clinicians 
and consultants to optimize pain care. Additionally, we will try to understand Veterans’ experiences with pain screening, 
assessment, and management (including use of opioid analgesics)  in order to better link their actual experience with 
provider assessment and management practices.  Interviews and focus groups will inform refinement of the items and 
screening interface; we will evaluate the interface including the PEG and ‘NRS now’ and the control arm for usability 
including among disabled veterans and for use in primary care.   

 
   
 
Page 10 of 25  
  
Aim 2. Evaluate in a three-arm RCT the effects of enhanced pain screening approaches using a tablet and comparing the 
PEG, the ‘NRS pain now’, and usual clinician-assessed pain screening.   
Hypothesis 2a:  One of three enhanced pain screening approaches using the PEG, but not the ‘NRS now’ will be 
associated with improved detection of pain related disability Hypothesis 2b: Enhanced pain screening approaches 
vs. usual pain assessment will be associated with equally low missingness, Veterans, providers, and clinic and 
facility staff will characterize their experience with the enhanced screening protocol and important barriers, 
facilitators, and practical steps to inform implementation  
4.0  
Resources and Personnel  
  
There are a total of four research sites for the Effective Screening for Pain study. The main research site is the  
West Los Angeles VA, the other sites are the Portland VA, Minneapolis VA, and the Palo Alto VA. Additionally, Robert 
Kerns PhD will participate in this study as a pain expert and assist with the analyses of qualitative and quantitative 
data as well as publication activities. He is located at the Connecticut VA and is therefore not mentioned in the 
categories below.   
West Los Angeles VA:  

 
   
 
Page 11 of 25  
  
Karl Lorenz: (Pre-move)  
Pete Glassman: Co-I  
Brian Mittman: Co-I  
Sangeeta Ahluwalia: Consultant  
Martin Lee: Statistician  
Hannah Schreibeis-Baum: Research Associate  
Karleen Giannitrapani: Project Director (Pre-Move)  
Ernest Raye: Administrative support  
  
Portland VA:  
Steven Dobscha: Co-I:  
Anne Kovas: Research Assistant  
Risa Cromer: Research Assistant  
Maura Pisiotta: Research Assistant  
Holly Williams: Research Assistant  
Stephanie Veazie: Research Assistant  
  
Minneapolis VA:  
Erin Krebbs: PI  
Agnes Jenson: Research Assistant  
David Leverty: Research Assistant  
  
Palo Alto VA:  
Karl Lorenz: PI  
Karleen Giannitrapani: Project Director (Post move)  
Matthew McCaa: Research Assistant  
Thomas Day:  Research Assistant  
  
Contracts:   
UCLA Contract: UCLA’s Office of Information Technology (OIT)’s Educational and Collaborative Technologies Group 
(ECTG) has expertise in the area of both  web development, disabilities computing (OIT runs UCLA’s Disabilities and 
Computing Program, see http://dcp.ucla.edu ) and assistive technologies.  UCLA will assist the ESP team in this grant by 
working closely with Dr. Karl Lorenz and other team members to determine and develop the best methods of collecting 
patient pain assessment data via tablet.  It is well understood by the ECTG Web Team, that accessibility of such a system 
is not only critical, but legally mandated.  
In support of this effort, the ECTG Web Team will work on a series of tasks in support of this grant.  The first of 
which is to gather the adaptive technology requirements in terms of both hardware such as headphones, braille labels 

 
   
 
Page 12 of 25  
  
etc…and software such as screen readers and other assistive technologies.  Given the ECTG Disabilities Computing 
expertise, this group is well suited to provide the pain RCT grant team with a set of recommendations for adaptive 
technology proof of concepts, research the available vendor technologies to meet these requirements and then 
recommend the appropriate vendor to meet these needs.    
The ECTG Web Team will work closely with the Pain RCT PI’s to first create and deliver a comprehensive user 
interface Pain RCT system design and specification, followed by the application itself.  The design process will include the 
creation of UML diagrams to describe the application navigation and flow.  The specification will also include the process 
for producing the random initial questionnaire.  The design document will also include the specification for the follow-on 
questionnaire.  The study questions will be able to be edited via the application’s administrative module.  The ECTG 
Programmer Analyst IV role will work with the PI’s to attain their formal approval and sign-off on the specification.  
  
The UCLA team will also be responsible for the usability testing of the pain screening protocol.  
5.0  
Study Procedures  
5.1 Study Design  
We are conducting a two phase mixed method study that will build on our prior work to develop and test enhanced pain 
screening approaches. We chose this design because of its suitability for pragmatic development and evaluation of 
enhanced pain screening approaches for primary care and Patient Aligned Care Teams (PACT). (28)  In the first phase of 
the study—the development phase—we will conduct semi-structured qualitative interviews and focus groups with 
primary care clinicians, other primary care team members including non-provider staff, and primary care Veteran 
patients to understand what patient-reported pain assessment data is most useful for clinical decision making and how 
this pain information can best be integrated into primary care team processes including the role of informatics to 
optimize primary care pain management and link pain screening to management. This will inform the development of 
the enhanced pain screening approaches used in the Aim 2 RCT. We will also submit the enhanced pain screening 
approaches after they have been integrated into a tablet based research protocol to usability testing by the University of 
California Office of Information Technology.   
Phase 1:   
a. Gain perspective of PACT providers and staff on how pain screening information should be collected and 
how screening can be integrated with informatics (qualitative): To ensure that enhanced pain screening 
approaches provide relevant information in an acceptable format to primary care providers and staff, we 
will initiate a qualitative study of provider and staff perspectives on pain assessment at the study outset. 
These focus groups and interviews will stimulate interaction and discussion around how nursing, physicians, 
social workers, and other clinic staff can work together, including how each disciplines skills, knowledge 
roles, tasks, and responsibilities can contribute to pain assessment and management.   
 Recruitment and Consenting:   
Providers: We will conduct a total of twelve focus groups, four at the West Los Angeles VA, four at the 
Portland VA and four at the Minneapolis VA. As well as up to 20 one-on-one interviews with providers 
and clinical staff at the West Los Angeles VA. if they should choose not to participate in the focus group 
and desire a more private setting. Each focus group will have an average of 6 participants. With four 
focus groups at each study site, we expect to have a total of 24 participants per site. If we are very 
successful in our recruitment, we may have up to 8 participants in each focus group. In that case, we 
expect a total of 32 participants per site. The focus groups will consist of individuals who regularly assess 
pain, manage it, and follow-up (e.g. MDs, PAs, NPs) as well as people who receive patient-reported pain 
information without formal assessment (e.g. clerks, phone screeners, MAs, other nurses). We will meet 
with all staff at the beginning of the study at each site during a regular staff meeting time for an 

 
   
 
Page 13 of 25  
  
informational session. The study will be described, informational sheets distributed, and staff questions 
will be answered regarding the study. At that time signup sheets will be placed in the primary care clinic 
staff break rooms for participation in the focus groups. To maintain privacy of those that want to 
participate we will make it known that an email will follow the in person sign up sheet if they would 
prefer to sign up in private.  After the staff meetings, department emails will also be sent out including 
the study information sheet and we will provide our contact information for providers to contact us with 
their interest in participating in a one-on-one interview instead. We will recruit two subgroups for 
qualitative evaluation—primary care providers (physicians, advanced practice nurses, and physician 
assistants) and primary care staff (nurses, social workers, medical assistants, clerks). Primary care 
providers and staff in any of the three sites (GLA, Minneapolis, Portland) will be eligible to participate. If 
participants are uncomfortable participating in a focus group, the private interview option will be best. 
Of course, potential participants can opt out at any time. At the beginning of each one-on -one 
interview as well as the start of the focus groups, verbal consent will be obtained for participation. . First 
research staff will distribute information sheets and go over the information provided in the sheet, then 
a question and answer time will follow. After all questions are resolved, participants will be verbally 
consented and this will be audio recorded. We will make sure participants understand this is research, 
the purpose, they will be recorded and they can refuse to answer any questions as well as contact us if 
needed. Providers’ signup on the signup sheets for participation in the focus group will be a sign of 
interest in participation.   
 Procedures: For provider interviews, a research team member from GLA experienced in conducting 
interviews will interview subjects in private VA offices at each site, and focus groups will be held in 
closed conference rooms at each site. Interviews and focus groups will be digitally audio-recorded and 
later de-identified and transcribed (by KeyStrokes) for analysis as they become available. We will 
continue to conduct interviews and focus groups until theoretical saturation is reached. After all 
provider and patient interviews and focus groups, are completed, the recordings will be transcribed and 
coded and a link file will be stored behind VA firewall on the secure server, with limited access. The 
digital recordings after transcription will be maintained on VA’s secure server at the West Los Angeles 
VA and VA Palo Alto for six years from the end of the fiscal year when the research project has been 
completed.. Based on recommendations for sampling subgroups, we expect to conduct up to 10 
interviews for each group, for a total of approximately 20 interviews (WLA). ). Focus groups will be led 
by an experienced researcher from GLA (Sangeeta Ahluwalia, Ph.D.), a trained research assistant who 
will be instructed in appropriate technique by Dr. Ahluwalia, and attended and supervised by the site PI 
at each site. We will conduct four provider FGs at each site (West Los Angeles, Minneapolis, and 
Portland) for a total of 12 FGs. We will also conduct a total of 10 Veteran patient focus groups, five at 
the Portland VA and five at WLA VA, as well as up to 20 individual interviews at each site if requested by 
the Veteran.    
 Data Analysis: We will conduct analysis concurrently with data collection. The analysis team will include 
Dr. Lorenz (Palo Alto), Dr. Krebs (Minneapolis), our qualitative analyst, Dr. Ahluwalia (GLA), and the 
research associate who participated in data collection, and start by individually reading initial 
deidentified transcripts to get an overall perspective on the data. The team will then progress to initial 
coding, using principles of grounded theory. (35,36) The team will code initial transcripts simultaneously 
and meet weekly in person and by conference call to compare codes, discuss discrepancies, and reach 
consensus. Once an initial codebook has been developed, the team will move to focused coding, which 
involves identifying the most relevant and common codes and applying them to the transcripts. NVivo 
software will be used to facilitate this process. Codes will be reassessed and reconsidered throughout 
this process and some codes will be eliminated, edited, or added. The team will continue to meet 
regularly to discuss emerging themes and interpretations of the data. Throughout this process, we will 
continually refer back to the data to remain grounded in participants’ words and experiences. Findings 

 
   
 
Page 14 of 25  
  
from the qualitative phase will provide Aim 1 academic products as well as inform the subsequent 
research in several ways:  
 Clinician perspectives on historical information useful to conducting clinical evaluation will be 
considered for the integrated enhanced pain assessment and research survey protocol.   
 Staff and clinician input on a tablet-administered enhanced pain screening approach will inform the 
initial implementation (e.g., how to locate tablets so staff can appropriately supervise patient input).  
 Staff and clinician views on the use of screening information to inform pain management will inform a 
‘map’ of intervention elements for improving pain care including staff training and education, reform of 
clinic processes, informatics elements and specific tools, workspace and virtual (e.g., shared notes) 
collaboration, elements of shared accountability (e.g., for pain-relevant performance measures) and 
other measures.   
 Detailed feedback on the use of screening information will inform general informatics approaches and 
specific prototypes. (e.g., clinicians can specify over what interval and in what format they would find 
most helpful comparable information on the chronology of pharmacologic intervention and changes in 
functional interference using PEG).  
  
b. Assess the usability of patient-self administered pain assessment via tablet: The integrated enhanced pain 
and patient survey protocol will be translated into a web-based environment for use on a touchscreen tablet 
with the support of our UCLA subcontractor/software provider. The tablet will meet standards for use with 
visually and physically impaired Veterans and facilitate accessibility through assistive technologies in 
compliance with American with Disabilities Act (section 504/508). The UCLA IT team will conduct 
performance based usability assessments at the UCLA Disabilities and Computing Lab to be sure it meets 
ADA standards. This lab provides an environment to capture usability data, assess user interaction with 
information systems, and also provide a centralized location for assessing optimal display and integration of 
clinical and operational data to integrate into clinical and management decision making. No Veterans will be 
involved in this process and instead usability of the tablet will be handled exclusively by UCLA as outlined in 
our contract with them.    
  
  
  
Phase 2:   
A. Multisite RCT to test pain screening approaches via tablet:   
Tablets will be deployed at VA Palo Alto first, and subsequently at Minneapolis and Portland. The screening trial will 
be undertaken within primary care clinics at each site, selected at random. Clinics at our three RCT sites operate in 
physical spaces that allow us to implement the tablet for a group of Veterans who are checking in for vital signs. 
Because of our goal to inform actual practice conditions (and subsequent VA wide use of the tool), after the 
installation of the tablet, we will institute screening to represent to the greatest extent possible actual practice. We 
also wish to minimize dropout of Veterans who might be reluctant to participate in research, but also most 
vulnerable to the imposition of tablet based screeners in under-estimating their pain (e.g. those with cognitive or 
mental health impairments). All Veterans who are presenting for an encounter at which vital signs are taken are 
eligible for the enhanced pain assessment protocol. We will place the tablet in a location that allows each Veteran 
to complete the assessment prior to making contact with a nursing staff vital signs screener.   
At each site, we will allow a brief run-in period to evaluate and refine the implementation prior to using the tablet to 
test the pain screening alternatives. This will allow us to focus on evaluation of the implementation and ensure that 
carrying out the RCT is not confounded by implementation difficulty. As part of the RCT we plan to observe patients 

 
   
 
Page 15 of 25  
  
using the tablet and ask them casually about their experience using the interface. This informal observation will 
allow us to discover implementation and usability issues that our patients and providers may experience. After the 
patient encounters the tablet we will casually ask them about their experience and any particular trouble they had.   
In order to compare study participants with non-participants, we will pull aggregate data via the VHA Support 
Service Center.  The information we plan to pull will include site level data on patients’ age, race/ethnicity, gender, 
distance from clinic, education, household income, marital status, diagnoses, and co-morbidities.  
1. Recruitment and Consenting:   
In general, we will solicit a visit-based sample of Veterans at each site so that we can ensure diversity in the 
population of Veterans and to minimize biases in our recruitment procedures.  We plan to enroll approximately 650 
participants among all three sites combined. We will recruit no more than 2000 respondents.   
After a brief run-in phase, patients presenting in specified primary care clinics in the Palo Alto, Portland, and 
Minneapolis VAs will be asked to complete the survey on a tablet before vital signs are taken by clinic staff. 
a.  Participant Eligibility:  
Veterans are eligible if they are attending a clinic visit in a participating primary care clinic and will have 
their vital signs taken as part of that visit and are not otherwise determined to be ineligible.   
Veterans would be determined ineligible if they are:  
• 
Severely visually impaired  
• 
Incapable of rationale responses  
• 
Physically unable to manipulate/control the tablet (unless a proxy is present and willing to input the 
Veteran’s responses)  
b. Recruitment  
The primary recruitment will take place in the primary care clinics by trained research assistants 
approaching Veterans after they have checked in for their appointment.  If a veteran expresses interest 
in participating, they will be informed of the study and screened for eligibility.  All veterans approached 
by clinic staff will receive an anonymous study identification.  Research staff will recruit in clinics for full 
or half-days selected from within the study enrollment window.  A Research Assistants will use a ‘Next 
Available’ method of randomizing which Veterans will be approached (i.e., When the R.A. is ready to 
begin/resume recruitment the next patient that completes the check in process for the clinic will be 
approached.)    
As a secondary recruitment method research staff will coordinate with each clinic to obtain a digital list 
of eligible participants and contact information 3 weeks in advance of their clinic appointment.  An 
(Introductory Recruitment Letter) will be mailed to potential participants describing the study and 
requesting that they arrive at least 20 minutes early to their appointment to learn more and to 
participate in the tablet survey. The (Introductory Recruitment Letter) will include information to 
contact Matt McCaa either by phone or by (Response Letter) to opt-out of further communication from 
the research staff. A (Telephone Interview Recruitment Letter) will be mailed to selected veterans a 
minimum of 1 week prior to telephone contact. This letter will also include information to contact Matt 
McCaa by phone or by the included (Response Letter) to opt-out of further communication by the 

 
   
 
Page 16 of 25  
  
research staff. After that week, a trained research assistant will contact the Veterans to inform them of 
the study and recruit them to participate using (Phone script).  A log of contact attempts will be 
maintained, and after a maximum of 3 contact attempts the Veteran will be removed from the contact 
list.  The lists and contact log will be encrypted and password protected.  All digital documents 
containing PKI will be encrypted and password protected and stored on a VA Palo Alto server.    
2.  Consent Process  
 In the waiting area of selected primary care clinics, patients will be approached by a research associate and 
be verbally consented. This process will include the research associate describing the study verbally as well 
as going over a study information sheet with the patient.  Subsequently, the research associate will go 
through the verbal consent process with the patient as long as they are cognitively fit enough and physically 
able to complete the tablet based screener.  A HIPAA authorization form will be filled out and signed prior to 
participation.  Research staff will mail a copy of the signed HIPAA form to the participant.  The name and last 
four of the participant’s SSN will be used for identification purposes to link the patient to the clinic pain 
screen (e.g. nursing staff documented ‘pain now NRS’) and additional health data that will allow us to assess 
the performance of the pain measure and tablet based screening method including describing variation in 
pain in socially and clinically vulnerable Veterans (e.g., those with disability, cognitive dysfunction, mental 
health disorders).  In general, the entire process will take place in the clinic waiting area.  However, in the 
unlikely case of an available private space within the clinic, we will provide Veterans with the option of 
completing the survey there.    
3. Data Collection Procedures  
The research associate will have a binder with 1 page Field Logs pre-labeled with randomly generated numbers 
to be used a participant ID.  When approaching a Veteran, the research associate will be using one field log to 
document necessary study information.  After the Veteran has been completed the verbal consent process and 
provided their first name and last 4 of their SSN the research associate will log into the tablet using the 
unidentifiable participant ID into the tablet.  Once logged in to the tablet based survey, the research assistant 
will hand the tablet to the participant.  Caregivers (i.e., family member, friend, significant others) will be allowed 
to operate the tablet if the participant so desires.  However, this will not be explicitly offered as an alternative 
by research staff.  A research associate will oversee the use of the tablets is PACT waiting areas at each selected 
site.  This process is expected to take approximately 10 minutes.  We believe that for each clinic we could 
reasonably approach 10 patients per half day at the PACT clinic and approximately half of those would consent 
to participate. At twice a week that would result in 10 participants per week, per site (30 total participants for all 
sites each week) for the tablet-based component.  
In addition, for those that decline participation in the tablet study we will ask the following:  
1. What is your main reason for not participating?  
2. What is your level of pain today (NRS-now)?  
In addition to these questions, research associates will record observed race and gender, but no additional 
identifiable information will be recorded. Responses may be recorded in the field log and/or in the tablet  
4. Data Security  
No identifiable information will be entered into the tablet.  Only the participant’s assigned identification number  
is used to log into the tablet.  The data entered by the participant will be transferred by a secure wireless 

 
   
 
Page 17 of 25  
  
connection from the tablet to a highly encrypted and secure server at UCLA (See Appendix A: Data Flow, See 
Appendix B: UCLA IT security specs). This site/server will only house data from the survey and the participant ID 
but will not house the patient’s name or SSN. This allows for storage of data outside of the VA environment in a 
non-identifiable manner.  The UCLA servers will be running the current version of CentOS, with current security 
patches. The servers will also be running intrusion detection software (fail2ban) and firewall (IPTables) to detect 
and deny access to systems that are trying to probe or breach security. The UCLA servers will be housed in the 
Math Sciences Data Center that is staffed every day of the year, all day, with limited access, cameras, and 
biometric door locks that record and log all coming and going personnel.   
  
UCLA’s network is monitored by the UCLA Network Operations Center for both security and availability. All 
network backbone routers and switches are in locked spaces with camera monitoring to record physical access 
to the network equipment. Regular traffic analysis is performed to note unusual behaviors.  
  
The 1 page field log will contain both identifiable (i.e., Full Name, last 4 digits of SSN, and contact information as 
needed for participation in follow up interviews) and unidentifiable data (i.e., participation status, participant ID, 
and observable demographics when necessary, and other research pertinent information).  While in the clinic 
field logs will be kept on person by research staff and in a locking carrying case or secured mail bag to be 
transferred to a designated office.  The paper documents will be stored in a locked filing cabinet inside of a 
locked office on VA campuses at the designated VA sites (i.e., Palo Alto, Minneapolis, Portland).  A designated 
research assistant from each site will input the data into an encrypted and password protected Excel sheet that 
is kept on a secured server at VA Palo Alto.  The designated research assistants at VA Portland VA Minneapolis 
will only have access to the cross-walk Excel sheet for their specific site. Research assistants from Palo Alto (i.e., 
Roger Day and Matthew McCaa) will have access to a site specific Palo Alto Excel sheet and a master cross-walk 
Excel sheet that is populated by site specific crosswalks.  Drs. Karl Lorenz and Karleen Giannitrapani will have 
view only access to the master cross-walk excel sheet.  At a later stage in the study, a designated programmer 
will have view only access to the master cross-walk excel sheet in order to compile data from both tablet survey 
data and medical records.    
  
5. Tablet based survey content:    
We will focus on content that is not available in existing VA administrative database the survey will include (not in this 
order):  
  
1) an introductory screen which allow for registration using the unique participant number, and  
2) an algorithm corresponding to the three intervention arms of the RCT and an underlying randomization engine 
to allocate Veterans to the three arms using a random number generator, with the corresponding pain screening 
measures (PEG, NRS one week, and DVPRS), and  
3) The Graded Chronic Pain Scale (the gold standard for comparison),and  
4) Self-assessed level of physical activity, and  
5) expectations and unmet need for pain treatment and management, and  
6) familiarity and experience with technology and satisfaction with the rating experience including whether the 
veteran has previously used a tablet and environmental context because these items reflect previous concepts 
salient to pain rating variation, and  
7) access to and experience with smart mobile technology  
  

 
   
 
Page 18 of 25  
  
Depending on the consideration of survey length which we will prioritize to keep the survey at approximately 10 
minutes, we will consider including several other items that previous research have shown to be important in 
considering pain outcomes including duration of pain in the previous year and the physical location of pain.  In order to 
minimize survey burden and in order to complete our planned analyses, we will consider using VA data (rather than 
participant responses to obtain information about BMI, comorbidity, copay exemption status). We will also ensure 
physical attributes of the tablet screen best accommodate visually impaired and other physically disabled veterans 
where possible.    
  
At the conclusion of the study in order to assess the degree to which implementation succeeded and Veterans who were 
missed by the tablet based protocol, we will compare all Veteran patients of the clinic during the study interval, as well 
as all Veterans who were approached but declined,  with all Veteran users of the tablet during the same time by age, 
race / ethnicity, gender, cognitive impairment and mental health diagnoses, and NRS pain ratings, to assure that the 
tablet based screening process does not disadvantage any Veterans. We will gather these factors using VA electronic 
databases and will request a HIPAA authorization and waiver of documentation of informed consent for this portion of 
the study.   
The usual care arm nursing staff rating of the ‘NRS pain now’ will be collected by linking the encounter to the Vital Signs  
Package to obtain the pain rating. Veterans’ health records accessed in Corporate Data Warehouse and CAPRI/ 
VistAWeb/CPRS, will inform basic demographics and clinical factors in order to allow us to minimize the burden of the 
tablet protocol.   
B. Patient Follow up Interviews:  
In order to learn about the experiences of Veterans with the pain screening via tablet and their experiences with the 
treatment of their pain, we will conduct up to 40 telephone interviews with Veterans after they have completed the 
tablet based survey.  Patient interviews will take approximately 25-45 minutes.  The interview will be audio recorded 
using VA approved recording devices (e.g., Olympus WS 822 or similar approved device), and transcribed by an external, 
VA-contracted Transcription Service.     
  
1. Recruitment and Consenting:   
In general, after Veterans complete the tablet-based survey, research staff will ask if it would be okay for us to 
contact them if we have follow up questions regarding their experience with using the tablet for pain screening.  
Not all Veterans will be contacted.  A quota sampling technique will be used to ensure we interview a 
comparable number of Veterans who reported relatively high pain and Veterans who reported relatively low 
pain.  
a. Participant Eligibility:   
Veterans are eligible to participate in the interview portion of the study if they have completed the tablet based 
pain screening survey.       
b. Recruitment:  
Upon completion of the tablet based survey in the primary care clinics, the research associate will ask 
the Veteran if it would be okay for research staff to contact them via telephone within a few weeks if we 
have any follow up questions regarding their experience with using the tablet. If necessary, additional 
contact information will be collected and recorded on the Field Log.  Of the Veterans that give us 
approval to contact them for follow up questions, a quota sampling approach will be used to ensure we 

 
   
 
Page 19 of 25  
  
select a comparable number of Veterans who reported relatively high pain and Veterans who reported 
relatively low pain.  If the Veteran is selected to participate in the follow-up interview, a research staff 
member will contact the veteran via telephone to complete the verbal consent process and the 
interview or to schedule a later time to complete the verbal consent process and interview..    
2. Consent Process:  
Prior to the interview, a research staff member will verbally consent the participant.  We have approval for a 
waiver of documentation of informed consent. Date of consent and person obtaining consent will be logged on 
the secured Excel sheets stored on VA Palo Alto Servers.    
3. Data Collection Procedures:  
We will conduct up to 40 telephone interviews that will take approximately 25-45 minutes.  The interviews will 
be audio recorded and later transcribed by a VA contracted transcription service.  For interview 
questions/content please see the Patient Interview Protocol.  
4. Compensation:  
After completion of the telephone interview, research staff will mail a $30 gift card to the Veteran’s preferred 
mailing address.  There will be no indication of their participation in the study sent with the gift card.  Veterans 
may also have the option of picking the gift card up in person if local research staff is able to safely 
accommodate.    
5. Data Security:  
Any information linking identifiable information with the assigned participant ID will be located on the Field Log 
and/or the secured Excel sheets. These documents will be secured and stored as previously mentioned under 
Phase 2: a: Data Security.    
Audio files will be uploaded to the secured server located at VA Palo Alto. The audio Files will be saved using the 
assigned participant id.  Audio files and de-identified transcripts will be transferred between VA servers and the 
transcriptions service via an approved secured FTP server.    
C. Provider interviews:  
In order to identify barriers and facilitators to implementation of the tablet based pain screening we will conduct 
telephone or in-person interviews with up to 40 VA providers and clinical staff.  This information will help guide future 
refinements to the tablet screening approach and provide valuable feedback for development of future interventions to 
improve integrated pain screening and management.     
1. Recruitment and Consenting:   
If possible, research staff will attend a local clinic staff meeting to briefly inform the staff of the study.  By partnering 
with local clinic leadership, we hope to be able to send emailed invitations to appropriate staff members.  Following this 
email invitation, we will contact the providers by telephone to either complete the verbal consent process and interview 
or to schedule a later time to complete the verbal consent process and interview.  A log of contact attempts will be 
maintained, and after a maximum of 3 contact attempts the staff member will be removed from the contact list.    

 
   
 
Page 20 of 25  
  
a. Participant Eligibility:   
VA Clinic staff may be eligible to participate in the interview if they are licensed medical staff within a clinic that 
is already included in this study.    
b. Recruitment:  
If possible, research staff will attend a local clinic staff meeting to briefly inform the staff of the study.  By 
partnering with local clinic leadership, we plan to send emailed invitations to eligible staff members.  A log of all 
contact attempts will be maintained and after a maximum of 3 contact attempts the staff member will be 
removed from the contact list.    
2. Consent Process:  
Prior to the interview, a research staff member will verbally consent the participant.  We have approval for a 
waiver of documentation of informed consent.  Date of consent and person obtaining consent will be logged on 
the secured Excel sheets stored on VA Palo Alto Servers.    
3. Data Collection Procedures:  
We will conduct up to 40 telephone interviews that will take approximately 20-30 minutes.  The interviews will 
be audio recorded and later transcribed by a VA contracted transcription service.  For interview 
questions/content please see the Provider Interview Protocol.  
4. Data Security:  
Any information linking identifiable information with an assigned participant ID will be located on the secured 
Excel sheets. These Excel sheets will be secured and stored as previously mentioned under Phase 2: a: Data 
Security.    
Audio files will be uploaded to the secured server located at VA Palo Alto. The audio Files will be saved using the 
assigned participant id.  Audio files and de-identified transcripts will be transferred between VA servers and the 
transcriptions service via an approved secured FTP server.    
  
  
Figure 1: Estimated enrollment scheme for ESP  
Due to randomization the proposed enrollment algorithm will allow analytic comparison of each assessment measure / 
mode to the other. The table below specifies the Arm to Arm comparisons but the outcomes of each comparison are 
‘clinically significant pain’ defined as pain that is moderate to severe, pain that is associated with disability, or unmet 
need for additional management.    
Main comparison (s) of interest  
Arm (s)  
Comment  
1  
Tablet: PEG vs. NRS 1 week vs. 
DVPRS  
1 to 2 to 3 tablet 
comparison  
Baseline arm to 
arm difference   
  

 
   
 
Page 21 of 25  
  
Table 1: Main analytic comparisons by screening measure and mode for outcomes of detection of pain, detection of 
moderate / severe pain (clinically relevant), pain associated with disability and unmet needs for treatment  
  
5.2 Informed Consent Procedures  
  
Summary of HIPAA and Consenting Requests:  
Phase 1:   
• 
Gain perspective of PACT providers,staff, and Veterans on how pain screening 
information should be collected and how screening can be integrated with informatics: 
HIPAA authorization not necessary for VA staff participation. Verbal consent was 
obtained, and a waiver of documentation of consent was approved.  HIPAA authorization 
forms would have been used for Veteran participants.  However, there were none.   
Phase 2:   
• 
Multisite RCT to test pain screening approaches via tablet, Patient follow-up interviews, 
Provider interviews: Waiver of Documentation of Consent necessary. HIPAA authorization 
is not necessary for the provider interviews.  A HIPAA authorization form will be filled out 
and signed by patient participants.  Research staff will a mail copy of the signed HIPAA 
form to the participant.    
  
Local site study personnel will be trained regarding human subjects protections requirements and how to obtain 
verbal consent. Study personnel are required to be up to date in the mandatory security and patient confidentiality 
trainings. Additionally site PI’s will practice consenting with research assistants and make sure the adequate skill 
level is obtained.  
  
  
5.3 Study Evaluations/Data Analysis  
  
Phase   
Data Type  
Source  
Outcome  
Analyses  
Comment  
1  
Qualitative  
Multidisciplinary 
providers, clinic 
staff, and Veterans  
n/a  
Grounded 
theory  
Characterize screening 
information use in care   
Phase   
Data Type  
Source  
Outcome  
Analyses  
Comment  
 
 
 
management  
 
 
1  
Quantitative  
Patient  
Pain-related 
disability, 
detection  
By intervention 
arm*  
At screening encounter  

 
   
 
Page 22 of 25  
  
1&2  
Quantitative  
Patient, linked to  
VA health data for 
Veteran 
demographics and  
clinical factors  
Pain 
missingness,  
By intervention 
arm*  
At screening encounter  
2  
Qualitative  
Provider, patient  
Satisfaction 
and 
experiences 
with screening 
including 
implementati 
on  
Grounded 
theory  
Characterize experiences 
with the tablet and 
enhanced pain screening 
approaches  
  
  
Outcome  
Measure  
Analysis  
Model  
Immediate pain related 
disability   
NHIS pain-related  
disability  
Arm 1,2,3 pairwise   
Linear regression, 
adjusted for patient 
factors  
Immediate pain missingness  
NRS (nurse), NRS  
(tablet), PEG  
(tablet);  
% missing pain ratings (Arm  
1,2,3)  
Logistic regression, 
adjusted for patient, rater 
(tablet perception), and 
environmental factors  
5.4 Withdrawal of Subjects  
  
• 
Veteran subjects will be withdrawn from the research without their consent if they are found to 
have any physical or cognitive abilities that makes them unable to participate in or tolerate an 
interview.   
• 
Participants of this study will have the opportunity to opt out of the study at any point in entire 
duration of the study if they so desire. IIf a subject decides to discontinue participation, only 
information collected up to that point will be used, and no additional information will be collected.  
  
6.0  
Reporting  
  
•  Our procedures for reporting unanticipated problems, serious adverse events, and protocol deviations 
include immediate notification of the Central IRB, local IRB, and our security officers. Adverse event reports 
will be executed if necessary.   
7.0  
Privacy and Confidentiality  
  

 
   
 
Page 23 of 25  
  
• 
This study will use Veteran subjects’ Protected Health Information (PHI).   
• 
Data will be secured in a number of ways:  
o All study staff with have adequate training in data security practicies and procedures o  
Access to such data sources will be extremely strict and only granted to those absolutely 
necessary for project execution  
o Data will be stored on a server protected by the VA’s firewall and password protection o 
Tablets will be designed with physical barriers to protect patients privacy  
  
8.0  
Communication Plan  
• 
SAEs that have the potential to affect implementation of the study will be communicated to all engaged 
participating sites. This will be achieved by doing the following: SAEs and other events this serious in nature will 
be discussed at weekly project meetings involving all sites.    
  
• 
Study events and interim results (if appropriate) are communicated regularly to engaged participating sites.  This 
will be achieved by doing the following: Study events and results will be communicated at weekly project 
meetings involving all sites.  
  
• 
LSIs must conduct the study appropriately.  This will be achieved by doing the PI/SC doing the following to 
ensure adequate monitoring:  
o Weekly conference calls among project leaders and members  
o Review of IRB communications to insure consistency in standards and measures along with critical 
documents.   
o Data management will be discussed among staff members on a weekly basis. o Maintaining study staff 
log with up to date information about trainings and data access and security.   
9.0  
References   
  
1. 
Pain as the 5th Vital Sign Toolkit. 2000; http://www.va.gov/PAINMANAGEMENT/docs/TOOLKIT.pdf. Accessed 
December 7, 2011.  
2. 
Krebs, E. E., Ramsey, D. C., Miloshoff, J. M., and Bair, M. J. (2011) Primary care monitoring of long-term opioid 
therapy among veterans with chronic pain, Pain Med 12, 740-746.  
3. 
Haskell, S. G., Mattocks, K., Goulet, J. L., Krebs, E. E., Skanderson, M., Leslie, D., Justice, A. C., Yano, E. M., and 
Brandt, C. (2011) The burden of illness in the first year home: do male and female VA users differ in health 
conditions and healthcare utilization, Womens Health Issues 21, 92-97.  
4. 
Krebs, E. E., Bair, M. J., Carey, T. S., and Weinberger, M. (2010) Documentation of pain care processes does not 
accurately reflect pain management delivered in primary care, J Gen Intern Med 25, 194-199.  
5. 
Lorenz, K. A., Sherbourne, C. D., Shugarman, L. R., Rubenstein, L. V., Wen, L., Cohen, A., Goebel, J. R., 
Hagenmeier, E., Simon, B., Lanto, A., and Asch, S. M. (2009) How reliable is pain as the fifth vital sign?, J Am 
Board Fam Med 22, 291-298.  
6. 
Krebs, E. E., Lorenz, K. A., Bair, M. J., Damush, T. M., Wu, J., Sutherland, J. M., Asch, S. M., and Kroenke, K. (2009) 
Development and initial validation of the PEG, a three-item scale assessing pain intensity and interference, J Gen 
Intern Med 24, 733-738.  

 
   
 
Page 24 of 25  
  
7. 
Lorenz, K. A., Krebs, E. E., Bentley, T. G., Sherbourne, C. D., Goebel, J. R., Zubkoff, L., Lanto, A. B., and Asch, S. M. 
(2009) Exploring alternative approaches to routine outpatient pain screening, Pain Med 10, 1291-1299.  
8. 
Zubkoff, L., Lorenz, K. A., Lanto, A. B., Sherbourne, C. D., Goebel, J. R., Glassman, P. A., Shugarman, L. R., 
Meredith, L. S., and Asch, S. M. (2010) Does screening for pain correspond to high quality care for veterans?, J 
Gen Intern Med 25, 900-905.  
9. 
NCI effort on PRO for pain and other things (PROMIS).  
10. 
Shugarman, L. R., Goebel, J. R., Lanto, A., Asch, S. M., Sherbourne, C. D., Lee, M. L., Rubenstein, L. V., Wen, L., 
Meredith, L., and Lorenz, K. A. (2010) Nursing staff, patient, and environmental factors associated with accurate 
pain assessment, J Pain Symptom Manage 40, 723-733.  
11. 
Dobscha, S. K., Corson, K., Perrin, N. A., Hanson, G. C., Leibowitz, R. Q., Doak, M. N., Dickinson, K. C., Sullivan, M.  
D., and Gerrity, M. S. (2009) Collaborative care for chronic pain in primary care: a cluster randomized trial, JAMA 
301, 1242-1252.  
12. 
(2009) Pain Management.  
13. 
Randell, R., Mitchell, N., Dowding, D., Cullum, N., and Thompson, C. (2007) Effects of computerized decision 
support systems on nursing performance and patient outcomes: a systematic review, J Health Serv Res Policy 12, 
242-249.  
14. 
Garg, A. X., Adhikari, N. K., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., and Haynes, 
R. B. (2005) Effects of computerized clinical decision support systems on practitioner performance and patient 
outcomes: a systematic review, JAMA 293, 1223-1238.  
15. 
Pearson, S. A., Moxey, A., Robertson, J., Hains, I., Williamson, M., Reeve, J., and Newby, D. (2009) Do 
computerised clinical decision support systems for prescribing change practice? A systematic review of the 
literature (1990-2007), BMC Health Serv Res 9, 154.  
16. 
Bodenheimer, T., Wagner, E. H., and Grumbach, K. (2002) Improving primary care for patients with chronic 
illness, JAMA 288, 1775-1779.  
17. 
Kerns, R. D., Sellinger, J., and Goodin, B. R. (2011) Psychological treatment of chronic pain, Annu Rev Clin Psychol 
7, 411-434.  
18. 
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., and Lowery, J. C. (2009) Fostering 
implementation of health services research findings into practice: a consolidated framework for advancing 
implementation science, Implement Sci 4, 50.  
19. 
Greenhalgh, T., Hinder, S., Stramer, K., Bratan, T., and Russell, J. (2010) Adoption, non-adoption, and 
abandonment of a personal electronic health record: case study of HealthSpace, BMJ 341, c5814.  
20. 
Greenhalgh, T., and Russell, J. (2010) Why do evaluations of eHealth programs fail? An alternative set of guiding 
principles, PLoS Med 7, e1000360.  
21. 
Quality Improvement Toolkit Series: Lung Cancer Care   
22. 
(2011) Integrating Care for People Eligible for Both Medicare and Medicaid.  
23. 
Williams, V. S., Smith, M. Y., and Fehnel, S. E. (2006) The validity and utility of the BPI interference measures for 
evaluating the impact of osteoarthritic pain, J Pain Symptom Manage 31, 48-57.  
24. 
Dworkin, R. H., Turk, D. C., McDermott, M. P., Peirce-Sandner, S., Burke, L. B., Cowan, P., Farrar, J. T., Hertz, S., 
Raja, S. N., Rappaport, B. A., Rauschkolb, C., and Sampaio, C. (2009) Interpreting the clinical importance of group 
differences in chronic pain clinical trials: IMMPACT recommendations, Pain 146, 238-244.  
25. 
Manchikanti, L. (2006) Prescription drug abuse: what is being done to address this new drug epidemic? 
Testimony before the Subcommittee on Criminal Justice, Drug Policy and Human Resources, Pain Physician 9, 
287-321.  
26. 
Cresswell JW, P. C. V. (2011) Designing and Conducting mixed methods research, 2 ed. ed., Sage Publications, 
Thousand Oaks, CA.  
27. 
(2005) The Sage Handbook of Qualitative Research, 3rd ed., Sage Publications, Thousand Oaks, CA.  
28. 
Rubin H, R. I. (1995) Qualitative interviewing: the art of hearing data, Sage Publications, San Diego, CA.  

 
   
 
Page 25 of 25  
  
29. 
Greenbaum, T. L. (2000) Moderating Focus Groups -- A Practical Guide for Group Facilitation, Sage Publications, 
Thousand Oaks, CA.  
30. 
Barach, P., and Johnson, J. K. (2006) Understanding the complexity of redesigning care around the clinical 
microsystem, Qual Saf Health Care 15 Suppl 1, i10-16.  
31. 
Brokel, J. M., and Harrison, M. I. (2009) Redesigning care processes using an electronic health record: a system's 
experience, Jt Comm J Qual Patient Saf 35, 82-92.  
32. 
Chan, B. C., Perkins, D., Wan, Q., Zwar, N., Daniel, C., Crookes, P., and Harris, M. F. (2010) Finding common 
ground? Evaluating an intervention to improve teamwork among primary health-care professionals, Int J Qual 
Health Care 22, 519-524.  
33. 
Strauss A, C. J. (1994) Grounded Theory Methodology - An Overview, Sage Publications, Thousand Oaks, CA.  
34. 
Glaser, B. G. (1992) Emergence vs. Forcing: Basics of Grounded Theory Analysis., Sociology Press, Mill Valley, CA.  
35. 
Kroenke, K., Spitzer, R. L., and Williams, J. B. (2003) The Patient Health Questionnaire-2: validity of a two-item 
depression screener, Med Care 41, 1284-1292.  
36. 
Callahan, C. M., Unverzagt, F. W., Hui, S. L., Perkins, A. J., and Hendrie, H. C. (2002) Six-item screener to identify 
cognitive impairment among potential subjects for clinical research, Med Care 40, 771-781.  
37. 
Turner, J. A., Fulton-Kehoe, D., Franklin, G., Wickizer, T. M., and Wu, R. (2003) Comparison of the Roland-Morris 
Disability Questionnaire and generic health status measures: a population-based study of workers' 
compensation back injury claimants, Spine (Phila Pa 1976) 28, 1061-1067; discussion 1067.  
38. 
National Health Interview Survey.  
39. 
Chodosh, J., Sultzer, D. L., Lee, M. L., Hahn, T. J., Reuben, D. B., Yano, E. M., Mittman, B. S., and Rubenstein, L. Z. 
(2007) Memory impairment among primary care veterans, Aging Ment Health 11, 444-450.  
40. 
Sherbourne, C. D., Asch, S. M., Shugarman, L. R., Goebel, J. R., Lanto, A. B., Rubenstein, L. V., Wen, L., Zubkoff, L., 
and Lorenz, K. A. (2009) Early identification of co-occurring pain, depression and anxiety, J Gen Intern Med 24, 
620-625.  
41. 
Ware, J. E., Jr., and Sherbourne, C. D. (1992) The MOS 36-item short-form health survey (SF-36). I. Conceptual 
framework and item selection, Med Care 30, 473-483.  
42. 
Cadogan, M. P., Edelen, M. O., Lorenz, K. A., Jones, M., Yosef, J., Hascall, T., Simon, B., Harker, J. O., Ferrell, B., 
and Saliba, D. (2008) The relationship of reported pain severity to perceived effect on function of nursing home 
residents, J Gerontol A Biol Sci Med Sci 63, 969-973.  
43. 
Russ, A. L., Baker, D. A., Fahner, W. J., Milligan, B. S., Cox, L., Hagg, H. K., and Saleem, J. J. (2010) A Rapid 
Usability Evaluation (RUE) Method for Health Information Technology, AMIA Annu Symp Proc 2010, 702-706.  
44. 
The CONSORT Statement.  
45. 
Consumer Assessment of Health Plans Surveys and Tools to Help Patient Care.  
46. 
Welch, H. G., Chapko, M. K., James, K. E., Schwartz, L. M., and Woloshin, S. (1999) The role of patients and 
providers in the timing of follow-up visits. Telephone Care Study Group, J Gen Intern Med 14, 223-229.  
47. 
DeSalvo, K. B., Block, J. P., Muntner, P., and Merrill, W. (2003) Predictors of variation in office visit interval 
assignment, Int J Qual Health Care 15, 399-405.  
48. 
Gallo, J. J., Meredith, L. S., Gonzales, J., Cooper, L. A., Nutting, P., Ford, D. E., Rubenstein, L., Rost, K., and Wells, 
K. B. (2002) Do family physicians and internists differ in knowledge, attitudes, and self-reported approaches for 
depression?, Int J Psychiatry Med 32, 1-20.  
49. 
Siegel S, T. J. (1960) A nonparametric sum of ranks procedure for relative spread in unpaired samples., J. Amer. 
Statist. Assoc. 55, 429-445.  
50. 
Lin, L. I. (1989) A concordance correlation coefficient to evaluate reproducibility, Biometrics 45, 255-268.