Concept Mapping as a Scalable Method for Identifying Patient-Important Outcomes
Conditions: Concept Mapping Versus Interviews
Sponsor: Thomas Jefferson University

 
 
 
 
 
 
Official Title: Concept Mapping as a Scalable Method for Identifying 
Patient-Important Outcomes 
NCT #: NCT02792777 
Date of Document: May 6, 2019 
 
 
 
 
 
 
 
 
 
 
 

VOICe Study Procedures Manual V.6 
ii 
 
 
 
 
 
 
 
Study Procedures Manual 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

VOICe Study Procedures Manual V.6 
iii 
 
 
I.  Overview  ...................................................................................................................................... 1 
A. Project aims  .......................................................................................................................................... 1 
B. Primary Outcomes  ................................................................................................................................ 1 
C. Timeline and key milestones  ................................................................................................................. 1 
D. Team member roles, responsibilities and contact information ............................................................... 5 
  
II. Training ............................................................................................................................................ 6 
A. Research Staff ............................................................................................................................................................ 6 
    B.  PAKSAB .................................................................................................................................................................... 6       
    C. Data management plan ............................................................................................................................................. 6 
                 i.     Data types, storage and person(s) responsible .................................................................................... 6 
 
                ii.     Metadata .................................................................................................................................................... 7 
                iii.    Access ........................................................................................................................................................ 7 
                iv.     Data sharing ............................................................................................................................................ 7 
                 v.    Data security and IRB compliance........................................................................................................ 7 
                vi.    Backup, archiving ..................................................................................................................................... 8 
 
III. Interview Protocols .......................................................................................................................... 9 
   A. Jefferson Emergency Department Interview Protocol ................................................................................... 9 
            A  i.     ED overview, triage process and patient flow ................................................................................ 9 
            A ii.     Screening process ................................................................................................................................ 9 
            A iii.    Scheduling process  ........................................................................................................................... 10 
            A iv.    Consent process ................................................................................................................................. 10 
            A v.     Interview process ............................................................................................................................... 10 
            A vi.    Financial Remuneration, Time Tracking, and Data Entry and Storage .................................... 11 
 
   B. Jefferson Family Medicine Associates Interview Protocol ........................................................................... 11 
            B  i.      JFMA overview and patient flow ................................................................................................... 11 
            B  ii.     Screening process ............................................................................................................................. 11 
            B iii.     Scheduling process  .......................................................................................................................... 12 
            B iv.     Consent process ................................................................................................................................ 12 
            B  v.     Interview process  ............................................................................................................................. 13 
            B vi.     Financial Remuneration, Time Tracking, and Data Entry and Storage ................................... 13 
 
  C. Post-Hospital Discharge Interview Protocol ................................................................................................... 13 
            C  i.     Hospital discharge overview  ........................................................................................................... 13 
            C ii.     Screening process............................................................................................................................... 13 
            C iii.    Scheduling process ............................................................................................................................ 15 
            C iv.    Consent process ................................................................................................................................. 15 
            C v.     Interview process ............................................................................................................................... 15 
            C vi.    Financial Remuneration, Time Tracking, and Data Entry and Storage .................................... 15 
 
       D. Financial Remuneration, Time Tracking, and Data Entry and Storage for all Interviews 
 
           D i.      Financial remuneration for patients  ............................................................................................... 15 
           D ii.     Interview time tracking procedure  ................................................................................................. 16 
           D iii.    Data entry and storage ....................................................................................................................... 16 
 
 
IV. Concept Mapping Protocol ................................................................................................................................. 17 
            i.      Screening process .................................................................................................................................. 17 

VOICe Study Procedures Manual V.6 
iv 
 
           ii.      Scheduling process ................................................................................................................................ 18 
          iii.      Consent process ..................................................................................................................................... 18 
          iv.      Concept mapping process .................................................................................................................... 18 
           v.      Financial remuneration for patients .................................................................................................... 23 
          vi.      Concept mapping time tracking procedure ....................................................................................... 23 
         vii.      Data entry and storage  ......................................................................................................................... 23 
 
V. Data Analysis and Dissemination ............................................................................................... 24 
A. Overview 
B. Interview analysis .................................................................................................................................................... 24 
C.  Concept mapping analysis .................................................................................................................................... 24 
D.  Aims 2 & 3 analysis ............................................................................................................................................... 24 
E.  Presentation and manuscript preparation .......................................................................................................... 25 
F.  Project website ........................................................................................................................................................ 25 
G.  Public announcements .......................................................................................................................................... 25 
 
VI. Financial Policies and Procedures ................................................................................................................ 26   
A.  General financial management ............................................................................................................................ 26   
B.  Purchasing  .............................................................................................................................................................. 26   
C.  Subcontractor invoicing ........................................................................................................................................ 26   
D.  Participant payment  ............................................................................................................................................. 26   
 
VII. Reporting Requirements ................................................................................................................................ 28   
A.  PCORI reporting ................................................................................................................................................... 28   
B.  Institutional reporting ............................................................................................................................................ 28   
C.  Risk Management and Adverse Event Reporting ............................................................................................. 28   
D.  Research and financial misconduct  ................................................................................................................... 28 
E.  Study completion and closeout procedures ....................................................................................................... 28   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

1 
 
I. Overview 
 
A. Project aims 
 
The VOICe project, funded by the Patient-Centered Outcomes Research Institute, will determine if group 
concept mapping (GCM), a structured brainstorming process, is a comprehensive and efficient method of 
identifying outcomes that are important to patients by comparing it to one-on-one interviews (the "gold 
standard").  The overarching objective of this methodology study is to provide high quality evidence to 
researchers, clinicians and policy makers about two methods used to elicit patient preferences for outcomes in 
order to inform best-practices for designing patient-centered research studies. 
Aim 1: To engage patients to identify their own patient-important outcomes for managing their chronic 
conditions through 1) one-on-one interviews and 2) concept mapping groups. 
 
Aim 2: To compare the comprehensiveness of concept mapping to one-on-one interviews for identifying 
patient-important outcomes for managing their chronic conditions. 
 
Aim 3: To compare the efficiency of concept mapping to one-on-one interviews for identifying patient 
important outcomes for managing their chronic conditions. 
 
B. Primary Outcomes 
 
Primary outcomes:  
1. Comprehensiveness of interviews as compared to one concept mapping group 
2. Comprehensiveness of interviews as compared to three concept mapping groups 
3. Comprehensiveness of concept mapping 
 
Secondary outcomes: Efficiency of interviews compared to GCM  
 
C. Timeline and key milestones 
 
Project timeline: January 1, 2016- December 31, 2018. 
 
Year 1: 
 All training and interview activities will occur during the first year of the project. 
Year 2: 
Final interview analysis and concept mapping sessions will occur during the second 
year.   
Year 3: 
Abstract preparation will and dissemination of the results will occur in the final year 
of the project. 
 
The full project timeline may be found on the project’s Sharepoint site 
(https://tjuv.sharepoint.com/sites/PCORI/_layouts/15/start.aspx#/SitePages/Home.aspx) 
Project milestones: 
Milestone Name 
Description 
Projected 
Completion Date 
A 
Contract Start Date 
- 
1/1/2016 
B1 
PAKSAB: Interview material preparation 
• Develop updated protocol 
• Develop study procedure manuals 
• Develop recruitment materials 
• Develop interview guide 
3/31/2016 

VOICe Study Procedures Manual V.6 
2 
 
B2 
IRB approval 
• Receive initial IRB approval for awarded 
study 
3/31/2016 
B3 
Pilot interviews 
• Train study personnel 
• Conduct pilot interviews (n= 9-12) 
6/30/2016 
B4 
PAKSAB: Interview pilot and guide review 
• Review pilot results 
• Refine interview guide 
• Submit to PCORI 
6/30/2016 
B5 
Completion of 25% of interview recruitment and 
participation 
• Recruit interview participants 
• Complete open-ended, semi-structured 
qualitative interviews (total goal n=90-120, 
25% goal n=22-30) 
• Generate interim progress report for PCORI 
6/30/2016 
B 
Report Submission 
Submit Progress Report, Using Interim 
Progress Report template 
6/30/2016 
C1 
Completion of 50% of interview recruitment and 
participation 
• Recruit interview participants 
• Complete open-ended, semi-structured 
qualitative interviews (total goal n=90-120, 
50% goal n=45-60) 
9/30/2016 
C2 
PAKSAB: Initial interview transcripts and codebook 
generation 
• Review initial interview transcripts 
• Generate codebook 
9/30/2016 
C3 
Dissemination 
• Publish initial project-specific website with 
detail including project goals, team members, 
and contact information 
9/30/2016 
C4 
Completion of 25% of interview analysis 
• Complete coding and initial analysis of 25% 
participant interviews  (total goal n=90-120, 
25% goal n=22-30) 
12/31/2016 
C5 
PAKSAB: Interview coding results and interim report 
• Review initial interview coding results 
• Generate interim report 
12/31/2016 
C6 
Completion of 75% of interview recruitment and 
participation 
• Recruit interview participants 
• Complete open-ended, semi-structured 
qualitative interviews (total goal n=90-120, 
75% goal n=67-90) 
12/31/2016 
C 
Report Submission 
Submit Progress Report, Using Interim 
Progress Report template 
12/31/2016 
D1 
PAKSAB: Concept Mapping material preparation 
• Develop Concept Mapping prompt 
• Plan recruitment process 
3/31/2017 
D2 
Completion of 100% of interview recruitment and 
participation 
• Recruit interview participants 
• Complete open-ended, semi-structured 
qualitative interviews (total goal n=90-120) 
3/31/2017 
D3 
Completion of 75% of one-on-one interview analysis 
• Complete coding and initial analysis of 75% 
participant interviews  (total goal n=90-120, 
75% goal n=67-90) 
3/31/2017 
F6 
Dissemination of results 
• Submit CM manuscript 
• Present interview abstract in at least 1 
national conference (pending acceptance 
from submissions with target date 9/30/17)) 
• Update project-specific website within 
Jefferson domain 
 
 
 
6/30/2018 

VOICe Study Procedures Manual V.6 
3 
 
D4 
Completion of 100% of interview analysis 
• Complete coding and initial analysis of 
100% participant interviews  (total goal n=90- 
120) 
6/30/2017 
D5 
PAKSAB: Interim Report 
• Generate interim progress report for PCORI 
6/30/2017 
D 
Report Submission 
Submit Progress Report, Using Interim Progress 
Report template 
6/30/2017 
E1 
Interview abstract submission and manuscript 
preparation 
• Prepare and submit abstract from interview 
data 
• Begin preparation of manuscript from 
interview 
 
9/30/2017 
E2 
PAKSAB: Interview analysis and abstract 
• Complete final interview analysis 
• Submit interview abstract to PCORI 
9/30/2017 
E3 
Concept Mapping preparation 
• Complete recruitment of 25 participants per 
group (n=75) 
• Schedule CM groups 
• Train study personnel in CM methods 
 
9/30/2017 
E4 
Concept Mapping data collection 
• Conduct 3 sessions for each CM group (n= 
3CM groups, n= 60 participants) 
12/31/2017 
E5 
PAKSAB: CM result discussion 
• Discuss initial CM results 
12/31/2017 
E6 
Dissemination of results 
• Update project-specific website within 
Jefferson domain – include members of team, 
overall project goals, links to any publications 
(once available) 
 
12/31/2017 
E 
Report Submission 
Submit Progress Report, Using Interim 
Progress Report template 
12/31/2017 
F1 
Analysis Aims 2 & 3: Start 
• Begin assessments for Aims 2 & 3 
(comprehensiveness and efficiency) 
3/31/2018 
F2 
PAKSAB Y3 Q1: Synthesis and reporting 
Convene PAKSAB to complete the following: 
• Discuss final CM results 
• Prepare/submit CM abstract 
• Begin preparation of CM manuscript 
• Begin development of dissemination plan 
with DAT
 
 
 
3/31/2018 
F3 
Analysis Aims 2 & 3: Finalize 
• Complete assessments for Aims 2 & 3 
(comprehensiveness and efficiency) 
6/30/2018 
F4 
CM manuscript completion 
• Complete draft of CM manuscript 
6/30/2018 
F5 
PAKSAB Y3 Q2: Synthesis and reporting 
Convene PAKSAB to complete the following: 
• Review CM manuscript and decide on 
journal for submission 
• Refine results for Aims 2 & 3 
• Develop dissemination plan with DAT 
• Plan next steps planning 
 
 
 
6/30/2018 

VOICe Study Procedures Manual V.6 
4 
 
F7 
PAKSAB: Results delivery 
• Deliver CM results to PCORI 
• Deliver results from Aims 2& 3 to PCORI 
• Generate interim progress report for PCORI 
 
6/30/2018 
F 
Report Submission 
Submit Progress Report, Using Interim 
Progress Report template 
6/30/2018 
G1 
Abstract and manuscript for Aims 1,2 & 3 
• Prepare and submit abstract for Aims 1,2 & 
3 
• Prepare and submit manuscript for Aims 1,2 
& 3 
 
9/30/2018 
G2 
PAKSAB Y3 Q3: Synthesis and reporting 
Convene PAKSAB to complete the following: 
• Review CM manuscript 
• Refine results for Aims 2 & 3 
• Develop dissemination plan with DAT 
• Plan next steps 
• Review manuscript for Aims 2 & 3 
 
 
 
9/30/2018 
G3 
PAKSAB Y3 Q4: Synthesis and reporting 
Convene PAKSAB to complete the following: 
• Refine results for Aims 2 & 3 
• Review manuscript for Aims 2 & 3 
• Develop dissemination plan with DAT 
• Plan next steps 
 
 
12/31/2018 
G4 
PAKSAB: Generation of final research and progress 
reports 
• Generate final progress report 
• Initiate final research report 
 
12/31/2018 
G5 
Dissemination of results 
• Submit manuscript from Aims 2 and 3 
• Revise/resubmit any other manuscripts as 
applicable (depending on prior results of peer 
review) 
• Present CM abstract in at least 1 national 
conference (pending acceptance based on 
submissions with target date 6/30/18) 
• Updating of project website with final results 
 
 
 
 
12/31/2018 
G 
Final Progress Report 
Submit Final Progress Report, Using Final 
Progress Report Template 
12/31/2018 
H 
Research Project Period End Date 
12/31/2018 
I 
Draft Final Research Report Submission 
Submit Draft Final Research Report 
according to instructions found here: 
http://pcori.org/awardee-resources 
6/30/2019 
J 
Final Research Report 
Upon receipt of written summary, and as 
applicable, PI will make revisions and submit 
Draft Final Research Report for acceptance in 
accordance to PCORI policy and process. 
12/31/2019 
K 
Approval / sign off of the Lay Abstract 
Sign off must be no later than 90 days 
beyond the date PCORI accepts the final 
research report 
See Description 
L 
Contract Term Date 
- 
12/31/2019 
M 
Final Expenditure Report 
Submit Final Expenditure Report (See 
Contract for Instructions) 
90 days from 
Contract Term Date 
N 
Notification of Public Acceptance 
See Contract for Instructions 
Within 30 Days of 
Acceptance 

VOICe Study Procedures Manual V.6 
5 
 
 
D. Team member roles, responsibilities and contact information   
 
Patient Advocates And Key Stakeholder Advisory Board (PAKSAB): The PAKSAB includes patient 
advocates who are experienced with helping chronically disabled and disadvantaged populations negotiate 
various healthcare settings as well as key representatives from diverse domains of the care pathway.  Members 
of the PAKSAB have been involved with the proposal from inception. Staff will provide updates at PAKSAB 
meetings, which are held quarterly.  
Research Team:  The research team includes all of the personnel listed below, who will work to complete 
the project objectives.  Some PAKSAB members will also participate as part of the research team by 
conducting interviews and concept mapping sessions, assisting with analysis and data interpretation, and 
disseminating the results.   
 
Name 
Role 
Department 
Kristin Rising, MD, MS 
Principal Investigator 
Emergency Medicine 
Marianna LaNoue, PhD 
Principal Investigator 
Family Medicine 
Judd Hollander, MD 
Co-Investigator 
Emergency Medicine 
Brendan Carr, MD, MA, MS 
Co-Investigator 
Emergency Medicine 
Geoffrey Mills, MD, PhD 
Co-Investigator 
Family Medicine 
Amy Cunningham PhD, MPH 
Project Manager 
Family Medicine 
Alexzandra Gentsch, LSW 
Project Coordinator 
Emergency Medicine 
Amanda Doty, MS 
Research Assistant 
Emergency Medicine 
Lori Latimer, LSW 
Research Assistant 
Emergency Medicine 
 
Contact info for Dr. Kristin Rising: 
Phone: 215-503-5507 
Email: Kristin.Rising@jefferson.edu 
 
Additionally, Jefferson students may participate in the project through the Emergency Department’s 
Academic Associates Program, the Dean’s Summer Research Program, and the Jefferson College of 
Population Health MPH clerkship/capstone program.  
Research team meetings are typically held on Fridays 10 AM-11:30 AM, in the Thomas Jefferson University 
Department of Emergency Medicine conference room.  
More information about the PAKSAB members and research team may be found on the VOICe website: 
http://www.jefferson.edu/university/skmc/research/voice.html 
Administrative Roles: Administrative roles for the grant are below: 
Role 
Department 
Financial Analyst/Grants Administrator 
Emergency Medicine 
Administrator 
Emergency Medicine 
Administrator 
Family Medicine 
Grants Administrator 
Research Administration 
Center of Excellence (RACE) 
 
 

VOICe Study Procedures Manual V.6 
6 
 
II. Training 
 
A. Research Staff 
All research members will complete the following trainings:  
• 
CITI training (basic human subjects protection, good clinical practice, responsible conduct of 
research-- must be taken prior to an individual starting research, and re-taken every two years): 
https://www.citiprogram.org/  
• 
Conflict of interest disclosure (must be completed annually): 
http://www.jefferson.edu/university/counsel/coi.html 
• 
Healthstream training (required modules vary by role): http://healthstream.tjuh.net/index.cfm 
Members involved in coding for interviews 
• 
NVivo (qualitative analysis software): The PIs will train research staff in NVivo. Additional NVivo 
resources are available here: http://www.qsrinternational.com/learning 
Project Coordinator and Research Assistants 
• 
EPIC (Needed for patient screening) 
• 
Concept mapping method: The PIs will train research staff in concept mapping. 
• 
Concept Systems Inc.: The PIs will train research staff how to use software needed for conducting 
analysis of concept mapping sessions. Concept mapping resources are available here: 
http://www.conceptsystems.com/trainings/ 
 
Additionally, the Jefferson Clinical Research Institute (JCRI) offers a full-day clinical research coordinator 
training and other periodic trainings on topics relevant to research staff. Updates are sent via the Office of 
Research Administration’s listserv: https://lists.jefferson.edu/mailman/listinfo/ora.clinical 
 
B. PAKSAB members  
i. General: All PAKSAB members will receive an orientation to the project, including its objectives, methods 
and key milestones involving the PAKSAB.  
 
ii. Members serving as researchers will complete basic training as identified above, depending on role. 
 
C. Data management plan 
 
i. Data types, storage and person(s) responsible: 
 
Data  
Storage 
Person(s) Responsible 
Research team documents (agendas, 
minutes, draft materials for discussion, 
timeline, presentations, manuscripts) 
Sharepoint (Web based secure file 
sharing system) 
 
Research team 
PAKSAB materials 
 
Sharepoint; also disseminated by 
email 
Project Coordinator, RA  
Eligible participant lists  
Selectrus (Imported from Epic) 
 
IT, Project Coordinator, 
RA 
Patient medical record 
 
Epic 
Project Coordinator, RA 
Patient Tracker (name, MRN, date 
approached, date enrolled, 
interview/concept mapping date, study 
ID, type of data) 
REDCap 
Project Coordinator, RA 
Ineligible/Refusal lists  
One Drive (Web based secure file 
Research team 

VOICe Study Procedures Manual V.6 
7 
 
sharing system) 
Consent forms 
Locked cabinet  
Project Coordinator 
Demographic forms 
Locked cabinet 
Project Coordinator 
Interview audio recordings 
One Drive 
Project Coordinator 
Interview transcripts 
One Drive 
Research team 
Concept mapping data (statements, 
sorting, rating, maps) 
Concept Systems Inc. software, web 
based platform 
Research team 
Time tracking  
Excel spreadsheet on Sharepoint 
Research team 
IRB documents (application, renewals) 
Sharepoint 
RA 
PCORI progress reports, research 
report 
Sharepoint 
Co-I’s, Project Manager 
Grant financial records 
RACE, PCORI 
RACE  
Project website 
Thomas Jefferson University 
website 
Project Manager, Marketing 
 
ii. Metadata 
Metadata are documentation for your data set. The VOICe project metadata includes the Patient Tracker in 
REDCap. 
 
 
iii. Access 
The research team—including PAKSAB members, students and others added as key personnel, will have 
access to study data as outlined in the IRB-approved protocol (15G.667). All research personnel must follow 
the data security steps outlined in section v. New research team members must be added to the study IRB via 
a new personnel amendment (http://www.jefferson.edu/university/human_research/forms.html) prior to 
viewing or collecting any study data.  
 
iv. Data sharing  
Research team members should discuss any data sharing requests with the PIs prior to sharing any procedural 
documents, data, or analytic study materials. Any data sharing must comply with the data security procedures 
below. 
 
v. Data security and IRB compliance 
Research personnel will follow the data entry, storage and security procedures outlined in the interview and 
concept mapping protocols, as well as the general guidance provided by the IRB. 
 
According to the IRB, the following steps must be taken to ensure identifiable data remains confidential and 
secure: 
a. If an individual is a Jefferson patient and research subject, a separate medical record must be 
maintained apart from any records required for the research. 
b. There are 18 identifiers described in 45 CFR 164.514 that make data identifiable.  To be considered 
de-identified, data must not contain any of the identifiers (also see OHR-5 for list of identifiers). 
c. When not in use, identifiable data should be stored in a locked cabinet or desk in a locked room. 
d. Access to the data should be limited.  Only the individuals who need the data should have access. 
e. If hardcopies of identifiable data must be taken to another building, a locked container such as a 
banker bag should be used.  The container should be marked with instructions for returning the 
container if misplaced. 
f. If hardcopies of identifiable data must be mailed, there must be a contract in place which specifies 
the method of doing this.  The data should be placed in one envelope inside of another envelope. 
Both envelopes should have tamper-evident seals and should be addressed to the specific recipient.  
Signatures should be required for receipt, or lockable mailboxes should be used. 

VOICe Study Procedures Manual V.6 
8 
 
g. If research data is stored on your Jefferson computer, encryption software must be installed on the 
computer.  Contact IT if you are not sure if the encryption software is installed. 
h. PHI may be emailed between Jefferson email addresses.  Jefferson email must not be sent from or 
forwarded to a non-Jefferson email address such as your personal email. 
i. 
Research data should not be stored on portable devices including laptops.  If research data must be 
stored on a portable device, contact IT.  Please see Jefferson University Policy 122.35 for complete 
information.  
j. 
External monitors must not view the eMR.   
 
vi. Backup, archiving  
 
All documents on Sharepoint and One Drive are automatically backed up by IT. The Project Coordinator will 
also periodically back up all Sharepoint and One Drive documents on a secure Jefferson computer. 
 
PCORI requires that “financial records, supporting documents, statistical records, and other records relevant 
to this Contract and performance under it must be retained by the Recipient for a period of five (5) 
years from the a) Contract Term Date, b) date of the final payment under this Contract, or c) 
conclusion of any audit or litigation related to this Contract, whichever is later.” 
 

VOICe Study Procedures Manual V.6 
9 
 
III. Interview Protocols 
 
Interviews and screening will take place in three different settings, with individual protocols listed separately 
below. Financial remuneration, interview time tracking and data entry and storage will be similar in all settings 
and outlined in Section D (page 15). 
 
A. Jefferson Emergency Department Interview Protocol 
 
A i. ED overview, triage process and patient flow 
Patients can enter the Emergency Department in several ways: 
1. Patients can come into the emergency department though the main entrance into the waiting room.  
• 
First they are “intaked”, which is a mini-registration process. 
• 
Then they are triaged by the triage clinician. 
• 
Then they are registered. 
• 
Patients wait until they are called into a room or hallway bed in the ED. 
2. Patients can come into the ED through “Fire Rescue” and are taken directly to the center desk.  
• 
First they are evaluated by the Senior Resident or Triage Nurse. 
• 
Then they are taken into a room or brought to a hallway bed. 
• 
A nurse registers them. 
• 
Then they receive formal triage 
3. Patients can come into the emergency department and go straight into the Trauma Bay where they are 
actively worked on by the hospital staff.  
4. After patients are in rooms (or hallway beds) and have been seen by the doctor it is determined which of 
several places they will go next: 
• 
Discharged home. 
• 
Placed in the Observation Unit.  
• 
Admitted to the hospital. 
• 
Transferred to another hospital or facility. 
 
A ii. Screening process 
Screening will take place during a random selection of days and times of day to ensure that we capture the full 
range of ED patients. Staff interviewers will sign up for three-hour shifts scattered on different days and at 
different times of day: morning, afternoon, evening, and night. Each PAKSAB interviewer will complete a 
minimum of two ED interviews. The research team will screen for eligible patients through the electronic 
medical record system. The research team uses a random generator to determine which patient to approach 
for enrollment, and will consult with the attending physician and nurse to determine the best time to 
approach the patient for enrollment.  
 
Inclusion criteria  
• 
Adult patient (age 18 and older) 
• 
English speaking 
• 
Capable of providing informed consent 
• 
In the Jefferson Emergency Department (ED) for a type 1 or type 2 diabetes-related problem 
determined to require medical treatment: diabetic ketoacidosis, hypo/hyperglycemia, soft-tissue 
infections/cellulitis, neuropathy, retinopathy, acute renal failure, chest pain, and receiving diabetes 
medication refills.  
• 
Patient has a less than fifty percent chance of being admitted to the hospital. 
 

VOICe Study Procedures Manual V.6 
10 
 
Exclusion criteria  
• 
Patient is newly-diagnosed with diabetes 
• 
Pregnancy 
• 
Patient has had a significant permanent complication related to diabetes mellitus (DM) including: 
• 
End stage renal disease/on dialysis/history of kidney transplant 
• 
History of amputation 
• 
Blindness related to diabetes complication 
• 
Patient undergoing medical clearance for a detox center or any involuntary court or magistrate 
             order 
• 
Patient in police custody or currently incarcerated individual 
• 
Patient who has, in their clinician's best judgment, major communication barriers such as visual 
or hearing impairment or dementia that would compromise their ability to give written informed 
consent 
 
After initial explanation of what the study is about, the research team will review an additional screening form 
to ensure exclusion criteria will not prevent them from participating in the study. The interviewer should 
document in the One Drive Excel sheet when patients a) initially appear eligible but are then found to be 
ineligible or b) decline participation, and list the reason for ineligibility/refusal.  The interviewer will also 
document the reason for ineligibility/refusal along with demographic information (gender, age, race, and 
ethnicity) to assess for selection bias. 
 
A iii. Scheduling process  
Interviews of patients in the acute care setting will be performed in-person in a private space within the ED at 
a time that is convenient for the patient and least disruptive to their care. This may be the patient room, the 
family room, or an ED doctor’s office. Interviewers will assign a study ID to each participant (ED participant 
ID numbers are in the 200’s). 
 
A iv. Consent process 
Prior to the interview, the research team member will conduct an informed consent discussion. By speaking 
to the patient (and when appropriate, to the patient’s clinician), the research team member will confirm that 
the patient has the current mental status and capacity to provide informed consent for this study.  
 
Obtaining consent will occur in a private setting. The research team member will inform the patient of the 
nature of the study, the study purpose, the study procedures, the risks, and the alternatives to participation, 
including the alternative to not participate. Potential participants will be given time to read the consent form 
and ask questions before making their decision. The research team member will verbally confirm that each 
interested patient understands the risks, purpose, procedure, and nature of the study before asking for 
consent.  
 
Patients who are interested will complete the informed consent discussion and may then decline or consent to 
participation in writing on an IRB-approved Informed Consent with HIPAA Authorization Form, prior to 
any further data collection.  Participants will be given a copy of the informed consent form; the original will 
be kept on file in the research office.  
 
Patients who are unable or prefer not to provide informed consent cannot be enrolled in this study. 
 
A v. Interview process 
Researchers will conduct the interview following the interview guide. Interviews will generally last about 30 
minutes and will be recorded using two recording devices simultaneously. 

VOICe Study Procedures Manual V.6 
11 
 
 
Interviews should be done towards the end of an ED visit or after being discharged. Interviewer will check in 
with both attending doctor AND nurse to see when is an appropriate time to do the interview during 
patient’s ED visit (can it be done during the visit without interfering or prolonging the visit?). 
 
After the interview is completed, researchers will ask the participants to complete the demographic form. 
Interviewers will then log the assigned study ID number and interview date in the study log.   
 
vi. Financial Remuneration, Time Tracking, and Data Entry and Storage 
See section D (page 15) for details.  
 
B. Jefferson Family Medicine Associates Interview Protocol 
 
B i. Jefferson Family Medicine Associates (JFMA) overview and patient flow 
Jefferson Family Medicine Associates (JFMA)’s main practice is located at 833 Chestnut Street, 3rd floor. The 
practice has nearly 80 family medicine faculty members and residents, 4 geriatric fellows, 2 sports medicine 
fellows, 7 nurse practitioners, and nearly 100 staff. JFMA is a Level III patient-centered medical home and 
has participated in the Southeastern Pennsylvania Chronic Care Initiative and other quality efforts.  
 
JFMA has approximately 35,000 patients making more than 80,000 visits each year. Although the population 
is remarkably diverse with patients from every socioeconomic group, JFMA maintains a special interest in the 
healthcare needs of marginalized communities.  More than half of JFMA patients are from minority groups, 
the majority of which are African Americans. Most of the visits to the practice are from individuals from 
medically underserved communities in Philadelphia. 
 
B ii. Screening process  
Research staff members will complete JFMA interviews each week; PAKSAB interviewers will complete a 
minimum of two JFMA interviews over the course of the study. The research team will screen the schedule 
of upcoming appointments at JFMA, available through Selectrus, on Monday and Wednesday of each week 
and will identify potentially eligible patients for enrollment who have visits that week. Research team 
members will verify each patient’s eligibility via EPIC, and may contact the patient’s provider if any questions 
arise regarding the patient’s eligibility for the study.  
To identify potential JFMA interview participants 
• 
Log in to Selectrus with your campus key and password 
• 
The following are the data elements of this report: 
o Patient’s MRN (EPIC) 
o Patient name 
o Patient’s date of birth 
o Patient’s age as of today (confirmed date calculation is correct) 
o Patient’s phone number (only one available in the data) 
o ICD9 codes from list provided that are active in patient’s record 
o ICD10 codes form list provided that are active in patient’s record 
o Most recent A1c result 
o Date of most recent A1c result 
o Second most recent A1c result 
o Date of second most recent A1c result 
o Yes/no for two or more A1c results greater than 7.5 within the past year 
o Patient’s next appointment date 
o Patient’s next appointment time 
o Provider scheduled for patient’s next appointment 

VOICe Study Procedures Manual V.6 
12 
 
• 
The following are the finalized specifications/inclusion criteria of this report: 
o Patients 18 years or older (as of today) who have an active diagnosis of diabetes on their 
problem list (ICD9 diagnosis of 250.XX OR ICD10 diagnosis of E11.XX or E10.XX), who 
have a HbA1c result >= 7.5 at least once within the past year and who have an appointment 
scheduled at the JFMA (833 Chestnut) location 
 
Inclusion criteria  
• 
Adult patient (age 18 and older) at JFMA (833 Chestnut)  
• 
Active diagnosis of type 1 or type 2 diabetes on problem list (ICD9 diagnosis of 250.XX OR ICD10 
diagnosis of E11.XX or E10.XX) 
• 
English speaking 
• 
Capable of providing informed consent 
• 
Scheduled for a routine follow-up visit at JFMA and has at least 1 measurement of HgbA1C > 7.5 in 
the prior year (with preference given to patients with 2 measurements of HgbA1C>7.5 in the past 
year). 
 
Exclusion criteria  
• 
Pregnancy 
• 
Patient has had a significant permanent complication related to DM including: 
• 
End stage renal disease/on dialysis/history of kidney transplant 
• 
History of amputation 
• 
Blindness related to diabetes complication 
• 
Patient undergoing medical clearance for a detox center or any involuntary court or magistrate 
             order 
• 
Patient in police custody or currently incarcerated individual 
• 
Patient who has, in their clinician's best judgment, major communication barriers such as visual 
       or hearing impairment or dementia that would compromise their ability to give written informed  
       consent 
• 
Newly diagnosed with diabetes at last doctor visit 
After initial explanation of what the study is about, the research team will review an additional screening form 
to ensure exclusion criteria will not prevent them from participating in the study. The interviewer will 
document in the One Drive Excel sheet when patients a) initially appear eligible but are then found to be 
ineligible or b) decline participation. The interviewer will also document the reason for ineligibility/refusal 
along with demographic information (gender, age, race, and ethnicity) to assess for selection bias.     
  
B iii Scheduling process 
The research team uses a random generator to determine which eligible patients to approach for enrollment, 
The project coordinator will call eligible patients 1-2 days before their appointment to explain the study and 
determine the patient’s eligibility and interest in enrolling. On the day of the visit, a research team member 
will be scheduled by the project coordinator to approach the eligible patient either before or after their 
doctor’s visit to obtain consent and conduct the interview. Interviewers will assign a study ID to each 
participant (JFMA participant ID numbers are in the 100’s) at the time interviews are done. 
 
B iv. Consent process 
When the research team member meets with a scheduled patient, he/she will obtain informed consent. The 
research team member will confirm that the patient has the mental capacity to provide informed consent for 
this study by speaking to the patient (and when appropriate, to the patient’s clinician).  
 

VOICe Study Procedures Manual V.6 
13 
 
Obtaining consent will occur in either the JFMA small or large conference room. The research team member 
will inform the patient of the nature of the study, the study purpose, the study procedures, the risks, and the 
alternatives to participation, including the alternative to not participate. Potential participants will be given 
time to read the consent form and ask questions before making their decision. The research team member 
will verbally confirm that each interested patient understands the risks, purpose, procedure, and nature of the 
study before asking for consent.  
 
Patients who are interested will consent to participation in writing prior to any further data collection.  
Participants will be given a copy of the informed consent form; the original will be kept on file in the research 
office.  
 
Patients who are unable or prefer not to provide informed consent cannot be enrolled in this study. 
 
B v. Interview process 
Interviews will take place in a private setting within Jefferson Family Medicine Associates.  
Researchers will conduct the interview following the interview guide. The interview will be taped using a 
digital recording device. Interviews will generally last about 30 minutes and will be recorded using two 
recording devices simultaneously. 
 
After the interview is completed, researchers will ask the participants to complete the demographic form. 
Interviewers will then log the assigned study ID number and interview date in the study log.   
 
B vi. Financial Remuneration, Time Tracking, and Data Entry and Storage 
See section D (page 15) for details.  
 
C. Post-Hospital Discharge Interview Protocol  
 
C i. Hospital discharge overview  
Jefferson Family Medicine Associates patients admitted to Thomas Jefferson University Hospital are admitted 
to JFMA’s inpatient service. After their discharge, if they are discharged to home they are contacted by a 
JFMA transition of care coordinator to schedule a follow-up appointment at JFMA. The processes are the 
same for Jefferson Internal Medicine Associates (JIMA) patients. 
 
C ii. Screening process 
Patients enrolled for the post-acute period will be contacted by phone within one week after discharge from 
the Thomas Jefferson University Hospital. The research team will screen the list of JFMA/JIMA patients 
recently discharged from JFMA/JIMA’s inpatient services, available through Selectrus, on Monday and 
Thursday of each week to identify potentially eligible patients.  
 
To identify potential post-hospital discharge interview participants 
• 
Log in to Selectrus with your campus key and password: http://reports.tjh.tju.edu/rs/login/auth 
• 
Next to “Patient Centered Outcomes Research Institute - Diabetes Inpatients JFMA Service” report, 
select “Run Now” and select a “START DATE” and “END DATE” based on your discharge dates 
of interest 
• 
The following are the data elements of this report: 
o Patient’s account number 
o Patient’s MRN (EPIC) 
o Patient name 
o Patient’s date of birth 
o Patient’s age as of today (calculation corrected) 

VOICe Study Procedures Manual V.6 
14 
 
o All available phone numbers for patient 
o Patient’s admit date/time 
o Patient’s discharge date/time 
 
If the patient has not yet been discharged, field will be blank 
o Discharge disposition description 
 
If the patient has not yet been discharged, field will be blank 
o Discharge Diagnoses  
 
If the patient’s admit has not yet been coded, field will be blank 
o Name of where patient is discharged to 
• 
The following are the finalized specifications/inclusion criteria of this report: 
o Patients aged 18 years or older (as of today) who have been admitted to the JFMA service  
• 
Note: the list will include all admitted JFMA and JIMA patients, only some of whom have 
diabetes—staff will need to confirm the patient’s diabetes diagnosis using the patient’s 
active problem list or past medical history in EPIC.  
 
Inclusion criteria  
• 
Adult patient (age 18 and older) 
• 
Active diagnosis of type 1 or type 2 diabetes on problem list/in past medical history (ICD9 diagnosis 
of 250.XX OR ICD10 diagnosis of E11.XX or E10.XX) 
• 
English speaking 
• 
Capable of providing informed consent 
• 
Diagnosis of moderately to poorly controlled diabetes mellitus (DM) defined as follows (for 
interview groups):  
• 
Patient was discharged from either the Jefferson Family Medicine Associates (JFMA) or Jefferson 
Internal Medicine Associates (JIMA) hospital services within the past 7 days after admission for a 
diabetes-related problem including: Stroke/transient ischemic attack (TIA), gastroparesis, 
hyperglycemia/diabetic ketoacidosis, hypoglycemia, renal failure, chest pain/cardiac problem, 
neuropathy, retinopathy, infections (of any kind) - including but not limited to: urinary tract 
infection/pyelonephritis, osteomyelitis, cellulitis, colitis, gastroenteritis) 
• 
Discharged in the last 7 days 
 
Exclusion Criteria  
• 
Pregnancy 
• 
Patient has had a significant permanent complication related to DM including: 
• 
End stage renal disease/on dialysis/history of kidney transplants 
• 
History of amputation 
• 
Blindness related to diabetes complication 
• 
Patient undergoing medical clearance for a detox center or any involuntary court or magistrate 
              order 
• 
Patient in police custody or currently incarcerated individual 
• 
Patient who has, in their clinician's best judgment, major communication barriers such as visual 
or hearing impairment or dementia that would compromise their ability to give written informed 
consent 
• 
Newly diagnosed with diabetes at last hospital visit 
• 
Disposition to hospice  
• 
Primary reason for admit is due to trauma or non-diabetes related health problem 
• 
Admitted anywhere other than center city hospital 
 

VOICe Study Procedures Manual V.6 
15 
 
The project coordinator will consult with one of the PIs regarding the eligibility of post-hospital discharge 
individuals for the study. After initial explanation of what the study is about, the research team will review an 
additional screening form with patients to ensure exclusion criteria will not prevent them from participating 
in the study. The project coordinator or interviewer will document in the One Drive Excel sheet when 
patients a) initially appear eligible but are then found to be ineligible or b) decline participation. The 
interviewer will also document the reason for ineligibility/refusal along with demographic information 
(gender, age, race, and ethnicity) to assess for selection bias.   
 
C iii. Scheduling process 
The research team will contact eligible patients once they are discharged from the hospital, within 7 days of 
discharge, to inquire about interest in study participation, using the IRB-approved email and phone scripts. 
Interviewers will assign a study ID to each participant (post-discharge participant ID numbers are in the 
300’s). 
 
C iv. Consent process  
At the time of the phone call, the research team member will seek informed consent. The research team 
member will assess that the patient has the mental capacity to provide informed consent for this study by 
speaking to the patient (and when appropriate, to the patient’s clinician). 
 
The research team member will inform the patient of the nature of the study, the study purpose, the study 
procedures, the risks, and the alternatives to participation, including the alternative to not participate. The 
research team member will verbally confirm that each interested patient understands the risks, purpose, 
procedure, and nature of the study before asking for verbal consent.  
 
Participants will be mailed a copy of the informed consent form; the original will be kept on file in the 
research office. Subjects who are interested in participating but prefer to come in for the interview will be 
consented at the time of their interview with an in-person interview consent form.   
 
Participants who are unable to or do not provide informed consent cannot be enrolled in this study. 
 
C v. Interview process 
Researchers will conduct the interview using an interview guide developed by the research team. The 
interview will be taped using a digital recording device. Interviews will generally last about 30 minutes.  
 
Interviews can be conducted in specified PI’s offices during normal business hours and at the research 
assistant’s desk after normal business hours. Phone interviews will be recorded using one recording device 
and in person interviews will be recorded using two recording devices simultaneously. 
 
After the interview is completed, study staff will ask the participants to complete the demographic form. 
Interviewers will then log the assigned study ID number and interview date in the study log.   
 
C_vi. Financial Remuneration, Time Tracking, and Data Entry and Storage 
See page 15—note that for phone interview participants, remuneration is via gift card.  
 
D. Financial Remuneration, Time Tracking, and Data Entry and Storage for all Interviews 
 
D i. Financial remuneration for participants 
There is payment remuneration for time and effort spent participating in this study.  Interview participants 
will be compensated with $25 at the end of the interview. In-person study participants will be remunerated 
using cash or clincards (see “Participant payment” in Financial Policies and Procedures section). Participants 
receiving payment must complete a W-9 and receipt. If the interview is done by phone, patients will be asked 

VOICe Study Procedures Manual V.6 
16 
 
if they would like a CVS, Wawa, or Walgreens gift card. Gift cards will be mailed to patients when done over 
the phone. (see page 26 for more details)   
 
D ii. Interview time tracking procedure  
Interviewers will track the time needed to complete each step of the interview process in an Excel 
spreadsheet on One Drive. 
This will include at a minimum: 
• 
How much time you spend calling patients to set up interviews 
• 
Time spent traveling to Jefferson to do interviews 
• 
Time spent reviewing consent forms with patients 
• 
Time spent doing actual interviews 
 
D iii. Data entry and storage  
In addition to demographic data collected at the end of each interview, patient records will be reviewed by 
trained research personnel with use of the medical record number (MRN) to determine: 
• 
Date of birth 
• 
Information on hospital and outpatient utilization in the prior 12 months, including date of visit, 
location of visit (i.e primary care office, ED, inpatient), date of discharge if applicable, reason for 
visit, and discharge diagnosis 
• 
Information on the visit associated with this study, including location of the visit, time and day of 
arrival, reason for visit, treatments provided, discharge diagnosis and instructions, and discharge 
disposition and time 
• 
Insurance status (and provider, if applicable) 
• 
Chronic medical conditions  
 
All paper documents will be secured in locked cabinets in the Project Coordinator’s office. Research team 
members will assign a Study ID to each participant and enter the Study ID, participant’s medical record 
number and demographic information in the Patient Tracker, which is stored in REDCap. Interview 
transcripts will be labeled with the study ID number and sent to ADA Transcriptions. Once data analysis is 
complete, audio recordings of the interview files will be destroyed. The research team will import the 
transcripts into the analytic database (NVivo 11.0 software) for coding.  
 
 
 
 

VOICe Study Procedures Manual V.6 
17 
 
IV. Concept Mapping Session Protocol 
 
i. Screening process 
Individuals for the concept mapping groups will be screened from a list of potentially eligible patients made 
available through Selectrus. Patients will be called by a research team member two weeks before the first 
GCM session takes place. Up to three phone attempts may be made leading up to the GCM session to ask a 
patient if they are interested in participating. An equal distribution of patients from each care setting will be 
recruited. 
 
To identify potential concept mapping participants 
Acute care setting:  
• 
The following are the data elements of the Selectrus report: 
o Patient’s account number 
o Patient’s MRN (EPIC) 
o Patient name 
o Patient’s date of birth 
o Patient’s age as of today (calculation corrected) 
o All available phone numbers for patient 
o Patient’s ED admit date/time 
o Patient’s ED discharge date/time 
 
If the patient has not yet been discharged, field will be blank 
o Discharge disposition description 
 
If the patient has not yet been discharged, field will be blank 
o Discharge Diagnoses  
 
If the patient’s admit has not yet been coded, field will be blank 
o Name of where patient is discharged to 
• 
The following are the finalized specifications/inclusion criteria of this report: 
o Patients aged 18 years or older (as of today) who have been admitted to the Jefferson ED 
Note: the list will include patient who has had a visit to the Jefferson ED over the past 6 who has 
diabetes (with preference for those discharged in the last 3 months) 
 
Post-acute care setting: Use Selectrus report procedure for post hospital discharge interviews with a filter for 
patients discharged from the JFMA/JIMA hospital service within the past 6 months. (page 11).  
 
Primary care setting: Use Selectrus report procedure for JFMA interviews with a filter for patients who have had 
a routine scheduled office visit within the past 6 months (page 13) 
 
Inclusion criteria  
• 
Adult patient (age 18 and older) 
• 
Active type 1 or type 2 diagnosis of diabetes on problem list (ICD9 diagnosis of 250.XX OR ICD10 
diagnosis of E11.XX or E10.XX) 
• 
English speaking 
• 
Capable of providing informed consent 
• 
Acute care setting: patient who has had a visit to the Jefferson ED over the past 6 months for a 
diabetes-related problem (with preference for those discharged in the last 3 months).  
• 
Post-acute care setting: patient was discharged from the JFMA/JIMA hospital service within the 
past 6 months for a diabetes-related problem (with preference for those discharged in the last 3 
months). 
• 
Primary care setting: patient has had routine scheduled office visit within the past 6 

VOICe Study Procedures Manual V.6 
18 
 
             months to the JFMA practice and has at least 1 measurement of HgbA1C > 7.5 in the prior 
             year (with preference given for 2 measurements of HgbA1C>7.5 in the past year).  
 
Exclusion Criteria  
• 
Pregnancy 
• 
Patient has had a significant permanent complication related to DM including: 
• 
End stage renal disease/on dialysis/history of kidney transplants 
• 
History of amputation 
• 
Blindness related to diabetes complication 
• 
Patient undergoing medical clearance for a detox center or any involuntary court or magistrate 
             order 
• 
Patient in police custody or currently incarcerated individual 
• 
Patient who has, in their clinician's best judgment, major communication barriers such as visual or 
hearing impairment or dementia that would compromise their ability to give written informed 
consent 
• 
Acute care and post-acute care settings: Primary reason for admit is due to trauma or non-diabetes related 
health problem 
 
After initial explanation of what the study is about, the research team will review an additional screening form 
to ensure exclusion criteria will not prevent them from participating in the study. The research team members 
will document in the One Drive Excel sheet when patients a) initially appear eligible but are then found to be 
ineligible or b) decline participation. The interviewer will also document the reason for ineligibility/refusal 
along with demographic information (gender, age, race, and ethnicity) to assess for selection bias.   
 
ii. Scheduling process 
Dr. LaNoue, in collaboration with participating PAKSAB members and study staff determined the dates and 
times for the concept mapping sessions: 
a. 9/23 & 9/25  
b. 11/11 &11/13  
c. 12/2 & 12/4  
 
o Patients will be contacted by phone in a random order and will be recruited for participation 
in one of the three concept mapping groups using the IRB-approved phone scripts. Each 
group will have a mix of patients from the three care settings (acute, post-acute and primary 
care). The research team will schedule up to 30 patients per group for an end target of 16-20 
total patient-participants in each group. Research team members will assign a study ID to 
each participant (Method is denoted with a “C”; Sessions are denoted by A, B, C; Participant 
setting numbers stay the same: 100’s, 200’s, 300’s (Examples: CA101, CB201, CC301)). 
 
iii. Concept mapping process 
The concept mapping process will consist of three steps that will take place over three sessions led by Dr. 
LaNoue and selected PAKSAB members. For each round of concept mapping, sessions 1 and 2 will take 
place at 833 Chestnut St on the 10th floor; session 3 will take place in the Department of Family & 
Community Medicine offices at 1015 Walnut Street, Suite 401. Before each session, the research team should 
make the preparations described below. 
 
Months Before 

VOICe Study Procedures Manual V.6 
19 
 
 Contact the facilities manager to reserve training rooms in the computer lounge with enough 
computers for each participant   
 Fill out a Project Activation Form (PAF) and email Jon Littlefield to set up a new project with the CS 
Global Max annual license (as well as one test project, if a practice run is needed) 
 If a new person will be responsible for the data analysis, designate them as the Project Administrator 
on the PAF and follow up to make sure they are set up with a log-on user name and password 
 Reserve a room for 3rd session, where participants can view the concept map, and have room for 
food  
 Submit EPIC request to generate reports containing patient contact info for recruitment  
 Create phone script for recruiting patients 
 Make a plan for how to randomize list of potential participants 
 Set up Patient Remuneration  
o Email greenphire contact with the following information:  study title, JeffTrial #, Jefferson 
account number that is paying for the study, the payment schedule, and the number of cards 
that you need 
o Pick up the remuneration cards  
o Create a log to identify which numbered ClinCard was given to each participant   
 Recruit patients (schedule up to 30 patients per group for an end target of 16-20 total in each group) 
Week before 
 Order food for all 3 sessions (Breakfast: fruit, yogurt, granola, oatmeal;  Lunch for session 2 and 
dinner for session 3: 1/2 salad, ½ wraps, sugar free cookies, fruit)  
o Make sure that tax will not be applied to the delivery if using a p-card to pay for the order 
o Keep the receipts to turn in to the p-card administrator 
 Mail letter with date, time, address, entrance to use, project coordinator and PI cell, agenda (doors 
open at x,  you must arrive by x to participate, we will be done by x ), breakfast and lunch will be 
provided, sign in at security desk and say you are here for the Concept Mapping Session  
 Double check that you have enough ClinCards 
Three days before 
 Let security know about the group session a week in advance and give a list of names “Here is a list 
of individuals who will be attending a Concept Mapping Session scheduled to be done on the x floor. 
Any questions or concerns can be directed to Dr. Kristin Rising (cell: xxx-xxx-xxxx)” 
 Print signs for outside doors 
 Assign numbers to participant names 
o Method is denoted with a “C”; Sessions are denoted by A, B, C; Participant setting stays the 
same: 100’s, 200’s, 300’s (Example: CA101) 
 
Session 1 preparation 
PREPARATION: 
Software Set Up 
1. Fill out a Project Activation Form and email Jon Littlefield to set up a new project with the CS 
Global Max annual license (as well as one test project, if a practice run is needed) 
2. If a new person will be responsible for the data analysis, designate them as the Project Administrator 
on the PAF and follow up to make sure they are set up with a log-on and password 
Food Delivery 
1. Arrange to have an appropriate amount of food delivered,  
2. Make sure that tax will not be applied to the delivery if using a p-card to pay for the order 
3. Keep the receipts and turn them in to the p-card administrator  
Patient Remuneration 

VOICe Study Procedures Manual V.6 
20 
 
1. Reach out to your Greenphire contact to get Greenphire training and your login id/password for the 
ClinCard system (this is NOT your Jeff sign-in info) 
2. Email Greenphire contact with the following information:  study title, JeffTrial #, Jefferson account 
number that is paying for the study, the payment schedule, and the number of cards that you need 
3. Pick up the remuneration cards from Greenphire contact 
Setting up the space: 
1. Talk to the security desk. Have 2-3 people waiting in the lobby of the building to greet and direct 
participants to the meeting room 
2. Set up refreshment table. Have 3+ people in room (with refreshments) to check participants in with 
sign in sheet, then to obtain informed consent, and finally to distribute demographic forms and W9s.  
Have the person who is collecting the forms check for completeness  
3. To set up the brainstorming room, make sure that the projector and computer are working, and pull 
up a blank word document to record all of the responses – immediately save the document 
 
SESSION 1: 
Materials: 
 Sign at the security desk (“Check in here”, project coordinator and PI cell) 
 Signs at both entrances of the building (“CM sessions inside”, project coordinator and PI cell” “Use 
the 9th st entrance around the corner”) 
 Sign-in sheet 
 Breakfast 
 Recorders 
 Schedule of events for team members (Roles to be assigned: 2 people as greeters, 3+ people to 
obtain informed consent, 1 research team member and 1 PAKSAB member to moderate the 
discussion, 1 person to record the brainstorming list, 1 person to accept/set up food order while 
brainstorming is in progress, 3-4 people to set up computers and explain sorting, and 2 people to 
distribute remuneration cards) 
 3 Folders, labelled as follows: Demographic Forms, W-9s + receipts, Informed Consent 
 Patient packets (2 copies of informed consent, demographic form, ClinCards/ informational 
brochures, receipt template for each card, W9)  
 ClinCard tracking log 
 pens + notecards for optional written brainstorming 
 pens for consenting 
 Printed prompt for each participant 
 Printed sorting instructions for each participant. 
 Example sorting sheet 
Steps: 
1. Check in process 
 Check in person to mark that patient is present and give name tag with assigned participant 
number  
 Consent process: Prior to the concept mapping sessions, the research team will obtain 
informed consent, either individually or in small groups. The research team member will inform 
the patient of the nature of the study, the study purpose, the study procedures, the risks, and the 
alternatives to participation, including the alternative to not participate. Potential participants will 
be given time to read the consent form and ask questions before making their decision. The 
research team member will verbally confirm that each interested patient understands the risks, 
purpose, procedure, and nature of the study before completing the consent discussion. Patients 
who are interested will consent to participation in writing on an IRB-approved informed consent 

VOICe Study Procedures Manual V.6 
21 
 
with HIPAA Authorization Form, prior to any further data collection.  Participants will be given 
a copy of the informed consent form; the original will be kept on file in the research office.  
The study team should collect demographic and W-9 forms, and check them for completion. 
2. Introduction: Start the session with PowerPoint introduction, then give the prompt.  Optional: 
Allow for 5-10 minutes of written brainstorming:  
 
Concept Mapping Introduction 
“Thank you for agreeing to participate in this research study. As we explained in the consent 
form, there are two parts to this study, which will happen on two separate days. This 
morning we’ll be doing the first activity, and then this afternoon we will do the second part. 
For this morning we are going to be brainstorming -  Brainstorming is a way we can get 
variety of ideas from a group of people – you! In a few moments, I will state the question we 
will use for this brainstorming session. Then, I will give you a few minutes to write your 
ideas, your answers to the question, on paper. Then I will ask you to share out loud any and 
all ideas that you think of. Don’t worry about repeating something that has already been said 
or saying something similar to what has already been said. There are no right answers or 
wrong answers.  
 
Two of my colleagues have joined us today and they will be writing down all of the 
responses that are said out loud.  
 
Are there any questions before we start? 
 
In the past, medical research has often focused on asking doctors what they think is most 
important in improving health care for their patients, but we have realized that patients may 
have different priorities and objectives when they seek health care. Today we would like to 
know what things patients think about when deciding whether their diabetes treatment has 
been successful. It can be anything patients consider important, including little things that 
matter on a day to day basis or things that patients might want to happen in the long term. 
We hope to use this information to improve the delivery of health care and prioritize the 
concerns patients tell us are important them. 
So with this in mind, the question I’d like to ask is: You are here as a person with diabetes; when 
people with diabetes seek care, what are they hoping to change or improve?” 
 
The moderator begins the discussion.  As the responses are listed, the moderator dictates 
responses to the recorder, and the recorder types them into a list that is projected in front of 
all of the participants.  Responses are collected until the session reaches theme saturation.   
3. After the session, the team members clarify all of the items on the list to ensure that they are written 
in a consistent format. In the GCM software, make sure that brainstorming is active (under project 
settings) and upload the list as a text (txt) file into the software. 
       4.  The Excel spreadsheet will then be uploaded to the Concept Systems Global Software and “sort      
            cards” will be generated. These cards each contain one statement generated from the brainstorming                               
 
session, and will be loaded into the Concept Systems Global Software for the second session.  
5. Two individuals should set up lunch towards the end of Session 1. 
 
SESSION 2: 
Materials: 
1. Sign-in sheet 
2. Food 
3. Meeting space with a computer for each participant  
4. Sorting example sheet for each participant 

VOICe Study Procedures Manual V.6 
22 
 
Prep Steps: 
1. Add participant ID into the software 
2. Set up a participant ids (these must be unique for each project that is put on the same license) in the 
system for each participant within the CS software 
3. Input rating question(s) into the software 
4. Print out sorting example sheet for each participant 
5. Print out of rules for each participant 
6. Before the second session begins, de-select brainstorming and select the rating and sorting sections 
(on the project settings tab) within the GCM software. 
7. Make a note of anyone who left during the break on the sign-in sheet  
Steps: 
1. Bring the participants into the computer room after team members have logged them into the system 
with the guest information. Explain the sorting process to the group using the example sheet/slides 
then explain the rules and hand them a set of the rules to keep with them.  After everything has been 
sorted, show them how to name the piles. 
2. Explain the rating system. Participants will rate items in response to these questions: 
• 
How important is this goal to you personally when thinking about your diabetes care?  
• 
How achievable is this goal for you personally? 
3. 3-4 team members should walk around the room to answer questions and to assist participants who 
have conceptual questions, questions about computer use, or software use. 
4. Before the patients leave, distribute the ClinCards by having each participant sign the receipt and 
record the participant name or study # next to the number of the ClinCard that they were given.  If 
the DOB was not collected on the demographics form, make sure it is collected here.  Make sure to 
specify when the card will be activated to the participant. 
5. After the second session, uncheck the sorting and rating options and check the analysis options in 
the software (on the project settings screen) to start the data analysis phase.  ** ONCE YOU 
ENTER DATA ANALYSIS, YOU WILL NOT BE ABLE TO CHANGE THE 
BRAINSTORMING/SORTING INFORMATION WITHIN THE SOFTWARE ** 
 
SESSION 3: 
Materials: 
1. Sign-in sheet 
2. Food 
3. Recorders 
4. Meeting space with projection/computer system 
5. Copy of final clustering solution and list of items for each of the participants, sorted numerically and 
by cluster 
Prep Steps: 
1. Talk to security to inform them of the research event 
Steps: 
1. Have the participants check in with the sign-in sheet.   
2. Start with an introduction of the GCM process and explain what a point map is (with examples) 
3. Read each statement that is contained within each cluster and ask the participants to come up with 
names that generally fit the majority of the cluster statements. Explain that statements that don’t 
seem to fit can be moved into a different cluster.  
4. After the names have been assigned, get consensus (by group majority) about which statements 
should be moved  
 
After all sessions have been completed, submit receipts to the p-card administrator. 
 

VOICe Study Procedures Manual V.6 
23 
 
v. Financial remuneration for participants 
Patients who complete all three sessions of the concept mapping will be compensated with $125, which will 
be allocated as follows: $50 upon completion of session 1 (brainstorming), $25 upon completion of session 2; 
$50 upon completion of session 3 (see below for session details). Study participants will be remunerated using 
Greenphire ClinCards (see “Participant payment” in Financial Policies and Procedures section).  
 
vi. Concept mapping time tracking procedure 
Research team members will track the time needed to complete each step of the concept mapping process in 
an Excel spreadsheet on Sharepoint. 
This will include at a minimum: 
• 
How much time you spend calling patients to set up concept mapping groups 
• 
Time spent traveling to Jefferson to do concept mapping 
• 
Time spent prepping for concept mapping groups 
• 
Time spent doing actual concept mapping group meetings (brainstorming, structuring of statements, 
and interpretation) 
 
vii. Data entry and storage  
In addition to demographic data collected at the end of each interview, patient records will be reviewed by 
trained research personnel with use of the medical record number (MRN) to determine: 
• 
Date of birth 
• 
Information on hospital and outpatient utilization in the prior 12 months, including date of visit, 
location of visit (i.e office, ED, inpatient), date of discharge if applicable, reason for visit, and 
discharge diagnosis 
• 
Information on the visit associated with this study, including location of the visit, time and day of 
arrival, reason for visit, treatments provided, discharge diagnosis and instructions, and discharge 
disposition and time 
• 
Insurance status (and provider, if applicable) 
• 
Chronic medical conditions  
 
All paper documents will be secured in locked cabinets in the Project Coordinator’s office. Research team 
members will assign a Study ID to each participant and enter the Study ID, participant’s medical information, 
and demographic information into the Patient Tracker stored in REDCap. Concept mapping data will all be 
entered into the Concept Systems Global Software.  
 
 
 
 
 
 
 
 
 
 
 
 
 
 

VOICe Study Procedures Manual V.6 
24 
 
V. Data Analysis and Dissemination 
A. Overview  
Data analysis will be done in consultation with the VOICe Research Advisory Council (RAC). The RAC will 
be composed of three PhD-trained researchers and two PAKSAB members who have already agreed to be 
representatives on this council. 
 
B. Interview analysis 
We will use a conventional content analysis approach for the analysis of the semi-structured interviews with 
the purpose of classifying the interview text into distinct categories that represent similar meanings and 
allowing themes to emerge from the data. Dr. Rising will oversee this analysis, and work with a coding team 
of four others (Project Coordinator, Research Assistant, and 2 PAKSAB members) to train them for coding 
interviews. The coding team will meet regularly during this process with the co-PIs to discuss codes and ideas 
that arise during coding. 
 
Each transcript will be labeled with the participant’s demographic and illness characteristics as well as the 
location along the care continuum at which the data were collected (acute, post-acute, primary care) and the 
mode of data collection (phone, in person). Transcripts from the same care continuum location will be 
analyzed together. Transcripts will be coded by members of the research team with a content analysis 
approach to identify thematic clusters within each set of care continuum location. Once data are coded, we 
will use the query function in NVivo to sort the data by various patient characteristics to examine patterns.  
 
C. Concept mapping analysis 
Data from each of the three CM groups will be entered into Concept Systems Global Software separately, 
with initial thematic clusters generated by the software and final clusters determined by the CM participants 
within each group. Each CM group will include participants from all three locations on the care continuum 
(acute, post-acute, primary care), thus generating three final sets of location-agnostic themes.  
 
D. Aims 2 & 3 analysis 
For Aims 2 & 3, we are comparing the comprehensiveness and efficiency of interviews and concept mapping.  
 
Our comparative analysis will start out with overall comparisons in which final thematic clusters from each of 
the three location-specific interview groups will be compared to the thematic clusters generated by the first 
CM group. This comparison will be a descriptive assessment of the presence/absence of each interview 
theme in the CM list. We will report proportions of the themes from each interview setting that were 
contained in the CM list as well as a detailed listing of each theme and in which lists it was present. Upon 
completion of this analysis, outputs from all three concept mapping groups will be aggregated and the same 
comparisons will be performed between the interview outcomes and the aggregated CM list.  
 
We will then work with the RAC to study patterns in our data that emerge by setting, participant 
characteristics, mode of data collection, and any other categories as identified by the PAKSAB or RAC as 
potentially important. These analyses will be operationalized with the use of matrices and other data display 
techniques to study the potential contributions of these various characteristics to the outcomes identified 
within the proposed study. For example, a matrix of thematic clusters in columns and features of individuals 
(acute, post-acute, and primary care) in rows will allow us to explore consistencies and divergences in the data 
which are associated with these non-method based descriptors. 
 
The efficiency of each method will be assessed by measuring overall resource utilization for conducting that 
method. We will determine the resource intensiveness for completion of a single concept mapping iteration 
through to completion, and for one set of interviews performed to saturation. The efficiency will be 
calculated based on the time required of research team and cost to conduct method. 
 

VOICe Study Procedures Manual V.6 
25 
 
We will also assess potential selection bias in who enrolled in interviews versus concept mapping using the 
data collected about who declines enrollment (gender, race, ethnicity, and age), and will assess on an on-going 
basis and at the end of the study how the enrolled patients compare to those who declined enrollment 
between the two groups. If we identify any systematic differences between the two groups, we will assess with 
our PAKSAB and our RAC how the difference(s) could have affected the thematic results obtained relative 
to the thematic results that we would have obtained.  
 
E.  Presentation and manuscript preparation 
Staff will maintain lists of upcoming conferences for presentation submissions. We anticipate presenting 
study findings annually. We plan to submit at least one manuscript based on interview findings (target date 
12/31/17) and one based on concept mapping (target date 6/30/18).  
 
Publications/Presentations – Research Awardees must acknowledge PCORI funding in scientific publications 
(e.g., peer-reviewed journal articles), scientific posters, slide presentations, study newsletters or brochures, 
study websites, reports and other non-commercial publications that relate to the PCORI-funded research 
project by including the following acknowledgement statement and disclaimer statement (as applicable). 
These statements will appear in legible size font in either introductory text or a footer and bracketed terms 
will be used only as applicable.  
 
Acknowledgement.  
Research reported in this [work, publication, article, report, presentation, etc.] was [partially] funded through a Patient-Centered 
Outcomes Research Institute (PCORI) Award (ME-1503-28476).  
 
Disclaimer. The following disclaimer must accompany the acknowledgement statement in any substantive 
works that present research findings, conclusions or other editorial content (e.g., journal publication, scientific 
poster).  
The [views, statements, opinions] presented in this [work, publication, article, report, etc.] are solely the responsibility of the 
author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of 
Governors or Methodology Committee.  
 
Any discussion of PCORI beyond these required statements in publications/presentations must be limited to 
the following factual reference.  
The Patient-Centered Outcomes Research Institute (PCORI) is an independent, nonprofit organization authorized by Congress 
in 2010. Its mission is to fund research that will provide patients, their caregivers, and clinicians with the evidence-based 
information needed to make better-informed healthcare decisions. PCORI is committed to continually seeking input from a broad 
range of stakeholders to guide its work. 
 
F.  Project website 
The VOICe website is designed to inform patients, researchers and other stakeholders about the project 
purpose, project team, and findings: http://www.jefferson.edu/university/skmc/research/voice.html 
 
G. Public Announcements 
Any public announcement intended to be distributed to media outlets (i.e., press release) announcing receipt 
of a PCORI award or any research findings that relate to a PCORI award by a Research Awardee requires 
coordination with PCORI. Awardees must provide notice to PCORI by sharing the public announcement 
and intended distribution date(s) via email to fundedpfa@pcori.org with a copy to the assigned Program 
Officer to enable proper coordination. 
 
 
 

VOICe Study Procedures Manual V.6 
26 
 
VI. Financial Policies and Procedures 
 
A. General Financial Management 
The Research Administration Center of Excellence (RACE) Grants Administrator reviews the project 
expenses on a monthly basis and meets with the PIs quarterly to review the grant’s financial status.  
 
B. Purchasing 
Contact Research Assistant regarding grant purchases.   
 
C. Subcontractor Invoicing 
The RACE Grants Administrator reviews subcontractor invoices, forwards them to the PIs for approval, and 
then sends to Accounts Payable for payment.  
 
D. Participant Payment  
There is payment remuneration for time and effort of participating in this study.  Interview participants will 
be compensated with a total of $25 at the end of the interview.  Patients who complete all three sessions of 
the concept mapping will be compensated with $125, which will be allocated as follows: $50 upon completion 
of session 1 (brainstorming), $25 upon completion of session 2, and an additional $50 upon completion of 
session 3.  
 
 In-person study participants will be remunerated with cash (interviews before1/1/17) or Greenphire 
ClinCards (interviews conducted after 1/1/17 and all CM group participants). They must complete a W-9 and 
receipt to receive either payment.  If the interview is done over the phone, patient’s will be asked if they want 
a gift card from Wawa, CVS, or Walgreens and the gift card will be mailed to the patient.  
 
Greenphire ClinCard Steps: 
1. Log on to https://www.ClinCard.com/login/ Select “Register Subject” at the left of the navigation 
bar 
2. From the drop down menu under study, select “Master Study” and then fill in the subject’s name and 
select “search” to see if they are already in the ClinCard system.  If not, proceed to add the 
participant to the master study.  In Subject Registration, complete the following steps: 
a. Ignore the study ID box for now 
b. From the drop-down menu, select “Master Study” 
c. Make sure “Enrolled” is selected under study status 
d. Make sure “Thomas Jefferson (MAIN)” study status 
e. Enter their first and last name 
f. 
Select “Register” 
3. Next, add the participant to your specific study through the following steps: 
a. Select “Edit Subject” button from the list of options on the right 
b. Don’t change the first “Study” dropdown list.  Instead, add the specific study information 
below starting with their Study ID, then select your specific study from the dropdown menu 
under this second “Study” option, and last make sure that “enrolled” is selected under 
“Subject Status.”  When all of the relevant information is added, make sure that you select 
the blue “Add Study” option in the top, right corner.  

VOICe Study Procedures Manual V.6 
27 
 
 
4. Once the participant is added to the study, first assign the participant a ClinCard (you will need to 
enter the full 16-digit ClinCard number) and then assign payments to that ClinCard with the 
following steps: 
a. Select “Assign ClinCard” from the list of options on the right 
b. Enter the 16-digit card number into the pop-up box 
c. Select “Assign” 
d. Select “Make Site Visit Payment” from the list of options on the right 
e. Select your specific study title from the drop-down list 
f. 
Use the sign-in sheets to determine which sessions the participant attended.  For each 
session of the study, select the appropriate session from the “Milestone” drop-down list  
• 
If the participant attended the session, select “Pay” 
• 
If the participant attended the session, select “Missed” 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

VOICe Study Procedures Manual V.6 
28 
 
VII. Reporting Requirements 
 
A. PCORI reporting 
PCORI requires progress reports on June 30 and December 31 of each grant year. 
PCORI also requires a final research report. Templates are available here: http://www.pcori.org/funding-
opportunities/awardee-resources 
 
The Project Manager will collaborate with the PIs to complete these reports. 
 
B. Institutional reporting 
Jefferson’s Office of Human Research requires an annual continuing review for expedited/full studies to 
maintain their approved status. Additionally, the PIs must submit amendments for protocol or study 
personnel changes. More information and forms can be found here:  
http://www.jefferson.edu/university/human_research/forms.html 
 
C. Risk Management and Adverse Event Reporting  
Psychological risk and/or risk to privacy is expected to be minimal, if any, and limited to the risk of 
discomfort in interviews or concept mapping sessions when asked to share their opinions and personal 
information. Risks will be mitigated by ensuring a private location to conduct interviews and concept 
mapping sessions. It will also be mitigated by communicating frequently with enrolled patients to ensure that 
their continued participation is voluntary and informing them that they may opt to decline to answer any or 
all of the interview or questions. There is the possible risk of loss of confidentiality. All participants will be 
asked to keep the contents of the concept mapping discussions confidential, and all data will be de-identified. 
All electronic data are password-protected on network-level-access-secured computers. All paper documents 
are secured in locked cabinets in a locked office accessible only to research personnel. PHI is only collected to 
the extent required by our study protocol, and records are de-identified as soon as possible. 
 
If a protocol deviation or adverse event occurs, the research team will document the incident in the protocol 
deviation or adverse event log and follow the procedures outlined in Thomas Jefferson University’s Office of 
Human Research Policy & Procedures Manual, Policy GA 120: Reporting and Reviewing Unanticipated 
Problems Involving Risks to Subjects or Others: 
http://www.jefferson.edu/content/dam/tju/human_research/irb/documents/PolicyandProceduresManual/
20151201%20DO%20NOT%20MODIFY%20Policy%20and%20Procedure%20Manual%20COMPLETE%
20CLEAN%20fixed.pdf 
 
D. Research and Financial Misconduct  
“Research misconduct is fabrication, falsification, or plagiarism in proposing, performing, or 
reviewing research, or in reporting research results. Financial misconduct refers to any act to acquire financial 
gain for oneself or for those of relatives, friends, or associates from or through activities and transactions 
related to the conduct of any PCORl-funded research. 
 
Recipient is required to report any findings of research or financial misconduct to PCORI within thirty (30) 
days of the conclusion of an investigation into research or financial misconduct related to any PCORl-funded 
research. Will research or financial misconduct occur with respect to any PCORl-funded research, the 
Recipient must notify PCORI, in writing, of the nature of the violation, the corrective actions that will be 
taken to correct the violation, and a timeline within which those corrective actions will be executed.” 
 
 
E. Study completion and closeout procedures 
i. PCORI  

VOICe Study Procedures Manual V.6 
29 
 
To close the study with PCORI, the research team must submit a draft final research report, sign off on the 
lay abstract no later than 90 days after PCORI accepts the final research report, and submit the final 
expenditure report 90 days from contract term date.  
 
ii. Thomas Jefferson University IRB 
Upon completion of the study, the research team will notify the IRB of study completion using an OHR-9 
final report: http://www.jefferson.edu/university/human_research/forms.html