Understanding What Matters Most to Patients: Establishing the Validity of a Best-Worst Scaling Survey
Conditions: Hematologic Neoplasms
Sponsor: UNC Lineberger Comprehensive Cancer Center

LINEBERGER COMPREHENSIVE CANCER CENTER 
Health Services Research 
CLINICAL ONCOLOGY RESEARCH PROGRAM 
September 2014 
UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL 
LCCC 2136:  Understanding What Matters Most to Patients: Establishing the 
Validity of a Best-Worst Scaling Survey 
Principal Investigator 
Daniel R. Richardson, MD 
Department of Medicine, Division of Hematology/Oncology 
University of North Carolina at Chapel Hill 
Houpt Building, 3rd Floor  
170 Manning Drive 
Chapel Hill, NC 27599 
Phone: (919) 445-0161 
Email: daniel.richardson@med.unc.edu 
Co-Investigator(s) 
Antonia Bennett 
William Wood 
Karthik Adapa  
Lukasz Mazur 
Amy Cole  
Stephanie Wheeler  
Jonathan Sorah 
Konan Beke 
Christopher Jensen  
Ashley Bryant 
Lorinda Coombs  
Callie Berkowitz  
Biostatistician 
Allison Deal  
Sponsor:  Lineberger Comprehensive Cancer Center 
Funding Source: Palliative Care Research Consortium Pilot Award 
Version Date: 1.0 September 1, 2021 

LINEBERGER COMPREHENSIVE CANCER CENTER 
Health Services Research 
CLINICAL ONCOLOGY RESEARCH PROGRAM 
 
September 2014 
UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL 
 
LCCC 2136:  Understanding What Matters Most to Patients: Establishing the 
Validity of a Best-Worst Scaling Survey 
 
 
 
Principal Investigator 
Daniel R. Richardson, MD 
Department of Medicine, Division of Hematology/Oncology  
University of North Carolina at Chapel Hill 
Physicians Office Building, 3rd Floor  
170 Manning Drive 
Chapel Hill, NC 27599 
Phone: (919) 445-0161 
Email: daniel.richardson@med.unc.edu 
 
 
Signature Page 
 
The signature below constitutes the approval of this protocol and the attachments, and 
provides the necessary assurances that this trial will be conducted according to all 
stipulations of the protocol, including all statements regarding confidentiality, and 
according to local legal and regulatory requirements and applicable U.S. federal regulations 
and ICH guidelines. 
 
Principal Investigator (PI) Name:__Daniel Richardson_____________________ 
 
PI Signature: ___________________ 
 
Date:____________________ 
 
 
 
Version Date:_______________________ 
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
i 
TABLE OF CONTENTS 
1.0 BACKGROUND AND RATIONALE .......................................................................1 
1.1 
Study Synopsis .....................................................................................................1 
1.2 
Background ..........................................................................................................1 
1.3 
Purpose and Rationale ..........................................................................................2 
2.0 STUDY OBJECTIVES AND ENDPOINTS ..............................................................2 
2.1 
Primary Objective ................................................................................................2 
2.2 
Secondary Objectives ...........................................................................................2 
3.0 PATIENT ELIGIBILITY ...........................................................................................3 
3.1 
Inclusion Criteria .................................................................................................3 
3.2 
Exclusion Criteria ................................................................................................4 
4.0 STUDY PLAN ..............................................................................................................4 
4.1 
Schema .................................................................................................................4 
4.2 
Duration of Study .................................................................................................4 
4.3 
Study Details ........................................................................................................5 
4.4 
Expected Risks ...................................................................................................11 
4.5 
Removal of Patients from Protocol ....................................................................13 
5.0 TIME AND EVENTS TABLE ..................................................................................14 
5.1 
Time and Events Table ......................................................................................14 
6.0 UNANTICIPATED PROBLEMS ............................................................................14 
6.1 
Definition ...........................................................................................................14 
6.2 
Reporting............................................................................................................15 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
ii 
7.0 STATISTICAL CONSIDERATIONS .....................................................................15 
7.1 
Study Design ......................................................................................................15 
7.2 
Sample Size and Accrual ...................................................................................15 
7.3 
Data Analysis Plans ...........................................................................................16 
7.4 
Data Management/Audit ....................................................................................17 
8.0 STUDY MANAGEMENT .........................................................................................18 
8.1 
Institutional Review Board (IRB) Approval and Consent .................................18 
8.2 
Required Documentation ...................................................................................19 
8.3 
Registration Procedures .....................................................................................19 
8.4 
Adherence to the Protocol ..................................................................................19 
8.5 
Amendments to the Protocol ..............................................................................20 
8.6 
Record Retention ...............................................................................................21 
8.7 
Obligations of Investigators ...............................................................................21 
9.0 REFERENCES ...........................................................................................................22 
10.0 APPENDICES ..........................................................................................................26 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
1 
1.0 
BACKGROUND AND RATIONALE 
1.1 
Study Synopsis 
This study will evaluate the content validity of using a survey to quantify patient 
preferences at the point-of-care for newly-diagnosed patients with hematologic 
malignancies. Our primary hypothesis is that by identifying the strength of patient 
preferences for outcomes with this survey, clinicians will be able to improve goal-
concordant care by aligning clinical recommendations with patients’ preferences.  
We will conduct cognitive interviews with 20 patients to establish the content 
validity of the survey. We will use purposive sampling based on the results of the 
survey to create two groups of patients (n=10 each) for the cognitive interviews. 
We plan to recruit 50 total patients to complete the survey to ensure adequate 
recruitment in each group. Secondary outcomes will evaluate acceptability and 
feasibility of administrating the survey. The information we obtain from these 
participants will be used to refine the survey. Interviews with oncologists and 
palliative care specialists (up to 10) will inform implementation.   
1.2 
Background 
 
Treatment decisions in oncology are increasingly preference sensitive. Cancer 
patients differ substantially in their preferences for achieving various outcomes.1–3 
For example in AML, we have shown that some patients prefer to maximize their 
overall chance of long-term survival and are willing to endure a high burden of side 
effects or a lengthy hospitalization for this opportunity; others prefer to minimize 
treatment effects in order to maintain their quality of life.1 As more therapies are 
becoming available across oncology, this preference heterogeneity suggests that 
treatment decisions will increasingly become preference sensitive—ideal therapy 
choice will depend on patients’ preferences for treatment outcomes.  
Routine shared decision-making (SDM) is currently inadequate to reliably capture 
patient preferences.4–8 Multiple systematic reviews have demonstrated consistent 
discordance after routine SDM between physician perceptions of what matters most 
to patients and patients’ stated preferences.8,9 A recent evaluation of a clearly 
preference-sensitive decision in oncology demonstrated that about half the time 
oncologists did not elicit patient preferences.10 Analyses of the quality of SDM in 
leukemia, for example, demonstrate a high rate of patient dissatisfaction with 
communication and very poor patient-provider concordance on goals and 
prognosis.11–13 As a stark example of patient-provider discordance following 
routine SDM, 80% of leukemia patients receiving symptom-directed therapy 
believe they are likely to be cured, when less than 5% will be.14 Palliative care 
clinicians can serve to improve these outcomes, but often have limited bandwidth 
to see most patients.  
Preference elicitation instruments can capture patient preferences, however 
fundamental challenges remain to their implementation in complex clinical care. 
Numerous stakeholders including the FDA strongly advocate for patient 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
2 
preferences to be integrated into SDM and clinical care.15–18 Preference elicitation 
instruments that were originally developed in economics to understand consumer 
behavior are increasingly being applied in healthcare to identify outcomes that are 
most valued by stakeholders. These instruments allow researchers to understand 
the relative importance of a group of items (attributes) to a population based on a 
series of choice tasks.19–21 In clinical trials evaluating dichotomized decisions 
where the risks and benefits of treatment options are well-established, these 
instruments have been shown to improve patient satisfaction, increase high-
quality decisions, and align treatment choice with risk.2,22–25 Fundamental 
questions remain, however, about how to assess and apply preference data to 
complex SDM in advanced cancer patients where risks and benefits of available 
options vary for each patient.26–28 Further, substantial methodological barriers 
exist in assessing individual preferences over time and pose a critical challenge to 
longitudinal goal-concordant care. To address these critical gaps, we will use 
treatment decisions for patients with hematologic malignancies as the context for 
developing generalizable strategies to improve the goal-concordance of complex 
treatment decisions in palliative care/oncology.  
1.3 
Purpose and Rationale 
In this study, we will recruit older (> 60 years) participants with newly-diagnosed 
hematologic malignancies to complete the BWS survey and provide quantitative 
and qualitative feedback to validate the survey. Interviews with a sample of 
oncologists and palliative care physicians will be completed to identify barriers 
and facilitators to implementation of the survey into routine clinical care.  
 
2.0 
STUDY OBJECTIVES AND ENDPOINTS  
2.1 
Primary Objective 
To evaluate the content validity of using a BWS survey to quantify the 
preferences of older patients with hematologic malignancies at the point-of-care.  
Content Validity: Content validity will be assessed in cognitive interviews with 
participants. We will first assess the respondent's comprehension of the 
questionnaire and its items in relation to their intended meaning. We will identify 
any formatting or wording difficulties with the survey itself. Next, we will 
evaluate participants comprehension of the content by use of probing questions to 
evaluate if other important aspects of treatment decision-making were absent in 
the survey.  
2.2 
Secondary Objectives 
1. To report the acceptability of the BWS survey to patients. This will be defined 
as participants answering agree/strongly agree to “I found the survey 
acceptable to clarify my preferences”.  
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
4 
- Willingness and ability to provide written informed consent 
 
Patients will become eligible for enrollment upon pathologic confirmation.  
 
Patients will be eligible for enrollment for 28 days after confirmatory diagnosis or 
until the start of chemotherapy.  
 
Chemotherapy will not include hydroxyurea, cyclophosphamide, or cytarabine if 
used only for emergent cytoreduction. 
 
Health care clinicians must be employed in hematology/oncology or palliative 
care as a nurse navigator, advanced practitioner, and/or physician. Together, these 
participants will be referred to as healthcare clinicians.  
 
3.2 
Exclusion Criteria 
 
All subjects meeting any of the exclusion criteria listed below at baseline will be 
excluded from study participation: 
3.2.1 Dementia, altered mental status, or psychiatric condition that would 
prohibit the understanding or rendering of informed consent or 
participation in the discrete choice experiment.  
3.2.2 Significant medical conditions, as assessed by the investigators, that 
would substantially increase the burden on the patient to complete study 
assessments (such as multiorgan failure, respiratory failure, or other 
critical illness). 
4.0 
STUDY PLAN 
4.1 
Schema 
 
4.1.1 This study will evaluate the validity of using this out(BWS) survey to quantify 
patient preferences at the point-of-care and the potential effectiveness of the 
survey to improve goal-concordant care. Our primary hypothesis is that by 
identifying the strength of patient preferences for outcomes with this survey, 
clinicians will be able to improve goal-concordant care by aligning clinical 
recommendations with patients’ preferences.  
Within this study we aim to evaluate the content validity, acceptability, and 
potential effectiveness of the BWS survey to improve goal-concordant care. 
Using the BWS survey, we will collect the preferences of 50 older (>60 years) 
newly-diagnosed hematologic cancer patients facing treatment decisions and 
evaluate content and construct validity of the survey, patient acceptability, and 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
5 
effect of preference elicitation on patient-reported quality of treatment decisions 
(effectiveness). Qualitative interviews involving cognitive interview questions 
(n=20) with patients will further explore the content validity of the survey and 
potential effect of the surveys on goal-concordant care. Interviews with palliative 
care clinicians, and oncologists will explore barriers and facilitators of 
implementation.  
We also aim to describe how patients’ preferences change over time and explore 
potential relationships between patients’ preferences and clinical outcomes. We 
will collect the preferences of the patients with hematologic cancer every other 
week for 3 months (repeat sampling up 5 times) and then every 3 months 
thereafter for up to 2 years. We will describe changes in preferences at each 
timepoint. An exploratory, hypothesis-generating analysis will evaluate 
correlations between preferences and standard clinical outcomes (e.g. treatment 
intensity and response), symptoms (collected with the PRO-CTCAE), health-
related quality of life (EQ-5D), and decisional satisfaction to provide direction for 
future studies.  
 
4.1.2 Trial Schema outline (patients)  
 
4.2 
Duration of Study 
The recruitment is planned to last for up to 24 months. Patients will complete the 
survey instruments up to 6 times over 3 months (Longitudinal Assessments) and 
then every 3 months for up to 2 years (7 times) (Extended Assessments). Patients 
will complete qualitative interviews within 3 months of treatment decision. 
Clinicians will be interviewed once over the course of the study.   
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
6 
4.3 
Study Details 
4.3.1 The study will recruit 50 total participants and conduct qualitative interviews on a 
subset of 20 participants.  
4.3.2 Pilot testing: The study team will review the inpatient census and clinic schedule 
to evaluate for potentially eligible patients. The study team will confirm patient 
eligibility with the primary oncologist or principal investigator. Eligible patients 
will be sequentially approached to be recruited to this study until 50 patients are 
recruited who complete BWS survey.  
Upon enrollment, the study team will record information about patients obtained 
from the patient or the medical record including gender, age, lab values including 
complete metabolic profile, complete and diffentiated blood counts, albumin, 
LDH,  a1c, cholesterol levels, hematopathology reports, clinician-determined 
performance status (Karnofsky and ECOG), and comorbidities present. 
Demographic information such as age, sex, and zip code will be obtained. 
Once consented, patients will complete initial study questionnaires within PRO 
Core either remotely if able or in-person with the assistance of study staff prior to 
a clinic visit. For patients who decline to participate in the study, reasons for 
refusal will be anonymously collected to examine the issue of selection bias as 
well as to assist with determination of feasibility.  
 
After the clinic visit, patients will be prompted to complete the first longitudinal 
assessment via PRO Core. This will be available for 7 days. The treating 
oncologist who saw the patient will be prompted to fill out an assessment via 
email. Clinicians may receive multiple reminder emails to complete the survey to 
improve completion rates. We will collect provider responses within 2 weeks of 
initial patient survey.   
 
Following, every 2 weeks (up to 5 times) participants will be prompted in PRO 
Core to complete the longitudinal assessment survey. Then, the extended 
assessment survey will be delivered by PRO Core every 3 months for up to 2 
years (7 times).  
 
Retention:  
 
Multiple strategies will be utilized to retain patients on study.  
1. Provide Detailed Rationale of the Study and Study Brochure: Upon enrollment 
participants will be informed of the rationale of the study: to help understand what 
is most important to each patient to tailor therapy to patient preferences. Patients 
will be informed that their responses will help adapt the survey that they will 
receive to help future patients. They will be given printed information about the 
study if recruitment is done in person.  

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
7 
2. Training: Each patient will receive training on how to complete the 
assessments in PRO Core. This will take place during enrollment (either in person 
or remotely).  
3. Reminders to complete the initial post-visit assessments: The day after the 
treatment encounter, patients will receive an automated reminder to complete the 
assessments.  
4. Reminders to complete the follow-up assessments: Staff will direct them to 
complete the assessment remotely via a secure link sent by email. PRO Core will 
send patients a reminder daily for up to 2 days in case of non-response. Study 
staff will call patients if no response.  
5. Reminders in case of non-response: Study staff will call patients if no response.  
6. Support to complete assessments: Staff will assist participants to log-on to PRO 
Core and to complete assessments if needed.  
7. Compensation: Participants will be provided with a $10 gift card following 
completion of each longitudinal and extended assessment.  
 
4.3.3 Qualitative Interviews: Based on responses to the survey, 20 patients will be 
selected to complete interviews. Patients will complete an initial interview and a 
follow-up interview.  
The study team will review patients who have consented to the study and confirm 
eligibility with the primary oncologist or principal investigator. Study staff will 
call eligible patients to participate in the interviews until the number of desired 
participants has been reached (n=20) according to the inclusion criteria above and 
recruitment goals below.  
 
We will use purposive sampling to stratify participants based on the results of the 
BWS survey. We will stratify into two groups of 10 participants each:  
 
1. Those participants who believe that “Living longer” is the most important 
attribute,  
 
and  
 
2. Those participants who believe another attribute is the most important attribute.  
Staff will setup a time to interview eligible participants. Interviews will be 
scheduled within 14 days of the treatment decision (+ 14 days). A reminder email 
and/or phone call will be sent to participants the day before and the day of the 
interview. If participants do not respond during the pre-scheduled time, staff will 
reschedule the interview/reminders. Participants will be provided with a $25 gift 
card following completion of the interview. 
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
8 
Follow-up interviews will be around 3 months after the initial interview (+ 30 
days). Interviews will follow the same format as the initial interview. Participants 
will be provided with a $25 gift card following completion of the interview.   
 
Interviews will also be completed with up to 5 oncology clinicians and up to 5 
palliative care clinicians. Clinicians will be referred to the study staff by the 
principal investigator. Interviews will take place over the course of 1 year and will 
not be time dependent upon patient enrollment. Participants will be provided with 
a $25 gift card following completion of the interview.   
4.3.4 Participant Recruitment and Stratification 
For the pilot testing, recruitment of patients will be stratified into the following 3 
groups: 
 
Aggressive lymphomas: 20 patients   
Acute leukemias: 20 patients 
Myeloma, Chronic lymphocytic leukemia (requiring treatment), chronic myeloid 
leukemia: 10 patients 
 
We will target a population that exceeds the following diversity thresholds: 30% 
non-white race, 50% female, 25% rural. 
 
For the qualitative interviews, recruitment of patients will be stratified into the 
following 2 groups: 
 
1. Those participants who believe that “Living longer” is the most important 
attribute (n=10),  
 
and  
 
2. Those participants who believe another attribute is the most important attribute 
(n=10).  
 
Recruitment of clinicians will be as follows:  
Oncology clinicians (including physicians, advanced practice providers, nurse 
navigators): 2-5  
Palliative Care clinicians (including physicians, advanced practice providers, 
nurse navigators): 2-5  
 
  
4.3.5 The BWS survey    
Priorities are how people value and rate the importance of multiple goals. In 
comparison, patient preferences reflect choices that patients make among several 
alternatives based on the expected benefit they get from each alternative. BWS 
surveys, a type of conjoint analysis, are one of several methods to measure 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
9 
preferences. They are emerging as potentially important tools to elicit preferences 
of patients in the clinical setting.20 BWS surveys involve asking participants a 
series of questions where they choose one attribute that is best and one attribute 
that is worst. In this survey, we will be asking patients to choose between the 
following 7 attributes:  
a. 
Maintaining usual activities  
b. 
Living longer 
c. 
Avoiding becoming dependent upon others 
d. 
Avoiding short-term side effects 
e. 
Avoiding long-term side effects 
f. 
Avoiding hospitalizations 
g. 
Avoiding high out-of-pocket costs 
 
 
We will be using a BWS survey that is adapted from a BWS survey we used to 
capture the preferences of patients with AML. Sampling 832 patients with the 
BWS survey, we illustrated that patients had the strongest concerns about 
treatment outcomes in psychosocial and physical domains.32 Patients were most 
worried about dying from their disease, the long-term side effects of treatment, 
and becoming a burden to others. 
 
 
4.3.6  Longitudinal Assessments 
 
Participants will complete longitudinal assessments every 2 weeks (x 5). These 
assessments will include the BWS survey, CollaboRATE, WIWI, Decisional 
conflict scale, EQ-5D-5L, distress thermometer, and 7 items from the PRO-
CTCAE. These assessments will begin after a treatment discussion with their 
oncology clinician.  
 
The first longitudinal assessment will include a follow-up questionnaire that will 
assess patient acceptability and preliminary efficacy of the BWS to improve 
treatment decisions.  
 
Participants will be given a $10 electronic gift card for each assessment.  
 
4.3.7 Extended Assessments 
After completing the longitudinal assessments (x6), participants will complete 
extended assessments every 3 months for the next 2 years (up to 7 assessments). 
These extended assessments will include the same questionnaires as the 
longitudinal assessments.  
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
10 
At 1 year on the study (the third extended assessment), participants will complete 
the 1 year questionnaire.  
 
Participants will be given a $10 electronic gift card for each assessment. 
4.3.8 Qualitative Interviews 
 
Patients: 
The qualitative interview will establish content validity of the BWS by cognitive 
interviews that will explore patients understanding of the content of the BWS. 
Specific domains to be covered will include:  
• Cognitive interview to evaluate patient understanding of each of the 
questions asked within the BWS 
• subjective value of potential outcomes including attributes elicited in 
BWS  
• patient understanding of how preferences influenced treatment decisions 
including elicitation of important factors in the decision (if applicable),  
• expected outcomes,  
• additional attributes not included in the BWS, and 
• preferences for potential interventions to improve preference elicitation 
and decision-making. 
 
In addition to open ended questions as part of the interview, patients will 
complete a digit span test to test working memory and recall.  
 
The follow-up qualitative interview will cover the same content as the first 
interview in addition to asking participants to reflect upon how their experience 
has changed over the time period between interviews.  
 
Interviews with clinicians will explore barriers and facilitators of implementation 
of surveys to capture patient values and preferences. There is no follow-up 
interview with clinicians. Clinicians will not complete the Digit span test.  
 
Interviews will be recorded to allow for qualitative analysis. Transcripts of the 
interviews will be made. The interviews should take about 30 minutes to 
complete. Following the interview a codebook will be created with emerging 
themes to allow for qualitative analysis similar to previous studies.11   
 
4.3.9 Participant questionnaires  
4.3.9.1 Baseline Participant Questionnaire 
Participants will complete a baseline questionnaire online via PRO Core upon 
enrollment. Questions related to the following domains will be asked of patients: 
race; ethnicity; household income; and insurance. This baseline questionnaire will 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
11 
also include the PROMIS Cognitive Function 8a short form (8 items) to assess the 
perceived cognitive impairment of patients.   
4.3.9.2 Decisional conflict scale (DCS):  
Decisional conflict will be evaluated with the validated survey measure decisional 
conflict scale.31,33 Patients will complete the subscales of decision uncertainty (3 
items) and perceived effective decision making (4 items).   
4.3.9.3 EQ-5D-5L:  
Health-related quality of life will be evaluated with the validated survey measure 
EQ-5D-5L.34 This is a 5 question survey.  
4.3.9.4 PRO-CTCAE:  
Symptoms will be evaluated with the validated survey measure PRO-CTCAE.35 
Patients will complete the following 7 items: pain, fatigue, nausea, vomiting, 
constipation, diarrhea, appetite.   
4.3.9.5 PROMIS emotional distress short-form 8a 
The PROMIS emotional distress short-form 8a is a 8 question survey developed 
to understand current emotional distress among patients and validated in 
numerous settings.  
4.3.9.6 Follow-up Questionnaire  
Patient acceptability and preliminary efficacy (both defined above) will be 
assessed in this 2-item study-specific questionnaire.   
 
4.3.9.7 1 year Questionnaire  
Patient acceptability and preliminary efficacy (both defined above) will be 
assessed in this study-specific questionnaire. Additionally, patients will be given 
open ended questions reflecting on how the BWS survey influenced the treatment 
decisions with their providers.    
4.3.9.8 CollaboRATE:  
Preference clarity will be evaluated with the validated survey measure 
CollaboRATE.30 This is a 3 question survey.  
4.3.9.9 WIWI 
The “was it worth it” (WIWI) patient questionnaire is a subjective tool to assess a 
patient’s decisional regret related specifically to the chemotherapy decision.38 We 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
12 
will use two questions to avoid overlap with other questionnaires: Was it 
worthwhile for you to undergo chemotherapy? And If you had to do it over, 
would you undergo chemotherapy again? 
4.3.9.10 Decision Regret Scale (DRS):  
Decisional regret will be evaluated with the validated survey measure DRS.29 This 
is a 5-item questionnaire that has been validated as an important outcome measure 
of preference/value clarification measures.    
 
4.3.9.11 Digit Span Test:  
The forward and backward assessments of the Digit Span test are subsets of the 
Wechsler Intelligence Scale, and when used separately, are considered validated 
measures of working memory.39,40 Participants are assessed on their ability to 
repeat strings of numbers, increasing in length by one, until an error occurs on the 
reiteration of the string, or they reach a string length equal to 9. Participants will 
run through all forward strings before moving to the digit span backwards test, in 
which they will be asked to repeat strings of numbers in reverse order.  
4.3.9.12 Provider questionnaire  
This survey will include 3 questions to capture treatment intent (curative vs. 
palliative), provider’s assessment of which outcomes are most important to the 
patient, and outcomes most important to treatment decision making. 
 
 
4.4 
Expected Risks 
4.4.1 Physical Risks 
No physical risks are expected with this study. 
4.4.2 Emotional Risks 
Some questions in the patient questionnaires or interviews could create emotional 
distress or confusion. Patients will be instructed that the questionnaires are for 
research purposes only and any problem or symptom should also be reported to 
their physician or nurse.  
4.4.3 Patient Confidentiality 
We do not anticipate any breach of confidentiality. Any information about the 
participants obtained from this research will be kept as confidential as possible. 
All data obtained will be stored on a secure server.   

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
13 
4.5 
Removal of Participants from Protocol  
Participants will be removed from the protocol if they decide they no longer want 
to participate in the study.    
 
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
15 
• Is unexpected (in terms of nature, severity, or frequency) given (a) the 
research procedures that are described in the protocol-related documents, 
such as the IRB-approved research protocol and informed consent 
document; and (b) the characteristics of the subject population being 
studied; 
• Is related or possibly related to a subject’s participation in the research; 
and  
• Suggests that the research places subjects or others at a greater risk of 
harm (including physical, psychological, economic, or social harm) related 
to the research than was previously known or recognized. 
6.2 
Reporting 
Any UPIRSO that occurs during the conduct of this study and that meets all three 
criteria listed in 6.1 must be reported to the UNC IRB using the IRB’s web-based 
reporting system.  The PI will also report any UPIRSO to the PCRC and the 
NINR. The PI will be responsible for assessing developments in the literature and 
results of related studies by conducting a literature review once during the grant 
period to identify relevant information that may impact the safety of participants 
or on the ethics for the research study.   
 
7.0 
STATISTICAL CONSIDERATIONS 
7.1 
Study Design 
This is a mixed methods pilot study designed to evaluate the content validity of 
the BWS survey.  
7.2 
Sample Size and Accrual 
Our primary objective is to establish content validity of the BWS survey. Based 
on established guidelines, 7-10 participants are needed to establish content 
validity of survey items in cognitive interviews.41–43 As we will be stratifying 
patients for cognitive interviews into 2 groups, we will have 10 participants per 
stratification to ensure saturation in both.  
 
Based on our prior data, we anticipate that we will have a 3:1 split between 
patients who value living longer most and those who value another attribute most. 
Therefore, we will need to recruit at least 40 patients to have 10 patients who 
value another attribute most to participate in cognitive interviews. Accounting for 
variability and drop out, we will plan to recruit 50 patients total to be able to 
recruit 10 patients in this group. 
 
10 clinicians will provide for saturation for qualitative interviews.  
 
This study was not designed to be powered to evaluate secondary endpoints or 
exploratory endpoints.  

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
16 
 
We expect to accrue up to 50 patients over the time of the study. We see over 500 
new patients with hematologic malignancies per year within the division of 
hematology.  
7.3 
Data Analysis Plans 
 
7.3.1 Primary Aim: Content Validity  
The primary aim of this study is to evaluate the content validity of the BWS 
survey. This will be established by mixed methods. We will conduct cognitive 
interviews with patients in accordance with FDA guidance on patient-reported 
outcomes measurement development to evaluate patient understanding of each 
concept within the BWS and explore other areas that were important but were not 
included in the survey.41 We will produce a cognitive interview summary based 
on published guidance to determine content validity from the interviews.41  
Additional interview questions from the qualitative interview will be used to 
establish the importance of individual attributes to patients using interpretive 
phenomenological analysis (IPA)44 and provide additional support to BWS 
content validity. Results from the ranking exercise (embedded within the BWS 
survey) will also be reported. To integrate data from each source, we will follow a 
sequential explanatory approach: we will produce narrative summaries of each 
aspect of the critical determinants identified in interviews followed by a summary 
of both quantitative and qualitative data. Together, these data will provide 
convergent support of the content validity of the BWS survey. 
 
7.3.2 Secondary Aims 
Patient acceptability:  
We will conclude the BWS is acceptable if  >70% of patients (35/50) report that it 
is acceptable. Hypothesizing 80%, the exact binomial 95% CI will be 66.3-90.0%.  
Preliminary efficacy:  
We will conclude the tool is potentially efficacious if efficacy (defined as 
participants answering “moderately effective/very effective” to the question: 
“How effective was the survey to help you and your doctor choose a treatment 
that was consistent with your goals?”) is reached in >50% of patients. Estimating 
5-7% missing data from enrollment to initial longitudinal  questionnaire (from 
other PRO assessments in advanced cancer patients45,46) and hypothesizing a rate 
of 75%, the exact binomial 95% CI will be 60.0-86.6%. 
 
Preliminary completion rates: 
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
17 
Frequencies will be reported at each time point.  Completion rate out of the 
original 50 participants will be reported at each month with exact binomial 95% 
CIs with reasonable precision, no wider than ±14.5%. Reasons for dropout will be 
summarized as available.  Associations of baseline patient characteristics with 
completion rates will be explored using Wilcoxon Rank sum and Kruskal-Wallis 
tests. 
 
 
7.3.3 Exploratory Aims  
 
Repeated measures analyses, using generalized linear mixed models, will be used 
to describe change in preferences over time and the relationship to clinical 
characteristics with a particular focus on evaluating the influence of response (e.g. 
remission) on preferences. Performance of the BWS survey will be evaluated 
including convergent validity, within-set dominated pairs, and attribute 
dominance.21,47 We will use frequency analysis to describe the strength of 
aggregate preferences for outcomes, and differences by age, race, gender, and 
clinical response.19 We will perform exploratory, hypothesis-generating analyses 
to compare within-person changes in preferences across timepoints, the relative 
stability of betas for attributes over time, and explore associations between 
preferences, decisional outcomes8,29–31, patient-reported symptoms,35 health-
related quality of life34, and provider perceptions. These analyses are exploratory 
as our sample size is relatively small. These exploratory analyses will inform 
future studies. 
 
7.4 
Data Management/Audit 
Data will be stored on a shared, Oracle-Based framework which provides a highly 
secure IT environment, utilizing a secure authentication mechanism, the Transport 
Layer Security (TLS) system. This security mechanism supports encryption in 
both directions (to and from the website) across all devices including mobile 
devices such as smartphones. Backup files of all data will be included on this 
server. Only those trained to work with sensitive data will have access to the 
server. This project will have a specific folder on the server that will require 
password access for all users.  
 
Data collection will be done by the study team. Study personnel will input data 
from patient interview or chart review into the database.  
 
The Principal Investigator will provide continuous monitoring of subject safety in 
this trial with periodic reporting to the Data and Safety Monitoring Committee 
(DSMC).  
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
18 
The Principal Investigator will provide continuous monitoring of subject safety in 
this trial with periodic reporting to the Data and Safety Monitoring Committee 
(DSMC). Data sources included in this study will include data from medical 
records, questionnaires, and audio recordings. Data will be extracted from 
medical records by trained clinical research assistants and placed in a study-
specific database per the protocol. Questionnaires will be delivered electronically. 
Data will be automatically transferred to the database. Audio recordings will be 
produced during the qualitative interviews and will be transcribed by trained 
personnel. The transcripts will be transferred electronically into the secure 
database. Data will undergo periodic review by the PI and the project manager for 
integrity. 
 
Meetings/teleconferences will be held at a frequency dependent on study accrual.  
These meetings will include the investigators as well as protocol nurses, clinical 
research associates, regulatory associates, data managers, biostatisticians, and any 
other relevant personnel the principal investigators may deem appropriate.  At 
these meetings, the research team will discuss all issues relevant to study 
progress, including enrollment, safety, regulatory, data collection, etc. 
 
The team will produce summaries or minutes of these meetings. These summaries 
will be available for inspection when requested by any of the regulatory bodies 
charged with the safety of human subjects and the integrity of data including, but 
not limited to, the Office of Human Research Ethics (OHRE) Biomedical IRB, the 
Oncology Protocol Review Committee (PRC) or the North Carolina TraCS 
Institute Data and Safety Monitoring Board (DSMB).   
 
As an investigator initiated study, this trial will also be audited by the Lineberger 
Cancer Center audit committee every six or twelve months, depending on the 
participation of affiliate sites. 
 
Once complete, data will be de-identified and transferred to the Palliative Care 
Research Consortium (PCRC) repository for dissemination. 
8.0 
STUDY MANAGEMENT 
8.1 
Institutional Review Board (IRB) Approval and Consent 
It is expected that the IRB will have the proper representation and function in 
accordance with federally mandated regulations.  The IRB should approve the 
consent form and protocol. 
 
In obtaining and documenting informed consent, the investigator should comply 
with the applicable regulatory requirement(s), and should adhere to Good Clinical 
Practice (GCP) and to ethical principles that have their origin in the Declaration 
of Helsinki. 
 
 
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
19 
Before recruitment and enrollment into this study, the patient will be given a full 
explanation of the study and will be given the opportunity to review the consent 
form. Each consent form must include all the relevant elements currently required 
by the FDA Regulations and local or state regulations. Once this essential 
information has been provided to the patient and the investigator is assured that 
the patient understands the implications of participating in the study, the patient 
will be asked to give consent to participate in the study by signing an 
IRB-approved consent form. 
 
Prior to a patient’s participation in the trial, the written informed consent form 
should be signed and personally dated by the patient and by the person who 
conducted the informed consent discussion. 
8.2 
Required Documentation 
Before the study can be initiated, the following documentation must be provided 
to the Clinical Protocol Office (CPO) at the University of North Carolina. 
• A copy of the official IRB approval letter for the protocol and 
informed consent 
• CVs and medical licensure for the principal investigator and any 
associate investigators who will be involved in the study 
• A copy of the IRB-approved consent form 
8.3 
Registration Procedures 
Patient enrollment will be tracked in a secure database as outlined above.  
8.4 
Adherence to the Protocol 
Except for an emergency situation in which proper care for the protection, safety, 
and well-being of the study patient requires alternative treatment, the study shall 
be conducted exactly as described in the approved protocol.   
8.4.1 Emergency Modifications 
UNC investigators may implement a deviation from, or a change of, the protocol 
to eliminate an immediate hazard(s) to trial subjects without prior UNC IRB 
approval.  
 
For any such emergency modification implemented, a UNC IRB modification 
form must be completed by UNC Research Personnel within five (5) business 
days of making the change.   
8.4.2 Single Patient/Subject Exceptions 
We will not allow single patient exceptions in the study. Any request to enroll a 
single subject who does not meet all the eligibility criteria of this study requires 
the approval of the UNC Principal Investigator and the UNC IRB.  

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
20 
8.4.3 Other Protocol Deviations/Violations 
According to UNC’s IRB, a protocol deviation is any unplanned variance from an 
IRB approved protocol that:  
• Is generally noted or recognized after it occurs 
• Has no substantive effect on the risks to research participants 
• Has no substantive effect on the scientific integrity of the research plan 
or the value of the data collected  
• Did not result from willful or knowing misconduct on the part of the 
investigator(s).  
 
An unplanned protocol variance is considered a violation if the variance meets 
any of the following criteria:  
• Has harmed or increased the risk of harm to one or more research 
participants. 
• Has damaged the scientific integrity of the data collected for the study. 
• Results from willful or knowing misconduct on the part of the 
investigator(s). 
• Demonstrates serious or continuing noncompliance with federal 
regulations, State laws, or University policies. 
 
If a deviation or violation occurs please follow the guidelines below: 
 
Protocol Deviations: UNC personnel will record the deviation in OnCore®  (or 
other appropriate database set up for the study), and report to any sponsor or data 
and safety monitoring committee in accordance with their policies.  Deviations 
should be summarized and reported to the IRB at the time of continuing review. 
 
Protocol Violations: Violations should be reported by UNC personnel within one 
(1) week of the investigator becoming aware of the event using the same IRB 
online mechanism used to report UPIRSO.   
 
Unanticipated Problems Involving Risks to Subjects or Others (UPIRSO): 
Any events that meet the criteria for “Unanticipated Problems” as defined by 
UNC’s IRB (see section 6.1) must be reported by the Study Coordinator using the 
IRB’s web-based reporting system. The PI will be responsible for reporting 
UPIRSO to the PCRC and the NINR.   
 
 
8.5 
Amendments to the Protocol 
Should amendments to the protocol be required, the amendments will be 
originated and documented by the Principal Investigator at UNC.  It should also 
be noted that when an amendment to the protocol substantially alters the study 
design or the potential risk to the patient, a revised consent form might be 
required.   

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
21 
 
The written amendment, and if required the amended consent form, must be sent 
to UNC’s IRB for approval prior to implementation.   
8.6 
Record Retention 
Study documentation includes all Case Report Forms, data correction forms or 
queries, source documents, Sponsor-Investigator correspondence, monitoring 
logs/letters, and regulatory documents (e.g., protocol and amendments, IRB 
correspondence and approval, signed patient consent forms). 
 
Source documents include all recordings of observations or notations of clinical 
activities and all reports and records necessary for the evaluation and 
reconstruction of the clinical research study. 
 
Government agency regulations and directives require that all study 
documentation pertaining to the conduct of a clinical trial must be retained by the 
study investigator.  In the case of a study with a drug seeking regulatory approval 
and marketing, these documents shall be retained for at least two years after the 
last approval of marketing application in an International Conference on 
Harmonization (ICH) region.  In all other cases, study documents should be kept 
on file until three years after the completion and final study report of this 
investigational study. 
8.7 
Obligations of Investigators 
The Principal Investigator is responsible for the conduct of the clinical trial at the 
site in accordance with Title 21 of the Code of Federal Regulations and/or the 
Declaration of Helsinki.  The Principal Investigator is responsible for personally 
overseeing the treatment of all study patients.  The Principal Investigator must 
assure that all study site personnel, including sub-investigators and other study 
staff members, adhere to the study protocol and all FDA/GCP/NCI regulations 
and guidelines regarding clinical trials both during and after study completion. 
 
The Principal Investigator at each institution or site will be responsible for 
assuring that all the required data will be collected and entered onto the Case 
Report Forms. Periodically, monitoring visits will be conducted and the Principal 
Investigator will provide access to his/her original records to permit verification 
of proper entry of data. At the completion of the study, all case report forms will 
be reviewed by the Principal Investigator and will require his/her final signature 
to verify the accuracy of the data. 
 
8.8 
Data and Safety Monitoring Plan 
The Principal Investigator will provide continuous monitoring of subject safety in 
this trial with periodic reporting to the UNC Lineberger Data and Safety Monitoring 
Committee (DSMC).  
 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
22 
Monitoring Procedures: 
Meetings/teleconferences will be held at a frequency dependent on study accrual.  
These meetings will include the investigators as well as protocol nurses, clinical 
research associates, regulatory associates, data managers, biostatisticians, and any 
other relevant personnel the principal investigators may deem appropriate.  At these 
meetings, the research team will discuss all issues relevant to study progress, 
including enrollment, safety, regulatory, data collection, etc. 
 
The team will produce summaries or minutes of these meetings. These summaries 
will be available for inspection when requested by any of the regulatory bodies 
charged with the safety of human subjects and the integrity of data including, but 
not limited to, the Office of Human Research Ethics (OHRE) Biomedical IRB, the 
Oncology Protocol Review Committee (PRC) or the North Carolina TraCS Institute 
Data and Safety Monitoring Board (DSMB).   
 
The UNC LCCC Data and Safety Monitoring Committee (DSMC) will review the 
study once the protocol is finalized. Based on the “minimal risk” nature of the study 
(behavioral, non-therapeutic trial), it is anticipated that the DSMC will review the 
study on an annual basis. The UNC PI will be responsible for submitting the 
following information for review: 1) safety and accrual data including the number 
of subjects treated; 2) significant developments reported in the literature that may 
affect the safety of participants or the ethics of the study; 3) preliminary response 
data; and 4) summaries of team meetings that have occurred since the last report.  
Findings of the DSMC review will be disseminated by memo to the UNC PI, PRC, 
and the UNC IRB and DSMB. Summaries will also be made available for review 
by the PCRC and the National Institute of Nursing Research (NINR). The PI will 
be responsible for submitting summaries to the PCRC and the NINR. In accordance 
with our institutional guidelines, studies deemed “minimal risk” will not be audited.   
 
Minimization of Risk:  
Some questions in the patient questionnaires, interviews or survey could create 
emotional distress or confusion. Patients will be instructed that the questionnaires 
are for research purposes only and any problem or symptom should also be reported 
to their physician or nurse. 
 
We do not anticipate any breach of confidentiality although recognize that a breach 
in confidentiality may result in psychological, social, or economic harm to a patient 
if the details of the medical record become available to others. Any information 
about the participants obtained from this research will be kept as confidential as 
possible. All data obtained will be stored on a secure server.   
 
 
9.0 
REFERENCES 
             

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
23 
1 Richardson DR, Crossnohere NL, Seo J, Estey E, O’Donoghue B, Smith BD, et al. 
Age at Diagnosis and Patient Preferences for Treatment Outcomes in AML: A 
Discrete Choice Experiment to Explore Meaningful Benefits. Cancer Epidemiol 
Biomark Prev Publ Am Assoc Cancer Res Cosponsored Am Soc Prev Oncol 
2020;29:942–8. https://doi.org/10.1158/1055-9965.EPI-19-1277. 
2 Bridges JFP, Mohamed AF, Finnern HW, Woehl A, Hauber AB. Patients’ preferences 
for treatment outcomes for advanced non-small cell lung cancer: A conjoint analysis. 
Lung Cancer 2012;77:224–31. https://doi.org/10.1016/j.lungcan.2012.01.016. 
3 Bien DR, Danner M, Vennedey V, Civello D, Evers SM, Hiligsmann M. Patients’ 
Preferences for Outcome, Process and Cost Attributes in Cancer Treatment: A 
Systematic Review of Discrete Choice Experiments. Patient - Patient-Centered 
Outcomes Res 2017;10:553–65. https://doi.org/10.1007/s40271-017-0235-y. 
4 Aiello Bowles EJ, Tuzzio L, Wiese CJ, Kirlin B, Greene SM, Clauser SB, et al. 
Understanding high-quality cancer care: a summary of expert perspectives. Cancer 
2008;112:934–42. https://doi.org/10.1002/cncr.23250. 
5 Wagner EH, Aiello Bowles EJ, Greene SM, Tuzzio L, Wiese CJ, Kirlin B, et al. The 
quality of cancer patient experience: perspectives of patients, family members, 
providers and experts. Qual Saf Health Care 2010;19:484–9. 
https://doi.org/10.1136/qshc.2010.042374. 
6 Beaussant Y, Mathieu-Nicot F, Pazart L, Tournigand C, Daneault S, Cretin E, et al. Is 
shared decision-making vanishing at the end-of-life? A descriptive and qualitative 
study of advanced cancer patients’ involvement in specific therapies decision-making. 
BMC Palliat Care 2015;14:61. https://doi.org/10.1186/s12904-015-0057-4. 
7 Brom L, De Snoo-Trimp JC, Onwuteaka-Philipsen BD, Widdershoven GAM, 
Stiggelbout AM, Pasman HRW. Challenges in shared decision making in advanced 
cancer care: a qualitative longitudinal observational and interview study. Health 
Expect Int J Public Particip Health Care Health Policy 2017;20:69–84. 
https://doi.org/10.1111/hex.12434. 
8 Harrison M, Milbers K, Hudson M, Bansback N. Do patients and health care providers 
have discordant preferences about which aspects of treatments matter most? Evidence 
from a systematic review of discrete choice experiments. BMJ Open 2017;7:e014719. 
https://doi.org/10.1136/bmjopen-2016-014719. 
9 Mühlbacher AC, Juhnke C. Patient preferences versus physicians’ judgement: does it 
make a difference in healthcare decision making? Appl Health Econ Health Policy 
2013;11:163–80. https://doi.org/10.1007/s40258-013-0023-3. 
10 Lee CN, Chang Y, Adimorah N, Belkora JK, Moy B, Partridge AH, et al. Decision 
making about surgery for early-stage breast cancer. J Am Coll Surg 2012;214:1–10. 
https://doi.org/10.1016/j.jamcollsurg.2011.09.017. 
11 LeBlanc TW, Fish LJ, Bloom CT, El-Jawahri A, Davis DM, Locke SC, et al. Patient 
experiences of acute myeloid leukemia: A qualitative study about diagnosis, illness 
understanding, and treatment decision-making. Psychooncology 2017;26:2063–8. 
https://doi.org/10.1002/pon.4309. 
12 Loh KP, Abdallah M, Kadambi S, Wells M, Kumar AJ, Mendler JH, et al. Treatment 
decision-making in acute myeloid leukemia: a qualitative study of older adults and 
community oncologists. Leuk Lymphoma 2020;0:1–12. 
https://doi.org/10.1080/10428194.2020.1832662. 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
24 
13 Rood J a. J, Nauta IH, Witte BI, Stam F, van Zuuren FJ, Manenschijn A, et al. Shared 
decision-making and providing information among newly diagnosed patients with 
hematological malignancies and their informal caregivers: Not ‘one-size-fits-all’. 
Psychooncology 2017;26:2040–7. https://doi.org/10.1002/pon.4414. 
14 El‐Jawahri A, Nelson‐Lowe M, VanDusen H, Traeger L, Abel GA, Greer JA, et al. 
Patient‐Clinician Discordance in Perceptions of Treatment Risks and Benefits in Older 
Patients with Acute Myeloid Leukemia. The Oncologist 2018:theoncologist.2018-
0317. https://doi.org/10.1634/theoncologist.2018-0317. 
15 Hunter NL, O’Callaghan KM, Califf RM. Engaging Patients Across the Spectrum of 
Medical Product Development: View From the US Food and Drug Administration. 
JAMA 2015;314:2499. https://doi.org/10.1001/jama.2015.15818. 
16 Rocque G, Miller-Sonnet E, Balch A, Stricker C, Seidman J, Stiles S, et al. Engaging 
Multidisciplinary Stakeholders to Drive Shared Decision-Making in Oncology. J 
Palliat Care 2019;34:29–31. https://doi.org/10.1177/0825859718810723. 
17 Bonamici S. H.R.34 - 114th Congress (2015-2016): 21st Century Cures Act. 2016. 
URL: https://www.congress.gov/bill/114th-congress/house-bill/34 (Accessed 27 May 
2020). 
18 Wicker RF. S.1597 - 114th Congress (2015-2016): Patient-Focused Impact 
Assessment Act of 2016. 2016. URL: https://www.congress.gov/bill/114th-
congress/senate-bill/1597 (Accessed 27 May 2020). 
19 Hauber AB, González JM, Groothuis-Oudshoorn CGM, Prior T, Marshall DA, 
Cunningham C, et al. Statistical Methods for the Analysis of Discrete Choice 
Experiments: A Report of the ISPOR Conjoint Analysis Good Research Practices Task 
Force. Value Health 2016;19:300–15. https://doi.org/10.1016/j.jval.2016.04.004. 
20 Bridges JFP, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, et al. 
Conjoint Analysis Applications in Health—a Checklist: A Report of the ISPOR Good 
Research Practices for Conjoint Analysis Task Force. Value Health 2011;14:403–13. 
https://doi.org/10.1016/j.jval.2010.11.013. 
21 Janssen EM, Marshall DA, Hauber AB, Bridges JFP. Improving the quality of 
discrete-choice experiments in health: how can we assess validity and reliability? 
Expert Rev Pharmacoecon Outcomes Res 2017;17:531–42. 
https://doi.org/10.1080/14737167.2017.1389648. 
22 Hawley ST, Li Y, An LC, Resnicow K, Janz NK, Sabel MS, et al. Improving Breast 
Cancer Surgical Treatment Decision Making: The iCanDecide Randomized Clinical 
Trial. J Clin Oncol Off J Am Soc Clin Oncol 2018;36:659–66. 
https://doi.org/10.1200/JCO.2017.74.8442. 
23 Jayadevappa R, Chhatre S, Gallo JJ, Wittink M, Morales KH, Lee DI, et al. Patient-
Centered Preference Assessment to Improve Satisfaction With Care Among Patients 
With Localized Prostate Cancer: A Randomized Controlled Trial. J Clin Oncol 
2019:JCO.18.01091. https://doi.org/10.1200/JCO.18.01091. 
24 Mansfield C, Masaquel A, Sutphin J, Weiss E, Gutierrez M, Wilson J, et al. Patients’ 
priorities in selecting chronic lymphocytic leukemia treatments. Blood Adv 
2017;1:2176–85. https://doi.org/10.1182/bloodadvances.2017007294. 
25 Mühlbacher AC, Nübling M. Analysis of physicians’ perspectives versus patients’ 
preferences: direct assessment and discrete choice experiments in the therapy of 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
25 
multiple myeloma. Eur J Health Econ 2011;12:193–203. 
https://doi.org/10.1007/s10198-010-0218-6. 
26 Herrmann A, Mansfield E, Hall AE, Sanson-Fisher R, Zdenkowski N. Wilfully out of 
sight? A literature review on the effectiveness of cancer-related decision aids and 
implementation strategies. BMC Med Inform Decis Mak 2016;16:36. 
https://doi.org/10.1186/s12911-016-0273-8. 
27 Steffensen KD, Vinter M, Crüger D, Dankl K, Coulter A, Stuart B, et al. Lessons in 
Integrating Shared Decision-Making Into Cancer Care. J Oncol Pract 2018;14:229–
35. https://doi.org/10.1200/JOP.18.00019. 
28 Austin CA, Mohottige D, Sudore RL, Smith AK, Hanson LC. Tools to Promote 
Shared Decision Making in Serious Illness: A Systematic Review. JAMA Intern Med 
2015;175:1213–21. https://doi.org/10.1001/jamainternmed.2015.1679. 
29 Brehaut JC, O’Connor AM, Wood TJ, Hack TF, Siminoff L, Gordon E, et al. 
Validation of a Decision Regret Scale. Med Decis Making 2003;23:281–92. 
https://doi.org/10.1177/0272989X03256005. 
30 Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing 
CollaboRATE: A fast and frugal patient-reported measure of shared decision making 
in clinical encounters. Patient Educ Couns 2013;93:102–7. 
https://doi.org/10.1016/j.pec.2013.05.009. 
31 Garvelink MM, Boland L, Klein K, Nguyen DV, Menear M, Bekker HL, et al. 
Decisional Conflict Scale Findings among Patients and Surrogates Making Health 
Decisions: Part II of an Anniversary Review. Med Decis Making 2019;39:315–26. 
https://doi.org/10.1177/0272989X19851346. 
32 Richardson DR, Oakes AH, Crossnohere NL, Rasthmill G, Reinhart CA, O’Donoghue 
B, et al. Prioritizing the Worries of AML Patients: Quantifying Patient Experience 
using Best-Worst Scaling. Psychooncology 2021. https://doi.org/10.1002/pon.5652. 
33 O’Connor AM. Validation of a decisional conflict scale. Med Decis Mak Int J Soc Med 
Decis Mak 1995;15:25–30. https://doi.org/10.1177/0272989X9501500105. 
34 Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and 
preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 
Int J Qual Life Asp Treat Care Rehabil 2011;20:1727–36. 
https://doi.org/10.1007/s11136-011-9903-x. 
35 Basch E, Reeve BB, Mitchell SA, Clauser SB, Minasian LM, Dueck AC, et al. 
Development of the National Cancer Institute’s patient-reported outcomes version of 
the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer 
Inst 2014;106:. https://doi.org/10.1093/jnci/dju244. 
36 Ma X, Zhang J, Zhong W, Shu C, Wang F, Wen J, et al. The diagnostic role of a short 
screening tool--the distress thermometer: a meta-analysis. Support Care Cancer Off J 
Multinatl Assoc Support Care Cancer 2014;22:1741–55. 
https://doi.org/10.1007/s00520-014-2143-1. 
37 Donovan KA, Grassi L, McGinty HL, Jacobsen PB. Validation of the distress 
thermometer worldwide: state of the science. Psychooncology 2014;23:241–50. 
https://doi.org/10.1002/pon.3430. 
38 Was it worth it (WIWI)? Patient satisfaction with clinical trial participation: Results 
from North Central Cancer Treatment Group (NCCTG) phase III trial N0147.: 
Journal of Clinical Oncology: Vol 29, No 15_suppl. n.d. URL: 

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
26 
http://ascopubs.org/doi/abs/10.1200/jco.2011.29.15_suppl.6122 (Accessed 19 
September 2018). 
39 Ramsay MC, Reynolds CR. Separate digits tests: a brief history, a literature review, 
and a reexamination of the factor structure of the Test of Memory and Learning 
(TOMAL). Neuropsychol Rev 1995;5:151–71. https://doi.org/10.1007/BF02214760. 
40 Tractenberg RE, Freas CE. Benchmarking a computerized test of immediate verbal 
memory. Behav Res Methods 2007;39:863–9. https://doi.org/10.3758/bf03192980. 
41 Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content 
validity--establishing and reporting the evidence in newly developed patient-reported 
outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good 
research practices task force report: part 1--eliciting concepts for a new PRO 
instrument. Value Health J Int Soc Pharmacoeconomics Outcomes Res 2011;14:967–
77. https://doi.org/10.1016/j.jval.2011.06.014. 
42 Willis GB. Cognitive interviewing: A tool for improving questionnaire design. sage 
publications; 2004. 
43 Leidy NK, Vernon M. Perspectives on Patient-Reported Outcomes. 
PharmacoEconomics 2008;26:363–70. https://doi.org/10.2165/00019053-200826050-
00002. 
44 Larkin M, Watts S, Clifton E. Giving voice and making sense in interpretative 
phenomenological analysis. Qual Res Psychol 2006;3:102–20. 
https://doi.org/10.1191/1478088706qp062oa. 
45 Pe M, Dorme L, Coens C, Basch E, Calvert M, Campbell A, et al. Statistical analysis 
of patient-reported outcome data in randomised controlled trials of locally advanced 
and metastatic breast cancer: a systematic review. Lancet Oncol 2018;19:e459–69. 
https://doi.org/10.1016/S1470-2045(18)30418-2. 
46 Stover AM, Urick BY, Deal AM, Teal R, Vu MB, Carda-Auten J, et al. Performance 
Measures Based on How Adults With Cancer Feel and Function: Stakeholder 
Recommendations and Feasibility Testing in Six Cancer Centers. JCO Oncol Pract 
2020;16:e234–50. https://doi.org/10.1200/JOP.19.00784. 
47 Johnson FR, Yang J-C, Reed SD. The Internal Validity of Discrete Choice Experiment 
Data: A Testing Tool for Quantitative Assessments. Value Health 2019;22:157–60. 
https://doi.org/10.1016/j.jval.2018.07.876. 
10.0 APPENDICES 
Included appendices:  
1. Baseline questionnaire  
2. PROMIS cognitive function 
3. BWS survey 
4. Decision Self-Efficacy Scale  
5. DCS  
6. EQ-5D-5L 
7. PRO-CTCAE  
8. PROMIS emotional distress SF 8a  
9. Follow-up questionnaire  
10. CollaboRATE  
11. DRS  

LCCC 2136 
 
CONFIDENTIAL 
PI: Richardson, Daniel  
 
UNIVERSITY OF NORTH CAROLINA 
Version/Date 1.0/Feb 26, 2021 
 
Health Services Research September 2014 Version 
 
27 
12. 1 year questionnaire  
13. Decision Self-efficacy  
14. Qualitative interview guide, initial, patients  
15. Qualitative interview guide, follow-up, patients  
16. Qualitative interview guide, clinicians  
17. Digit span test  
18. WIWI 
19. Provider questionnaire