QUESTIONS (make sure to ask each item not covered spontaneously) Would you do it again the same way? PROMPTS: What would you keep the same? What would you do differently? Lessons learned “if you were talking to a group of colleagues doing this kind of work, what would you advise and why?” QUESTIONS (make sure to ask each item not covered spontaneously) Identification of users Recruitment of users Numbers and mix of people Preparation or training Involvement (type, level and timing) The particular experience or perspective that users bring Incentives or other supports to participation Do you have any other thoughts that you would like to share? Thank you so very much for making the time to talk to me. Appendix 2. Included Projects Article(s) Did the project specifically involve members of vulnerable populations? Reason(s) people included or involved may be vulnerable Clinical context of patient decision aid Allen et al., 2009[28] Yes Race and Ethnicity Prostate Cancer Anderson et al., 2011[29] No (but they were included within a diverse population) Older Age Ovarian Cancer Arterburn et al., 2011[30] Yes Other: Obesity Obesity Au et al., 2011[31], Lam et al., 2013[32] No (but they were included within a diverse population) Lower Literacy Breast Cancer Auvinen et al., 2004[33] No members of vulnerable population(s) reported Prostate Cancer Bailey Jr. et al., 2012[34], Bailey Jr. et al., 2013[35] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income Fragile X Syndrome Barnieh et al., 2011[36] No members of vulnerable End-Stage Renal Disease
population(s) reported Barry et al., 1995[37], Barry et al., 1997[38], Piercy et al., 1999[39], Rovner et al., 2004[40], Holmes-Rovner et al., 2006[41], Wills et al., 2006[42], Pylar et al., 2007[43] No (but they were included within a diverse population) Older Age; Lower Education or Income Benign Prostatic Hyperplasia Bass et al., 2013[44] Yes Race and Ethnicity; Lower Education or Income; Lower Literacy Colorectal Cancer Bastian et al., 2002[45], McBride et al., 2002[46] No members of vulnerable population(s) reported Reproductive Health; Breast Cancer Becker et al., 2009[47] Yes Other: Mobility Impairments Reproductive Health Bekker et al., 1999[48], Bekker et al., 2004[49] No members of vulnerable population(s) reported Reproductive Health; Down Syndrome Berman et al., 2011[50] No (but they were included within a diverse population) Older Age Abdominal Aortic Aneurysm Berry et al., 2010[51], Berry et al., 2013[52] No (but they were included within a diverse population) Race and Ethnicity Prostate Cancer Breslin et al., 2008[53], Mullan et al., 2009[54] No (but they were included within a diverse population) Older Age Diabetes Brink et al., 2000[55], Fagerlin et al., 2004[56] No members of vulnerable population(s) reported Prostate Cancer Brohan et al., 2014a[57], Brohan et al., 2014b[58] Yes Mental Health Condition(s) Mental Health Chewning et al., 1999[59] Yes Younger Age; Race and Ethnicity; Lower Education or Income; Lower Literacy Reproductive Health Chiew et al., 2008[60] No (but they were included within a diverse population) Older Age; Race and Ethnicity; Lower Education or Income Breast Cancer Consoli et al., 1995[61] No members of vulnerable Cardiovascular Health (Heart
population(s) reported Disease, Stroke, etc…) Costanza et al., 2011[62] No (but they were included within a diverse population) Race and Ethnicity; Lower Literacy Prostate Cancer Cox et al., 2012[63] No (but they were included within a diverse population) Race and Ethnicity Prolonged Mechanical Ventilation Dales et al., 1999[64], Wilson et al., 2005[65] No (but they were included within a diverse population) Older Age; Lower Education or Income Advance Care Planning; Chronic Obsructive Pulmonary Disorder (COPD) Davis et al., 1998[66] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income Vaccine Preventable Disease Dillard et al., 2010[67] No members of vulnerable population(s) reported Colorectal Cancer Dodin et al., 2001[68] No (but they were included within a diverse population) Older Age; Lower Education or Income Reproductive Health Dolan & Frisina, 2002[69] No members of vulnerable population(s) reported Colorectal Cancer Dolan et al., 2013[70] No (but they were included within a diverse population) Race and Ethnicity Knee Osteoarthritis Dowding et al., 2004[71], Thomson et al., 2006[72] No (but they were included within a diverse population) Older Age; Other: Computer Literacy Benign Prostatic Hyperplasia; Cardiovascular Health (Heart Disease, Stroke, etc…) Drake et al., 2010[73] Yes Race and Ethnicity Prostate Cancer Durand, 2009[74], Durand et al., 2010[75], Durand et al., 2012[76] Yes Race and Ethnicity Reproductive Health Elit et al., 1996[77] No members of vulnerable population(s) reported Ovarian Cancer Emmett et al., 2007[78], Montgomery et al., 2007[79], Frost et al., 2009[80], Hollinghurst et al., 2010[81] Yes Reproductive Health
2001[136], Wise et al., 2008[137] Hamann et al., 2006[138], Hamann et al., 2007[139] Yes Mental Health Condition(s) Schizophrenia Hawkins Virani et al., 2013[140] No members of vulnerable population(s) reported Huntington's disease Heinrich et al., 2012[141] No members of vulnerable population(s) reported Diabetes Henderson et al., 2013[142] Yes Race and Ethnicity Diabetes Hightow-Weidman et al., 2011[143], Muessig et al., 2013[144], Muessig et al., 2014[145], Hightow-Weidman et al., 2015[146] Yes Race and Ethnicity; Lower Education or Income; Mental Health Condition(s); Sexual Orientation Sexually Transmitted Infections (STIs); HIV Hill-Briggs et al., 2008[147], Schumann et al., 2011[148], Majid et al., 2012[149] No (but they were included within a diverse population) Older Age; Race and Ethnicity Diabetes Hoffner et al., 2012[150] No members of vulnerable population(s) reported Cancer Holbrook et al., 2007[151] No members of vulnerable population(s) reported Overall health; Cardiovascular Health (Heart Disease, Stroke, etc…) Hollen et al., 2013a[152] Yes Younger Age; Cognitive Impairments Substance Use Hollen et al., 2013b[153] No members of vulnerable population(s) reported Prostate Cancer; Breast Cancer; Lung cancer Holmes-Rovner et al., 2005[154] Yes Lower Literacy Prostate Cancer Hong et al., 2013[155] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Hooker et al., 2011[156] No members of vulnerable population(s) reported Breast Cancer
Hope & Rombauts, 2010[157] No members of vulnerable population(s) reported Reproductive Health Hutchison & Campbell, 2002[158] No members of vulnerable population(s) reported Cancer Ickenroth et al., 2014[159], Ronda et al., 2014[160] No members of vulnerable population(s) reported Diabetes; Cardiovascular Health (Heart Disease, Stroke, etc…) Irwin et al., 1999[161] No (but they were included within a diverse population) Lower Education or Income Breast Cancer Jackson et al., 2010[162], Shourie et al., 2013[163], Tubeuf et al., 2014[164] No members of vulnerable population(s) reported Vaccine Preventable Disease Jenkinson et al., 1998[165] No (but they were included within a diverse population) Older Age; Sexual Orientation Prostate Cancer Jibaja-Weiss et al., 2006a[166], et al., 2006b[167], et al., 2011[168] Yes Lower Literacy Breast Cancer Johnson et al., 2006[169] No members of vulnerable population(s) reported Endodontics Johnson et al., 2010[170], Langston et al., 2010[171] No members of vulnerable population(s) reported Reproductive Health Juan et al., 2008[172] No members of vulnerable population(s) reported Breast Cancer; Ovarian Cancer Juraskova et al., 2008[173], Juraskova et al., 2014[174] No (but they were included within a diverse population) Older Age Breast Cancer Kellar et al., 2008[175], Mann E. et al., 2010[176], Marteau et al., 2010[177], Kellar et al., 2011[178] No members of vulnerable population(s) reported Diabetes Kennedy et al., 2002[179], No members of vulnerable Reproductive Health Weiss
Pignone et al., 2011[199] Income; Lower Literacy Li et al., 2013[200], Li et al., 2014[201] No members of vulnerable population(s) reported Rheumatoid Arthritis Liao et al., 1996[202], Morgan et al., 2000[203] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Loh et al., 2007[204] No members of vulnerable population(s) reported Depression Lurie et al., 2011[205] No members of vulnerable population(s) reported Herniated Disc Mancini et al., 2006[206] No members of vulnerable population(s) reported Breast Cancer; Ovarian Cancer Man-Son-Hing et al., 1999[207], Man-Son-Hing et al., 2000[208], McAlister et al., 2005[209] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Mathers et al., 2012[210], Ng et al., 2014[211] No (but they were included within a diverse population) Older Age; Lower Education or Income Diabetes Mathieu et al., 2007[212] Yes Older Age; Lower Education or Income Breast Cancer Mathieu et al., 2010[213] No members of vulnerable population(s) reported Breast Cancer Matlock et al., 2014[214] No members of vulnerable population(s) reported Advance Care Planning Mayer et al., 2010[215] No members of vulnerable population(s) reported Hematopoietic Stem Cell Transplant McCaffery & Irwig, 2005[216], McCaffery et al., 2006[217], McCaffery et al., 2008[218], McCaffery et al., 2010[219] No members of vulnerable population(s) reported Reproductive Health; Cervical Cancer
McKay et al., 2005[220] No members of vulnerable population(s) reported Breast Cancer Miller et al., 2005[221] No members of vulnerable population(s) reported Breast Cancer; Ovarian Cancer Milne et al., 2009[222] No members of vulnerable population(s) reported Reproductive Health Mitchell et al., 2001[223], Hanson et al., 2011[224], Snyder et al., 2013[225] Yes Lower Education or Income; Mental Health Condition(s) Dementia Montgomery et al., 2003[226], Emmett et al., 2005[227] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Montori et al., 2007[228], Weymiller et al., 2007[229], Jones et al., 2009[230], Nannenga et al., 2009[231], Mann D.M. et al., 2010[232] No members of vulnerable population(s) reported Diabetes; Cardiovascular Health (Heart Disease, Stroke, etc…) Murray et al., 2001a[233] No members of vulnerable population(s) reported Benign Prostatic Hyperplasia Murray et al., 2001b[234] No members of vulnerable population(s) reported Reproductive Health Nassar et al., 2006[235], Nassar et al., 2007[236] No members of vulnerable population(s) reported Reproductive Health Nozaki et al., 2007[237] No members of vulnerable population(s) reported Unruptured Cerebral Aneuryms O’Connor et al., 1998a[238], O’Connor et al., 1998b[239], Rostom et al., 2002[240] No (but they were included within a diverse population) Older Age; Lower Education or Income Reproductive Health Onel et al., 1998[241] No members of vulnerable Prostate Cancer
population(s) reported Ozanne et al., 2007[242] No members of vulnerable population(s) reported Breast Cancer Peate et al., 2011a[243], Peate et al., 2011b[244], Peate et al., 2012[245] No members of vulnerable population(s) reported Reproductive Health; Breast Cancer Pencille et al., 2009[246], Montori et al., 2011[247] No members of vulnerable population(s) reported Osteoporosis Perestelo-Perez et al., 2010[248] No (but they were included within a diverse population) Lower Education or Income Benign Prostatic Hyperplasia Perestelo-Perez et al., 2010[248] No (but they were included within a diverse population) Lower Education or Income Depression Perestelo-Perez et al., 2010[248] No (but they were included within a diverse population) Lower Education or Income Hip or Knee Osteoarthritis Permuth-Wey et al., 2010[249] Yes Race and Ethnicity Breast Cancer; Ovarian Cancer Pierce et al., 2010[250], Hess et al., 2012[251] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Pignone et al., 2000[252], Kim et al., 2005[253], Griffith et al., 2008a[254], Griffith et al., 2008b[255] No (but they were included within a diverse population) Older Age; Race and Ethnicity; Lower Education or Income Colorectal Cancer Pignone et al., 2004[256], Sheridan et al., 2006[257], Sheridan et al., 2009[258], Sheridan et al., 2010[259], Sheridan et al., 2011[260] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income; Numeracy Cardiovascular Health (Heart Disease, Stroke, etc…) Raats et al., 2008[261] No members of vulnerable population(s) reported Prostate Cancer; Reproductive Health; Depression; Down Syndrome; Breast Cancer; Cardiovascular Health (Heart
Disease, Stroke, etc…) Raynes-Greenow et al., 2009[262], Raynes-Greenow et al., 2010[263] No members of vulnerable population(s) reported Reproductive Health Rothert et al., 1997[264], Holmes-Rovner et al., 1999[265] No members of vulnerable population(s) reported Reproductive Health Ruthman & Ferrans, 2004[266] No (but they were included within a diverse population) Older Age Prostate Cancer Saver et al., 2007[267] No members of vulnerable population(s) reported Reproductive Health Sawka et al., 1998[268], Goel et al., 2001[269] No (but they were included within a diverse population) Older Age; Lower Education or Income Breast Cancer Schackmann et al., 2013[270] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income Breast Cancer; Ovarian Cancer Schapira & VanRuiswyk, 2000[271] No (but they were included within a diverse population) Older Age; Lower Education or Income Prostate Cancer Schapira et al., 1997[272] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income Prostate Cancer Schapira et al., 2007[273] No members of vulnerable population(s) reported Reproductive Health Schonberg et al., 2014[274] Yes Older Age Breast Cancer Schroy et al., 2011[275], Schroy et al., 2012[276], Schroy et al., 2014[277] No (but they were included within a diverse population) Race and Ethnicity; Lower Literacy Colorectal Cancer Schwalm et al., 2012[278] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Schwartz, L.M. et al., 2009[279] No members of vulnerable population(s) reported Cardiovascular Health (Heart Disease, Stroke, etc…) Schwartz, M.D. et al., 2009[280] No members of vulnerable population(s) reported Breast Cancer Sepucha et al., 2009[281] No (but they were included within a diverse population) Older Age; Lower Education or Income Breast Cancer
Shaffer et al., 2013[282] No members of vulnerable population(s) reported Breast Cancer Sheppard et al., 2010[283] Yes Breast Cancer Sheridan et al., 2004[284] No (but they were included within a diverse population) Older Age; Race and Ethnicity Prostate Cancer Sherman et al., 2014[285] No members of vulnerable population(s) reported Breast Cancer; Ductal Carcinoma In Situ (DCIS) Shorten et al., 2004[286], Shorten et al., 2005[287] Yes Lower Education or Income Reproductive Health Simon et al., 2012[288] Yes Mental Health Condition(s) Depression; Acute Low Back Pain Smith et al., 2008[289], Trevena et al., 2008[290], Smith et al., 2009[291], Smith et al., 2010[292] Yes Lower Literacy Colorectal Cancer Solberg et al., 2010[293] No members of vulnerable population(s) reported Reproductive Health Spunt et al., 1996[294], Deyo et al., 2000[295], Phelan et al., 2001[296] No (but they were included within a diverse population) Older Age Low Back Problems (Herniated Disks; Spinal Stenosis; and other diagnoses) Sridhar et al., 2015[297] No members of vulnerable population(s) reported Reproductive Health Stacey et al., 2003[298] No members of vulnerable population(s) reported Breast Cancer Stacey et al., 2014[299] No members of vulnerable population(s) reported Knee Osteoarthritis Stalmeier & Roosmalen, 2009[300] No members of vulnerable population(s) reported Breast Cancer; Ovarian Cancer Stalmeier & Roosmalen, 2009[300] No members of vulnerable population(s) reported Breast Cancer; Ovarian Cancer
Steckelberg et al., 2004[301], Steckelberg et al., 2011[302] No members of vulnerable population(s) reported Colorectal Cancer Stein et al., 2013[303] No members of vulnerable population(s) reported Advance Care Planning; Cancer Stiggelbout et al., 2008[304] No members of vulnerable population(s) reported Abdominal Aortic Aneurysm Stirling et al., 2012[305] Yes Lower Education or Income; Mental Health Condition(s) Dementia Sudore et al., 2014[306] Yes Older Age; Race and Ethnicity; Lower Education or Income; Lower Literacy; Mental Health Condition(s) Advance Care Planning Tan et al., 2012[307], Tan et al., 2014[308] No members of vulnerable population(s) reported Psoriasis Taylor et al., 2006[100], Dorfman et al., 2010[309], Kassan et al., 2012[310], Taylor et al., 2013[311], Williams et al., 2013[312] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income; Other: Internet Access Prostate Cancer Taylor et al., 2006[100], Dorfman et al., 2010[309], Kassan et al., 2012[310], Taylor et al., 2013[311], Williams et al., 2013[312] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income; Other: Internet Access Prostate Cancer Taylor et al., 2006[100], Dorfman et al., 2010[309], Kassan et al., 2012[310], Taylor et al., 2013[311], Williams et al., 2013[312] No (but they were included within a diverse population) Race and Ethnicity; Lower Education or Income; Other: Internet Access Prostate Cancer
Appendix 3. Variables Considered In this table, we use the term users as shorthand for patients, caregivers, family members, or surrogate decision makers, whichever was appropriate for the decision aid under consideration. Clinician users refers to clinicians who were not on the research team and who might potentially use the decision aid with their patients. Variable Projects that specifically involved members of vulnerable populations (n=30) Projects that did not specifically involve members of vulnerable populations (n=157) Decision aid is intended to be used at home or in a private setting 13 (43%) 85 (54%) Decision aid is intended to be used in a health care setting but not during a clinical encounter 8 (27%) 47 (30%) Decision aid is intended to be used during a clinical encounter 9 (30%) 61 (39%) Decision aid users were involved in a formal needs assessment 5 (17%) 24 (15%) **Decision aid users were involved in an informal needs assessment 22 (73%) 63 (40%) *Decision aid users were involved in a content review prior to prototype development 9 (30%) 25 (16%) Decision aid users were involved in prototype development 7 (23%) 38 (24%) Decision aid users were involved in a content format review after prototype development 11 (37%) 57 (36%) *Decision aid users were involved in a pilot test 26 (87%) 111 (71%) Decision aid users were involved in a second pilot test 3 (10%) 17 (11%) Decision aid development was iterative 24 (80%) 134 (85%)