Recruitment Recruitment and Retention Retention of Wom

Wom

Health Research Participants: Health Research Participants: Lesso

Lesso

Learned From NIH Trials (And ErroLearned From NIH Trials (And Erro

Barbara B. Cochrane, PhD, RN, FAAN de Tornyay Endowed Professor for Healthy Aging School of Nursing y g , University of WashingtonWomen’s Health Initiative Clinical Coordinating Center, 

Fred Hutchinson Cancer Research Center 

Seattle,WA Nancy Reame, MSN, PhD, FAAN Mary Dickey Lindsay Professor School of Nursing School of Nursing Pilot Studies Director, 

Irving Institute for Clinical and Translational Research 

Columbia University Columbia University New York, NY 

SoWH CyberSeminar, Janua
 

 

Poll Question #1 

• What is your primary role in VA?
 

– student, trainee, or fellow 

– clinician 

– researcher 

– manager or policy-maker
 

– other
 
 

 

Poll Question #2 

•	 Which best describes your research 

experience? 

–	 have not done research 

–	 have collaborated on research 

–	 have conducted research myself 

–	 have applied for research funding
 

–	 have led a funded research grant 
 

 

 

The WHI is funded by the
 

National Heart, Lung, and Blood Institute,
 

National Institutes of Health,
 

U.S. Department of Health & Human Services
 

(through contracts N01WH22110, 24152, 32100-2, 32105-6, 32108-9, 

32111-13, 32115, 32118-32119, 32122, 42107-26, 42129-32, and 44221) 
 

Why WHI? 

• Address major causes of morbidity and
 mortality in postmenopausal women:
 –	 coronary heart disease 

–	 breast cancer 

– colorectal cancer
 

– osteoporotic fractures
 

•	 Inadequate research on 

women’s health 

– concerns about hormonal cycles
 

and pregnancy
 

–	 lack of women investigators 

•	 Inadequate research in minority populations
 
 

 

 

A Vast Scientific Undertaking ... 

• 161,808 participants followed ≤12 years in main WHI (1993-2005)
 

• 115,403 enrolled in WHI Extension Study (2005-2010) 

• Over 93,000 enrolled in Extension Study-2 (2010-2015) 

• 40 Clinical Centers in urban, suburban and rural areas 

– 3 CCs designated bone density centers 

– 10 CCs with satellite sites 

– 10 minority centers w/access to American Indian, African 

American, Asian American/Pacific Islander, Hispanic 

populations 
WHI Clinical Centers 

Clinical Center 

Seattle 

Portland 

Sacramento

Reno 

Oakland 

Stanford 

Los Angeles 

Torrance Orange 

Honolulu 

San DiegoTucson 

San Antonio 

Houston 

Gainesville 

Miami 

Birmingham 

Atlanta 

Memphis Chapel Hill 

Winston-Salem 

Washington, DC 

Cincinnati 

Columbus

Pittsburgh 

Newark

Stony Brook 

Pawtucket 

Boston 

Worcester 

Bronx 

Detroit Buffalo 

Minneapolis 

Milwaukee 

Madison 

Iowa CityChicago 

Minority Clinical Center 
 

 

 

Hormone Therapy Trials: 

Coronary Heart Disease & Fractures 

Adverse effect for Breast Cancer? 

(16,608 E+P; 10,739 E-Alone) 

Calcium/Vitamin D Trial: 

Fractures & Colorectal Cancer 

Dietary Modification Trial: 

Breast & Colorectal Cancers & Coronary 

Heart Disease 

93,676 

Observational Study 

48,835 

36,282 

3 Controlled Trials 1 Observational Study 

27,347 

161,808 women total 

WHI Components & Primary Outcomes 
 

 

Enrollment Goals 

• Study component (HT, DM, CaD, OS)
 

•	 Age group: 

10% 50-54 years 

20% 55-59 years 

45% 60-69 years 

25% 70-79 years 

•	 Minority centers - 60% racial/ethnic 

minority participants to ensure 

proportions representative of 

general population 
 

 

 

WHI Timeline
 

• Recruitment/screening: 1993 to 1998
 

– Vanguard CCs started in September 1993 

– New CCs started in February 1995 

• Follow-up for 7-12 years 

– CT: contacts every 6 months 

– OS: annual mailings; 3-year clinic visit
 

• Close-out: Oct. 2004 – Mar. 2005 

• WHI Extension Study: 2005-2010
 

• WHI Extension Study-2: 2010-2015
 
 

 

 

 

Eligibility Criteria
 

•	 General criteria for inclusion in CT and OS:
 

–	 Ages 50 to 79 years 

–	 Postmenopausal 

–	 Planning to reside in the area for ≥ 3 years 

– Able/willing to provide written informed
 

consent
 

•	 Additional eligibility criteria specific to each 

study component, related to: 

–	 Safety 

–	 Competing risk 

–	 Adherence/retention 
Type of Initial Contact with WHI 
 

80
 

70
 

60
 

50
 

40
 

30
 

20
 

10
 

0
 

Mail TV Paper/Mag Friend/rel 

50-59
 

60-69
 

70-79
 
 

80 

Type of Initial Contact with WHI
 

70
 

60
 

50
 

40
 

30
 

20
 

10
 

0
 

Mail TV Paper/Mag Friend/rel 

Amer Ind. 

Asian 

Black 

Hispanic 

White 
 

WHI Initial Screening
 

• Mass mailing of recruitment brochure
 

– provided basic information 

– postage-paid return postcard to
 

indicate interest
 

• Telephone screening by 

trained interviewers
 

• Clinic visits 

– 3-5 for randomization in 

Clinical Trial (HT or DM)
 

– 1 for enrollment in
 

Observational Study
 
 

 

Retention and Adherence 

Primary Prevention 

During screening: 

•	 Fully inform participants about study burden, do not 

minimize inconvenience and effort required 

•	 Fully inform that benefits may not accrue to the 

participant in a research trial 

•	 Fully inform about the meaning of randomization 

•	 Inquire about their past intervention behavior 

•	 Have a “trial run” (HT run-in, 4-day food record, CaD taste test) 

•	 Attend to subtle and overt cues during screening visits 

(rescheduling, lateness, difficulty contacting, hesitancy) 
 

 

 

 

 

Baseline Characteristics of WHI
 

Final Enrollment Participants
 

HT DM CaD OS 

N=27,347 N=48,835 N=36,282 N=93,676 

N % N % N % N % 

Race/Ethnicity 

American Indian 131 0.5 203 0.4 149 0.4 422 0.5 

Asian/Pacific Island’r 527 1.9 1107 2.3 722 2.0 2671 2.9 

Black 2741 10.0 5266 10.8 3317 9.1 7639 8.2 

Hispanic 1543 5.6 1854 3.8 1507 4.2 3623 3.9 

White 22027 80.5 39760 81.4 30153 83.1 78013 83.3 
 

 

 

 

Notes about Data Collection 

•	 Data collection protocols (forms & frequency) 

based on study component, study year, WHI 

priorities, and subsampling 

•	 All participant materials, 

including self-report forms, 

available in Spanish 

•	 Self-report forms: 

–	 Most were scannable 

– Most could be completed by
 

interview or w/assistance
 

•	 Follow-up outcome reports obtained from 

participants, proxies (as needed & available), 

National Death Index (vital status) 
 

 

 

Notes about Coordinating and
 

Monitoring Recruitment Efforts
 

• Recruitment Coordinator (RC)
 

at each site & CCC RC liaison
 

•	 Weekly national RC calls & 

email newsletter to share ideas 

•	 Use of logo, study colors, & 

catch-phrase on all recruitment 

materials (central review) 

•	 Performance Monitoring 

Committee focus on 

recruitment and assistance 
 

 

 

WHI Websites 

• WHI Study - www.whi.org 

– Scientific resources - assist scientists with 

proposing papers or ancillary studies using WHI 

data or biospecimens 

– Participant information - for lay public 

• NHLBI - www.nhlbi.nih.gov/whi 

Thanks to the WHI participants! 
 

 

 

 

 

 

 

WHI Investigators (A Short List)
 

Program Office: (NHLBI, Bethesda, Maryland) Barbara Alving, Jacques Rossouw, Shari Ludlam, Linda Pottern, Joan McGowan, Leslie Ford, and Nancy Geller.
 

Clinical Coordinating Center: (Fred Hutchinson Cancer Research Center, Seattle, WA) Ross Prentice, Garnet Anderson,
 Andrea LaCroix, Charles L. Kooperberg, Ruth E. Patterson, Anne McTiernan; (Wake Forest University School of
 Medicine, Winston-Salem, NC) Sally Shumaker; (Medical Research Labs, Highland Heights, KY) Evan Stein; (University
 of California at San Francisco, San Francisco, CA) Steven Cummings.
 

Clinical Centers: (Albert Einstein College of Medicine, Bronx, NY) Sylvia Wassertheil-Smoller; (Baylor College of 

Medicine, Houston, TX) Jennifer Hays; (Brigham and Women's Hospital, Harvard Medical School, Boston, MA) JoAnn 

Manson; (Brown University, Providence, RI) Annlouise R. Assaf; (Emory University, Atlanta, GA) Lawrence Phillips; (Fred 

Hutchinson Cancer Research Center, Seattle, WA) Shirley Beresford; (George Washington University Medical Center, 

Washington, DC) Judith Hsia; (Harbor-UCLA Research and Education Institute, Torrance, CA) Rowan Chlebowski; (Kaiser 

Permanente Center for Health Research, Portland, OR) Evelyn Whitlock; (Kaiser Permanente Division of Research, 

Oakland, CA) Bette Caan; (Medical College of Wisconsin, Milwaukee, WI) Jane Morley Kotchen; (MedStar Research 

Institute/Howard University, Washington, DC) Barbara V. Howard; (Northwestern University, Chicago/Evanston, IL) 

Linda Van Horn; (Rush Medical Center, Chicago, IL) Henry Black; (Stanford Prevention Research Center, Stanford, CA) 

Marcia L. Stefanick; (State University of New York at Stony Brook, Stony Brook, NY) Dorothy Lane; (The Ohio State 

University, Columbus, OH) Rebecca Jackson; (University of Alabama at Birmingham, Birmingham, AL) Cora E. Lewis; 

(University of Arizona, Tucson/Phoenix, AZ) Tamsen Bassford; (University at Buffalo, Buffalo, NY) Jean Wactawski- Wende; (University of California at Davis, Sacramento, CA) John Robbins; (University of California at Irvine, CA) F. Allan 

Hubbell; (University of California at Los Angeles, Los Angeles, CA) Howard Judd; (University of California at San Diego, 

LaJolla/Chula Vista, CA) Robert D. Langer; (University of Cincinnati, Cincinnati, OH) Margery Gass; (University of 

Florida, Gainesville/Jacksonville, FL) Marian Limacher; (University of Hawaii, Honolulu, HI) David Curb; (University of 

Iowa, Iowa City/Davenport, IA) Robert Wallace; (University of Massachusetts/Fallon Clinic, Worcester, MA) Judith 

Ockene; (University of Medicine and Dentistry of New Jersey, Newark, NJ) Norman Lasser; (University of Miami, Miami, 

FL) Mary Jo O’Sullivan; (University of Minnesota, Minneapolis, MN) Karen Margolis; (University of Nevada, Reno, NV) 

Robert Brunner; (University of North Carolina, Chapel Hill, NC) Gerardo Heiss; (University of Pittsburgh, Pittsburgh, PA) 

Lewis Kuller; (University of Tennessee, Memphis, TN) Karen C. Johnson; (University of Texas Health Science Center, San 

Antonio, TX) Robert Brzyski; (University of Wisconsin, Madison, WI) Gloria E. Sarto; (Wake Forest University School of 

Medicine, Winston-Salem, NC) Denise Bonds; (Wayne State University School of Medicine/Hutzel Hospital, Detroit, MI) 

Susan Hendrix. 
 

 

 

 

Recruitment and Retention of Women’s Health
 

Research Participants: Lessons Learned from
 

NIH trials (and errors)
 

Nancy Reame, MSN, PhD, FAAN 

School of Nursing & Irving Institute for Clinical & Translational Research
 

Columbia University, New York, NY
 

With contributions by Barbara Cochrane, PhD, RN
 

School of Nursing, University of Washington
 
 

 

WHI Working Group: advisory role
 

to the NHLBI Director
 

• Judith Hochman, MD 

• Linda Burhansstipanov, DrPH 

• Deborah Dingell 

• Stephen Fortmann, MD 

• Lou Glasse, MSW 

• Elmer Huerta, MD, MPH 

• M. Margaret Kemeny, MD 

• Gennell Knatterud, PhD 

• Shiriki Kumanika, PhD 

• Eugene Passamani, MD 

• Alfred Poindexter, MD 

• Nancy Reame, PhD, RN 

NIH NHLBI
 

Director
 

WHI CO-DIRECTORS WHI CO-DIRECTORS 

WHI WORKING GROUP 
 

 

 

WHI recruitment sites
 

Minority = 60% women of color; 

Other sites = proportional to their 

general population; GOAL: 20% 

minorities enrolled 
 

 

 

 

 

WHI Recruitment required wise planning!
 

Recruitment is a process, not a single step (Walson,1999)
 

•	 Identifying & attracting the eligible patient populations 

•	 Adequately explaining the study 

•	 Recruiting an adequate, representative sample 

•	 Obtaining true informed consent (in many cultures, this 

may hinge on family/spouse’s views) 

•

•	 Minimizing cost-benefit ratio 

Retaining participants until study completion 
Women’s Health Initiative
 

Timeline
 

Planning Recruitment Follow-up Close-out 

1993
 1998
 

1990 2005
 
 

National Recruitment Activities
 

•	 Central development of materials 

(brochures, videos) 

•	 Toll free national recruitment 

telephone line (1-800-54-WOMEN) 

•	 National public awareness campaign 

to develop materials and aid with 

national media placement 

•	 Central training and support of staff 

•	 Study-wide advisory groups 
 

 

 

 

WHI Special Strategies to Encourage a 

Diverse Population of Elderly Women
 

•	 Allowing extra time at screening visits for older women 

•	 Providing extra assistance for women with physical

limitations 

•	 Using large size print, good color contrasts 

•	 6th grade reading level 

•	 Spanish translation 

•	 All materials printed with WHI logo/colors 

•	 Consistent use of visual images (three heads logo) 

•	 And verbal messages: “Be part of the answer,” passing on a 

legacy of good health 
 

WHI “Branding”
 

•	 All materials printed 

with WHI logo/colors 

•	 Consistent use of 

visual images (three 

heads logo) 

•	 And verbal 

messages: “Be part of

the answer,” passing 

on a legacy of good 

health 

 

 

 

 

 

Clinical Center
 

Recruitment Strategies
 

Mass mailing was primary method for initial contact 

Each CC mailed an average of 1000-5000 brochures per 

month for 3-5 years! 

Supplemental Strategies: 

• Community presentations 

• Local newsletter articles and ads 

• Public service announcements (TV and radio) 

• Name-a-friend programs 

• Health fairs 

• National and local press releases 

• Health care provider referrals 
 

WHI Creative Local Strategies 

• Airplane message flying over OSU football game
 

•	 Local and national celebrities including Ms. Senior

America, Olympia Dukakis 

•	 Collaboration with local businesses to fund incentive 

items, newsletters, and transportation 

•	 Planned events such as fashion shows 

•	 Paid advertisements on buses, billboards, coupon packs,

employee payroll stuffers 
Engaging Volunteers in 
 

Recruitment: examples
 

Postcards were 

developed for 

recruited women to 

mail to their friends 

and family 
 

Focus Group Feedback is Key!
 

•	 Branding messages 

•	 Celebrity testimonials 

•	 Incentives 

•	 Videos 

•	 Posters and flyers 

•	 Public Service 

Announcements for radio, 

TV, and print media 

• WHI Logo magnets,
 

buttons, stickers
 
 

 

 

 

 

 

 

 

 

 

 

Recruitment Issues: Making Trials “User-Friendly”
 

• Engaging community gate-keepers/stakeholders in the research agenda 

Detroit Church pastors, Hispanic women city representatives in Miami 

• Tailoring Site-Specific recruitment strategies in response to focus groups 

- motivational messages “Be part of the Answer” ; “for the next generation” 

• Developing TRUST of the scientific/medical communities- listening to the 

participants about travel costs, clinic hours, communication methods 

- hiring participants to work as recruiters, educators, team members 

• Clinic site personnel and researchers should “look like me” 

- women of color represented in mass media alerts, recruitment flyers, 

websites 

- at study onset, very few PIs were women!... 
 

 

 

 

 

 

Adapting Study Questionnaires/Tools for 

WHI Populations: Physical Activity
 

•Tracked self-reported “moderate” intensity physical activities 

that included child care, housework, occupation, and not just 

leisure-time conditioning and sports activities 

• monthly 4-day sets of 24hr PA diaries x 6 mos, kept by AA 

women in Birmingham, Alabama and 107 NA residents of 

Pueblo or Navajo reservations in New Mexico 

• top 5 PA: occupation, walking for exercise, house cleaning,
 

child care, lawn & garden; few reported sports/conditioning
 

• ethnic differences in patterns and categories of exercise 

(Ainsworth et al, J Women’s Health Gender-based Med, 1999) 
 

 

 

Retention Issues: Making Trials “User-Friendly”
 

•	 Modifying interventions in line with lifestyle needs 

- Site-specific cooking classes, dietary modifications to reduce fat intake 

•	 Active monitoring of recruitment goals successes & failures for timely recalibrations 
 

 

 

 

Improving Adherence in the CaD: A Case Example* 

•	 Early CaD Adherence was low (~60% had >80% adherence). 

•	 CCC performed survival analyses with time to first non-adherent visit

as the outcome. Factors predicting poor adherence were: 

•	 younger age 

•	 minority ethnicity 

•	 unmarried status 

•	 not having a 4-week adherence phone contact 

•	 moderate to severe symptoms of gas or constipation 

•	 Women who stopped CaD reported problems taking the pills. 

* (LaCroix, 2005)
 
LaCroix, 2005
 
 

 

Improving Adherence in the CaD: A Case 

Example*
 Increased awareness of problems with chewable formulation: 

• taste 

• consistency 

• size 

• Hardness
 

Taste test (7/96)
 

Participant material revisions (9/96)
 

4-week call (10/96)
 

CaD survey of 100 participants (late 1996)
 

New swallowable formulation (10/97)
 

Swallowable re-formulation (early 1998)
 

* LaCroix, 2005 
 

 

 

Hot Tips: Adherence*
 

•	 Screen for good adherers during recruitment
 

•	 Conduct the 4-week call to assess possible 

problems using consistent staff 

•	 Develop a structure to support adherence: 

- designate a key retention staff person 

- have monthly retention committee meetings 

- identify candidates for and implement

interventions 

•	 Offer “step-down” and “step-up” options or pill

“holidays” 

*Lacroix,2005 
 

 

 

 

 

Feasability of Recruiting Postmenopausal Women from Diverse Racial & Social
 

Backgrounds into a Primary Prevention Trial Requiring Marked Reductions in
 

Dietary Fat. (Lewis et al, 1998, fig 2)
 

Cumulative percent recruitment goal by ethnicity:
 

AA = 605, NHW- 1253, Hispanic = 350 
 

 

 

 

 

 

 

 

 

Conclusions: WHI Recruitment*
 

1.Retention begins at recruitment, really.
 

2.WHI was privileged to have a national recruitment infrastructure and $1 

million to spend on national recruitment efforts. 

3.Mass mailing was the backbone of our recruitment success. 

4.Minority recruitment was a challenge, but tenacious effort paid off. 

5. The greatest success comes from systematized pro-active efforts and 

individualized reactive efforts by staff. 

6. Adherence monitoring and special studies can help to inform innovations

that improve adherence. 

7. Encouraging clinic staff (e.g. nurses) to become more data savvy is an 

important future direction. 

*Trial Implementation II: Recruitment, Retention and Adherence, A. LaCroix, 2005 
 

 

 

 

From Gerontological Society of America 

64th Annual Scientific Meeting 

November 18 – 22, 2011: 

What Have We Learned from the WHI about
 

Racial/Ethnic Minority Women’s Health?
 Cochrane, Benavente, Coker
 
 

 

Even with WHI’s large numbers, our understanding of older

racial/ethnic minority women’s health continues to be 

challenged by: 

 Comparisons of “white” with “non-white” 

 Comparisons across groups 

 Lack of subgroup differential analyses by racial/ethnic 

group: 

 Hispanic/Latinas 

 Asian/Pacific Islanders 
 

 

 Future studies should include analyses by racial/ethnic 

group: 

 To gain new insights about potential mechanisms for 

study findings 

 To guide methods for future studies that incorporate 

alternative approaches to typical linear models used 

in epidemiology 

 Structural equation modeling 

 Interpretive/inductive methods 

 To generate hypotheses for future research 
 

 

 

 

 

 

Effective Recruitment & Retention of Minority Research 

Participants
 AK Yancey, AN Ortega, SK Kumanyika, Ann Rev Pub Health, 2006 

• Systematic review of 95 studies 1999-2005 of methods to 

increase minority participation 

•“federally-defined” under-represented minority ethnic groups 

•10 factors emerged re recruitment success: 

- 6 related to features of the participants (religiosity, attitudes 

about disease, psychosocial health, trust of medical/scientific 

community; cultural adaptations) 

- 4 related to the study design, sampling, incentives, 

community involvement 
 

 

 

 

 

 

 

Women’s Health Research: Progress, Pitfalls &
 

Promises: Recommendations (IOM, 2010)
 

1. Mainstream women’s health: routinely assess sex differences 

in all health studies; 

2. Govt agencies should develop targeted initiatives to enhance 

research on groups at highest risks/burdens 

3. Outcomes research should include quality of life 

4. Social determinants of health behaviors/disease risk 

5. Sex-stratified analysis required for publication; add to journal 

guideline 

6. Research on how to translate findings in clinical guidelines 

7. HSS task force to develop evidence-based health messaging 

strategies to the public
 
 

 

 

 

 

 

Women’s Health Research, 1990: “We need a 

moon walk for women” B. Healy, NIH Director.
 

•Until the 1990’s, female subjects (humans and rats!) were
 

underrepresented in biomedical research
 

•Women scientists <15% NIH grant awardees
 

•IN RESPONSE, 

•* Offices of Women’s Hlth/Minority Hlth
 

* The Women’s Health Initiative (WHI) 

* The National Centers for Infertility Research (U54) 

•Legislation mandated that women be included in RCTs 
Questions??
 

Thank you! 

To the 161,808 WHI participants