**Transcript of Cyberseminar**

**Spotlight On Women’s Health**

**Recruitment and Retention of Women's Health Research Participants: Lessons Learned from NIH trials (and errors)**

**Presenters:  Barbara B. Cochrane, PhD, RN, FAAN;  Nancy Reame, MSN, PhD, FAAN**

**January 17, 2013**

**Recruitment and Retention of Women's Health Research Participants: Lessons Learned from NIH trials (and errors)**

**January 17, 2013**

Molly:So as we are approaching the top of the hour, I would like to take this time to introduce our two presenters. Once again, today’s topic is “Recruitment and retention of Women’s Health Research Participants: Lessons Learned from NIH Trials (And Errors).” Speaking first we have Dr. Barbara Cochrane, a de Tornyay Endowed Professor for Healthy Aging School of Nursing at the University of Washington and Women’s Health initiative Clinical Coordinating Center at the Fred Hutchinson Cancer Research Center in Seattle, Washington. Joining her is Dr. Nancy Reame, a Marcy Dickey Lindsay Professor School of Nursing and Pilot Studies Director at the Irving Institute for Clinical and Translational Research at Columbia University in New York. So at this time I would like to turn it over to you, Dr. Cochrane ... You should see a popup that says show my screen. Go ahead and click on that. Excellent ... And we are ready to go.

Dr. Barbara Cochrane:Okay. Well, welcome everyone and thank you so much for joining us today. As Molly said, my name is Barbara Cochrane and I am with the University of Washington School of Nursing; but today the affiliation that is probably most relevant is that I am at the WHI Clinical Coordinating Center. So Dr. Nancy Reame and I are going to do a sort of a tag-team presentation today. I am going to offer some of the background, setting a foundation for the WHI. And then Dr. Reame is going to talk to you more about the lessons learned, the trials and errors of the Women’s Health Initiative.

But first, we would like to, if I can move the slide here ... okay. We have some poll questions for you. And I am going to turn it over to Molly right now.

Molly:Thank you. We would like to get an idea of who is joining us in our audience today, so the question is, what is your primary role within VA? Please select one of the following options on your screen. You can click directly on the circle next to your answer. The options are student, trainee or fellow; clinician; researcher; manager or policy-maker; or other. Looks like we have had about two-thirds of our audience vote, so we will give people just a few more seconds to get their entry in ... and the answers have stopped streaming in. We have had about 80 percent of our audience vote, so I am going to go ahead and close the poll and share the options. Nancy, would you like to talk through those real quick?

Dr. Nancy Reame:Sure. Hi, everybody. I am Nancy Reame from Columbia University. I think I need to point out that it is the Mary Lindsay Professorship that I hold. Mary Lindsay is actually the sister-in-law of the former New York mayor, John Lindsay. 

I see. It is interesting. I have never done this before, these polls, and it looks like the majority of you are researchers. Is there anything else you wanted me to say, Molly?

Molly:No. And I apologize.

Dr. Nancy Reame:That is okay.

Molly:I forgot Barbara was presenting at this time. I will just talk through them real quick.

Dr. Nancy Reame:Okay.

Molly:So we have 7 percent student, trainees or fellows; 4 percent clinicians; 47 percent researchers; 9 percent manager or policy-maker; and we have 30 percent other. So thank you to those of you who responded. I am now going to share the other poll. We would like to know what best describes you research experience? So the answers are, have not done research, have collaborated on research, have conducted research myself, have applied for research funding, or have led a funded research grant. And we have had about 70 percent of our audience vote now, so we will give people a few more seconds to get their responses in. We do appreciate your contribution as it helps the presenters gear the presentation towards your understanding of the topic. And once again, we have about 80 percent of our audience vote, so I am going to go ahead and close the poll and share the results. And Barbara, would you like to talk to the percentages real quick.

Dr. Barbara Cochrane:Okay. So it looks like the majority of you have conducted research yourself. We have 8 percent who have not done research. I am hoping you are emerging researchers; let us put it that way. Thirty-two percent have collaborated on research; 40 percent have conducted research themselves; 4 percent have applied for research funding; and a wonderful 16 percent have actually let a funded research grant. So thank you so much for answering our poll.

Molly:Thank you. And I am going to go ahead and turn the screen sharing back over to you ... okay.

Dr. Barbara Cochrane:Okay. Skipping past this, I wanted to mention we are going to be talking about the Women's Health Initiative as our example of a women’s health NIH trial. And the Women's Health Initiative has been funded by the National Heart, Lung and Blood Institute of the National Institutes of Health. But I would also like to acknowledge that particularly in the earlier years of the study, the WHI represented a consortium of institutes. All were collaborating together and the WHI was funded out of the Director’s Office before it moved into the National Heart, Lung and Blood Institute.

Why the WHI? The WHI was formed to address major causes of morbidity and mortality in postmenopausal women and the focus of the clinical conditions were coronary heart disease, breast cancer, colorectal cancer, and osteoporotic fractures, so we described those as our primary outcomes.

The reason for the WHI came about from some early research agenda-setting meetings nationally because there was inadequate research on women’s health. Many reasons for that. Two reasons that are relevant are that there were concerns about hormonal cycles in pregnancy and so women were excluded from studies and also there was a lack of women investigators. And as you can probably well imagine, women are most tuned in to what might be issues for women. And so by encouraging women investigators’ participation within the WHI, we also have identified and responded/answered many other interesting research questions.

Another key reason why the WHI came about and was designed in the way it was is because there has been inadequate research in minority populations.

The WHI was a vast scientific undertaking—still is. We had almost 162,000 participants who were followed for up to 12 years in the main study and then 115,000 re-consented to participate in the five-year Extension Study. And after that five years was up, another 93,500 participants re-consented to stay with the WHI for the second Extension Study. The two extension studies are essentially observational studies, but I will talk about the other components of the WHI in a minute.

There were 40 clinical centers across the United States who came from urban, suburban, and rural areas. Three of those clinical centers also were designated bone density centers and they obtained bone density measurements on their participants over the life of the study. Ten of the clinical centers had satellite sites and essentially we worked with them and they interacted as fairly distinct sites such that we often considered there were 50 centers within the WHI. And then ten of the centers were designated minority centers that had greater access to American Indian, African American, Asian American/Pacific Islander and Hispanic populations.

This is a map of the United States showing the WHI clinical centers. You can see actually that there is kind of a big swath through middle America where there are no centers. A key rationale for that is that centers had to go through an application process and so the centers that were named were those who were viewed as having the best proposals as well as access to a recruitment/catchment area of at least a million age-eligible women. 

Another thing I would like to point out is that in magenta, the minority clinical centers are noted; and you can probably imagine by looking at the regions of the United States where those minority clinical centers are, some of the racial ethnic minority populations that they had key access to.

This shows the WHI components for the main WHI that went on from, as I said, 1993 to 2005. It also describes the primary outcomes relevant to those components.

So the WHI had three clinical trials, randomized clinical trials: a Hormone Therapy Trial that was actually two separate trials of combined hormones or estrogen alone depending on whether a woman respectively still had a uterus or had already had a hysterectomy. We had a dietary modification trial that involved an intervention with a high fruits, vegetables and grain and the low fat diet. And then a year into the WHI, we invited women to join the Calcium and Vitamin D Trial.

Participants could be in one, two, or all three of the clinical trials. Participants in the Calcium and Vitamin D trial all were part of one of the other clinical trials.

In addition to the clinical trials component, we had an observational study of almost 94,000 participants, and these were women who were either not eligible for the clinical trials, or on hearing more about them decided that they were not interested in participating in the clinical trials but still wanted to be part of the answer and joined the WHI and so they were invited to participate in a no-intervention, long-term observational or cohort-based study.

We had specific enrollment goals for the WHI. They depended on the study components. And you can imagine that the enrollment goals per study component were based on power calculations and specific design assumptions that would allow us to test the hypotheses. 

We also had specific enrollment goals set out by age groups, and you notice that a much higher percentage of participants in the older age group and clinical centers actually had what we call recruitment cells close down once it reached a certain percentage of participants within a specific age group.

As I mentioned, we had the minority centers, ten of those. And their recruitment goals involved recruiting 60 percent of their participants from racial/ethnic minority populations with the idea that this would ensure that over across the entire WHI that we would have proportions that are representative of the general population.

The timeline for the WHI was that recruitment and screening started in 1993, but we actually started in ’93 with just 16 vanguard clinical centers who sort of tried out the protocols, the recruitment strategies, the messaging that was used. And then a year into the study, after we had tried all that out and made a few tweaks and adjustments, then 24 new clinical centers were starting up in February of 1995. So they had, over the course of the study, 40 clinical centers recruiting across the United States.

Depending on when the clinical center started up and the participants joined, they were followed up for seven to 12 years. Participants who were in one of the clinical trials had contacts every six months. Participants in the observational study had annual mailings but only came back into the clinic for a three-year clinic visit with measurements.

You may have heard about the hormone trials closing down or stopping the intervention in 2002 and 2004 for the combined hormone and estrogen-alone trials, respectively; but actually, the main WHI closed down in October 2004 to March of 2005. So our hormone trial participants, even though the intervention stopped earlier, they continued to be followed until 2005.

In 2005 we had, as I mentioned, the first WHI Extension Study. It went on for five years. One of the focuses of that Extension Study was to determine what occurred in terms of participants’ health after interventions were stopped. And then we are now right in the middle of WHI Extension Study-2. A key focus of that Extension Study is now looking at issues of aging for older women as well as really opening our arms and embracing and encouraging scientists from across the country and worldwide to take advantage of the resources that are available with the WHI.

And I should mention that our first WHI participant will be turning 100 in 2014, so you can see that it is quite a treasure trove of data and it represents an incredible contribution of these participants.

Eligibility criteria for the main WHI were fairly general overall, and they included that the woman needed to be between 50 and 79 years when she joined the study; postmenopausal, and that was based on a specific algorithm of experiences and symptoms; plans to reside in the area for three or more years; and able and willing to provide written informed consent.

In addition to those general criteria, we had additional eligibility criteria that were specific to each study component. And so we had criteria for safety. You can imagine that that was a key area for eligibility criteria in the hormone trial. Also eligibility based on competing risks, so for example the dietary modification trial had breast cancer as the primary outcome. And so participants who had already been diagnosed with breast cancer were not eligible for that trial. And then specific eligibility criteria related to adherence and retention in part because this was a tremendous commitment on the part of participants and we wanted to give them an opportunity to really think carefully about whether this was a challenge they wanted to take up, as well as us within the WHI to be able to determine if they were going to be able to complete the protocols over time.

Dr. Reame is going to talk more about recruitment and retention, but I thought you might be interested to see that the type of initial contact that was made by WHI with participants. So we asked participants when they joined the study, how did you hear about our study? And you can see that overwhelmingly most of them heard about the study through mail. And that is because, as Dr. Reame will mention, mass mailings were the key recruitment strategy for the WHI. But we also had a national campaign and local campaigns, so there was information on the television and newspapers and magazines. And then we had not only word-of-mouth, but some actual recruitment campaign efforts to have friends and relatives tell others about the study.

This breaks down those same kinds of initial contacts by racial/ethnic groups and you can see that for all of the racial/ethnic groups, again the mailings were the key way the participants heard about the WHI. But there are some interesting variations within the other strategies, and I think it is interesting that, for example, that the friends and relatives—that that was among – across the racial/ethnic groups, the Hispanic women saw that as a key way that they heard about the WHI.

Initial screening for the WHI really, then, started at the time of recruitment, as it should be in any clinical trial with mass mailings. The recruitment brochure provided basic information about the study including the basic eligibility criteria. They included a postage-paid return postcard to indicate interest. And although you are seeing the National Institutes of Health generic recruitment brochure, each of the clinical centers had a clinical center-specific brochure. So when they did mailings, the mailings would go out to people in their area with their recruitment information.

You will notice at the bottom of the brochure is a 1-800 number so that anybody who heard about the WHI could call that number and they would automatically be routed to their clinical center phone number based on their area code. Or if there was not a clinical center in the area, they would be routed to a voice mail to get additional information about the study.

When participants were contacted, there was an initial telephone screen by trained interviewers and if they were still interested in the study and eligible during that screening, then they were invited to come in and schedule for a clinical visit.

There were at least, for all participants, one clinic visit at baseline or the beginning of the study. For participants in the clinical trials, if they stayed interested and eligible after that first visit, there were an additional two to four more visits as they went through the screening and eligibility process. Participants in the observational study generally came into the WHI after that first clinic visit. 

Recruitment and retention all begin at the beginning of the study, and so we were addressing a sort of primary prevention strategy for retention and adherence during screening. And so some strategies that took place were that we would fully inform participants about the study burden, not trying to sugarcoat it, but really emphasizing the potential inconvenience and the effort that might be required. We reminded them that if they came in at this point this year, then they would be asked to come in at the same time each year or every six months. 

We fully informed them about the aspects of the study such that they would not misconstrue benefits that might occur. We did so many clinical physical measurements that it was a real challenge for us to convey to participants that this was not a study in which they would be getting free annual health checkups, that the measurements were targeted specifically to the outcomes and intermediate outcomes of the studies. So we wanted participants to understand that the benefits of participating in the WHI would not necessarily occur for them, but it would be benefiting future generations of women.

We also wanted to fully inform participants about the meaning of randomization. You can well imagine that participants joining had a perhaps somewhat of a view of whether they wanted to be in the intervention or the control arm of any one of the three clinical trials. And so our goal was to have them understand before they ever were randomized into the study that there would be a chance that they could be in one or the other and none of us would know. And so we sort of told them that if you think that you would not want to be in one arm or the other, then please let us know now before you are randomized. Because once you are randomized, no one can take your place. That was another catchphrase we had with participants.

They were asked about their past intervention behavior and specifically whether they were participating in any other intervention studies out of concern that the protocols and rigors of the WHI would be a burden if put on top of another intervention study. And then in our clinical trials, we essentially had a kind of a trial run. 

For the Hormone trial we had a four-week placebo run in really geared towards could women remember to take their pills on a daily basis. For the Dietary Modification trial, participants were asked to complete a four-day food record. And so this gave the participants as well as the study staff a sense of what it meant for the participant to be monitoring their food. And for the Calcium and Vitamin D trial, there was a taste test that was done to see which of the two formulations that we ended up with, if any, would be tolerable. And Dr. Reame is going to talk about that a little bit more.

And then another aspect of primary prevention for retention and adherence was attending to subtle and overt cues during screening visits. Participants who were rescheduling a lot, who were often very late for the visits, for whom we had difficulty making contact or who seemed hesitant, made some statements that would say, they were not really sure about the study and they hoped they got in one arm or the other, that was a cue for the study staff to talk with them a bit more and see if this was really a good choice for them.

This just shows you the baseline characteristics of the WHI final enrollment participants broken down by racial/ethnic groups. And so you can see how many and what percentage were participating in the Hormone trials, the Dietary Modification trial, the Calcium and Vitamin D trial and the Observational Study. And you can see from this table that we had anywhere from 17 to almost 20 percent of our participants in any one component coming from a racial/ethnic minority group.

A few notes about data collection that might be relevant are that the data collection protocols were actually quite complicated. Our effort was to ensure that those complications were relegated to a computer or the staff rather than study participants necessarily having to keep track of it. So they were informed each step of the way what we needed from them as opposed to them having to remember to fill out this form or another.

But it was based on which study component they were in; the study year because some measures were taken every three years, for example; the WHI priorities in terms of our primary outcomes; and then we had some subsampling strategies such that a small subsample of participants contributed blood every three years, whereas others just did it at the beginning of the study. Some participants participated in what we called the physical function subsample, and so every three years they had physical performance measures such as a chair stand and a timed walk done. And then our Hormone Trial participants who were 65 and over participated in the cognitive assessment subsample in which a Mini-Mental Status exam was done every one to two years depending upon whether they were participating in the ancillary WHI memory study.

All of our participant materials, recruitment materials, information materials, self-report forms, newsletters, were available to participants in Spanish and they could identify whether they preferred to get their materials in that way.

Self-report forms were mostly scannable. And you can imagine in part that is because there were just such huge numbers; and even at every clinical center the number of participants filling out any form could be upwards of 2,000. 

Most of the forms could be completed by interview or with assistance if necessary. And we did follow-up outcome reports every six months to a year from participants, but we also set up a system for the participants to identify and designate proxies so that if needed and if they were available, we could contact the proxy to get health information updates. And on a regular basis we also, I guess you could say, dipped into the National Death Index to determine vital status on participants. Perhaps we had not been able to get a hold of them and so we were able to get at least some confirmation of vital status by using the National Death Index.

Some additional notes. Since I came from the Coordinating Center, I wanted to mention about coordinating and monitoring recruitment efforts. Each clinical center had a designated recruitment coordinator. And then at the Coordinating Center, there was an RC liaison who was sort of the point person who worked with the recruitment coordinators and was always readily available to sort of keep those communications flowing. 

On the slides that you have – or the handout you have, it actually said, Weekly National RC Calls, but I realized that that is really from the Coordinating Center perspective. What the Recruitment Coordinators had was a monthly regional call and then we held four regional calls. It was pretty near impossible to have a really good discussion among 40 to 50 recruitment coordinators all on one conference call. So we broke out the clinical centers into four regions so that there would be a more manageable number of RCs on each conference call, and also so that they were sort of sharing some similar regional considerations.

Most of the materials for the WHI used the standardized logo, the study colors and the catchphrase “Be Part of the Answer” on their materials, even materials that were developed locally, were asked to include some aspects of that branding; but there were centrally-developed recruitment materials that were also made available to recruitment coordinators. So the locally developed materials actually went through a central review just to ensure that the information was consistent and accurate and sort of conveying an appropriate message.

There was a Central Performance Monitoring Committee set up that focused on recruitment but also specifically on assistance for clinical centers that were having some difficulty meeting their recruitment goal; and we would go out to clinical centers on performance enhancement visits. For those centers who wanted some assistance, we could observe activities going on at the clinical centers and offer some suggestions.

I would like to conclude by letting you know about the WHI website: www.whi.org. It is the WHI Study website that is still incredibly active. There are essentially two parts to the website, one for scientists and one for participants or the lay public. The scientific website has an incredible breadth and depth of information about the WHI, the data that are available, the outcomes that we have determined so far to assist them in proposing papers or secondary analyses or ancillary studies using our database or biospecimens.

The participant information still includes when important papers come out—a lay summary of that paper—and then we have an archive of lay summaries within that.

And you will notice at the bottom of the screen the thanks to the WHI participants. Part of the participant website that I find particularly interesting is the quilts, and you can see the quilts in a drop-down menu on the participant tab. Each of the clinical centers did all kinds of activities to build rapport and communication among their participants.

Many of the clinical centers allowed and encouraged participants to contribute squares to their WHI quilt. And the cool thing about the quilt section of the website is you can click around and look at the quilts coming from the different centers. Some of those quilts, when you click on a particular square, what pops up is information about why the participant designed their quilt square in a particular way. It is a wonderful way to stay grounded in who was participating in the study and why do participants – why do people and particularly women join studies like this.

NHLBI also has a WHI-specific website and you see the URL for that. That has much of the same information as the WHI Study website, but also will include sometimes updates on new funding initiatives on opportunities that are coming out that would use the WHI data and biospecimens.

This is just to sort of give you a sense that there was a huge number of people—still are—involved in the WHI, hundreds and hundreds of investigators, staff, and volunteers. We had participants volunteering, helping out in the clinical centers, which is amazing. So what this represents is just a short list of our WHI principal investigators. Many, many more people involved that deserve recognition.

And now I am going to turn things over to Dr. Nancy Reame to actually share the experiences that we had.

Molly:Thank you very much, Dr. Cochrane. And, Dr. Reame, you should now have a popup that says, show my screen. Great. Thank you.

Dr. Nancy Reame:Thanks. Thanks so much. Barbara, that was terrific. As someone who was a part of that for so long, I am still amazed at how much I learned—or else, how much I forgot. But in any case, that was a very thorough, detailed evaluation. And I just want to remind everybody, that website will get you to a section for Researchers. And you may be very interested, because in looking at the kinds of research questions that have already been done, I think there are roughly, I do not know, 600 publications and they are always looking for new research ideas. And all the rules and regs on that are there on that website. And I would be happy to assist anyone if you wanted more information, and I am sure Barbara would, too.

And so in my—I think I have ten minutes—I wanted to just mention that this presentation actually evolved or emerged ...

Molly:Dr. Reame?

Dr. Nancy Reame:Yes?

Molly:I am sorry to interrupt. We actually do not have any pending questions right now, so feel free to take more than ten minutes. I will interrupt if a bunch come in.

Dr. Nancy Reame:Oh, great. Please do. Yes. So this presentation we are doing today actually emerged from one that was part of a panel, a VA panel, actually, that we presented just a month ago, right before Christmas, at the NIH Summit on Eliminating Health Disparities. It had actually been originally scheduled—you will see the date there—right around the Hurricane Sandy crisis and so we had to reschedule.

But it was a wonderful opportunity to highlight the fabulous work of the VA and the Women's Health Initiatives and Practice, Policy and Research and I was fortunate to be a part of that group along with Linda Lipson, Jillian Shipherd, Kristen Maddox, and Sylvina Levis. I do not think I left anybody out. I am sorry I do not have our pictures here. That would have been a nice slide.

So I am going to now go forward. So my presentation today really comes out of my perspective as a member of the working group, the WHI working group, which was an advisory task force that was actually sitting and reporting to the NIH director originally. We were reporting to the director of the NIH in the early years.

And then when the WHI was transferred over to Heart, Lung and Blood, we then became the advisory task force or working group to the NHLBI director. And our challenge and our message and our goal and our mission was really to act as the interface between the public and the NIH and to serve as the voice, take questions, and raise concerns with ... I am hearing a lot of noise on my speaker. I just want to make sure ...

Molly:Yeah. I am wondering, Dr. Cochrane, do you have your line muted?

Dr. Barbara Cochrane:It is muted, yes.

Molly:Okay. I will make sure all of our attendees do as well.

Dr. Nancy Reame:Okay, thanks. Okay. I do not think it is happening at my end. So we were – our charge was literally to raise questions, concerns, ethics, anything we wanted to. And the group, I do not have all their affiliations—it would have been a very busy slide—but every six months we came in for our meetings to listen to the progress that was being made across all the activities of these 40 sites in terms of recruitment and in terms of even data analysis and outcomes, and to ask anything. We were free to ask any questions. 

And the group was a wonderful mix of mostly women, the majority were women; but academic scientists, clinicians, epidemiologists, activists at the national level, highly-respected public health leaders, and representing a variety of racial and ethnic minorities. So in that sense – and we were chaired by Judith Hochman, who is a cardiologist at NYU, I wanted to point out. So in that sense, it was a great, terrific group to work with. 

In some ways, it was an interesting relationship. I think there was kind of a love-hate relationship or maybe even a healthy tension. This is my own view I am expressing between – our relationship with the WHI investigators because in some ways on some days a thorn in their sides. But in other days, we were good ammunition if they felt there was a need for perhaps more budget for recruitment and retention, and that did come up, actually, as an issue.

And clearly, I am not assuming that any of you on the call today are going to be involved yet, at least, in the massive size of an undertaking like the WHI. But nonetheless, advisory work groups or advisory groups are wonderful tools to think about when you are doing your intervention trials, no matter how small, because they can be used in many ways to help support and advocate for you if you need to with your sponsor, for example.

I just also want to mention—you will see the red circle drawn around Detroit, Michigan. I also served as a member of the local advisory board. The Detroit Clinical Center was designated a minority center. It had a large population in Detroit of African American residents. And it was funded through Wayne State University. I was not working at the time for Wayne State, but I was a Detroit resident and active in the Detroit community and so was asked by Susan Hendricks to serve as a local advisory board member. And most folks on that board were community activists and community leaders who provided a great deal of wonderful input and feedback about – especially about recruitment and retention.

So just to remind all of us, one of the first lessons learned, and I am going to quickly skim over them, but I want to make sure to make the point that right up front there was an important planning that clearly we learned that enough time had to be dedicated to planning in order to prevent some potential missteps or perhaps even – at least modify and move forward if there were some missteps.

And understanding that recruitment is a process and not a single step, and that it involves, as you see, the entire range of research activities involving not only recruitment but retention. And as you have heard, retention starts at the time of recruitment; and all of this has to be done in a very cost-efficient way. 

In terms of the time frame, the planning process actually took three years, so a very careful attention to designing the study and going forward. And then a five-year time frame was allocated to recruitment. This is a wonderful way of tweaking the system, learning from what had gone on at the vanguard centers that you heard Dr. Cochrane refer to. So that throughout even the follow-up process as more feedback was coming in about meeting recruitment goals, interventions could be tweaked a bit here and there without compromising the scientific merit.

So lesson number one: Always develop ample time for your planning and for recruitment; and most people say double whatever you think is going to be your maximum time for each of these steps because it usually takes much longer than you plan on.

You have heard about the national recruitment activities. I am not going to go into those to any extent. Again, though, budgets for recruitment. I am assuming nobody on the call is going to have a $1 million budget for recruitment, which was the initial dollars allocated for the WHI, some $600 million altogether. So this was the mother of all clinical trials. But having said that, another important lesson learned is to take very seriously and plan well for the recruitment budget, because it really does take considerable money and time of staff. And I think sometimes we do not carefully plan for a percentage of effort or hiring additional staff to be specifically assigned to recruitment and retention activities will take a bite out of budget. And including costs that might have to be generated for your local advisory groups who will play an important role in specific recruitment and retention activities. So that is another important lesson you can take away.

We talked about the importance of these mass mailings, especially, in terms of recruitment, brochures, and also obviously translation into Spanish. But in addition because this was primarily an elderly population of women, the average age was 63, but as you saw 45 percent, I think, or so were 65 and older, there had to be special considerations in terms of their reading ability, the size of the print—any kind of physical disabilities that might hinder their ability or cognitive abilities to listen and understand and remember. 

And so consistency was key in all of the information, in the logos, in the colors. People, if there was a slight—I learned this from the Detroit site—if there was a slight change in the size of the logo or the colors did not come out right, some people questioned that and wanted to know because they paid very careful attention to all of this information, as was the same in the messaging, the wording.

So please keep that in mind when you are working with especially elder and an elderly population, given the veterans population you are working with.

And, of course, the branding we talked about, and some of this came out of the focus groups that were developed at each of the sites. But again, consistency in management and imaging were very important in the recruitment and retention. 

And just to point out that sometimes Spanish or translating into a second language, sometimes the quality of those recruitment materials are not the same or the color is different. It should not just be in black and white or printed out on a copier, because that will also perhaps interfere with recruitment successes. So everything should be carefully thought out and planned for in the budget.

And we mentioned that each site took advantage of the mass mailings, but also each site undertook very specific strategies based on input from their own volunteers from their own local advisory boards to seriously attend to the preferences and the interests of that specific community and that recruitment area and target population. So that in some cases TV and radio public service announcements worked very well. In other cases, they did not. Health fairs were another example that were more specific to some sites than others.

Other very creative local strategies included the airplane messages that went over your head at state football stadiums in the fall. That might have worked well in Columbus, but it would not necessarily work in other areas, as you might guess.

Billboards and the use of advertisements on the sides of busses, or subways, although we did not have a New York City site. But using the transportation facilities is an important area where people might see the ads, and selecting important local or national celebrities that resonated with the target population. In some cases Olympia Dukakis was an appropriate person, but not in others.

And we mentioned the person-to-person approach, especially when volunteers became active participants. As members of the research team, I cannot stress that enough, an important lesson that was learned. The women volunteers did indeed play important roles, especially in reaching out to other women in the community. So the use of postcards in some cities worked very well.

Actually, I just want to make sure that I note this. I do have some references for a number of community-based approaches around recruitment and retention have been described and are on the website under Publications.

I mention here in the next slide about focus group feedback being key. For example, Angela Lansbury was recruited by a national PR firm early on as someone who, according to the PR folks who were hired originally for the national advertising, was seen as having a great deal of respect with older women. But imagine, if you will, that this may or may or not work, depending on the site. And it turned out that in Miami, for example, folks that were most highly respected in the community and who women might listen to were Hispanic women members of the city council for Miami.

So again, these seem like no-brainer truisms. But this really was something that early on, as we were reaching out to minority populations, had to be learned. And the study designs and advertising approaches had to be modified.

So in this slide here I mention some of the important key points that talked about the Hispanic women city council members. Early on, every clinical site recognized and identified, with the help of their focus groups, important people in the community. Back then the word was gatekeeper or stakeholder in terms of recruitment. In Detroit it was the church pastors, mostly men. And so if permission and approval came from the pulpit, that was the entre to the community, and the advisory members of our local board in Detroit were composed of very important, highly respected members of that community, and women especially.

The messaging was important. African American women found that they resonated more to the message of “being part of the answer” or doing this “for the next generation,” and so that message tended to be favored, especially in the African American sites.

And certainly developing trust with the scientific community. Even though some of these academic settings were in the middle of urban centers, the level of trust with the community at times needed improvement. And so it was not enough to have focus groups talk about general issues around recruitment and retention, but it was important to drill down to what seemed like minutiae to the investigators. But very important, key issues that women themselves talked about that we needed to attend to and listen to and follow through with changes in clinic hours, for example, or reimbursement for costs. 

People in Detroit, for example, did not in the beginning appreciate that grandmothers might need – that we might need babysitters at the clinical site because this was a study of elderly women, and they needed to understand that many times they were the caretakers of young children. So the cost for babysitting had to be incorporated into the budgets of some of these sites.

And certainly Shiriki Kumanyika, who was a member of our advisory board, taught us very early on the importance of institutionalizing diversity in research, not only at the level of the PI, which was a very important issue that had to be changed and modified very early on, but also across every level of the clinical site, including the staff support. And just as important as skin tone was also being able to identify or coming from the community, so that hiring a Harvard-educated African American PhD student to be the study coordinator might be important, but at the same token, you also needed women who could understand the transportation problems and understand why someone might show up an hour late, or two hours late, and not be scolded about the importance of that, or leaving the clinic open longer because folks were working. So the need to also be diverse and be responsive to representing the community across socioeconomic status and not just ethnicity was an important lesson that we all learned.

The PI issue was an interesting one, and that was one of the first essential issues where our advisory board kind of came up against the NIH because we had to in a way scold the NIH about this and actually demand that there be a mentoring process put into place. Because at the very beginning most of the sites selected, there was a predominance of male—and VPIs—and yet there was no process put in place for the next generation or the training of the next generation. And so it was one of our, I think, successes as an advisory group to get that going and get that in place to really push for greater numbers of women scientists, which was one of the key reasons why the WHI was funded in the first place and why Bernadine Healy was so effective.

I just quickly want to—I am looking at the clock here—mention that it was not only the details of the administration of the recruitment and retention approaches, but also an important need that our advisory committee took to heart, as well as the scientists of the WHI, was to change up and modify and validate many of the instruments that were being used at the time, and health scales, to make them relevant for women in our study across groups, across ages.

Just one story I want to give. One of the members of our group, who was representing a Native American cancer organization said, this example, this physical activity scale, is absolutely, totally irrelevant to women in the tribal nations. These are women who do not go to gyms, who do not play golf—which were some common questions at the time on how much golf you might play. But they walk with their bags of flour. They might walk up to five miles to the center’s village to bake their bread at the village ovens and then come back to their rural residence. And so those questions and those concerns were taken to heart. And, indeed, I chose the Ainsworth and colleagues paper published in 1999 that talks about some of the modifications, especially around physical activity, that were made in response to thinking about the special needs of at the time we called them special populations in research. Moving forward now, clearly we still consider them vulnerable in many ways, but they’re certainly a growing population in terms of research.

One of the biggest changes, and I am going to quickly end here, because I see we are really close to ending, was around interventions. A lot of changes had to be made in terms of the lifestyle interventions and especially around calcium modifications and the actual size and the taste of the calcium supplements. It was found very early on that from 28 percent of the folks in the study were reporting health problems and that was the key reason why people were stopping the calcium supplements and dropping out of the trials.

So a whole feasibility study and thus considerable budget and effort and time had to be taken to reformulate, downsize the size of these medications, then which clearly enhanced the success of the interventions and the recruitment and retention goals.

So I have left you with a whole list of important hot tips around adherence. And as you can see and Dr. Cochrane talked about some of them that were required, even including pill holidays in order to keep women committed and in the trials.

And in the end, and here is a paper by Lewis, et al from 1998, in the end it was not so much ethnic or minority differences that showed discrepancies in lag times in terms of recruitment. It was more about educational status or language status. So keep that in mind, that the lower the socioeconomic status and the level of education, the more difficult it is to recruit and retain.

But eventually many of these feasibility studies demonstrated that you can be highly successful with significant efforts across minorities.

So I have left you with some important conclusions that many folks before me have provided at various sites and at conferences around key lessons learned around recruitment and retention, including Dr. Cochrane, who’s talked about this before. 

One important thing: I am going to skip just to the last actual recommendation that came out from the Institute of Medicine in 2010, and I was privileged to be an editor of that report, was that it is very important now to report sub-analyses by sex but also now by race and ethnicity. And this is going to be demanded increasingly by journals in order to be published. So we have to take these recommendations very seriously.

And finally, I really do want to give credit where credit is due to the late Dr. Bernadine Healy and to Vivian Pinn, who has now formally retired as the founding director of the Office of Women’s Health, and urge you to get online and take a look at the potential wonderful research opportunities that you will find at the WHI website. Thank you very much.

Molly:Thank you very much, Dr. Reams. We do have just two pending questions. I think we can get those fairly quickly. During the data collection period, were forms and print materials available in any other languages besides Spanish?

Dr. Barbara Cochrane:Unfortunately, no. They were available only in Spanish, but we did go through a very careful process of translation to make sure that the Spanish translation was relevant for both Hispanic Latinas in Miami as well as California and Arizona.

Molly:Thank you for that reply. The next question: do you have any suggestions in recruiting active-duty military women and veterans using websites? Have you any experience recruiting from VA women’s wellness centers?

Dr. Nancy Reame:Let me just say that I have only been involved in one VA study with Sarah Krein, my VA research colleague at the University of Michigan, and my former doctoral student, Patricia Rowan, and that used the national mailing. It was a national survey that went out through the auspices of the VA after very careful review and was linked to electronic medical records. I do not have any experience now working as a research collaborator with any of the clinic sites, although I have to encourage all of you to take part in the WHI conference of monthly phone calls that are going on, because that is exactly the point is to get more VA researchers engaged together with new studies in the VA system. And I’m member of the reproductive health one. There are several SIGs, maybe ten now. There is one on chronic pain. There is one on PTSD. There is a new one, I believe, coming out on lesbian and gay transsexual bi-gender issues. Jillian Shipherd can provide more information about a lot of that, as can I think Dr. Klapp or certainly Linda Lipson. But that is an area where I know there is a push to get more VA research going.

Molly:Thank you very much for that response. We do have a couple people that wrote in saying, thank you for this great presentation. And one woman wrote, thank you for the WHI update; I was a WHI lead nutritionist and our volunteers were truly an amazing group. 

Dr. Nancy Reame:Oh! Terrific.

Molly:I also would like to echo the audience’s thanks for you two sharing your expertise with the field. And I did just get a message from Dr. Ruth Klapp saying that people are welcome to contact her regarding information about the work group. So that’s ruth.klapp@va.gov. And with that, I would like to thank our speakers and our audience for joining us today. Do either of you have any concluding comments you would like to make?

Dr. Nancy Reame:Just thank you. It was wonderful.

Dr. Barbara Cochrane:Yes, it was a great experience. I wish there was a little bit more opportunity for a longer-term interaction. Anyone can feel free to get in touch with me by email if you have any questions or want to share other ideas. I am at barbc@uw.edu.

Molly:Excellent. Thank you very much. All right. Well, that does conclude today’s HSR&D Cyberseminar. And everyone, enjoy the rest of your day.

Page 1 of 15

Page 15 of 15