Promoting Regular Mammography Screening I. A Systematic Assessment of Validity in a Randomized Trial

Most health promotion trials of interventions to change behavior evaluate efficacy rather than effectiveness (1). Efficacy trials are typically limited in scope, focusing on internal validity under optimal conditions (2). Internal validity refers to the comparability of the intervention and control groups and the extent to which their distributions on important confounding variables are equal or balanced to support a causal interpretation of any postintervention between-group differences on the study outcome of interest (2). In contrast to efficacy trials, effectiveness trials reach more broadly to a specific population, seeking external validity under real-world circumstances (3). External validity refers to the extent to which study findings can be generalized or extrapolated to the target population of interest (2). As emphasized in a recent report by the Agency for Healthcare Research and Quality (4), efficacy and effectiveness trials exist along a continuum defined by the target population and study setting, the strictness of eligibility and exclusion criteria, the health outcome measures of primary interest, the intervention regimen and mode of delivery, the adequacy of the sample size and duration of follow-up to detect minimally important differences, and the use of intention-to-treat analyses to increase the generalizability of intervention effect estimates. The development of behavior change interventions that can be broadly disseminated is a high priority of the National Institutes of Health (5). For interventions to achieve their maximum public health impact, results from intervention trials must be generalizable (6–9). A 2004 supplement of the journal Cancer, jointly sponsored by the National Cancer Institute, American Cancer Society, and Centers for Disease Control and Prevention, urged researchers involved in cancer screening interventions to pay more attention to external validity (such as participant representativeness) in the design of efficacy trials (10). The supplement further emphasized that “The sooner researchers and funding agencies understand the constraints on intervention delivery in the real world, the more likely programs are to be adopted” (10). Efficacy trials typically exclude from their analyses nonrespondents, those who refuse to participate, those who withdraw from the study, and participants with any missing data on the outcome variable of interest (11). In addition, intervention trials are seldom population based and are often restricted to a specific geographic location, health-care system or provider, and demographic profile of study participants (4). Thus, the internal and external validity of study findings and the potential for broad dissemination of most health promotion interventions remain uncertain. This uncertainty further challenges the validity of the cost-effectiveness analyses that form the basis of public and private health-care policy (12). We sought to overcome limitations in internal and external validity by targeting the population of US women veterans in a patient-directed, nationwide intervention trial to promote regular mammography screening, Project HOME (Healthy Outlook on the Mammography Experience) (13). Guided by the CONSORT statement (14), the Code of Professional Ethics and Practices of the American Association for Public Opinion Research (15, 16), the Evidence-Based Behavioral Medicine Committee (17), the Agency for Healthcare Research and Quality’s criteria for distinguishing effectiveness from efficacy trials (4), and criteria specifically recommended to improve external validity in health promotion research by Glasgow et al. (11), we systematically assessed Project HOME’s internal and external validity. Our approach included the following elements: 1) flow diagrams showing the passage of study candidates through each stage of the trial; 2) internal validity assessments comparing randomized study groups on correlates of mammography screening measured at three time points: the date of randomization, baseline survey completion, and the trial’s end; and 3) three external validity assessments—the first evaluating possible “cueing” effects of baseline and interim surveys on mammography screening outcomes in our study population; the second comparing mammography screening trends over the 3.25-year study period in the study population, independent of intervention effects, with the corresponding trends in the US female population; and the third comparing characteristics of nonparticipants with those of participants remaining at the end of the trial. This approach can be adapted to future intervention trials in cancer prevention and control, regardless of whether they are efficacy trials of relatively limited scope or large-scale, population-based effectiveness trials.