Physical Activity Interventions in Latin America Expanding and Classifying the Evidence

ing Studies to Evaluate the Evidence and Calculate Net Effects The abstraction phase followed the same procedure as those in the U.S. Community Guide and had been applied previously. Two trained bilingual researchers independently abstracted each study and came to consensus on the suitability of the design and quality of intervention execution—the two measures by which the studies were evaluated. Design suitability was coded as greatest, moderate, and least using established criteria. Based on the number of limitations, quality of execution was categorized as good (0 or 1 limitations); fair (2–4); or limited (! 5). Limitations were counted in the following nine categories, as described elsewhere: (1) description of the study population and intervention; (2) sampling; (3) measurement of exposure; (4) measurement of outcome and independent variables; (5) confounding bias; (6) data analysis; (7) participation; (8) comparability and bias; and (9) other biases. If an intervention had the potential to achieve strong or suffıcient evidence of effectiveness using the Community Guide criteria, net effects were calculated as the relative, net percent change from baseline for participants in both the intervention and control groups. The formula for calculating the net intervention effect varied depending on the study design (with or without a control group and/or pre-intervention measurement of outcome). Net intervention effects were calculated for all reported measurements of aerobic capacity and physical activity behavior, for all time periods relative to baseline, and for all subgroups (e.g., gender, country of study). Suffıcient and large intervention effects are based on the quality of the outcomes, magnitude of effect sizes, and population reach of the intervention. Results from the assessments of design suitability, quality of execution, and net intervention effects were combined to determine whether an intervention had strong, suffıcient, or insuffıcient evidence of effectiveness for recommendation. Assessing External Validity Along with evaluating intervention effectiveness (internal validity), the systematic review included an assessment of external validity elements among a subset of reviewed articles. Articles were selected if they evaluated interventions that (1) possessed suffıcient evidence for recommendation (i.e., school-based physical education); (2) were candidates for recommendation (i.e., multicomponent instructional programs, based on number and quality of studies included in the review); or (3) included specifıc programs in Latin America, namely Academia da Cidade, Ciclovia, Curitibativa, and Agita São Paulo, with extensive practice-based evidence (i.e., evidence of institutionalization or spread to other communities) although with limited researchbased evidence (i.e., limited number or quality of peer-reviewed articles) based on knowledge of the research team. The assessment included all articles of these specifıc intervention strategies reviewed to date. An external validity assessment tool (EVAT) was developed based on previous work by others. The EVAT adapted the RE-AIM model and was organized into fıve sections: (1) reach; (2) adoption; (3) implementation; (4) outcomes for decision making; and (5) maintenance and institutionalization (www.projectguia. org/en/research_publications.html). Although the items were the same for programs and environmental and policy interventions, the protocol defınitions of items were tailored. Examples of differences in defınitions between the external validity factors are presented in Appendix B (available online at www.ajpmonline.org). With the exception of one item, all asked whether a specifıc element of external validity was reported, with response choices yes, no, or not applicable. Not applicable was assigned to studies inwhich the interventionwas eithermandatory, and thus no selectionwas involved (e.g., students in school physical education program for target population participation rate); only targeted one setting (e.g., citywide program for setting participation rate); or was delivered as part of standard care or curriculum (e.g., existing physical education teachers as delivery agent items). Five trained abstractors completed an electronic form, with two abstractors assigned to independently assess each study and with subsequent discussion about any discrepancies. Interrater agreement was 96%–100% among a sample of seven articles. Descriptive analyses included the frequency of studies reporting each element, as well as the average number of elements reported per study. Hoehner et al / Am J Prev Med 2013;44(3):e31–e40 e33