Evaluation of the Coach Across 

America Sports-Based Youth 

Development Program 

FINAL REPORT 

Amy Windham, Ryan Basen, Quy Nhi Nguyen, Denise 

Mitchell 

OCTOBER 2014

Evaluation of the Coach Across America 

Sports-Based Youth Development 

Program 

FINAL REPORT 

October 2014

Amy Windham, Ryan Basen, Quy Nhi Nguyen, Denise Mitchell 

6003 Executive Blvd., Suite 300 

Rockville, MD 20852 

301.592.8600

www.air.org 

Copyright © 2014 American Institutes for Research. All rights reserved. 

10/10/14
American Institutes for Research Evaluation of Coach Across America —i 

Contents 

Page 

Executive Summary 1 

Background and Significance 2

Evaluation Aims...........................................................................................................................2

Up2Us and Coach Across America .............................................................................................3

Evaluation Methods 5

Data Collection and Measures .....................................................................................................5

Sample and Follow-up .................................................................................................................8

Data Analysis .............................................................................................................................13

Results 14

Overall........................................................................................................................................14

High intensity versus low intensity programming .....................................................................16

Conclusions 20

Physical Activity ........................................................................................................................20

Nutrition .....................................................................................................................................20

High Impact Attributes ..............................................................................................................21

References 23

Appendices 1

Appendix A. PAQ Scoring Instructions.......................................................................................1

Appendix B. HIA Scoring Instructions ........................................................................................4

Appendix C. Program outcomes by youth attendance .................................................................7

Appendix D: Number of Sessions Offered and Attended by CAA Program ..............................9

Appendix E: Survey Response Rates by CAA Program ...........................................................12

Appendix F: Data Cleaning Steps ..............................................................................................15

Appendix G. Program Outcomes by Grade Level .....................................................................17

Appendix H. Outcomes by Sex ..................................................................................................19

Appendix I. Outcomes by Program Intensity ............................................................................21
American Institutes for Research Evaluation of Coach Across America —1 

Executive Summary 

American Institutes for Research (AIR) conducted external data analysis to evaluate the impact 

of the Coach Across America (CAA) program on underserved youth nationwide during the 

2013-2014 school year. Specifically, AIR examined changes in youth outcomes concerning their 

physical activity levels, nutrition habits, and manifestation of attributes shown to contribute to 

sports and life skills (High Impact Attributes). 

CAA is the flagship program run by Up2Us, a national coalition of 1,000 youth sports 

organizations committed to using sports for social change. Up2Us recruits, trains, and places 

CAA coaches with its affiliates, preparing the coaches to work with underserved youth who are 

the primary participants in the affiliates’ programs. 

Up2Us collected survey data from more than 2,000 youth in 19 cities both when they began and 

completed their participation in an affiliate’s program that school year. Up2Us shared the data 

with AIR, which then conducted the data analysis to measure youth outcome changes over this 

time span – both overall changes and changes based on dosage (as defined by how many sessions 

affiliate programs offered youth). 

Overall findings revealed that CAA was correlated with significant improved physical activity 

levels as well as reduced consumption of soda and fried potatoes. Youth reported a decline in 

100 percent fruit juice consumption, however. Regarding High Impact Attributes (HIA), youth 

demonstrated an increase in manifesting the attributes labeled Prosocial connections and Plan B 

thinking. They also reported improvements in their answers to three decision-making questions 

designed in part to measure their self-esteem and independent thinking. 

Findings by dosage indicated significant, linear correlations between how many events the 

affiliate programs offered and changes in both youth physical activity levels and consumption of 

fried potatoes. Other findings by dosage were not conclusive. 

Many of this study’s results are reflected by previous research into the influence of coaching on 

youth development, and the impact of organized sports on youth nutrition habits and physical 

activity levels. 

 
American Institutes for Research Evaluation of Coach Across America —2 

Background and Significance 

We need to know more about how coaches can encourage youth to build life skills and make 

better decisions concerning their physical health, particularly youth in underserved communities. 

Research shows that coaches have the ability to affect youth’s decision-making. A young 

athlete’s personal interactions with coaches are an important influence on the athlete’s positive 

development. Further, coaches trained in research-based, coaching-education programs and 

positive youth development can minimize negative experiences reported by youth sports 

participants. High-quality informal coach training alone can yield an increase in young athletes’ 

personal skills. The addition of formal training, including classroom-based and practical training, 

substantially enhances coaching effectiveness. Many, if not the majority of, youth coaches lack 

specific training in the science of coaching, and coach training itself is an emerging science; as 

such, many coaching interventions lack solid theoretical bases (Beatty & Fawver, 2013; 

Bornstein, 2011; Vella, et. al., 2013a). 

Given these research findings and challenges, offering established coaching guidelines to 

coaches can empower them to provide critical aid to disadvantaged youth. Coach Across 

America (CAA), a sports-based youth development (SBYD) program run by the coalition 

Up2Us, assesses, places, trains, and supports adult role models to handle coaching duties. CAA 

also demands that host organizations commit to a rigorous process for preparing these coaches 

(Up2Us, http://up2us.org). 

American Institutes for Research (AIR) conducted an external evaluation of CAA to assess how 

CAA impacts the thousands of young athletes it serves. The purpose of this evaluation was to 

expand the understanding of what ways and to what extent the coaching and role modeling of 

CAA-trained coaches correlate with improved outcomes for youth—and to inform CAA and 

other SBYD programs nationally . 

Evaluation Aims 

CAA aims to reduce youth violence; promote physical, mental, and social health; and improve 

academic achievement for youth in underserved communities. Focusing on health outcomes, the 

evaluators specifically examined CAA’s ability to influence the physical activity, healthy 

decision-making, and nutrition habits of youth participants. We addressed the following research 

questions: 

1. To what extent does working with a CAA coach at a host site increase the amount of 

physical activity that the youth participants are engaging in? 

2. To what extent does working with a CAA coach at a host site improve the nutrition habits 

of youth participants? 

3. To what extent does working with a CAA coach at a host site increase the development 

of attributes that contribute to healthy decision-making? 

In addition to assessing these changes in outcomes, we tested whether program effects were 

larger in programs offering a greater intensity of activity as measured by number of sessions or 

practices offered. These analyses addressed the research question: 
American Institutes for Research Evaluation of Coach Across America —3 

4. Do youth participating in programs that meet more frequently demonstrate larger effects 

on the physical activity, nutrition, and personal attribute outcomes compared to youth 

participating in lower-intensity programs? 

This report describes Up2Us and the CAA program, followed by the methodology and measures 

used in the evaluation, results of the data analyses, and conclusions about CAA’s impact on 

youth participants contextualized within research detailing impacts of coaching in SBYD 

programs. Appendices include the data collection instruments, scoring procedures, CAA site 

level descriptive data, and summary tables of results. 

Up2Us and Coach Across America 

Up2Us is a national coalition of more than 1,000 organizations committed to using sports for a 

social change in addition to its athletic mission via sports-based youth development (SBYD) 

programs. Striving to harness the power of sports to reduce youth violence and promote health 

and academic success, Up2Us organizes nationwide community training programs and maintains 

a research center in Boston. Headquartered in New York, Up2Us seeks to empower sports 

programs to become a united force for meaningful social change. 

SBYD is the core of Up2Us’ mission. SBYD activities feature organized sports geared towards 

positive youth development for disadvantaged youth. The SBYD environment involves a coach 

serving as a mentor, who often hails from the same community as participants; and a primary 

focus on youth development outcomes including life skills, nutrition, violence prevention and 

academic achievement. Up2US champions SBYD because participating in sports programs has 

been proven to increase youth physical activity levels (Mandic, et. al., 2012; Nelson, et. al., 

2011), and SBYD programs have been credited with helping youth reduce obesity and 

participation in violence; and increase knowledge of nutrition, physical activity, and academic 

achievement (Rosewater, 2010; Bohnert and Ward, 2013; Berlin, et.al, 2007; Le Menestrel & 

Perkins, 2007; University of Chicago, 2012; Gould, et. al., 2012). 

Up2Us’ cornerstone SBYD program is Coach Across America (CAA), which develops a 

workforce of coach-mentors by training and placing adult role models within sports programs, to 

foster SBYD in underserved communities. Launched in 2010, CAA served 50,000 youth in 33 

states during the 2013-2014 program year. CAA’s desired outcomes include increasing youth 

levels of physical activity and improving their ability to make health-seeking decisions (Program 

Model); its overarching goal is “to develop a dedicated corps of SBYD coaches who inspire 

youth in marginalized urban communities to live healthier, safer and more prosperous lives.”

Organizers developed CAA to address alarming and interrelated problems. Childhood obesity 

has become a major national issue, particularly for disadvantaged youth. Obesity prevalence for 

children from low-income, low-education households increased more than twice as quickly as 

obesity prevalence for all American children ages 10-17 over a four-year span, according to a 

study by Singh, et. al. (2010). CDC has identified unhealthy eating habits and physical inactivity 

as major sources of this obesity problem (CDCa), classifying them as priority health risk 

behaviors “that contribute markedly to the leading causes of death, disability and social 

problems” on par with alcohol, tobacco, and drug use (CDCb). Schools in low-income 
American Institutes for Research Evaluation of Coach Across America —4 

communities have simultaneously deemphasized physical activity by cutting and eliminating free 

physical education and sports programs, or offering “pay-to-play” models because of fiscal 

problems (Rausch, 2006). The consequence is clear: “With even fewer opportunities to 

participate in quality, structured sports programs,” Up2Us notes, “all children – and particularly 

those from low-income families – become even more vulnerable to unhealthy lifestyles and risky 

behavior.” Up2Us therefore steers CAA to youth in low-income communities with aboveaverage childhood obesity rates. 

Prospective CAA coach-mentors are required to undergo at least 53 hours of training orienting 

them to CAA and their individual program, as well as youth development and mentoring 

strategies—including SBYD foundations, the brain and trauma, behavior management and 

culture—to help them cultivate caring, trust-based relationships with youth, and to foster SBYD 

outcomes. By participating in the requisite CAA training, coaches should be able to manifest to 

youth the links between sports skills and life skills, devise strategies for managing difficult youth 

behavior, and develop positive cultures on their teams and at their host organizations (among 

other coach certification program outcomes). Each coach serves about 100 youth per year and is 

encouraged to recruit five community volunteers to assist with CAA services at the host site. 

Some of these coaches work full-time (1,700 hours annually), while part-time coaches work 

between 300-900 annual hours depending on their status. They typically serve one of these roles: 

Coach, program manager, referee, camp leader, nutrition educator, recess leader. 

Up2Us-sanctioned host sites must submit program plans encompassing SBYD elements and 

meet additional criteria including: serving as a liaison between CAA staff and coaches, 

completing CAA evaluation and reporting requirements, purchasing General Liability Insurance, 

and supporting coaches’ professional development as well as program costs. Up2Us applies best 

practices derived over four years from an AmeriCorps grant to select host sites. 

Up2Us has engaged hundreds of AmeriCorps members as CAA coach-mentors since securing a 

Corporation for National and Community Service (CNCS) National Direct Grant in 2009, and 

these coach-mentors have practiced SBYD programming focused on health and wellness 

outcomes addressing CNCS National Performance measures H5 (physical activity) and H6 

(nutrition) with thousands of youth. Up2Us plans to continue working with AmeriCorps to 

operate CAA, addressing the CNCS Healthy Futures priority by helping underserved youth: 

increase physical activity, improve physical fitness, and develop skills leading to healthy 

decision-making. 
American Institutes for Research Evaluation of Coach Across America —5 

Evaluation Methods 

This evaluation plan was designed to examine change in outcomes as a result of taking part in 

CAA-affiliated programs. Randomized designs provide the strongest evidence of program 

effectiveness. Random assignment to a treatment versus control condition was not feasible for 

this project, however. Therefore, we employed a quasi-experimental pre-post design to compare 

outcomes for youth before and after participation in the CAA program. Secondly, because there 

was no opportunity to select a comparison group (either randomized or based on a convenience 

sample), we conducted analyses testing whether high intensity programming was more effective 

than low intensity programming. 

Data Collection and Measures 

Up2Us collected data on 6,288 youth participants spanning 80 CAA programs in 19 cities during 

the 2013-2014 school year. They employed 400 coaches, including 187 AmeriCorps coaches. 

For the evaluation, the organizations collected baseline and follow-up (endline) surveys 

assessing the outcomes of interest. In addition to the survey data, Up2Us recorded participant 

demographic data and program data. The participant data included grade, ethnicity, and sex. 

Program data included the youth’s coach, sport, location of the program, number of sessions 

offered, and number of sessions attended by each youth. 

Physical Activity 

Physical activity was measured by the Physical Activity Questionnaire (PAQ), including the 

PAQ-C (tailored to grades 4 through 8) and PAQ-A (grades 9 through 12). The PAQ was 

designed to measure moderate to vigorous physical activity levels for youth from grades 4 

through 12 during the school year. Researchers developed it “in response to the need for a valid 

and feasible self-report measure for large-scale (physical activity) research with children and 

adolescents” (Kowalski, et al., 2004). 

A youth’s PAQ score is tallied by first finding an activity score between 1 and 5 for each survey 

item (excluding the final item, which asks if the youth was sick or if anything prevented the 

youth from engaging in normal physical activities over the previous week). Item one asks 

whether a youth participated in specific activities during their spare time as well as the amount of 

times the youth participated in that activity; the mean of all activities in this item is calculated to 

find an individual activity score. Item eight is similarly structured; it asks how often the youth 

engaged in physical activity for each day of the last week, and the mean of these responses is 

calculated to find an individual activity score. After creating these two summary scores, the 

mean across all items (items one, eight, and two through seven) is calculated to find an overall 

activity score. (The final question does not factor into the final score, but is used instead to 

identify youth who engaged in unusual activity). 

Up2Us adopted most questions from the original PAQ items, but modified them by removing a 

PAQ question inquiring about which sports youth had participated in and how often youth had 

participated in these sports over the previous week. 
American Institutes for Research Evaluation of Coach Across America —6 

The original items in the questionnaire included questions soliciting physical activity done in the 

participant’s spare time, physical activity done in different settings over the previous seven days 

(e.g. during physical education classes/recess, at lunch, right after school, evenings, etc.), a 

question seeking the level of physical effort exerted during a participant’s free time over the 

previous seven days, how often the participant was physically activity for each day last week, 

and whether or not the participant was sick in the last week or was prevented from participating 

in normal physical activity. 

We followed the developer’s instructions for scoring the instrument (see Appendix A); however, 

since Up2Us modified the original PAQ and removed the first item inquiring about youth’s 

physical activity in their spare time, this item was not calculated into the overall activity score. 

The score is reported on a continuous scale ranging from 1 to 5, with high scores indicating 

higher levels of physical activity. 

Nutrition 

Questions about nutritional intake came from the Physical Health & Nutrition Module of the 

California Healthy Kids Survey (California Dept. of Education, 2013). This measure asks how 

many times the respondents consumed the following items over the previous 24 hours: 

x Milk or yogurt 

x Soda pop 

x 100 percent fruit juices, such as orange, apple, or grape 

x French fries, potato chips, or other fried potatoes 

x Fruit 

x Vegetables 

Response options included: 0 times (scored 1), 1 time (scored 2), 2 times (scored 3), 3 times 

(scored 4), and 4 or more times (scored 5). Each nutrition item was scored as an individual 

outcome and reflected the mean frequency ranging from 1 to 5. 

Two additional questions about nutritional choices included: 

When I have to make a decision about what to eat or drink...

x I know the difference between what is healthy for me and what is not healthy 

x I usually choose the healthier option 

These items were rated on a 1 to 5 scale from strongly agree to strongly disagree. They were 

coded so that a higher score indicated a better nutritional choice. 

High Impact Attributes 

The High Impact Attributes (HIA) survey questionnaire was developed to measure the extent to 

which participants have a set of attributes shown to help them in sports as well as in life. The 

eight attributes include: 

1. Positive identity 
American Institutes for Research Evaluation of Coach Across America —7 

2. Situational awareness 

3. Plan B thinking 

4. Future focus 

5. Discipline 

6. Social confidence 

7. Prosocial connections 

8. Self-awareness 

Each attribute was measured using two survey items. Per the scoring instructions, the two items 

were averaged to produce a score ranging from 1 to 5 for each attribute with higher scores 

indicating a higher amount of the attribute. 

Additionally, the instrument included five questions measuring general decision-making ability.

The items included the following: 

1. I can resist peer pressure. 

2. When I have to make a hard decision in life, I make the decision before thinking about all 

of the options. 

3. When I have to make a hard decision in life, I talk to someone I trust about the decision. 

4. When I have to make a hard decision in life, I feel helpless. 

5. When I have to make a hard decision in life, I make the decision based on what will make 

other people like me. 

Each item was rated on a 1 to 5 scale from strongly agree to strongly disagree. We treated each 

decision-making item as a separate outcome scored based on the mean agreement rating with 

high scores indicating better decision-making skill. The HIA measure, including the decisionmaking questions and scoring instructions can be found in Appendix B. 

Program Dosage 

We used program data on the number of sessions provided by the programs and the number of 

sessions attended by youth to differentiate high- versus low-intensity1

 programming, and high 

versus low attendance. Both are indicators of dose. Intensity (number of sessions offered) 

reflects the intended or planned program intensity. Attendance reflects the actual achieved dose. 

Number of sessions offered reflects an intent-to-treat (ITT) approach because it estimates the 

effects on everyone the program “intended to treat”. Whereas, the total number of classes 

attended is an “effects on the treated” approach. ITT is more rigorous and scientifically justified. 

However, the effect on the treated often is of greater interest to programs. 

For the primary dosage analysis, we used the intensity variable and those results are reported in 

the body of the report. Another practical limitation of using the attendance variable was that 

 

1

 We explored using dates of baseline and endline survey administration to construct a variable indicating the length 

of time of the program. However, there was too much missing data to create a program length variable. 
American Institutes for Research Evaluation of Coach Across America —8 

there was substantial missing data. However, because of the potential interest by program 

administrators, we report the attendance results in Appendix C. 

We examined the frequency distributions of the intensity and attendance variables to determine 

how to best categorize intensity and attendance. Intensity ranged from 1 to 128 sessions with a 

mean of 39 and a median of 34. We defined low intensity as 33 or fewer sessions and high 

intensity as 34 or greater sessions. Attendance ranged from 0 to 113 sessions with a mean of 26 

and a median of 21. We defined low attendance as attending 21 or fewer sessions and high 

attendance as 22 or more sessions. Intensity and attendance data by CAA program are reported in 

Appendix D. 

Sample and Follow-up

Up2Us received evaluation data from 80 youth sports programs, collecting data from between 11 

and 579 youth participants per program. The majority of programs provided data on fewer than 

100 youth; the median was 44. 

Completed baseline survey data were collected from 4,771 youth and endline survey data were 

collected from 2,299 youth. Of the 80 programs, 57 provided both baseline and endline surveys 

from some portion of their participants. While the remaining 23 of these programs failed to 

provide endline data, another program did not provide either baseline or endline data, and two 

others failed to provide baseline data but did submit endline data. Survey response rates for each 

program are reported in Appendix E. 

Given the availability of baseline and/or endline survey data, we provide descriptive statistics for 

all participants (Table 1). Up2Us provided at least some demographic data on 5,797 (92.2 

percent) participants (Table 1). Participants spanned all grade levels from Kindergarten through 

12th grade, youth from 11 different racial or ethnic backgrounds, and a mix of new and veteran 

participants. Programs were instructed to exclude youth younger than age eight, however, there 

are a small number youth in this age range represented. Of the sample where data were provided: 

x The sample was nearly evenly split among boys (50.6 percent) and girls (49.4 percent). 

x The majority were enrolled in grades 3-8 (76.7%). Broken down by school level, nearly 

60% were enrolled in middle or high school, with the remaining youth in elementary 

school. 

x 78.1% identified themselves as either Hispanic or African-American. 
American Institutes for Research Evaluation of Coach Across America —9 

Table 1: Sample Descriptive Characteristics, n=6,288 

Grade n %

Kindergarten 5 0.1

1st 124 2.0

2nd 244 3.9

3rd 701 11.1

4th 851 13.5

5th 715 11.4

6th 698 11.1

7th 652 10.4

8th 638 10.1 

9th 289 4.6 

10th 238 3.8 

11th 209 3.3 

12th 185 2.9 

Not disclosed 739 11.8 

Race/ethnicity n % 

Caucasian 421 6.7 

African American 1577 25.1 

Native American 24 0.4

Asian American 111 1.8

Hispanic 1850 29.4

Other 64 1.0

Asian 170 2.7

African 63 1.0

Multi-ethnic 76 1.2

Pacific Islander 3 0.0

Middle Eastern 27 0.4

Not disclosed 1902 30.2

Sex n %

Male 2936 46.7

Female 2861 45.5

Not disclosed 491 7.8

Length of enrollment n %

New participant 1091 17.4

1 year or less 1088 17.3

1-2 years 926 14.7

2-4 years 743 11.8

5 or more years 158 2.5

Not disclosed 2282 36.3

Location n %

New York 536 8.5

Chicago 942 15

Los Angeles 551 8.8

New Orleans 695 11.1

Miami 502 8

Boston 655 10.4

DC/Baltimore 306 4.9

Denver/Mountain Region 808 12.8

Bay Area CA 248 3.9

Flint/Mil./Cleve./Det. 234 3.7

Philadelphia 334 5.3

New Mexico 11 0.2

Dallas 92 1.5

Atlanta 113 1.8

Seattle 261 4.2
American Institutes for Research Evaluation of Coach Across America —10

For the outcome analyses, we excluded programs that did not provide baseline or endline data 

(23 programs), defining the analysis sample as programs providing both baseline and endline 

data (57 programs). Of the 57 programs with baseline and endline data, response rates across 

programs varied from 4.8 to 100 percent. Overall, 39 of the 57 programs secured fewer than 50 

percent on both data points. 

We conducted a response bias analysis by comparing completers versus non-completers on 

baseline demographic and outcome variables (Tables 2 and 3). Collection of the surveys was 

more dependent on the level of effort put forth by the programs, as opposed to the individual 

youth; nonetheless it is important to understand who was represented in the followed sample. In 

addition, non-response is a combined function of attrition from the program and failure to 

complete the endline survey; however, it is not possible to know whether this was the case (i.e.: 

was the lack of endline data the result of attrition and therefore also of non-participation in the 

survey, or simply non-participation/non-completion of the survey?). 

Table 2 presents the results of the response bias comparisons based on youth demographic 

characteristics. High school students were significantly less represented in the endline surveys 

than elementary and middle school students (p=.001). Caucasian and Hispanic youth were 

significantly more likely to complete surveys compared to other racial and ethnic groups 

(p<.001). 

There also were differences in completers versus non-completers in some, but not all, of the 

outcome variables measured at baseline (Table 3). Non-completers had higher physical activity 

levels (p<.001). Their higher levels likely relate to the higher inclusion of elementary school 

students, who had higher baseline rates of physical activity than the middle and high school 

youth. This circumstance is preferred to having higher non-response by youth with lower activity 

levels because it is more important to have lower activity youth represented in the outcome 

analyses. Youth with lower levels on the Discipline attribute were significantly less likely to 

complete the endline survey (p<.001). The discipline items include, “I have a hard time waiting 

when I want something” and “If I’m feeling mad, it is hard to control what I say or do.” It is 

unclear if this distinction reflects impulsivity and true differences in proclivity to complete either 

the survey or to complete the sports program, but it is worth monitoring both program 

participation and survey completion to see if this attribute is related to participation. 
American Institutes for Research Evaluation of Coach Across America —11

Table 2: Demographic comparisons between youth with endline data and youth without 

endline data, analysis sample 

Completed Did not complete 

 endline endline p value 

n=2299 n=2472 

N % n % 

Grade 0.001 

Elementary (K-5th) 975 42.4% 1080 43.7% 

-8th Middle (6 ) th 768 33.4% 906 36.7% 

-12th High school (9 ) th 556 24.2% 486 19.7% 

Sex 0.085 

Male 1048 45.6% 1254 50.7% 

Female 1009 43.9% 1088 44.0% 

Not Disclosed 242 10.5% 130 5.3% 

Race/ethnicity <.001 

Caucasian 233 10.1% 125 5.1% 

African American 427 18.6% 682 27.6% 

Native North American 12 .5% 10 .4% 

Asian American 31 1.3% 45 1.8% 

Hispanic 799 34.8% 689 27.9% 

Other 32 1.4% 26 1.1% 

Asian 28 1.2% 54 2.2% 

African 24 1.0% 29 1.2% 

Multi-ethnic 37 1.6% 25 1.0% 

Pacific Islander 0 0.0% 3 .1% 

Middle Eastern 7 .3% 19 .8% 

Not Disclosed 669 29.1% 765 30.9% 
American Institutes for Research Evaluation of Coach Across America —12

Table 3: Comparisons on the outcomes, between youth with endline data and youth 

without endline data. 

Completed

endline

Did not complete 

endline p value

n mean n mean

Physical Activity - PAQ Score 1702 3.11 1633 3.21 <0.001

Nutrition

In the past 24 hours, how many times did you...

drink white milk? 309 2.85 133 2.81 0.928

drink soda? 309 2.18 133 2.24 0.703

drink 100% fruit juice? 309 2.85 133 2.92 0.645

eat French fries, potato chips, other fried 

potatoes? 306 2.38 132 2.42 0.767

eat fruit? 289 3.18 116 3.23 0.756

eat vegetables? 293 2.75 116 2.75 0.987

High Impact Attributes

Positive Identity 1617 4.36 1577 4.34 0.442

Situational Awareness 1613 3.37 1578 3.34 0.341

Plan B Thinking 1639 3.69 1589 3.69 0.911

Future Focus 1636 4.13 1583 4.08 0.108

Discipline 1635 2.90 1588 2.80 <0.001

Social Confidence 1636 3.54 1583 3.50 0.238

Prosocial connections 1637 3.65 1579 3.59 0.057

Self Awareness 1636 3.71 1578 3.67 0.201

Decision-making

I can resist peer pressure. 724 3.66 428 3.65 0.890

When I have to make a hard decision in life...

I make the decision before 

of the options.

thinking about all 1593 2.95 1528 2.88 0.159

I talk with someone I trust about the 

decision. 1587 3.86 1535 3.81 0.315

I feel helpless. 1590 3.21 1526 3.16 0.251

I make the decision based on what will 

other people like me.

make 1591 3.20 1530 3.13 0.127
American Institutes for Research Evaluation of Coach Across America —13

Data Analysis

The data were provided by Up2Us to AIR in a Microsoft Excel spreadsheet and prepared for 

analysis. AIR merged the demographic and program data with the survey data, conducted quality 

checks to address errors in data entry and missing data prior to analysis, and ran preliminary 

descriptive analyses to determine how to best define and code the analysis variables. Detailed 

data cleaning and preparation steps are provided in Appendix F. 

We conducted a two-stage analysis to: 

1) Estimate the pre-post change in outcomes across the study population, and 

2) Estimate the difference in pre-post change for youth participating in a low intensity 

versus high intensity program. 

For the Stage One analysis, we used an observational one-group, pretest–posttest design to 

examine changes in outcomes between baseline and completion of the program. Specifically, we 

constructed a mixed-effect regression model to estimate change in outcomes from pretest to 

posttest. Using Maximum Likelihood estimation, we obtained regression coefficients adjusted 

for the missing values based on the assumption that the data were missing at random. The 

parameter estimates from the Maximum Likelihood estimation are similar to those obtained 

using multiple imputations. The former, however, is more efficient because adjusted regression 

coefficients and standard errors can be obtained without generating multiple complete data sets 

with imputed post-test scores. The models will account for potential correlations among 

participants resulting from both repeated measures on the same participant and from potential 

correlations between participants of the same program. 

For the Stage Two analysis, we expanded the Stage One model to examine the change in 

outcomes for youth participating in high versus low intensity programs. In preliminary analyses, 

we looked at each outcome stratified by the two intensity categories (low versus high). Secondly, 

we looked at the interaction between time (pre-to-post) and each intensity category (i.e. 

time*intensity) to test for an interactive effect. The interaction effect determines whether the 

change in the pre-post scores is significantly different for youth participating in high versus low 

programs. 

All the regression models adjusted for the potential effects of sex, race/ethnicity, and grade. In 

the analyses, grade was collapsed into: elementary school (grades K through 5), middle school 

(grades 6 through 8), and high school (grades 9 through 12). 

In addition to comparing the results by program intensity, we conducted additional descriptive 

analyses to assess differences in program effects by variables of secondary interest to Up2Us: 

attendance, grade (elementary school versus middle and high school combined, and sex. Because 

these were of secondary interest, they are reported in tabular form in Appendix C, G, and H. 
American Institutes for Research Evaluation of Coach Across America —14

Results 

Overall 

Physical Activity 

Overall, youth significantly increased their level of physical activity from a mean of 3.11 at 

baseline to 3.21 at endline (p < .001). 

Nutrition 

Changes in the nutrition outcomes from baseline to endline were mixed. There was significant 

improvement on two of the six nutrition frequency items: youth reported drinking less soda 

(p=.03) and eating fewer French fries, potato chips, and other fried potato (p<.001). However, 

they also significantly reduced the amount of 100 percent fruit juice they reported drinking 

(p=.004). There was no overall impact on drinking milk or eating fruit or vegetables. There were 

no differences on the healthy choices questions. 

High Impact Attributes. 

There was a significant effect on two of the eight attributes: Plan B thinking (p=.008) and 

Prosocial connections (p.001). Additionally, there was significant improvement in three of the 

five decision-making items: feeling helpless when having to make a hard decision (p=.003), 

making decisions based on what will make others like me (p=.005), and resisting peer pressure 

(p=.001). 

See Table 4 for a summary of the overall results for each outcome measured. 
American Institutes for Research Evaluation of Coach Across America —15

Table 4. Model estimated baseline and endline means for each outcome 

Predicted Predicted * n Baseline 

mean

Endline 

mean

p value

Physical Activity - PAQ Score 2599 3.11 3.21 <.001 * Nutrition

In the past 24 hours, how many times did you...

drink white milk? 1133 2.80 2.82 .768

drink soda? 1128 2.20 2.04 .033

drink 100% fruit juice? 1133 2.88 2.63 .004

eat French fries, potato chips, other fried 

potatoes? 1125 2.45 2.12 <.001

eat fruit? 1106 3.15 3.24 .304

eat vegetables? 1116 2.82 2.72 .227

When I have to 

or drink...

make a decision about what to eat 

I know the difference between what is healthy 

for me and what is not healthy 2571 3.99 4.06 .238

I usually choose the healthier option 2560 3.47 3.47 .998

High Impact Attributes

Positive Identity 2512 4.35 4.37 .550

Situational Awareness 2514 3.34 3.38 .094

Plan B thinking 2525 3.69 3.75 .008

Future Focus 2522 4.12 4.14 .433

Discipline 2521 2.87 2.92 .052

Social Confidence 2518 3.53 3.56 .209

Prosocial connections 3.63 3.71 .001

Self-Awareness 2517 3.70 3.71 .873

Decision-making

* I can resist peer pressure 1572 3.55 3.71 .001

When I have to make a hard decision in life...

I make the decision before 

the options.

thinking about all of 2488 2.94 2.99 .185

I talk with someone I trust about the decision. 2485 3.84 3.84 .951

I feel helpless. 2486 3.21 3.32 .003

I make the decision based on what will 

other people like me.

make 2485 3.20 3.30 .005

*

The overall sample size for these analyses was 2,599. However, the nutrition questions and the 

decision-making item “I can resist peer pressure” were not included on all survey versions. For the other 

outcomes, the sample size is lower than 2,599 due to youth skipping some survey items. 
American Institutes for Research Evaluation of Coach Across America —16

High Intensity versus Low Intensity Programming 

Next the results for the dosage analyses are presented. We highlight outcomes where there were 

differences in effects for youth in high- versus low-intensity programs. The complete results for 

the intensity comparison are reported in Appendix I. 

Physical Activity 

Youth in both high and low intensity programs achieved significant increases in physical 

activity. Notably, youth in high intensity programs started off at a higher level of physical 

activity and saw a steeper increase. This may reflect selection differences between the groups, 

i.e., more active youth selecting higher intensity programs. However, it also is important to 

monitor data collection to be sure there are not systematic differences in data collection 

procedures, e.g., baseline data collection being delayed until after the start of the program when 

activity levels have increased. 

Figure 1. Mean Physical Activity Questionnaire (PAQ) scores at baseline and endline for 

youth participating in high versus low intensity programs 

4.00

Score 3.50 PAQ 

Low intensity

Mean 

High intensity

3.00

2.50

Baseline Endline
American Institutes for Research Evaluation of Coach Across America —17

Nutrition 

The dosage analysis revealed different patterns of effects for low- versus high-intensity 

programs. The effect on consumption of fried potato products was significant in both groups. 

Reduced soda consumption, however, was observed only in the low intensity group. The high 

intensity group saw a small but statistically significant effect on consumption of vegetables 

(p=.04). Additionally, the decrease in consumption of 100 percent fruit juices was only observed 

in the low intensity group. There was no change in frequency of drinking 100 percent fruit juices 

in the high intensity group. 

Figure 2. Mean scores on milk and juice consumption, for youth participating in high 

versus low intensity programs 

Figure 3. Mean scores on fruit and vegetable consumption, for youth participating in high 

versus low intensity programs

1.50

2.00

2.50

3.00

3.50

Baseline Endline

Mean Frequency Score 

Milk Low intensity

Milk High intensity

Juice Low intensity

Juice High intensity

1.50

2.00

2.50

3.00

3.50

Baseline Endline

Mean Frequency Score 

Fruit Low intensity

Fruit High intensity

Vegetables Low intensity

Vegetables High intensity
American Institutes for Research Evaluation of Coach Across America —18

Figure 4. Mean scores on soda and fried potatoes consumption, for youth participating in 

high versus low intensity programs 

 

1.50

2.00

2.50

3.00

3.50

Baseline Endline

Soda Low intensity

Soda High intensity

Chips, fries Low intensity

Chips, fries High intensity
American Institutes for Research Evaluation of Coach Across America —19

High Impact Attributes 

The dosage analyses also revealed patterns of differences in the HIA outcomes. In the overall 

analyses, we reported significant effects on Plan B thinking, Prosocial connections and three 

decision-making items. However, the dosage analyses showed that youth in high intensity 

programs had significant changes on just Plan B thinking (p=.001) and ability to resist peer 

pressure (p<.001). Low intensity programs, however, had significant effects on Prosocial 

connections (p=.01) and two decision-making items: feeling helpless when having to make a 

hard decision (p<.001) and making decisions based on what will make others like me (p<.001). 

These differences in patterns are illustrated in Figures 5 and 6. 

Figure 5. Mean scores on HIA outcomes, for youth participating in high versus low 

intensity programs 

Figure 6. Mean scores on decision-making outcomes, for youth participating in high versus 

low intensity programs 

3.00

3.50

4.00

Baseline Endline

Mean Score 

Can resist peer pressure High

Intensity

Can resist peer pressure Low

Intensity

Makes decisions based on

others' opinions High

Intensity

Makes decisions based on

others' opinions Low

Intensity

3.00

3.50

4.00

Baseline Endline

Mean Score 

Plan B thinking High intensity

Plan B thinking Low intensity

Prosocial Connections High

intensity

Prosocial Connections Low

intensity
American Institutes for Research Evaluation of Coach Across America —20

Conclusions 

Results revealed that youth who interacted with CAA coaches during the 2013-2014 school year 

increased their physical activity, and decreased their consumption of unhealthy items including 

soda and fried potatoes. CAA’s overall impact on youth nutritional habits was mixed, however, 

because youth also reported a decline in 100 percent fruit juice consumption and insignificant 

results concerning changes in other nutritional habits. 

Interacting with a CAA coach correlated with youth’s enhanced manifestation of the HIAs 

labeled Prosocial connections and Plan B thinking, and the youth also reported improvements in 

their answers to three of the five decision-making questions. Mixed findings concerning changes 

in other HIA outcomes, however, lead us to conclude that more research is needed to determine 

an illuminating, detailed CAA impact on youth development. 

Physical Activity 

The improvements in short-term physical activity (PA) manifested by youth in this study 

correspond with recent studies concluding that sports participation can positively impact youth 

physical activity levels in the short term, including disadvantaged youth within organized sports 

programs such as those that worked with CAA coaches in this study (Vella, et. al., 2013b; 

Mandic, et. al., 2012). 

The larger PAQ increases, reported in particular by middle school youth in this evaluation, and 

insignificant results from the elementary-school cohort also are validated by results and 

conclusions from a recent study. Basterfield, et. al. (in press) found that middle school-aged 

youth increasing PA levels correlated with participating on a sport club. “This provides some 

evidence that sports club participation is the source of increased physical activity,” wrote the 

authors, who examined English youth born in 1999 or 2000 three times between 2006 and 2012.

Elementary school-aged youth who participated on these clubs, by contrast, did not exhibit 

higher levels of PA. The authors thus concluded that sports participation may be more influentia

as youth age: “Sports participation may become an increasingly significant source of physical 

activity over late childhood and early adolescence when physical activity is likely to decline 

dramatically and rates of obesity incidence are highest” (p. 4).

Other recent studies of organized sports have concluded that coaches are especially influential 

because of the extent and variety of means they can assist youth, including enhancing youth’s

physical activity levels up to recommended guidelines (Fenton, et. al., 2014; Gould, et. al., 2012

Camire, et. al., 2011; Blom & Watson, 2010). Considering this evidence, the statistically 

significant PAQ increases revealed in this CAA evaluation—while not isolating for coaching 

impact—suggest that CAA coaches were at least partly responsible for inducing the youth in thi

study to be more physically active on a regular basis. 

l 

; 

s 

Nutrition 

Mixed findings in this evaluation on the impact of sports participation on nutrition and diet are 

consistent with recent research. Nelson, et al. (2011) reviewed 19 studies comparing weight 
American Institutes for Research Evaluation of Coach Across America —21

status, physical activity and diet of sports participants versus nonparticipants; they did not find 

any clear pattern or association between body weight and sports participation—even though 17 

of these studies revealed sports participants are more physically active. “The existing research 

suggests that youth in sports are more likely to consume greater amounts of calories and 

consume some unhealthy foods and beverages,” the authors concluded. “It is unclear whether the 

higher energy expenditure associated with sport compensates adequately for this additional 

energy intake” (p. 369).

Vella, et al. (2013b) failed to find associations between organized sport participation and 

consumption of high-fat foods or sugar-sweetened beverages when they surveyed 12,000 

Australian youth ages 12 and 17. Nelson, et. al., echoed the Vella research team in concluding: 

“Whether youth sport participation predicts future dietary intake is understudied” (p. 368). 

It fits, then, that the results from this CAA study concerning sports participation, nutrition and 

diet are not conclusive. (One potential solution, suggested by Vella, et. al., is to revamp 

quantitative instruments designed to measure these relationships. “The consumption of high-fat 

foods and sugar-sweetened beverages may be so pervasive in the population,” the authors noted, 

“that measures of organized sports participation are unable to differentiate between those who do 

and do not consume these unhealthy options on a regular basis” [p. 119].)

High Impact Attributes 

The statistically significant results revealed by this CAA evaluation concerning Plan B Thinking, 

Prosocial connections and general decision-making reflect research concluding that coaches are 

influential in aiding youth development, particularly for underserved youth. Gould, et. al. (2012) 

surveyed 239 urban youth sports participants from an underserved community; they found that 

the more coaches cultivate caring, mastery-oriented environments, the more likely they are to 

yield positive developmental gains for their young athletes. These results, the authors wrote, 

supported previous research “showing that coaching actions and behaviors have an important 

influence on personal and social development of young people involved in sport in general and 

in underserved populations in particular” (p. 85).

In addition, MacDonald et al. (2010) found that athletes who played for coaches who received 

training through their sport program—which, of course, CAA coaches are expected to do—

reported higher rates of personal and social skills than athletes who played for untrained coaches. 

The researchers split up youth participants ages 9-17 into one group (n=41) coached by the 

trained coaches and a group (n=68) coached by the untrained coaches, surveyed both groups, and 

found that “youth experiences can be influenced by informal coach training...Any training may 

be better than no training in developing positive skills in youth.” The authors concluded that 

“incorporating discussion about positive development into a program could have significant 

effects on a large number of youth” (p. 371). 

More specifically, Blom and Watson (2010) found significant results for the interaction between 

caring coaching behaviors and athlete self-confidence over time. The researchers divided their 

youth sample into three groups, including a group led by coaches who received formal training 

from a professionally sanctioned trainer-coach. These youth reported feelings of improved self 

confidence in comparison to the other two groups. 
American Institutes for Research Evaluation of Coach Across America —22

More recently, however, Langan, et al. (2013) systematically reviewed and evaluated research 

into the effectiveness of coach education interventions more broadly; they concluded that, “due 

to the diversity in athlete outcomes and intervention design, it is difficult to draw firm 

conclusions around the effectiveness of coach education interventions” (p. 37). The researchers 

examined four coaching studies, finding mixed effects on athlete developmental outcomes 

including anxiety, self-esteem, fear of failure, and motivational orientation. 

Despite the findings divulged by MacDonald, et al. (2010), Blom and Watson (2010) and by this 

CAA evaluation, then, researchers and practitioners should consider this recommendation from 

the Langan research team (2013) before drawing any ironclad conclusions about trained coaches’ 

impacts on youth development: “More research is needed to further our understanding of (coach 

education) intervention effectiveness to allow for growth and improvement in coach education” 

(p. 37). 

Meeting this request from the Langan team would also satisfy a challenge from another research 

team. After Flett, et al. (2012) conducted 66 observations of inner-city baseball and softball 

practices and games, across Under-18, Under-14 and Under-12 groups, they ultimately 

concluded: “This observation study has detailed how difficult it is to conduct research in 

underserved competitive sport programs. However, these are the contexts that sport science 

researchers can make a positive contribution to social well-being (sic). It is our challenge as a 

field to improve the lives of youth using sport as a vehicle” (p. 288). 
American Institutes for Research Evaluation of Coach Across America —23

References 

Basterfield, L., Reilly, J.K., Pearce, M.S., Parkinson, K.N., Adamson, A.J., Reilly, J.J., Vella, 

S.A. (in press). Longitudinal associations between sports participation, body composition and 

physical activity from childhood to adolescence. Journal of Science and Medicine in Sport. 

Beatty, G., & Fawver, B. (2013). What is the status of youth coach training in the U.S.? Sport 

Policy & Research Collaborative. Retrieved from http://www.aspeninstitute.org. 

Berlin, R. A., Dworkin, A., Eames, N., Menconi, A., & Perkins, D. F. (2007). Examples of sports 

based youth development programs. New Directions for Youth Development, (115), 85-106. 

Blom, L.C., Watson, J.C. (2010). The Impact of a Coaching Intervention on the Coach-Athlete 

Dyad and Athlete Sport Experience. Athletic Insight, 12(3). 

Bohnert, A.M. & Ward, A.K. (2013). Making a difference: Evaluating the Girls in the Game 

(GIG) after-school program. Journal of Early Adolescence, 33(1), 104-130. 

Bornstein, D. (2011, October 20). The Power of Positive Coaching. New York Times 

Opinionator. Retrieved from www.nytimes.com. 

California Dept. of Education (2013). California Healthy Kids Survey: Version H18. 

http://chks.wested.org. 

Camire, M., Forneris, T., Trudel, P., Bernard, D. (2011). Strategies for helping coaches facilitate 

positive youth development through sport. Journal of Sport Psychology in Action, 2: 92-99. 

Centers for Disease Control and Prevention (n.d., a). The Obesity Epidemic and United States 

Students. Retrieved from http://www.cdc.gov/healthyyouth/yrbs/pdf/us_obesity_combo.pdf. 

Centers for Disease Control and Prevention (n.d., b). Youth Risk Behavior Surveillance System: 

Overview. Retrieved from http://www.cdc.gov/healthyyouth/yrbs/index.htm. 

Fenton, S.A.M., Duda, J.L., Quested, E., Barrett, T. (2014). Coach autonomy support predicts 

autonomous motivation and daily moderate-to-vigorous physical activity and sedentary time in 

youth sport participants. Psychology of Sport and Exercise, 15, 453-463. 

Flett, M.R., Gould, D., Lauer, L. (2012). A study of an underserved youth sports program using 

the Youth Program Quality Assessment. Journal of Applied Sport Psychology, 24: 275-289. 

Gould, D., Flett, R., Lauer, L. (2012). The relationship between psychosocial development and 

the sports climate experienced by underserved youth. Psychology of Sport and Exercise, 13, 80-

87. 
American Institutes for Research Evaluation of Coach Across America —24

Kowalski, K.C., Crocker, P.R.E., Donen, R.M. (2004). The Physical Activity Questionnaire for 

Older Children (PAQ-C) and Adolescents (PAQ-A) Manual. College of Kinesiology, University 

of Saskatchewan. 

Langan, E., Blake, C., Lonsdale, C. (2013). Systematic review of the effectiveness of 

interpersonal coach education interventions on athlete outcomes. Psychology of Sport and 

Exercise, 14, 37-49. 

Le Menestrel, S., & Perkins, D.F. (2007). An overview of how sports, out-of-school-time, and 

youth well-being can and do intersect. New Directions for Youth Development, 115, 13-25. 

MacDonald, D.J., Cote, J., Deakin, J. (2010). The impact of informal coach training on the 

personal development of youth sport athletes, International Journal of Sports Science & 

Coaching, 5(3), 363-372. 

Mandic, S., Bengoechea, E.G., Stevens, E., de la Barra, S.L., Skidmore, P. (2012). Getting kids 

active by participating in sport and doing it more often: focusing on what matters. International 

Journal of Behavioral Nutrition and Physical Activity. 9, 86. 

Nelson, T.F., Stovitz, S., Thomas, M., La Voi, N.M., Bauer, K.W., Neumark-Sztainer, D. (2011). 

Do youth sports prevent pediatric obesity? A systematic review and commentary. Current Sports 

Medicine Reports, 10(6), 360-370. 

Rausch, K. (2006). Pay-to-play: a risky and largely unregulated solution to save high school 

athletic programs from elimination. Suffolk Law Review, 39 (2), 583-611. 

Rosewater, A. (2010). Organized Sports and the Health of Children and Youth. Oakland, CA: 

Team Up for Youth. 

Singh, G. K., Siahpush, M., & Kogan, M. D. (2010). Rising social inequalities in U.S. childhood 

obesity, 2003-2007. Annals of Epidemiology, 20(1), 40-52. 

University of Chicago (2012). BAM-Sports Edition. Crime lab research and policy brief. 

Up2Us. (n.d.). CAA description. Retrieved from http://up2us.org. 

Up2Us. (n.d.). Coach Across America introductory guide: hosting a coach. Retrieved from 

http://up2us.org. 

Vella, S.A., Oades, L.G., Crowe, T.P. (2013a). The relationship between coach leadership, the 

coach–athlete relationship, team success, and the positive developmental experiences of 

adolescent soccer players. Physical Education and Sport Pedagogy, 18(5), 549-561. 

Vella, S.A., Cliff, D.P., Okely, A.D., Scully, M.L., Morley, B.C. (2013b). Associations between 

sports participation, adiposity and obesity-related health behaviors in Australian adolescents. 

International Journal of Behavioral Nutrition and Physical Activity, 10, 113. 
American Institutes for Research Evaluation of Coach Across America —25
American Institutes for Research Evaluation of Coach Across America —Appendix 1 

Appendices 

Appendix A. PAQ Scoring Instructions 

Physical Activity Questionnaire for Adolescents (PAQA) Popular

The Physical Activity Questionnaire for Adolescents is a nine-item, seven day self-report recall 

questionnaire designed and extensively used for surveillance and monitoring. The PAQ-A is a selfadministered. It was developed to assess general levels of physical activity for high schools students in 

grades 9 to 12 and approximately 14 to 19 years of age. 

The PAQ-A can be administered in a classroom setting and provides a summary physical activity score 

derived from eight items (the 9th item does not factor into the overall score), each scored on a 5-point 

scale. Estimated completion time is 20 minutes.

Population 

High School

Administration Method

Self-Report Questionnaire

Number of Questions 9 

Creator(s) of Tool 

Kowalski, K., Crocker, P., & Donen, R. The Physical Activity Questionnaire for Older Children (PAQ-C) and 

Adolescents (PAQ-A) Manual. College of Kinesiology, University of Saskatchewan. 

Kent C. Kowalski, Ph.D. 

College of Kinesiology 

University of Saskatchewan 

Peter R. E. Crocker, Ph.D. 

School of Human Kinetics 

University of British Columbia 

Rachel M. Donen, Bsc. Honours 

College of Kinesiology 

University of Saskatchewan 

Scoring / Benchmarking 

Overall process - Find an activity score between 1 and 5 for each item (excluding item 9) 

1) Item 1 Spare Time Activity 

- Take the mean of all activities ("no" activity being a 1, "7 times or more" being a 5) on the activity checklist to form a 

composite score for item 1. 

2) Item 2 to 7 (PE, lunch, right after school, evening, weekends, describes you best) 
American Institutes for Research Evaluation of Coach Across America —Appendix 2 

- The answers for each item start from the lowest activity response and progress to the highest activity response. 

- Use the reported value that is checked off for each item (the lowest activity response being a 1 and the highest 

activity response being a 5). 

3) Item 8 

- Take the mean of all days of the week ("none" being a 1, "very often" being a 5) to form a composite score for item 

8.

4) Item 9 

- Can be used to identify students who had unusual activity during the previous week, but this question is NOT used 

as part of the summary activity score. 

5) How to calculate the final PAQ-A activity score 

- Once you have a value from 1 to 5 for each of the 8 items (items 1 to 8) used in the Physical Activity composite 

score, you simply take the mean of these 8 items, which results in the final PAQ-A activity summary score. 

- A score of 1 indicates low physical activity, wheareas a score of 5 indicates high physical activity. 

Background / Quality 

Consistently high validity; reliability is considered to be moderate. 

(Richardson D, Cavill N, Ells L, Roberts K (2011) Measuring Diet and Physical Activity in Weight Management 

Interventions: A Briefing Paper. Oxford: National Obesity Observatory.) 

Retrieved from: http://www.performwell.org/index.php/find-surveyassessments/outcomes/health-a-safety/good-healthhabits/physical-activity-questionnaire-for-adolescents 

Physical Activity Questionnaire for Children (PAQC) Popular

The Physical Activity Questionnaire for Children is a self-administered, 7-day recall instrument. It was 

developed to assess general levels of physical activity throughout the elementary school year for students 

in grades 4 to 8 and approximately 8 to 14 years of age. The PAQ-C can be administered in a classroom 

setting and provides a summary phsycial physical activity score derived from nine items, each scored on 

a 5-point scale. Estimated completion time is 20 minutes.

Scoring / Benchmarking 

Overall process - Find an activity score between 1 and 5 for each item (excluding item 10) 

1) Item 1 Spare Time Activity 

- Take the mean of all activities ("no" activity being a 1, "7 times or more" being a 5) on the activity checklist to form a 

composite score for item 1. 

2) Item 2 to 8 (PE, recess, lunch, right after school, evening, weekends, and describes you best) 

- The answers for each item start from the lowest activity response and progress to the highest activity response. 

- Use the reported value that is checked off for each item (the lowest activity response being a 1 and the highest

activity response being a 5). 

3) Item 9 

- Take the mean of all days of the week ("none" being a 1, "very often" being a 5) to form a composite score for item 

9.

4) Item 10 

- Can be used to identify students who had unusual activity during the previous week, but this question is NOT used 
American Institutes for Research Evaluation of Coach Across America —Appendix 3 

as part of the summary activity score. 

5) How to calculate the final PAQ-A activity score 

- Once you have a value from 1 to 5 for each of the 9 items (items 1 to 9) used in the Physical Activity composite 

score, you simply take the mean of these 8 items, which results in the final PAQ-A activity summary score. 

- A score of 1 indicates low physical activity, whereas a score of 5 indicates high physical activity.
American Institutes for Research Evaluation of Coach Across America —Appendix 4 

Appendix B. HIA Scoring Instructions 

HIA survey scoring instructions 

The HIA survey questions are meant to measure the extent to which participants have a set of attributes 

that research shows will help them in sport and in life. 

The following questions measure the following attributes: 

Question Attribute 

1. I believe in myself. Positive Identity 

2. I get into bad situations without knowing it until it is too late. Situational Awareness

3. I consider more than one option when trying to solve the problems Plan B thinking 

I face in my life.

4. My future goals influence the decisions I make. Future Focus 

5. I have a hard time waiting when I want something. Discipline 

6. I am afraid to express my opinions to other people. Social Confidence 

7. The friends I spend time with make me a better person. Prosocial connections 

8. When I am having bad thoughts, I know how to change them to Self Awareness 

happier thoughts.

9. There are a lot of good things about me. Positive identity 

10. When I walk into a room, I look around to see if anything is unsafe. Situational Awareness 

11. If something doesn’t work out as I planned, I usually have a backup Plan B Thinking 

plan.

12. If I am feeling mad, it is hard to control 

I 

what I say or do. Discipline 

13. None of the adults in my life understand who I really am. Pro Social connections 

14. When I am in a bad mood, can name the reason why. Self Awareness

15. I know how to make plans t o reach my goals. Future Focus

16. If I see something happening that I think is wrong, I will speak up. Social Confidence 

These five questions are meant to measure general decision-making ability:

17. When I have to make a hard decision in life, I make the decision before thinking about all of 

the options.

18. When I have to make a hard decision in life, I talk to someone I trust about the decision.

19. When I have to make a hard decision in life, I feel helpless.

20. When I have to make a hard decision in life, I make the decision based on what will make 

other people like me.

21. I can resist peer pressure.

For some of the questions, “strongly agree” is an indicator of a higher amount of the attribute and for 

other questions, “strongly disagree” is an indicator of a higher amount of the attribute. In order to 

compare all of the questions to one another, we need to standardize the responses in a way that 1 is 

low for all questions and 5 is high for all questions. 
American Institutes for Research Evaluation of Coach Across America —Appendix 5 

Instructions for data entry and analysis: 

1. When entering survey data, start by entering responses exactly how they are in the survey: 1 for 

strongly agree, 2 for agree, 3 for not sure, 4 for disagree, and 5 for strongly disagree. 

2. Next, you will want to “flip” responses for the highlighted/bolded questions below so that all 1’s 

become 5’s, 2’s become 4’s, and so on: 

1. I believe in myself.

2. I get into bad situations without knowing it until it is too late.

3. I consider more than one option when trying to solve the problems I face in my life.

4. My future goals influence the decisions I make.

5. I have a hard time waiting when I want something.

6. I am afraid to express my opinions to other people.

7. The friends I spend time with make me a better person.

8. When I am having bad thoughts, I know how to change them to happier thoughts. 

9. There are a lot of good things about me. 

10. When I walk into a room, I look around to see if anything is unsafe. 

11. If something doesn’t work out as I planned, I usually have a backup plan.

12. If I am feeling mad, it is hard to control what I say or do. 

13. None of the adults in my life understand who I really am. 

14. When I am in a bad mood, I can name the reason why. 

15. I know how to make plans to reach my goals. 

16. If I see something happening that I think is wrong, I will speak up. 

17. When I have to make a hard decision in life, I make the decision before thinking about all 

of the options. 

18. When I have to make a hard decision in life, I talk to someone I trust about the 

decision.

19. When I have to make a hard decision in life, I feel helpless.

20. When I have to make a hard decision in life, I make the decision based on what will make 

other people like me.

21. I can resist peer pressure.

Doing this “flip” by hand is very time consuming and will probably result in errors, so the best way to do 

it is using the CHOOSE function in Excel. The following shows an example of the first 5 questions, with 

the original survey entry in purple and the appropriate questions flipped in green. Note for questions 

number 2 & 5 (red columns) the numbers were kept the same and for questions number 1, 3, and 4 the 

numbers were flipped using the formula 

=CHOOSE(cell#, 5,4,3,2,1) Æ cell# is a place holder for cell you want to flip (the original) 
American Institutes for Research Evaluation of Coach Across America —Appendix 6 

3. Once all of the questions have been standardized so that 5 indicates a high amount of the attribute 

and 1 indicates a low amount of the attribute (green in example above), pair the questions that 

measure the same attribute and average them for each participant: 

x Positive identity: #1 & #9 

x Situational awareness: #2 & #10 

x Plan B thinking: #3 & #11 

x Future focus: #4 & #15 

x Discipline: #5 & #12 

x Social confidence: #6 & #16 

x Prosocial connections: #7 & #13 

x Self awareness: #8 & #14 

This will give you a score from 1-5 for each attribute for each participant, where 5 is high and 1 is 

low. You can also average each participant’s overall decision-making score by averaging #17-21.

4. Next you can calculate things like: 

o Average participant score for each HIA 

o Average participant overall HIA score 

o Average participant decision-making score 

o Average participant total survey score 

o Percentage of participants who averaged 4 or above 

o Percentage of participants who averaged 2 or below 

o If survey is administered to same participants multiple times: 

 Change score for each HIA, overall HIA score, and decision-making score 

 Percentage of participants who improved, stayed the same, or decreased HIA score 

o MANY MORE! 
American Institutes for Research Evaluation of Coach Across America —Appendix 7 

Appendix C. Program Outcomes by Youth Attendance 

LOW ATTENDANCE HIGH ATTENDANCE

n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Differencep 

value n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Differencep 

value

Physical Activity - PAQ Score 939 3.07 3.12 -.06 .086 1256 3.19 3.32 -0.13 <.001

Nutrition 

In the past 24 hours, how many 

times did you... 

drink white milk? 497 2.75 2.79 -.04 .800 566 2.81 2.89 -0.07 .423

drink soda? 495 2.22 1.95 .27 .066 563 2.19 2.07 0.12 .169

drink 100% fruit juice? 497 2.87 2.50 .37 .027 566 2.88 2.78 0.11 .295

eat French fries, potato chips, 

other fried potatoes? 493 2.33 2.00 .33 .021 564 2.51 2.18 0.33 <.001

eat fruit? 487 3.09 3.18 -.09 .611 550 3.20 3.29 -0.10 .316

eat vegetables? 493 2.68 2.81 -.13 .465 553 2.79 2.64 0.15 .098

When I have to make a decision 

about what to eat or drink...

I know the difference 

between what is healthy for 

me and what is not healthy

704 4.05 4.01 .04 .679 853 3.99 4.11 0.12 .078

I usually choose the healthier 

option 704 3.53 3.44 .09 .412 855 3.50 3.55 -.05 .496

High Impact Attributes

Positive Identity 917 4.40 4.39 .00 .898 1238 4.33 4.34 -0.02 .608

Situational Awareness 916 3.33 3.36 -.03 .409 1240 3.39 3.43 -0.04 .228

Plan B thinking 926 3.74 3.73 .01 .735 1242 3.65 3.75 -0.10 .003

Future Focus 924 4.16 4.10 .05 .142 1241 4.11 4.16 -0.05 .121

Discipline 923 2.92 3.00 -.08 .112 1241 2.88 2.90 -0.02 .649
American Institutes for Research Evaluation of Coach Across America —Appendix 8 

LOW ATTENDANCE HIGH ATTENDANCE

n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Differencep 

value n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Differencep 

value

Social Confidence 924 3.56 3.60 -.04 .260 1241 3.52 3.54 -0.01 .697

Prosocial connections 921 3.70 3.79 -.09 .012 1241 3.63 3.68 -0.06 .103

Self Awareness 924 3.73 3.70 .03 .421 1239 3.68 3.70 -0.02 .630

Decision-making 

I can resist peer pressure. 628 3.66 3.72 -.06 .434 817 3.44 3.67 -0.23 <.001

When I have to make a hard 

decision in life... 

I make the decision before 

thinking about all of the 

options.

912 2.87 2.98 -.11 .093 1227 2.99 2.95 0.03 .536

I talk with someone I trust 

about the decision. 908 3.85 3.78 .07 .206 1225 3.83 3.87 -0.04 .380

I feel helpless. 911 3.19 3.43 -.24 <.001 1227 3.25 3.23 0.02 .752

I make the decision based on 

what will make other people 

like me.

914 3.17 3.31 -.14 .014 1225 3.23 3.25 -0.02 .756
American Institutes for Research Evaluation of Coach Across America —Appendix 9 

Appendix D: Number of Sessions Offered and Attended by CAA Program 

Sessions Offered Sessions Attended

Mean Minimum

MaximumMini- Mean

mum

Maximum

Total 39 1 128 26 0 113

America SCORES Bay Area 92 46 98 78 30 95

America SCORES Boston 37 6 39 29 4 38

America Scores Chicago 24 16 38 22 3 38

America SCORES Cleveland 46 6 47 43 0 47

America SCORES Dallas 73 73 73 58 5 73

America SCORES DC 34 4 36 19 0 32

America SCORES Denver 49 26 83 45 26 76

America Scores New York 21 9 26 19 0 26

America SCORES Seattle 43 11 47 41 11 47

A's and Aces 60 60 60 . . .

Baltimore SquashWise . . . . . .

Beat the Streets Wrestling 

Philadelphia 51 51 51 26 0 47

Beyond the Ball 40 38 42 31 15 40

Black Women in Sport 

Foundation 45 45 45 29 10 41

Boston Scholar Athlete 28 28 28 . . .

Boys & 

Atlanta

Girls Club of Metro 35 35 35 27 1 35

Boys & Girls Clubs of SELA . . . . . .

Bridge Lacrosse 40 30 40 26 18 39

Brotherhood Crusade LA 33 32 34 30 22 34

Chicago Run 54 21 79 44 13 78

Chris Paul (CP3) Afterschool 

Zone . . . . . .

Coach Art LA 63 10 89 27 0 83

CoachArt Bay Area 6 6 6 6 4 6

Cohen College Prep Middle 

School 49 49 49 46 37 49

Collegiate Academies . . . . . .

Colorado 

Club

Rapids Youth Soccer 34 34 34 27 5 34

Community Rowing . . . . . .

Crim Fitness Foundation 5 3 7 4 0 7

El Monte CBI 36 36 36 23 0 36
American Institutes for Research Evaluation of Coach Across America —Appendix 10

Sessions Offered Sessions Attended

Mean Minimum

MaximumMini- Mean

mum

Maximum

Family & Children Faith 

Coalition (d/b/a Hope for 

Miami)

30 16 51 18 3 33

Firstline 84 66 128 . . .

Girls in the Game . . . . . .

Harlem Lacrosse & 

Leadership - FDA Site 35 12 54 28 6 54

Heart of Los Angeles (HOLA) 12 8 12 10 7 11

Henry Street Settlement 45 16 68 19 1 54

I Challenge Myself 39 26 56 28 9 48

Illinois Youth Soccer 

Association 32 32 32 28 22 31

JT Dorsey Foundation 

(Kensington Soccer Club) 20 20 20 9 5 16

Lagniappe Academies . . . . . .

Lake Forest Elementary . . . . . .

Legacy Youth Tennis and 

Education . . . . . .

Metro Lacrosse 22 14 28 14 0 26

MetroSquash 34 3 51 26 3 47

MGR Foundation 8 5 10 5 1 10

Netball America 33 13 35 12 3 31

Notah Begay III Foundation 22 22 22 9 3 17

Oakland Lacrosse Club 37 25 39 32 20 38

Oakland Strokes . . . . . .

Parks and People Foundation . . . . . .

Philly Triple Threat . . . . . .

Play Rugby LA 10 10 10 9 4 10

Play Rugby NY 28 10 31 26 4 31

Raquet Up Detroit 40 10 48 32 2 47

Row New York 90 77 100 82 47 100

School of Basketball 13 13 13 3 1 5

Sheriff's Youth Foundation 

Los Angeles County

of 113 113 113 69 12 113

Skate Like A Girl . . . . . .

Soccer Without Borders 

Baltimore 90 74 102 41 1 76

Soccer Without Borders 

Boston 50 50 50 22 0 39
American Institutes for Research Evaluation of Coach Across America —Appendix 11

Sessions Offered Sessions Attended

Mean Minimum

MaximumMini- Mean

mum

Maximum

Soccer Without Borders 

California 97 58 110 44 2 104

Soccer without Borders 

Greenley 32 28 35 22 0 34

SOS Outreach 9 1 23 8 0 21

South Bronx United, Inc. 14 14 14 3 0 9

Southern California Golf 

Association 8 8 8 6 2 8

Sportsmen's Tennis Club 11 9 11 7 1 11

Squashbusters 83 22 111 69 15 100

StreetSquash . . . . . .

Tenacity . . . . . .

The Sanneh Foundation . . . . . .

The Service Board 34 26 36 27 1 34

The Youth Foundation 20 4 44 17 1 33

Touching Miami with Love 70 70 70 63 42 69

Triple Threat Mentoring 10 5 12 8 0 12

United for DC Soccer Club 35 35 35 28 4 35

Urban Initiatives 38 24 44 31 7 43

USTA National Junior Tennis 

& Learning Chapter-Adams 

50

21 17 22 . . .

Washington Nationals Youth 

Baseball Academy 29 29 29 27 14 29

Westhab, Inc. . . . . . .

World Fit for Kids 77 58 81 65 37 81

World Sport Chicago 29 25 32 6 0 23

YET NFL . . . . . .

Youth Enrichment Services 10 2 10 5 0 8

Youth Speak Collective 15 8 17 7 1 16

Zhang Sah . . . . . .
American Institutes for Research Evaluation of Coach Across America —Appendix 12

Appendix E: Survey Response Rates by CAA Program 

# of Baseline Endline Both

Participants n % n % n %

Total 6288 4173 66% 2338 37% 1702 27%

America SCORES Bay Area 23 20 87.0 16 69.6 15 65.2

America SCORES Boston 58 40 69.0 38 65.5 27 46.6

America Scores Chicago 69 60 87.0 57 82.6 50 72.5

America SCORES Cleveland 85 61 71.8 64 75.3 51 60.0

America SCORES Dallas 50 41 82.0 26 52.0 20 40.0

America SCORES DC 61 25 41.0 28 45.9 16 26.2

America SCORES Denver 41 32 78.0 18 43.9 11 26.8

America Scores New York 21 18 85.7 15 71.4 13 61.9

America SCORES Seattle 165 150 90.9 29 17.6 18 10.9

A's and Aces 16 14 87.5 9 56.3 7 43.8

Baltimore SquashWise 69 50 72.5 27 39.1 22 31.9

Beat the Streets Wrestling 56 18 32.1 18 32.1 17 30.4 Philadelphia

Beyond the Ball 71 51 71.8 33 46.5 29 40.8

Black Women in Sport 44 33 75.0 11 25.0 10 22.7 Foundation

Boston Scholar Athlete 45 33 73.3 0 0.0 0 0.0

Boys & Girls Club of Metro 84 4 4.8 4 4.8 4 4.8 Atlanta

Boys & Girls Clubs of SELA 15 11 73.3 4 26.7 0 0.0

Bridge Lacrosse 42 21 50.0 36 85.7 21 50.0

Brotherhood Crusade LA 29 29 100.0 26 89.7 26 89.7

Chicago Run 64 39 60.9 23 35.9 10 15.6

Chris Paul (CP3) Afterschool 28 12 42.9 0 0.0 0 0.0 Zone

Coach Art LA 59 35 59.3 23 39.0 16 27.1

CoachArt Bay Area 12 12 100.0 0 0.0 0 0.0

Cohen College Prep Middle 24 20 83.3 0 0.0 0 0.0 School

Collegiate Academies 26 26 100.0 0 0.0 0 0.0

Colorado Rapids Youth Soccer 20 9 45.0 13 65.0 4 20.0 Club

Community Rowing 35 35 100.0 35 100.0 35 100.0

Crim Fitness Foundation 65 64 98.5 46 70.8 45 69.2

El Monte CBI 55 55 100.0 25 45.5 25 45.5
American Institutes for Research Evaluation of Coach Across America —Appendix 13

# of Baseline Endline Both

Participants n % n % n %

Family & Children Faith 

Coalition (d/b/a Hope for 326 123 37.7 116 35.6 73 22.4

Miami)

Firstline 502 405 80.7 160 31.9 67 13.3

Girls in the Game 241 128 53.1 0 0.0 0 0.0

Harlem Lacrosse & Leadership - 133 95 71.4 75 56.4 52 39.1 FDA Site

Heart of Los Angeles (HOLA) 21 17 81.0 13 61.9 11 52.4

Henry Street Settlement 74 52 70.3 33 44.6 27 36.5

I Challenge Myself 38 34 89.5 14 36.8 14 36.8

Illinois Youth Soccer 26 11 42.3 19 73.1 5 19.2 Association

JT Dorsey Foundation 20 11 55.0 8 40.0 8 40.0 (Kensington Soccer Club)

Lagniappe Academies 18 13 72.2 14 77.8 9 50.0

Lake Forest Elementary 56 42 75.0 39 69.6 25 44.6

Legacy Youth Tennis and 108 107 99.1 0 0.0 0 0.0 Education

Metro Lacrosse 91 90 98.9 50 54.9 50 54.9

MetroSquash 95 64 67.4 53 55.8 36 37.9

MGR Foundation 19 15 78.9 0 0.0 0 0.0

Netball America 29 17 58.6 0 0.0 0 0.0

Notah Begay III Foundation 11 8 72.7 7 63.6 5 45.5

Oakland Lacrosse Club 32 32 100.0 32 100.0 32 100.0

Oakland Strokes 26 8 30.8 0 0.0 0 0.0

Parks and People Foundation 16 15 93.8 0 0.0 0 0.0

Philly Triple Threat 61 21 34.4 0 0.0 0 0.0

Play Rugby LA 48 37 77.1 29 60.4 24 50.0

Play Rugby NY 84 80 95.2 31 36.9 31 36.9

Raquet Up Detroit 38 32 84.2 22 57.9 18 47.4

Row New York 27 27 100.0 0 0.0 0 0.0

School of Basketball 141 31 22.0 0 0.0 0 0.0

Sheriff's Youth Foundation of 58 33 56.9 32 55.2 13 22.4 Los Angeles County

Skate Like A Girl 28 17 60.7 0 0.0 0 0.0

Soccer Without Borders 102 41 40.2 51 50.0 12 11.8 Baltimore

Soccer Without Borders Boston 26 15 57.7 14 53.8 6 23.1

Soccer Without Borders 155 38 24.5 51 32.9 17 11.0 California
American Institutes for Research Evaluation of Coach Across America —Appendix 14

# of Baseline Endline Both

Participants n % n % n %

Soccer without Borders 61 0 0.0 29 47.5 0 0.0 Greenley

SOS Outreach 579 481 83.1 354 61.1 311 53.7

South Bronx United, Inc. 16 15 93.8 0 0.0 0 0.0

Southern California Golf 179 61 34.1 0 0.0 0 0.0 Association

Sportsmen's Tennis Club 33 27 81.8 22 66.7 20 60.6

Squashbusters 133 118 88.7 120 90.2 109 82.0

StreetSquash 114 77 67.5 0 0.0 0 0.0

Tenacity 103 68 66.0 37 35.9 33 32.0

The Sanneh Foundation 46 29 63.0 0 0.0 0 0.0

The Service Board 68 35 51.5 21 30.9 17 25.0

The Youth Foundation 25 25 100.0 19 76.0 19 76.0

Touching Miami with Love 35 17 48.6 16 45.7 5 14.3

Triple Threat Mentoring 134 88 65.7 49 36.6 35 26.1

United for DC Soccer Club 18 17 94.4 0 0.0 0 0.0

Urban Initiatives 186 144 77.4 53 28.5 38 20.4

USTA National Junior Tennis & 42 41 97.6 0 0.0 0 0.0 Learning Chapter-Adams 50

Washington Nationals Youth 40 0 0.0 0 0.0 0 0.0 Baseball Academy

Westhab, Inc. 29 19 65.5 0 0.0 0 0.0

World Fit for Kids 98 77 78.6 77 78.6 57 58.2

World Sport Chicago 37 25 67.6 16 43.2 11 29.7

YET NFL 10 0 0.0 10 100.0 0 0.0

Youth Enrichment Services 131 90 68.7 0 0.0 0 0.0

Youth Speak Collective 44 22 50.0 10 22.7 6 13.6

Zhang Sah 45 20 44.4 18 40.0 14 31.1
American Institutes for Research Evaluation of Coach Across America —Appendix 15

Appendix F: Data Cleaning Steps 

The initial dataset was provided to AIR in a Microsoft Excel spreadsheet which included two 

tabs: one which contained baseline data from the Coach Across America programs included in 

the evaluation, and another which contained the labels and values for each of the variables in the 

dataset. As Up2Us received additional data for the dataset, it updated the spreadsheet and sent 

this updated dataset to AIR. AIR also communicated any issues, questions or concerns (e.g. data 

that was out of the range of a particular variable) they encountered while preparing the data for 

analysis and worked with Up2Us to resolve these issues. The most updated version of the dataset 

included data from the baseline survey and endline survey as well as attendance data. The data 

manager conducted the following steps to clean the data and transfer it to the statistical software 

package Stata 13.1:

x Checked all data for duplicate identification numbers 

x An “X” was placed in the dataset to indicate missing data. Removed all X’s from the 

dataset so that Stata would properly read the data as missing. 

x Deleted missing and invalid dates from the birth date (variable: birth) and interview date 

(variable: date1) columns. 

x Changed the “K’s” (which stood for kindergarten) in the grade column to “0” since all of 

the grade data was numeric (e.g. a participant in eighth grade would have an “8” in this 

column). 

x Separated the demographic data into a separate sheet. This would allow for easier 

processing when reshaping the dataset. The demographic variables in this sheet were as 

follows: code, code2, program, birth, grade gender, ethnicity, city, enrollment, coach1 

and coach2. 

x Removed the “code” variable from the attendance data. The attendance data would be 

merged with the demographic data later in the process, and as such only one key 

identifier was needed. 

x Renamed the column headers of survey variables in the dataset to their measure labels 

(e.g. hia01, paq01). 

x Added a “time” variable to the baseline dataset. Added value of “1” to the time variable 

for all baseline data. 

x After double checking that information in baseline and endline data sheets aligned, 

copied and pasted endline data into the same sheet as baseline data. 

x Added time value of “2” to all endline data.

Up2Us added a second “code” variable (code2). This was a numeric code that allowed the AIR 

team to easily import the data into Stata. The previous code variable was alphanumeric, which 

created an error in running frequencies on the dataset when in Stata. 

After the final dataset was created, the data was loaded into Stata and labeled with variable and 

value labels. Since data from two time points was included in this dataset, the data needed to be 

reshaped from “long” data—where each participant had baseline and endline data but these two 

sets of data were contained in two separate rows—to “wide” data, where the baseline and endline 

data of one participant were contained in one row. The following steps were taken to reshape the 

dataset: 
American Institutes for Research Evaluation of Coach Across America —Appendix 16

x Imported the attendance data sheet into Stata. 

x Reshaped all variables in the attendance data sheet from “long” to “wide.”

x Checked that reshape was successful. 

x Saved attendance data as separate dataset 

x Imported the demographic data sheet into Stata. 

x Reshaped all variables in the demographic data sheet from “long” to “wide.”

x Checked that reshape was successful. 

x Saved demographic data as separate dataset. 

x Imported the baseline and endline data sheet into Stata. 

x Reshaped all variables in the baseline and endline data sheet from “long” to “wide.”

x Checked that reshape was successful. 

x Saved attendance baseline and endline data as separate dataset. 

x Cleared all data from Stata. 

x Imported the newly saved attendance dataset. 

x Merged attendance dataset with demographic dataset. 

x Checked that merge was successful. 

x Saved merged attendance data and demographic data into separate dataset. 

x Imported newly saved baseline and endline data. 

x Merged baseline and endline data with recently merged attendance and demographic 

data. 

x Checked that merge was successful. 

x Reapplied variable and value labels where missing. 

x Saved merged dataset as final dataset. 
American Institutes for Research Evaluation of Coach Across America —Appendix 17

Appendix G. Program Outcomes by Grade Level 

ELEMENTARY SCHOOL MIDDLE AND HIGH SCHOOL

n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p

Physical Activity - PAQ Score 1081 3.3840 3.4066 -0.02 .443 1478 2.9205 3.0698 -0.15 <.001

Nutrition

In the past 24 hours, how many 

times did you... 

drink white milk? 505 2.87 3.01 -0.14 .234 628 2.78 2.72 0.06 .566

drink soda? 504 2.19 2.09 0.10 .390 624 2.23 2.03 0.20 .038

drink 100% fruit juice? 506 2.93 2.84 0.09 .483 627 2.84 2.48 0.36 .001

eat French fries, potato chips, 

other fried potatoes? 499 2.48 2.21 0.27 .020 626 2.41 2.06 0.34 <.001

eat fruit? 502 3.21 3.42 -0.21 .074 604 3.11 3.10 0.00 .984

eat vegetables? 505 2.73 2.80 -0.07 .522 611 2.91 2.67 0.25 .019

When I have to make a decision 

about what to eat or drink...

I know the difference between 

what is healthy for me and 

what is not healthy

753 4.02 3.97 0.05 .558 1132 3.96 4.09 -0.13 .039

I usually choose the healthier 

option 750 3.62 3.65 -0.04 .715 1133 3.36 3.37 -0.01 .845

High Impact Attributes

Positive Identity 1066 4.2958 4.3469 -0.05 .159 1446 4.3794 4.3693 0.01 .697

Situational Awareness 1066 3.4221 3.4690 -0.05 .256 1448 3.2831 3.3210 -0.04 .229

Plan B thinking 1069 3.5874 3.6770 -0.09 .032 1456 3.7598 3.8072 -0.05 .091

Future Focus 1068 4.0147 4.0880 -0.07 .061 1454 4.1745 4.1593 0.02 .579
American Institutes for Research Evaluation of Coach Across America —Appendix 18

ELEMENTARY SCHOOL MIDDLE AND HIGH SCHOOL

n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p

Discipline 1067 2.8878 2.9078 -0.02 .687 1454 2.8563 2.9383 -0.08 .022

Social Confidence 1069 3.4218 3.5153 -0.09 .028 1452 3.5835 3.5756 0.01 .798

Prosocial connections 1067 3.6933 3.7841 -0.09 .026 1451 3.5855 3.6569 -0.07 .019

Self-Awareness 1066 3.7039 3.7007 0.00 .940 1451 3.6947 3.7100 -0.02 .622

Decision-making 

I can resist peer pressure. 537 3.37 3.55 -0.18 .081 1035 3.67 3.83 -0.16 .001

When I have to make a hard 

decision in life...

I make the decision before 

thinking about all of the 

options.

1052 2.87 2.90 -0.03 .588 1436 2.97 3.03 -0.06 .228

I talk with someone I trust 

about the decision. 1045 3.82 3.87 -0.05 .421 1440 3.84 3.81 0.02 .532

I feel helpless. 1053 3.12 3.19 -0.08 .188 1443 3.28 3.40 -0.12 .006

I make the decision based on 

what will make other people 

like me.

1048 3.02 3.08 -0.06 .307 1437 3.31 3.43 -0.12 .007
American Institutes for Research Evaluation of Coach Across America —Appendix 19

Appendix H. Outcomes by Sex 

Male Female

N

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p N

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p

Physical Activity - PAQ Score 1370 3.25 3.34 -0.10 <.000 1189 2.99 3.07 -0.08 .003 

Nutrition

In the past 24 hours, how many 

times did you... 

drink white milk? 646 2.90 2.96 -0.06 .616 487 2.71 2.66 0.05 .651

drink soda? 642 2.30 2.11 0.20 .059 486 2.08 1.97 0.11 .270

drink 100% fruit juice? 646 2.91 2.64 0.26 .023 487 2.84 2.64 0.21 .075

eat French fries, potato chips, 

other fried potatoes? 642 2.48 2.16 0.32 .002 483 2.44 2.09 0.35 .001

eat fruit? 630 2.94 3.17 -0.24 .041 476 3.40 3.31 0.09 .441

eat vegetables? 634 2.76 2.73 0.03 .805 482 2.92 2.72 0.20 .074

When I have to make a decision 

about what to eat or drink...

I know the difference between 

what is healthy for me and 

what is not healthy

996 3.92 4.11 -0.19 .014 889 4.07 4.02 0.05 .463

I usually choose the healthier 

option 996 3.49 3.57 -0.09 .282 887 3.45 3.38 0.07 .333

High Impact Attributes

Positive Identity 1338 4.34 4.36 -0.01 .696 1174 4.36 4.38 -0.01 .651

Situational Awareness 1340 3.27 3.35 -0.08 .026 1174 3.42 3.43 -0.01 .862

Plan B thinking 1347 0.02 3.78 -3.76 .016 1178 3.67 3.71 -0.04 .224

Future Focus 1346 4.08 4.10 -0.02 .476 1176 4.17 4.17 0.00 .773

Discipline 1344 2.90 2.94 -0.04 .308 1177 2.82 2.90 -0.08 .059
American Institutes for Research Evaluation of Coach Across America —Appendix 20

Male Female

N

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p N

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Difference p

Social Confidence 1344 3.50 3.55 -0.06 .097 1177 3.55 3.56 -0.01 .867

Prosocial connections 1341 3.58 3.72 -0.15 <.000 1177 3.70 3.71 -0.01 .787

Self-Awareness 1341 3.70 3.74 -0.04 .280 1176 3.71 3.67 0.04 .290

Decision-making

I can resist peer pressure. 852 3.60 3.74 -0.13 .043 720 3.50 3.67 -0.17 .008

When I have to make a hard 

decision in life...

I make the decision before 

thinking about all of the options. 1325 2.88 2.95 -0.06 .225 1163 2.99 3.03 -0.04 .470

I talk with someone I trust about 

the decision. 1319 3.74 3.74 -0.01 .907 1166 3.95 3.95 0.00 >.99

I feel helpless. 1319 3.28 3.35 -0.08 .113 1167 3.13 3.26 -0.13 .010

I make the decision based on 

what will make other people like 1321 3.07 3.19 -0.12 .014 1164 3.32 3.40 -0.08 .131

me.
American Institutes for Research Evaluation of Coach Across America —Appendix 21

Appendix I. Outcomes by Program Intensity 

LOW INTENSITY (OFFERED) HIGH INTENSITY (OFFERED)

n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Differ

-ence p value n

Predictied 

Baseline 

mean

Predictie

d Endline 

mean

Differencep 

value

Physical Activity - PAQ Score 1013 3.07 3.14 -0.07 .029 1277 3.18 3.31 -0.13 <.001

Nutrition

In the past 24 hours, how many 

times did you... 

drink white milk? 530 2.71 2.77 -0.07 .724 533 2.78 2.88 -0.10 .280

drink soda? 527 2.60 1.95 0.66 <.001 531 2.12 2.09 0.02 .789

drink 100% fruit juice? 530 2.94 2.48 0.45 .013 533 2.90 2.80 0.10 .324

eat French fries, potato chips, 

other fried potatoes? 526 2.53 2.02 0.51 

.001

531 2.49 2.20 0.29 .002

eat fruit? 521 3.06 3.18 -0.11 .541 516 3.18 3.25 -0.07 .472

eat vegetables? 528 2.55 2.78 -0.23 .215 518 2.82 2.64 0.18 .044

When I have to make a decision 

about what to eat or drink...

I know the difference between 

what is healthy for me and 

what is not healthy

892 4.05 3.94 0.11 .237 1150 3.93 4.12 -0.19 .005

I usually choose the healthier 

option 890 3.50 3.46 0.04 .675 1138 3.47 3.55 -0.08 .229

High Impact Attributes

Positive Identity 995 4.36 4.37 -0.01 .714 1255 4.35 4.35 0.00 .980

Situational Awareness 994 3.36 3.37 -0.01 .773 1257 3.35 3.41 -0.06 .100

Plan B thinking 1000 3.72 3.70 0.01 .732 1263 3.65 3.77 -0.12 .001

Future Focus 998 4.16 4.12 0.04 .214 1262 4.10 4.15 -0.05 .143

Discipline 998 2.93 3.01 -0.08 .067 1261 2.83 2.85 -0.02 .715
American Institutes for Research Evaluation of Coach Across America —Appendix 22

LOW INTENSITY (OFFERED) HIGH INTENSITY (OFFERED)

n

Predicted 

Baseline 

mean

Predicted 

Endline 

mean

Differ

-ence p value n

Predictied 

Baseline 

mean

Predictie

d Endline 

mean

Differencep 

value

Social Confidence 998 3.54 3.59 -0.05 .147 1262 3.52 3.52 0.00 .988

Prosocial connections 997 3.69 3.79 -0.10 .006 1260 3.61 3.65 -0.04 .227

Self Awareness 998 3.76 3.73 0.03 .468 1259 3.66 3.68 -0.02 .680

Decision-making 

I can resist peer pressure. 660 3.71 3.73 -0.01 .843 792 3.41 3.67 -0.26 <.001

When I have to make a hard 

decision in life... 

I make the decision before 

thinking about all of the 

options.

986 2.90 3.01 -0.11 .063 1243 2.96 2.91 0.05 .367

I talk with someone I trust 

about the decision. 984 3.85 3.78 0.08 .124 1241 3.83 3.87 -0.04 .401

I feel helpless. 984 3.24 3.45 -0.21 <.001 1244 3.19 3.16 0.03 .588

I make the decision based on 

what will make other people 

like me.

988 3.18 3.35 -0.17 .002 1238 3.20 3.20 0.00 .943
LOCATIONS 

Domestic 

Washington, D.C. 

Atlanta, GA 

Baltimore, MD 

Chapel Hill, NC 

Chicago, IL 

Columbus, OH 

Frederick, MD 

Honolulu, HI 

Indianapolis, IN 

Naperville, IL 

New York, NY 

Rockville, MD

Sacramento, CA

San Mateo, CA 

Waltham, MA

International 

Egypt 

Honduras 

Ivory Coast 

Kyrgyzstan 

Liberia 

Tajikistan 

Zambia 

ABOUT AMERICAN INSTITUTES FOR RESEARCH 

Established in 1946, with headquarters in Washington, D.C., 

American Institutes for Research (AIR) is an independent, 

nonpartisan, not-for-profit organization that conducts behavioral 

and social science research and delivers technical assistance 

both domestically and internationally. As one of the largest 

behavioral and social science research organizations in the world, 

AIR is committed to empowering communities and institutions with 

innovative solutions to the most critical challenges in education, 

health, workforce, and international development. 
6003 Executive Blvd., Suite 300 

Rockville, MD 20852 

301.592.8600

www.air.org