School Turnaround AmeriCorps National Evaluation:

Year 1 Final Report Technical Appendixes

School Turnaround AmeriCorps National Evaluation: 

Year 1 Final Report

TECHNICAL APPENDIXES

September 28, 2015 

Prepared for:

Corporation for National and

Community Service

1201 New York Avenue NW

Washington, DC 20525 

Submitted by:

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138 

Authors: 

Jennifer Bagnell Stuart

Beth Gamse

Anna Jefferson

Ed Bein

Hiren Nisar

Brian Freeman

Samantha Burke 

Amy Checkoway

Erin Sullivan

Katherine Murphy

Marissa Personette

Will Huguenin

Alex Silverman

nationalservice.gov
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

School Turnaround AmeriCorps National Evaluation: 

Year 1 Final Report Technical Appendixes 

Contents 

A. TECHNICAL APPENDIXES .................................................................................................................................... 1
 

A.1 Survey Sampling............................................................................................................................................. 1
 

A.2 Survey Weighting, Nonresponse Adjustments, and Standard Errors .................................................... 1
 

A.2.1 Survey Weighting and Adjustment for Nonresponse ................................................................ 1
 

A.2.2 Standard Error Calculations ........................................................................................................... 3
 

A.3 Supplemental Survey Data Tables ............................................................................................................... 3
 

A.3.1 Grantee Survey................................................................................................................................. 4
 

A.3.2 School Leader Survey ...................................................................................................................... 9
 

A.3.3 Instructional Staff and Counselor Survey .................................................................................. 19
 

A.3.4 Parent Interviews ........................................................................................................................... 29
 

A.4 Supplemental Survey Exhibits.................................................................................................................... 30
 

A.5 Qualitative Methodology............................................................................................................................. 34
 

A.5.1 Qualitative Data Collection .......................................................................................................... 34
 

A.5.2 Qualitative Analysis ...................................................................................................................... 36
 

A.6 Data Collection Instruments........................................................................................................................ 36
 

B. REFERENCES ........................................................................................................................................................... 37
 

nationalservice.gov i
 
A. TECHNICAL APPENDIXES 

A.1 Survey Sampling

The main body of this report describes findings from Grantee, School Leader, and School Staff surveys, based on 

responses provided by 13 grantees1, 38 school leaders2, and 215 instructional staff and/or counselors, respectively. All 57 

school leaders whose schools were in their second year of School Turnaround AmeriCorps program implementation were 

surveyed.3 The study team used a multi-step process to survey a sample of instructional/counseling staff from those 57 

schools. First, the team requested rosters of instructional staff and counselors from all 57 schools, and 48 schools 

subsequently provided rosters. Next, the team selected a random sample of 543 staff from a sampling frame of 1743 

relevant staff (those who work with the grades served by AmeriCorps members). Approximately 31 percent of 

instructional staff and counselors at each school were surveyed for the staff survey, with at least five staff from each 

school included in the sample.

The school leader and school staff surveys were administered online between January 1, 2015 and February 13, 2015, and 

the grantee survey was administered online between May 4, 2015 and June 4, 2015. Response rates differed by survey (see 

Appendix Exhibit A-1 below). The grantee survey had a response rate of 100 percent, the school leader survey had a 

response rate of 68 percent, and despite repeated reminders, the response rate for the instructional staff and counselors 

was 40 percent.

Exhibit A-1: Response Numbers and Rates for School Leaders and Instructional Staff and Counselors

Respondent Type Number Selected Number Responded Response Rate

Grantees 13 13 100%

School Leaders 56a 38b 68%

Instructional Staff and Counselors 543 215 40%

NOTES: a The School Turnaround AmeriCorps program was in its second year of implementation in 57 schools, but two of those 

schools shared a principal. This principal was surveyed only once to minimize respondent burden.

b One school leader respondent reported having misunderstood the survey and asked to retake it, although then did not complete the 

survey upon starting it a second time. Consequently, this survey was excluded from analysis. Two respondents indicated that they 

occupied positions other than principal, assistant principal, or site director. The study team followed up with these two respondents and 

determined that their roles were equivalent to those of principals in other schools.

Exhibit reads: 100% of grantees, 68% of school leaders and 40% of instructional staff and counselors responded to the surveys.

A.2 Survey Weighting, Nonresponse Adjustments, and Standard Errors

A.2.1 Survey Weighting and Adjustment for Nonresponse

For the school leader and school staff surveys, the set of respondents did not encompass the entire population of interest, 

and therefore the study team developed survey weights whose application would enhance the representativeness of the 

survey responses. The weights were developed in two steps. First, the study team created a base weight for each 

individual invited to take a survey. A base weight is the inverse of the probability that the given individual would be 

selected to take a survey. For the school leader survey, the goal was to obtain a census of all treatment schools, and 

therefore invitations were extended to those schools with certainty. This meant that the base weights for the leader survey 

were 1. For the school staff survey, approximately 30 percent of relevant teachers at each school were randomly selected 

for the staff survey, yielding typical base weights of about 3.3. While all staff members at a given school were assigned the 

1 There are 12 grantee organizations and 13 grantee programs. Four grantee organizations implement their own programs. Eight 

grantee organizations are state service commissions with subgrantee organizations which implement the grantee programs. One 

state commission has two subgrantee organizations, each of which operates one grantee program. The term “grantee staff” 

specifically refers to grantee and/or subgrantee organizations’ staff who participated in the grantee focus groups, and is generally 

synonymous with the term “program staff.” Eleven of the School Turnaround AmeriCorps programs are single-state programs 

that must apply for funding through state service commissions and address local needs in only one state, and two are national 

programs that must apply for grants directly from CNCS and address local needs in at least two states.

2 School leaders are typically the administrators in charge of the school, typically school principals.

3 Two schools in the sample shared a principal. The study team surveyed this principal about one of the two schools to reduce 

respondent burden.

nationalservice.gov 1 

 
same base weight, the base weights differed slightly from school to school, reflecting the slightly different school-specific 

sampling rates.4

Second, the study team modified the base weights to adjust for survey nonresponse in accordance with the federal Office 

of Management and Budget (OMB) Standards and Guidelines for Statistical Surveys (sections 1.3 and 3.2) to address the 

potential concern that respondents may differ systematically those who did not respond, and hence would not accurately 

represent the views of the population from which they were drawn. A standard mitigation approach, therefore, is to 

adjust the base weights of respondents to make them more representative of the selected sample as a whole (that is, both 

respondents and non-respondents) on baseline characteristics measured on the entire sample. For instance, because 

gender and school enrollment were known for all individuals invited to take either survey, it was possible to make weight 

adjustments separately for the two surveys, based on these and other measured characteristics, to make respondents 

resemble the entire sample. Consider, as an example, the first row of Exhibit A-2. Among all school leaders, 66 percent 

were female, whereas 58 percent of respondents were female. A potential concern given these different proportions is that 

the representativeness of any survey item responses correlated with gender would be compromised given the relative 

underrepresentation of females among the respondents. However, once the final nonresponse-adjusted weights are 

applied, the estimate of 67 percent nearly matches the sample-wide value of 66 percent, and addresses the relative lack of 

representativeness. Similarly, in the second row of Exhibit A-3, base-weighted respondents over-represent the wholesample base-weighted estimate of proportion of females in the staff population, and this has been rectified via the use 

of nonresponse-adjusted weights. Though imperfect, the nonresponse adjustments generally improved the 

representativeness of the respondents on the measured baseline characteristics. See Exhibit A-2 for details about the 

leader survey and Exhibit A-3 about the staff survey. One caveat: this adjustment does not guarantee improvements of 

similar magnitudes regarding the representativeness of the survey responses of the respondents.

To develop the nonresponse adjustments, the study team fit a logistic regression model separately for each survey to 

predict the probability that each sample member would respond to the survey based on his or her observed 

characteristics. These response probabilities are called propensity scores. The propensity scores were then sorted and 

divided them into quintiles. Each sample member in a quintile was assigned a nonresponse weighting factor equal to the 

inverse of the average propensity score in the quintile. This factor, multiplied by a respondent’s base weight, gave the 

respondent’s final weight. The final weights of non-respondents were set to zero. These final weights were used in all 

survey item data analyses.5

Exhibit A-2: Characteristics of School Leader Survey Sample and Respondents

Variable

Sample

(base weight)

Respondents 

(base weight)

Respondents 

(final weight)

Female 66% 58% 67%

Student enrollment (school) 512 545 514

Student-teacher ratio (school) 15.1 15.7 14.9

Percent minority students (school) 83% 77% 84%

Percent of students eligible for free or reduced-price lunch 

(school) 85% 82% 86%

Percent of students proficient in math (school) 33% 37% 33%

Percent of students proficient in reading (school) 38% 41% 36%

Region

Northeast 25% 32% 24%

Midwest 32% 29% 34%

South 29% 26% 31%

West 14% 13% 11%

School Level

Elementary 57% 55% 59%

High 43% 45% 41%

4 Though the study’s sampling rate target was 30 percent, the effective percentage of instructional staff and counselors sampled 

from each school varied slightly, as not all schools necessarily had a sampling frame cleanly divisible by ten. Also, since the study 

team sampled a minimum of five instructional staff and/or counselors per school, smaller schools (those with relatively few 

instructional staff and counselors) had sample percentages significantly larger than 30 percent.

5 See Valliant, Dever, and Kreuter (2013, p. 316-338) for a discussion of nonresponse adjustment.

nationalservice.gov 2 

 
Exhibit A–3: Characteristics of Instructional Staff and Counselor Survey Sample and Respondents

Variable

Sample

(base weight)

Respondents

(base weight)

Respondents

(final weight)

Teacher 84% 84% 86%

Female 70% 77% 70%

Student enrollment (school) 666 684 675

Student-teacher ratio (school) 16.8 15.9 16.7

Percent minority students (school) 82% 76% 81%

Percent of students eligible for free or reduced-price 

lunch (school) 84% 81% 84%

Percent of students proficient in math (school) 37% 38% 36%

Percent of students proficient in reading (school) 43% 44% 43%

Region

Northeast 28% 25% 31%

Midwest 23% 20% 23%

South 41% 48% 40%

West 7% 8% 6%

School Level

Elementary 23% 17% 22%

Middle 15% 15% 16%

Elementary & Middle 16% 21% 16%

High 40% 41% 41%

Middle & High 6% 7% 6%

A.2.2 Standard Error Calculations

Data analyses for the school leader and school staff surveys were implemented using the Stata 13 suite of svy survey 

commands. The study team collapsed four-level Likert scale survey items to binary endorsement indicators 

(e.g., “strongly agree/agree/disagree/strongly disagree” collapsed to “agree/disagree”), and used the final survey weights 

to estimate endorsement rates (e.g., proportion of the population agreeing with the item). To quantify the uncertainty in 

estimates based on a survey sample, jackknife resampling was used to generate standard errors and confidence intervals. 

Jackknife resampling involves creating subsamples that systematically exclude one or more sample members, 

recalculating their survey weights, and then re-estimating the population endorsement rate. The initial standard error is 

then derived from the set of re-estimates.6 The study team then constructed 95 percent confidence intervals around the 

jackknife point estimates by applying a finite population correction7 to the initial standard errors and multiplying the 

product by 1.96 to obtain the margin of error, which is half the width of a standard 95% confidence interval. When the 

estimated endorsement rate was very high in the leader survey, the study team limited the upper bound of the confidence 

intervals to one minus the proportion of sample members who did not endorse the item, as the population proportion 

cannot possibly exceed this figure. Likewise, when the estimated endorsement rate was very low, the lower bound of the 

confidence intervals was limited to the proportion of sample members who did endorse the item.

A.3 Supplemental Survey Data Tables

This section contains supplemental exhibits for all grantee survey, school leader survey, instructional staff and counselor 

survey items, along with quantitative parent interview questions. The supplemental exhibits present estimated 

percentages measured on complete categorical scales (e.g., strongly agree to strongly disagree). This is in contrast to the 

presentation of only positive response options (e.g., strongly agree and agree) for the report exhibits. The estimated 

percentages in several report graphs are displayed along with 95% confidence intervals, in order to indicate the degree of 

precision of these estimates. Due to the proliferation of estimated percentages reported in the supplemental exhibits, 

6 See Valliant, Dever, and Kreuter (2013, p. 418-426) for a description of the jackknife procedure. For the school leader survey, the 

study team created 56 subsamples, each of which excluded one sampled school leader. For the instructional staff and counselor

survey, 177 subsamples were created, each of which excluded between 1 and 8 sampled staff, following Valliant, Dever, and 

Kreuter (p. 437-441).

7 The finite population correction accounts for the added precision of collecting data from a sample that approaches the size of the 

target population. Following Lohr (2009), the study team used a finite population correction equal to √((NN − nn)/NN), where NN is 

stratum size and nn is the number of respondents.

nationalservice.gov 3 

 
confidence intervals are not provided, but the reader is alerted to the fact that the degree of precision of these estimates 

will be roughly comparable to the degree of precision of the analogous estimates presented in the report exhibits. In 

addition, percentages for survey item response options may differ between the report exhibits and the supplemental 

exhibits because the study team excluded respondents who selected “Don’t Know” or “Not Applicable” from the 

denominator in the report exhibits only.

A.3.1 Grantee Survey

Exhibit A-4: Relationship with Target School Prior to School Turnaround AmeriCorps Program

Statement n

Any prior relationship (N=13)

Yes, with some schools 8

Yes, with all schools 4

No 1

Duration of prior relationship (N=12)

One year or less 1

Two years 3

Three or more years 4

Varies by school 4

NOTES: (N=12-13, Missing=0)

Duration question limited to grantees whose relationship with the school(s) existed before the School Turnaround AmeriCorps 

partnership agreement.

Among the four grantees where duration of relationship varied by school, one grantee had a relationship of less than six months and all

four had relationships of one year, two years, and three years.

Exhibit reads: 8 grantees had a relationship with some of their target schools prior to the School Turnaround AmeriCorps program. 1 of 

the grantees with any prior school relationships indicated that their prior relationship lasted one year or less.

SOURCE: Grantee Survey Q1 (Did your organization’s relationship with the school(s) your grant is operating in exist before you 

established a partnership agreement for the School Turnaround AmeriCorps program?)

Grantee Survey Q1a (If yes, how long has your organization been collaborating with the school(s)?)

Exhibit A-5: Unit of Service for School Turnaround AmeriCorps Direct Services (2014-2015)

Target n

Individual students 11

Whole classroom 7

All students 4

Varies by school 3

NOTES: (N=13, Missing=0)

Frequencies do not sum to 13 because multiple responses were permitted.

Exhibit reads: Eleven grantees indicated that School Turnaround AmeriCorps members provided direct services to individual students.

SOURCE: Grantee Survey Q2 (To the best of your knowledge, did School Turnaround AmeriCorps members provide direct services to 

individual students, to the whole classroom, or to all students in the school during the 2014-15 school year?)

Exhibit A-6: Grantee Knowledge of Students Served by School Turnaround AmeriCorps (2014-2015)

Knowledge Level n

Yes 11

Sometimes, but not always 2

No 0

NOTE: (N=13, Missing=0)

Exhibit reads: Eleven grantees knew which students were served by School Turnaround AmeriCorps members.

SOURCE: Grantee Survey Q3 (Do you know which students were served by School Turnaround AmeriCorps members?))

nationalservice.gov 4 
Exhibit A-7: Target Number of Students Expected to be Served by School Turnaround AmeriCorps

 

Average number of 

students per school 

expected to be 

served

(n)

Average number 

of students per 

school expected 

to complete 

services

(n)

Complete/

Serve

(%)

Average 

number of 

AmeriCorps 

members per 

school

(n)

Caseload 

(Complete/Members)

(%)

Mean 210 189 89 8 28

Standard deviation 150 170 30 5 20

Maximum 540 673 165 20 87

Minimum 46 38 45 1 8

Median 160 135.5 78 6 24.5

1st Quartile 102 60.75 74.75 4 16.25

3rd Quartile 230 226.75 100 10 34.25

NOTE: (N=13, Missing=0)

Exhibit reads: The mean number of students per school expected to be served across all 13 grantees is 210. The mean number of 

students per school expected to complete services across all 13 grantees is 189. The mean percentage of students expected to 

complete services out of students expected to be served is 89%. The mean number of AmeriCorps members per school is eight. The 

mean caseload (number of students per school expected to complete services divided by the number of AmeriCorps members) is 28. 

SOURCE: Grantee Survey Q5 (Please review the list below to confirm the schools to which your organization assigned School 

Turnaround AmeriCorps members. Fill in the number of members who serve at each school during 2014-15, and the targeted number 

of students that you expect to serve and complete the program this school year. If you don’t know, please write in “DK.”))

Exhibit A-8: Frequency of Student Progress Meetings Between School Turnaround AmeriCorps Members and 

School Staff

Occurrences n

More than twice per month 5

Once per month 3

Once per year 1

Very different from school to school 3

NOTE: One grantee did not know how often School Turnaround AmeriCorps members meet with school staff to discuss student 

progress data. (N=12, Missing=0)

Exhibit reads: Five grantees indicated that meetings between School Turnaround AmeriCorps members and school staff were held 

more than twice per month. 3 grantees indicated that it is very different from school to school.

SOURCE: Grantee Survey Q7 (On average, how often do the School Turnaround AmeriCorps members meet with school staff to 

discuss data on the progress of all students? Please select the option that is closest to your members’ experience.)

Exhibit A-9: Mechanism to Identify Students to Participate in School Turnaround AmeriCorps (2014-2015)

n

Teacher recommendation 9

Grades 7

Counselor recommendation 6

Standardized test scores 5

Student request 4

Parent request 2

NOTE: (N=13, Missing=0)

Exhibit reads: Nine grantees chose "Teacher recommendation" as a mechanism to identify students for School Turnaround 

AmeriCorps.

SOURCE: Grantee Survey Q8 (To the best of your knowledge, which mechanisms did the school(s) use to identify students to 

participate in activities led by School Turnaround AmeriCorps members?) 

nationalservice.gov 5 
Exhibit A-10: Reason Students Identified to Participate in School Turnaround AmeriCorps (2014-2015)

n

Improve academic achievement 12

Improve academic engagement 10

Improve behavior 9

Improve self-esteem or socio-emotional health 8

Assist students at risk for dropping out 7

Sustain performance 6

NOTE: (N=13, Missing=0)

Exhibit reads: Twelve grantees indicated that students were identified to participate to "Improve academic achievement."

SOURCE: Grantee Survey Q9 (To the best of your knowledge, what are the reasons that students were identified to participate in 

School Turnaround AmeriCorps programming?)

Exhibit A-11: Grantee Perceptions of Most Important Student Outcomes in Next Two Years (2014-2015)

Rank of Importance

School Turnaround Student Outcome Number Ranked Mean Rankings (Standard Deviations)

Enhanced academic achievement 12 2.8 (2.7)

Improved grades 12 3.5 (2.4)

Improved attendance 12 4.1 (1.9)

Increased motivation 12 4.3 (2.0)

Improved behavior 12 4.6 (1.7)

Improved completion of assignments 11 5.4 (1.9)

Improved socio-emotional health 12 5.5 (2.5)

Increased self-esteem 12 5.8 (2.0)

NOTE: (N=13, Missing=0)

Ranks range from 1 - 9, with 1 being the most important; not all responses were given a ranking. One grantee indicated that this 

question was different for different schools, but still provided a ranking.

Exhibit reads: Grantees ranked "Enhanced academic achievement" as the most important student outcome, with a mean ranking of 3.0 

on a 9-point scale; 100% of grantees ranked this particular outcome.

SOURCE: Grantee Survey Q10 (What do you consider to be the most important school turnaround outcomes for students over the next 

two years?)

Exhibit A-12: Grantee Perceptions of Areas of Improvement in Student Outcomes (2014-2015)

Quantity of Schools

Degree of 

Improvement

Student Outcome

Total

(n)

All

(n)

Most 

(>50%)

(n)

Some

(25-49%)

(n)

Small 

Number 

(<25%)

(n)

Don't 

Know

(n)

Mean 

Rankings 

(Standard 

Deviations)

Number 

Ranked

Enhanced academic achievement 13 7 1 2 0 3 2.4 (2.0) 9

Improved grades 13 4 1 2 0 6 2.6 (2.1) 7

Improved completion of assignments 13 4 0 3 0 6 4.0 (2.2) 6

Increased motivation 13 2 1 1 0 9 4.0 (3.6) 4

Increased self-esteem 13 2 1 1 0 9 4.3 (1.3) 4

Improved attendance 13 4 0 1 1 7 4.5 (2.1) 5

Improved socio-emotional health 13 2 1 1 0 9 4.5 (2.2) 4

Improved behavior 13 3 2 0 2 6 4.8 (2.6) 6

NOTE: (N=13, Missing=0)

Ranks range from 1 - 9, with 1 being the most important. Not all responses were given any ranking. Means are calculated for grantees 

who ranked the option.

Table rows are sorted in ascending order by the mean rankings of grantees.

Exhibit reads: Seven grantees perceived that students at all schools demonstrated "Enhanced academic achievement." One grantee 

perceived "Enhanced academic achievement" in most schools, two in some schools. Three grantees did not know. Grantees ranked 

"Enhanced academic achievement" as the student outcome with the greatest degree of improvement across schools, with a mean 

ranking of 2.4 on a 9-point scale; 9 grantees ranked this particular outcome. 

SOURCE: Grantee Survey Q11 (Please fill in the following table about student outcomes)

nationalservice.gov 6 
Exhibit A-13: Grantee Opinions About Partner Collaboration

Statement

Total

(n)

Strongly 

Agree

(n)

Agree

(n)

Disagree

(n)

Strongly

Disagree

(n)

Very 

Different

(n)

It is easy for me to get in touch with someone from the school(s) 13 8 2 0 0 3

There is frequent communication between my organization and 

the school(s) (e.g., visits to each other’s offices, meetings, 

written information and telephone communications)

13 8 3 0 0 2 

The school(s)responds, if needed, when I make contact 13 7 4 0 0 2

The school(s) is (are) committed to making our collaboration a 

success

13 7 3 0 0 3 

The school(s) has (have) the ability to accomplish set goals 13 6 5 0 0 2

The school(s) puts forth effort to maintain relationship(s) with my 

organization 13 6 3 0 0 4 

NOTE: (N=13, Missing=0)

Exhibit reads: Eight grantees strongly agree and two agree that "It is easy for me to get in touch with someone from the school(s)." 

Three grantees believe that this is very different from school to school.

SOURCE: Grantee Survey Q12 (Please indicate your level of agreement/disagreement about your organization’s collaboration with

your school partner(s) for each statement listed below. Please try to respond in reference to the typical school, if you work with more 

than one.))

nationalservice.gov 7 
Exhibit A-14: Grantee Satisfaction with Elements of School Turnaround AmeriCorps (2014-2015)

Element

Total

(n)

Very 

Satisfied

(n)

Satisfied

(n)

Dissatisfied

(n)

Very 

Dissatisfied

(n)

Don't 

Know

(n)

Very 

Different

(n)

Not 

Applicable

(n)

Communication between 

school(s) and grantee 13 5 4 0 0 0 4 0 

Communication and 

collaboration between 

teachers and School 

Turnaround AmeriCorps 

members

13 1 6 0 0 0 4 2 

Communication and 

collaboration between 

school leadership and 

School Turnaround 

AmeriCorps members

13 3 7 0 0 0 3 0 

Implementation of the roles 

and responsibilities outlined 

in the school partnership 

agreements

13 3 8 0 0 0 1 1 

Placement of members in 

meaningful service 

activities

13 5 7 0 0 0 1 0 

Referral of students to 

receive services offered by 

School Turnaround 

AmeriCorps members

13 5 5 0 0 0 1 2 

Matching of members to 

students in need of 

academic strengthening 

and/or social/emotional 

supports

13 6 4 0 0 1 1 1 

Alignment of School 

Turnaround AmeriCorps 

member activities with 

school turnaround plans

13 6 6 0 0 0 1 0 

Sharing of outcome data by 

the school/district 13 7 5 0 0 0 1 0 

NOTE: (N=13, Missing=0)

Number of respondents who did not know ranged from 0-1. Number of respondents who indicated the question was not applicable 

ranged from 0-2.

Exhibit reads: Five grantees are very satisfied and four are satisfied with the "Communication between school(s) and grantee." Four

grantees perceived it as very different from school to school.

SOURCE: Grantee Survey Q14 (For this school year, please indicate how satisfied you are with each of the statements below:)

nationalservice.gov 8 
Exhibit A-15: Grantee Perceptions of Important Elements of School Turnaround AmeriCorps Implementation 

(2014-2015)

Element

Total

(n)

Very 

important

(n)

Important

(n)

Somewhat 

important

(n)

Not at all 

important

(n)

Not 

Applicable

(n)

Orientation and training of AmeriCorps members 

before they serve at the school 13 12 1 0 0 0 

Comprehensive trainings of AmeriCorps 

members and program support staff during their 

year(s) of service

13 8 5 0 0 0 

Multi-layered supervisory structure to ensure 

fidelity of program implementation 13 10 2 1 0 0 

Defined framework (e.g., RTI) to guide objective 

instructional choices and allow for the 

assessment of program effectiveness

13 6 5 2 0 0 

Limited set of highly scripted interventions that 

have been shown to be effective (i.e., research 

based) in achieving desired student-level 

outcomes

12 4 4 2 2 0 

Recruitment and selection process that 

effectively identifies members with 

characteristics/skills that correspond with the 

program objectives

13 9 4 0 0 0 

NOTE: (N=13, Missing=0-1)

Exhibit reads: Twelve grantees perceived that "Orientation and training of AmeriCorps members before they serve at the school" is very 

important. 1 grantee perceived it as important.

SOURCE: Grantee Survey Q16 (How important are the following characteristics to successfully implementing your School Turnaround 

AmeriCorps program at a typical school?)

A.3.2 School Leader Survey

Exhibit A-16: School Leader Title/Role (2014-2015)

Title/Role %

Principal 89

Assistant Principal 2

Site Director 2

Other 7

NOTES: The "Other" response consisted of an Instructional Facilitator. (N=38, Missing=1)

Table rows are sorted in descending order.

Exhibit reads: 89% of school leaders surveyed were principals.

SOURCE: School Leader Survey Q1 (What is your title/role?)

Exhibit A-17: School Leader’ Experience at Current School (2014-2015)

Years Worked at Current School %

Fewer than 3 years 28

Between 3 and 4 years 21

4 years or more 51

NOTES: The three school leaders that selected “Not applicable” have likely already met their School Improvement Grant (SIG) 

requirements. (N=38, Missing=1)

Exhibit reads: 28% of school leaders had worked fewer than 3 years at their current school.

SOURCE: School Leader Survey Q2 (How many years total have you worked at this school, including this year?)

nationalservice.gov 9 
Exhibit A-18: School Improvement Grant Models (2014-2015)

School Improvement Grant (SIG) Change Model %

Turnaround 64

Transformation 22

Restart 7

Not applicable 8

NOTES: The three school leaders who selected “Not applicable” may have already met their School Improvement Grant (SIG) 

requirements. (N=38, Missing=1)

Table rows are sorted in descending order.

Exhibit reads: 64% of schools follow the Turnaround school improvement grant model.

SOURCE: School Leader Survey Q4 (What School Improvement Grant (SIG) change model does your school follow?)

Exhibit A-19: Number of Students Served by School Turnaround AmeriCorps per School (2014-2015)

Number of Students Served %

Not sure 17

Fewer than 10 students 44

Between 10 and 100 students 13

100 or more students 26

NOTES: Five recipients (13%) saw the question text "This school year, how many AmeriCorps members are serving at your school as 

part of the School Turnaround AmeriCorps program?" (N=38, Missing=0)

Exhibit reads: 17% of school leaders were not sure how many students were served by School Turnaround AmeriCorps at their school.

SOURCE: School Leader Survey Q5_1 (This school year, how many students are AmeriCorps members serving at your school as part 

of the School Turnaround AmeriCorps program?)

Exhibit A-20: Average Hours per Week Members Serve in School (2014-2015)

Hours per week %

Fewer than 20 hours per week 4

Between 20 and 30 hours per week 19

Between 30 and 40 hours per week 38

Between 40 and 50 hours per week 27

50 or more hours per week 12

NOTES: (N=38, Missing=0)

Exhibit reads: 4% of AmeriCorps members served fewer than 20 hours per week on average in the school.

SOURCE: School Leader Survey Q5_3 (On average, how many hours per week does each of these AmeriCorps members serve this 

school year (2014-15)?)

Exhibit A-21: Average Number of Weeks per Year Members Serve in School (2014-2015)

Weeks %

Fewer than 30 weeks per year 14

Between 30 and 40 weeks per year 50

40 or more weeks per year 37

NOTES:(N=38, Missing=2)

Exhibit reads: 14% of AmeriCorps members served fewer than 30 weeks per year in the school.

SOURCE: School Leader Survey Q5_4 (On average, how many weeks do these AmeriCorps members spend in your school this 

school year (2014-15)?) 

Exhibit A-22: Unit of Service for School Turnaround AmeriCorps Direct Services (2014-2015)

Target %

Individual students 74

All students in the school 54

Whole classroom 45

NOTES: Percentages do not sum to 100 because multiple responses were allowed. 

(N=38, Missing=0)

Table rows are sorted in descending order.

Exhibit reads: 74% school leaders reported that School Turnaround AmeriCorps services are provided to individual students.

SOURCE: School Leader Survey Q6 (To the best of your knowledge, to whom do School Turnaround AmeriCorps members provide 

direct services this school year? (Check all that apply.))

nationalservice.gov 10
Exhibit A-23: School Leaders’ Knowledge of Students Served by School Turnaround AmeriCorps (2014-2015)

Knowledge Level %

Yes 74

Sometimes, but not always 26

No 0

NOTES: (N=38, Missing=2)

Table rows are sorted in descending order.

Exhibit reads: 74% of school leaders knew which students were served by School Turnaround AmeriCorps.

SOURCE: School Leader Survey Q7 (Do you know which students are served by School Turnaround AmeriCorps members this school 

year?)

Exhibit A-24: School Leaders’ Communications with and Monitoring of School Turnaround AmeriCorps Activities 

and Members (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Not 

Applicable

(%)

Teachers in this school are supportive of the AmeriCorps 

program

37 41 47 7 5 0 

The principal and/or school leadership team monitors 

performance and progress of implementation of turnaround 

activities at this school

38 36 58 2 4 -- 

The principal and/or school leadership team communicates 

a clear vision of turnaround to AmeriCorps members 38 34 51 12 4 -- 

The principal and/or school leadership team monitors 

performance and progress of students and share this 

information with AmeriCorps members

38 29 53 15 4 -- 

AmeriCorps members are integrated into regular staff 

meetings and communication 38 24 61 12 4 0 

NOTES: Items marked with "--" could not be entered as "Not Applicable".

(N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected “Strongly Agree”.

Exhibit reads: 41% of school leaders strongly agreed and 47% agreed that “Teachers in this school are supportive of the AmeriCorps 

program.”

SOURCE: School Leader Survey Q8 (Please indicate your level of agreement/disagreement with each of the elements listed below for 

this school year (2014-15): (Mark one response in each row))

nationalservice.gov 11
Exhibit A-25: School Leaders’ Perceptions of Important Elements of School Turnaround AmeriCorps 

Implementation (2014-2015)

Element

Total

(n)

Very 

important

(%)

Important

(%)

Somewhat 

important

(%)

Not at all 

important

(%)

Not 

Applicable

(%)

AmeriCorps member recruitment and selection 

process that identifies and selects members with skills 

aligned with the program's objectives

38 74 26 0 0 0 

Alignment of AmeriCorps activities to the strategies 

outlined in the school’s turnaround plan 38 69 29 2 0 0 

Clearly defined supervisory structure to ensure fidelity 

of program implementation 38 67 33 0 0 0 

Orientation and training of AmeriCorps members 

before they serve at the school 38 65 33 2 0 0 

Highly defined set of research-based interventions to 

improve desired student-level outcomes 38 64 32 4 0 0 

Comprehensive trainings of AmeriCorps members and 

program support staff during their year(s) of service 37 57 41 2 0 0 

Clearly defined framework (e.g. RTI) to guide 

instructional choices and allow for the assessment of 

program effectiveness

38 56 44 0 0 0 

NOTES: (N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected “Very Important”.

Exhibit reads: 74% of school leaders perceived that the “AmeriCorps member recruitment and selection process that identifies and 

selects members with skills aligned with the program’s objectives” is very important. 26% of school leaders perceived it as important.

SOURCE: School Leader Survey Q9 (How important are the following to the successful implementation of School Turnaround 

AmeriCorps program in your school(s)?)

Exhibit A-26: School Leaders’ Perceptions of Most Important Student Outcomes in Next Two Years (2014-2015)

Rank of Importance

School Turnaround Student Outcome Mean Rankings Standard Errors Percent Ranked

Enhanced academic achievement 1.5 0.3 100

Increased motivation 3.5 0.4 94

Improved grades 4.1 0.7 77

Improved attendance 4.4 0.8 84

Increased self-esteem 4.5 0.4 81

Improved socio-emotional health 5.0 0.5 93

Improved completion of assignments 5.8 0.7 76

Improved behavior 6.4 0.6 82

Other 2.0 0.0 7

NOTES: Ranks range from 1 - 9, with 1 being the most important; not all responses were given a ranking. Means are calculated for 

respondents who ranked the option.

No respondents indicated that this question was not applicable. (N=38, Missing=0)

Table rows are sorted in ascending order by the mean ranking of school leaders.

Exhibit reads: School Leaders ranked “Enhanced academic achievement” as the most important student outcome, with a mean ranking 

of 1.5 on a 9-point scale; 100% of School Leaders ranked this particular outcome. 

SOURCE: School Leader Survey Q10 (What do you consider to be the most important school turnaround outcomes for students over 

the next two years? (Please rank in order of importance with 1 as the most important. Please only rank outcomes that you consider 

important.))

nationalservice.gov 12
Exhibit A-27: School Leaders’ Perceptions of Areas of Improvement in Student Outcomes (2013-2014)

Any Improvement Degree of Improvement

Area of Improvement n Yes (%) No (%)

Don't 

Know (%)

Mean 

Rankings

Standard 

Errors

Percent 

Ranked

Enhanced academic achievement 31 100 0 0 1.7 0.3 88

Increased motivation 27 74 0 26 3.2 0.3 83

Improved attendance 26 96 0 4 3.5 0.9 95

Improved grades 30 92 0 8 3.7 0.5 85

Improved behavior 33 97 0 3 4.0 1.1 79

Increased self-esteem 25 70 0 30 4.3 0.7 78

Improved completion of assignments 31 91 0 9 4.3 0.6 71

Improved socio-emotional health 28 84 0 16 5.2 0.6 76

Other 1 100 0 0 NA NA 0

NOTES: Ranks of improvement range from 1 - 9, with 1 being the greatest improvement; not all responses were given a ranking. 

Means are calculated for respondents who ranked the option.

Any Improvement: (N=38, Missing=0)

Degree of Improvement: (N=1-33, Missing=0)

Any improvement n column describes the number of survey respondents who provided an answer in each row. The remaining columns

present nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1A.2.1 for details). 

Table rows are sorted in ascending order by the mean ranking of school.

Exhibit reads: School leaders ranked “Enhanced academic achievement” as the most important student outcome, with a mean ranking 

of 1.7 on a 9-point scale; 88% of school leaders ranked this particular outcome.

SOURCE: School Leader Survey Q11 (Please answer the following about school outcomes for students. Was there improvement in this 

area at your school(s) last year (2013-14)?)

Exhibit A-28: School Leaders’ Perceptions of School Climate and Student Supports (2014-2015)

Statement about School

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Don't 

Know

(%)

Support for Learning

Promotes academic success for all students 38 66 30 2 2 0

Offers students a supportive and inviting environment 

within which to learn 38 65 31 2 2 0 

Emphasizes helping students academically when they 

need it 38 64 34 0 2 0 

Offers staff a supportive and inviting environment within 

which to work 38 59 40 0 2 0 

Promotes trust and collegiality among staff 38 52 44 2 2 0

Emphasizes teaching lessons in ways relevant to 

students 37 52 41 6 2 0 

Sets high standards for academic performance for all 

students. 38 46 50 2 2 0 

Provides the materials, resources, and training 

(professional development) needed to do your job 

effectively

38 46 44 9 2 0 

Provides adequate counseling and support services for 

students 38 43 49 7 2 0 

Provides the materials, resources, and training 

(professional development) needed to work with special 

education (IEP) students

37 43 48 8 2 0 

Has sufficient teaching staff to meet the needs of 

students 38 42 38 18 2 0 

Has sufficient support staff to meet the needs of students 38 33 41 22 4 0

nationalservice.gov 13
Statement about School

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Don't 

Know

(%)

Student Autonomy and Access to Opportunities

Gives all students equal opportunity to participate in a 

variety of extracurricular activities 37 45 50 4 0 0 

Gives all students equal opportunity to participate in a 

variety of enrichment activities 37 43 38 19 0 0 

Gives all students equal opportunity to participate in 

classroom discussions or activities 38 39 61 0 0 0 

Encourages students to enroll in rigorous courses (such 

as honors and AP), regardless of race, ethnicity, or 

nationality

34 38 54 6 0 3 

Gives students opportunities to “make a difference” by 

helping other people, the school, or the community (e.g., 

service learning)

36 36 55 9 0 0 

Encourages opportunities for students to decide things 

like class rules 37 31 52 16 0 2 

Diversity and Culture

Considers closing the racial/ethnic achievement gap a 

high priority 37 56 42 0 0 2 

Emphasizes showing respect for all students’ cultural 

beliefs and practices 38 55 41 2 2 0 

Fosters an appreciation of student diversity and respect 

for one another 37 46 53 2 0 0 

Has staff examine their own cultural biases through 

professional development 36 35 46 18 0 0 

Emphasizes using instructional materials that reflect the 

culture or ethnicity of our students 37 34 55 7 0 4 

Discipline Environment

Handles discipline problems fairly 38 67 26 5 2 0

Clearly communicates to students the consequences of 

breaking school rules 38 67 23 9 2 0 

Effectively handles student discipline and behavioral 

problems. 38 46 48 4 2 0 

School Safety and Physical Appearance

Is a safe place for staff 38 67 31 0 2 0

Is welcoming to and facilitates parent involvement 38 56 42 0 2 0

Is a safe place for students 38 55 44 0 2 0

Has clean and well-maintained facilities and property 38 49 39 10 2 0

Makes information and resources available to 

parents/guardians about how they can support their 

children’s education

38 39 59 0 2 0 

Exhibit reads: 66% of school leaders strongly agreed and 30% agreed that the school “Promotes academic success for all students”. 

SOURCE: School Leader Survey Q12 (Please indicate your level of agreement/disagreement with the following statements about your 

school during the 2014-15 school year. (Mark one response in each row.))

nationalservice.gov 14
Exhibit A-29: School Leaders’ Perceptions of School Challenges (2014-2015)

Topic

Total

(n)

Severe 

Challenge

(%)

Moderate 

Challenge

(%)

Not a 

Challenge

(%)

Student academic performance 37 59 41 0

Student attendance 37 41 50 8

Student depression or other mental health problems 37 27 55 18

Disruptive student behavior 37 22 66 12

Student engagement in school 36 19 71 11

Student aspirations for college and/or career 37 18 58 24

Student fatigue/lack of sleep 36 13 58 29

Student behavior and discipline 37 11 78 10

Lack of respect of staff by students 37 11 47 42

Student safety 37 9 28 63

Harassment or bullying among students 37 8 67 25

Student alcohol and drug use 37 4 34 62

Cutting classes or being truant 36 3 39 58

Theft 37 3 22 75

Physical fighting between students 37 2 40 58

Student tobacco use 37 2 29 69

Racial/ethnic conflict among students 37 2 20 77

Gang-related activity 37 0 23 77

Vandalism (including graffiti) 37 0 10 90

Weapons possession 37 0 8 92

NOTES: (N=38, Missing=1-2)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected “Severe Challenge”.

Exhibit reads: 59% of school leaders perceived “Student academic performance” as a severe challenge, and 41% perceived it as a 

moderate challenge. 

SOURCE: School Leader Survey Q13 (Please indicate whether the following topics represent challenges in your school this school 

year (2014-15). (Mark one response in each row.))

Exhibit A-30: School Leaders’ Perceptions of Students, Teachers, and Families (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Teachers and other school staff communicate and collaborate 38 42 58 0 0

Students treat AmeriCorps members with respect 38 4 90 6 0

Students treat teachers with respect 37 4 87 9 0

Students take their school work seriously 38 4 78 18 0

Students treat each other with respect 37 2 83 15 0

Families play an active role in our school 38 0 65 30 5

NOTES: (N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected strongly agree.

Exhibit reads: 42% of school leaders strong agreed and 58% agreed that “Teachers and other school staff communicate and 

collaborate.”

SOURCE: School Leader Survey Q14 (Please indicate your level of agreement/disagreement with the following statements about your 

school this school year (2014-15). (Mark one response in each row.))

nationalservice.gov 15
Exhibit A-31: School Leaders’ Perceptions about Out-of-Classroom Student Supports (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Not 

Applicable

(%)

Make appropriate support services available to students 

with special needs 38 41 52 7 0 0 

Deliver wraparound services and non-academic 

(social/emotional) supports to students 38 40 53 7 0 0 

Expose students to post-secondary education 

opportunities and increase student interest in and 

knowledge about college

37 36 49 4 3 8 

Provide academic enrichment, extended learning time or 

other academic supports to students 38 36 58 6 0 0 

Connect parents/guardians to information and resources 

to help them support their children’s education 38 30 66 2 0 2 

Increase awareness about and access to health 

resources/services (e.g. targeting drug use, mental health, 

teen pregnancy)

38 29 58 7 0 7 

NOTES: Number of respondents who indicated the question was not applicable ranged from 0-3. (N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Strongly Agree" with “Not Applicable” 

excluded from the denominator.

Exhibit reads: 41% of school leaders strongly agreed and 52% agreed that they “Make appropriate support services available to 

students with special needs”. 

SOURCE: School Leader Survey Q15 (Please indicate your level of agreement/disagreement with the following statements about your 

school this school year (2014-15). (Mark one response in each row.))

Exhibit A-32: School Leaders’ Satisfaction with Elements of School Turnaround AmeriCorps (2014-2015)

Element

Total

(n)

Very 

Satisfied

(%)

Satisfied

(%)

Dissatisfied

(%)

Very 

Dissatisfied

(%)

Not 

Applicable

(%)

Implementation of the roles and responsibilities 

outlined in the school partnership agreements 36 30 53 3 0 14

Communication and collaboration between school 

leadership and School Turnaround AmeriCorps 

members

37 34 63 3 0 0 

Communication and collaboration between teachers 

and School Turnaround AmeriCorps members 37 29 68 3 0 0 

Overall quality of School Turnaround AmeriCorps 

programming 37 28 64 2 7 0 

Communication between school leadership and 

grantee staff 37 25 57 10 0 8 

Collaboration between school leadership and 

grantee staff 37 25 57 10 0 8 

Matching of members to students in need of 

academic strengthening and social/emotional 

supports

37 25 65 3 7 0 

Referral of students to receive services offered by 

AmeriCorps members 37 23 67 10 0 0 

Placement of members in meaningful service 

activities 37 19 71 10 0 0 

NOTES: Number of respondents who indicated this question was not applicable ranged from 0-3. (N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Very Satisfied" with “Not Applicable” 

excluded from the denominator.

Exhibit reads: 30% of school leaders are very satisfied and 53% are satisfied with the “Implementation of the roles and responsibilities 

outlined in the school partnership agreements”. 

SOURCE: School Leader Survey Q16 (For this school year (2014-15), please indicate your level of satisfaction/dissatisfaction with 

each of the elements listed below. (Mark one response in each row.))

nationalservice.gov 16
Exhibit A-33: School Leaders’ Perceptions of Success of School Turnaround AmeriCorps (2014-2015)

Success Statement

Total

(n)

Very

Successful

(%)

Somewhat

Successful

(%)

Somewhat 

Unsuccessful

(%)

Very

Unsuccessful

(%)

Not

Applicable

(%)

Overall success 37 47 47 7 0 0

Success in improving the school’s 

capacity to implement its turnaround 

model

37 46 42 2 7 3 

Success in improving school climate 37 43 45 6 7 0

Success in improving student socioemotional health 37 36 53 12 0 0 

Success in improving student 

academic achievement 37 34 54 8 0 3 

NOTES: Number of respondents who indicated the question was not applicable ranged from 0-1. (N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected “Very Successful”.

Exhibit reads: 47% of school leaders perceived School Turnaround AmeriCorps to be very successful overall, and 47% of school 

leaders perceived it to be somewhat successful. 

SOURCE: School Leader Survey Q17 (In your opinion, how successful is the School Turnaround AmeriCorps program in the following 

areas this school year (2014-15)? (Mark one response in each row.))

Exhibit A-34: School Leaders’ Perceptions of Value of School Turnaround AmeriCorps (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Not 

Applicable

(%)

School Turnaround AmeriCorps members provide helpful 

support to the students in this school 38 45 55 0 0 0 

School Turnaround AmeriCorps members are important 

partners in improving student outcomes 38 40 53 0 6 0 

School Turnaround AmeriCorps members offer supports that 

are beneficial to the teachers in this school 38 38 51 5 6 0 

School Turnaround AmeriCorps activities occur frequently 

enough to be valuable 38 26 59 9 0 6 

School Turnaround AmeriCorps members provide access to 

information and resources to parents/guardians about how 

they can support their children’s education

38 13 64 12 3 7 

School Turnaround AmeriCorps members engage 

parents/guardians to become involved in their children’s 

school

38 13 58 21 3 4 

NOTES: Number of respondents who indicated the question was not applicable ranged from 0-3. (N=38, Missing=0)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected “Strongly Agree”.

Exhibit reads: 45% of school leaders strongly agreed and 55% agreed that “School Turnaround AmeriCorps members provide helpful 

support to the students in this school”.

SOURCE: School Leader Survey Q18 (Please indicate your level of agreement/disagreement with the following statements this school 

year (2014-15). (Mark one response in each row.))

nationalservice.gov 17
Exhibit A-35: School Leaders’ Perceptions about School Improvement Grant Strategy Most Influenced by School 

Turnaround AmeriCorps (2014-2015)

Rank of Influence

School Improvement Strategy (SIG) Mean Rankings Standard Errors

Percent 

Ranked

Academic achievement 1.8 0.4 90

School culture and environment 2.5 0.3 85

Increased learning time 2.7 0.5 91

Family and community engagement 3.6 0.3 91

Graduation rates 4.2 0.3 59

College enrollment rates 4.7 0.5 69

NOTES: Ranks range from 1 - 6, with 1 being the most important; not all responses were given a ranking. Means are calculated for 

respondents who ranked the option. (N=31, Missing=3)

Table rows are sorted in ascending order by the mean ranking of school.

Exhibit reads: School leaders ranked “Academic achievement” as the most important school improvement strategy, with a mean 

ranking of 1.8 on a 6-point scale; 90% of school leaders ranked this particular outcome. 

SOURCE: School Leader Survey Q19 (Which School Improvement Grant (SIG) strategies are influenced the most by the School 

Turnaround AmeriCorps members this school year?)

Exhibit A-36: School Leaders’ Perceptions of Level of Influence of School Turnaround AmeriCorps on School 

Turnaround Goals (2014-2015)

Element of School's Turnaround Goals

Total

(n)

Substantial 

influence

(%)

Some 

influence

(%)

Minimal 

influence

(%)

No 

influence

(%)

Not 

Applicable

(%)

Establishing a school culture and environment that 

fosters school safety, attendance, and discipline 38 23 61 13 0 2 

Improving academic performance in ELA and/or math 38 23 59 6 9 3

Increasing rates of high school graduation 37 9 35 9 7 41

Increasing college readiness and enrollment rates 36 6 48 4 9 33

Providing ongoing mechanisms for family and 

community engagement 37 10 51 30 6 4 

NOTES: Number of respondents who indicated the question was not applicable ranged from 1-16. (N=38, Missing=0-1)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Substantial Influence" with “Not 

Applicable” excluded from the denominator.

Exhibit reads: 23% of school leaders perceived that School Turnaround AmeriCorps has a substantial influence on the School 

Turnaround goal of “Establishing a school culture and environment that fosters school safety, attendance, and discipline”. 61% 

perceived they have some influence, 13% perceived they have minimal influence, 0% perceived they have no influence, and 2% said 

the question was not applicable to them.

SOURCE: School Leader Survey Q20 (Please indicate the level of influence School Turnaround AmeriCorps members have over the 

following elements of your school’s turnaround goals? (Mark one response in each row.))

nationalservice.gov 18
A.3.3 Instructional Staff and Counselor Survey

Exhibit A-37: School Staff Primary Roles (2014-2015)

Primary Role/Position %

All subjects 19

Mathematics 19

Special education/resource 14

Science and technology/engineering 12

English language arts 11

History and social science 10

English language learners 6

Instructional coach for teachers 5

Reading/literacy support 4

Other electives 3

School counselor 3

Foreign languages 2

Comprehensive health/physical education 2

Vocational Technical programs 2

Visual and performing arts 1

Speech, physical, or occupational therapist 1

Librarian 0

Nurse 0

Others 8

NOTES: Percentages do not sum to 100 because multiple responses were permitted.

All subjects selected if elementary school teacher provides instruction in most or all core academic subjects.

(N=215, Missing = 0)

Table rows are sorted in descending order.

Exhibit reads: 19% of school staff teach all subjects.

SOURCE: Instructional Staff and Counselors Survey Q1 (What is your primary role/position this school year?)

Exhibit A-38: Grades Served by School Staff (2014-2015)

Grade %

K 12

1 11

2 9

3 13

4 14

5 9

6 20

7 22

8 23

9 36

10 37

11 36

12 35

NOTES: Percentages do not sum to 100 because multiple responses were permitted. (N=215, Missing=0)

Exhibit reads: 12% of school staff teach kindergarten.

SOURCE: Instructional Staff and Counselors Survey Q2 (What grades do you work with? (Check all that apply.))

nationalservice.gov 19
Exhibit A-39: School Staff Years of Experience (2014-2015)

Years Worked

Total

(%)

At the School

(%)

Fewer than 2 years 11 25

Between 2 and 6 years 36 53

Between 6 and 12 years 27 13

12 years or more 26 10

NOTES: Total: (N=215, Missing=4)

At the school (N=215, Missing=2)

Exhibit reads: 11% of school staff have less than 2 years of teaching or counseling experience. 25% of school staff have less than 2 

years of teaching or counseling experience at their current school.

SOURCE: Instructional Staff and Counselors Survey Q3 (How many years total have you worked as a teacher/counselor and how 

many years total have you worked at this school, including this year?)

Exhibit A-40: Number of Students Served by School Staff and Average Class Size (2014-2015)

%

Number of Students Overall (N=215)

Fewer than 50 students 27

Between 50 and 100 students 21

Between 100 and 200 students 25

200 students or more 27

Average per class (N=185)

Fewer than 10 students 7

Between 10 and 20 students 22

Between 20 and 30 students 49

30 students or more 21

NOTES: Overall: (N=215, Missing=0)

Average per class: 30 respondents indicated this question was not applicable. (N=185, Missing=4)

Exhibit reads: 27% of school staff work with fewer than 50 students overall; 7% of school staff have an average class size smaller than 

10 students.

SOURCE: Instructional Staff and Counselors Survey Q4 (Across all your responsibilities, approximately how many students do you 

work with this year?) - Q5 (On average, how many students do you teach in each class? (If not applicable, enter NA))

Exhibit A-41: Proportion of Staffs’ Students Served by School Turnaround AmeriCorps (according to staff 

familiar with the program) (2014-2015)

Target of AmeriCorps Programming %

Less than 25 percent of students 35

Between 25 and 50 percent of students 16

Between 50 and 75 percent of students 9

75 percent of students or more 40

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Sixty respondents were not familiar with the School Turnaround AmeriCorps program.

Sixty nine respondents did not know how many of their students were served by School Turnaround AmeriCorps. (N=86, Missing=0)

Exhibit reads: 35% of staff reported that less than 25 percent of their students were involved in School Turnaround AmeriCorps 

programming.

SOURCE: Instructional Staff and Counselors Survey Q6 (Approximately how many students with whom you have worked this school 

year (2014-15) are/were involved in School Turnaround AmeriCorps programming?)

nationalservice.gov 20
Exhibit A-42: Mechanisms to Identify Students for School Turnaround AmeriCorps Activities (2014-2015)

Rank of Frequency

Mechanism Mean Rankings Standard Errors Percent Ranked

Teacher recommendation 2.1 0.1 89

Counselor recommendation 2.6 0.2 66

Standardized test scores 2.6 0.2 62

Grades 3.1 0.2 71

Parent request 3.6 0.3 43

Student request 3.8 0.2 46

Other 2.3 0.6 15

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Ranks range from 1 - 7, with 1 being the most important.

Not all responses were given any ranking. Means are calculated for respondents who ranked the option.

Fifty one respondents did not know which mechanisms were most frequently used. (N=104, Missing=3)

Table rows are sorted in ascending order by the mean ranking of school staff.

Exhibit reads: School staff ranked “Teacher recommendation” as the most important mechanism, with a mean ranking of 2.1 on a 7-

point scale; 89% of school leaders ranked this particular outcome. 

SOURCE: Instructional Staff and Counselors Survey Q7 (To the best of your knowledge, which of the following mechanisms are most 

frequently used in your school to identify students for activities led by School Turnaround AmeriCorps members)

Exhibit A-43: Reasons Students were Identified to Participate in School Turnaround AmeriCorps (2014-2015)

Rank of Frequency

Reason Mean Rankings Standard Errors Percent Ranked

Improve academic achievement 1.8 0.2 88

Improve academic engagement 2.3 0.1 79

Improve behavior 3.1 0.2 61

Improve self-esteem or socio-emotional health 3.4 0.2 60

Assist students at risk for dropping out 3.5 0.2 53

Sustain performance 4.4 0.2 52

Other 4.2 1.1 8

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Ranks range from 1 - 7, with 1 being the most important.

Not all responses were given any ranking. Means are calculated for respondents who ranked the option.

Forty one respondents did not know the reasons students were identified. (N=114, Missing=1)

Table rows are sorted in ascending order by the mean ranking of school staff.

Exhibit reads: School staff ranked “Improve academic achievement” as the most important reason, with a mean ranking of 1.8 on a 7-

point scale; 88% of school leaders ranked this particular outcome. 

SOURCE: Instructional Staff and Counselors Survey Q8 (To the best of your knowledge, what are the reasons that students were 

identified to participate in School Turnaround AmeriCorps programming this school year?) 

Exhibit A-44: Target for Direct Services in School Turnaround AmeriCorps (2014-2015)

Target %

Individual students 80

All students in the school 34

Whole classroom 33

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Percentages do not sum to 100 because multiple responses were allowed.

Twenty three respondents did not know the targets of School Turnaround AmeriCorps programming. (N=132, Missing=0)

Table rows are sorted in descending order.

Exhibit reads: 80% of school staff said School Turnaround AmeriCorps members served individual students.

SOURCE: Instructional Staff and Counselors Survey Q9 (To the best of your knowledge, to whom do School Turnaround AmeriCorps 

members provide direct services this school year? (Check all that apply.))

nationalservice.gov 21
Exhibit A-45: School Staff Knowledge of Students Served by School Turnaround AmeriCorps Members (2014-

2015)

Do you know which students are served by School Turnaround AmeriCorps members? %

Yes 37

Sometimes, but not always 42

No 21

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program. 

(N=155, Missing=6)

Table rows are sorted in descending order by the proportion of school leaders who selected each option. Exhibit reads: 37% of school 

staff knew which students were served by School Turnaround AmeriCorps members.

SOURCE: Instructional Staff and Counselors Survey Q10 (Do you know which students are served by School Turnaround AmeriCorps 

members this school year?)

Exhibit A-46: School Staff Perceptions of Most Important Student Outcomes in Next Two Years (2014-2015)

Rank of Importance

School Turnaround Student Outcome Mean Rankings Standard Errors Percent Ranked

Enhanced academic achievement 2.6 0.2 91

Increased motivation 3.4 0.2 85

Improved grades 4.3 0.2 75

Improved attendance 4.5 0.2 73

Increased self-esteem 4.2 0.2 75

Improved socio-emotional health 4.1 0.2 73

Improved completion of assignments 4.8 0.2 67

Improved behavior 4.4 0.2 76

Other 5.2 2.6 3

NOTES: Ranks range from 1 - 9, with 1 being the most important.

Not all responses were given any ranking. Means are calculated for respondents who ranked the option.

Ten respondents (5%) indicated that this question was not applicable. (N=205, Missing=0)

Table rows are sorted in ascending order by the mean ranking of school staff.

Exhibit reads: School staff ranked “Enhanced academic achievement” as the most important school turnaround student outcome, with a 

mean ranking of 2.6 on a 9-point scale; 91% of school staff ranked this particular outcome. 

SOURCE: Instructional Staff and Counselors Survey Q11 (What do you consider to be the most important school turnaround outcomes 

for students over the next two years? (Please rank in order of importance with 1 as the most important. Please only rank outcomes that 

you consider important.))

Exhibit A-47: School Staff Perceptions of Improvement in Student Outcomes (2014-2015)

Quantity of Students

Student Outcome

Total

(n)

All

(%)

Most

(%)

Some

(%)

Few

(%)

None

(%)

Don't Know

(%)

Enhanced academic achievement 153 9 23 22 6 1 39

Increased motivation 152 7 24 21 5 1 42

Improved completion of assignments 151 7 19 23 9 2 40

Improved grades 150 7 16 26 8 1 42

Increased self-esteem 153 8 22 21 4 1 45

Improved behavior 152 8 20 23 7 1 42

Improved socio-emotional health 153 7 17 20 7 3 46

Improved attendance 152 5 15 19 6 4 50

Other 54 0 9 0 4 4 83

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Ranks range from 1 - 9, with 1 being the most important.

Not all responses were given any ranking.

Number of respondents who did not know ranged from 47-76. (N=155, Missing=2-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the mean ranking of student outcomes.

Exhibit reads: 9% of school staff perceived that all students demonstrated enhanced academic achievement. 23% perceived enhanced 

academic achievement in most students, 22% in some students, 6% in few students, and 1% in no students. 39% of school staff did not 

know how many students demonstrated enhanced academic achievement.

SOURCE: Instructional Staff and Counselors Survey Q12 (For how many of the students served by School Turnaround AmeriCorps at

your school this year (2014-2015) are there improvements in the following areas?)

nationalservice.gov 22
Exhibit A-48: School Staff Perceptions of Improvement in Student Outcomes (2014-2015)

Degree of Improvement

Student Outcome Mean Rankings Standard Errors Percent Ranked

Enhanced academic achievement 2.6 0.3 74

Increased motivation 2.9 0.2 74

Improved completion of assignments 3.1 0.3 59

Improved grades 3.3 0.3 54

Increased self-esteem 3.8 0.4 62

Improved behavior 4.1 0.5 56

Improved socio-emotional health 4.6 0.5 56

Improved attendance 5.1 0.5 48

Other 9.0 7.8 10

NOTES: Ranks range from 1 - 9, with 1 being the most important.

Not all responses were given any ranking. Means are calculated for respondents who ranked the option. (N=5 95, Missing=0)

Table rows are sorted in ascending order by the mean ranking of school staff.

Exhibit reads: School staff ranked “Enhanced academic achievement” as the most important student outcome, with a mean ranking of 

2.6 on a 9-point scale; 74% of school staff ranked this particular outcome. 

SOURCE: Instructional Staff and Counselors Survey Q12 (For how many of the students served by School Turnaround AmeriCorps at 

your school this year (2014-2015) are there improvements in the following areas?)

Exhibit A-49: School Staff Communications with and Monitoring of Members (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Not 

Applicable

(%)

Teachers in this school are supportive of 

the AmeriCorps program 150 44 44 5 2 4 

Teachers and AmeriCorps members 

successfully collaborate to support 

students

151 32 41 12 10 5 

Teachers in this school discuss their 

expectations for students with 

AmeriCorps members

150 31 44 12 7 6 

Teachers in this school share and 

discuss behavioral management

strategies with AmeriCorps members

151 30 38 16 8 8 

Teachers in this school share and 

discuss instructional practices with 

AmeriCorps members

151 27 43 16 7 8 

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Number of respondents who indicated the question was not applicable ranged from 5-10. (N=155, Missing=4-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school staff who selected “Strongly Agree”.

Exhibit reads: 44% of school staff strongly agreed and 44% agreed that “Teachers in this school are supportive of the AmeriCorps 

program.”

SOURCE: Instructional Staff and Counselors Survey Q13 (Please indicate your level of agreement/disagreement with the following 

statements about teacher relationships with School Turnaround AmeriCorps members this school year (2014-15). (Mark one response 

in each row.))

Exhibit A-50: School Staff Perceptions of School Climate and Student Supports (2014-2015)

Statement about School

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Support for Learning

Promotes academic success for all students 211 32 55 11 2

Offers students a supportive and inviting environment within which 

to learn 212 31 56 12 2 

Emphasizes helping students academically when they need it 210 40 57 2 1

Offers staff a supportive and inviting environment within which to 

work 210 25 52 14 9 

Promotes trust and collegiality among staff 207 24 60 11 4

nationalservice.gov 23 
Statement about School

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Emphasizes teaching lessons in ways relevant to students 211 27 59 12 2

Sets high standards for academic performance for all students. 212 36 46 14 4

Provides the materials, resources, and training (professional 

development) needed to do your job effectively 210 21 54 19 6 

Provides adequate counseling and support services for students 211 24 43 23 10

Provides the materials, resources, and training (professional 

development) needed to work with special education (IEP) 

students

209 13 51 22 14

Has sufficient teaching staff to meet the needs of students 214 21 45 23 11

Has sufficient support staff to meet the needs of students 214 18 45 25 11

Student Autonomy and Access to Opportunities

Gives all students equal opportunity to participate in a variety of 

extracurricular activities 212 26 57 14 3 

Gives all students equal opportunity to participate in a variety of 

enrichment activities 213 24 56 17 3 

Gives all students equal opportunity to participate in classroom 

discussions or activities 213 33 60 5 2 

Encourages students to enroll in rigorous courses (such as honors 

and AP), regardless of race, ethnicity, or nationality 206 25 47 23 5 

Gives students opportunities to “make a difference” by helping 

other people, the school, or the community (e.g., service learning) 212 20 49 25 6 

Encourages opportunities for students to decide things like class 

rules 213 21 54 21 4 

Diversity and Culture

Considers closing the racial/ethnic achievement gap a high priority 213 29 51 17 3

Emphasizes showing respect for all students’ cultural beliefs and 

practices 212 32 55 13 1 

Fosters an appreciation of student diversity and respect for one 

another 213 30 54 15 1 

Has staff examine their own cultural biases through professional 

development 210 19 41 30 11

Emphasizes using instructional materials that reflect the culture or 

ethnicity of our students 211 20 57 20 3 

Discipline Environment

Handles discipline problems fairly 214 23 47 17 12

Clearly communicates to students the consequences of breaking 

school rules 214 32 43 14 10

Effectively handles student discipline and behavioral problems 210 22 41 25 12

School Safety and Physical Appearance

Is a safe place for staff 214 33 57 6 4

Is welcoming to and facilitates parent involvement 213 27 56 15 1

Is a safe place for students 214 30 57 9 4

Has clean and well-maintained facilities and property 213 28 52 16 4

Makes information and resources available to parents/guardians 

about how they can support their children’s education 211 23 64 11 2 

NOTES: (N=215, Missing=1-9)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Strongly Agree".

Exhibit reads: 32% of school staff strongly agreed and 55% agreed that the school “Promotes academic success for all students.”

SOURCE: Instructional Staff and Counselors Survey Q14 (Please indicate your level of agreement/disagreement with the following 

statements about your school during the 2014-15 school year. (Mark one response in each row.))

nationalservice.gov 24
Exhibit A-51: School Staff Perceptions about Out-of-Classroom Student Supports (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Not 

Applicable

(%)

Make appropriate support services available to students 

with special needs 208 28 48 14 5 5 

Deliver wraparound services and non-academic 

(social/emotional) supports to students 209 23 60 9 4 5 

Expose students to post-secondary education 

opportunities and increase student interest in and 

knowledge about college

206 28 52 9 <1 10

Provide academic enrichment, extended learning time or 

other academic supports to students 210 29 49 11 2 9 

Connect parents/guardians to information and resources to 

help them support their children’s education 209 23 57 13 3 4 

Increase awareness about and access to health 

resources/services (e.g. targeting drug use, mental health, 

teen pregnancy)

208 20 47 19 6 8 

NOTES: Number of respondents who indicated the question was not applicable ranged from 7-22. (N=215, Missing=5-9)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Strongly Agree".

Exhibit reads: 28% of school staff strongly agreed and 48% agreed that they “Make appropriate support services available to students 

with special needs”. 

SOURCE: Instructional Staff and Counselors Survey Q15 (Please indicate your level of agreement/disagreement with the following 

statements about community involvement and partnerships with your school this school year (2014-15). (Mark one response in each 

row.) )

Exhibit A-52: School Staff Perceptions of School Challenges (2014-2015)

Topic

Total

n

Severe 

Challenge

(%)

Moderate 

Challenge

(%)

Not a 

Challenge

(%)

Student academic performance 212 50 45 5

Student attendance 212 33 57 11

Student depression or other mental health problems 212 22 56 22

Disruptive student behavior 213 45 48 7

Student engagement in school 213 38 53 9

Student aspirations for college and/or career 210 21 60 19

Student fatigue/lack of sleep 212 23 60 18

Student behavior and discipline 213 43 50 7

Lack of respect of staff by students 212 27 50 23

Student safety 210 6 48 46

Harassment or bullying among students 213 18 63 19

Student alcohol and drug use 211 11 41 49

Cutting classes or being truant 212 26 44 31

Theft 211 11 46 43

Physical fighting between students 212 19 48 33

Student tobacco use 211 8 32 60

Racial/ethnic conflict among students 213 4 30 67

Gang-related activity 213 6 32 62

Vandalism (including graffiti) 213 6 36 57

Weapons possession 210 4 19 78

NOTES: (N=215, Missing=2-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Severe Challenge".

Exhibit reads: 50% of school staff perceived “Student academic performance” as a severe challenge. 45% of school staff perceived it as 

a moderate challenge, and 5% did not perceive it as a challenge at all. 

SOURCE: Instructional Staff and Counselors Survey Q16 (Please indicate whether the following topics represent challenges in your 

school this school year (2014-15). (Mark one response in each row.))

nationalservice.gov 25
Exhibit A-53: School Staff Perceptions of Students, Teachers and Families (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Teachers and other school staff communicate and 

collaborate 213 13 68 16 3 

Students treat AmeriCorps members with respect † 153 10 68 18 4

Students treat teachers with respect 211 6 58 26 10

Students take their school work seriously 214 6 49 38 7

Students treat each other with respect 213 5 54 34 6

Families play an active role in our school 213 3 30 51 16

NOTES: (N=155-215, Missing=1-5)

† These items were restricted to respondents who worked with at least one student in the School Turnaround AmeriCorps program.

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school staff who selected “Strongly Agree”.

Exhibit reads: 13% of school staff strongly agreed and 68% agreed that “teachers and other school staff communicate and collaborate”. 

SOURCE: Instructional Staff and Counselors Survey Q17 (Please indicate the level of agreement/disagreement with the following 

statements about your school this school year (2014-15). (Mark one response in each row.))

Exhibit A-54: School Staff Perceptions of Value of School Turnaround AmeriCorps (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

AmeriCorps members provide helpful support to the students in this 

school † 150 31 63 4 2 

AmeriCorps members are important partners in improving student 

outcomes † 153 30 59 8 3 

AmeriCorps members offer supports that are beneficial to the teachers in 

this school † 150 27 58 11 4 

AmeriCorps activities occur frequently enough to be valuable † 152 26 49 20 5

NOTES: (N=155-215, Missing=1-5)

† These items were restricted to respondents who worked with at least one student in the School Turnaround AmeriCorps program. 

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school staff who selected “Strongly Agree”.

Exhibit reads: 31% of school staff strongly agreed and 63% agreed that “AmeriCorps members provide helpful support to the students 

in this school”.

SOURCE: Instructional Staff and Counselors Survey Q17 (Please indicate the level of agreement/disagreement with the following 

statements about your school this school year (2014-15). (Mark one response in each row.))

nationalservice.gov 26
Exhibit A-55: School Staff Satisfaction with Elements of School Turnaround AmeriCorps (2014-2015)

Element

Total

(n)

Very 

Satisfied

(%)

Satisfied

(%)

Dissatisfied

(%)

Very 

Dissatisfied

(%)

Don't 

Know

(%)

Not 

Applicable

(%)

Implementation of the roles and 

responsibilities outlined in the 

school partnership agreements

150 24 46 17 2 0 11

Communication and collaboration 

between teachers and School 

Turnaround AmeriCorps members

151 28 47 14 6 0 5 

Overall quality of School 

Turnaround AmeriCorps 

programming

152 30 46 16 3 0 6 

Matching of members to students 

in need of academic strengthening 

and social/emotional supports

150 25 49 15 3 0 7 

Referral of students to receive 

services offered by School 

Turnaround AmeriCorps members

152 23 52 15 2 0 8 

Placement of School Turnaround 

AmeriCorps members in 

meaningful service activities

153 30 48 11 6 0 6 

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Number of respondents who indicated the question was not applicable ranged from 7-17. (N=155, Missing=2-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Very Satisfied".

Exhibit reads: 24% of school staff are very satisfied and 46% are satisfied with the “Implementation of the roles and responsibilities 

outlined in the school partnership agreements”.

SOURCE: Instructional Staff and Counselors Survey Q18 (For this school year (2014-15), please indicate your level of 

satisfaction/dissatisfaction with each of the elements listed below. (Mark one response in each row.))

Exhibit A-56: School Staff Perceptions of Success of School Turnaround AmeriCorps (2014-2015)

Success Statement

Total

(n)

Very 

Successful

(%)

Somewhat 

Successful

(%)

Somewhat 

Unsuccessful

(%)

Very 

Unsuccessful

(%)

Not 

Applicable

(%)

Overall success 150 39 41 11 4 6

Success in improving the school’s 

capacity to implement its turnaround 

model

152 40 35 14 6 6 

Success in improving school climate 151 34 36 15 7 7

Success in improving student socioemotional health 151 30 42 14 6 8 

Success in improving student 

academic achievement 152 34 45 11 5 5 

NOTES: Responses limited to school staff who worked with at least one student in the School Turnaround AmeriCorps program.

Number of respondents who indicated the question was not applicable ranged from 6-11. (N=155, Missing=3-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of school leaders who selected "Very Successful".

Exhibit reads: 39% of school staff perceived School Turnaround AmeriCorps to be very successful and 41% perceived it to be 

somewhat successful overall. 11% perceived it to be somewhat unsuccessful and 4% very unsuccessful. 6% said the overall success of 

School Turnaround AmeriCorps was not applicable.

SOURCE: Instructional Staff and Counselors Survey Q19 (In your opinion, how successful is the School Turnaround AmeriCorps 

program in the following areas this school year (2014-15)? (Mark one response in each row.))

nationalservice.gov 27
Exhibit A-57. School Staff Perceptions of Changes in Behavior of Student "A" Served by School 

Behavior

Total

(n)

Significant 

Improvement

(%)

Moderate 

Improvement

(%)

No 

Change

(%)

Moderate 

Decline

(%)

Significant 

Decline

(%)

Did Not Need 

to Improve

(%)

Participating in class 93 28 50 17 1 0 5

Coming to school 

motivated to learn 92 24 49 23 0 0 3 

Getting along well with 

other students 93 22 49 21 1 0 7 

Volunteering (e.g., for 

extra credit or more 

responsibilities)

92 21 30 39 0 0 10

Being attentive in class 92 21 55 20 1 0 3

Attending class regularly 90 20 42 26 0 0 12

Completing homework to 

your satisfaction 92 20 47 28 0 1 4 

Turning in his/her 

homework on time 91 20 44 32 1 0 4 

NOTES: Each respondent was asked about two students, one whose last name is closest to the beginning of the alphabet, and another 

whose last name is closest to the end of the alphabet. (N=95, Missing=2-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of students for whom school staff who selected "Significant Improvement" 

with “Did Not Need to Improve” excluded from the denominator.

Exhibit reads: 28% of school staff perceived a significant improvement in class participation for the student whose last name is closest 

to the beginning of the alphabet. 50% perceived a moderate improvement, 17% perceived no change, 1% perceived a moderate 

decline, 0% perceived a significant decline, and 5% reported that the student did not need to improve.

SOURCE: Instructional Staff and Counselors Survey Q20 (Since beginning to work with a School Turnaround AmeriCorps member(s),

to the best of your knowledge, what extent has student with the last name closest to the beginning of the alphabet changed his or her 

behavior in terms of Turnaround AmeriCorps (2014-2015)

Exhibit A-58: School Staff Perceptions of Changes in Behavior of Student "Z" Served by School Turnaround 

AmeriCorps (2014-2015)

Behavior

Total

(n)

Significant 

Improvement

(%)

Moderate 

Improvement

(%)

No 

Change

(%)

Moderate 

Decline

(%)

Significant 

Decline

(%)

Did Not Need 

to Improve

(%)

Participating in class 91 26 48 21 0 0 5

Coming to school motivated to 

learn 92 24 48 21 1 1 4 

Getting along well with other 

students 91 20 52 20 0 0 9 

Volunteering (e.g., for extra 

credit or more responsibilities) 91 14 37 36 1 0 12

Being attentive in class 90 23 56 15 1 0 4

Attending class regularly 91 24 41 23 1 1 9

Completing homework to your 

satisfaction 92 19 44 31 0 0 6 

Turning in his/her homework 

on time 92 15 47 31 0 0 7 

NOTES: Each respondent was asked about two students, one whose last name is closest to the beginning of the alphabet, and another 

whose last name is closest to the end of the alphabet. (N=95, Missing=2-5)

Total n column describes the number of survey respondents who provided an answer in each row. The remaining columns present 

nonresponse-weighted percentages to approximate the population of interest (see Appendix A.2.1 for details). 

Table rows are sorted in descending order by the proportion of students for whom school staff who selected "Significant Improvement" 

with “Did Not Need to Improve” excluded from the denominator.

Exhibit reads: 26% of school staff perceived a significant improvement in class participation for the student whose last name is closest 

to the end of the alphabet. 48% perceived a moderate improvement, 21% perceived no change, 0% perceived a moderate decline, 0% 

perceived a significant decline, and 5% reported that the student did not need to improve.

SOURCE: Instructional Staff and Counselors Survey Q20 (Since beginning to work with a School Turnaround AmeriCorps member(s),

to the best of your knowledge, what extent has student with the last name closest to the end of the alphabet changed his or her 

behavior in terms of:)

nationalservice.gov 28
A.3.4 Parent Interviews

Exhibit A-59: Parent Perceptions of Students, Teachers and Families (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Don't Know

(%)

Teachers and leaders at my child’s school care 

about my child’s academic performance 50 50 40 6 2 2 

Teachers and leaders at my child’s school care 

about my child’s social and emotional well-being 50 42 50 4 2 2 

Families play an active role in our school 49 37 43 16 0 4

NOTES: (N=50, Missing=0-1)

Number of respondents who did not know ranged from 1 - 2.

Table rows are sorted in descending order by the proportion of parents who selected “Strongly Agree”.

Exhibit reads: 50% of parents strong agreed and 40% agreed that “Teachers and leaders at my child’s school care about my child’s 

academic performance”. 

SOURCE: School Turnaround AmeriCorps Parent Interviews Q10 (Now I will read several statements about your child’s school. For 

each statement, please tell me whether you strongly disagree, disagree, agree, strongly agree, or don’t know.)

Exhibit A-60: Parent Perceptions of Value of School Turnaround AmeriCorps (2014-2015)

Statement

Total

(n)

Strongly 

Agree

(%)

Agree

(%)

Disagree

(%)

Strongly 

Disagree

(%)

Don't 

Know

(%)

AmeriCorps members provide helpful support to 

the students in this school † 38 74 18 0 3 5 

AmeriCorps members are important partners in 

improving student outcomes † 38 58 34 0 3 5 

AmeriCorps activities occur frequently enough to 

be valuable † 38 47 42 0 3 8 

NOTES: Responses limited to those familiar with the School Turnaround AmeriCorps program (N=38, Missing=0).

Number of respondents who did not know ranged from 2 - 3.

Table rows are sorted in descending order by the proportion of parents who selected “Strongly Agree”.

Exhibit reads: 74% of parents strongly agreed and 18% agreed that “AmeriCorps members provide helpful support to the students in 

this school”. 

SOURCE: School Turnaround AmeriCorps Parent Interviews Q10 (Now I will read several statements about your child’s school. For

each statement, please tell me whether you strongly disagree, disagree, agree, strongly agree, or don’t know.)

Exhibit A-61: Parent Perceptions of Success of School Turnaround AmeriCorps (2014-2015)

Overall success of the School 

Turnaround AmeriCorps 

program in terms of...

Total

(n)

Very 

Successful

(%)

Somewhat 

Successful

(%)

Somewhat 

Unsuccessful

(%)

Very 

Unsuccessful

(%)

Don't 

Know

(%)

Overall success 38 66 24 0 0 11

Success in improving the school’s 

capacity to implement its turnaround 

model

38 45 34 0 0 21

Success in improving school climate 38 55 21 3 3 18

Success in improving student socioemotional health 38 61 18 5 0 16

Success in improving student 

academic achievement 38 66 29 0 0 5 

NOTES: Responses limited to those familiar with the School Turnaround AmeriCorps program (N=38, Missing=0).

Number of respondents who did not know ranged from 2 - 8.

Table rows are sorted in descending order by the proportion of school leaders who selected "Very Successful".

Exhibit reads: 66% of parents perceive School Turnaround AmeriCorps to be very successful overall.

SOURCE: School Turnaround AmeriCorps Parent Interviews Q11-Q15 (On a scale of 1-4, what is your perception of the School 

Turnaround AmeriCorps program's success in terms of ...)

nationalservice.gov 29
A.4 Supplemental Survey Exhibits

This section contains supplemental exhibits that combine information from multiple supplemental survey data tables or 

present table data in graphical format for ease of interpretation.

Exhibit A-62: Grantee and School Leader Knowledge of Students Served by School Turnaround AmeriCorps 

(2014-2015)

Knowledge Level

Grantees

(n)

School Leaders

(%)

Yes 11 74

Sometimes, but not always 2 26

NOTES: Grantee (N=13, Missing=0), School Leader (N=38, Missing=2)

Exhibit reads: Eleven of 13 grantees and 74% of school leaders reported that they knew which students were served by School 

Turnaround AmeriCorps members.

SOURCE: Grantee Survey Q3 (Do you know which students were served by School Turnaround AmeriCorps members?)

School Leader Survey Q7 (Do you know which students are served by School Turnaround AmeriCorps members this school year?)

Exhibit A-63: Target Number of Students Expected to be Served by School Turnaround AmeriCorps

Students 

expected to 

be served

(n)

Students expected 

to complete 

services

(n)

Complete/

Serve

(%)

Average number of 

AmeriCorps members 

per school

(n)

Caseload 

(Complete/ 

Members)

(%)

Mean 210 189 89 8 28

Standard deviation 150 170 30 5 20

Maximum 540 673 165 20 87

Minimum 46 38 45 1 8

Median 160 135.5 78 6 24.5

1st Quartile 102 60.75 74.75 4 16.25

3rd Quartile 230 226.75 100 10 34.25

NOTES: (N=13, Missing=0)

Exhibit reads: The mean number of students per school expected to be served across all 13 grantees is 210. The mean number of 

students per school expected to complete services across all 13 grantees is 189. The mean percentage of students expected to 

complete services out of students expected to be served is 89%. The mean number of AmeriCorps members per school is eight. The 

mean caseload (number of students per school expected to complete services divided by the number of AmeriCorps members) is 28. 

SOURCE: Grantee Survey Q5 (Please review the list below to confirm the schools to which your organization assigned School 

Turnaround AmeriCorps members. Fill in the number of members who serve at each school during 2014-15, and the targeted number 

of students that you expect to serve and complete the program this school year. If you don’t know, please write in “DK.”))

Exhibit A-64: Proportion of Staffs’ Students Served by School Turnaround AmeriCorps (among staff familiar with 

the program) (2014-2015)

nationalservice.gov 30
NOTES: See Appendix Exhibit A-41 for a tabular presentation of these data. 

Sixty respondents were not familiar with the School Turnaround AmeriCorps program.

Sixty-nine respondents did not know how many of their students were served by School Turnaround AmeriCorps. (N=86, Missing=0)

Exhibit reads: 35% of staff reported that less than 25% of their students were involved in School Turnaround AmeriCorps programming.

SOURCE: Instructional Staff and Counselors Survey Q6 (Approximately how many students with whom you have worked this school 

year (2014-15) are/were involved in School Turnaround AmeriCorps programming?)

Exhibit A-65: School Leaders’ Communications with and Monitoring of School Turnaround AmeriCorps Activities 

and Members (2014-2015) 

nationalservice.gov 31

NOTES: See Appendix Exhibit A-24 for a table with all response options.

95% confidence intervals (represented by black capped lines) may not be symmetric because the upper bound is limited to one minus 

the proportion of sample members who did not endorse the item and the lower bound is limited to the proportion of sample members 

who did endorse the item.

(N=38, Missing=0-1)

Bars are sorted in descending order by the proportion of school leaders who selected “Strongly Agree.”

Exhibit reads: An estimated 41% of all school leaders strongly agree and 47% agree that “Teachers in this school are supportive of the 

AmeriCorps program.” With 95% confidence, the proportion of all school leaders in School Turnaround AmeriCorps schools who 

strongly agree or agree that “Teachers in this school are supportive of the AmeriCorps program” falls between 79% and 93%.

SOURCE: School Leader Survey Q8 (Please indicate your level of agreement/disagreement with each of the elements listed below for 

this school year (2014-15): (Mark one response in each row))
Exhibit A-66: School Staff Communications and Collaboration with Members (2014-2015) 

nationalservice.gov 32

NOTES: See Appendix Exhibit A-49 for a table with all response options.

95% confidence intervals (represented by black capped lines) may not be symmetric because the upper bound is limited to 100 percent 

and the lower bound is limited to 0 percent.

Responses limited to respondents who worked with at least one student in the School Turnaround AmeriCorps program. Number of 

respondents who indicated the question was not applicable ranged from 5-10. (N=155, Missing=4-5)

Bars are sorted in descending order by the proportion of school leaders who selected “Strongly Agree.”

Exhibit reads: An estimated 46% of all school staff strongly agree and 46% agree that “Teachers in this school are supportive of the 

AmeriCorps program.” With 95% confidence, the proportion of all school staff in School Turnaround AmeriCorps schools who strongly 

agree or agree that “Teachers in this school are supportive of the AmeriCorps program” falls between 86% and 98%.

SOURCE: Instructional Staff and Counselors Survey Q13 (Please indicate your level of agreement/disagreement with the following 

statements about teacher relationships with School Turnaround AmeriCorps members this school year (2014-15). (Mark one response 

in each row.))
Exhibit A-67: Grantee and School Staff Perceptions of Areas of Improvement in Student Outcomes for Students 

Served by School Turnaround AmeriCorps (2014-2015)

nationalservice.gov 33

NOTES: See Appendix Exhibits A-12 and A-48 for a table with all response options.

Grantee (N=13, Missing=0)

Number of school staff who did not know whether students improved on an outcome ranged from 47-76. (N=79-99, Missing=2-5)

Ranks range from 1 - 9, with 1 being the most important. Not all responses were ranked by respondents. Means are calculated for 

respondents who ranked the option.

Table rows are sorted in ascending order by the mean rankings of grantees.

Exhibit reads: Grantees and school staff ranked "Enhanced academic achievement" as the student outcome with the greatest degree of 

improvement across schools, with a mean ranking of 2.4 and 2.6, respectively, on a 9-point scale. SOURCE: Grantee Survey Q11 

(Please fill in the following table about student outcomes. Was there improvement in this area for students in your schools served by 

School Turnaround AmeriCorps members this year? If you marked “Yes,” what were the outcomes with the greatest degree of 

improvement, across schools?)

Instructional Staff and Counselors Survey Q12 (For how many of the students served by School Turnaround AmeriCorps at your school 

this year (2014-2015) are there improvements in the following areas?)
A.5 Qualitative Methodology

The Year 1 evaluation encompasses qualitative data gathered from a large number and variety of stakeholders in order to 

triangulate findings based on multiple perspectives. A summary of data sources is below, followed by more detailed 

discussion of the considerations specific to each data source.

Exhibit A-68. Qualitative Data Sources By Stakeholder, Mode, and Timing of Collection

Stakeholder N Mode of Data Collection Timing of Data Collection

Grantee staff 13 Telephone interviews November 17-December 1, 2014

3 groups (11 participants) Online focus groups February 12-24, 2015

12 Telephone interviews June 2015

Principals 25 One-on-one telephone interviews December 2014—February 2015

3 groups (9 participants) Small group telephone interviews April 28 – May 5, 2015

12 Site visit interviews (4 in-person, 8 

telephone)

May 4 – May 19, 2015

36 Survey narrative responses May 4-28, 2015

Parents 50 Telephone interviews January 21-February 23, 2015

Members 3 groups (9 participants) Online focus groups March 5-21, 2015

26 One-on-one telephone interviews February 27-April 2, 2015

School staff 32 Case study interview (8 in- person, 24 

over telephone)

April 29-June 11, 2015

4 groups (14 participants) In-person focus groups May 4 – May 19, 2015

Grantee Progress 

Reports

12 mid-year

12 end-of-year plus 16 

supplements

Provided by CNCS N/A

Partnership 

Agreements

12 Provided by grantees N/A

A.5.1 Qualitative Data Collection

Most qualitative data for this project was collected through interviews (mostly by telephone) and focus groups (mostly 

online). The research team interviewed a knowledgeable representative from each grantee organization at two times—

first in fall 2014 for a “pre-interview,” then again in spring 2015 for a “post-interview.” Grantee staff members were 

included in focus groups if they were involved with and knowledgeable about the program.

The principals selected for the 25 one-on-one interviews were selected at random from the 57 schools in their second year 

of implementing the School Turnaround AmeriCorps program. The research team interviewed principals affiliated with 

between one and four schools per grantee for 12 of the 13 grantees. Interviews were conducted by telephone between 

December 23, 2014 and February 19, 2015 using a standard protocol, then transcribed verbatim. Principal surveys asked 

closed-ended questions about their individual schools, about the school’s overall turnaround plan, components of and 

support for the AmeriCorps program, improvements and challenges in the school, and the general school environment; it 

included one open-ended narrative question about individual schools’ turnaround plan activities.8, 9 Principals 

interviewed in small group telephone interviews were selected at random from those principals who had not participated 

in one-on-one interviews. The research team initially tried to match principals whose schools offered similar 

interventions; however, this proved prohibitive because of scheduling. A condition of participating in the case studies (inperson or by telephone) was that the principal or another school leader (e.g., assistant or vice principal) with knowledge 

of the program would be interviewed. 

Eight of the 13 School Turnaround AmeriCorps programs provide family and community engagement services, and the 

50 parents selected for interviews were associated with such programs. Parents were recruited to participate in interviews 

through convenience sampling: the research team asked program staff and, as needed, school leaders to provide contact 

information for parents who might be willing to be interviewed for the School Turnaround AmeriCorps evaluation. Those 

parents who agreed to be interviewed are potentially already among the most engaged parents in the school. Further, it is 

possible that parents with a positive view of the program were more likely to be referred for interviews and to complete 

8 The survey item reads as follows: “Briefly describe the activities in your school’s turnaround plan.”

9 Nineteen principals both responded to the survey and participated in interviews.

nationalservice.gov 34

 
them. It is not clear how representative—or not—the feedback was from parents interviewed for this evaluation relative 

to feedback from all parents whose children are involved in the program.

Forty-two of 50 parent interviews were conducted in English and eight in Spanish. Thirty-six parents knew something 

about the School Turnaround AmeriCorps program and provided open-ended answers about the program itself. It is 

important to note that three interviewed parents were also teachers in the local district, and four parents were members of 

their schools’ Parent Teacher Organization (PTO); all seven of these individuals knew about the program. The remaining 

14 parents answered only scalar questions about the overall school climate; the 36 parents who knew details about the 

program also answered these questions, and these 50 responses are integrated in the discussion of the survey findings in 

the main body of the report. 

Members recruited for interviews and focus groups were drawn at random from programs’ rosters. After several 

unsuccessful attempts to contact a member, she or he was replaced with another member drawn at random from the 

program’s cohort. Overall, the sample sizes for the member focus groups are small, but the sample size alone is not cause 

for concern because of the random selection. While it is possible that there are systematic differences between those 

members who did and did not elect to participate in focus groups, the study team does not have evidence to substantiate 

that. 

School staff members (teachers and guidance counselors) interviewed for case studies were suggested by their principals 

as staff who had sufficient knowledge to discuss the school’s turnaround activities and partners and School Turnaround 

AmeriCorps program, as applicable. 

For all interviews, the interviewing team was trained in the OMB-approved grantee interview protocol prior to 

conducting the interviews. The interviews were recorded and transcribed, with all personally identifying information 

(PII) expunged from the transcripts. Interviews followed a standard interview protocol; however, not all interviewees 

provided similar information or the same level of detail to each question posed. Interviews rely on interview participants’ 

recall, impressions, and details shared during the interview. Because a given respondent shared more information on one 

topic during an interview does not mean that others who provided less detail did not necessarily have relevant 

experiences with that topic. For example, one principal elaborated in an interview about how individual members’ 

strengths influence the services offered each year, because fulfilling members’ own educational goals represented an 

important part of the school’s model for its School Turnaround AmeriCorps program. While other schools may have also 

tailored services based on individual members’ personal strengths, other principals did not recall that aspect of program 

delivery or were not compelled to mention it in the interview. As a result, the frequencies reported in this document 

represent lower bounds—when a frequency is reported, at least that many respondents shared the experience, but it does 

not mean that all other respondents disagreed. 

Focus group moderators and note-takers were trained on the OMB-approved discussion guide used for member and 

grantee focus groups; the same discussion guide was used for each group to facilitate the comparisons of member and 

grantee staff perspectives. Online focus groups were conducted using the iTracks platform for video, audio, and chatenabled online focus groups. Each focus group was video-recorded. A note-taker from the research team took real time 

notes on participants’ words, body language, and group dynamics; notes were supplemented as needed with a later 

review of the video. Online focus groups were conducted with grantee staff from February 12 to 24, 2015 and with 

members from March 5 to 21, 2015. 

The research team encountered a few challenges with administration of the online focus groups. Some participants 

experienced technological challenges that inhibited their participation. For example, slow internet connections meant that 

in at least one focus group, a participant lost audio and video connections but maintained chat connection to the group; in 

another group, a participant was lost entirely due to connection problems. Members experienced technological problems 

more often, especially if they only had access to their smartphones for Internet connectivity. Lastly, the team also 

experienced challenges with obtaining member participation because of difficulty in scheduling focus group sessions at 

times when members could attend due to their varying schedules as well as time zone differences. Communicating with 

members was difficult due to their variable schedules and varying ability to take a personal phone call at work, as well as 

their not having access to or checking the email addresses issued by the programs. Members also were not incentivized to 

participate, which might have helped with the participation rate in the focus groups. 

nationalservice.gov 35
The research team received grantee end-of-year and mid-year progress reports from CNCS. Grantees provided written 

partnership agreements and responded to follow-up questions via email about how they use the agreements. 

A.5.2 Qualitative Analysis 

The study team developed a codebook documenting the source documents and the relevant open-ended questions and 

narrative responses to be analyzed from each source, and created a hierarchical coding structure of topics and subtopics, 

or content categories to be coded. Topics were associated with each of the study’s research questions and further 

developed through an iterative coding process as new data sources were incorporated. For example, based on grantee 

pre-interviews and GPRs, high level topic categories were identified that consisted of activities and interventions (such as 

“direct services”), administration and oversight, barriers and challenges, program changes, context, and members and 

staffing. Examples of second level topics are “Tutoring” under “Direct Services” and “Retention” under 

“Members/Staffing.” 

The codebook was developed iteratively as new data sources became available and their information incorporated into 

the coding structure. As this happened, new codes were added based on findings in the prior rounds that, for example, 

more precisely identified the activities members lead, the structure of activities (e.g., one-on-one, small group), and the 

training and professional development activities in which members participate. Additionally, some codes were redefined 

to accommodate changes arising out of stakeholder perspectives. For example the node “school-level intervention” was 

redefined after principal interviews to code a school’s turnaround plan as a whole instead of to program activities 

targeting the whole school (which were found to be few).

The analysis team was trained in the codes and their definitions and ensured quality by performing peer coding of 

samples of the dataset throughout the coding process. Training and peer coding help to ensure that team members 

interpret the meaning of the text in the same way, similarly define the unit of meaning when coding and categorizing a 

piece of text, and consistently apply the appropriate codes to the data, thereby increasing inter-rater reliability and 

reducing sources of bias introduced to the study. The research team then coded all narrative text from these sources 

following the coding structure. 

Once the data were coded, the team identified the most prominent themes by counting the number of respondents that 

made a reference to the topic or subtopic. In calculating prevalence for each topic, respondents were only counted once 

even if they had provided multiple comments on the topic. During the analysis phase, the team identified sub-themes for 

many issues, including for example the specific components of school turnaround plans and a higher degree of precision 

in describing members’ activities during the school day. 

The team then used NVivo query functions to identify patterns in the data. Each member of the analysis team also wrote 

detailed memos linking coded materials to query output and to draft text describing patterns they observed in coded and 

queried data. This memo-writing process provides a clear record of the analytic process and a link between raw data and 

summarized findings. 

Partnership agreements and surveys were also subjected to thematic analysis and included mixed-methods analysis of 

these sources’ attributes. For partnership agreements, the team first analyzed the composition of the agreements to 

determine if certain features were present or not (e.g., evidence of plans to manage collaboratively). The team also posed 

three follow-up questions to program staff to help describe how they use and how frequently they update partnership 

agreements—issues that are not evident from a content analysis of the documents themselves. The information on both 

document content and usage were imported to NVivo as a survey so that closed-ended answers could be used to create 

attributes for documents and open-ended answers analyzed by theme. Similarly, the survey data were uploaded so that 

narrative descriptions of programs were coded and closed-ended questions used to segment the data. This strategy 

allowed the team to run queries that identify, for example, if schools following a turnaround model provide different 

services than those following a transformation model. 

A.6 Data Collection Instruments

The Year 1 data collection instruments will be delivered as a separate document from the final report. CNCS will not post 

the document publicly and therefore it does not need to be made 508 compliant. CNCS will make it available upon 

request.

nationalservice.gov 36
B. REFERENCES 1. Bagnell Stuart, J., Gamse, B., Jefferson, A., Nisar, H., Freeman, B., Burke, S., Personette, M. (not published). School 

Turnaround AmeriCorps National Evaluation: Interim Report. Corporation for National and Community Service.

2. Center on Innovation & Improvement and Council of Chief State School Officers. School Improvement Grant (SIG) 

Intervention Models (March 2010). Webinar prepared for National Network of State School Improvement Leaders. 

www.centerii.org/webinars/resources/Turnaround3.pptx. 

3. Corporation for National and Community Service (2013). Announcement of Federal Funding Opportunity, School 

Turnaround AmeriCorps FY13. 

http://www.nationalservice.gov/sites/default/files/nofa/schoolturnaround_ac_notice.pdf

4. Corporation for National and Community Service (2014). Draft announcement of Federal Funding Opportunity, 

School Turnaround AmeriCorps FY14. 

5. Corporation for National and Community Service (2013). Notice of Federal Funding Opportunity Addendum: 

Definitions, Suggestions regarding Data Collection, and Additional Notes.

6. Lohr, S. L. (1999). Sampling: Design and Analysis. New York: Cengage Learning.

7. Office of Management and Budget (2006). Standards and Guidelines for Statistical Surveys. 

https://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf

8. U.S. Department of Education (2014). School Improvement Grants. http://www2.ed.gov/programs/sif/index.html

9. U.S. Department of Education Office of Elementary and Secondary Education. Guidance on Fiscal Year 2010 School 

Improvement Grants under Section 1003(G) of the Elementary and Secondary Education Act of 1965. March 1, 2012.

10. U.S. Department of Education, National Center for Education Statistics (2013). Digest of Education Statistics, 2012

(NCES 2014-015), Chapter 2. http://nces.ed.gov/fastfacts/display.asp?id=64. 

11. Valliant, R., Dever, J. A. & Kreuter, F. (2013). Practical tools for designing and weighting survey samples, pp. 418-422.

nationalservice.gov 37