POLICY

STUDIES

ASSOCIATES, INC.

POLICY

STUDIES

ASSOCIATES, INC.

POLICY STUDIES ASSOCIATES, INC.

YEAR 4 EVALUATION OF CITY YEAR NEW YORK’S 

WHOLE SCHOOL WHOLE CHILD INITIATIVE 

Monica Mielke 

Alisha Butler 

Andrea Palmiter 

Leslie M. Anderson 

April 11, 2014 

Policy Studies Associates, Inc. 

1718 Connecticut Avenue, NW 

Suite 400 

Washington, DC 20009 

www.policystudies.com

Contents 

Page 

I. Introduction ................................................................................................................................. 1

Summary of Previous Evaluation Findings ........................................................................ 2

Year 4 Evaluation Design ................................................................................................... 3

II. School Partnerships .................................................................................................................... 7

III. Corps Member Team Quality ................................................................................................. 18

Training ............................................................................................................................. 18

Team Leadership and Support .......................................................................................... 26

IV. Implementation of Literacy Interventions .............................................................................. 31

Student Selection .............................................................................................................. 31

Literacy Tutoring Models and Dosage ............................................................................. 32

Content of Literacy Interventions ..................................................................................... 34

Monitoring Student Progress ............................................................................................ 41

V. City Year Outcomes ................................................................................................................. 45

Benefits for Schools .......................................................................................................... 45

Benefits for Students ......................................................................................................... 49 

City Year Impacts on Student Achievement in Reading/Language Arts ......................... 55 

VI. Conclusions............................................................................................................................. 63

Summary of Findings ........................................................................................................ 63

Recommendations ............................................................................................................. 67

Appendix: Implementation Indices ............................................................................................... 68
1

I. Introduction 

City Year is an education-focused nonprofit organization that partners with public schools 

to help keep students in school and on track to graduate. Founded in Boston in 1988, City Year 

works in 24 cities across the United States and has international affiliates in London and 

Johannesburg, South Africa. City Year corps members are 17 to 24-year-olds who commit to one 

year of full-time service in elementary or middle schools. Working on 10- to 12-person schoolbased teams, City Year corps members provide a variety of services, including literacy tutoring 

for targeted students and in-class support for teachers. A team leader manages the team’s day to 

day activities and a program manager supports the team leaders and communicates with school 

staff regarding school needs and City Year services. City Year developed the Whole School 

Whole Child (WSWC) model to address the central factors—known as the “ABC’s”—affecting 

student academic success: attendance, behavior, and course performance. The strategies designed 

to monitor and address each of the ABCs include the following: 

■ Attendance. Corps members conduct activities throughout their partner schools 

to support increased attendance both across the school and for small groups of 

students identified for focus lists. Corps members use check-in phone calls when 

focus list students are absent and have follow-up conversations when students 

return to school. Corps members also use incentives, including attendance 

certificates and access to social events, to encourage students to attend school 

regularly. 

■ Behavior. City Year corps members serve as near-peer role models (i.e., younger 

than teachers, but older than students) and provide behavior coaching to targeted 

students. All nine visited sites used the 50 Acts of Greatness (elementary school) 

or the 50 Acts of Leadership (middle school) curricula to guide behavior 

coaching. At both school levels, corps members met with small groups of 

targeted students weekly to discuss acts of “greatness” or “leadership” in their 

daily lives. In addition to the support provided to focus list students, corps 

members might also emphasize behavior management during in-class support, 

one-on-one tutoring, and in other areas of service. 

■ Course performance. Through one-on-one and small group tutoring as well as 

through in-class support, corps members aim to improve student performance in 

literacy and math. 

 City Year seeks to customize the specific types and intensity of supports corps members 

provide to meet the needs of students served, using a data-informed approach and building a 

student mindset and skills for school achievement and civic participation. Although the work of 

each City Year team varies somewhat from school to school, all WSWC City Year teams share 

the following three central goals: (1) improve the school climate, (2) improve student literacy, 

and (3) foster positive youth development and civic engagement among students. Within each of 

those broad goals, City Year teams focus on achieving specific outcomes over time, including 

the following: 
2

Short-term outcomes 

■ Students show improved performance in English/Language Arts and literacy 

■ Students show improved school attendance 

■ Students show improved behavior (decrease negative behaviors, increase positive 

behaviors) 

■ Students show improved performance in math 

Intermediate outcomes 

■ Students have a growth-mindset; they feel capable and committed to their learning 

and effort

■ Students have an improved connection to school 

■ Students are community-minded 

Longer-term outcomes 

■ Students graduate from high school 

■ Students are college ready and job competitive 

■ Students have a strong civic identity 

 In New York City, WSWC operations began in the 2009-10 school year. During the 

2012-13 school year (the fourth year of WSWC operations), City Year New York (CYNY) 

deployed teams of corps members in 19 schools—12 elementary schools and 7 middle schools—

throughout the city to implement the model. This evaluation report focuses on the 

implementation, outcomes, and impacts of the WSWC literacy intervention in 2012-13.

Summary of Previous Evaluation Findings 

 Policy Studies Associates, Inc. (PSA) has evaluated the WSWC initiative in each year of 

its implementation by CYNY. During the 2011-12 academic year, PSA conducted a study 

intended to yield a description of the implementation and outcomes of the literacy component of 

the WSWC model. The Year 3 evaluation relied on corps member, teacher, and principal survey 

data collected by City Year New York as well as a student survey designed by City Year but 

modified by PSA. Evaluators also analyzed student QRI-5 scores to assess improvement in 

literacy skills.1

 In addition to survey data, evaluators conducted site visits to three of the six 

middle schools implementing the WSWC model. Site visitors observed program activities and 

conducted interviews with corps members, school principals, teachers, and participating students. 

The Year 2 (2010-11) and Year 3 (2011-12) evaluations found: 

■ Implementation of City Year’s Whole School Whole Child model in both Year 2 

and Year 3 varied by school and by classroom with respect to curricula, structure, 

and implementation. 

 

1

 The QRI-5 is an assessment of students’ reading abilities and uses word lists and passages of narrative and 

expository text as the basis for determining a student’s “independent” reading level, “instructional” level, “and 

“frustration” level with certain types of text. 
3

■ Reports of the training and preparation of corps members varied by school in both 

Year 2 and Year 3. Few corps members felt that the trainings that the trainings 

they received were “very helpful” for planning and carrying out service in their 

school. Most corps members did report that they felt very prepared to provide 

whole-classroom academic support and homework assistance. 

■ Corps member survey results in both years suggested that team leadership could 

be improved. In Year 3, 58 percent of corps members agreed or strongly agreed 

that their program managers were very effective as leaders; the same was true of 

62 percent of corps members in Year 2. To improve team quality, in Year 3 

evaluators recommended a review of the program manager role, including 

increasing the coaching offered to corps members by program managers. 

■ In Year 2, analysis of literacy tutoring participants’ state test scores found that 

participating in literacy tutoring did not have a significant effect on students’ test 

scores. Regression models were created using 107 of the literacy participants in 

the WSWC schools and 210 matched comparison students, and analyses showed 

that participation in literacy tutoring did not significantly predict students’ ELA 

test scores after one year of tutoring. 

■ In both Year 2 and Year 3, the vast majority of students who responded to the 

survey were enthusiastic about the involvement of City Year corps members in 

their schools. In addition, the majority of teachers in each year believed that 

corps members had positive impacts on their students – both on their 

English/language arts performance and on their attitudes towards learning – and 

on their classroom environment. 

Year 4 Evaluation Design 

The Year 4 evaluation built on PSA’s prior work evaluating the WSWC model in New 

York City and aimed to assess the implementation, outcomes, and impact of City Year’s literacy 

component of the WSWC model on student performance. As in past years, the evaluation aimed 

to measure school-level variation in model implementation using corps member, teacher, and 

principal survey data collected by City Year. The evaluation team also conducted site visits to 

half of the elementary and middle schools that implemented the model during the 2012-13 school 

year (six elementary schools and three middle schools). In addition, and to measure City Year’s 

impact on student performance, the evaluation included a comparison of participating students’ 

performance on the New York City Department of Education (DOE) tests of English language 

arts (ELA) to that of a matched comparison group of students from similar schools that did not 

implement the WSWC model. 

Through analysis of survey data, site visit interview data, and ELA achievement data, the 

Year 4 evaluation addressed the following research questions related to implementation, 

outcomes, and impacts of the literacy tutoring component of the WSWC model: 
4

Implementation of literacy tutoring: 

■ To what extent is the literacy component of the WSWC model being implemented 

as intended? What are the common program features among the 12 participating 

elementary schools and 7 middle schools? To what extent does WSWC 

implementation vary across the 19 CYNY sites (e.g., types of school-wide 

activities/programming; frequency and duration of corps member visits to 

participating schools; literacy tutoring strategies employed, etc.)? How does 

implementation vary by school level? How does WSWC model implementation 

in 2012-13 compare with model implementation in previous years? 

■ Overall, what are the successes and challenges that City Year teams experience 

when implementing the literacy tutoring component of the WSWC model? What 

are the perceptions of school partners (i.e., teachers and principals) regarding the 

quality and effectiveness of literacy tutoring provided to students? Do different 

stakeholder groups (e.g., principals, team leaders, corps members, teachers, 

students) experience different successes and challenges? 

Student outcomes and impacts of literacy tutoring: 

■ To what extent do the literacy outcomes across the 19 participating elementary 

and middle schools vary by students’ exposure levels (i.e., frequency and duration 

of tutoring received)? To what extent do literacy outcomes vary across the 19 

sites by corps member team quality, training, efficacy, and level of satisfaction? 

■ How do students’ literacy skills compare to those of students in un-served 

conditions? That is, how does the performance of City Year literacy tutoring 

participants on the DOE test of ELA compare with that of a matched comparison 

group of students who attend schools that are similar in terms of their learning 

environments2but do not participate in the CYNY program and have not 

implemented the WSWC model? 

Measurement of Program Implementation 

 As in the 2010-11 and 2011-12 evaluation, the Year 4 evaluation assessed program 

implementation using surveys of corps members, teachers, and principals collected by City Year 

New York. School-level implementation indices calculated in previous evaluation reports were 

updated and revised, and details regarding these indices can be found in Appendix A. In addition 

to evaluating implementation in the 19 City Year schools – 12 elementary and seven middle 

schools – using survey data, the evaluation team conducted site visits to six elementary and three 

middle schools implementing the WSWC model. During site visits, team members conducted 

interviews with corps members, teachers, principals/liaisons, and participating students. 

 

2

 As measured by the NYC DOE 2011 Learning Environment Survey. 
5

Measurement of Student-Level Outcomes 

The evaluation analyzed teacher and corps member survey data to determine whether 

teachers and corps members believed that the literacy tutoring program helped improve student 

attitudes toward and engagement in learning. In addition, to determine whether there were 

desired outcomes among students receiving literacy tutoring services that can be attributed to 

participation in the CYNY literacy tutoring program, the evaluation employed a quasiexperimental design that compared the average performance of CYNY WSWC literacy tutoring 

participants on the New York State English Language Arts (ELA) test with the average 

performance of a comparison group of non-participants who share similar demographic and prior 

performance characteristics and who attend similar schools that do not implement the WSWC 

model. 

Measurement of City Year Impacts on Student Performance 

To measure the impact of City Year’s WSWC model on participating students, the 

evaluation first identified a matched comparison group of students who attended similar schools 

but did not participate in City Year services and then measured the impacts of City Year literacy 

tutoring—or any City Year services—on student performance on the ELA state test compared 

with the comparison group of non-participants. We employed hierarchical linear regression 

modeling (HLM) to differentiate among the various levels of influence (e.g., student and school) 

on student performance and to account for the wide variability of student and school 

characteristics at each level that affect student performance. In addition, because of the 

differences in program implementation in elementary and middle schools as identified in site 

visit interviews, and the differences in the number of hours of student participation across school 

levels, PSA conducted separate analyses of the impact of participation in City Year programming 

by school level. 

Data Collection 

 City Year administered mid-year and end-of-year surveys to teachers and 

principals/school liaisons in each of the 19 WSWC schools included in the study. The surveys, 

designed and administered by City Year headquarters, were sent by e-mail to 172 teachers and 

46 principals/liaisons. Surveys were completed by 123 teachers and 41 principals/liaisons, for 

response rates of 72 and 89 percent, respectively. In addition, all 221 elementary and middle 

school corps members completed the start-of-year, mid-year, and end-of-year surveys. 

 The evaluation team visited six elementary schools and three middle schools participating 

in the WSWC initiative in May of 2013. During these one-day visits, PSA staff conducted 

interviews with school administrators, teachers, team leaders, corps members, and small groups 

of students at each of the schools. Additionally, in April 2013, the evaluation team conducted 

telephone interviews with the program managers who oversaw the teams at each of the nine 

schools included in the site visit sample. 
6

Analysis 

 Analyses comprised both quantitative and qualitative data collected for this study. 

Quantitative analysis included descriptive analysis of survey data, construction of scales, and 

exploratory cross-tabulations. 

Scale construction. Teacher, principal, and corps member survey data were used to 

create scales or indices that measured each of the following: (1) corps member/teacher 

partnerships, as rated by teachers; (2) teacher involvement in City Year activities; (3) corps 

member preparation and training to carry out work in schools; (4) the quality of corps member 

teams in terms of their leadership strength and cohesion; (5) corps member satisfaction; (6) the 

level of coordination between City Year and partner schools; and (7) teacher and principal 

satisfaction with corps member service. After identifying survey items that were theoretically 

associated with the construct to be measured, evaluators analyzed them for possible inclusion in 

an index. To the extent that responses varied, selected items were included in the index and then 

added together to create each index score. As part of the focus on the literacy component of 

WSWC, the corps member indices included only those corps members who provided literacy 

tutoring, either during one-on-one tutoring or in class. (See the Appendix A for further details 

regarding the definition of each of these indices.) 

Descriptive and explanatory analyses. Data in this report include descriptive statistics 

from surveys and City Year’s participation records. Evaluators also examined the relationships 

among the indices and school-level variables such as the percent of students in the school who 

received literacy tutoring, the primary intervention setting (e.g., push-ins, pull-outs), and the 

average number of literacy intervention hours that corps members delivered at each school. 

Comparisons between elementary and middle school were also conducted to detect potential 

differences in model implementation or City Year’s work in schools. Evaluators examined the 

relationships among variables using correlation analysis or independent samples t-tests for 

continuous variables, and chi-square tests for categorical variables. 
7

II. School Partnerships 

The success of City Year’s work depends upon a number of factors, including building 

strong partnerships with host schools. This chapter describes the strategies that City Year teams 

used to develop their partnerships with schools. It also examines the supports that corps members 

received from schools over the course of the year, the extent to which teachers were involved in 

City Year activities in their schools, and principal perspectives on their school’s partnership with 

City Year. 

Setting Service Expectations 

Setting clear expectations for the service initiative is the first step toward successful 

partnerships with schools, according to City Year program managers, team leaders, corps 

members, school principals, and teachers who participated in site visit interviews. For some 

City Year teams that participated in site visit interviews, setting expectations began with the 

program managers meeting with school administrators to discuss the “big picture” for service, 

including how corps members would deliver services. Program managers described their early 

conversations with school administrators as having helped them define the goals for corps 

member service, the number of service hours, the method of literacy intervention (e.g., pull-outs 

versus push-in), and which students would receive targeted literacy support. For some corps 

member teams, establishing clear expectations for corps member service supported the teams’ 

ability to deliver the service. For example, when asked whether the team had been able to 

provide the required dosage hours for literacy tutoring, one program manager said, “[It’s] less 

hard this year because we [built the interventions] into the [school] schedule. [The students 

targeted for services] have a City Year period during the school day.”

In addition to the “big picture” conversations between City Year program managers and 

school administrators, City Year team leaders and corps members helped set expectations among 

school staff by meeting with teachers and other school staff to describe City Year, its mission, 

and the WSWC model. At mid-year, the majority of surveyed principals (94 percent) reported that 

City Year staff conducted an orientation for key school stakeholders to explain its organization and 

service model; 82 percent of surveyed teachers reported that they attended an introductory event at 

the start of the year. In their respective Mid-Year Surveys, teachers and principals were asked to 

report the extent to which they felt well-informed about City Year’s mission and goals. The 

overwhelming majority of principals (94 percent) agreed or agreed strongly that they were wellinformed, as did 84 percent of surveyed teachers. When asked whether they were knowledgeable 

about City Year’s WSWC model and program activities, 94 percent of principals/liaisons agreed or 

agreed strongly that they were knowledgeable about City Year’s model and program activities. 

Seventy-one percent of teachers agreed or agreed strongly that they were familiar with City Year’s 

approach to instructional support.

Interview data highlight differences in WSWC implementation in schools where teachers 

and administrators were familiar with City Year’s model and supportive of its objectives and in 

schools where they were not. In one school, for example, school administrators were supportive 

of City Year’s objectives, and the school principal and the City Year liaison, together with 
8

teachers, defined corps members’ roles and set expectations for corps member performance in 

classrooms. The liaison said, “[Administrators, teachers, and corps members] talked about the 

mission of City Year as well as what [school staff] want, what our expectations are in working 

together, and then [corps members and teachers] spent some time to just talk.” Interviews with 

school staff and corps members suggested that this process helped corps members work more 

effectively with teachers, helped corps members be viewed positively in the school, and helped 

corps members’ secure time for student literacy and behavioral interventions. 

One program manager suggested, however, that City Year New York’s transition from 

physical service to academic service had not been clearly communicated to its partner schools. 

In one of City Year’s longstanding partner schools, the program manager described tensions 

between the school’s expectations and City Year’s new focus: “It’s harder to transition to ‘City 

Year 2.0’ when we’ve been in a school for a long time and [staff] have expectations. We can 

work with your school and [continue to provide the traditional City Year services and events], 

but not at the expense of working directly with students.” Several corps members who 

participated in site visit interviews also mentioned the negative consequences of the lack of 

communication with some partner schools about City Year’s service transition. Corps members 

in one school, for example, said they were frustrated that school staff asked them to carry boxes 

and do other menial tasks. At a second school, corps members were asked to perform duties 

outside of their City Year agreement, such as supervising lunch, which reduced the time they 

could dedicate to their student focus lists. A corps member at a third site said, “The school has 

such different expectations of us, and I think that is really due to the fact that nobody has 

communicated to the school how City Year has changed. We’re not just there to be extra bodies 

in the school. We have real goals that we’re aiming for.” 

Determining Corps Member Placements 

Strategic corps member placement, both in terms of the grade levels and the teachers 

they serve, can contribute to the quality and strength of the relationship forged between City 

Year New York and its partner schools. Corps member placement varied across the schools 

visited, with some schools assigning corps members to work with specific grades and others 

assigning corps members based on which students would receive services. At one of the schools 

where the corps member team wanted to serve third- through fifth-graders, the school liaison 

said, “We wish that we could have City Year in every class but if we had to choose, I think [they 

should work in older grades] because the need is there.” At one school, corps members were 

assigned to classrooms based on the number of eligible focus list students; the team leader 

explained, “I know there was one teacher who asked [for a corps member].... It was contingent 

on if [teachers had] a lot of students who are eligible for our focus list.”

Once school administrators and City Year team leaders determine the grade levels or 

students who will receive service, corps member teams might employ a matching process to 

match corps members and teachers whose work styles and backgrounds are similar. As 

suggested by interviews with corps members at one school, matching corps members with 

teachers can positively affect service. Corps members interviewed described their in-class 

support assignment as “a match made in heaven” and felt that they could effectively 
9

communicate with their teachers, especially through conversations during class. At this 

particular school, City Year asked teachers to invite corps members to their classrooms; from 

there, small groups of corps members observed these teachers to gain an understanding of 

teachers’ instructional styles and classroom structures. Corps members then wrote and submitted 

summaries of their classroom observations and teacher evaluations. Based on corps members’ 

reactions and teacher profiles, the program manager and the team leader worked together to 

match corps members with teachers, based on their preferences for and fit with a teacher. 

Most teachers surveyed at mid-year (75 percent) reported that there was a corps member 

matching process in place at their schools. However, interviews with corps members suggested 

that there was variation in how the matching process was defined and carried out in schools. 

Although interviewed team leaders and corps members often described the matching process as 

corps members observing teachers and being assigned based on their preferences, as described 

above, not all teams were able to carry out this process as planned. Two schools, for example, 

both encountered challenges to matching corps members with teachers. At one school, teachers 

worried that the matching process would be disruptive for students and would delay corps 

members’ work in their classrooms. At another school, corps members observed teachers but 

were ultimately assigned to teachers other than those they observed. 

Supporting City Year Teams 

 The City Year Statement of Partnership defines the roles and responsibilities of City Year 

staff and school personnel in support of the service initiative; the statement includes an 

agreement to implement the WSWC model, describes when services are to occur, negotiates 

access to student data, and outlines the intended outcomes of the partnership (e.g., the percentage 

of students receiving services who will improve in WSWC areas). As an agreement between 

City Year and the school, the statement of partnership is signed by the school liaison, principal, 

district leader, and City Year leadership. For example, schools are expected to include City Year 

leadership in school committee and staff meetings, and work with City Year staff to assess 

progress toward achieving the partnership goals and objectives. In addition, partner schools are 

expected to provide an orientation to the City Year team to introduce corps members to the 

school’s structure, culture, and operating environment. Partner schools are also expected to 

include City Year team leaders and corps members in school trainings and professional 

development activities that support the development of corps members’ instructional skills.

Corps member survey data suggested that corps members infrequently participated in 

school-based activities. For example, 28 percent of middle school corps members and 34 percent 

of elementary school corps members had attended training or professional development sessions 

offered by the partner school or district two or more times during the year. Principals, however, 

reported more frequent opportunities for corps members to attend professional development 

activities; by mid-year, 85 percent of middle school principals and 60 percent of elementary 

school principals said that their school had offered trainings to corps members or invited them to 

participate in staff professional activities once a month or more. 
10

Site visit interviews suggest that partner schools were not integrating City Year team 

leaders and corps members as closely into school operations as was suggested by the Statement 

of Partnership. One program manager explained that the City Year corps member team would 

like to become more integrated into school operations, especially as it related to the decisionmaking process, but that school leadership was not supporting it: “[P]art of the challenge of this 

partnership has been getting the administration to include us [in school-level decision making]. 

We’re not integrated into the community. The level of investment at the top isn’t as high [high 

as we’d like it to be].” Corps members and team leaders who participated in site visit interviews 

described how breakdowns in communication affected their overall relationship with the schools. 

For example, although corps members at one school felt supported by the individual teachers 

with whom they worked, the larger school community did not understand why they were there; 

in some cases, the corps member team felt blamed for things that went wrong in the school and 

had had teachers bar them from using their classrooms during afterschool hours. At another 

school, a similar communication breakdown negatively affected corps members’ ability to 

achieve the required number of student intervention hours. The program manager said that there 

was a disconnect between the message the corps member team received from the administrators 

and what the teachers allowed. Taking students out of class, for example, was approved by the 

administration, but that message was not transmitted to or enforced with the teachers. 

 Corps members participating in site visit interviews also described challenges related to 

school policies and teachers’ schedules limiting the time they had available to work with 

students. During the test preparation window, for example, corps members at one school could 

not work with their focus list students. At a second school, the program manager described 

challenges resulting from teachers not following the established class schedule: “[A] teacher is 

supposed to allow the corps member to take the student out, but they do something completely 

different where the student can’t leave.... I have some corps members who are afraid of the 

teacher and don’t want to interrupt the teacher’s flow, so they just won’t pull their students out.”

Building Corps Member-Teacher Partnerships 

Strong partnerships and effective communication between corps members and teachers 

have the potential to contribute to the overall quality of City Year services by helping corps 

members gain insight into student needs, student progress, and effective instruction. At midyear, teachers were asked to report whether they met with their corps members at the start of the 

year to discuss expectations. At both the elementary and middle school levels, the overwhelming 

majority of teachers reported meeting with their corps members, with 94 percent of elementary 

school teachers and 97 percent of middle school teachers reporting that they discussed 

expectations with corps members. Similarly, large percentages of elementary and middle school 

teachers strongly agreed or agreed that they and their corps members had established clear 

expectations for corps members’ work in the classrooms (Exhibit 1).
11

Exhibit 1 

Extent to which teachers agree that they established clear expectations 

for corps members’ work in their classrooms 

38

32

49

54

0 20 40 60 80 100

Middle school

(N=37)

Elementary

school

(N=68)

Percent of teachers

Strongly agree Agree

My corps member(s) and I have established clear expectations 

for their work in my classroom.

Exhibit reads: Thirty-two percent of elementary school teachers strongly agreed and 54 percent of teachers 

agreed that they and their corps members had established clear expectations for the corps members’ work in 

their classrooms. 

Source: Teacher Mid-Year Survey, Question 20 

Despite the survey results, teachers interviewed at two of the partner schools said that 

they believed that the partner leaders needed to do more at the beginning of the year to establish 

expectations for teachers and corps members. One teacher said that she wondered whether other 

teachers knew how they could use corps members in their classrooms. A teacher at another 

school echoed this comment, saying that she asked her corps member to “touch base with the 

[students] on the independent activity and then go into [the planned City Year intervention]” 

when the corps member pulled students out of the class. However, she felt that not all teachers 

in her school knew that they could instruct corps members in this way; she said, “[P]eople would 

like to know the direction.... It’s not clear to all of us.” 

 Survey data suggested that the majority of elementary and middle school corps members 

strongly agreed or agreed that teachers had regularly integrated them into their classrooms 

(Exhibit 2). However, a larger percentage of elementary school corps members strongly agreed 

or agreed than did their middle school counterparts. Furthermore, 60 percent of elementary 

school corps members strongly agreed that teachers regularly integrated them into their 

classrooms, compared with 34 percent of middle school corps members (p<0.01, effect 

size=0.31). 
12

Exhibit 2 

Corps members’ perspectives on whether teachers 

integrated them into the classroom 

34

60

43

26

0 20 40 60 80 100

Middle school

(N=67)

Elementary

school

(N=105)

Percent of corps members

Strongly agree Agree

*

The teacher(s) I work with regularly integrated me into their classroom

Exhibit reads: Sixty percent of elementary school corps members strongly agreed that the teachers they 

work with regularly integrated them into their classroom. Twenty-six percent of elementary school corps 

members agreed with this statement. 

* Statistically significant at p<0.01. 

Source: Corps Member End-of-Year Survey, Question 40 

 Although survey data showed that corps members generally felt that the teachers with 

whom they worked had regularly integrated them into their classrooms, teacher survey data 

suggested that there were limited interactions between corps members and teachers in discussing 

student performance. At mid-year, teachers were asked how often they met with corps members 

to discuss the students with whom corps members were working. As displayed in Exhibit 3, 

approximately one third of elementary and middle school teachers reported meeting with corps 

members at least once per week to review student-level data. Larger percentages of elementary 

and middle school teachers reported meeting with corps members at least once per week to 

discuss student performance. 
13

Exhibit 3 

Frequency of teacher-corps member meetings to discuss targeted students 

47

33

58

31

0 20 40 60 80 100

Meet to discuss the performance

of the students with whom

they are working

Review student-level data

for the students with whom

they are working

Elementary (N=67) Middle (N=36)

Percent of teachers who reported that they did the 

following with corps members at least once per week

Exhibit reads: Thirty-one percent of elementary school teachers said that they meet with corps members at least 

once per week to review student-level data for the students with whom they are working. Thirty-three percent of 

middle school teachers reported doing the same at least once per week. 

Source: Teacher Mid-Year Survey, Question 21 

Even when both groups met to establish expectations for service and support, interviews 

with corps members suggested that corps member-teacher partnerships still encountered some 

challenges. At one school, for example, corps members and the team leader described initial 

challenges to establishing relationships between City Year and the teachers and to setting 

expectations for service activities. The corps member team initially focused their activities on 

addressing school needs as defined by the teachers with whom they were partnering. The team, 

however, did not establish a formal plan to work with the students on their focus lists and, as a 

result, the team had a smaller role in the school than had been anticipated. Indeed, corps 

members were frustrated that they were not given more autonomy to work with students and that 

teachers were unsure of what City Year was and how to use corps members in their classrooms. 

 To measure the strength of corps member/teacher partnerships within classrooms, 

evaluators developed two indices. The first index measured the strength of the corps 

member/teacher partnership from the perspective of teachers, including whether teachers worked 

with City Year to identify students for one-on-one or small-group tutoring, whether teachers 

strongly agreed that they and their corps members established clear expectations for corps 

members’ work in the classroom; whether they met with corps members regularly to review their 

performance; whether teachers regularly contributed to their corps members’ professional 
14

development; whether teachers reported that their corps members worked well with them; and 

whether teachers reported that their corps members integrated smoothly into their classrooms. 

Teachers received one point for each response of “yes” or “strongly agree.” Out of a possible 

seven points for the index, teachers’ scores ranged from zero to six points. Across all schools, 

the average school-level score was three points. 

Exhibit 4 

Strength of teacher partnerships with corps members, by school 

3 3

6

3

4

5

6

3

5

6 6

4 4

5 5

6

2 2 2 2 2 2

3 3

4

2 2 2 2

4

0

1 1 1 1

0

2

0

3

0

1 1

0

1

0

1

2

3

4

5

6

Sch 17

(N=2)

Sch 8

(N=5)

Sch 9

(N=6)

Sch 13

(N=4)

Sch 10

(N=6)

Sch16

(N=7)

Sch 18

(N=6)

Sch 12

(N=7)

Sch 19

(N=10)

Sch 15

(N=6)

Sch 11

(N=4)

Sch 3

(N=5)

Sch 5

(N=7)

Sch 2

(N=6)

Sch 6

(N=7)

Sch 4

(N=7)

Maximum score

School average

Minimum score

Corps Member Teacher Partnership

Teacher Perspective

Elementary School Teachers Middle School Teachers

Index Score

Exhibit reads: Among the 11 City Year elementary schools, School 17 had the lowest average score for teacher 

partnerships, with two points. The teacher partnership index scores for this school ranged from a low of zero points 

to a high of three points. 

*An index score was not calculated for School 14 because only one end-of-year teacher survey was received for that 

school. 

A second index measured teacher involvement in City Year activities. The index was 

based on five questions from the mid-year teacher survey, allowing teachers to earn up to seven 

points. This index included whether teachers attended an introductory meeting or event hosted 

by City Year at the start of the school year, whether teachers reported that there was a corps 

member matching process at their schools, and whether teachers had the opportunity to observe 

corps members perform ELA/literacy services. The index also included whether teachers 

strongly agreed that they were well informed about City Year’s mission and goals, whether 

teachers were familiar with City Year’s approach to instructional support, whether teachers had 

adequate opportunities to communicate with the team leader, and whether the team leader was 
15

responsive to their questions or concerns. Each response of “yes” or “strongly agree” earned 

teachers one point; teacher scores ranged from zero to seven points (out of a possible seven). 

The extent to which teachers were involved in City Year activities varied both within and across 

schools. Across all schools, the average school-level score was three points, suggesting that 

more could be done to strengthen teacher involvement in City Year activities and thereby bolster 

the strength of the City Year/school partnership. 

Exhibit 5 

Strength of teacher involvement with City Year, by school 

2

6

2

3

6

4

6

7

6 6

7

4

3

5 5

6

7

1 1

2

3 3 3

4 4

1

2

3 3

4 4

0 0

1 1

0

2

1

2

0

1 1

2

1

0

1

2

3

4

5

6

7

Sch 14

(N=2)

Sch 9

(N=4)

Sch 8

(N=5)

Sch 10

(N=6)

Sch 11

(N=6)

Sch 13

(N=4)

Sch 12

(N=5)

Sch 18

(N=5)

Sch 19

(N=10)

Sch 15

(N=5)

Sch 16

(N=7)

Sch 6

(N=5)

Sch 5

(N=6)

Sch 3

(N=5)

Sch 2

(N=5)

Sch 4

(N=7)

Sch 7

(N=2)

Maximum score

School average

Minimum score

Teacher Involvement Index

Elementary School Teachers Middle School Teachers

Index Score

Exhibit reads: Among the 11 City Year elementary schools, School 14 had the lowest average score for teacher 

involvement, with one point.** The teacher involvement index scores for this school ranged from a low of zero points 

to a high of two points. 

*An index score was not calculated for School 17 because only one mid-year teacher survey was received for that 

school. 

 Evaluators identified a strong, positive correlation between the teacher involvement index 

and the percent of students who received City Year literacy tutoring in the school (r=0.67, 

p<0.01). This association suggests that teachers in schools in which a greater proportion of the 

student population was enrolled in literacy tutoring reported higher levels of involvement in City 

Year activities. Indeed, where a greater proportion of the student population was enrolled in 

literacy tutoring, the average teacher involvement index score was 3.5 points compared with an 

average teacher involvement score of 2.5 among schools where a smaller proportion of students 

were enrolled in literacy tutoring (p<0.05, effect size=1.02). It may be that City Year’s goals are 
16

more widely recognized among teaching staff and teachers in schools in which larger proportions 

of students receive services. 

Principal Ratings of City Year-School Partnership 

As was true in 2011 and 2012, principals/liaisons in 2013 had strongly favorable 

assessments of the City Year program overall and the services that corps members provided to 

their respective schools. This suggests that City Year’s partnerships with schools are generally 

strong and continue to improve. Indeed, the vast majority of surveyed principals and liaisons 

strongly agreed or agreed that corps members worked well with teachers and staff (93 percent), 

integrated smoothly into the school (90 percent), served as positive role models (87 percent), and 

were well prepared for the work they did in schools (80 percent). These findings are consistent 

with principal and liaison assessments of City Year and the services corps members provided in 

2011 and 2012; in both years, more than 90 percent of principals and liaisons strongly agreed or 

agreed that corps members served as positive role models, worked well with teachers and staff, 

integrated smoothly into the classroom, and were well prepared for the academic work corps 

members did in their schools. 

 To measure the strength of the City Year/school partnership across the 19 schools, as 

reported by surveyed principals and liaisons, evaluators developed an index that combines 

myriad indicators of school partnership into one overall summary score, by school. The index 

includes whether schools offered trainings to corps members or invited them to participate in 

staff professional activities more than once per month. The index also includes whether 

principals and liaisons agreed strongly with statements about City Year’s relationship with the 

school, including if City Year’s initiatives were well-aligned with school priorities, and whether 

the principal/liaison had adequate opportunities to communicate with the team leader. Out of a 

possible 10 points, individual principal/liaison scores ranged from 0 to 10 points across the 19 

schools (Exhibit 6). Across all schools included in the study, the average school-level score was 

six points, suggesting that there was room for improvement in enhancing City Year/school 

relationships. 

The school partnership index was positively correlated with the percent of students 

served (r=0.69, p<0.01). Principals in schools in which corps members provided literacy 

interventions to a greater percentage of students had an average index score of 7.1 compared to 

the average school partnership index score of 4.1 among schools where smaller percentages of 

students received services (p<0.05, effect size=1.2). 
17

Exhibit 6 

Principal ratings of the school’s partnership with City Year, by school 

1

2

5

5 5

6

8 8

9

10 10

2

4

4

5

6

8 8

0

2

4

6

8

10

Sch 14

(N=2)

Sch 11

(N=2)

Sch 15

(N=2)

Sch 12

(N=4)

Sch 9

(N=1)

Sch 19

(N=1)

Sch 8

(N=2)

Sch 13

(N=2)

Sch 18

(N=1)

Sch 17

(N=2)

Sch 16

(N=1)

Sch 6

(N=2)

Sch 3

 (N=2)

Sch 2

(N=1)

Sch 4

(N=3)

Sch 5

(N=2)

Sch 1

(N=1)

Sch 7

(N=1)

Elementary school principals Middle school principals

Index Score

City Year-School Partnership

Exhibit reads: Among the 11 City Year elementary schools, School 14 had the lowest average score for the 

City Year-school partnership, with one point.** 

*Light blue bars denote schools in which corps members provide literacy tutoring to a larger percentage of the 

student population relative to other schools in the sample. 

**An index score was not calculated for School 10 because no end-of-year principal/liaison surveys were 

received for that school. 

 Additionally, stronger school partnerships may be associated with greater levels of 

teacher involvement with City Year. Principals’ ratings of school partnerships and teachers’ 

ratings of their involvement with City Year were positively correlated (r=0.66, p<0.01). As 

suggested in site visit interviews, schools with higher partnership scores might have principals or 

liaisons who were proactive in their approach to supporting City Year teams, including involving 

corps members in planning, providing them with professional development, and supporting the 

relationships between corps members and teachers. This investment from the school leadership 

might send a clear message to teachers about City Year’s role in the school, encouraging teachers 

to become more involved in their corps members’ service.
18

III. Corps Member Team Quality 

 This chapter describes the training that City Year New York provided to help prepare 

corps members and team leaders to effectively serve the needs of its partner schools. The City 

Year program offers intensive training to corps members prior to the start of the school year and 

then offers additional training and professional development opportunities throughout the year. 

This chapter also examines team leaders’ and program managers’ roles in building strong teams 

and supporting corps members, as well as identifies potential factors affecting team quality and 

corps member satisfaction ratings. 

Training 

Overall, City Year’s training goals included preparing corps members to deliver effective 

support in literacy and math instruction, and attendance and behavior management; to build 

strong relationships with teachers and school staff; and to learn the City Year culture so that 

corps members can serve as ambassadors for the program. 

Type and Frequency of Corps Member Training 

Most corps members had entered City Year with some education and experience that 

provided background knowledge for their service. Eighty-one percent of corps members 

reported having completed a bachelor’s degree, six percent had attended some college courses, 

and one percent had earned an associate’s degree. Although many corps members had 

experience working with children and in educational environments prior to their year of service, 

few had received any formal training. 

As an introduction to City Year, all corps members participated in City Year’s Basic 

Training Academy (BTA) for three weeks in the summer prior to starting the school year. The 

training addresses City Year culture, literacy intervention skills, and other techniques to help 

support work in literacy tutoring, such as behavior management. Corps members also learn 

lesson planning skills. Fifty-eight percent of corps members reported that they were satisfied or 

very satisfied with their experience at the Basic Training Academy. 

After the school year began, members of the City Year service team provided additional 

training to corps members and team leaders. The service team comprises City Year New 

York’s service director, along with service managers who specialize in each of the topic areas for 

which City Year provides service, such as literacy and math. The service team supports all 

school-based corps member teams throughout the school year. A team leader may request 

training on a particular topic from the service team, and the service team member who 

specializes in that topic area provides targeted professional development. During interviews, 

some team leaders reported that the service team trainings had been very helpful to corps 

members and, as a result, corps members frequently requested supplemental training over the 

course of the school year. 
19

Much of the ongoing training corps members received throughout the school year 

occurred on Learning and Development days. Each Friday, all corps members participate in 

city-wide activities that vary from week to week. Learning and Development days may involve 

either training from service team members or a visit to a City Year school to observe other corps 

members. Some Learning and Development days have included community service projects or 

support for corps members to plan for life after City Year. According to one program manager, 

the Learning and Development manager sought to align the Friday activities with the service that 

corps member teams provided in schools throughout the year. 

Corps member training also included program managers and team leaders observing 

corps members at work during the school year and providing feedback. Team leaders and 

program managers scheduled regular observations of all corps members: team leaders observed 

corps members once a month and program managers observed corps members once a quarter. In 

site visit and telephone interviews, corps members, team leaders, and program managers reported 

that the frequency of service team observations varied throughout the year. Due to other 

commitments, not all team leaders and program managers were able to conduct as many 

observations as they were assigned. 

 Program managers and team leaders observed corps members in each area of service: 

literacy intervention, math intervention, attendance coaching, and behavior coaching. Observers 

have a few templates to choose from when structuring their observations. After the observation, 

the observer and the corps member meet to discuss areas in which the corps member can 

improve. One program manager described how she targets her feedback to behavior 

management skills that can help support academic interventions: 

I try to observe twice a week to see what they are doing and have an immediate follow-up 

with them. I give them tips on how I thought things were going. Both schools have big 

behavior problems. A lot of the feedback is about how to control behavior so that I can 

get through the whole intervention. It’s more a way of how they can get through the 

intervention using behavior management techniques. 

 Another program manager said that he tried to get a sense of the full spectrum of a corps 

member’s service when he conducted his observations: 

I structured my observation schedule differently. I spent a whole day with one corps 

member, following one corps member. I saw the holistic day of what that corps member 

is like. I saw the transitions of their day. It was a cool opportunity for me. It’s a 

structure I’m going to keep for next year. 

Service team members also visited schools to conduct observations periodically 

throughout the year. Because service team members have subject area expertise, they can offer 

strategies to address specific issues observed. They are not familiar with individual corps 

members or the school environment, however, especially because different service team 

members visit each school throughout the year. One corps member expressed frustration that 

several different service team members observed her during the year, and because there was no 

continuity among the observers, corps members could not build on previous feedback. Still, 61 
20

percent of corps members reported that they were satisfied or very satisfied with the observation 

and coaching they received from their program manager and the service team. 

Of the various supports offered to corps members, survey data show that corps 

members most frequently participated in training sessions offered by City Year New York. 

Thirty-seven percent of corps members attended a training related to school-based service four or 

more times, and an additional 45 percent attended a training at least twice during the school year 

(Exhibit 7). In addition, 46 percent of corps members reported being observed by a senior corps 

member, such as a team leader, second-year corps member, or program manager, at least twice 

during the school year, although 27 percent said that they were never observed. 

Exhibit 7 

Frequency of participation in literacy/ELA training and coaching 

(N=184) 

7

7

7

8

37

23

25

34

38

45

15

34

29

27

10

56

35

29

27

8

0 20 40 60 80 100

Observation and related coaching by your

teachers or another school staff member

during ELA/literacy sessions

Training and professional development

offered by your school or district

Observation and related coaching by other

City Year staff member (e.g., Training

Manager, Site Literacy Lead, etc.) during

ELA/literacy sessions

Observation and related coaching by your

SCM and PM during ELA/literacy sessions

Training sessions related to school based

service offered by CYNY

Percent of corps members

4 or more times 2-3 times Once Never

How often have you participated in the following literacy/ELA related trainings or coaching?

Exhibit reads: Thirty-seven percent of corps members participated in training sessions related to school-based 

service offered by their site four or more times by the middle of the school year. 

Source: Corps Member Mid-Year Survey, Question 25 

Some corps members found that the training provided by City Year was helpful to their 

service, but others did not. Of the training and supports available, corps members most 

frequently identified Learning and Development days as moderately or very helpful (58 percent) 

(Exhibit 8). Forty-two percent of corps members found Learning and Development days to be 

only slightly helpful or not at all helpful, and 47 percent found Basic Training Academy to be 

slightly helpful or not at all helpful. Though some corps members interviewed found these 
21

trainings useful, others found them to be irrelevant or repetitive. During interviews, corps 

members commented that experience was the best teacher in learning how to better serve their 

school. Continued training throughout the school year provided on Learning and Development 

days allowed corps members to apply lessons to the context of their own schools based on their 

classroom experience. 

Exhibit 8 

Corps member perceptions of the usefulness 

of ELA-focused training provided by City Year 

(N=174) 

22

31

14

15

22

20

39

43

0 20 40 60 80 100

Observation and coaching

by my program manager

Observation and coaching

by my team leader

Basic training academy

Learning and development days

Percent of corps members

Very helpful Moderately helpful

How helpful were the following in supporting you in your service this year?

Exhibit reads: Fifteen percent of corps members found Learning and Development days to be very 

helpful, and another 43 percent found them moderately helpful. 

Source: Corps Member End-of-Year Survey, Question 44 

The relative helpfulness of team leader and program manager observations and 

coaching varied. Forty-nine percent of corps members reported that observations from their 

team leader were slightly or not at all helpful, and 56 percent said the same regarding 

observations from their program manager. Nevertheless, when looking across all types of 

training that corps members received, more corps members found observations and coaching to 

be “very helpful” than any other type of training (Exhibit 8). For some corps members, program 

managers and team leaders effectively observed their work and provided very helpful feedback, 

demonstrating the potential that exists from this type of training. 
22

 One major obstacle to effective observation feedback was that several team leaders 

interviewed were not comfortable observing and giving feedback to corps members. Although 

program managers and members of the service team still observed and provided feedback at 

these schools, corps members in the schools visited did not receive the regular frequent feedback 

from the team leader. 

 In the mid-year survey, corps members identified which aspects of trainings worked well 

and which could be improved. In general, corps members found that ELA trainers and 

presenters were well-prepared to deliver content, as shown in Exhibit 9. They also reported that 

ELA trainings increased their knowledge of ELA and provided knowledge and skills that they 

could readily apply. However, fewer corps members found the trainings to be relevant to their 

service and school or the sequence of the ELA trainings to be logical. 

Exhibit 9 

Corps member perceptions of ELA-focused trainings 

(N=205) 

12

14

12

14

15

14

26

37

50

54

52

51

53

51

0 20 40 60 80 100

Ordering of the ELA trainings has

been logically sequenced

ELA trainings have increased my level

of preparedness for service delivery

ELA trainings have been relevant to

my service and school

ELA trainings have been high-quality

ELA trainings have increased

my knowledge about ELA

ELA trainings have provided knowledge

and skills that I can readily apply on the job

ELA trainers and presenters have been

well-prepared to deliver the content

Percent of corps members

Strongly agree Agree

Indicate your level of agreement with the following statements about ELA trainings:

Exhibit reads: Twenty-six percent of corps members strongly agreed that ELA trainers and presenters 

have been well-prepared to deliver the content. 

Source: Corps Member Mid-Year Survey, Question 35 
23

Corps Member Perceptions of Service Preparation and Training 

Most corps members reported feeling prepared to carry out the literacy and in-class 

support components of service. Fifty-five percent of corps members reported that they felt very 

prepared to provide in-class support on ELA or literacy, and 55 percent felt very prepared to give 

one-on-one or small-group tutoring in ELA or literacy. Corps members assigned to elementary 

schools tended to feel more prepared, with 64 percent feeling very prepared for one-on-one or 

small-group tutoring, compared to 39 percent of middle school corps members, a statistically 

significant different (p<0.01, effect size=0.24). There was less differentiation across school 

levels in preparation for whole classroom support, with 57 percent of elementary school corps 

members and 51 percent of middle school corps members feeling very prepared for whole 

classroom support. 

In order to effectively tutor students in literacy, corps members needed training in 

other skill areas as well. For example, corps members needed training on collaborating 

effectively with their City Year team, teachers, and other school staff. As one corps member 

described, “We had a training on how to do an ‘Hour One,’ which is the first time you meet the 

teacher to set expectations. I thought that was really useful.”

Corps members also reported that behavior management beyond the dedicated behavior 

coaching was a major part of their role because behavior issues arise during tutoring, in-class 

support, and during transition times. One corps member said, “Whether you're doing [it] in class 

or you're doing a pull-out, whether you're doing attendance, behavior—there's always a focus on 

behavior management.” At the beginning of the school year, 70 percent of corps members 

strongly agreed or agreed that they felt good about their strategies for handling behavior 

management issues in their day-to-day work (five or above on a scale of one to seven). Forty-three 

percent felt very prepared to carry out whole class or homeroom behavior support by mid-year. 

In interviews, several corps members and team leaders said that the most useful type of 

training was actually practicing literacy tutoring techniques. Many corps members 

interviewed were frustrated with the PowerPoint-centric structure of the Basic Training 

Academy and felt that some of the presenters did not have expert knowledge of the topics they 

were teaching. Practicing literacy tutoring techniques helped corps members remember skills 

that they had been taught and understand how to apply them. The team leader at one school said 

that the best training she received was during the summer in which she and other new team 

leaders (as well as some program managers) practiced observing each other tutor students (the 

training was held at a summer youth program). They then gave each other feedback on the 

tutoring—and proceeded to coach each other on how to give more useful feedback. 

Corps members also said in interviews that trainings that addressed specific strategies to 

use with students were the most helpful. For example, one corps member reported applying the 

Frayer model of using graphic organizers (presented during Basic Training Academy) while 

working with her literacy focus list students during in-class support. This corps member’s 

classroom teacher, who had used the strategy in the past, began incorporating it into her lessons to 

create continuity between classroom instruction and the corps member’s work with students.
24

 Corps members at several schools said that they would benefit from an online resource 

hub. One corps member noted that having online videos available to learn a specific skill would 

allow them to be trained on techniques relevant to their work in a timely manner. Another corps 

member requested a collection of worksheets and activities on particular literacy skills to help 

address the individual needs of their students. 

Corps members identified specific areas in which they needed additional training. 

During interviews, corps members at several schools that only used the push-in model to deliver 

literacy interventions (i.e., they tutored students within the classroom instead of pulling them 

out) felt that the Basic Training Academy focused too heavily on strategies to be used during 

pull-outs. Although all corps members provided in-class support, not all corps members pulled 

out students for tutoring. Additional time spent on in-class support techniques as well as explicit 

training on push-in interventions would benefit a greater number of corps members. 

In addition, several corps members at different schools reported feeling unprepared to 

work with special education students. The City Year Whole School Whole Child model does not 

include service targeting special education students because of the level of difficulty working 

with students with special needs. In practice, however, corps members at many schools worked 

with special education students because these students appeared on their literacy focus lists or 

because they worked with these students during in-class support. This was particularly true for 

corps members in elementary schools, as New York City is moving toward an inclusion model 

and many general education classrooms include special education students. Because City Year 

did not intend corps members to work with special education students, corps members did not 

receive any training on how to work with special education students, and as a result were 

unprepared to assist these students. 

Overall, corps members reported a moderate level of preparedness for their service 

tasks. To determine whether there was variation across schools in the overall quality of corps 

member teams, evaluators developed an index to measure the extent to which corps members felt 

that they were prepared to carry out their service tasks. The Preparation and Training index 

includes whether the corps member holds at least an associate’s degree, a degree in education, 

has worked with youth in a formal education setting (e.g., as a tutor, classroom aide, or 

classroom teacher), and is comfortable tutoring students in literacy/ELA. In addition, the index 

includes corps member ratings of the skills they need to work with youth, how often they 

participated in literacy/ELA trainings, their preparedness to perform school-based activities, and 

how helpful they found coaching from City Year staff. Corps members received one point for 

responses on the most positive end of the response scale (e.g., “very comfortable,” “strongly 

agree”) and for responses of “yes.” 

 Out of a possible 22 points, corps member scores ranged from a low of two points to a 

high of 18 points (Exhibit 10). Across all schools, the average preparation score was 10 points, 

suggesting that City Year could do more to prepare corps members for service. 
25

Exhibit 10 

Corps member ratings of their preparation and training, by school 

16

11

13

16 16

14 14 14 14

16

18

16

15

12

11

14

12

8 8

9

10 10 10 10

11 11

12 12

7 7

8

9 9

2

5 5

4

6 6 6 6

8 8

6

4

2

4

5

2

0

2

4

6

8

10

12

14

16

18

Sch 11

(N=7)

Sch 13

(N=7)

Sch 19

(N=6)

Sch 18

(N=6)

Sch 15

(N=7)

Sch 12

(N=9)

Sch 17

(N=2)

Sch 8

(N=7)

Sch 14

(N=9)

Sch 16

(N=7)

Sch 10

(N=9)

Sch 9

(N=8)

Sch 2

(N=7)

Sch 7

(N=6)

Sch 1

(N=5)

Sch 6

(N=8)

Sch 3

(N=6)

Maximum score

School average

Minimum score

Corps Member Preparation and Training

Elementary School CMs Middle School CMs

Index Score

Exhibit reads: Among the 12 City Year elementary schools, School 11 had the lowest average score for preparation 

and training, with eight points. The corps member index scores for this school ranged from a low of two points to a 

high of 16 points. 

 At the school level, on average, corps members working in elementary schools had an 

average school-level score of 10 points, compared with the average school-level score of eight 

points for middle school corps members (p<0.05, effect size=1.05). Middle school corps 

members may have felt unprepared because middle schools more frequently pushed-in for 

literacy intervention rather than pulled-out. Corps members interviewed at the middle schools 

visited for this evaluation noted that their training had focused on pull-outs and was thus not 

relevant to service in their school. Developing corps member trainings that are specifically 

tailored to the middle school environment, and that focus more on push-in interventions, might 

help corps members assigned to middle schools feel more prepared for service. 
26

Teacher Perceptions of Corps Member Preparation and Training 

Teachers generally felt that corps members had sufficient training to perform their 

support role in the classroom. Indeed, despite the views of corps members about their own 

training and level of preparedness, 83 percent of teachers strongly agreed or agreed that the corps 

members assigned to their classrooms were well-prepared for the academic work they did with 

students at mid-year. At the end of the year, 86 percent of teachers reported that they were very 

satisfied or satisfied with the overall preparation and training of corps members for the services 

they provide. An ELA teacher in a middle school said that the corps member in her class was 

indeed prepared to work with students to support her lessons and help students “who need a little 

bit more support in trying to complete their task.” She described:

[The corps member is] able to sit down with them, break down the article, and use the 

strategies that we've discussed in class. And he's able to work on a one-to-one basis with 

them and it frees me up to help students with the revision portion. So it takes a load off 

me in a way, because I'm able to move forward and not have to redo an entire lesson.... I 

think that he doesn't have an education background, but I think that he is a very smart 

and very capable young man. So I think that his joining the program also said a lot 

about his desire to work with young people and wanting to help them. So I think that he 

is someone who is very well-prepared and capable of helping students. 

Most teachers reported providing additional training and support to corps members to 

help them better serve their focus list students and the classroom overall. Sixty-six percent of 

teachers strongly agreed or agreed on the mid-year survey that they met regularly with their 

corps member to review the corps member’s performance. Fifty-three percent of teachers 

strongly agreed or agreed that they regularly contributed to their corps member’s professional 

development. One teacher noted that although her corps member was prepared overall to help 

in her classroom, the corps member had gaps in knowledge that needed to be addressed: 

I had to speak to [my corps member], but I think that having the basics is fine; we can all 

use refresher courses on what needs to be done. Just going back and refreshing for her 

not to forget, just like [I need to do]. 

Team Leadership and Support 

 In the end-of-year survey, team leaders were asked to report the three responsibilities on 

which they spent the most time during the school year. As displayed in Exhibit 11, team leaders 

most often selected instructional coaching with corps members, coordinating after school 

programming and extended-learning time activities, and corps member-teacher scheduling and 

relationship management. 
27

Exhibit 11 

Responsibilities on which team leaders spent the most time 

(N=18) 

2

4

4

14

14

16

0 2 4 6 8 10 12 14 16 18

Leading on Idealist Journey sessions and

preparation

Direct Tier 2 service

Data entry

Other

Corps member-teacher scheduling and

relationship management

Coordinating after school programming/

extended-learning time

Instructional coaching with

corps members

Number of team leaders

0

Exhibit reads: Sixteen out of 18 team leaders selected “instructional coaching with corps members” 

as one of the three responsibilities on which they spent the most time. 

Source: Corps Member End-of-Year Survey, Question 12 

 In addition to the team leader, all City Year teams are supported by a program manager, 

who oversees up to two teams of corps members. Program managers are tasked with supporting 

the growth and professional development of team members, managing relationships with school 

administrators and teachers, ensuring program compliance, and participating in the team 

selection process for the following school year. 

 When asked to describe their role, the eight interviewed program managers highlighted the 

direct support they offer teams, including observation and coaching for both team leaders and 

corps members. For corps member observations, one program manager said that he/she provides 

feedback that aims to improve intervention services and especially to ensure that corps members 

are connecting intervention lessons to previous sessions: “Is there a connection to the last lesson? 

Are [corps members] giving a clear goal for the session?” Another program manager, who usually

observed corps members twice each week, supported corps members by providing feedback about 

behavior management so that corps members are able to deliver their interventions. For team 

leaders, program managers’ support often begins during the summer. One program manager 

stated, “I want to make sure that [the team leaders] can communicate effectively with the staff, that 

they have a schedule for checking in with corps members, that they understand high-quality 

service, and [that they] know how to do good observations and give good feedback.” 
28

Program managers who managed teams at multiple schools described challenges 

related to the amount of time they were able to spend at each school in order to fully support 

their teams. One challenge faced by program managers related to scheduling the appropriate 

time to spend at the school. In one program manager’s words, “We’re supposed to be in schools 

75 percent of the time. When you straddle two partnerships, it’s hard to say where you’re going 

to be and when. It’s hard to determine how to split that 75 percent evenly.” A second program 

manager stated that he’s found it challenging to manage two partnerships and feels “pulled in 

different directions.” 

 The end-of-year corps member survey asked corps members to rate the effectiveness of 

their program manager as a leader. As displayed in Exhibit 12, slightly more than half of corps 

members (58 percent) strongly agreed or agreed that their program manager was, overall, an 

effective leader. Seventy-three percent of corps members strongly agreed or agreed that their 

program managers maintained high expectations of them individually or of the team, and 63 

percent of corps members strongly agreed or agreed that their team leaders made them feel like 

they could trust the program manager. 

Exhibit 12 

Corps member attitudes toward their program manager 

(N=171) 

17

20

20

20

25

19

28

28

23

33

34

38

33

39

35

45

0 20 40 60 80 100

Facilitated strong relationships

with service partners

Helped me understand the

culture of City Year

Helped me and/or

my team resolve conflicts

Communicated effectively with

me and/or my team

Was very effective as a leader overall

Helped to facilitate leadership

development experiences

Made me feel like I could

trust him/her

Maintained high expectations of

me and/or my team

Percent of corps members

Strongly agree Agree

My program manager...

Exhibit reads: Forty-five percent of corps members agreed and 28 percent of corps members agreed 

strongly that their program manager maintained high expectations of them or their team. 

Source: Corps Member End-of-Year Survey, Question 42 
29

Corps members were somewhat satisfied with the overall quality of their City Year 

team. To measure the strength of City Year teams’ leadership and cohesion, evaluators 

developed an index that includes measures of team function (e.g., how often teams met to discuss 

best practices), corps member ratings of the program manager’s leadership, frequency of one-onone meetings with the program manager, and corps members’ overall ratings of their team 

experience. Out of a possible 16 points, corps members ranged from a low of zero points to a 

high of 15 points, and the average school-level score across all schools was four points. As 

displayed in Exhibit 13, there is variation of individual corps member scores within schools; this 

variation might be driven by corps members’ perceptions of their program managers’ leadership. 

Exhibit 13 

Corps member perceptions of team quality, by school 

 

6

4 4

6

9 9

14

6

7

13

15

7

9

5

6

8

7

1

2

3 3 3 3 3 3

4

8

9

1 1

2 2 2

3

0

1

0 0 0 0 0 0 0

1

4

0 0 0 0 0 0

2

4

6

8

10

12

14

16

Sch 11

(N=11)

Sch 14

(N=8)

Sch 13

(N=7)

Sch 10

(N=9)

Sch 19

(N=10)

Sch 18

(N=6)

Sch 9

(N=9)

Sch 15

(N=10)

Sch 17

(N=5)

Sch 12

(N=8)

Sch 16

(N=7)

Sch 2

(N=10)

Sch 7

(N=10)

Sch 6

(N=8)

Sch 3

(N=8)

Sch 5

(N=9)

Sch 4

(N=10)

Maximum score

School average

Minimum score

Team Quality: Leadership and Cohesion

Elementary School CMs Middle School CMs

Index Score

Exhibit reads: Among the 12 City Year elementary schools, School 11 had the lowest average score for team 

quality, with one point. The corps member index scores for this school ranged from a low of zero points to a high 

of six points. 

Corps Member Satisfaction 

Corps members at both the elementary and middle school levels reported moderate 

levels of satisfaction with their training and their City Year experience. Overall, when asked on 

the Corps Member End-of-Year survey how they would rate their service experience, 6 percent of 

all corps members selected “excellent,” and 70 percent selected “good” or “very good.”
30

Corps members gave high ratings to City Year as a learning experience: 31 percent of 

corps members rated it as “excellent,” and 55 rated it as “good” or “very good.” Fourteen 

percent of corps members rated City Year as an excellent leadership development experience, 

while another 56 percent rated it as “good” or “very good.” Similarly, when asked how they 

would rate their overall team experience, 31 percent responded “excellent” and another 55 

percent responded “good” or “very good.”

To measure corps member satisfaction, evaluators developed an index that includes the 

extent to which corps members were satisfied with the ELA/literacy and student engagement 

training they received, and their overall ratings of their team experience, leadership development 

experience, and learning experience in City Year. Corps members received one point for every 

response of “strongly agree,” “very good,” or “excellent.” Out of a possible 11 points on the 

index, individual corps member satisfaction scores ranged from 0 to 11 points, with variation 

within and across schools (Exhibit 14). Across all schools, the average corps member satisfaction 

score was two points, emphasizing the fact that few corps members gave top-level ratings to their 

overall training and City Year experiences. 

Exhibit 14 

Corps member satisfaction with City Year, by school 

5

3

4

3 3

9

7

11

7

9

11 11

3

5

4 4

6

1 1 1

2 2

3 3 3 3

4 4

5

1 1

2 2 2

0 0 0 0 0 0 0 0

1

0 0

1

0 0 0

1

0

2

4

6

8

10

Sch 11

(N=9)

Sch 15

(N=11)

Sch 17

(N=5)

Sch 14

(N=8)

Sch 12

(N=7)

Sch 18

(N=10)

Sch 16

(N=8)

Sch 10

(N=8)

Sch 8

(N=9)

Sch 13

(N=7)

Sch 9

(N=10)

Sch 19

(N=9)

Sch 7

(N=11)

Sch 2

(N=10)

Sch 5

(N=8)

Sch 4

(N=9)

Sch 6

(N=11)

Maximum score

School average

Minimum score

Corps Member Satisfaction

Elementary School CMs Middle School CMs

Index Score

Exhibit reads: Among the 12 City Year elementary schools, School 11 had the lowest average score for corps 

member satisfaction, with one point. The corps member index scores for this school ranged from a low of zero points 

to a high of five points. 
31

IV. Implementation of Literacy Interventions 

 In 2012-13, corps members provided literacy tutoring services to a total of 1,086 students 

in 12 elementary and seven middle schools. Of these students, 689 were elementary school 

students and 397 were middle school students. This chapter describes how students were 

selected to receive City Year literacy tutoring services, the literacy intervention models that 

corps members used to deliver the literacy tutoring services, and the content of the literacy 

intervention. In addition, this chapter describes the ways that corps members monitored and 

communicated with teachers about the progress of the students they served. 

Student Selection 

Each corps member who provided literacy intervention supports had a “focus list” of 

four to six students with whom they worked on literacy skills. According to program manager 

interviews, City Year recommends that corps members work with students whose scores on the 

previous year’s state English-language arts assessments were not proficient but who were not the 

highest need students, known as the “high 1’s and low 2’s.” (“Proficiency” on the state ELA 

assessment is defined as scoring a 3 or 4; scores of 1 or 2 are categorized as below proficient.) 

Sixty-eight percent of corps members reported that City Year had aligned their focus 

list students with their in-class support assignments for the 2012-13 school year. In previous 

years, as reported in Year 2, some corps members tutored focus list students who were not in the 

same classes in which corps members offered in-class support. As a consequence, corps 

members had to coordinate schedules and follow the curriculum of multiple classrooms. 

Interviews during this year’s visits showed that providing all literacy services to students in the 

same class gave corps members the opportunity to build relationships with their classroom 

teacher and to become familiar with the literacy work taking place during class time. It also 

allowed the corps members to understand more about the needs of their focus list students and 

the work they were doing in class. 

Eighty-eight percent of teachers surveyed reported that they had at least some input 

into the selection of students receiving one-on-one or small-group support from corps 

members. City Year teams and school staff used a combination of assessment data—including 

grades, scores on the state ELA and math tests and the Fountas and Pinnell assessment—

attendance and disciplinary data, and personal knowledge of the students to decide who they 

should target for tutoring services. Corps members administered the QRI-5 reading assessment 

to focus list students to confirm that the selected students were appropriate candidates for 

tutoring. As the City Year liaison at one school described: 

The teachers were involved in the selection of focus list kids. We gave [corps members] 

the children’s performance scores. Then [corps members] made their preliminary list 

after seeing the kids who had need. Sometimes teachers would suggest other students, 

like kids who [had behavioral problems] to go on the focus list. That decision, they made 

with the teachers. 
32

Some schools selected students who fit the City Year profile for service in that they were 

“high 1’s and low 2’s,” while others selected the students they felt would benefit most from 

working with corps members. In some schools, low-performing students with significant needs, 

including special education students, were selected to work with City Year corps members. As 

one program manager described, special education students can benefit greatly from working 

with corps members even though they are not City Year’s target population:

At one school we are placed with SPED students. I don’t want to say that they’re the 

wrong students, but they’re not City Year’s ideal. But they can benefit from the help. 

They’ve gotten a lot of help that they wouldn’t have otherwise gotten. There aren’t any 

students that we’re working with who don’t need the help. 

Corps members working with these high-needs students noted that they wished they had had 

more training that applied to the particular type of students that they actually worked with, rather 

than the type of students they were supposed to work with. 

Literacy Tutoring Models 

 In 2011-12, in an effort to improve the uniformity and consistency of the WSWC model 

implementation and service quality across participating schools, leadership at City Year New 

York defined the range of approved models of literacy service delivery that follows WSWC 

model guidance which is based on previous research on other literacy programs. 

Literacy Tutoring Delivery Models 

City Year’s goal for literacy intervention is for each corps member to work with students 

on their focus lists for a minimum of 15 hours over the course of the year, in sessions of at least 

15 minutes in length. City Year offers push-in, pull-out, and other modes of literacy tutoring 

service delivery from which schools can choose. The following describes each mode of literacy 

tutoring service delivery: 

■ Pull-outs. Corps members take an individual student or a small group of students 

out of the classroom and provide literacy tutoring in the hallway or in a separate 

space in the school such as the City Year office. 

■ Push-ins. Corps members work with an individual student or a small group of 

students directly on classroom-related tasks or other material. 

■ Other intervention settings. Corps members deliver literacy interventions to 

individual students or small groups during before- or after-school programming, 

or during extended learning time at the beginning or end of the school day. 

Corps member survey data suggest that corps members delivered literacy interventions 

in ways that varied among and within schools. When surveyed mid-year, corps members who 
33

provided literacy tutoring most commonly reported using the pull-out model (30 percent) or 

push-in model (29 percent) during class time. Twenty-two percent of corps members, however, 

reported that their primary setting for intervention was a combination of push-ins and pull-outs. 

Corps members also reported providing literacy tutoring after school (11 percent), or during the 

school day (6 percent) (i.e., Extended Learning Time may fall under this category). 

 During site visit interviews, corps members in several schools noted that non-focus list 

students wanted to spend time with their corps member, and felt left out if they were not allowed 

to do so. One corps member explained: 

A lot of my kids are like, “Oh. Let me come with you.” It's the kids that don't need to 

be pulled out at all. But they don't understand that the kids that are coming up [to the 

City Year office] need the help, and that's why they're being pulled out. And 

sometimes I'll just bring them just because they never get to come up or whatever. 

But they don't really understand what we do when we take kids out. And they're like, 

you guys are having fun without us. I'm like, “No. Really. They're probably doing 

math problems.”

Service Delivery Preferences among School Staff 

In site visit interviews, principals and teachers who encouraged pull-outs reported that 

students were only pulled out during independent or small-group work time and did not miss 

teacher-led whole class instruction. For example, in one visited school, corps members were 

not allowed to pull students out of classrooms during a 90-minute instructional block because 

the school was implementing a particular structured lesson during that time. Instead, and with 

the teacher’s permission, corps members pulled students out during independent work time. 

Teachers in these schools reported that students benefitted from adult attention that they might 

not otherwise receive. 

Teachers also weighed the potential class benefits versus drawbacks when deciding 

whether corps members should use a pull-out or push-in service delivery model. Teachers and 

administrators in one elementary school that reportedly struggled with student behavior problems 

reported that the smaller class sizes resulting from the pull-out model made the class more 

manageable for the teacher and allowed her to work individually with other students. At another 

school, teachers believed that the pull-out model was less distracting for the other students 

compared with the push-in model. One corps member described a common scenario: “If I’m 

sitting in the room and a student is trying to ask me a question...even if it’s whispering, it’s 

distracting to the other students.” Even a teacher at a school that did not allow pull-outs said that 

she wished her corps member could work with students outside of the classroom in order to 

benefit the easily distractible students. 
34

Exhibit 15

Primary literacy intervention model, by school level 

4

2 2 2 2

1

3

1 1 1

0

1

2

3

4

5

Pull-out Push-in Pull-out/

push-in

After/before

school

No majority

setting

Primary literacy intervention model

Elementary (N=12) Middle (N=7)

Number of schools 

Exhibit reads: In four elementary schools, the majority of corps members who delivered literacy interventions 

reported doing so primarily in pull-outs during class time. The same was true for the majority of corps 

members in one middle school. 

Source: Corps Member Mid-Year Survey, Question 20 (aggregated by school) 

 According to teacher survey responses, teachers in schools in which pull-outs were the 

primary setting for literacy intervention (as determined by corps member surveys) (Exhibit XX) 

were more satisfied than those in schools where push-ins were primarily used: the school-level 

means on the Teacher Satisfaction Index (described later in this chapter) were 9.4 and 4.3 for 

schools using the pull-out model versus schools using the push-in model, respectively (p<0.05, 

effect size=1.8). This association suggests that teachers in schools that use the pull-out model 

are more likely to be satisfied with their corps members’ performance and/or with City Year.

Content of Literacy Interventions 

As reported in the Year 3 (2012) evaluation report, corps members in the 2011-12 school 

year used a variety of externally developed literacy tutoring curricula and activities, including 

Great Leaps, Worldly Wise 3000, Read Naturally, and Newbridge Fluency to provide literacy 

tutoring and support to their focus list students. Some City Year teams, however, used their host 

school’s literacy curricula to provide literacy tutoring services to focus list students. In site visits 

to three middle schools in 2011-12, evaluators also found that some corps members were 

creating their own lessons and tailoring them to their focus list students, and that the freedom to 
35

tailor activities to particular students and circumstances had its benefits. The following describes 

the design and content of literacy interventions in 2012-13. 

Designing Literacy Interventions 

In 2012-13, City Year New York dispensed with scripted programs such as Great Leaps 

and Read Naturally, and gave corps member teams more freedom to design a literacy 

intervention that would address the specific needs of their focus list students. According to 

City Year program leaders, although prescribed programs provided corps members with 

uncomplicated and easily implemented instructional strategies, they recognized that prescribed 

programs were sometimes a source of tension with the schools because those programs did not 

always address students’ needs. Indeed, surveys of corps members in 2012-13 revealed far fewer 

uses of set, structured curricula during literacy interventions and more frequent development of 

their own lessons or usage of class material as the starting point for skill work with students. 

To help corps members implement high-quality literacy interventions, City Year created 

a “Quick Reference Guide.” The guide describes how corps members should plan their literacy 

tutoring sessions and select appropriate materials or strategies to use with their focus list 

students. Unless the school selects a specific intervention that they would like corps members to 

use, this guide is an important resource that City Year provides to support corps members in 

designing the appropriate literacy intervention for their students. The guide informs corps 

members that they can use books being read in class, books or other texts at the students’ reading 

level, other text books, or other materials related to the content covered in class. The guide states 

that corps members are supposed to work with all students on vocabulary and comprehension, 

but that not all students will need to work on fluency. The guide states that “the ultimate goal is 

to supplement or reinforce what is being read or taught in class.” 

Corps members’ approaches to lesson design varied widely, with over a third (37 

percent) of corps members reporting that they primarily worked alone and 23 percent 

reporting working with a classroom teacher to develop their literacy tutoring sessions. Corps 

members participating in site visit interviews at one school reported that they planned their 

literacy interventions independently. At this school, corps members worked with City Year’s

3

site literacy lead who provided them with useful websites and guided reading materials to help 

them design their literacy lessons. These corps members reported reinforcing what students 

learned in class by using flashcards, games, and by modeling reading skills such as making 

inferences (strategies suggested in City Year’s literacy intervention guide) to help students 

practice their literacy skills. As one corps members explained: 

We do a lot of flash cards. We’re supposed to have an independent reading book every 

week so I will flip through my kids’ books when they’re working...and pull out 

vocabulary words and have them make flash cards with definitions because a lot of times 

they’ll just skip right past the word [they don’t know]. – Corps member 

 

3

 The site literacy lead is a member of the city-wide service team and provides support to corps member teams 

across New York City related to literacy interventions. 
36

Corps members in a few schools described how they used whatever text they could find that 

seemed interesting and appropriate for their students, and used reading strategies that their City 

Year training emphasized, such as “summarize” or “find the author’s point of view.” 

Some corps members participating in site visit interviews reported that the content of 

their literacy interventions was defined by the classroom teacher with whom they were working. 

At one of these schools, a corps member said, “[The content] is based on what my teacher thinks 

is best for the week. Normally, my teacher recommends that I work with [the focus list students] 

on vocabulary and comprehension.” The corps member also explained that the content of her 

lessons changed prior to the state test administration because her teacher thought it would be 

important to work with the students on test-taking strategies. 

Literacy Tutoring Dosage 

Corps members are required to provide a minimum of 15 hours of literacy tutoring for 

each literacy focus list student over the course of the school year. In addition, City Year required 

that corps members work with focus list students individually or in small groups for a minimum 

of 15 minutes per session, and 45-60 minutes per week. Finally, corps members are required to 

track the amount of time they spend tutoring each of their students.

Student participation data for the 2012-13 school year suggested that the extent to 

which City Year corps members achieved the program goal of providing a minimum of 15 

hours of literacy tutoring for their focus list students varied both within and across schools. 

4 Overall, the average amount of literacy tutoring corps members delivered to the 1,059 students 

served in 2012-13 was 15 hours. Of these students, 689 were in elementary schools and 397 

were in middle schools and corps member teams served an average of 57 students per school at 

each school level. In addition, elementary school students received more hours of tutoring, on 

average, than did middle school students. The average number of hours of literacy tutoring 

received by an elementary school student was 17 hours compared with 11 hours for middle 

school students. This difference was statistically significant (p<.01, effect size=.69). 

As shown in Exhibit 16, the average number of hours of literacy tutoring students 

received by school ranged from an average of five hours per student (School 3) to an average of 

33 hours per student (School 19). In addition, the minimum number of hours ranged from one 

hour of literacy tutoring (Schools 1, 2, 3, 4, 6, 11, 13, and 17) to a maximum of 50 hours. 

 

4

 Forty-one students were excluded from the analyses because they received no literacy tutoring (zero hours) and 27 

students were excluded because they received less than one hour of tutoring. 
37

Exhibit 16 

Literacy tutoring dosage hours, by school 

(N=1,059) 

School

Average number 

of Literacy 

dosage hours 

achieved

Minimum 

number of 

Literacy dosage 

hours achieved

Maximum number of 

Literacy dosage 

hours achieved

School 19 (n=70) 33 8 50

School 8 (n=57) 21 9 35

School 12 (n=51) 20 10 46

School 13 (n=48) 18 1 31

School 15 (n=49) 17 3 42

School 9 (n=55) 17 2 26

School 10 (n=58) 16 2 30

School 14 (n=43) 15 4 33

School 5 (n=28) 15 3 29

School 4 (n=61) 15 1 38

School 7 (n=60) 15 2 22

School 17 (n=54) 14 1 37

School 1 (n=54) 14 1 29

School 18 (n=47) 14 2 22

School 16 (n=102) 12 2 20

School 6 (n=59) 11 1 30

School 11 (n=48) 10 1 18

School 2 (n=60) 6 1 13

School 3 (n=55) 5 1 16

Exhibit reads: The average number of hours of literacy tutoring students received in School 19 was 33 hours. 

Overall, City Year corps members did not achieve the City Year benchmark of providing 

a minimum of 15 hours of literacy tutoring to all focus list students. That is, approximately half 

the focus list students received 15 hours or more of literacy tutoring. Of the remaining half, 

about 28 percent received about 75 percent of the City Year benchmark of 15 hours of 

instruction (7.51 to 14.99 hours of literacy tutoring); 11 percent received about 50 percent of the 

City Year goal (3.76 and 7.5 hours of literacy tutoring); and 11 percent received about 25 percent 

of the City Year goal (one to 3.75 hours of literacy tutoring) (Exhibit 17). 
38

Exhibit 17 

Percent of students in 2012-13 receiving the 

literacy tutoring dosage benchmark of 15 hours or less 

(N=1,059) 

Number of 

students 

receiving 

literacy 

tutoring

Percent of 

students

receiving 

literacy 

tutoring

Students who received 1 to 3.75 hours of literacy 

tutoring (25% of CY goal)

121 11

Students who received 3.76 to 7.5 hours of literacy 

tutoring (50% of CY goal)

111 11

Students who received 7.51 hours to 14.99 hours of 

literacy tutoring (75 of CY goal)

300 28

Students who received 15 hours or more of literacy 

tutoring (CY goal)

527 50

Exhibit reads: Approximately 11 percent of students who received City Year literacy tutoring services 

received approximately 1 to 3.75 hours of literacy tutoring during the 2012-13 school year. This represents 

approximately 25 percent of the City Year literacy tutoring dosage benchmark. 

By school, the extent to which City Year met the literacy tutoring dosage benchmark 

varied tremendously. That is, the percent of students receiving the literacy tutoring dosage 

benchmark of 15 hours ranged from 2 percent to 95 percent of students in 18 of the 19 schools 

implementing the WSWC model. In one school, however, none of the students received 15 

hours or more of literacy tutoring. 

 The majority of corps members (70 percent) who provided literacy tutoring to students 

reported that they typically spent at least 30 minutes with a student per literacy session. End-ofyear survey data indicated that corps members working in elementary schools were more likely 

to provide frequent literacy tutoring to students on a focus list. The vast majority of corps 

members (89 percent) at the elementary school level provided literacy tutoring for the same 

students on a focus list two days per week or more, whereas this was true for only 63 percent of 

corps members working in middle schools. This difference was statistically significant (p<.001, 

effect size=.30). In addition, while it was rare for corps members in elementary schools to 

report that they did not consistently provide tutoring for the same students on a focus list (6 

percent), this was true for 29 percent of corps members in middle schools (p<.001, effect 

size=.32). 
39

Exhibit 18 

Percent of student receiving in 2012-13 receiving the literacy tutoring dosage 

benchmark of 15 hours or less, by school 

(N=1,059) 

Exhibit reads: In School 8, five percent of students who received City Year literacy tutoring services 

received approximately 7.51 to 14.99 hours of literacy tutoring, or 75 percent of the literacy tutoring dosage 

benchmark during the 2012-13 school year. The remaining 95 percent of students in School 8 who received 

City Year literacy tutoring services received approximately 15 hours or more of literacy tutoring, or 100 

percent of the literacy tutoring dosage benchmark during the 2012-13 school year. 

Disruptions and irregularities in teachers’ and schools’ schedules negatively impacted 

corps members’ schedules for delivering literacy tutoring services (both push-in and pull-out) 

in a number of the schools visited. To begin, schools did not always set regular schedules for 

students’ literacy tutoring sessions. Among the schools that did, corps members reportedly had 

less trouble logging the required number of literacy tutoring hours than did corps members in 

schools with an irregular schedule or with ad-hoc intervention times. Some schools and 

teachers also had inconsistent or unpredictable schedules such that even if an intervention had 

been previously scheduled, the student might not be available at the scheduled time. 

 In site visit interviews, some corps members reported struggling to provide the required 

literacy dosage in schools using the push-in model. In schools where corps members primarily 

pushed into classrooms to deliver literacy tutoring and support, corps members reported that they 

did not usually schedule a time for when push-in interventions would take place. Instead, corps 
40

members reported providing interventions when the opportunity arose within the classroom. For 

example, in a middle school in which corps members provided literacy interventions exclusively 

through push-ins, corps members essentially targeted their in-class supports to their focus list 

students. These corps members reported struggling to find enough 15-minute blocks of time to 

work with their students to achieve the 15 hour dosage goal. 

The few schools visited where corps member teams were able to secure consistent 

intervention time with students in a pull-out setting took similar steps to secure that time. 

First, the corps member team leader negotiated an agreement with the school principal for a 

specific time when corps members would be permitted to pull out students from class. Next, the 

school administration communicated this decision and their rationale for making it to teachers in 

the school. The City Year liaison, who is either the principal or an assistant principal, then 

talked with teachers individually to help them work out any frustrations they had about working 

with corps members. 

 After gaining the backing of the school administration and setting teacher expectations 

appropriately, team leaders and corps members at the schools administering consistent pull-out 

interventions worked with teachers individually to set up a day-to-day schedule for intervention 

time with focus list students. At one of the schools, the team leader led this effort, and set up 

schedules by working with each corps member-teacher pair. At the other school, there was no 

centralized effort to create a schedule for the school, but corps members communicated 

frequently with their teachers and were able to organize their own pull-out schedules.

 Along with gaining permission for pull-outs and setting a schedule that can be followed 

consistently, team leaders and corps members at these schools reported that it was important for 

them to build teachers’ trust so that they would support the time corps members spent with 

students. The team leader at one of these schools said that building teachers’ trust is essential to 

implementing interventions well: 

I stress this to my corps members, the most important thing at the beginning of the year is 

building that relationship with those teachers to get in the practice of sharing your 

lessons, and if they have feedback, showing that you're taking it in so that the teachers 

trust what you're doing with the students and see that it's helping them because they're so 

busy. But I think it definitely makes a difference. And the teachers are much more 

willing to let corps members have their students if they like believe that it's valuable time. 

 At another school, one corps member related how she started her positive relationship 

with her teacher by asking for feedback: 

I was just very open with my teacher and I told her, ‘If I mess up, let me know so I can 

change whatever I'm doing and I can be whatever you need me to be.’ And she was 

grateful. When I do little things that maybe are distracting rather than helpful because I 

think they're helpful, she redirects me. 
41

Monitoring Student Progress 

 In the interest of continuous improvement, City Year encourages corps members to monitor 

the progress of their focus list students as well as to reflect on their students’ work over the course 

of the school year. The following describes corps member efforts to monitor and reflect on student 

progress. 

Types of Student Data Used 

 Most corps members reported using student-level performance data at some point during 

their service year. For example, 86 percent of corps members reported that they used studentlevel grades, assessments, and coursework in some part of their service. Almost all corps 

members who tutored students in literacy in middle schools reported using student-level 

performance data (96 percent); whereas 81 percent of corps members in elementary schools 

reported doing so. This difference was statistically significant (p<.05, effect size=.20). The types 

of data that corps members reported using to monitor students on the literacy focus lists were 

similar across school levels (Exhibit 19), although a greater percentage of middle school corps 

members reported using grades than did elementary school corps members, and this difference 

was statistically significant (p<.05, effect size=.18). 

Exhibit 19 

Types of data used to monitor students on ELA/literacy focus lists 

Types of student data

Classwork/ assignments/ homework 83

Percent of corps members

who provided literacy tutoring

Elementary Middle

(N=86) (N=47)

81

All

(N=133)

82

Informal check for understanding 78 64 73 (e.g., questioning, exit slips, etc.)

Grades 69 85 74

Regular assessments (e.g., DIBELS, 

Anet, MAPS) 20 23 21

Exhibit reads: Eighty-three percent of corps members in elementary schools who provided literacy 

tutoring reported using classwork, assignments, and homework to monitor students’ progress.

Source: Corps Member End-of-Year Survey, Question 35 

In site visit interviews, corps members reported using the QRI-5 to assess focus list 

students’ progress. Corps members administer the QRI-5, which is designed to assess students’ 

reading abilities through a series of diagnostic activities, three times during the school year (i.e., 

in fall, winter, and spring). In addition, corps members reported working with their teachers to 

monitor student progress using course grades (as shared by teachers), as well as teachers’ 

estimates of students’ performance on classwork. In addition to monitoring student progress, 

corps members who used student-level performance data also reported regularly using student 
42

data to plan for their tutoring sessions with literacy focus list students (48 percent strongly 

agreed or agreed that they had done so). 

Tracking Student Data 

City Year provided corps members with dosage tracking sheets and Learner-Leader 

binders to help them track their work with their focus list students. While the dosage tracking 

sheets were relatively simple for corps members to complete, corps members participating in site 

visit interviews gave the binders mixed reviews. 

Dosage tracking sheets. Corps members who participated in site visit interviews 

consistently mentioned using tracking sheets to record the time they spent with students and the 

skills they covered during the literacy interventions. One team leader related how she collects a 

tracking sheet from each of her corps members every Friday, telling her corps members: “’You 

cannot leave this building until you turn in your tracker.’”

Learner-leader binders. Corps members on a number of teams mentioned feeling 

overburdened with the amount of information that they were supposed to track on all their focus 

list students. At one school, tracking information daily was too burdensome for corps members, 

and so they instead filled out the log on a weekly basis and reflected on and planned their work 

during in-school Fridays. Corps members at another school used the learner-leader binders to 

track students’ progress towards weekly and monthly goals (both academic and behavioral) for 

the students with whom they worked, but felt the process was daunting. At yet another school, 

corps members had the option of tracking information in their binders daily or weekly. In site 

visit interviews, no team leader reported actively monitoring how corps members used their 

Learner-Leader binders. Still, in the opinion of one team leader, the Learning-Leader binders 

were a helpful way to see if the tutoring techniques were helping students. 

Communicating with Teachers 

Communication with teachers about student progress varied by corps member. According to 

mid-year survey results, not all corps members met with their teachers to discuss student performance 

data. Indeed, 40 percent of corps members who provided literacy tutoring reported mid-year that they 

had never reviewed student ELA/literacy performance data with their teacher or with another school 

staff member. Nevertheless, site visit interviews revealed the modes of communication some corps 

members were using to communicate with teachers about student progress. 

Formal meetings to track student progress. In some of the schools visited, corps 

members set up formal meetings with teachers a few times a year to discuss student progress. At 

one school, according to a teacher, corps members met with teachers at the middle and end of 

each marking period in order to “see which students were not doing so well and what plan of 

action we can take moving forward.” Team leaders and corps members at two schools described 

completing a profile of their students a few times a year. In City Year’ Reference Guide to 

implementing literacy interventions, City Year recommends that corps members create a start-of-
43

year, mid-year, and end-of-year profile for each of their focus list students that summarizes their 

progress and includes, among other documents, lesson plans and student progress data. At one of 

the schools visited, corps members described sitting down formally with their teachers for a full 

class period and discussing the progress of each student in order to complete their profiles. 

Informal communication about student needs and progress. In addition to meeting 

formally with teachers a few times a year, corps members and teachers reported conversing 

frequently about student progress. Teachers and corps members reported holding informal, dayto-day conversations about students’ comprehension of class lessons, as well as how students 

performed during the tutoring sessions. In addition to helping inform the design of their in-class 

support activities, these conversations with teachers helped corps members determine the 

structure and content of the tutoring sessions. A teacher at one school reported: 

Once the kids leave the room [the corps member and I] were able to talk a little bit more 

informally about how the students did.... So we always have those conversations, like 

Ariel did great, Gene was off task just so you know. You know, he didn’t finish the donow. So he’s always giving me feedback and he’s always keeping an eye on them. It’s 

hard with 34 kids. It’s nice to have another set of eyes in the room.

Teacher Perceptions of Quality and Effectiveness of Literacy Interventions 

The majority of teachers who had City Year corps members in their classrooms felt that 

corps members had helped them to better serve their students. The majority of teachers 

surveyed at the end of the year agreed (45 percent) or strongly agreed (40 percent) that corps 

members helped them to feel supported in their work (Exhibit 20). Similarly, a majority of 

teachers agreed (43 percent) or strongly agreed (35 percent) that corps members helped them 

differentiate their instruction. 

In site visit interviews, several teachers mentioned the importance of the one-on-one or 

small-group support corps members provided in helping them to deliver instruction without 

letting individual students fall behind. One middle school ELA teacher, for example, described 

how her corps member’s support of individual students allowed her to continue instructing the 

rest of the class without interruption: 

[My corps member] was able to sit down with [students who needed extra help, break 

down the article and use the strategies that we discussed in class. And he's able to work 

with [students] on a one-to-one basis and it frees me up to help students with [the next 

element of the lesson]. So it takes a load off of me in a way, because I'm able to move 

forward and not have to re-teach an entire lesson. 

Another teacher described how difficult it would be to give each student individual attention if 

they did not have City Year at their school: 

Every little bit helps. [My corps member has] totally helped me with one-on-one, with 

their writing, they're really struggling writers, so we have to get them to write a certain 
44

way for the test. Again, me being by myself, one-on-one with 23 students would take me 

forever and he could take one-on-one also—not even his group of six kids—just whoever 

I asked. 

 Teachers also reported that corps members had affected the way they interacted with 

students. Twenty-five percent of teachers strongly agreed and 48 percent of teachers agreed that 

corps members had helped them to have a positive relationship with their students. Twenty-four 

percent of teachers strongly agreed that corps members helped improve the quality of their 

interactions with students, while another 26 percent agreed. At one school with a lot of behavior 

management issues, students commented on how corps members had taught them to respect their 

teachers and other adults, saying, “They teach you not to argue with the teacher,” and “Even if 

you don’t like your teacher, you should respect them.”

Exhibit 20

Teachers’ perceptions of the impact 

of corps members in their classrooms (N=123) 

24

25

35

40

26

48

43

45

0 20 40 60 80 100

Improve the quality of my interactions

with my students

Help me to have a positive relationship

with my students

Help me differentiate my instruction

Help me to feel supported in my work

Percent of teachers

Strongly agree Agree

My corps members...

Exhibit reads: Forty percent of teachers strongly agreed that corps members helped them to feel supported in 

their work. An additional 45 percent of teachers agreed that corps members helped them to feel supported in 

their work. 

Source: Teacher End-of-Year Survey, Question 17 
45

V. City Year Outcomes 

As was true in previous evaluation years, school staff believed that City Year positively 

affected their school and students in many ways in 2012-13. Overall, the teachers, City Year 

liaisons, and principals who were surveyed and who participated in site visit interviews pointed 

to numerous benefits for their schools and students from City Year’s work. Student benefits 

included improvements in literacy skills, overall academic performance, behavior, and 

attendance. In surveys and site visit interviews, school staff also reported changes in students’ 

attitudes towards school and academic learning as a result of having City Year in their schools. 

In interviews, school staff and corps members said they believed that the improvements in the 

school climate and school morale were linked to improvements in student achievement. 

Benefits for Schools 

School staff believed that City Year had positive effects on school climate and learning 

environments and were very satisfied with corps member performance. The majority of school 

staff who responded to survey and interview questions were very positive about the overall 

impact of City Year corps members. 

Impact on School Climate 

As in 2011 and 2012, the majority of school staff surveyed and interviewed believed 

that City Year corps members had a positive impact on school climate. In conversations with 

evaluators during site visits, teachers, principals, liaisons, and corps members all mentioned that 

school climate was one of the top areas for visible City Year impact. Interviewees said that City 

Year corps members encouraged students and helped them to feel welcome and valued in the 

school. One elementary school liaison said that the relationships forged between corps members 

and students improved students’ self-confidence which, she believed, ultimately led to improved 

student performance: 

I think this group of corps members has a very good rapport with their children. And 

because of the rapport, sometimes the things that [children] are not able to talk to us 

about, they're able to talk to them about.... Where they're pulling them as well and 

working with them in small groups, it gives the children [confidence to speak up]—

instead of being embarrassed to [speak] inside of the classroom.... It really does assist 

them in growing academically because now they do have an outlet.... They're 

participating now in the classroom because [City Year corps members] gave them 

confidence. 

Among surveyed principals, 77 percent responded that City Year had a great deal of 

positive impact on the school climate, and another 20 percent said City Year had a moderate 

amount of positive impact. Principal and liaison views on City Year’s impacts on school climate 

were similar across school levels, as the differences in responses were not statistically 

significant. Eighty-one percent of responding principals/liaisons from elementary schools said 
46

that City Year had a great deal of positive impact on school climate, and 71 percent of 

responding principals/liaisons from middle schools also felt this way. One middle school liaison 

described the school’s corps members as follows:

They're here early in the morning. They do teacher appreciation, staff appreciation. 

They are always looking for the positive, always doing something for the kids in the 

schools. Bringing that school spirit is what they usually do. 

Teachers viewed the overall impact of City Year on school culture positively as well. 

Sixty-eight percent of all responding teachers reported that City Year had a great deal of positive 

impact on school climate, with another 29 percent responding that City Year had a moderate 

amount of positive impact on school climate. Like principals/liaisons, the views of elementary 

and middle school teachers were similar. Seventy-one percent of elementary school teachers 

believed that City Year had a great deal of positive impact on school climate, and 65 percent of 

middle school teachers said this. 

Impact on the School Learning Environment 

School staff believed that City Year corps members had helped to improve the learning 

environment within their schools. City Year seeks to improve not only the academic performance 

of individual students but also the overall learning environment within schools. Fostering an 

environment where learning is valued and students see the connection between academic performance 

and fulfilling their life dreams is a critical step in moving students onto a path for success. 

In the views of surveyed teachers, corps members had a positive impact on their schools’ 

learning environment: 97 percent of all responding teachers strongly agreed or agreed that City 

Year corps members helped to foster a positive learning environment in their school. Similarly, 

87 percent of surveyed principals/liaisons strongly agreed or agreed that City Year corps members 

served as positive role models for students, and 73 percent strongly agreed or agreed that City 

Year corps members help to establish a college-going and career-aspiring culture in their school. 

 School staff who participated in site visit interviews mentioned that City Year corps 

members helped foster this positive learning environment by serving as role models for students 

and establishing a college- and career-aspiring culture in the school. According to school staff 

who participated in site visit interviews, City Year corps members often came from similar 

demographics and backgrounds as the students they served, many of whom did not have positive 

adult role models in their lives to whom they could turn for guidance. As one middle school 

liaison said: “The students form relationships with corps members and want to do well in their 

classes,” speaking to the fact that students regard corps members as people they want to emulate. 

An elementary school liaison echoed this sentiment, saying, “The kids really look forward to 

spending a lot of time with [City Year corps members]. And they're role models for our 

students. So I think that's something that's not necessarily quantifiable but that really adds to the 

value of having them here.”
47

School Staff Satisfaction with Corps Member Performance 

While the vast majority of surveyed teachers reported being satisfied with City Year 

corps members’ impact on their students, the levels of teacher satisfaction varied within and 

across schools. Overall, 54 percent of teachers said that they were very satisfied, and 39 percent 

reported that they were satisfied with the impact of City Year corps members in their class and 

on their students (Exhibit 21). Although the percentages of elementary and middle school 

teachers reporting that they were very satisfied with the impact of City Year in their class and on 

their students appear to be quite different (60 and 46 percent, respectively), the difference in 

ratings was not statistically significant. 

Exhibit 21

Teachers’ satisfaction with the impact of City Year 

on their class/students, by school level 

60

46

54

34

48

39

0

20

40

60

80

100

Elementary

(N=73)

Middle

(N=48)

All schools

(N=121)

Satisfied

Very satisfied

Percent of teachers

Exhibit reads: Sixty percent of elementary school teachers said they were very satisfied 

with the impact of City Year on their class/students; 34 percent said they were satisfied. 

Source: Teacher End-of-Year Survey, Question 24 

Recognizing that there were several dimensions to teachers’ level of satisfaction, 

evaluators created a Teacher Satisfaction index based on 19 items from the end-of-year teacher 

survey. The index included various survey data, including the extent to which teachers agreed 

strongly about the impact of corps members’ work in their classrooms, the impact of corps 

members on their own work, the extent to which teachers were satisfied with corps member 

performance, and the likelihood that teachers would recommend City Year to a teacher at 

another school. For answers on the highest end of the response scale, such as “strongly agree” or 

“very satisfied,” teachers received one point, and could score up to 19 points on the index.
48

The results of the index analyses suggest that teacher satisfaction varied within schools 

and, to some extent, across schools (Exhibit 22). The average school-level teacher satisfaction 

score across all schools was eight points, or 42 percent of the total possible points. Within-school 

variation, including four schools with individual scores ranging from a low of zero points to a high 

of 19 points, suggests that not all teachers were completely satisfied with corps member 

performance, even if others in their school held very positive opinions. Within schools, evaluators 

expected to see a narrow range of scores, signaling similarities in teachers’ experiences at a 

particular site. Schools with the most variation (e.g., scores ranging from 0 to 19 points), however, 

tended to be those for which a greater number of teachers responded to the survey, suggesting that 

teachers’ experiences within schools might vary significantly. Still, in general, these findings are 

more encouraging than those in the Year 3 report, in which the average teacher satisfaction index 

score was 7 out of a possible 37 points, or 19 percent of the total possible points. Nonetheless, the 

variation within and across schools in the 2012-13 teacher survey suggested that there may still be 

room for improvement in some teachers’ experiences working with corps members. 

Exhibit 22

Teacher satisfaction with corps member performance, by school 

12

18 18

19 19

17

13

18

19 19 19

17

10

4

10

18

19

2

5

6

8 8

9

11

12

13 13

14

2

3

4

7

8

0

1

0 0 0

2

5

6

0

4

5

12

0 0 0 0 0

2

4

6

8

10

12

14

16

18

Sch 15

(N=5)

Sch 8

(N=6)

Sch 11

(N=6)

Sch 9

(N=6)

Sch 10

(N=4)

Sch 18

(N=7)

Sch 17

(N=2)

Sch 13

(N=3)

Sch 12

(N=9)

Sch 19

(N=10)

Sch 16

(N=6)

Sch 14

(N=2)

Sch 5

(N=7)

Sch 3

(N=3)

Sch 6

(N=6)

Sch 7

(N=7)

Sch 2

(N=8)

Maximum score

School average

Minimum score

Teacher Satisfaction Index

Elementary School Teachers Middle School Teachers

Index Score

Exhibit reads: Among the 12 City Year elementary schools, School 15 had the lowest average score for teacher 

satisfaction, with two points. The teacher satisfaction index scores for this school ranged from a low of zero points to 

a high of 12 points. 

Evaluators computed a similar index to measure principal/liaison satisfaction with corps 

member performance by school. The Principal Satisfaction index is based on eight items from the 
49

principal end-of-year survey, including principal perceptions of corps members’ preparedness for 

the academic work in their schools, their views on the overall impact of corps members on their 

school climate, the level of satisfaction with the quality of corps member service and the overall 

experience of working with City Year, and the likelihood that they would recommend City Year to 

someone else in a similar position. Principals/liaisons received one point for answers on the 

highest end of the response scale (e.g., “very satisfied” and “strongly agree”) and could earn up to 

eight points. Across all schools, the average school-level satisfaction score was four out of eight 

points, indicating a moderate level of satisfaction with corps member performance among liaisons 

and principals. 

Site visit interviews with some school staff revealed that staff had observed variation in 

corps members’ work from year to year. One principal spoke highly of her corps members in 2012-

13, saying they had become more integrated into the school than past teams had: “This happens to 

be a very good team this year. They have found ways to become seamless. This year they are very 

seamless. They are part of the whole environment.” A teacher at a different school was also very 

positive about the school’s corps members in 2012-13, in contrast to some corps members from 

years past: 

I’ve had corps members in the past who are not helpful, that needed my guidance. [My 

corps member this year] didn’t need my guidance, she took initiative. She asked what 

can I do in the room.... I didn’t need to be on top of her. [With others], every day, I had 

to ask, can you do this, or that. Or talking to the kids, it became distracting, sometimes it 

was so distracting. I think this year I’ve seen a good pick of the litter. They were a good 

bunch of kids. I haven’t seen such a good group, ever. 

A liaison at another school, who was also a teacher, had observed variation in team leader 

work from year to year as well: 

I've seen years where there was a better [team] leader. [...] I think the leader makes a 

big difference. This year, one of the other members stepped up more as the face for the 

leader where administration would go to him before going to [the team leader]. I know 

teachers weren't happy because the City Year [corps member] wasn't always in the class 

doing stuff. They were just sitting there. But if the leader checked in on them or had more 

of a relationship with the school, that could've been a plus. 

Benefits for Students 

School staff saw improvements in students’ literacy skills, overall academics, and academic 

behaviors for those students who worked with City Year corps members. School staff who 

participated in site visit interviews felt that corps members’ presence in their schools benefitted 

students in many ways. Some mentioned seeing improvements in students’ literacy skills and 

overall academic achievement. Others focused on City Year’s impact on students’ academic 

behaviors, such as more sustained focus, increased participation in class, willingness to ask 

questions, and improved engagement in learning. In interviews, teachers frequently mentioned the 

close mentoring relationships that City Year corps members forged with students during 

interventions and after school. Corps members served as positive role models whom students 
50

connected with in a different way from their teachers. Several students participating in focus 

group interviews mentioned how City Year corps members were able to explain material and 

concepts to them in different ways than was presented in class, or planned fun academic activities 

that helped further their learning. 

Improvement in Student Literacy Skills 

Teachers reported that corps members had improved students’ reading and writing 

skills. When surveyed about the impact of City Year corps members’ literacy interventions with 

their students, teachers responded with positive feedback. Thirty-eight percent of surveyed 

teachers said they strongly agreed, and 52 percent agreed, that City Year corps members helped 

to improve students’ English and language arts performance (Exhibit 23). Elementary and 

middle school teachers responded similarly to this question, with 37 percent of elementary 

school teachers and 43 percent of middle school teachers strongly agreeing that corps members 

helped to improve student performance in English language arts. 

Exhibit 23 

Teachers’ ratings of corps members’ impact on students’ 

English/language arts performance, by school level 

37

43 38

54 43 52

0

20

40

60

80

100

Elementary

(N=68)

Middle

(N=28)

All schools

(N=96)

Agree

Strongly agree

Percent of teachers

Exhibit reads: Thirty-seven percent of surveyed elementary school teachers strongly agreed that corps 

members helped to improve their students’ English/language arts performance; 54 percent agreed. 

Source: Teacher End-of-Year Survey, Questions 9, 11 and 13 

Teachers who participated in site visit interviews shared stories about the positive impact 

that City Year corps members had on their students. For example, one elementary teacher 
51

reported that her students’ skills had improved in reading, writing, and spelling as a result of 

working with a City Year corps member. She spoke about one student in particular: 

I have a student who was held back in third grade prior to being in my class. In the 

beginning of the year, for literally every [task] he was asking “How do you spell this?” 

“How do you make a sentence?” He can now pretty much write two paragraphs; that’s 

a big improvement. That same student can read so much better [now]. City Year has 

worked with him in small groups and individually. 

A middle school teacher also commented on her students’ improved writing skills: 

You can see [that the difference between] their first writing piece [compared with] now is 

day and night. You know, is it the most fabulous writing that you're ever going to see? 

Perhaps not, but it's definitely [evidence of ] growth and it's definitely a lot better. And I 

feel [these students are] a lot better prepared to face the challenges in high school, 

because they've had a lot of support, a lot of people trying to help move them forward.

 A student who participated in a focus group interview also mentioned how City Year 

corps members had helped him: 

They helped me with my reading and writing because when I used to read, I saw the 

words wrong but they helped me with that. Just keep your finger on the word and then 

read what you are saying and they helped me with my writing because when I used to 

write it didn’t make any sense, but they made me write better so I can make more sense. 

Some teachers said that they had seen varying degrees of corps members’ impact on 

students’ literacy skills. A corps member at an elementary school expressed a similar view, 

saying that they had success with some students but not with others: 

I have a group of six. I can honestly say two have improved. One student doesn’t come 

frequently. Another one doesn’t push herself hard enough. [The classroom teacher] and 

I try to push her. She doesn’t use her talent. I want to push her but she’s in that stage 

where she wants to play. 

Improvement in Students’ Overall Academic Skills

School staff saw improvements in the academic skills and behaviors of the students 

who worked with corps members, leading to changes in their overall academic performance. 

Overall, the majority of the teachers, principals, and liaisons surveyed felt that corps members 

positively impacted students’ overall academic performance. Eighty-eight percent of all 

responding teachers strongly agreed or agreed with the statement, “My corps members help 

improve the overall academic performance of my students.” No responding teachers strongly 

disagreed, and teacher ratings were similar across school levels (Exhibit 24). 
52

Exhibit 24 

Extent to which teachers agree that corps members 

helped students’ overall academic performance 

39 36 38

48 52 50

0

20

40

60

80

100

Elementary

(N=71)

Middle

(N=44)

All schools

(N=115)

Agree

Strongly agree

Percent of teachers

Exhibit reads: Thirty-nine percent of surveyed elementary teachers strongly agreed that corps members 

helped to improve students’ overall academic performance. An additional 48 percent agreed. 

Source: Teacher End-of-Year Survey, Question 16 

Similarly, 84 percent of all responding principals/liaisons strongly agreed or agreed that 

the City Year program helped to strengthen students’ academic performance (Exhibit 25). While 

the level of satisfaction reported varied slightly by school level, the difference between 

elementary and middle school principals/liaisons’ ratings was not statistically significant. 
53

Exhibit 25 

Extent to which principals/liaisons agree that City Year 

helped to strengthen students’ academic performance

25

40

32

63

40 52

0

20

40

60

80

100

Elementary

(N=16)

Middle

(N=15)

All schools

(N=31)

Agree

Strongly agree

Percent of principals/liaisons

Exhibit reads: Twenty-five percent of surveyed elementary principals/liaisons strongly agreed that the City 

Year program helped to strengthen students’ academic performance. An additional 63 percent agreed. 

Source: Principal End-of-Year Survey, Question 11 

 While survey respondents were positive about the impact of corps members on academic 

performance, participants in the site visit interviews were less inclined to draw direct connections 

between corps members’ work and students’ academic improvements. They were more comfortable 

discussing how corps members had changed students’ academic mindsets and behaviors, such as 

school attendance and improved academic confidence. Changes in such behaviors may have been in 

part responsible for academic improvement in the school. It was difficult, however, for the school 

staff interviewed to attribute gains to corps members alone, because in most schools, numerous 

reforms and initiatives were underway. For example, one liaison reported: 

From the last time that we looked at the data, I think from the scores last year, I think 

they did show that there was an improvement with the children that did work with City 

Year corps members, from last year's test scores. You know that they were actually kids 

that moved across the continuum, I think. But we had a decrease in level ones and an 

increase in the students that were scoring in level twos and threes. But again, that was–

we had a lot of different things going on. 

In site visit interviews, students frequently mentioned that they viewed the relationships 

that their corps members forged with them as key to their academic improvement. One student 

who was interviewed said that, in general, students liked working with corps members because the 
54

corps members found different ways of explaining materials so that students could better 

understand concepts: “In elementary school, I didn’t like math. When City Year came, they taught 

me tricks to figure it out in a better, cool way.” A middle school teacher said that students often 

found it easier to ask corps members questions rather than their teachers because they felt less 

embarrassed: 

The kids [the corps member is] dealing with often feel like they’re drowning. Some 

wouldn’t ask a question to say they’re not getting it.... Knowing he’s there to talk them 

through it without embarrassment is important. 

Changes in Student Attitudes towards School and Academic Learning 

School staff saw improvements in students’ enjoyment, engagement, and participation in 

academic learning resulting from their work with City Year corps members. Teachers surveyed 

believed that corps members positively impacted their students’ attitudes towards school. For 

example, 90 percent of teachers said that corps members helped to improve their students’ 

motivation to learn, with no teachers disagreeing or strongly disagreeing (Exhibit 26). Similar 

percentages of teachers reported that corps members helped to improve students’ confidence in 

their ability to learn (90 percent) and their active engagement in learning (93 percent). One teacher 

summed up the effects that their corps member had on students in her class with regard to their 

“work ethic,” attendance in school, and academic motivation: 

Definitely the students are trying more. My students love my corps member to death. 

They are excited when she is around. They ask for her when she is not there because 

there are a lot of incentives that City Year offers. [Students think,] ‘I’m going to come to 

school because you are going to have that attendance party.’
55

Exhibit 26 

Extent to which teachers agree that corps members have positively affected 

students’ attitudes towards school and learning (N=119)

28

37

38

39

37

42

53

52

52

56

0 20 40 60 80 100

Attendance.

Enjoyment of school.

Motivation to learn.

Confidence in their ability

to learn.

Active engagement in learning.

Percent of teachers

Strongly agree Agree

Corps members have helped to improve students’...

Exhibit reads: Thirty-seven percent of surveyed teachers strongly agreed that corps members have helped to 

improve students’ active engagement in learning. An additional 56 percent agreed. 

Source: Teacher End-of-Year Survey Questions, 9, 11, 13 and 16 

Several participants in site visit interviews at a school with a City Year afterschool 

program also mentioned a noticeable change in the amount of homework that students were 

completing, citing the structure that the afterschool help provided. Corps members saw this as 

an area in which they had an impact as well. Said one corps member, “That’s where I’ve seen 

the biggest change [among afterschool participants]. I’ve seen a huge attitude adjustment. 

They’re forced to do their homework every night. Most of them fail because they don’t do their 

homework.” 

At another school, teachers participating in site visit interviews said that having corps 

members in the classroom has made students more enthusiastic about school and become more 

confident in their academic skills. An upper-grade math teacher at this school explained that he 

noticed that students who work one-on-one with corps members participate more in class. 

Similarly, a middle school language arts teacher at another school told a story about the change 

in a student’s self-confidence as a result of his participation in a school-wide spelling bee that 

City Year planned: 

I have a student in my class who wasn't sure he wanted to participate [in the spelling 

bee], but we [offered him] a prize [if he did]. So he memorized all the words and how to 

spell them correctly. He was the spelling champ for this class. He was able to 
56

[participate] in the school-wide spelling bee. He didn't win, but that instilled in him a 

sense of accomplishment and, from that day on, [in terms of] his attendance and 

participation in class, he was a completely different kid. It was that one activity that City 

Year structured for the students that instilled so much confidence in him.... One activity 

completely changed his attitude and the way that he projected his voice in class and he 

was like, I can do this, no big deal. 

The majority of teachers (90 percent) also agreed or strongly agreed that City Year corps 

members had increased their students’ enjoyment of school. Student engagement and enjoyment 

in learning are important factors in improving student attendance. More frequent attendance is 

widely known to be correlated with higher student achievement. Over two-thirds (70 percent) of 

teachers strongly agreed or agreed that corps members had helped to improve student attendance. 

During interviews at one school, all the students participating in the focus group said that 

City Year helped them become more interested in attending school each day. One middle school 

student said that, before interacting with City Year staff, she felt that going to school was 

something to do because she had to; now, she is more excited about going to school each day. 

Another student echoed similar feelings about going to school; she explained that knowing that 

the City Year team cares about her encouraged her to come to school each day. A student at 

another school said, “I used to think school wasn’t cool. But [City Year corps members] made it 

fun. It’s the ticket to a good job and college.”

City Year Impacts on Student Achievement in 

Reading/Language Arts 

PSA analyzed the students who participated in literacy tutoring, and City Year 

programming more broadly, to determine the extent to which their participation in City Year 

(City Year dosage) affected their ELA state test scores. The analyses yielded the following 

findings: 

■ For elementary school students, both the number of literacy tutoring hours and the 

number of total programming hours they received positively affected their 

performance on the Common Core ELA test and this effect approached statistical 

significance. 

■ Elementary school students who received more than the average number of total 

dosage hours (i.e., more than 61 hours) performed statistically significantly better 

on the ELA test than those who received fewer than the average dosage hours. 

Methodology of Impact Analyses 

To measure the impact of City Year’s WSWC model on participating students, PSA 

employed hierarchical linear regression modeling (HLM) to differentiate among the various 

levels of influence (e.g., student and school) on student performance and to account for the wide 
57

variability of student and school characteristics at each level that affect student performance. 

That is, HLM addresses the variation in students’ characteristics within and across schools, 

measuring the impact of these characteristics at the school level while allowing for inclusion of 

information that is student-specific. In addition, because of the differences in program 

implementation in elementary and middle schools as identified in site visit interviews, and the 

differences in the number of hours of student participation across school levels, PSA conducted 

separate analyses of the impact of participation in City Year programming by school level. 

City Year provided PSA with student participation and program implementation data. 

Student-level data included the number of hours of literacy tutoring and overall hours of City 

Year programming (literacy tutoring, math tutoring, behavior coaching, attendance coaching, and 

after school programming) each student received. School-level information included index 

scores calculated by PSA based on corps member, teacher, and principal surveys; number of 

students served by City Year corps members within the school; and average literacy dosage rates 

for participating students in the school. 

 Participants were only included in the impact analyses if they received at least five hours 

of literacy tutoring. 

Exhibit 27 

Literacy dosage hours for students included in City Year 

impact analyses, by school level 

Total 

number of 

schools 

receiving 

City Year 

literacy 

tutoring 

services

Total number 

of students 

receiving 

City Year 

literacy 

tutoring 

services

Average 

number of hours 

of City Year 

literacy tutoring 

services 

received per 

student

Minimum 

number of hours 

of City Year 

literacy tutoring 

services 

received

Maximum 

number of hours 

of literacy 

tutoring 

services 

received

Elementary 

Schools 12 360 20 5 50

Middle Schools 7 293 15 5 38

Overall 19 653 17 5 50

 Exhibit reads: At the 19 City Year schools, 653 students participated in an average of 17 hours of literacy 

 tutoring. 

In addition to examining literacy dosage, evaluators analyzed the total number of hours 

that students participated in any City Year activity, including literacy tutoring, math tutoring, 

behavior coaching, attendance coaching, and afterschool programming (although not all schools 

offered all programming types). Total dosage hours reflect the number of hours a student 

participated in any of the various City Year activities over the course of the school year. All 

students included in the impact analyses participated in at least five hours of any City Year 

programming. Some students participated in as many as 270 hours of City Year programming 

over the course of the school. As was the case with literacy tutoring hours, elementary schools 

were able to provide more hours to participating students than middle schools (Exhibit 28).
58

Exhibit 28 

Total City Year dosage hours, by school level 

Number 

of 

schools

Total number 

of students 

receiving 

ANY City 

Year services

Mean number of 

hours of ANY 

City Year 

programming 

received

Minimum 

number of hours 

of ANY City Year 

programming 

received

Maximum 

number of hours 

of ANY City Year 

programming

received

Elementary Schools 12 812 67 5 270

Middle Schools 7 1,005 56 5 226

Overall 19 1,817 61 5 270

Exhibit reads: At the 19 City Year schools, 1,817 students participated in an average of 61 hours of City Year 

programming. 

 Although City Year served a greater number of elementary schools (12 elementary 

schools compared to 7 middle schools), they served a greater number of middle school students, 

averaging 144 middle school students with more than 5 hours of programming per school 

compared to 68 students per elementary school. 

Because City Year teams served a greater number of 

students in middle schools, they provided fewer hours 

per student. 

 

Selecting a Matched Comparison Group. 

The NYC Department of Education (the DOE) 

provided PSA with 2012-13 administrative data for 

the 3,322 students who participated in City Year 

programming in 2012-13; in addition, the DOE was 

able to provide 2011-12 administrative data for 94 

percent (3,154) of those students. Of these students, 

2,666 had complete data for all indicators 

(demographics, school attendance, and state test 

scores) for both the 2011-12 baseline year and the 

2012-13 program year. PSA also received 2012-13 

DOE administrative data for all students in grades 

three through eight from which to draw a comparison 

group for analyses. The DOE data included 

demographic information such as gender, race, 

English language learner status, free and reducedprice lunch status, special education status, school 

attendance rates, and performance on state assessments. PSA used these data not only to assign 

matches for comparison to each of the participating students, but also to control for variation in 

each of the regression analyses. 

 Using a technique known as propensity score matching, evaluators matched City Year 

participants with non-participating students who attended similar schools that had not 

implemented the WSWC model. Propensity matching minimizes the “distance” (i.e., the overall 

Special Note about 2012-13 changes to 

the New York state test

In spring 2013, New York administered a new 

standardized test for its state assessment to 

align with its Common Core implementation. 

Scale scores on the Common Core ELA test 

range from 100 to 425, a change from the 400 

to 800 scale for the New York state 

assessments in previous years. The state 

assigned all students to one of four proficiency 

levels in both 2012 and 2013, but the cut 

points for each level changed for 2013. 

Because the scores are on different scales, it 

is not possible to calculate student growth from 

one year to the next. However, there is a 

statistically significant correlation between 

student performance on the 2012 and 2013 

ELA tests (a=.712, p=.000). This strong 

correlation allows PSA to consider 

performance on the 2012 ELA as a predictor 

for the 2013 test and it is therefore used as a 

criterion for identifying a matched comparison 

group and as a control variable for the 

regression analyses measuring program 

impact on student performance. 
59

difference in group means and/or frequencies on the matching variables) and reduces all 

observed baseline characteristics into one indicator: the propensity score of being a recipient of 

City Year literacy tutoring. PSA selected nonparticipants to match as closely as possible on 

demographic characteristics and baseline achievement, as measured by ELA tests taken during 

the 2011-12 school year. To be included in the matching procedure, participants and students in 

the pool of possible nonparticipants needed to have sufficient data recorded in their DOE 

administrative records.5 To maximize the evaluation’s power to detect statistically significant 

differences, PSA identified three matched nonparticipating students for each City Year 

participant. The comparison group was included in all regression analyses unless otherwise 

noted. 

Ultimately, 1,817 City Year participants—of which 653 received literacy tutoring 

services and had test score data from the prior year—and their 6,309 matched comparisons were 

included in the final analyses.6 Exhibit 29 displays the characteristics of City Year participants 

and matched nonparticipants included in the analyses. 

Exhibit 29 

Characteristics of City Year participants and matched nonparticipants 

Percent of City 

Year

participants

(n=1,817)

Percent of 

Matched

nonparticipants

(n=5,451)

Average school attendance 94 94

Average ELA scale score 656 656

Gender 

Male 54 54

Female 46 46

Race/ethnicity

African American 32 32

Hispanic 59 60

Asian 6 6

Other 3 2

Free and reduced-price lunch recipient 97 97

ELL 20 20

Special education 21 21

 

5 New York’s ELA test is administered to students in grades 3 through 8. Because City Year participants enrolled in 

the third grade for the first time in 2013 (i.e., they were promoted from the second grade at the end of the 2012 

school year) were ineligible to take the ELA exam during this evaluation’s baseline year, they were excluded from 

the matching procedure. 

6

Forty-six students were removed from the analysis because although they were marked as City Year participants, 

their school of record was not a City Year school. This is likely because the students transferred during the school 

year and were recorded as attending the other school. Because PSA could not match these students to school and 

City Year team characteristics, they were not included in the analysis. Another 626 students were removed from the 

analysis because, although they were identified as participants, they participated in fewer than five hours of any City 

Year programming over the course of the school year. 
60

Impact of City Year Service Dosage on Student Performance 

At the elementary school level, the total number of dosage hours a student received of 

ANY City Year activity or service had a positive effect on their ELA scale score and this effect 

approached statistical significance7

(Exhibit 30). That is, each additional hour of participation 

in City Year service or activities was associated with an average of 0.08 more scale score points 

on the state ELA test. 

Exhibit 30

HLM predicting elementary school students’ 2013 ELA

scale scores on the New York stat test, by TOTAL City Year dosage hours 

Fixed Effect Coefficient

Standard

error

Intercept, β0

 Intercept, γ00 -329.62 (16.14)***

Received services for English language learners, β1

 Intercept, γ10 -4.26 (1.08)*** 

Received special education services, β2

 Intercept, γ20 -10.56 (1.16)***

Qualified for free or reduced price lunch, β3

 Intercept, γ30 -6.27 (2.53)*

Student Attendance Rate in 2011-12, β4

 Intercept, γ40 20.70 (6.39)**

2012 ELA score, β5

 Intercept, γ50 0.93 (0.02)***

Hispanic, β6

 Intercept, γ60 -2.84 (2.14)

Asian, β7

 Intercept, γ70 7.53 (2.58)**

Black, β8

 Intercept, γ80 -2.86 (2.27) 

Total Dosage Hours, β9

 Intercept, γ90 0.08 (0.04)†

† indicates p≤0.10; * indicates p≤0.05; ** indicates p≤0.01; *** indicates p≤0.001

Exhibit reads: After controlling for all other variables and for the clustering of students in schools, 

elementary school students scored an additional 0.08 points on their scale score for each additional 

hour of City Year programming. The effect was statistically significant.

 The total number of dosage hours of any City Year service or activity did not, however, 

have a statistically significant impact on the performance of middle school students on their ELA 

test (Appendix B).8 

Elementary school students performed better on the Common Core ELA test with more 

hours of literacy tutoring. As shown in Exhibit 31, the coefficient demonstrating the effect of 

literacy dosage hours for elementary school students is 0.14, meaning that increasing the number 

of literacy dosage hours had a greater impact on the performance of elementary school students 

 

7

 The regression analysis included participants who attended at least five hours of any type of City Year activity, as 

well as their matched comparison group. 

8 The model controlled for students’ English language learner status, special education status, free or reduced-price 

school lunch status, prior year test scores, prior year school attendance rate, and race/ethnicity. The regression 

analysis included participants who attended at least five hours of any type of City Year activity, as well as their 

matched comparison group.
61

than increasing the overall dosage of programming9and the relationship approaches statistical 

significance. 

Exhibit 31

Hierarchical linear model predicting students’ 2013 ELA 

scale scores on the New York state assessment, by City Year literacy dosage 

hours for elementary school students 

Fixed Effect Coefficient

Standard

error

Intercept, β0

 Intercept, γ00 -356.80 (25.96)***

Received services for English language learners, β1

 Intercept, γ10 -4.31 (1.11)***

Received special education services, β2

 Intercept, γ20 -8.97 (1.13)***

Qualified for free or reduced price lunch, β3

 Intercept, γ30 2.06 (2.52)

Student Attendance Rate in 2011-12, β4

 Intercept, γ40 12.17 (6.29)†

2012 ELA score, β5

 Intercept, γ50 0.96 (0.04)***

Hispanic, β6

 Intercept, γ60 -2.50 (2.20)

Asian, β7

 Intercept, γ70 4.85 (2.60)†

Black, β8

 Intercept, γ80 -3.76 (2.30)

Literacy Dosage Hours, β9

 Intercept, γ90 0.14 (0.08)†

† indicates p≤0.10; * indicates p≤0.05; ** indicates p≤0.01; *** indicates p≤0.001

Exhibit reads: After controlling for all other variables and for the clustering of students in schools, 

elementary school students scored an additional 0.14 points on their scale score for each additional 

hour of literacy tutoring. The effect approached statistical significance.

 The number of hours of literacy tutoring did not have a statistically significant effect 

on the ELA scale scores of middle school students.10 Like the total dosage hours of middle 

school students, the number of hours of literacy tutoring a middle school student received did not 

have an effect on their Common Core ELA test performance (see Appendix C). 

 Elementary school students who received more than the average number of total 

dosage hours of ALL City Year programming performed statistically significantly better on the 

ELA test than those that received fewer than the average dosage hours. As shown in the 

regression analysis in Exhibit 32, elementary students who received 61 hours or more of City 

Year programming, the average number of total dosage across all elementary and middle school 

students included in the impact analyses, scored an average of 3.69 more scale score points on 

 

9 The model controlled for students’ English language learner status, special education status, free or reduced-price 

school lunch status, prior year test scores, prior year school attendance rate, and race/ethnicity. The analysis 

includes only students who participated in at least five hours of literacy tutoring and their matched comparison 

students. 

10 The analysis includes only students who participated in at least five hours of literacy tutoring and their matched 

comparison students. 
62

the state ELA test than participating students who received fewer hours than average, and 

comparison students. When analyzing the performance of only City Year students, elementary 

school students who received more than the program-wide average number of dosage hours 

scored an average of 4.32 more scale score points than students who received fewer than the 

average total City Year dosage hours. 

Exhibit 32

Predicting students’ 2013 ELA scale scores on the New York state assessment, 

for elementary school students receiving above average 

dosage hours of ALL City Year service activities 

Fixed Effect Coefficient

Standard

error

Intercept, β0

 Intercept, γ00 -327.76 (16.21)***

Received services for English language learners, β1

 Intercept, γ10 -4.24 (1.09)*** 

Received special education services, β2

 Intercept, γ20 -10.59 (1.16)***

Qualified for free or reduced price lunch, β3

 Intercept, γ30 -6.03 (2.56)*

Student attendance rate in 2011-12, β4

 Intercept, γ40 20.13 (6.25)**

2012 ELA score, β5

 Intercept, γ50 0.92 (0.02)***

Hispanic, β6

 Intercept, γ60 -2.91 (2.13)

Asian, β7

 Intercept, γ70 7.41 (2.59)**

Black, β8

 Intercept, γ80 -2.95 (2.24)

Above average total dosage hours, β9

 Intercept, γ90 3.69 (0.81)***

† indicates p≤0.10; * indicates p≤0.05; ** indicates p≤0.01; *** indicates p≤0.001

Exhibit reads: After controlling for all other variables and for the clustering of students in schools, 

elementary school students scored an additional 3.69 points on their scale score for each additional 

hour of literacy tutoring. The effect was not statistically significant. 

PSA conducted similar regression analyses to determine if elementary or middle school 

students who received more than the average number of literacy tutoring hours than other City 

Year participants performed better on their ELA test than students who received fewer than the 

average number of literacy tutoring hours, as well as for middle school students with above 

average total dosage hours. None of these analyses produced statistically significant results. 
63

VI. Conclusions 

Summary of Findings 

 During the 2012-13 school year, City Year New York deployed teams of corps members 

in 19 schools—12 elementary schools and 7 middle schools—throughout the city to implement 

the Whole School Whole Child model. Corps members in these schools delivered literacy 

tutoring supports to more than 1,000 students and also provided attendance, behavior, and inclass academic supports. Implementation of the literacy intervention component of the WSWC 

model, however, varied from school to school (and sometimes within school) in both form and 

content. 

 Overall, teachers and school staff appreciated the presence of City Year in their schools: 

for example, the majority of teachers surveyed who had City Year corps members in their 

classrooms felt that corps members had helped them to better serve their students. Many 

teachers and school administrators interviewed reported that City Year corps members provided 

needed services to students, both through academic supports and through their “near peer” 

relationships with students. 

 The following section of this report summarizes the study findings with respect to the 

partnerships forged between City Year and participating New York City schools; the quality of 

the corps member teams providing services to partner schools; the various literacy interventions 

City Year offers its partners; and the reported benefits of the partnerships to schools and to 

students. Finally, this chapter concludes with a set of recommendations regarding next steps for 

program implementation and operations. 

City Year-School Partnerships 

■ Setting expectations. Interviews with program managers, team leaders, corps 

members, and school personnel suggest that setting clear expectations for the 

service initiative is the first step toward successful service. Program managers 

described their early conversations with school administrators as having helped 

them define the goals for corps member service, while team leaders and corps 

members met with teachers and other school staff to describe City Year’s mission 

and goals and discuss intervention strategies. 

■ Support from schools. Survey and interview data suggest that corps members 

and team leaders were not integrated as closely into school operations as 

suggested in the school partnership agreement. Several interviewed corps 

members reported that although they felt supported by the teachers with whom 

they worked, for example, the larger school community did not understand corps 

members’ role in the school.

■ Corps member-teacher partnerships. Survey and interview data suggest that 

strong partnerships and effective communication between corps members and 
64

teachers have the potential to contribute to the overall quality of City Year 

services. Interviewed teachers suggested that more can be done at the beginning 

of the year to establish expectations for corps members and teachers. 

■ City Year-school partnerships. Principals and liaisons had strongly favorable 

assessments of the City Year program overall and the services that corps members 

provided to their respective schools, as was true in 2011 and 2012. The vast 

majority of principals and liaisons agreed or agreed strongly that corps members 

worked well with teachers, integrated smoothly into the school, and were well 

prepared for the work that they did. 

Corps Member Team Quality 

■ Corps member training. Corps members preferred the training that took place 

during the school year to training that occurred before the school year began. 

After spending time in schools, corps members had a better understanding of the 

skills they needed and could better apply the lessons learned during training to the 

context of their service. Corps members also reported that training provided by 

City Year did not address some areas of their work, such as intervention strategies 

for push-in tutoring and working with special education students. In particular, 

corps members assigned to middle schools felt significantly less prepared for their 

work than corps members in elementary schools. 

■ Team leadership and support. Team leaders spent most of their time coaching 

corps members and coordinating in-school and after-school schedules. Although 

corps members and team leaders generally reported positive opinions of program 

managers with regard maintaining high expectations, trust and communication, 

program managers who supported two schools often did not have the time to fully 

support both teams. 

Literacy Interventions 

■ Student selection. Most corps members reported that their literacy focus list 

students came from the classroom in which they provided in-class support. Most 

teachers reported that they had at least some role in selecting the students who 

would receive literacy tutoring supports from corps members. Although City 

Year attempted to serve students who were not proficient but were also not the 

highest-need students, in some cases corps members were asked to serve students 

with significant needs. 

■ Literacy tutoring models. Corps members delivered literacy interventions in ways 

that varied both among and within schools: some corps members used a pull-out 

model (working with students in a separate space), while others used a push-in 

model (working inside the classroom), or worked with students during before- or 

after-school programming. In schools in which pull-outs were the primary setting 
65

for literacy intervention, surveyed teachers reported higher levels of satisfaction 

with their corpsmembers’ performance and/or with City Year than did teachers in 

schools where push-ins were the primary setting for literacy intervention. 

Interviews with team leaders and corps members suggested that gaining 

permission from teachers for pull-outs hinged on setting clear expectations for 

City Year’s work in the school, and on building teachers’ trust.

Student participation data for the 2012-13 school year suggested that the extent to 

which City Year corps members achieved the program goal of providing a minimum of 15 

hours of literacy tutoring for their focus list students varied both within and across schools. 

Overall, the average amount of literacy tutoring corps members delivered to the 1,059 students11

served in 2012-13 was 15 hours. Of these students, 689 were in elementary schools and 397 

were in middle schools and corps member teams served an average of 57 students per school at 

each school level. In addition, elementary school students received more hours of tutoring, on 

average, than did middle school students. The average number of hours of literacy tutoring 

received by an elementary school student was 17 hours compared with 11 hours for middle 

school students. This difference was statistically significant (p<.01, effect size=.69). 

As shown in Exhibit 16, the average number of hours of literacy tutoring students 

received by school ranged from an average of five hours per student (School 3) to an average of 

33 hours per student (School 19). In addition, the minimum number of hours ranged from one 

hour of literacy tutoring (Schools 1, 2, 3, 4, 6, 11, 13, and 17) to a maximum of 50 hours. 

Overall, City Year corps members did not achieve the City Year benchmark of providing 

a minimum of 15 hours of literacy tutoring to all focus list students. That is, approximately half 

the focus list students received 15 hours or more of literacy tutoring. Of the remaining half, 

about 28 percent received about 75 percent of the City Year benchmark of 15 hours of 

instruction (7.51 to 14.99 hours of literacy tutoring); 11 percent received about 50 percent of the 

City Year goal (3.76 and 7.5 hours of literacy tutoring); and 11 percent received about 25 percent 

of the City Year goal (one to 3.75 hours of literacy tutoring) (Exhibit 17). 

 By school, the extent to which City Year met the literacy tutoring dosage benchmark 

varied tremendously. In about half the schools, corps members met the literacy tutoring dosage 

benchmark of 15 hours for approximately 50 percent or more of the students. In one school 

(School 2), none of the students received 15 hours or more of literacy tutoring, and in School 3, 

only 2 percent of the students received the benchmark dosage of literacy tutoring. 

■ Content of literacy interventions. In 2012-13, City Year New York dispensed 

with scripted programs such as Great Leaps and Read Naturally, and gave corps 

member teams more freedom to design a literacy intervention that would address 

the specific needs of their focus list students. Surveys of corps members in 

2012-13 revealed far fewer uses of set, structured curricula during literacy 

interventions and more frequent development of their own lessons or usage of 

class material as the starting point for skill work with students. 

 

11 Forty-one students were excluded from the analyses because they received no literacy tutoring (zero hours) and 27 

students were excluded because they received less than one hour of tutoring. 
66

■ Literacy tutoring dosage. Student participation data for 2012-13 suggest that 

City Year corps members did not achieve the City Year benchmark of providing a 

minimum of 15 hours of literacy tutoring to all focus list students. That is, only 

approximately half the focus list students received 15 hours or more of literacy 

tutoring. By school, the extent to which City Year met the literacy tutoring 

dosage benchmark varied tremendously. The percent of students receiving the 

literacy tutoring dosage benchmark of 15 hours ranged from 2 percent to 95 

percent of students in 18 of the 19 schools implementing the WSWC model. In 

one school, however, none of the students received 15 hours or more of literacy 

tutoring. 

■ Monitoring student progress. Most corps members reported using student-level 

data, such as classwork or grades, to monitor students on their literacy focus lists. 

Not all corps members met with their teacher to review student performance data, 

but corps members who communicated with their teachers about student progress 

did so in both formal meetings and in frequent informal conversations. 

Benefits for Schools 

■ School climate and learning environment. The majority of the teachers, 

principals, and City Year liaisons with whom evaluators interacted with this year 

felt very positively about the impact of City Year on their schools. As in the Year 

2 and Year 3 evaluations, school staff mentioned tangible improvements in school 

climate, which they said improved morale among teachers and students. They 

also said that City Year had helped to cultivate a positive learning environment in 

the schools in which students felt more comfortable asking questions and were 

confident about their abilities. 

Benefits for Students 

■ Relationships. Numerous school staff interviewed observed the strong bonds that 

corps members developed with students and reported that corps members served 

as role models for students who lacked other positive role models in their lives. 

■ Academic performance. In surveys and interviews, teachers said that City Year 

corps members had helped to improve their students’ literacy skills, as well as 

students’ overall academic performance. Students as well as teachers discussed 

how the one-on-one or small group support that corps members provided enabled 

students to receive individualized instruction in their areas of weakness. 

■ Attitudes. Echoing teachers’ observations that students’ enjoyment of school had 

increased, students said that corps members had helped them to feel excited about 

learning and coming to school. Teachers reported seeing positive changes in 
67

students’ focus and participation in class, student attendance, and the amount of 

homework that students completed. 

Impacts on Student Performance 

■ For elementary school students, both the number of literacy tutoring hours and the 

number of total programming hours they received positively affected their 

performance on the Common Core ELA test and this effect approached statistical 

significance. 

■ Elementary school students who received more than the average number of total 

dosage hours performed statistically significantly better on the ELA test than 

those that received fewer than the average dosage hours. 

Recommendations 

■ Consider providing additional support to program managers and team leaders as 

they negotiate a City Year-school partnership agreement. If school leaders 

strongly support and promote City Year, teachers might be more inclined to better 

integrate corps members into classroom activities, which might ultimately 

contribute to improved student outcomes. 

■ Continue to differentiate training for corps members working in elementary 

versus middle schools. That is, the WSWC model varies by school level and 

corps members in middle schools, overall, felt less prepared for their work than 

corps members in elementary schools. In particular, help corps members better 

prepare for their service activity by providing them with more information about 

their school assignment as early as possible and targeting training based on those 

assignments. 

■ Continue to draw focus list students from the classrooms in which corps members 

provide in-class support. Working in these classrooms allowed corps members to 

familiarize themselves with their students’ class work and better understand their

needs. In addition, they were able to build relationships and regularly 

communicate with the classroom teachers. 

■ Continue to support corps members as they seek to create and sustain positive 

working relationships with their teachers. Corps members, team leaders, and 

teachers interviewed described the important role that teachers’ trust in corps 

members plays in their work with students. 
Appendix A
 The following tables list the survey items used to calculate the index scores described in 

this report. To calculate each index, we have a respondent one point for every item to which 

they respondent at the most positive end of the answer scale (e.g., “strongly agree” or “very 

satisfied”). We then added up the respondent’s scores on each of the items in the index to create 

a respondent-level index score. To receive a respondent-level index score, a respondent must 

have answered all questions within an index. Finally, we averaged all of all of the respondentlevel scores within each school to create a school-level index score. 

 The indices rely on data from the Start-of-Year (SOY), Mid-Year (MY) and End-of Year 

(EOY) corps member surveys, as well as the Mid-Year and End-of-Year teacher and 

principal/liaison surveys. All surveys were administered by City Year headquarters. 
Corps Member Preparation and Training12

Survey

number Survey items

Value

(0-23 points)

CM SOY Q9 The highest level of education completed by corps members

Associate’s degree=1

Bachelor’s degree=1

Graduate degree=1

Else=0

CM SOY Q12 Whether the corps member has a degree in education

Education=1

Else=0

CM SOY Q19

Whether corps members worked with children as a tutor, classroom 

aide, or classroom teacher prior to joining City Year

Tutor=1

Classroom aide=1

Classroom teacher=1

Else=0

CM SOY Q31

The extent to which corps members feel comfortable tutoring students: 

-in English coursework

-in reading as a skill Very comfortable=1

CM SOY Q32

On a scale of 1 to 7, the extent to which corps member agree with the 

following:

-I have many effective strategies for building positive relationships with 

students

-I feel good about my strategies for handing behavior management 

issues with students in my day-to-day work

-I have a strong base of knowledge about the needs of young people 

at different times in their development

-I have many specific strategies to work with students who are fidgety 

and distracting to others

-I have very good strategies to deal well with students who don’t follow 

rules

6, 7= 1

Else=0

CM MY Q25

How often corps members participated in the following literacy/ELA 

related trainings:

-Training/professional development sessions offered by your school 

district

-Training sessions related to school-based service offered by your site

-Observation and related coaching by your SCM or PM during 

Literacy/ELA sessions

-Observation and related coaching by another City Year staff member 

during Literacy/ELA sessions

-Observation and related coaching by your teacher(s) or another 

school staff member during Literacy/ELA sessions

-Review of student Literacy/ELA performance data with your teacher or 

another school staff member

4 or more times=1

Else=0

CM MY Q38

How prepared corps members feel to perform the following school-based 

activities:

-One-on-one/small group tutoring in literacy

-Whole classroom academic support in ELA or literacy 

-Whole class and/or homeroom behavior support Very prepared=1

Else=0

CM EOY Q44

The extent to which corps members found the following helpful:

-Observation and coaching by PM

-Observation and coaching by Team Leader

Very helpful=1

Else=0

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

10

7

12

 

12 The Corps Member Preparation and Training Index was not included in the 2012 report. 
Corps Member Team Quality: Leadership and Cohesion 

Survey 

number Survey items

Value

(0-16 points)

Item included in 2012 

Index (Yes/No)

CM EOY 

Q41

Corps member reports of how often their teams did 

the following:

-Meet as a team

-Meet to discuss best practices

-Get feedback from each other about what is 

working and what needs improvement in your 

service

-Discuss issues relating to team 

functioning/dynamics

-Meet with the PM

-Meet with the Team Leader

A few times a week=1

Daily=1

Else=0 Yes

CM EOY 

Q42

The extent to which corps members agreed with the 

following statements:

My PM...

-Communicated effectively with me and/or my 

team

-Maintained high expectations of me and/or my 

team

-Helped and/or my team resolve conflicts

-Made me feel like I could trust him/her

-Helped me understand the culture of City Year

-Helped to facilitate leadership development 

experiences

-Facilitated strong relationships with service 

partners

-Was very effective as a leader

Strongly agree=1

Else=0 Yes

CM EOY 

Q43

How often corps members had one-on-one 

meetings with their PM

Every couple of weeks=1

About once a week=1

A few times a week=1

Daily=1

Else=0 Yes

CM 

EOYQ53

Corps members’ overall ratings of their team 

experience

Very good=1

Excellent=1

Else=0 No

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

4

1

9
Corps Member Satisfaction13

 

Survey 

number Survey items

Value

(0-11 points)

CM MY Q35

The extent to which corps members agreed with the following 

statements:

-ELA trainings have increased my knowledge about ELA

-ELA trainings have increased my level of preparedness for service 

delivery 

-ELA trainings have been relevant to my service/school

-ELA trainings have provided knowledge and skills that I can readily 

apply on the job

Strongly agree=1

Else=0

CM MY Q37

The extent to which corps members agreed with the following 

statements:

-Student engagement trainings have increased my knowledge about 

ELA

-Student engagement trainings have increased my level of 

preparedness for service delivery 

-Student engagement trainings have been relevant to my 

service/school

-Student engagement trainings have provided knowledge and skills 

that I can readily apply on the job

Strongly agree=1

Else=0

CM EOY Q53 Corps members’ overall rating of their team experience

Very good/excellent=1

Else=0

CM EOY Q54

Corps members’ overall rating of City Year as a leadership 

development experience 

Very good/excellent=1

Else=0

CM EOY Q55 Corps members’ overall rating of City Year as a learning experience

Very good/excellent=1

Else=0

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

2

1

5

 

13 The Corps Member Satisfaction Index was not included in the 2012 report. 
Teacher Involvement Index14

 

Survey 

number Survey items

Value

(0-7 points)

Teacher

MY Q10

Whether the teacher reports that there was a corps member 

matching process Yes=1

Teacher MY 

Q14

Whether teachers had the opportunity to observe corps members 

perform ELA service Yes=1

Teacher MY 

Q18

Whether the teacher attended an introductory meeting/event hosted 

by City Year at the start of the year Yes=1

Teacher MY 

Q20

The extent to which teachers agree with the following statements:

-I feel well informed about City Year’s mission and goals.

-I am familiar with City Year’s approach to instructional support.

Strongly agree=1

Else=0

Teacher MY 

Q23

The extent to which teachers agree with the following statements:

-I have adequate opportunities to communicate with the team 

leader

-The team leader is responsive to my questions and concerns

Strongly agree=1

Else=0

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

3

1

5

 

14 The Teacher Involvement Index was not included in the 2012 report. 
Corps Member-Teacher Partnership Index (Teacher Perspective) 

Survey 

number Survey items

Value

(0-6 points)

Item included in 2012 

Index (Yes/No)

Teacher 

MY Q8

Whether City Year worked with teachers to identify 

students for one-on-one or small group tutoring Yes=1 No

Teacher 

MY Q20

The extent to which teachers agree with the following 

statements:

-My corps members and I have established clear 

expectations for their work in my classroom.

-My corps members and I meet regularly to review 

their performance.

-I regularly contribute to my corps members’ 

professional development

Strongly agree=1

Else=0 Yes15

Teacher 

MY Q24

The extent to which teachers agree with the following 

statements:

-My corps members work well with me.

-My corps members have integrated smoothly into 

my classroom

Strongly agree=1

Else=0 Yes

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

3

2

5

 

15 In the 2012 report, the item “My corps members and I have established clear expectations for their work in my classroom” was not 

included in the Corps Member-Teacher Partnership Index (Teacher Perspective). 
Teacher Satisfaction 

Survey 

number Survey items

Value

(0-19 points)

Item included in 2012 

Index (Yes/No)

Teacher 

EOY Q16

The extent to which teachers agree with the following 

statements:

My corps members help...

-foster a positive environment for learning

-increase my students’ respectfulness to each other

-increase my students’ enjoyment of school

-improve the overall academic performance of my 

students

-improve overall student focus and order in the 

classroom

-reduce the number of conflicts between students

Strongly agree=1

Else=0 Yes

Teacher

EOY Q17

The extent to which teachers agree with the following 

statements:

My corps members...

-help me have a positive relationship with my 

students

-help me to feel supported in my work

-improve the quality of my interactions with my 

students

-help me differentiate my instruction

-give me more time for planning

-have effective communications from schoolto-home about school programs and student 

progress

-help me to engage parents and families effectively

Strongly agree=1

Else=0 Yes

Teacher 

EOY Q23

The extent to which teachers believe that City Year 

has had an overall positive impact on the climate of 

their schools

A great deal of impact=1

Else=0 No

Teacher 

EOY Q24

The extent to which teachers are satisfied with the 

following:

-The quality of service provided by your City Year 

corps members

-The overall impact of City Year on your 

class/students

-The overall experience of having City Year in your 

school

-The overall training and preparation of corps 

members for the service they provide

Very satisfied=1

Else=0 Yes

Teacher 

EOY Q25

One a scale of 1 to 10, the likelihood that teachers 

would recommend City Year to someone else who 

serves in their position at another organization

10=1

Else=0 No

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

8

2

15
City Year – School Partnership16

 

Survey 

number Survey items

Value

(0-10 points)

Principal 

MY Q12

The extent to which principals/liaisons agree with the following 

statements:

-My school’s priorities and City Year’s initiatives are well-aligned

-We have an agreed upon plan for our City Year team’s initiatives

-City Year staff conducted an orientation for key school/program 

stakeholders to explain its organization and service model

-City Year staff establish an effective process with us to set 

expectations for their work

-We have an effective feedback system in place with City Year 

that allows us to course correct when needed

-City Year staff communicated clearly with us regarding our 

participation in their data collection processes and conducting 

other views of progress.

-Our City Year team provides us with a regularly updated calendar 

to show when team members are present

Strongly agree=1

Else=0

Principal 

MY Q13

How often the school has offered trainings to corps members and/or 

invited them to participate in staff professional activities

More than once a month=1

Else=0

Principal 

MY Q15

The extent to which principals/liaisons agree with the following 

statements:

-I have adequate opportunities to communicate with the team 

leader

-The team leader is responsive to my questions and concerns

Strongly agree=1

Else=0

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

6

1

10

 

16 The City Year-School Partnership Index was not included in the 2012 report. 
Principal/Liaison Satisfaction17

Survey 

number Survey items

Value

(0-8 points)

Principal 

EOY Q13

The extent to which principals/liaisons agree with the following 

statements:

Corps members(s)...

-are well prepared for the academic work they do in our 

school/program

-work well with our teachers and/or staff

-have integrated smoothly into our school/program

-serve as positive role models

Strongly agree=1

Else=0

Principal 

EOY Q16

The extent to which principals and liaisons believe that City Year 

has had an overall positive impact on the climate of their schools

A great deal of impact=1

Else=0

Principal 

EOY Q17

The extent to which principals/liaisons are satisfied with the 

following:

-The quality of service provided by your City Year team

-The overall experience of working with City Year

Very satisfied=1

Else=0

Principal 

EOY Q18

One a scale of 1 to 10, the likelihood that the principal/liaison would 

recommend City Year to someone else who serves in their position 

at another organization 10=1

Else=0

Average School Score:

Minimum School-Level Score: 

Maximum School-Level Score:

4

0

8

 

17 The Principal/Liaison Satisfaction Index was not included in the 2012 report. 
Appendix B 

Total Number of Hours of 

City Year Services Students Received, by School
Exhibit B.1 

Total number of hours of City Year literacy tutoring services students 

received in 2012-13, by participating elementary and middle schools 

School 

TOTAL

number of 

students receiving 

literacy tutoring 

services

Average 

Number of literacy 

tutoring hours 

received per 

student

Minimum 

number of 

literacy tutoring 

hours received

Maximum 

number of 

literacy 

tutoring hours 

received

Middle schools

 1 27 15 5 29

 2

29 9 5 13

 3

17 9 6 16

 4

54 16 5 38

 5

39 16 5 29

 6

39 14 5 30

 7

52 16 6 22

Elementary Schools

 8 39 19 9 32

 9 38 17 11 23

10 45 16 7 29

11 26 12 7 18

12 27 15 10 19

13 21 19 13 28

14 20 17 5 33

15 34 18 6 26

16 33 20 9 34

17 31 22 5 37

18 35 14 6 20

19 47 35 9 50

Exhibit reads: At one school, 27 students participated in an average of 15.2 hours of literacy tutoring. 
Exhibit B.2 

Total number of hours of ANY City Year service students 

received in 2012-13, by participating elementary and middle schools 

School 

TOTAL

number of students 

receiving ANY City 

Year services

Mean Number of 

hours of ANY City 

Year services 

received, per 

student

Minimum 

number of hours 

of ANY City Year 

services

received

Maximum number 

of hours of ANY 

City Year services 

received

Middle Schools

1 99 64 5 224

 2 168 70 5 180

 3 158 101 5 218

 4 142 18 5 52

 5 108 62 5 193

 6 110 23 5 103

 7 137 46 6 226

Elementary Schools

 8 90 64 6 173

 9 91 70 8 220

10 108 97 6 270

11 49 90 7 200

12 66 19 5 53

13 74 40 5 207

14 49 62 5 163

15 34 18 6 26

16 67 57 10 133

17 59 32 5 77

18 113 95 5 264

19 95 67 5 182

Exhibit reads: At one school, 99 students participated in an average of 64.1 hours of City Year programming. 
APPENDIX C 

HLM Analyses for Middle School Students 
Exhibit C.1. 

Hierarchical linear model predicting middle school students’ 2013 

ELA scale scores on the New York state assessment, by City Year 

literacy dosage hours 

Fixed Effect Coefficient

Standard

error

Intercept, β0

 Intercept, γ00 -501.02 (41.65)***

Received services for English language learners, β1

 Intercept, γ10 -6.37 (1.45)***

Received special education services, β2

 Intercept, γ20 -6.45 (1.43)***

Qualified for free or reduced price lunch, β3

 Intercept, γ30 0.17 (3.18)

Student Attendance Rate in 2011-12, β4

 Intercept, γ40 11.98 (7.88)

2012 ELA score, β5

 Intercept, γ50 1.18 (0.07)***

Hispanic, β6

 Intercept, γ60 -2.06 (2.42)

Asian, β7

 Intercept, γ70 3.91 (3.15)

Black, β8

 Intercept, γ80 -6.27 (2.59)*

Literacy Dosage Hours, β9

 Intercept, γ90 0.01 (0.10)

† indicates p≤0.10; * indicates p≤0.05; ** indicates p≤0.01; *** indicates p≤0.001

Exhibit reads: After controlling for all other variables and for the clustering of students in schools, middle school 

students scored an additional 0.01 points on their scale score for each additional hour of literacy tutoring. The effect 

was not statistically significant.
Exhibit C.1. 

HLM predicting middle school students’ 2013 ELA scale scores on the

New York state test, by TOTAL City Year dosage hours 

Fixed Effect Coefficient

Standard

error

Intercept, β0

 Intercept, γ00 -542.77 (22.20)***

Received services for English language learners, β1

 Intercept, γ10 -5.37 (0.90)***

Received special education services, β2

 Intercept, γ20 -8.43 (0.90)***

Qualified for free or reduced price lunch, β3

 Intercept, γ30 0.45 (1.82)

Student Attendance Rate in 2011-12, β4

 Intercept, γ40 16.23 (4.51)**

2012 ELA score, β5

 Intercept, γ50 1.24 (0.03)***

Hispanic, β6

 Intercept, γ60 -1.78 (1.48)

Asian, β7

 Intercept, γ70 4.42 (1.84)*

Black, β8

 Intercept, γ80 -4.63 (1.50)**

Total Dosage Hours, β9

 Intercept, γ90 0.01 (0.08)

† indicates p≤0.10; * indicates p≤0.05; ** indicates p≤0.01; *** indicates p≤0.001

Exhibit reads: After controlling for all other variables and for the clustering of students in schools, middle school 

students scored 0.01 fewer points on their scale score for each additional hour of participation in City Year 

programming. The effect was not statistically significant.