AARP Foundation 

Experience Corps 

Implementation 

Evaluation of 

AmeriCorps-Funded 

Sites 

 December 15, 2016 

Prepared for: 

Kathy Brennan 

AARP Experience Corps 

601 E Street NW 

Washington, DC 20004 

Submitted by: 

Abt Associates 

4550 Montgomery Avenue

Suite 800 North

Bethesda, MD 20814
CONTENTS 

Abt Associates EC AC Implementation Evaluation Report ▌pg. i

1. Introduction ............................................................................................................................................1

2. The Experience Corps (EC) Program Model ......................................................................................2

3. Implementation Research Questions ....................................................................................................3

4. Data Sources and Measurement Plan ...................................................................................................4

4.1 Who was involved .........................................................................................................................4

4.2 Program Director phone interviews ...............................................................................................5

4.3 Site visits .......................................................................................................................................5

4.4 On-site interviews ..........................................................................................................................6

4.5 Tutor and teacher surveys ..............................................................................................................6

5. Evaluation Findings ...............................................................................................................................8

5.1 Research Question 1: How are core program inputs (e.g., tutor training) currently 

implemented across EC AC sites?................................................................................................. 8

5.1.1 EC Input: Staffing ...........................................................................................................8

5.1.2 EC Input: Recruitment ...................................................................................................8

5.1.3 EC Input: Training .........................................................................................................9

5.1.4 EC Input: Management and Coordination....................................................................10

5.1.5 EC Input: Site / School Contribution and Involvement ................................................10

5.1.6 Student Selection and Assessment ................................................................................11

5.1.7 Use of Stipends .............................................................................................................12

5.1.8 Consistency of Program Inputs .....................................................................................12

5.2 Research Question 2: How are the two EC strategies, sustained tutoring and classroom 

literacy assistance, implemented across EC AC sites across the network? ................................. 13

5.2.1 Strategy Delivery ..........................................................................................................14

5.2.2 Setting ...........................................................................................................................17

5.2.3 Materials and Use of Curriculum ..................................................................................18

5.3 Research Question 3: What other factors appear to most influence program 

implementation? ..........................................................................................................................19

5.3.1 In-service Tutor Support ...............................................................................................19

5.3.2 Targeted Use of Curriculum and Instructional Materials ..............................................19

5.3.3 Tutors’ Prior Teaching Experience ...............................................................................20

5.4 Research Question 4: What supports from AARP and the school/site itself are necessary 

for successful program implementation? .....................................................................................20

5.4.1 Additional Tutor Training .............................................................................................20

5.4.2 Access to Curriculum ....................................................................................................20

5.4.3 Tutor and Teacher Interaction .......................................................................................21

5.5 Research Question 5: Which sites are most ready for impact evaluation and why? ...................21

6. Conclusion .............................................................................................................................................23
Abt Associates EC AC Implementation Evaluation Report ▌pg. 1

1. Introduction

In June 2015, the Experience Corps (EC) national office contracted Abt Associates to conduct an 

implementation evaluation that builds on current internal research and answers new research 

questions that have emerged as the EC program has evolved and matured. The overarching goal of 

this implementation evaluation is to better understand what implementation “on the ground” looks 

like for the two EC strategies – sustained tutoring (one-on-one or small group) and classroom literacy 

assistance – and the key internal and external factors that influence how each is implemented in 

AmeriCorps-funded sites. With a more in-depth understanding of how the program is implemented 

across sites, EC stakeholders can improve program delivery across the entire network. 

The three major products of the implementation evaluation are: 1) a refined EC program model, 2) the 

development of a system to measure implementation fidelity, and 3) a qualitative assessment of 

implementation fidelity which enables EC national office to address targeted research questions. 

Results from this implementation evaluation are also being used to inform the EC impact study which 

has been funded by CNCS, with data collection beginning in the 2018-2019 school year. 

The implementation evaluation was conducted in four phases: 

• Phase 1: Further refinement of the EC program model

• Phase 2: Identification of research questions and development of a measurement plan to address

those questions

• Phase 3: Utilization of the measurement plan and engagement in data collection activities

• Phase 4: Analysis of implementation data and reporting of findings
Abt Associates EC AC Implementation Evaluation Report ▌pg. 2

2. The Experience Corps (EC) Program Model

AARP and Abt worked to clearly identify essential program components in a program model. A clear 

articulation of a program’s program model is a critical step in assessing whether any program is 

implemented with fidelity. As expected, the implementation evaluation required the explicit 

identification of the following components: (1) the inputs and supports needed to ensure successful 

implementation (e.g., volunteer training, management infrastructure); (2) key program outputs (e.g., 

sustained one-on-one tutoring, sustained small group tutoring, engagement of students); and (3) the 

experiences and changes hypothesized to occur as a result of participation (e.g., volunteer skill 

development, improved student reading achievement). After identifying those essential elements of 

EC program delivery, the evaluation team possessed a comprehensive framework for assessing 

implementation—determining which components were delivered, how well they were implemented, 

and what challenges were faced by schools, teachers, tutors, and students participating in the 

program. Ultimately, measuring implementation fidelity will help EC program developers understand 

how implementation varies across sites and will guide program developers in refining any program 

elements that have not been implemented as intended. Figure 1 depicts the most recent Experience 

Corps program model. 

Figure 1. Experience Corps program model 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 3

3. Implementation Research Questions

At the onset of the implementation evaluation, EC national staff, in collaboration with the evaluation 

team, identified five primary research questions. These questions and the corresponding analysis and 

conclusions can be mapped to elements of the program model. 

Research questions related to Experience Corps inputs and outputs (see first two columns in the EC 

program model): 

I. How are core program components currently implemented across the network (e.g., tutor 

training, coordination with classroom teachers, program model selection, EC leadership 

structure, tutor stipends, student assessment)? Is there strong implementation 

standardization/efficacy across the network? What program components have the most 

variability and the most consistency in implementation across sites? 

II. How are the two EC strategies – sustained tutoring in a one-on-one or small-group setting 

and classroom literacy assistance – being implemented? 

Research questions exploring factors related to the implementation of the Experience Corps model: 

III. What other internal and external variables appear to most influence program results (i.e., 

degrees of implementation)? 

IV. What supports from AARP and the school/site itself are necessary for successful program 

implementation? 

Research questions addressing program sites’ readiness for rigorous evaluation of effectiveness: 

V. Which sites are most ready for impact evaluation and why? 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 4

4. Data Sources and Measurement Plan

The evaluation team examined these research questions by cross-referencing and analyzing multiple 

data sources. In the initial phases of the implementation evaluation, the evaluation team and EC 

national staff identified viable data sources necessary to capture multiple perspectives of EC 

implementation. These data sources included EC program data and documentation, program director 

phone interviews, site visits and on-site interviews of key staff and tutors, and online teacher and tutor 

surveys. The timeline of data collection, and the corresponding summary reports, is presented in 

Figure 2. 

Figure 2: Timeline of Abt data collection and dissemination activities for EC AmeriCorps 

implementation evaluation 

4.1 Who was involved 

This implementation evaluation involved all eight AC sites who implemented EC in 2015-2016: 

Baltimore, Bay Area, Boston, Minneapolis/St. Paul, New York City, Philadelphia, Portland, and 

Washington DC. Experience Corps New York City discontinued operations in 2016-2017, but 

information about program implementation in 2015-2016 is included in this report. Some data 

collection activities, described more below, took place at every site, while others took place in a 

selection of the eight sites. Table 1 shows an overview of the AC sites including which strategies 

were used where, how many districts, schools, and tutors were involved, and what grades were served 

by the EC program. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 5

Table 1. Description of Implementation and Characteristics of AmeriCorps Sites in 2015-2016 

 

Year 

Program 

Started

EC Strategies

Districts Schools Tutors Tutor 

Stipends

Grades 

Served

 Sustained 

Tutoring Liter. 

1-on- Assist. 

1 

Small 

Grp 

Baltimore 1998   1 27 284 98% Pre-K-3 

Bay Area 1998    3 22 169 14% K-5 

Boston 1998    2 19 266 69% Pre-K-3 

Minneapolis/

St. Paul 1995    2 15 87 87% K-3 

New York* 1996   3 6 49 100% K-3 

Philadelphia 1996    1 26 275 90% K-3 

Portland 1995    7 9 45 66% K-5 

Washington 

DC 1999    1 9 92 77% Pre-K-4 

*Experience Corps in New York City closed after the 2015-2016 school year 

4.2 Program Director phone interviews 

In the first phase of the evaluation, Abt staff reviewed annual EC program data and conducted a series 

of eight phone interviews in August 2015 with EC program staff from each AmeriCorps-funded site. 

The results of these interviews were provided in Program Model Case (PMC) summaries for each 

site, which describe tutoring strategies and delivery mechanisms, including recruitment, training, tutor 

coordination, strategy implementation, staffing, participant relationships, tutor assignments to 

classrooms, student assignment to tutors, and challenges faced. The PMCs provide a “snapshot” of 

programming across EC sites and have been periodically updated to reflect changes in program 

implementation within each site. Abt staff conducted a second round of Site Director interviews in 

October 2016 to confirm the accuracy of the PMCs and capture changes in programming in the 2016-

2017 school year. The updated PMCs for all AmeriCorps sites (excluding New York) were submitted 

to AARP in November 2016. 

4.3 Site visits 

The Abt team conducted five site visits between January and March 2016 to gather more in-depth 

qualitative data about the EC model and its implementation in select sites. These site visits were 

conducted in Boston, Massachusetts; Oakland, California; Philadelphia, Pennsylvania; Baltimore, 

Maryland; and Portland, Oregon. These site visits enabled the Abt team to gather first-hand 

observations of EC program delivery and to determine whether elements included in the current 

program model were being implemented in participating schools. The site visits also provided 

opportunities to identify elements that should be included in the EC model but were not at the time. 

The Abt team summarized key takeaways from the site visits in the AmeriCorps memo which was 

presented to EC National in March 2016. Table 2 shows the distribution of data collection across AC 

sites from the site visits. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 6

Table 2. Distribution of School Visits, Interviews, and Observations across AmeriCorps Sites 

Boston Bay Area Philadelphia Baltimore Portland 

Total 

(across 

sites) 

Site Interviews 

Program Director 1 1 1 1 1 5 

Other EC Staff 1 1 3 2 1 8 

Principal / Leader 2 3 1 2 2 10

Teacher 0 3 2 1 0 6 

Tutor 0 1 2 2 5 10

Total Interviews 

(per site) 4 9 9 8 9 39

Site School Visits 

School Visits 2 3 2 2 2 11

Total School Visits 

(per site) 2 3 2 2 2 11

Site Observations 

Sustained One-onOne 1 1 5 0 4 11

Sustained Small 

Group 4 1 1 0 3 9 

Literacy Assistance 1 0 1 6 0 8 

Total 

Observations 

(per site) 6 2 7 6 7 28 

4.4 On-site interviews 

Each site visit included a number of data collection activities (Table 2). In order to get a range of 

perspectives and experiences related to key EC inputs like staff roles, recruitment, training, tutor 

management, and decisions around strategy and targeted student selection, the Abt team interviewed a 

number of different individuals depending on the structure of each site. Interviews typically included 

the following key EC stakeholders: 1) EC Program Directors, 2) Additional EC Staff Members, 3) 

School Principals, 4) K-3 Classroom Teachers, and 5) EC Tutors. In addition, Abt team members 

observed tutoring sessions at each program site. Direct observation provided substantial opportunity 

to understand EC strategy delivery and to assess the experience of students receiving assistance 

and/or tutoring. The Abt team used the same data collection tools and protocols across the 

AmeriCorps site visits. 

4.5 Tutor and teacher surveys 

The Abt team designed online EC tutor and teacher surveys in March 2016 to supplement the 

information collected during the five site visits. All EC tutors in the eight AmeriCorps sites 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 7

(Baltimore, Bay Area, Boston, Minneapolis/St. Paul, New York City, Philadelphia, Portland, and 

Washington DC) whose contact information was accessible in the EC Salesforce database were 

emailed the survey and given the option to respond. Teachers from five AmeriCorps sites (Baltimore, 

Bay Area, Minneapolis/St. Paul, Portland, Washington DC) were randomly selected to participate in 

the survey. The surveys were sent to 1,009 tutors and 288 K-3 classroom teachers in AmeriCorpsaffiliated sites on April 4, 2016, and the survey remained open until April 25, 2016. By the time the 

survey closed, the Abt team received 526 EC tutor responses and 61 teacher responses. The Abt team 

analyzed responses across and between sites, as well as for certain tutor/teacher subgroups (e.g. tutors 

who implement sustained small groups, tutors with prior teaching experience). Preliminary findings 

were presented in the Experience Corps Sustained Small Group Topical Report published in July 

2016. The Abt team published the final Experience Corps Survey Results Memo in November 2016. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 8

5. Evaluation Findings

Results from the implementation evaluation are presented here in conjunction with the associated 

research question. The evaluation team used the previously described data sources to evaluate each 

research question and identify differences in implementation across all eight sites. The team 

conducted site visits to only five sites; however, the sample of sites visited was designed to be 

representative of the full EC AC network. The first two research questions corresponded to program 

inputs (e.g. staffing) and outputs (e.g. strategy selection) as shown in the EC program model in Figure 

1. Program outcomes were not a focus of this evaluation. The third and fourth research questions

required the research team to identify internal and external factors that influence program

implementation and best support tutors. Finally, the fifth research question addressed future impact

evaluation plans and required the evaluation team to identify sites that are “ready” for the next impact

phase in the larger Abt EC study.

5.1 Research Question 1: How are core program inputs (e.g., tutor training) 

currently implemented across EC AC sites? 

One of the goals of this implementation evaluation was to examine the variation in inputs across sites 

and compare what is being done “on the ground” with what is outlined in the program model. 

According to the EC program model provided above in Figure 1, there are six key program inputs: 

funding, staffing, recruitment, training and coaching, management/coordination, and site/school 

support. These inputs, or resources invested in the program, have been identified as essential for 

delivering EC services. The following section will summarize key findings across EC AC sites and 

highlight unique ways that certain sites have adapted the EC model. Ultimately, AC sites appear to 

implement these inputs with varying levels of fidelity, and factors like program size, strategy type, 

and school leadership affect how these activities are carried out. 

5.1.1 EC Input: Staffing 

Depending on the size of the program, the staffing structure varies in size and scope. In Portland, 

which only serves nine schools across seven districts, the program coordinator/director serves as the 

primary contact for school staff and EC tutors. The Portland coordinator is involved in day-to-day 

tutor management activities (e.g., interacting with teachers, collecting timesheets, and discussing 

make-up hours), as well as general program support (e.g., tutor training, school recruitment). Larger 

sites may utilize a more complex staffing structure which may include a Branch Director, Program 

Director, Recruitment Coordinator, and several Team Leaders. EC Philadelphia, one of the largest 

branches in the EC network with 275 tutors serving 6,306 students in 26 schools, has 7.5 full-time EC 

staff members who each fulfill a unique role in program delivery. In several EC sites, EC leadership 

has recently increased the use of literacy coaches and other instructional personnel to train and 

support tutors in a more pedagogically-focused manner. Both Philadelphia and Boston demonstrate 

very promising approaches to integrating these staff into existing training and scheduling frameworks.

5.1.2 EC Input: Recruitment 

Program staff use a variety of strategies to recruit and retain new tutors, including but not limited to 

mailings/email blasts, information sessions, and print or radio/TV advertisements. In addition to these 

efforts, current EC tutors act as a primary recruiting tool by spreading the word about EC to other 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 9

potential tutors. Consequently, recruitment “by another EC tutor” serves as the most frequently cited 

way (27%) for EC AC tutors to learn about EC and decide to enroll in the program (see Table 3). 

Other successful strategies include AARP state mailings (22%) and community newspaper/print 

advertisements (17%). Across EC sites, program staff are heavily involved in tutor recruitment. 

Larger sites may have a designated Recruitment Coordinator who is responsible for leading program 

recruitment efforts. 

Table 3: Primary Recruitment Strategies Used across EC AmeriCorps Sites 

Recruitment Strategy Tutor Survey Respondents (%) 

By another EC tutor 27.3 

EC information session 11.3 

AARP state mailings 22.0 

Community newspaper/ print advertisement 17.4 

Radio or TV advertisement 2.9 

Social Media 0.80 

Online volunteer database 6.7 

Other (e.g., other advertisements, Internet 

searches, community fair, word of mouth) 

24.2 

Source: Tutor Survey Q4. How were you recruited to be an EC tutor? (n=524) 

*Percentages sum to more than 100% because tutors were able to select more than one response 

5.1.3 EC Input: Training 

Pre-service and ongoing tutor training is delivered in a fairly consistent manner across all five sites. 

EC Portland provides 12 hours of training to all tutors and an additional half-day training to new 

tutors in the later summer / early fall. Tutors are expected to participate in an additional 10-12 hours 

of training throughout the year. In Boston, tutors receive extensive pre-service and ongoing training 

that aligns with the instructional practices and content area priorities of Boston Public Schools (BPS). 

Team leaders often observe tutors and provide feedback on a weekly basis. EC Bay Area currently 

offers tutors a three-session pre-service training, monthly meetings with the Site Coordinator, and 

feedback from observations conducted by the Site Coordinator. EC Bay Area recently hired a literacy 

trainer who is revamping training materials, activities, and opportunities for peer-tutoring 

observations. EC Philadelphia also augments initial tutor training with monthly meetings – almost 

half of which are facilitated by new literacy coaches and focus on instructional strategies. EC 

Baltimore follows a similar approach and plans to roll-out a new training model that organizes tutors 

into grade-level cohorts and trains these tutors together. Tutors in Baltimore have recently also had 

the opportunity to attend trainings delivered to Baltimore City Public Schools (BCPS) teachers, and 

EC staff have also provided trainings for BCPS teachers. This cross-collaboration between BCPS and 

EC Baltimore has helped the program align their tutor training with classroom instruction. Across 

most sites, we observed a very systematic use of literacy coaches as well as Site Coordinators and 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 10

Team Leaders with strong backgrounds in literacy instruction to provide tutors with necessary 

pedagogical training and support. 

5.1.4 EC Input: Management and Coordination 

Team Leaders, veteran EC tutors who supervise other EC tutors and facilitate meetings, often provide 

on-the-ground leadership and management. In EC Baltimore, for example, EC tutors interact most 

frequently with teachers and onsite Team Leaders. The same was true in EC Philadelphia, where 

tutors most frequently interact with Team Leaders, literacy coaches, classroom teachers, and other 

tutors. Tutors look to the Team Leader for logistical support (e.g., keeping track of hours worked, 

stipend checks, etc.) as well as instructional mentoring. In EC Bay Area, Site Coordinators fulfill 

similar responsibilities to what was expected of Team Leaders. Site coordinators in EC Bay Area 

interact with tutors on a regular basis and are available to provide increased support, especially to new 

tutors. This intensive ongoing tutor management and support is a key feature of EC Bay Area, and EC 

leadership in this site hold tutor management at the forefront of their site-specific program model. 

Across sites, EC staff also use detailed documentation of tutoring sessions (e.g., session length, 

content) and tutor hours to monitor and manage programming. EC Boston, Bay Area, Philadelphia, 

Baltimore, and Portland all require tutors to log hours of tutoring per day and per week, as well as 

number and length of literacy assistance sessions, and tutoring sessions with specific students. During 

site visits, EC staff from all five sites indicated they use the paper-format tracking logs to document 

assistance and tutoring time. Across the EC network, sites have started transitioning to electronic 

monitoring systems like the FIS and New Volunteer Portal (NVP) in the 2016-2017 school year. The 

Abt evaluation team was able to observe the implementation of this new process in the two schools 

visited during the site visits to EC Philadelphia. Each Team Leader made a Kindle e-reader available 

to each tutor throughout the day to enter hours into a Kindle-compatible app. Despite the fact that EC 

program staff across the network are concerned that older tutors may have varying abilities and 

comfort levels in using technology, EC Philadelphia has made the process more accessible to this 

population. By using Team Leaders as facilitators of both the equipment and the process, EC 

Philadelphia has successfully made the transition from paper and computerized session logging. 

5.1.5 EC Input: Site / School Contribution and Involvement 

Across the five visited sites, school staff (e.g., teachers, principals, literacy specialists) play varying 

roles in supporting EC implementation. Schools allocate money for EC services in different ways, 

pulling from specialized pools of money (e.g., Title I) or from the general school budget. For 

instance, principals in Baltimore noted that Title I funds can be used to fund the program, and this 

was an important funding source for EC services. In Portland, however, schools are not required to 

pay for EC services. Principals across sites have been highly supportive of the EC program in their 

schools regardless of the monetary investment required by schools. However, the level of principal 

involvement in the day-to-day EC activities varies from school to school and is dictated by the 

priorities and experience level of the principal in that particular school. Through the site visits, the 

evaluation team identified extensive principal involvement in Boston, Philadelphia, Baltimore, and 

the Bay Area. The team also observed authentic connections between tutor, teacher, and school 

resources in Boston and among tutors and EC Team Leaders in Philadelphia. 

Teachers also provide varying levels of guidance and instructional materials to EC tutors. According 

to tutor survey results, tutors most frequently turn to teachers for guidance when they have questions 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 11

(see Table 4). The specific level of teacher involvement varies widely across sites. In some sites, 

tutors work closely with classroom teachers and incorporate classroom-based resources into their 

tutoring sessions. In EC Baltimore, which primarily operates under the literacy assistance strategy, 

tutors work closely with teachers within the same classrooms. Consistent with the literacy assistance 

strategy, tutors in EC Baltimore serve as a general resource for the classroom and its teachers. In 

other sites, tutors bring materials from home or look to the Team Leader for resources. In Portland, 

tutors are given an “EC toolkit” (i.e., bags with books, games, and other instructional materials) and 

are free to use classroom materials, but tutors are fairly autonomous and rarely consult with the 

classroom teacher when using classroom materials 

Table 4: EC or School Staff Members Providing Instructional Support 

EC or School Staff Member Percentage of Tutor Survey 

Respondents (%) 

EC Program Director 2.7 

EC literacy coach 4.6 

School principal 0.0 

School literacy coordinator 3.8 

School teacher 38.7 

Other EC tutor 3.4 

Don’t know 1.2 

EC team leader 25.8 

Other EC program staff (e.g., site coordinator, site liaison) 16.6 

Other (Please specify) 3.2 

Source: Tutor survey item: If you have a tutoring or academic content question, who would you ask for 

guidance from first? (n=524) 

5.1.6 Student Selection and Assessment 

Classroom teachers are often responsible for selecting students to receive Experience Corps tutoring. 

Especially in sites that focus more heavily on literacy assistance (e.g., Baltimore), the delivery of 

tutoring and additional classroom assistance is largely driven by teachers. In these cases, tutoring 

assistance can be either focused on a single concept (e.g. fluency, phonemic awareness) or on a more 

long-term goal to improve reading skills (e.g. reading at grade level). In one site, Bay Area, students 

are selected to receive Experience Corps tutoring either by a classroom teacher, principal, or by a 

literacy coach who works with struggling readers schoolwide. EC Bay Area staff set clear 

expectations with school personnel about selecting Tier 2 students (in a RTI framework) and referring 

students with more intensive academic needs to other specialized programs. 

Regardless of the primary intervention strategy, students selected to receive sustained Experience 

Corps tutoring generally have a similar profile: they are struggling readers at least one year behind 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 12

grade level. Teachers, literacy coaches/specialists, and other EC or school staff frequently utilize 

student assessment data to identify eligible students. Student performance on standardized 

assessments like the DIBELS, AIMSweb, DRA, and the Fountas and Pinnell heavily influences 

student selection across AmeriCorps sites. In the literacy assistance strategy, students are often 

grouped into ability levels within classrooms and provided the appropriate level of assistance for their 

needs. Assessment data less consistently influences student selection in classrooms implementing 

literacy assistance only. 

5.1.7 Use of Stipends 

Across AmeriCorps sites, the majority of tutors (65%) receive a stipend, but over one third (36%) of 

tutors receive no source of funding. Within sites, stipends may either be provided to all tutors or given 

to a select group of tutors. For example, Boston provides stipends for 69% of all tutors regardless of 

funding, while Bay Area only provides stipends to AC-funded tutors. Some sites (e.g., Philadelphia, 

Baltimore) provide stipends to all tutors regardless of tutoring hours completed, but stipend amounts 

differ based on tutors’ time commitment to the program. In EC Baltimore, tutors must serve a 

minimum of 10 hours per week, but quarter-time and reduced-half-time options are also available. 

AC tutors on minimum time schedules (10 hours/week) receive a stipend of $150 per month, and 

stipends rise to $225 per month for a quarter time commitment and $300 per month for a reduced 

half-time commitment. 

There is some disagreement among interviewees as to whether using stipends increases recruitment or 

retention rates. In particular, some EC staff expressed that the use of stipends increased the 

professionalism and the credibility of the program. While future research should investigate whether 

the use of stipends can lead to increased recruitment and retention, interviews with EC staff and tutors 

provided more in-depth information about the importance of stipends for certain tutors. For tutors in 

several communities, the stipend represents a consistent source of income that may be used to cover 

basic living expenses. 

5.1.8 Consistency of Program Inputs 

At AC sites, the inputs underlying the EC model were implemented in line with EC standards when 

applicable. Many of the EC standards, however, are intentionally broad since maintaining program 

flexibility and adapting to local school needs is a hallmark of the EC model. For instance, EC 

standards require a minimum of 25 hours of annual training for new volunteers and 20 hours of 

annual training for returning volunteers. The annual training requirements can involve a mix of preservice and in-service training, and there is evidence that each EC site offers a different mix of these 

trainings. This local flexibility was coupled with a deliberate effort on the part of site leadership to 

maintain fidelity to EC standards; all but three AC sites exceed annual training standards. Moreover, 

the flexibility afforded to EC sites allowed for local innovations such as linking EC training to district 

curricula (Boston), hiring professional literacy trainers to modify the training curriculum (Bay Area), 

enlisting literacy coaches (Philadelphia), and organizing training into grade-level cohorts (Baltimore). 

While we observed the most variability among inputs in training, staffing arrangements also varied 

substantially depending upon the size of the EC site. As expected, larger EC sites (e.g., Philadelphia) 

had greater division of responsibilities while smaller EC sites (e.g., Portland) required a smaller 

number of staff that fulfilled multiple concurrent roles. Despite the differences in staffing 

arrangements, we observed a great deal of similarities in the leadership structures across sites. All EC 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 13

sites had central office coordination with site-level leadership (fulfilled by site coordinators, Team 

Leaders [typically senior tutors], or even project directors in some cases). Moreover, tutors 

consistently kept session logs of sustained tutoring activities, though the content of those session logs 

differed across sites. Classroom teachers were generally a consistent source of support and 

coordination as well. 

A key remaining challenge for the evaluation is to specify measures of implementation fidelity that 

provide guidance on how to further define the EC model while not hindering local innovation. Many 

of the EC standards pertaining to program inputs are very broad (e.g., training requirements of 25+ 

hours per year for new tutors), focused on general site management (e.g., consistent branding for site 

recruitment activities), or are simply not existent (e.g., staffing requirements). It is clear that the 

implementation fidelity metrics must be developed in a manner that is consistent with EC standards; 

however, the current EC standards should not be the sole source of our fidelity rubric. 

5.2 Research Question 2: How are the two EC strategies, sustained tutoring 

and classroom literacy assistance, implemented across EC AC sites 

across the network? 

Across the network of AmeriCorps-funded sites, tutors deliver a combination of EC strategies. The 

two EC strategies include 1) sustained tutoring in a one-on-one or small-group setting and 2) 

classroom literacy assistance. Four of the five visited EC sites deliver both strategies and delivery 

approaches (one-on-one and small group), and in many instances these strategies are used in 

combination with one another depending on the site, school, and population of students. Figure 3 

displays a map of all eight AmeriCorps-funded sites and EC strategies currently delivered by each 

program. EC National has recently advised sites to increasingly implement sustained small group 

tutoring in order to serve more students, but the five visited AC sites are in varying stages of making 

this transition. One-on-one tutoring remains the most common mode of EC service delivery across 

AC sites, except in EC Baltimore where a blended form of literacy assistance is the predominant 

approach. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 14

Figure 3. Experience Corps strategies delivered across AmeriCorps-funded sites 

The five visited AC sites have also implemented each strategy to varying levels of fidelity, often 

because of differences in tutor training, resources, and site leadership. EC National has produced 

guidelines for each strategy, but as will be discussed in the following section, each strategy is 

implemented in slightly different ways. The Abt evaluation team observed a substantial amount of 

variation in the delivery of specific components of each strategy across AmeriCorps sites, but also by 

tutors within sites. As expected, we found this to be true for the delivery of literacy assistance, a 

strategy that promotes tutor flexibility to provide assistance as needed for any particular lesson in any 

particular day. However, we also found this to be true when tutors implement sustained small group 

tutoring and even sustained one-on-one tutoring, a strategy that is more prescriptive at its core. Each 

strategy will be discussed individually as defined by AARP’s EC National standards, and then 

compared with data collected in interviews, observations, and surveys. 

5.2.1 Strategy Delivery

Sustained tutoring 

Sustained one-on-one delivery 

Sustained one-on-one tutoring is one of two delivery approaches for the EC-endorsed sustained 

tutoring strategy. As defined by AARP’s EC standards, this strategy is delivered when an individual 

student meets with a tutor for two to five times per week with sessions lasting 20-40 minutes 

depending on age and need. The matched pair remains together for a sustained period with a target of 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 15

35 sessions throughout the school year. Students referred for one-on-one tutoring are below grade 

level in one or more of the foundational literacy skills or in overall reading ability. All students 

receiving tutoring should have attainment goals established by the classroom teacher or another 

relevant school staff or administrator (teacher, literacy specialist, liaison), and are tracked for progress 

through pre- and post-tutoring surveys. 

One-on-one tutoring has historically been the most common EC strategy and is used in four of the 

five visited AC sites. The Abt team’s observations of one-on-one tutoring generally matched the 

strategy as prescribed by EC National. EC Baltimore was the only AC site that did not use sustained 

one-on-one tutoring, but tutors worked individually with students while providing classroom-based 

literacy assistance. Across the other four visited AC sites, one-on-one sessions typically lasted for 30 

minutes and occurred 2-3 times per week. Generally, tutors appeared to work with the same students 

across the entire year. Tutors typically worked with students in grades K-3, but the Abt team saw 

evidence of tutors working with students in grades 4-5 in one EC AC site. A number of AmeriCorps 

sites, including EC Boston, Baltimore, and Washington DC, have expanded into Pre-K. Although 

teachers were often responsible for identifying eligible students based on test scores or other factors, 

tutors were often responsible for determining the focus and content of one-on-one sessions with 

minimal guidance from the classroom teacher. 

The Abt team observed one-on-one sessions both in the classroom and in separate spaces (e.g., 

hallway, library). Sessions taking place inside the classroom often occurred at a designated table or 

corner. The content of the observed tutoring sessions also varied widely depending on student needs 

and abilities. For example, in EC Boston, the Abt team observed a tutoring session that touched on 

more rudimentary literacy skills, incorporating elements of print awareness, reading fluency, 

phonetics, vocabulary, and comprehension. In this session, the tutor worked individually with one 2nd

Grade student at a desk outside of the classroom. The teacher provided the tutor with specific 

activities to use with the student (a story writing worksheet and flashcards with words to read). The 

Abt team observed another tutoring session in EC Bay area that focused more exclusively on reading 

comprehension and advanced vocabulary. During this session, the tutor prompted the student to read 

one page at a time and then pause, after which the tutor facilitated a discussion about the vocabulary 

words they encountered and then asked reading comprehension questions. Some of the vocabulary 

words included, “precious,” “mumble,” and “plump.” The tutor added to the student’s cultural 

knowledge as well, explaining how to cook a “fried” and “poached” egg (both referenced in the 

story). 

Tutors also used a variety of materials in the observed sessions. While traditional printed books were 

the primary teaching tool, the Abt team also observed multiple instances in which tutors used iPads, 

worksheets, flashcards, “EC toolkits” (i.e., bags with books, games, and other instructional materials), 

and other classroom materials. For instance, in EC Portland, the Abt team observed tutors using an 

assortment of materials and resources. Unlike other sites, tutors in Portland did not use any 

commercial, published curricula since the Common Core guides the instructional focus of Portlandarea schools. During our observations, one Portland tutor used a bingo game from the classroom, and 

another tutor from the same school pulled materials from her EC bag. Tutors in other schools and 

sites may integrate more “high-tech” resources into their tutoring sessions. In EC Philadelphia, tutors 

are encouraged to use iPads or other technological teaching tools to deliver tutoring. We observed 

several tutors using iPads across the two schools involved in the site visit. EC leadership indicated 

that they used program funds to purchase a series of inexpensive literacy-based apps for each iPad. As 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 16

technology continues to become a larger part of education, the use of iPads and other assistive devices 

may be an effective tool for engaging students across all EC sites. 

Sustained small group delivery 

This delivery approach of the sustained tutoring strategy is adopted when one tutor is matched with a 

group of up to 4 students. The group of students should meet with a tutor two to five times per week 

with sessions lasting 20-40 minutes depending on age and need. The group remains together for a 

sustained period throughout the school year, or throughout the course of the after-school program. 

Like one-on-one tutoring, students referred for group tutoring are below grade level in up to three of 

the same foundational literacy skills and should have specific attainment goals. 

The five visited AC sites have taken different approaches to integrating sustained small group 

tutoring. Four of the five sites offered sustained small group tutoring, and in each of those sites, at 

least one sustained small group tutoring session was directly observed by the Abt team. Tutors 

generally worked in groups of 2-4 students, but there were instances where tutors worked with five 

students at a time. Across the eight observed sessions, the Abt team observed a direct relationship 

between behavior management and group size—as the size of the small group increased, the amount 

of time spent on behavior management also increased. Tutors also adapted their instruction to engage 

multiple students at once. Many tutors played literacy games or facilitated interactive activities with 

their students to keep every member of the group focused and engaged. In a few cases, the Abt team 

observed tutors working with a small group on materials that might have been more appropriate in a 

one-on-one tutoring session (e.g., completing a worksheet assigned by the teacher, guided reading). In 

these cases, the tutor usually relied on teacher-provided materials with the small group as the rest of 

the class completed the same task individually. 

During the site visits and also in some survey responses, the Abt team heard concerns from some EC 

staff that tutors may have difficulty working with multiple students at the same time. Despite these 

reservations, the Abt team observed several tutors successfully facilitating a small group session. At

one elementary school served by EC Philadelphia, the Abt team observed a tutor working with three 

first grade students on developing phonemic awareness. The tutor led the students in an activity 

focused on recognizing consonant sounds and building words with the letters “s, “t,” “r,” “u,” “c,” 

and “k.” In this activity, the students created a series of words from laminated letter cards that became 

increasingly more difficult (e.g., “us” “cuts” “stuck” “trucks”). The activity was engaging and 

required involvement from each student in the group, and the tutor rarely had to provide behavioral 

support because each student was focused on completing the task. 

Literacy assistance 

As defined by AARP’s Experience Corps standards, literacy assistance tutoring is intended to 

reinforce literacy skills or concepts introduced in daily lessons. The teacher assigns the tutor to work 

with a small group, individual student, or an entire class while the teacher works with other students 

in the classroom. Literacy assistance is employed to meet a specific lesson-related goal, rather than 

individual student goals. Tracking individual student progress for classroom literacy assistance is not 

required. In new sites, tutors are expected to devote no more than 20% of their tutoring time to 

Literacy Assistance. 

Literacy assistance is used in all five of the EC AC sites, but is implemented to varying frequencies. 

Programs typically provide literacy assistance on an as-needed basis or in conjunction with sustained 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 17

one-on-one or small group tutoring. In EC Baltimore, however, literacy assistance is the primary 

strategy. Consistent with the literacy assistance strategy, tutors in Baltimore serve as a general 

resource for the classroom and facilitate short lessons. In one observed classroom in Baltimore, the 

tutor sat behind students during a whole-class presentation and provided behavioral management 

support by ensuring that students were focused on the teacher. The Abt team also observed instances 

in Baltimore where the tutor facilitated small group sessions while the rest of the class did other 

activities, but since the groupings were not sustained throughout the year, this would qualify as 

literacy assistance. 

In other sites, literacy assistance may be implemented less frequently and often in combination with 

the sustained tutoring strategy. In EC Boston, literacy assistance typically occurs in one of two ways: 

1) before and/or after sustained one-on-one and small groups are complete, or 2) as a scheduled 

tutoring session. At the beginning of the literacy assistance session that the Abt team observed, the 

tutor initially sat with a small group of students at their clustered desks, helping a few of them with 

spelling and writing as the teacher led a compare and contrast activity with the whole class. The EC 

tutor circulated among five students who needed varying levels of support to complete the activity. 

Similar to what was observed in other sites, tutors in Boston supported students who needed help 

focusing or provided specific assistance with teacher-determined activities or assignments (including 

worksheets, whole-group reading, individual reading, etc.). 

Literacy assistance is generally related to reading and writing instruction, but the Abt team observed 

multiple instances where EC tutors provided assistance in other subject areas. In EC Portland, literacy 

assistance is frequently related to literacy instruction, but may also involve other disciplines or 

activities. In passing, one tutor shared that she spent some time that day working with a student on 

fractions during math class. During one observation of a second grade classroom with two EC tutors, 

tutors worked at separate tables while three other adults (e.g., classroom teacher, foster grandmother, 

other paraprofessional) worked with small groups of students at other tables in an interdisciplinary 

center-based instructional period. Similarly, in EC Philadelphia, the Abt team observed an EC tutor 

providing classroom support during a math activity. The tutor worked with three students at a 

separate table in the classroom while the lead teacher explained a math activity projected onto the 

whiteboard. The tutor provided individualized support to this group for the duration of the activity, 

often repeating and clarifying the teacher’s directions to promote students’ understanding. 

5.2.2 Setting 

Tutoring sessions take place in different settings (e.g., hallway, classroom, designated room), often 

according to EC strategy type and limitations in the school layout. According to responses to the tutor 

survey shown below in Table 5, tutors most commonly delivered services within the classroom. The 

most common tutoring locations included a separate tutoring area within the classroom (35%), 

hallways outside the classroom (30%), and classrooms supporting teacher instruction (23%). In 

Boston, the Abt team observed one-on-one tutoring exclusively in a pull-out setting, often in a 

hallway or common area just outside the classroom. In Philadelphia, the observed one-on-one tutoring 

was delivered as pull-out in the hallway adjacent to the classroom as well as within the classroom – 

these approaches seemed to be dictated in large part by the available physical space. EC Portland 

provides yet another approach to one-on-one delivery. Abt staff observed four pull-out sustained oneon-one tutoring sessions in a single school delivered in a variety of settings (e.g., in the hallway just 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 18

outside the classroom, in a hallway outside a Physical Education class in progress, and in the school 

library). 

Table 5: Location of Tutoring Sessions 

Tutoring Session Location EC AC Tutor Survey 

Respondents (%) 

In the classroom at a separate tutoring area 34.8 

In the classroom at student work areas 29.6 

In the classroom supporting teacher instruction 22.8 

In a different room outside of the classroom 16.5 

In the hallway outside of the classroom 30.2 

In a different area of the school (i.e. library, cafeteria) 12.5 

Other 9.1 

Source: Tutor survey item: Where do your tutoring sessions usually take place? (check all that apply) 

(n=514) 

5.2.3 Materials and Use of Curriculum 

Across AmeriCorps sites, tutors occasionally used commercial curricula in their tutoring sessions. 

Approximately 53% of surveyed tutors reported that they use a specific curriculum with students 

during each session, and the remaining 47% either reported that they did not use a curriculum or were 

unsure. In addition, over half of tutors (56%) reported using teacher provided materials, but 

information gathered from site visits suggests that a significant number of tutors use materials from 

home or from EC staff members. In EC Boston, tutors engaged students on a variety of skills using 

instructional materials provided by the teacher or freely available within the classroom or school 

library. In other sites, tutors may rely more heavily on Team Leaders and their own resourcefulness to 

find materials that match the needs of their students. One tutor in Philadelphia shared that after every 

day of tutoring, he goes home to select and print worksheets from the Internet to use during his next 

session. Team Leaders may also identify and distribute instructional materials for tutoring sessions. 

One Team Leader, also from Philadelphia, explained that she receives several requests for additional 

material that focus on blending, for example, since this is a fundamental literacy skill often addressed 

in EC tutoring sessions. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 19

Table 6: Curriculum Use in EC AmeriCorps Tutoring Sessions 

Curriculum Used in Tutoring Session EC AC Tutor Survey 

Respondents (%) 

Reading A-Z 15.0 

Read Naturally 18.0 

Opening the World of Learning 2.6 

Teacher provided materials 56.5 

Other curriculum 36.1 

None 7.6 

Don't Know 4.8 

Source: Tutor survey item: What curriculum do you use with your student(s)? (check all that apply) 

(n=501) 

5.3 Research Question 3: What other factors appear to most influence 

program implementation? 

Results from the implementation evaluation suggest that three primary factors influence the degree to 

which programs implement the EC program model. These factors are: 1) Approaches to in-service 

tutor training and supports, 2) Tutor use of curriculum and other instructional materials, 3) Tutors’ 

prior teaching experience and educational background. 

5.3.1 In-service Tutor Support 

In several EC sites, we note the increased use of literacy coaches and other instructional personnel to 

train and support tutors in a more pedagogically-focused manner. Both Philadelphia and Boston 

demonstrate very promising approaches to integrating these staff into existing training and scheduling 

frameworks. We see this as a very positive step in expanding tutor knowledge of and comfort with 

effective EC strategies to move struggling readers forward. Research suggests that creating an 

instructional match with a reader – that is, selecting texts that both reinforce background knowledge 

and previously mastered skills as well introduce new words and content – is an essential step in 

increasing reading achievement. Providing an instructional match is important, yet many would argue 

that it is one of the most difficult aspects of teaching. Despite observing several instances where 

tutors unknowingly created an instructional “mismatch” by selecting (or having students select) texts 

that were either too difficult or too easy, we see the increased use of literacy coaches as a very viable 

method in helping tutors master this skill. 

5.3.2 Targeted Use of Curriculum and Instructional Materials 

The evaluation team observed substantial variation across sites with regard to sources of instructional 

materials utilized by tutors. In some sites, tutors work closely with classroom teachers and accessed a 

range of classroom-based resources. These observations aligned with the EC model, which stipulates 

that tutors work with the curriculum materials used in the classroom so that their instruction is 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 20

relevant. In other sites, tutors brought materials from home or looked to the Team Leader for 

resources. In and of itself, variation in the way in which tutors acquire materials is not problematic as 

long as the instructional value of these materials remains high, appropriate to each learner, and 

relevant to the other literacy instruction they receive. At sites in which tutors lack access to teacherendorsed materials and/or teachers are not supplying materials or using a curriculum, EC national’s 

recent recommendation of Reading A-Z will provide tutors across the network with high quality 

materials. Use of classroom materials and/or curricula will continue to serve as the preferred option to 

best align literacy instruction. 

5.3.3 Tutors’ Prior Teaching Experience 

Data collected from program director interviews and tutor surveys have revealed the benefits of 

recruiting and maintaining tutors who have had experience teaching prior to volunteering with 

Experience Corps. According to tutor survey responses, 34% of AC-supported tutors indicated that 

they have previous teaching experience. Survey results also indicate that tutors with extensive 

experience working with schools, as classroom teachers or in other roles, tend to feel most prepared 

to lead small group tutoring sessions – arguably the most challenging tutoring format, as it requires 

more adept behavior management and engagement skills. The EC network cannot expect to enlist 

only tutors with teaching backgrounds, but awareness of past tutor experiences in the timing and 

assignment of specific tutors may reduce discomfort and concerns with tutoring. 

5.4 Research Question 4: What supports from AARP and the school/site 

itself are necessary for successful program implementation? 

The evaluation team was able to utilize the varied data collection activities and subsequent analyses to 

identify key supports in implementation that are delivered at both a broad scale via EC national staff, 

and a more local scale via EC site staff. Results from the implementation evaluation suggest that 

tutors would benefit from additional training, increased access to a formalized curriculum or other 

intentional instructional materials, and support and encouragement in fostering frequent interactions 

with classroom teachers. 

5.4.1 Additional Tutor Training 

For several years, EC national staff have provided sites with well-developed training materials, 

scheduling and dosage expectations, and guidelines on how and why to deliver focused training. 

During site visits, many tutors emphasized the importance of training and had generally positive 

feedback about the quality of their training. Approximately 75% of tutors responding to the EC 

survey indicated that trainings both prepare them to work with students and support their tutoring 

activities during the year. When asked how tutors could be further supported, tutors indicated their 

desire for even more training, specifically in areas related to behavior management, supporting social 

and emotional learning, and delivering the five components of reading instruction. 

5.4.2 Access to Curriculum 

The EC national office has made strides to provide top-down supports by endorsing Reading A-Z and 

encouraging local sites to acquire Reading A-Z licenses as a resource for tutors. Based on data 

collected through site visits and the tutor survey, the evaluation team views the increased emphasis on 

using sponsored curriculum as a viable support in EC implementation. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 21

During site visits, members of the evaluation team observed tutors exhibiting varying levels of 

success in establishing an instructional match with their students. Formal curricula, like Reading A-Z 

and Read Naturally, often take the guess work out of aligning reading material with children’s 

reading ability. The EC network has, at large, deliberately used curricula to guide tutoring sessions. In 

fact, a little over 50% of tutors report using a curriculum. Based on these observations and supporting 

evidence generated by survey respondents, the evaluation team concludes that providing a curriculum 

to tutors who do not currently use one may be helpful in maintaining a higher degree of fidelity to the 

EC program model.

5.4.3 Tutor and Teacher Interaction 

A variety of data sources indicate that the majority of tutors and teachers have a strong relationship 

and that teacher-tutor pairs interact with a high degree of frequency. Seventy-two percent of tutors 

and 68% of teachers report “checking in” with one another at least once per week. In addition, 64% of 

tutors and 69% of teacher reported having a strong relationship with one another. 

Data also indicate that a substantial portion of these interactions are used to communicate issues 

concerning tutoring delivery. Close to 70% of tutors reported being observed and receiving feedback, 

most often a few times per year. Relatedly, three-quarters of teachers reported feeling comfortable 

giving tutors feedback directly. The evaluation team sees this as an essential element in supporting 

EC implementation at the local level. Strong tutor-teacher relationships and regular check-ins help 

foster open communication. Tutor observations and feedback allow tutors to adapt their tutoring 

methods continually, not only during formal assessment periods. 

5.5 Research Question 5: Which sites are most ready for impact evaluation 

and why? 

Implementation is a necessary but insufficient piece of discovering the impact of a program or 

intervention. Arguably, implementation is the most important piece to enacting change. In order to be 

able to examine the effect of a program, it is critical that the program be implemented well and with 

fidelity. Low levels of implementation fidelity can lead to biased conclusions about the effectiveness 

of a given intervention in changing student skills. Such implementation requires key program 

supports and components are in place. Two EC AC sites, Boston and Bay Area, applied to be a part of 

the Social Innovation Fund EC evaluation and were accepted. As we know from site visits, 

interviews, and other collected information, those sites are currently implementing EC at reasonable 

levels of fidelity to be primed for an impact evaluation. Boston has an established relationship with 

the school district and individual schools within that system, as well as an existing data sharing 

agreement. There is an existing system of data collection, management, and analysis at the site, and 

staff have developed a plan to implement a specific combination of the two primary EC tutoring 

strategies in new classrooms. EC Bay Area also has significant experience with the program model 

and has established a strong reputation in the area. Though they do not have a data sharing agreement 

currently in place, the program is planning to hire a Program Coordinator who will be responsible for 

the majority of data management and analysis activities. In addition to Boston and Bay Area, we also 

saw other sites during our data collection process that showed potential to be excellent candidates for 

impact evaluation based on their level of implementation. 

The degree to which a site supports tutors and maintains fidelity to the EC model are important 

factors in determining the readiness of a site to participate in an impact evaluation. Certain supports 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 22

such as a willingness to participate, the ability to provide district-level data and support outcome data 

collection, and sufficient buy-in from school staff are vital to the success of the program. 

Additionally, the implementation of EC needs to be consistent within each participating school across 

all sites. If eligibility criteria for participation are different, if assignment procedures are different, or 

if the delivery of EC differs between schools within a site varies substantially, our ability to detect 

site-level effects will be compromised. 
Abt Associates EC AC Implementation Evaluation Report ▌pg. 23

6. Conclusion

Ultimately, findings from the implementation evaluation enable EC national staff, as well as program 

leadership across all AC-funded sites, to better understand what implementation on the ground looks 

like for the two EC strategies – sustained tutoring (i.e., delivered in one-on-one and small group 

formats), and classroom literacy assistance. We observed generally strong levels of implementation 

across sites. The vast majority of tutoring observed was within the EC Standards (e.g., 20-40 minute 

sessions for two-five times per week for sustained one-on-one and small-group tutoring). There were, 

however, some deviations in implementation (e.g., some sustained small-group tutoring sessions 

involved five students), but these deviations were determined to be largely driven by local needs and 

context. A key strength of the EC program is that it meets schools and teachers “where they’re at” to 

provide a range of tutoring strategies that address superintendent, principal, and/or teacher 

preferences. Local innovation, when employed deliberately and aligned with core components of the 

EC model, does not appear to be a weakness of the EC model but rather a strength of it. 

There is evidence that the EC model is becoming more “professionalized,” with a greater reliance on 

literacy coaches and education professionals in the administration of training. We have also observed 

a growing emphasis among EC sites on engaging tutors with prior educational experience to take on 

additional tutoring responsibilities. Moreover, new recommendations encouraging tutors to use a 

standardized curriculum (Reading A-Z) when teachers’ materials and/or curricula do not provide 

sufficient guidance will provide EC staff and tutors with more clarity around the EC model. When EC 

finalizes these program components and the Abt team develops a fidelity rubric for each tutoring 

strategy, we expect that EC sites will soon arrive at a common understanding of the core 

implementation components that are necessary to successfully execute the EC model. Given these 

developments, coupled with an upcoming experimental study of the sustained small group strategy, 

there are numerous learning opportunities on the near horizon.