City of Lakes AmeriCorps 

Minneapolis Community Education 

Compiled Impact Report: 2011-2016 

December 2016 
Table of Contents 

Introduction ................ .. ............. ....................... ... ...................... .. ....... 1 

About Col ... .... ................ . ...... ..... . ...... .... ........... .... .... .. ..... ........ .... ...... 1 

Program Structure .......... .................. ..... ........................ ...................... 2 

Anticipated Outcomes and Methods of Evaluation ...... ................. . ....... .. ....... 2 

Data Collection Methods ...... ................ .. ..... . ......... ...... ... ...... ... ........ . ...... 4 

Continuous Improvement ...... ... .............................................................. 4 

Results ... ........ . ............. .. ............. ........ ... ................. . .................. ... .... S 

Conclusions .... .. ..... . ... ...... ....... .. .... .. ... ... ..... . ... .... ...... .. ..... . ... ...... .......... 13 

Appendix: 20 Year Program Totals ............. ...... ... ......... ... ...... .... ........... ...... .. 14 

Appendix: Teacher Feedback......................... ....... ................ . ...... ... ....... .. .. 14 

City of Lakes 

AmeriCorps 

With support fromthe Corporationfor 

Nationar & CommunityService& ServeMinnesota ... . . . . . . . .. . . . . . .. . . . . . . ... . . . . . .. . . . . . .. .. . . . . . . . . . . . . . . ... . . . . . ... . . . . . .. . . . . . . ... . 0 

Apmgram of 11,!imapms Comnwllity &lLQtia, &lhe MI'S M1.dtiingual Department 
Introduction 

As of the writing of this report, City of Lakes AmeriCorps has completed five years of programming 

with its current focus on middle school Long-Term English Learners in Minneapolis Public Schools. 

There have been many changes made to the program over the course of these five years, as we 

have analyzed and evaluated what works best for our target student group. This report summarizes 

the evaluations completed over the years, and the resulting continuous improvement decisions 

made to the program as a result of those evaluations. 

About City of Lakes AmeriCorps 

Who We Are: City of Lakes AmeriCorps (Col) is a partnership between Minneapolis Community 

Education and the Multilingual Department of Minneapolis Public Schools. We strive to bridge the 

language gap for Long-Term English Learner students (LTELs) in Minneapolis Public Schools. 

History: The City of Lakes AmeriCorps program started in 1994 as a partnership between the YMCA 

and Minneapolis Community Education. Over the years the program has continued to evolve and 

refine it s scope. In 2000 the program was housed entirely with Community Education in the 

Minneapolis Public Schools, although the YMCA cont inues to be an important partner for specific 

programs and·events. 

Beginning in the 2011-2012 school year, the program began its focus specifically on middle school 

English Learner (EL) students. Each COL AmeriCorps member provides a caseload of students with 

lit eracy and language-based 1:1 or small-group tutoring throughout the school day. Members 

collaborate closely with content, ESL, and Reading teachers to ensure a cohesive approach in 

meeting students' individual learning needs. Members also partner with Community Education to 

provide classes during out-of-school time (after school, Saturdays, and during the summer). These 

classes aim to reinforce literacy skills in unique and engaging ways. 

Mission: The mission of City of Lakes AmeriCorps is to provide Long-Term English Learners access 

to grade-level content through dedicated and rigorous language learning support . Our mission for 

members is to build a learning community through service, reflection and inquiry. 

Vision: Our vision is to serve, and not fix*, in order to eradicate the opportunity gap for Long-Term 

English Learners. 

* Basedon the poem "A Fixer" 

Goals: Our goal for our students is that they are engaged and empowered to achieve their full 

potential. Our goal for our members is that they are engaged and empowered to serve as valuesdriven leaders in their communities. 

Strategies 

1. Building relationship with students through intentional interactions and youth development 

best practices. 

2. Utilizing a gradual release of responsibility and tutoring strategies from Kate Kinsella and 

other research-based approaches. 

3. Creating a home connection between members and students' families. 

4. Collaborating closely with teachers to form a teacher-member-student partnership. 

1 
Program Structure 

Synopsis of Need/Prob lem: LTELs are at significant educational risk. On average, LTELs tend to have lower 

grades, lower standardized test scores, and face a higher risk of dropping out than non-L TELs. After years of 

falling behind, they often have developed habits of disengagement in school. 74% of the 1592 middle school 

English Learners (Els) in Minneapolis Public Schools ( or 1,178 students) are LTELs. 82% of these students are 

scoring as non-proficient on standard ized tests, with 37% of them sco ring in the "Falling Behind" categories 

of growth. 

Synopsis of Rationale : Research tells us that effect ive second language instruction must include explicit 

teaching of that language (Claude Goldenberg , 2008), as well as intentional support in transferring those 

skills into the classroom (Margarita Calderon, 2012). Research has also documented the impact afte r school 

programs (Mahoney et al, 2007) and parent involvement (William Jeynes, 2012) can have both on student 

engagement and academic outcomes. 

City of Lakes AmeriCorps' Theory of Change is that by providing small group evidence-based English 

language instruction and support, supplemented by after-school programming that utilizes literacy skills in 

engaging ways, and by involving parents, LTELs in grades 6 -8 will significantly improve their academic 

outcomes. Participation in the Col program will result in increased engagement in school as well as 

increased academic langua ge skills, as evidenced by teacher and student survey, pre- to post- test 

assessments, and a propensity matched comparison of standardized test scores. 

Anticipated Outcomes and Methods of Evaluation 

In looking at the impact of the program, City of Lakes measures effect within the following anticipated 

outcomes: 

1. Col students will show improved academic performance (ED5) 

City of Lakes has consistently looked at WIDA ACCESS growth as the ultimate measure of academic language 

growth in our students. Data to identify these students comes from Google Reporting forms submitted by 

Col members on stud ents they had served. WI DA ACCESS is large -scale English language proficiency 

administered annually to all MPS students receiv ing ESL services. It is part of the WIDA consortium of 

assessments for Els, used in over 35 states throughout the nation. 

The WIDA ACCESS has been a difficult mea sur e to use because, until rece ntly, there were no growth norms 

to help guide target setting and comparison. As a result, Col often fell short of this goa l in previous program 

years (for example, only 3 out of 194 Col students met our WIDA ACCESS growth goal in 2014-15 of 1.0 

increase in WIDA ACCESS score; however, average growth for middle school Els that year was -.3 points, 

indicating the goal we set was a very unrealistic one) . In addition, the WIDA ACCESS and other standardized 

tests may be too far removed from the work we do with students to truly reflect our impact. Because of th is 

difficulty, Col will utilize curriculum-imbedded pre- and post-tests to determine improved academic 

performance for PY2015-16. This tool has a much more direct tie to the serv ice our members provide to 

middle school Long-Term English Learners in Minneapo lis Public Schools, and should be a much more 

accurate measure of student growth in academic language . We will continue to also ana lyze the WIDA 

ACCESS growth of our students as one way of comparing growth of our students to other LTELs in th e 

district. 

2. Col students w ill show greater growth on standardized test scores compared to middle school 

l TELs not being served by Col. 

City of Lakes has utilized a variety of different assessments to analyze compa rative growth - comparing 

growth of students who received se rvices versus other L TEL students who did not . This ha s often involved 

2 
an evaluation of other stari:dardized test scores - either the Measure of Academic Progress (MAP) or :.:. 

Minnesota Comprehensive Assessment (MCA). The determination of which test to use has been made in 

conjunction with the district's Researc h Evaluat ion Assessment and Ac~ol.mtability Department (REAA, 

formerly REA), and in response to testing parameters . For example,°MAP was the preferred tool and was . 

used in 2011-12, but was_not required for all sites to give in follovying'years. MCAs were then used as not all of. ' . . 

our sites had MAP .scores for students . 

To determine gro.wth comparison using MCA scores , an e~~luator from the district's REAA Departm~r;it" 

completes an ~t'JOVA analysis of growth scores for COL participants (treatment) vs. eligible students · . 

( control). Thi~eva luation involves a propensity matching ~ethodology to compare growth of students who 

had received English 3D tutoring services from a Col member to students who were middle school LTELs 

who had not received services from a Col member. Data to identify these students came from Google 

Reporting forms submitted by Col members on students they had served, and from district databases with 

student demographic and test score information . Students have to have two MCA scores in order to be 

included in this growth comparison. Means used in th ese results are z score growth means. 

The method of evaluation used in analyzing comparative academic impact is quasi-experimental. We use 

treatment and control groups in comparing st udent growth on standa rdized test sco re s. This method was 

chosen over random assignment because intent ional se lection of students exists through factors such as if 

the school a student attends is also one where members are serving and if a student's daily schedule allows 

for the intervention to be added during a time when a member is also available. ESL teacher input on which 

students might most be receptive to, and benefit from, the intervention is also factored into the student 

selection process. Quasi-experimental evaluation is also possible for our program due to the ease of access 

to data from a comparative student group within our school district that has the exact same characteristics 

(middle school, Limited English Proficiency status for 5 or more years , attending the same high-poverty, lowperforming MPS schools) as our served students . The evaluation was designed by City of Lakes Program 

Director Lisa Lambert in partnership with the MPS REM Department 

3. Col students will show increased student engagement (ED27) 

The original plan for measuring student engagement was using an annual district assessment, the MPS 

Student Survey. The Student Survey contains a small portion of questions that get to student engagement in 

school; however, it is a relatively new tool. We determined under the advice of REM not to utilize the annual 

MPS Student Survey to measure student engagement for psychometric reasons. According to REA, the 

psychometric reasons for not utilizing this assessment yet in this way are around not having a large enoug h 

data set with enough variability to use the results with any degree of confidence. Basically, the applicable 

questions used this year on the Student Survey were piloted items on smaller samples, and REA is waiting 

until there is enough data to have stable psychometric properties to more formally use it in evaluations. Until 

that time, we have been advised to utilize other methods of measuring student engagement. 

In place of the Student Survey, Col has utilized both attendance reco rds as well as teacher surveys to 

measure student engagement behaviors. 

In addition , student voice has recently been included as part of our program eva luation as well. For PY2015-

16, along with the teacher survey for student engagement, Col utilized a student survey around engagement 

in school. MPS has been developing a standardized tool to help measure student engagement of students 

districtwide. It is our goal that once this tool is developed and past the piloting stage, Col will be able t o 

analyze results of our Col student engagement compared to district results for other middle school LongTerm English Learners. In the meantime, we have been utilizing a student survey with questions taken from 

the SAYO (Survey of Academic and Youth Outcomes) tool commonly used by Community Educatio n to 

eva luat e its programs . 2015-16 results from this survey indicate that 86% of students enjoy working with a 

Col member, and 97% report that they learn from working with a memb er. 81% of Col students also reported 

that working with a Col member helped them fee l more confident in class. 

3 
4. Col students will have increased parent engagement. 

Col members are required to maintain consistent positive communication with the parents of caseload 

students at least twice per month, and to facilitate quarterly family nights. To measure the impact of these . 

outreach efforts, members disseminate a parent survey at their final two family nights of the year. This 

written survey is available in English, Spanish, Somali and Hmong. Interpreters are also often present at 

family nights and help in translating if needed. This survey asks for parent feedback on the even t itself, as 

well as how the Col parent connection piece is impacting them. 

For 2015-16 results, 73% of parents agreed and 23% strongly agreed that family nights and regular contacts 

from Col Members helped them to feel more connected to their child's school. 55% of parents agreed and 

45% strongly agreed that they were more likely to attend other school events because of family nights and 

regular contacts from Col mempers . 64% of parents agreed and 32% strongly agreed that they were more 

likely to contact their child's teacher or other school staff with questions about their child's education 

because of family night and regular contacts from Col. This information helps Col members and program 

staff be responsive to parents as well; for example, while many parents felt the amount they were contacted 

by Col members was just right, almost 20% of respondents felt like it was too much. In response, Col 

lowered parent communication requirements for members from 3 to 2 times per month for the 2016-17 

program year. 

Data Collection Methods 

Program data is collected weekly (English 3D tutoring logs and parent communication logs), monthly 

(student progress logs), quarterly (after school activity participation logs, family night surveys), biannually 

(member and teacher surveys); and annually (standardized tests, focus groups/ in-depth interviews) . All 

reports from members (the logs mentioned above) are submitted through Google Forms Reporting . These 

forms are only accessible to members logging in with MPS login info, and once entered, are not editable by 

members. Members are trained in data collection and database tools during orientation and on-site over the 

first few weeks of service. Members collect baseline data in September, then benchmarking data monthly 

from October through May. Program Director Lisa Lambert analyzes data quarterly and annually to ensure 

accurate and timely submissions, and to guide the program in a process of continuous improvement. The 

MPS REAA Department oversees data collection and management of standardized test scores bot h for Col 

students and for the comparison group. Evaluations of stakeholders --including members, parents, teachers 

and others --are facilitated by the Program Director through surveys administered twice each year through 

Survey Monkey, or by pen -and-paper. Focus groups and in-depth interviews have also been used, facilitated 

by both the REAA Department and Program Director Lisa Lambert , as another way of gathering stakeholder 

input . 

Continuous Improvem ent 

City of Lakes is committed to continuous improvement. Each month, an ESL Teacher and the member(s) at 

eac h site review benchmark findings of student progress and use this data to inform or modify individual 

student interventions and accelerate their progress. Twice each year, internal stakeholder meetings review 

program progress towards goals, as well as feedback from members , teachers , parents, and students, to 

determine whether any chang es are nee ded to ensure outcomes are met. Performance Measure progress 

results are reported to ServeMinnesota and CNCS quarterly. A full program performance measure 

evaluation is reported to all funders and stakeholders annually, and used in planning program s, training and 

operations for the coming year. 

4 
Results 

The chart on the following pages illustrates the impact from, and changes to, the City of Lakes 

AmeriCorps program over the past five years of our focus on middle school Long-Term English 

Learners (LTELs). In-depth data is included in the end notes. 

5 
2011-12 

1st Year of ESL Focus 

2012-13 

2nd Year of ESL Focus 

2013-14 

3rd Year of ESL Focus 

2014-15 

4th Year of ESL Focus 

2015 -16 

5th Year of ESL Focus 

Changes made Established specific curriculum Increased parent Additional tim e with students, plus Curriculum change from Kansas 

to program (Kansas Writing Strategies) for communication, increased addition of monthly meetings Writing Strategies to English 3D (a 

structure 

. 

members to use student involvement in after 

school, increased/restructured 

tutoring time during the day, 

and enhanced bench marking. 

between member and site's ESL 

teacher to review student progress 

and make any adjustments 

more engaging curriculum geared 

specifically for Long-Term English · 

Learners) 

Assessment Measuring language growth on Measuring speaking growth Measuring writing growth using Measuring writing growth using Measuring language growth using 

Plan TEAE (Test of Emerging Academic using the OLAl-2 (Oral WIDA ACCESS Writing scores WIDA ACCESS Writing scores overall WIDA ACCESS score AND 

English) Language Acquisition growth qu intiles to determine# of 

Inventory, Second Edition) students who had above average 

growth Vii 

Outputs 343 students tutored (target 275) 

Total of 947 students served 

through tutoring, after school 

and/or mentoring 

341 students tutored {target 245) 

Total of 898 students served 

through tutoring, after school 

and/or mentoring 

291 students tutored (target 275) 

Total of 618 students served 

through tutoring, after school 

and/or mentoring 

175 students tutored {ta rget 180) 

Tota l of 1,071 students served 

through tutoring, after school 

and/or mentoring 

154 students t utored {target 180) 

Total of 836 students served 

through tutoring , after school, 

and/or mentoring 

Outcomes 232 students received required 

dosage 

In Spring 2011, 0 (zero) COL students 

achieved the highest proficiency level 

(Level 5) on the TEAE Reading test, 

compared to 87 COL students 

achieving the highest proficiency 

levels (Levels 5 & 6) on the WIDA 

ACCESSReading test in Spring 2012. 

244 students received required 

dosage 

137 out of 244 increased their 

WIDA ACCESS Speaking score 

by the state standard amount 

(0.5) (Target 145) 

160 students received required 

dosage 

5 students out of 160 increased 

their WIDA ACCESS Writing 

score by one full point (target 

145) See Challenges for more 

information 

31 students received required 

dosage (target 135) - for first time, 

dosage based on curriculum 

progress rather than time/amount 

of tutoring. 

3 out of 141 students with 

comparable scores had an ACCESS 

writing growth of 1.0 or higher 

114 students received required 

dosage (target 135} 

76 of 121 (63%) of Col students 

with comparab le scores w ho 

completed the program scored in 

the top 3 quintiles of growth on 

the WIDA ACCESS. 

6 
Engagement In Spring 2012, twenty-six teachers 95.59% of Col students met the 77.4% of Col students met the 95% Teachers reported growth in at 

Outcomes representing all sites responded to an 

impact survey. Among teachers of 

students who are working with COL 

AmeriCorps members: • 100% of 

teachers reported increased/ 

improved 1) comprehension,2) usage 

of academic vocabulary, 3) writing 

skills, and 4) speaking skills among 

participants; • 96% of teachers 

'reported increased/ improved 1) 

participation in academic discussions, 

2) listening skills, 3) reading skills, and 

4) homework completion among 

participants; • 72% of teachers 

reported an increase in student 

attendance among participants.i 

95% attendance requ irement 

during the 2013-14 school year 

(with a standard deviation of 

less than .OS). This can be 

compared to 93% of eligible Col 

students who did not receive 

services (standard deviate of 

.08286) and approximately 61% 

of MPS students overall who 

had 95% or better attendance 

during the 2013-14 school year. 

attendance requirement during the 

2015-16 school year, compared to 

71.8% of eligible Col students who 

did not receive services. ii 

least one engagemen t area for 

92.5% of students . Growth was 

attributed at least in part to a 

student's participation in the Col 

program . viii 

Attendance reco rds of Col 

stude nts were also evaluated as a 

way of examining student 

engagement impact. For PY2015-

16, 79% of served Col students 

met the district goal of 95% 

attendance. This can be compared 

to 56% of students meeting this 

goal districtwide . 

Comparison Col students had higher average Greater percentage of Col MPS l TEL Students' average Overall, Col impact on MCA scores Overa ll, Col students showed 

Data scale score increases in MAP 

Reading and Vocabularyii i 

students had strong speaking 

growth on WIDA ACCESS when 

compared to MPS overall iw 

growth on WIDA ACCESS Writing 

from Spring 2013 to Spring 2014 

was negative, at -0.3, which is in 

line with very low growth being 

seen statewide on WIDA ACCESS 

for the 2013-14 school year. 

Col Students showed higher 

growth on MCAs compared to 

eligible students who did not 

receive services.vi 

was not statistically significant; 

however, one MCA analysis done 

by REA did show that w hen 

comparing among implementation 

(number of lessons), the more 

lessons our students received, the 

more growth they made o n MCAs. 

higher growth on MCAs but not 

quite significant (p=.128) . Results 

indicate t hat the more of the 

intervention results in higher 

growth. ix 

In looking at MCA Reading 

percent ile ranking growth, Col 

students grew an average of 3.842 

points compared to an average of 

3.28 point growth for middle 

school English Learners overall. 

7 
Teacher "My developing readers have "I have seen a positive change "I have worked with AmeriCorps "This program helps encourage "Ana's confidence and task 

Comments never read as many books as they 

have read this year. They also 

enjoy discussing the books with 

their tutors. I am having an 

incredible year this year because 

of my tutors." 

in many of the students that [a 

Col Member] has worked with 

- a few who have finally 

become confident in science." 

volunteers for the last 3 years. I 

have REALLY seen a difference in 

the kids on the volunteer 's 

caseload. It's difficult to explain. 

The kids seem to feel SPECIAL, 

and so they become more 

motivated to do good work!" 

students to be proactive about 

their learning." 

"It provides students with an 

opportunity for additional 

academic support that they 

wouldn't get otherwise." 

comp letion has grown over t he 

course of the year . I think that she 

now believes that she is capable of 

succeeding so she is more willing 

to try each assignment. " 

. Challenges Moving from the TEAE test (Test 

of Emerging Academic English) to 

the ACCESS test (Assessing 

Comprehension and 

Communication in English Stateto-State) - reporting data includes 

TEAE test scores from Spring 2011 

and ACCESS test scores from 

Spring 2012. The scores are not 

directly comparable. 

Newness of and changes in 

ACCESS Test - The ACCESS tests 

run on a cycle, changing every 3 

years. This was a year the test 

changed. In addition, this is only 

the third year both Col and MPS 

have used the WIDA ACCESS 

exam. As Col and MPS become 

more familiar with the WIDA 

ACCESS exam and have a better 

sense of typical growth to 

expect, we will be better able to 

assess Col's comparative impact 

and create more realistic 

outcome measures around this 

particular exam. 

Limited time with students -

members provided only one 30-

minute session per week 

Schedule unpredictability and 

uncertainly around WIDA Growth 

Norms - determination that our 

original goal of 1.0 growth in 

Writing in an unrealistic figure 

Kansas Writing Strategies 

curriculum was good in that it 

focused very specifically on one 

skill (writing); however, members 

report it was not a very engaging 

curriculum at times seemed too 

basic for some students. 

Low recruitment and retention 

rates led to fewer total students 

being served; increasingly, 

members cite financial difficulties 

and mental health issues as 

barriers to successful service with 

our program. 

8 
i. 2011-12 Engagement Outcomes based on teacher post-survey 

In general, for students working with a COL AmeriCorps members, have you 

noticed any of the following? 

No,ie 

- A hlllc 

- s.oo-., - A lat 

;;2014-15 Engagement Outcomes 

Met 95% attendance goal Attended 90% or more Average Attendance 

Col Students 77.4% 97.0% 96 .61% 

(stand. dev of .038) 

Other MPS LTELs 71.8% 91.3% 95.69% 

(stand. dev . of .0527) 

Basedon an ANOVAanalysis of the 106 served student s with attendance records comparedto 415similar/eligible 

students (students meeting eligibility requirements for Col but ultimately not served) 

Average attendance of served students was 96.61% with a standard deviation of .03817 while average 

attendance of eligible but not served students was 95.69% with a standard deviation of .05273. This 

finding was statistically significant at the .10 leve l. In addition, wh en looking at these two groups in 

terms of meeting the district standard of 95% or better attendance, 77.4% of served students met this 

standard compared with 71.8% of eligible, non-served students. This trend continues when looking at 

students attending school 90% or mor e of the time, with 97.0% of stud ents meeting this mark 

compared to 91.3% of eligible but non-served students. This data supports what members and 

teachers are seeing, namely that throughout the year , served students become more engaged 

throughout the school day and in their education in general . 

9 
iii 2011-12 MAPResults -

Average Scale Score increases for Col vs. non-Col students 

Non-Col MPS Students Col Students 

MAP Reading 1.0 1.03 

Vocabulary Strand 2 4 

In addition -The MAP Reading scale score gap between participating and non-participating 

students was 7 points in Fall 20111 but only 3 points by Spring 2012. The Vocabulary Scale 

Score gap between participating and non-participating students was 7 points in Fall 2011, but 

only 5 points in Spring 2012. 

v 2012-13 Comparison Results - WIDA ACCESS Speaking growth 

Col Students with 12+ Col Students with 1- MPS Middle School 

weeks of tutoring 11 weeks of tutoring ELL Students 

Strong Speaking 

growth (1.0 or more) 

47.2% 38.1% 30.8% 

vi 2013-14 Comparison Results - MCA growth 

Low Growth Medium Growth High Growth 

Eligible students who did 

not receive services 

30.7% 40.1% 29.1% 

Col Students overall 31.6% 37.8% 30.6% 

Col students with 12+ 

weeks of tutoring* 

13% 20% 49% 

Vii WIDA Growth Adjustment Justification 

MPS began using WIDA in the spring of 2012. WIDA proficiency scores range from 1.0 to 6.o; 

lower range is defined as 1.0 to 3.0 with mid-to upper-range being 3.0 and above . A few 

years of analysis of district-wide results shows that students in the lower range of scores 

(typically newcomer English Learners) demonstrate faster growth on the WIDA ACCESS. 

Students in mid- to upper -range WIDA scores (where our target population of Long-Term 

English Learners typically fall) have much slower growth . 

For example , the 2014-15 average growth for a ih grade English Learner with a 1.5 starting 

WI DA ACCESS level was 1.3. The 2014-15 average growth for a ih grade English Learner with 

a 4.5 starting WIDA ACCESS level was actually negative at-.2. The negative average growth 

10 
trend continues for students with a beginning score range of 4.5 to 6-4. This is because 

language growth actually slows (and sometimes regresses a bit) as students switch from 

developing conversational English to developing academic English. 

MPS has addressed the need for more accurate growth goals for students by creating 

growth quintiles for each grade and WI DA level. This quintiles place students' growth into 5 

ranges - Q1. Low growth (1st to 19th percentile), Q2. Below average growth (20th to 39th 

percentile), Q3. Average growth (40 th to 59th percentile), Q4. Above average growth (60 th to 

79th percentile) and Q5. High growth (80 th to 99th percentile). To determine which quintile a 

student falls into, their growth is compared to other students at their grade level with a 

beginning WIDA score similar to theirs. This essentially allows us to factor in starting 

proficiency levels to get a more accurate picture of that student's yearly growth. 

Viii2015-16 Engagement Outcomes 

Col was unable to utilize the MPS Student Survey as originally planned due to the pilot stage 

of the tool and questions from REM about the validity of the results. In its place, the 

teacher survey was administered asking teachers about growth of students as well as how 

much they would attribute that growth to Col. Because this was not our initial plan, these 

resu lts we re given in a post-test only format. 

For PY2015-16, 120 teacher respondents completed an engagement survey on 81 total 

students. Of those 81 students, 75 ( or 92.5%) showed growth in at least one of the identified 

areas of academic engagement . Identified areas of engagement were Attendance, In-Class 

Participation, Assignment Completion, and Use of Academic Language. 

29 teachers identified improvements in attendance, with 12 reporting that this was definitely 

due to the City of Lakes program and the remaining 17 saying it may or may not be due to 

Col. For assignment completion, 57 teachers identified improvement with 27 of those 

reporting the improvement as definitely due to Col. For in-class participation, 57 teachers 

identified improvement with 21 of those reporting the impro vement as definitely due to Col. 

For academic language, 74 teachers reported improvement with 37 of those reporting the 

improvement as definitely due to Col. 

For PY2016-17, Program Director Lisa Lambert will disseminate a survey to teachers of all Col 

Students at the beginning of the year (in Oct) and again at the end of the year (in May). This 

survey will ask teachers to rate students on the following academic engagement areas : 1) 

class attendance, 2) assignment completion, 3) in-class participation, and 4) disciplinary 

referrals. To help gauge academic language impact , the survey also asks teacher to rate the 

student's use of academic language. All questions use a ranking system of 1 to 5. In orde r to 

count towards our growth target, students much receive a total score increase of at least 

20% when factoring in scores from the four academic engagement areas. For example, a 

student receiving a 2 in each of the 4 categories for a total score of 8 ( out of 20 possible 

points , or 40%) must receive a total score of 12 ( or 60%) in order to count towards our 

growth target with student engagement . 

11 
ix 2015-16ComparativeResults using MCA Reading scores 

For 2015-16, the treatment sample was quite a bit smaller than the control group, with 125 students 

falling within the served/treatment category and 511 in the eligible/control group, but did not violate 

homogeneity of variance. Overall, the mean was higher for the treatment group but not quite 

significant (p=.128). When it was run by grade, grade 6 had negative impact results -the treatment 

group had slightly lower z score growth. Grade 7 were very positive - the control group had negative 

z score growth and treatment had positive z score growth. Results were pretty close to significant 

with a p value of .082. Grade 8 also had higher z score growth for the treatment group but the 

difference was not statistically significant. 

The -grade breakdown is notable as it corresponds with the amount of intervention delivered . Grade 

7 had the highest positive impact and also the most intervention received. On average, Col students 

in 7th grade received 31 hours of English 3D instruction, completed 23 lessons, received 6.7 hours per 

week of push-in support, participated in 2 days per week of after school, and received 8 parent 

contacts over the course of the year. Grade 8 showed the next highest difference between the 

treatment and control groups, and received the next highest amount of inte rvention. On average, 

Col students in 8th grade received 26 hours of English 3D instruction, completed 20 lessons, 

rec e ived 4.7 hours per week of push-in support, participated in 1.5days per week of after school, and 

received 7.75 parent contacts over the course of the year. Grade 6 had negative results and received 

the least amount of the intervention. On average, Col students in 6th grade received 18 hours of 

English 3D instruction, completed 18 lessons , received 3.8 hours per week of push-in support, 

part icipated in less than 1 day of after school, and received 7 parent contacts. 

12 
Conclusions 

Much of the past five years has been spent adjusting program structure and content 

(specific curriculum} in order to best meet the specific academic needs of middle school 

Long-Term English Learners (LTELs). There is promising evidence that the City of Lakes 

AmeriCorps inte rvention in its current form leads to improved academic outcomes for 

middle schoo l LTELs in Minneapolis Public Schools, both in academic language skill and 

schoo l engagement . Moving forward with a consistent program structure in place, a system 

for continuous improvement, and continued evaluation will allow this program to broaden 

that impact. 

13 
APPENDIX 

i.._Q_y_e _ar Program Totals 

2013-14 marked the 20 th year of Col! Here are our 20 year totals -

► 533 members who have served 

► $2,965,000 in Education Awards earned 

► 10,205 youth tutored 

► 4,734 youth engaged in Service Learning 

► Over 1,270 youth mentored 

► 15,702 youth participated in before or after school programs 

► Over 760 Adults received ESL or GED Tutoring 

► 11374 additional volunteers recruited 

Highly qualified and trained candidates for permanent MPS positions* 

*Almost one-third (32.5%)of our alumni reported that they worked in MPS after their year of service, 

and almost half (43.75%) are still working in Education. The vast majority report that their experience 

in Col directly impacted their decision to work in this field and their ability to get a position in the 

field of their choice. 

Teacher Feedback 

2011-12 

"My developing readers have never read as many books as they have read this year. They 

also enjoy discussing the books with their tutors. I am having an incredible year this year 

because of my tutors." 

"The COL AmeriCorps member in my class does an excellent job of fostering positive 

relationships with all of the students in the class. She has been a great asset in my classroom 

and I'll be sorry to see her go at the end of the school year." 

"I have seen a positive change in many of the students that [a Col Member] has worked 

with - a few who have finally become confident in science." 

"Mr. Menard has been working with a few students in science over the past few months. 

During an independent learning unit (Astronomy) Mr. Menard worked very closely with K. 

He has made such strides with him - he is not as shy anymore and willing to participate with 

groups in science now. It has been an amazing transformation for K - an invaluable resource 

to have." 

"One particular student comes to mind. He was not turning in homework and had a bad 

attitude most of the time. An AmeriCorps member made a point to give him regular positive 

and encouraging feedback. His attitude and homework completion completely changed for 

the better." 

14 
"The students that worked with Ms. Dianna have gained more than one level on their MCA 

Scores. Two have gone from below grade level to at grade level. Students who work with 

Ms. Dianna are more confident in Math and have a deeper understanding of what they have 

learned." 

"J (works with Ms. Jenny) has gone from a score of 325 in September to a score of 771 (an 

approximate four grade level improvement) on one measure of reading proficiency." 

"Adam T. has been instrumental in assisting me with motivating a student who suffered 

from a lack of motivation. This student refused to do any work or put forth any effort in or 

out of school. His parent could not get him to attend after school to receive additional 

academic support . Adam and I encouraged, coaxed, and pleaded with this student until one 

day he showed us what he was capable of by completing an assignment. This student has 

continued to show effort by setting the goal to achieve an A in all of his classes. He has an A 

in Language Arts for the first time in 2 years. I couldn 't have convinced him alone without 

the help of Adam T." 

"Eis painfully shy and Jane's encouragement has kept him from being completely 

unnoticed . He has turned in every assignment and has received As every quarter in my class. 

He is no longer afraid to ask her for help. Jane has helped M to stay grounded when he 

would like to goof off with his friends. E went from being very apathetic to someone who 

takes pride in her accomplishments and wants Jane and I to be proud of her, too. Since Jane 

has been doing literature circles in my class, all of the students have bonded with her. They 

are now reading independently because of her. Many of them had rarely ever read a book 

on their own until they joined her book clubs." 

2014-15 

"I have worked with AmeriCorps volunteers for the last 3 years. I have REALLY seen a 

difference in the kids on the volunteer's caseload. It's difficult to explain. The kids seem to 

feel SPECIAL, and so they become more motivated to do good work!" 

"T his program helps encourage students to be proactive about their learning." 

"It provides students with an opportunity for additional academic support that they 

wouldn't get otherwise. " 

"Students develop a relationship with AmeriCorps staff that allows students to excel during 

small group project work." 

2015-16 

. "E has been taking assignment completion even more seriously and regularly checks his 

grades. He asks about his scores, showing great ownership of his grades and achievement." 

"V benefits from the small group attention that Col offers . She is very conscientious of her 

grades and retains and uses academic vocabulary." 

" The student has been very positively impacted academically and personally." 

15