USAID/Zambia Education Program
Evaluation

July 2009This publication was produced for review by the United States Agency for International Development.
It was prepared by DevTech Systems, Inc.

USAID/Zambia Education Program
Evaluation
Authors:
John R. Swallow, Ph.D.
Eldora Burns Nielson, Ed.D.
Peggy Nsama Chakufyali, M.Ed.

July 2009
Prepared by DevTech Systems, Inc. under MOBIS Contract #GS-10F-0048L,
Purchase Order # 611-O-00-09-00054.

DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect
the views of the United States Agency for International Development or
the United States Government.

Table of Contents
Acronyms ............................................................................................................................... i
Acknowledgements ............................................................................................................. iii
Preface .................................................................................................................................. iv
1. Summary of USAID/Zambia Education Program Evaluation ........................................ 1
Overview of USAID/Zambia Program .................................................................................. 1
Overview of USAID Impact on Zambian Education Sector ................................................. 2
Moving Forward ................................................................................................................... 5
2. Evaluation Methodology .................................................................................................. 5
3. Limitations of the Study ................................................................................................... 8
4. School Visits and Classroom Observations .................................................................. 8
5. Programs ......................................................................................................................... 10
CHANGES2 ....................................................................................................................... 10
Background: .................................................................................................................................. 10
Implementation and Results: ........................................................................................................ 11
Overall CHANGES2 Findings: ......................................................................................................... 13
QUESTT ............................................................................................................................ 15
Background: .................................................................................................................................. 15
Implementation and Results: ........................................................................................................ 16
QUESTT’s Work with Communities: .............................................................................................. 16
Challenges and Limitations: .......................................................................................................... 17
Overall QUESTT Findings: .............................................................................................................. 17
EQUIP2 ............................................................................................................................. 19
Background: .................................................................................................................................. 19
Overall EQUIP2 Findings: .............................................................................................................. 20
Information management and IT infrastructure: ......................................................................... 21
Policies, procedures, and guidelines: ............................................................................................ 22
District and school level capacity building: ................................................................................... 22
Continuous Assessment: ............................................................................................................... 24
6. Sector Analysis ............................................................................................................... 26
Institutional Development .................................................................................................. 26
ICT and Information Management ..................................................................................... 29
Teacher Training ............................................................................................................... 31
Teaching and Learning Materials ...................................................................................... 34

School Health and Nutrition............................................................................................... 35
Grants and Scholarships ................................................................................................... 35
7. Sustainability................................................................................................................... 37
8. Recommendations .......................................................................................................... 39
ANNEXES
Annex A: Key Documents Reviewed
Annex B: Places Visited and People Met/Interviewed
Annex C: Data Collection Instruments
Annex D: Results of Classroom Observations and Results of Anonymous
Questionnaires

Acronyms
AIR
AED
CA
CBO
CDC
CHANGES2
COE
CS
CST
DEBS
DESO
DODE
DRC
DRCC
ECZ
EDC
EFA
EMIS
ESO
EQUIP2FHT
FNDP
FTI
HIV/AIDS
HRIS
GRZ
ICT
IGA
IM
IRI
LTM
MOE
M&E
NISTICOL
NGO
ODL
OVC
PCSC
PEO
PEPFAR
PESO
POC
PRC
PRCC
QUESTT
SCP
SEOAmerican Institutes of Research
Academy for Educational Development
Continuous Assessment
Community Based Organization
Curriculum Development Center
Community Health and Nutrition, Gender and Education Support 2
College of Education
Community School
Community School Teacher (volunteer)District Education Board Secretary
District Education Standards Officer
Directorate of Open and Distance Education
District Resource Centre
District Resource Centre Coordinator
Examination Council of Zambia
Education Development Center
Education for All
Educational Management and Information Systems
Education Standards Officer
Educational Quality Improvement Program
Family Health Trust (Zambian NGO)
Fifth National Development Plan
Fast Track Initiative (USAID)
Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome
Human Resources Information System
Government of Zambia
Information Communication Technology
Income Generating Activity
Institutional Management
Interactive Radio Instruction
Learning at Taonga Market
Ministry of Education
Monitoring and Evaluation
National In-Service Training College
Non-Governmental Organization
Open and Distance Learning
Orphans and Vulnerable Children
Parent Community Schools Committee
Provincial Education Officer
President’s Emergency Plan for AIDS Relief
Provincial Education Standards Officer
Provincial Outreach Coordinator
Provincial Research Centre
Provincial Research Centre Coordinators
Quality Education Services Through Technology
School-Community Partnership
Senior Education Officer
i

SESO
SHN
SIC
SO
SPRINT
TA
TED
TESS
VCT
USAID
USG
ZATEC
ZCSS
ZIC
ZNBC

Senior Education Standards Officer
School, Health and Nutrition
School In-service Coordinator
Strategic Objective
School Program of In-service for the Term
Technical Adviser
Teacher Education Development
Teacher Education and Specialized Services
Voluntary Counseling and Treatment
United States Agency for International Development
United States Government
Zambia Teacher Education Course
Zambia Community Schools Secretariat
Zonal In-service Coordinator
Zambia National Broadcasting Corporationii

Acknowledgements
The evaluation team wishes to thank a number of organizations whose individuals helped us
during the conduct of this evaluation. The leadership and rank-and-file personnel of the three
USAID-funded programs were unfailingly gracious and supportive in supplying us with
documents and information, answering our questions, helping arrange visits to the field andto their program sites, and explaining the background and various elements of their work.
The schools, teachers, head teachers, pupils, parents/guardians and other concerned
citizens of the various rural and urban communities we visited gave us much valuable timeand information. They allowed us to interrupt their busy days to tell us of their devotion toand specific involvement in the education of children and adults in their villages andneighborhoods.
Ministry of Education personnel, beginning with the Permanent Secretary, at all levels were
cordial and collaborative, sharing with us their views on education and how they felt one, twoor all three of the USAID-assisted programs were working. MOE personnel at the zonal,
district and provincial levels especially described for us the effects of the programs on theirgeographical areas and the teachers, head teachers and other MOE employees working
therein. Personnel in USAID/Zambia were interested and helpful with this evaluation, andmindful how the information gathered and analyzed might inform their education
programming and investments for the coming years.

iii

Preface
As a brief point of introduction, this USAID/Zambia Education Program Evaluation reportfocuses on the overall impact of the USAID education program on the Zambian education
sector. Currently, the majority of USAID funding in this sector goes to three projects: (1) the
Education Quality Improvement Program 2 (EQUIP2); (2) the Quality Education Services
through Technology (QUESTT) Project; and (3) the Community Health and Nutrition, Genderand Education Support 2 (CHANGES2) Program. As such, the evaluation team spent muchof their time meeting with staff and stakeholders related to these three projects. A large partof this report focuses on the findings of these meetings and site visits.
This report does not contain a traditional Executive Summary per se. Rather, the first section
serves to summarize the overall impact of USAID education funding, and can be read as an
overall summary of the evaluation team’s findings in the field.

iv

1. Summary of USAID/Zambia Education Program Evaluation
The purpose of this study is to conduct an independent assessment of the impact of
USAID/Zambia’s education program between 2003 and 2009, and to makerecommendations for USAID's new educational strategy, programming options andinvestments for the coming five years. In order to develop a “big picture” evaluation thatmoves beyond an assessment of discreet programs, the evaluation team conducted both (1)a summative evaluation of the three largest USAID-funded education programs, an end-ofproject assessment, as two programs are scheduled to end in late September, 2009 and athird in September, 2010, and (2) a sectoral evaluation that assesses the synergies among
projects for affecting overall program sub-sectors such as teacher training and information
management systems. Thus, the findings conclusions and recommendations can inform andhopefully be useful to USAID and the Ministry of Education (MOE) as they decide how tocontinue education assistance to Zambia. Especially weighed are the merit, relevance, andcost effectiveness of USAID education programs and to what extent they have promoted
educational quality, equity and access. Also, suggestions will be offered to USAID/Zambia
as it prepares to update its Country Strategic Plan (CSP).
The three-person team spent six weeks in Zambia utilizing six complementary, inter-related
forms of gathering information: (1) review of pertinent documents throughout the evaluation
process; (2) individual and group interviews; (3) a select few focus group discussions; (4)visits to schools, classroom observations and conversations with local citizens; (5) meetings
with MOE personnel at the zonal, district, provincial and central levels, grantees' offices,non-governmental Organizations (NGOs), among others; and (6) the administration ofanonymous questionnaires to individuals and groups of people involved in andknowledgeable of one or more of the program activities.

Overview of USAID/Zambia Program
The Government of Zambia (GRZ) and the MOE have recognized the vital role that
education plays in the development of the country’s youth, and ultimately in poverty
alleviation and economic growth. The Government has taken a number of steps to increase
access to education in the past decade. In 2002, the GRZ abolished school fees for basic
education, which dramatically improved access for many Zambian children. Also, the Fifth
National Development Plan 2006-2010 (FNDP) recognized education as a priority sector forthe country. An additional development has been the increase in support for the over 3,000
community schools (CSs) in the country. NGOs, community-based organizations (CBOs),
and the GRZ have each increased their support to CSs, which has provided an additional
access point for education opportunities.
Despite these advances, the education system in Zambia remains constrained by funding
issues; the devastating impact of the HIV/AIDS epidemic on teachers, administrators,
parents and family support networks; and challenges of meeting the increased demand of
free primary education. Though access to education has increased, the quality of basic
education has not followed suit. Education statistics show high repetition and low completion
rates. Particularly at risk are orphans and vulnerable children (OVC), who remain
disadvantaged and often excluded from educational opportunities.

1

It is in this context that USAID/Zambia has been focusing on collaboration with the Zambian
MOE to support the Government’s efforts to both increase access and the quality of thelearning environment. USAID education programs have focused on ultimately enabling the
MOE to promote their basic education sector agenda while supporting policy reforms,
capacity building efforts, and improving the quality of teaching and learning.
The USAID/Zambia education program has evolved dramatically, from a relatively small
program eight years ago working on small pilot interventions to a large program working
across multiple areas including HIV/AIDS and community schools with funding in the range
of $16-20 million per year. The three largest programs in the USAID/Zambia education
portfolio are: (1) the Education Quality Improvement Program 2 (EQUIP2); (2) the Quality
Education Services through Technology (QUESTT) Project; and (3) the Community Healthand Nutrition, Gender and Education Support 2 (CHANGES2) Program. Each program is acooperative agreement, a type of grant. EQUIP2 and QUESTT began in 2004 andCHANGES2 in 2005, and the total US Government (USG) investment in these threeprograms is approximately $70 million dollars over a period of five years. These threeprograms are examined in more detail in Section 5. Programs of this report. The USAID
education program also provides budget support to the Ministry of Education through a nonproject assistance mechanism upon the attainment of agreed performance milestone. This
assistance mechanism is valued at $6.4 million over a period of six years.

Overview of USAID Impact on Zambian Education Sector
As discussed above, all USAID education programs and activities have notableachievements for the 2004/5 to 2009 period, and the three main projects have competent,
professional personnel, who are working with underserved, disadvantaged populations ofchildren and adults in rural districts and communities. Some of the significant
accomplishments and immediate impacts of USAID support include:
Enhanced teaching and learning in classrooms and non-formal settings (such ascommunity listening groups). The team noted the high regard for the value and qualityof teacher and head teacher training provided by USAID programs. Currently the issue ofteacher supply reverberates throughout the system – the need for more and higher
quality teachers, better human resource systems for teacher records, teacher retention,
improved teacher certification pre-service programs, leadership training for headteachers, in-service programs for veteran teachers. Several efforts are underway toaddress the issues from a variety of directions, but the need remains for acomprehensive plan to reach the levels of teacher supply envisioned by the year 2015.
Non-formal settings also provide unique and productive learning opportunities. There are
currently about 900 adult listening groups in Zambia, many of which are exposed tochallenging topics about life skills and community support of OVCs.
Dozens of high-quality teachers' manuals and supplementary learning materials
developed, distributed and used by different audiences. Zambian schools haveacutely lacked adequate materials for students and teachers: desks, books, instructional
materials. CSs have few to no materials, and many government schools have little more.
Most do not have access to the adopted curriculum guides and syllabi. Less than five
percent of basic schools reported having a library in 2008 and no report is given on
available textbooks. Children cannot learn to read if they have nothing to read, nor can2

teachers follow the curriculum if they have no materials. The quality educational
materials developed by QUESTT and CHANGES2 and provided to schools and colleges
are valued and widely requested.
Major educational quality and enrollment gains due to community schools andinter-active radio instruction (IRI). The immense amount of community collaborationand support for community schools (CS) is extraordinary. Over 100,000 children – inremote settings, poor, vulnerable, orphaned, half of them girls – have access to
education as a result of the work of communities, project support, IRI via radio andiPods, and MOE recognition that CSs should be formally transitioned into the Zambian
education system. Many factors must be considered in the transition, which has not been
given formal acceptance yet. In the interim these schools and communities need
continued support to maintain their level of commitment and hope for a future for theirchildren.
Strengthened management and administration of schools and MOE offices at alllevels. The Ministry of Education's (MOE's) structural capacity and role in articulating
educational needs have been strengthened dramatically since 2004. Data collection has
become more universal and streamlined, reporting mechanisms more useful and timely,
and CSs are included in the data collection. ICT infrastructure provides newcommunication capacity within the MOE and is now being extended to provinces.
Colleges have received equipment and networking assistance. Working within the MOE
structure, EQUIP2 has been able to guide planning, budgeting, and policy development
in a variety of areas – health, HIV/AIDS, community schools. More and more divisions of
MOE are requesting and benefiting from their services.
Heightened parent, guardian and community participation in educational services.
The involvement of parents, guardians, and community members in the education
system provides added value to the learning environment and a student’s ability to grow.
Community schools are just one example of how USAID support has provided support toa child’s entire support network.
Vital support for thousands of orphans and other vulnerable children (OVCs). The
team was struck by the need and success of the modest scholarship and support
program given to 20,000 orphans and vulnerable children (OVCs) in secondary schools,
most single or double orphans living below the poverty line, and demand far exceeded
supply. Scholarships made it possible for these OVCs to attend and remain in school,and directly contribute to educational access, equity, human rights and humanitarian
goals.
Providing support to school health and nutrition (SHN), further improving thelearning environment for young students. Time and again the team heard, read about
and witnessed the critical importance of USAID-supported efforts in school health andnutrition (SHN), including de-worming, vitamin A supplements, bilharzia treatment, andHIV/AIDS prevention. The importance of school health and nutrition is obvious. Sick andhungry children do not learn, and often do not attend school.

3The USAID/Zambia education program has also succeeded in supporting an impressive
range of quantifiable results. Table 1 below summarizes some of the outputs coming from
USAID programs:
Table 1. Selected USAID/Zambia Education Program Results
Indicator
Pre-service teachers trained
In-service teachers trained
Pupils reportedly benefiting from improved teaching
No. of trainers of teachers trained in 31 districts
No. of CSs assisted in six provinces
No. of community school teachers (CSTs) trained
Total number of small grants provided (QUESTT and CHANGES2)

Results
8,823
5,095
1,753,404
4,347
1,832
1,919
902

USAID support has put systems and programs in place that can also continue to havevaluable long-term impact in the future. These impacts may ultimately be better appreciatedover time, but with some addition support to maintain momentum, these activities could
continue to have positive impact for some time. Examples include:
IRI and small grants for community schools. IRI has already shown great success inZambia in contributing to educational quality, access and equity, and as a valuable
teaching aid for both GRZ schools and CSs. It often serves as the interim educational
avenue open to tens of thousands of poor Zambian youth and adults until such time ascommunity schools receive government support and are supplied with certified teachers.
At the same time, small grants to communities have been well-administered and used.
They frequently boost formal education in poor areas, provide cost-effective school
construction, are a community action and strengthening medium, and are positive nonformal education activities. Both IRI and small grants will continue to have long-term
impacts in the Zambia education sector.
School Program of In-service for the Term (SPRINT) training and manuals fortargeted areas. The School Program of In-service for the Term (SPRINT) training andmanuals, including the popular SPRINT Teacher’s Guide, have proven to be wellreceived and successfully utilized at the basic and community schools levels. Theircontinued utilization and roll out to new areas will provide additional positive impact. In
addition, it would be useful to develop a version designed specifically for secondary
school teachers.
Increased availability of data for the MOE at the administrative and political level.
Much time and effort has been put into improving the ability of the MOE to collect,manage, and utilize data for decision making. Key advances have been made, but withadditional improvements, access to accurate data can have further, lasting impact. The
development of a functioning human resources record-keeping system within the MOE
will positively affect the employment status of all Ministry personnel, from headquarters
all the way down to the schools. In addition, initial training efforts in evidence-based
decision making could be further scaled up to additional districts, and empower district
personnel to understand and assist schools, rather than “monitor” and use checklists toserve headquarters requirements. The creation of such systems, and developing them in4such a way that they are lasting and functional, will be invaluable in the long term to the
MOE and its schools.

Moving Forward
As with any development effort, there are concerns about the long-term sustainability of the
programs and the impacts that they leave behind. Like most countries, there are politicalconsiderations that can impact what types of activities receive government funding andwhich will not. A new President or Minister of Education can target new priorities, whileleaving past priorities neglected. Nevertheless, deeply embedded activities are more likely tobe sustained even if the political will is not present.
As referenced above, there have been numerous positive developments resulting from
USAID’s support in the education sector in Zambia. Many excellent materials have beendeveloped, and their continued use will be sustainable for a long period of time, particularly ifthey are reproduced in quantities that allow them to be distributed more broadly thancurrently possible. Sustainability of USAID-funded efforts to support community schools
ultimately will depend on the transition of the GRZ being able to assume full responsibility forthese community schools. Until that point, continued support for the volunteer efforts thatkeep these schools functioning is necessary to sustain the momentum of communities who
may be beginning to show signs of fatigue.
In combination, five of the current USAID-funded activities have the potential to contribute toa sustainable educational structure that will improve teacher recruitment and quality, andultimately, quality learning:
1. A MOE human resources information system (HRIS);
2. Education Leadership and Management Training course for heads of schools;
3. Workshops supporting districts in the use of data for decision making;
4. Support for COEs with technology development; and5. Integration of continuous assessment into the education system.
These points are discussed in more detail below in Section 7. Sustainability.

2. Evaluation Methodology
As graphically outlined in Figure 1, the evaluation team assessed programs independently
and across education sub-sectors receiving program investment. This approach yielded two
sets of conclusions. The first set is program-specific and assessed the coverage, impact
and degree of local participation in each program. The second set is sub-sector specificand considered the pattern in program areas investments, and in combination withinterviews and observations, identified sectors that have received funding support and theircontribution Zambia’s education sector.
The team spent six weeks in Zambia utilizing six inter-related, complementary forms ofgathering information: (1) review of relevant documents throughout the evaluation process;(2) individual and group interviews; (3) a select few focus group discussions, such as withadults in the OVC support radio listening groups; (4) visits to schools, including classroom
observations and conversations with local citizens; (5) meetings with MOE offices andpersonnel at the zonal, district, provincial and headquarters levels, grantees' offices, NGOs,5etc.; and (6) the administration of anonymous questionnaires to individuals and groups ofpeople involved in and knowledgeable of one or more of the programs and its/their activities.
Annex C includes copies of the classroom observation form and anonymous questionnaires.
The team spent eight days visiting 29 schools of all types, six District Education Board
Secretary (DEBS) and three Provincial Education Officer (PEO) offices, district, zonal andprovincial resource centers, three community radio stations and two teacher training collegesin Lusaka, Eastern and Southern provinces. Time constraints and resource limitations
obviated visits to the other provinces in which these programs function.
The formulation and administration of anonymous questionnaires and classroom observation
forms helped the evaluation team gather useful qualitative and quantifiable information andare included in Annex C. The results are reported in Annex D by sub-groups of respondents
with similar response patterns. No further breakdown by respondent group is presented inorder to retain the intended anonymity of the questionnaires. Because time was scarce –both for the busy Zambians we met and for evaluation team members – the anonymousquestionnaires offered a convenient, confidential, honed and efficient means to registerjudgments on the principal components of each project, plus a few open-ended questions at
the end so respondents could comment on any topics they wished.
The team used classroom observation forms to record what was happening in classrooms
and to discover what USAID-funded activities were evident in the schools. The forms wereadapted for Zambia from a more elaborate form developed by the Northwest Regional
Educational Laboratory in Portland, Oregon and completed by four different people, with theresults collated. The forms included information about the setting and available resources,
as well as a rating scale for components of organization and the learning process. In most
instances LTM classes were observed only during the broadcast time, but in a few instances
there was an opportunity to observe the pre-session. During the lessons, notes andobservations were records concerning the activities and resources. Then the notes wereused as a reference to rate the level of occurrences of a variety of processes as well as
record the percent of time devoted to lecture, interaction with students, and other classroom
activities. The observers looked for items such as radios, iPods, posters, teachers’ guides,
teacher or student made materials, curriculum materials and toolkits. Since mostobservations were of Learning at Taonga Market (LTM) lessons, the results provide a pictureof the process delivered via the packaged programs.

6

Figure 1: Evaluation Framework for Program and Sector Assessments
CHANGES2EQUIP2How have the USAID supported programs promoted institutional development in the MOE?
Has Technical Assistance (TA) enhanced the MOE institutional structures particularly in policy and research areas?
How have USAID programs enhanced the roles/functions of key directorates?
To what extent are the headquarters, provincial, district and zonal MOE structures and staffing able to sustain theimplementation of USAID supported programs in community and GRZ schools?

Institutional
Development

Have radio programs beeneffective in mobilizing schools
and communities to provide
enhanced support for OVC?

ICT andInformation
Management

Sub-sector (area of investment)

QUESTT

What has been the impact of theeducation management
information system on education
development?

SectorConclusions

What has been the impact of teacher education programs on pedagogy in GRZ and community schools?
What strategies work best in improving teaching skills of community school teachers?
How have teaching tools (IRI and continuous assessment) improved
teaching practice and learning in community and government schools?

TeacherTraining- Identify level offunding support andcontribution toeducation
improvement

How have education materials impacted the quality of education?
Toolkits, Basic Skills Training
iPods, mentor guides, radios, wall
Modules, activity books, sample
chartsManual, Grade 1-7 Syllabus
forms and tests.

Teaching &Learning
Materials

What has been the impact of theschool health and nutrition
programs on education delivery
systems and learning?

School Healthand Nutrition

Grants andScholarships

What has been the benefit of the
HIVAIDS prevention activities to
education quality/system
improvement?

What benefits have accrued to schools and communities as a result of
implementing the sub-grants? How has this support improved thegovernance of schools? Have scholarships increased equity?

Program Conclusions••••

What USAID supported programs have had the most impact on Zambia’s basic education system?
How have education programs leveraged other sources of funding?
What are the MOE’s perceptions (sustainability/effectiveness) of the three USAID supported programs?
How have education interventions benefited pupils in government and community schools?

Overall Conclusions andRecommendations••

Provide summary findings andconclusions and document lessons
learnedProvide recommendations for
future education programming3. Limitations of the Study
The team spent eight days in the field to observe classroom teaching and learning, visit
program sites and meet with relevant personnel who have been involved and areknowledgeable of the three programs. Field visits were carried out in Lusaka, Eastern andSouthern Provinces. Major program activities occurred with CHANGES2 in Central andCopperbelt Provinces, and in all provinces with QUESTT and EQUIP2 but other provinces
were not included in the field visits due to time restraints. Some 34 U.S. Peace Corps
Volunteers work in education in Zambia – including with QUESTT and CHANGES2 – thoughtime did not allow the team to meet with them individually or in groups.
Also, each of these three programs has many components and multiple activities, and two
and frequently all three are involved with various aspects of the same activities. For
instance, CHANGES2, QUESTT and EQUIP2 all work – to varying degrees – in the areas ofteacher training and professional development, school health and nutrition, HIV/AIDS,
community schools, school management, capacity building and institutional strengthening,and other areas. At times, it is difficult to attribute credit when things go well and constructive
criticism when results are occasionally disappointing.
The programs themselves differ in their visibility and approaches. QUESTT and CHANGES2
carry out most of their activities at the community, school, district and provincial levels, withparticipation of citizens, teachers, MOE officials and others. Their work can be photographedand documented clearly, certificates and awards are often given after training courses andother events, many publications are jointly produced, distributed and used, and people inZambia know these two programs and their decentralized personnel by name. EQUIP2,
however, is a program advocating and promoting policies, processes and procedures, andits staff are integrated with MOE mainly at the headquarters level. Its work and effects aredesigned to filter down to the provincial, district, community and school levels. Its labors do
not, for the most part, produce good photo opportunities. As two EQUIP2 staff and one of
their documents noted, "Our outputs are largely invisible, besides it would be impossible toseparate our accomplishments from those of the MOE."

4. School Visits and Classroom Observations
The heart of education does not reside in the Ministry of Education or the provincial anddistrict offices or in the quality of the publications and statistical reports. It resides in theclassrooms with the teachers and the pupils. So the evaluation team went to the schools andthe classrooms to see if the work of the USAID-funded activities had reached into theclassrooms of Zambia.
The team visited 29 schools of all types in Eastern, Southern, and Lusaka Provinces –community, basic, girls high schools, boys high schools, orphanages, grant-assisted, private
schools, basic school recently transitioned from a community school, some urban, someextremely rural. Community members spontaneously gathered to talk with the team andshow us the results of their efforts to support education for their children. In one instance, 75
adults gathered in a school building with three classrooms, but no furniture, windows, ordoors – and they all sat on the floor like the pupils do. A small grant from QUESTT allowed
them to complete the roof, but they made their own bricks and mortar from local materials,
8had completed three toilets and were working on a fourth one, and had rows of bricks laidout drying for constructing teachers’ housing. Other communities shared similar stories,
some showing off sewing and knitting machines and school uniforms made on thosemachines. Others were generating a small amount of income to help support the classrooms
and the mentors. In one community they did not have enough money to complete the roof
but the walls were up.
Most community schools were extremely poor with a high percent of orphans. Some had dirt
floors, most had no materials or books, perhaps a poster and a teacher made alphabet, or aset of numbers or a diagram of the digestive system on the wall. Chalkboards and chalkwere the only materials common across the classrooms, and in one CS the blackboard was
flattened scrap metal. Children sat on the floor, the dirt, bricks, wobbly benches, andcrowded around tables.
When possible, team members did classroom observations – not to evaluate teachers but tolook at the learning processes (see form in Annex C). A total of 18 lessons were observed,
with about 500 students present, in all grades from 1 to 7, in very different circumstances
and including a special education classroom. The visit to the special education classroom,and subsequent inquiries informed the team that USAID-funded student materials had notbeen adapted for those special education students who require a different learning modality.
Zambia has made an effort to include special education students in the educational systemand to train teachers for those students. The 2008 Education Statistical Bulletin reports
168,866 special needs students in their schools, nearly 5% of the total enrollment.
Fourteen of the classrooms were in community schools, and 13 of those classrooms used
Learning at Taonga Market (LTM) with radios or IPods. The other classrooms were in basic
schools, but all had been impacted by the USAID funded education programs with training
sessions from QUESTT, CHANGES2, or both, and all had been included in the EMIS
reporting.
The approximate percentage of time spent on various activities was recorded (see results inAnnex D). The design of LTM lessons assures that little lecturing takes place and that agreat deal of interaction occurs, so it is no surprise the observations indicate that in LTM
classrooms almost three-fourths of the time is devoted to interaction with the pupils andassessing what they are learning. In the non-LTM classrooms less time is devoted to thoseinteractions, but there is more time for reflection and for pupils to explore and be involved intheir own learning. In most instances, with the guidance of the interactive LTM programs, theobservations credit the mentors with conducting the learning process in a professionalmanner. The problem of the fast pacing of the directions was observed, but if the presession had been conducted, the pupils and teacher were familiar with what might comenext so the fast pace was not as inconvenient.
Overall, it was obvious that most of the students were eager for the lessons to begin andwere engaged, the mentors interacted with pupils, and the lessons were well done. It isremarkable that in such stark settings, with so little resources or time in school, the students
are learning.
Obviously, the work of the USAID-funded programs has filtered to the classrooms. Certainly
the mentors and teachers were on their best behavior for visitors, but nearly all of themshowed us that they can teach children. The mentors could not be doing this work without
9

the LTM instructional tools supported and delivered through the programs. The teachers inthe basic schools had all been affected by training and/or materials prepared by both
QUESTT and CHANGES2. Most of the classes we observed would have been meeting
under trees without the assistance of the program grants, and community schools would nothave the reached the current level of recognition if their existence had not been accounted
for and reported at central government levels via the assistance of EQUIP2 and thedevelopment of guidelines for CSs.

5. Programs
CHANGES2Background:
CHANGES2 began June 20, 2005 and will conclude September 30, 2009. The original
cooperative agreement (grant) between USAID and the American Institutes of Research(AIR), the principal consulting firm implementing the program was for an estimated amountof $17.5 million for the 51 month period. New, additional requests and funds from the U.S.
Government (USG) channeled through USAID increased the total life-of-project funding to$28.45 million.
CHANGES2 is a follow-on to the USAID-financed CHANGES program, which functioned
from late 2000 to early 2005, concentrating on School, Health and Nutrition (SHN) needs inEastern Province and three districts of Southern Province. An evaluation of CHANGES
carried out in February-April, 2005 found solid evidence that SHN had positive effects on the
cognitive development of students, and that teachers can be trained to be full participants inthe administration of drugs and improvement of the health and nutrition in their schools.
Community Sensitization and Mobilization (CSMC), focusing on issues of gender andHIV/AIDS, spread to all 11 districts of Southern Province and was credited with changing
deeply rooted cultural beliefs and behaviors. A third program component was a small grants
mechanism, which drew plaudits in the report.
The main program areas of CHANGES2 are: (1) Teacher Education; (2) School Health andNutrition (SHN); (3) HIV/AIDS; (4) Small Grants; (5) School-Community Partnerships; and(6) Scholarships for Orphans and Vulnerable Children (OVCs). The program aims tostrengthen basic education teachers' professional skills, with a special focus on HIV/AIDS
prevention and mitigation. Also, it supports the MOE's SHN activities to new programmaticand geographical areas, with the goal of empowering pupils, teachers and community
members to improve education, gender equity and health in schools and communities.
The central office is in Lusaka, and the program assists the Government of Zambia (GRZ) indelivering strengthened basic education in a decentralized manner in four provinces:
Southern; Lusaka; Central; and Copperbelt. Provincial program offices are in Livingston,
Lusaka, Kabwe and Ndola to coordinate district and zonal education activities. CHANGES2
also delivers, with the assistance of a Zambian NGO, scholarship support services for needy
secondary student OVCs in the same four provinces, plus Eastern and North Western
Provinces. At its apex, CHANGES2 had 56 employees, many of them deployed at the
district, zonal and provincial levels.

10Implementation and Results:
CHANGES2 is a multi-faceted program with many inter-related components or elements, allcontributing to one or more of the six core program areas. Much like QUESTT and EQUIP2,
it has added tasks and activities both due to conditions on the ground in Zambia and throughnew or heightened emphases from the USG and USAID. All three programs have had tocope with fluctuating exchange rates in Zambia, which have presented problems forplanning, financial forecasting, budgeting and procurement. Lastly, all three have had toreport on different sources or "streams" of USG funding. The original goal was for
CHANGES2 to work in 400 new basic schools annually, for a four-year total of 1,600
schools. As the program ends, it has worked in 1,673 schools.
Collaborating with the MOE at various levels, CHANGES2 has concentrated on improving
curricula and teaching methods in basic schools and teacher training colleges throughtraining courses and the development, testing and use of textbooks, teachers' manuals andguides, supplementary materials, posters for students, community committees and others.
The principal activity in pre-service teacher education has been Teaching in the Window of
Hope (TWH), which concentrates on HIV/AIDS education and prevention. In-service teacher
education has emphasized development of a SPRINT (School Program of In-service for aTerm) Teacher's Guide, training of teachers and MOE institutional strengthening at theprovincial, district and zonal levels.
Table 2. Selected Program Results for CHANGES2
Indicator
Pre-service teachers trained
In-service teachers trained
Pupils reportedly benefiting from improved teachingStudents participating in SHN interventions
No. of COE tutors receiving TWH pilot-testing (2008)
No. of student teachers in TWH pilot testing (2008)
No. of trainers of teachers trained in 31 districts
No. of student teachers trained, mainly in SPRINT

Results
8,823
5,095
1,753,404
911,897
279
5,398
4,347
6,600

Evaluation team members found when visiting program sites in the provinces that
CHANGES2 helped build increased capacity in zonal, district and provincial resource
centers, the delivery of valued library books, and capacity building in SHN, HIV/AIDS,SPRINT and other areas. District Resource Centre (DRC) and Provincial Research Centre(PRC) staff were particularly praiseful and enthusiastic about the process of capacity
building and the products themselves.
The SHN sub-sector represents a plethora of activities. Building on the efforts of CHANGES,GRZ health workers collaborate fully with basic schoolteachers, continuing the collaborationof the Ministries of Health and Education, working closely together at the local school andcommunity levels. Besides addressing USAID/Zambia's education SO and indicators,
CHANGES2 contributes to USAID's Health SO, especially through improved child health,adolescent reproductive health, and malaria prevention. The Health Promoting Schools
Initiative covers de-worming and micronutrients, water and sanitation, and classroom
instruction. 3,155 teachers/head teachers and 523 health workers have been trained in SHN,
while 676,010 students have been de-wormed, nearly 100,000 were given vitamin A
supplements, and a large number was treated for bilharzia. CHANGES2, teaming with11

Africare, has worked with 200 schools in sustainable school gardens. From what theevaluation team could discern, it appears that these gardens have had only mixed success,much like school gardens and "production units" noted in the 2005 evaluation of CHANGES.
School-community partnerships involve community mobilization, capacity building andtraining, as well as opening up dialogue between education personnel and local leaders andcitizens, all geared to enhance the quality of education. They aim to contribute to the MOE's
capacity building and decentralization initiatives, and USAID's emphases in civil society
strengthening, democratization, and a stronger role for communities in the management ofeducation. Theater and drama in schools and communities are part of this effort. A review of
documents and conversations with Zambians in and outside the MOE reveal differences of
opinion regarding the centrality of SCPs.
CHANGES2's sub-grants or small grants activities seem to have contributed to several ends,
including SCPs, capacity building, access to and quality of education, and OVC support andHIV/AIDS prevention. Capacity building includes Sub-grant Management Committees being
training and supported in 31 districts across the four focus provinces, 183 grantmanagement committees – involving schools and Community Based Organizations (CBOs)– trained and supported, and ties with and strengthening of four local partner NGOs. OVC
support and HIV/AIDS prevention-related grants have aided 183 schools and CBOs, 12
resource centers, one College of Education (COE), and a reported 122,732 children.
CHANGES2 has produced and distributed various practical, high-quality publications in thesmall grants area. One such document is "Infrastructure Improvement Grants for Community
Schools: Operational Guidelines." It would be highly useful to any GRZ ministry, particularly
one with decentralized services dealing with citizens and communities, as well as NGOs and
CBOs throughout the country.
The high school scholarships for OVCs relate to USAID's humanitarian and human rights
goals as well as to the MOE and USAID objectives of access to and equity of education. Six
high schools with this program were visited, two of them unannounced. This is a highly
popular program area with the GRZ, MOE, secondary schools and the Zambian public. To
date, 20,654 girls and boys – most of whom could not otherwise attend school due topoverty and lack of support – have benefited. A modest sum to cover tuition, examination
fees, boarding and spending money is given each recipient. Importantly, given to each girl is
a comfort kit, which includes three pairs of underpants, five pads, a plastic bag and a bar ofsoap for her to use during her menses.
The evaluation team met with a group of OVC girl scholarship recipients. One pupil said, "Mymother passed away when I was eight months old. My dad passed away when I was five. I
want to go to the university to study economics and help other people." Another girl OVC
told us, "I am a double orphan and, with the help of people in my neighborhood, was only
able to come to school this year. I want to study journalism and bring information to people."
AIR and CHANGES2 partnered with the Shoprite supermarket chain to provide the soap,
while the Johnson & Johnson firm supplied some funds. In 2009, the program is supporting
8,000 scholarships, 5,600 of whom are girls, and 1,720 of these secondary students areworking as HIV/AIDS peer educators. Data collected by CHANGES2 and Family Health
Trust, a Zambian NGO and sub-grantee, reveal that the attendance, school performance,and progression of scholarship recipients were better than the national average.
12

Psychosocial support (PSS) to the OVCs is another facet of the scholarship activity.
CHANGES2 and the MOE developed a PSS model for children traumatized by HIV/AIDS,
and trained 239 teachers and community members at 142 schools so that more than 2,000
children received PSS.
What became, in effect, a seventh program area – while reinforcing several of the previous
six – is CHANGES2's multi-pronged support of community schools. The main program areas
for this have been training of teachers and head teachers, MOE capacity building, provisionof quality materials, and infrastructure improvement grants, all geared to quality education in
CSs. All 1,832 CSs in CHANGES2's four focus provinces were included, plus Luapula andEastern Provinces.
Table 3. Results for CHANGES2 support to community schools (2006-2009)
Type of Support
ResultNo. of CSs assisted in six provinces (all CSs)
1,832
No. of CSTs trained (approximately one per school)
1,919
Tool Kits of teaching-learning materials provided (one per school)
1,832
No. of COE (ZATEC) student teachers supported in 135 CSs
230
Another notable activity that evaluation team members heard about concerned CS headteachers. CHANGES2 and the MOE designed a five-day school management course for CS
head teachers, and trained 1,807 of them in personnel management and support, record
keeping and working with the community. CHANGES2 provided sub-grants for infrastructure
improvement to 235 CSs, and strengthened 235 Parent Community School Committees(PCSCs). Related activities directly concerned MOE capacity building. CHANGES2 gave
technical assistance for policy development and system strengthening regarding CSs. These
included Operational Guidelines for Community Schools, a much-used MOE publication
developed with EQUIP2’s assistance, in both full and abridged versions that was utilized in
CS Teacher (CST) training, training of district officers in five provinces in effective support to
CSTs, and two MOE support visits to each of 1,832 CSs. Finally, overall program field-levelmonitoring and reporting were carried out by MOE, CHANGES2 and partner personnel tocore schools, CSs and high schools.

Overall CHANGES2 Findings:
Evaluation team members heard universal high praise for CHANGES2, its personnel andwork in the 10 districts of Eastern, Southern and Lusaka provinces that were visited, as well
as from MOE officials at headquarters. About the only complaint, or shortcoming (if one canlabel it as such), was that CHANGES2 should be operating in more provinces and districts
so that they also could benefit from the program. Teachers, head teachers and community
leaders and activists are particularly appreciative of collaboration with CHANGES2. On the
anonymous questionnaires, these respondents rated CHANGES2 very high in all 13 listed
categories of activities – higher than 4.0 on a 1 to 5 scale with 5 being the highest rating.
The SPRINT program and value of the small grants to communities to support their schools
were rated exceptionally highly at 4.9. It should be noted that district, provincial and zonal
MOE personnel also ranked CHANGES2 very highly in all categories, with the average for
only one question below 4.0 – at 3.9. For overall results see Annex D.

13

Teachers, head teachers, and district, zonal and provincial personnel throughout the areas
visited – and presumably throughout the remainder of Zambia as well – are in great need ofquality teaching texts, guides and manuals, and supplementary materials for pupils at each
grade level. These respondents indicated that CHANGES2 delivered these, trained them intheir use, and its personnel have followed up with them to monitor their use and answer
questions. Especially appreciated are the SPRINT methodology and materials, many SHNrelated booklets and materials, TWH by COE personnel, practical publications on community
schools, and useful, well-written documents and guidelines on the implementation of
infrastructure improvements grants by communities. As the Senior Planning Officer in the
PEO Southern Province remarked, "All of our 11 districts have worked with CHANGES2,
and there has been much beneficial training with the District Resource Centre Coordinators(DRCCs) and Provincial Research Centre Coordinators (PRCCs). Teachers and schools
appreciate their work. CSTs have particularly benefited, such as with head teacher training
and being able to manage their schools more easily. CHANGES2 has helped us complete
dozens of small grant projects, some of them for income generating activities (IGAs)."CHANGES2 and its local NGO partner, Family Health Trust (FHT), received accolades forestablishing uniform, clear and equitable criteria for selection, working with secondary school
personnel, and then helping administer the scholarship program for OVCs. Evaluation teammembers visited six secondary schools – including two unannounced visits – where theprogram operates and found it to be functioning smoothly, with many of the scholarship
recipients serving as HIV/AIDS peer educators. In all sites visited, school personnel
administering the program and the students said that these modest scholarships are thereason they can enroll and stay in school. The comfort kits given to the girl scholarship
recipients are much appreciated and of considerable physical and psychological benefit. The
main laments cited were that there were not many more such scholarships available (in theschools visited, demand outstrips supply by roughly 250%), and these OVCs foresee little orno possible manner in which they can proceed to post-secondary education.
MOE personnel, largely in the field, gave a rating of 4.0 on the question of to what degree
the MOE is stronger now because of CHANGES2 in-service training and services forteachers. In addition, a number of district and provincial MOE officials, including Senior
Education Standards Officers (SESOs), remarked very positively about the manner in which
CHANGES2 staff have worked collaboratively with them and their colleagues, and of thelasting achievements and institutional strengthening the program has left behind. They told
evaluation team members about the cascading nature in which CHANGES2 frequently
worked to develop materials and training courses, involving and integrating MOE personnel
from the school, zonal, district, provincial and headquarters levels in the development of, forexample, SPRINT, and in the field testing, deployment and use of the materials, as well as
the M & E that followed.
When respondents rank ordered the importance and value of nine aspects of CHANGES2'swork, teacher education and professional development was by far the highest ranked. Next
most valuable in their experiences and opinions was training courses for teachers and headteachers, closely followed by school health and nutrition. Among other factors, this reveals
how concerned district and provincial officials – in addition to teachers themselves – areabout the quality of the teaching/learning process, and how much better teachers canfunction with quality training, follow-up support and good, readily available manuals, guides
and other materials.
14

QUESTT
Background:
As Andrea Bosch writes in a World Bank publication, "Learners retain approximately 10
percent of what they read, 26 percent of what they hear, 50 percent of what they see andhear, but 90 percent of what they say and do." It is no wonder that interactive radio
instruction (IRI) – a methodology developed to turn a typically one-way technology into a tool
for active learning inside and outside the classroom – not only continues to be popular 33
years after its invention, but also is growing in popularity, particularly in developing countries.
Newer than the Internet, IRI is the use of interactive lessons in which an audio component
delivered by an "audio teacher" through a radio, audio cassette, IPod or MP3 player andclassroom activities carried out by the learners are carefully integrated. Bosch goes on to
say:"IRI is distinct from most other forms of distance education because its primary
goal has been the improvement of educational quality. Unlike many distantlearning efforts designed to address issues of access, IRI began as a classroom
tool to counteract low levels of teacher training, poor achievement among
learners, and limited resources. While IRI has demonstrated that it can be used
to expand access and increase equity in both formal and non-formal educationalsettings, it retains an emphasis on quality improvement through a development
strategy and methodology that require active learning, attention to pedagogy,
and formative evaluation to be included in the design."
IRI began in Zambia in 1999/2000, when the MOE and Educational Broadcasting Service
asked EDC, with USAID funding, to help develop and deliver IRI programs for grades 1-5 forout-of-school children in Zambia. Many of the most disadvantaged, poor, difficult-to-reachchildren in Zambia were targeted. The programs lasted merely 30 minutes and wereassisted by volunteer mentors from the communities. The sites were rustic, frequently undera tree, or in roofless walled structures, or an open-air space with a crude roof. Several
assessments of student learning show that IRI learners achieve as well as and sometimes
better than students in GRZ schools. This first phase of IRI lasted until 2004, with enrollment
reaching 38,513 children.
USAID/Zambia selected EDC under USAID’s dot.EDU mechanism covering the period
October, 2004 to September, 2009. QUESTT now provides technical assistance to support
teacher training, IRI program development for grades 1-7, and capacity development of the
MOE's Directorate of Distance Education (DODE). It also provides support to local
communities through partnerships with NGOs and community radio stations, assists OVCs
and trains CSTs. The overall goal was improved quality of basic education delivery systems.
Included aims were: (1) IRI adoption in GRZ schools; (2) in-service and pre-service teacher
training strengthened; (3) math and science learning outcomes improved; and (4) access togood quality basic education offered to more out-of-school children. Under the general goalof mitigating the impact of HIV/AIDS, community care, community radio stations, andadvocacy and support networks were also included as components. The five-year USAID
grant of $7,879,816 was eventually increased to $16,598,522, due principally to the successof the project and to additional, new requests being made to EDC/QUESTT by the USG and
15

USAID. PEPFAR and USAID's Fast Track Initiative (FTI) represented sizable portions of thisadditional funding.
Implementation and Results:
A priority of QUESTT was to complete Learning at Taonga Market (LTM) for grades 1-7.
Other aims were to enhance the quality of services offered in IRI centers, strengthen thecapacity of the DODE, improve the broadcasting signal, help increase community andprivate sector support, and monitor the quality of learning. For lower basic schools in 2004,
150 IRI programs for grades 1-3 were revised, and grade 4 was revised in 2006. At the
middle basic school level, grades 6 and 7 were developed, field-tested and made operationalin 2006 and 2007, respectively. Grade 5's 150 IRI programs were created in 2003. The
Director of Teacher Education and Specialized Services (TESS) in 2005 gave approval toutilize IRI in GRZ schools which, over the last four years, has led to approximately 3,000 of
the 4,000 GRZ basic schools using IRI to one degree or another.
EDC and DODE have kept records on the costs of their IRI programs over the years. Even
with broadcast expenses increasing in later years, the most recent data show that thebroadcast expense to supply IRI programs to children is US 11 cents annually per child, or
currently about K550. If costs for radios, teachers’ guides, and program schedules forschools are included the total recurrent costs per child is 32.5 cents. This is in sharp contrast
to the current MOE statistic of US $27.00 yearly to support one child in a GRZ basic school.
Most research comparing student performance in GRZ schools or CSs using IRI against
control GRZ and community schools without it show that students with IRI outperform thestudents having no IRI. For example, in grade 2 testing in 2006, IRI students in GRZ schools
outperformed their peers in non-IRI schools in English (10.2 vs. 8.1) and mathematics (7.1
vs. 5.6). Grade 4 assessments conducted in 2007 by QUESTT showed increased learningby students having IRI programs, though it was less than in 2003 when EDC's antecedent
IRI project had assessments carried out by external evaluators. IRI in Zambia has had thegreatest impact on student learning in local language literacy and mathematics, but less so
in social studies. QUESTT and EDC training for GRZ teachers and supplying them withteachers' guides and radios were much appreciated, according to documents reviewed andpersons contacted.
QUESTT’s Work with Communities:
Other activities fortified QUESTT's outreach and boost to communities and their schools.
Ten Provincial Outreach Coordinators (POCs) were hired and deployed to the provinces.
Working with PEO's offices, their main tasks have been community sensitization, face-toface training and radio training, mobilizing NGOs and supporting CBOs, monitoring, datacollection, and managing grants. The effective work of these POCs, coordinated by anexperienced former educator and MOE official (ex-DEBS), account for much of the successof QUESTT's decentralized approach. Conversations with many PEO and DEBS employees
reveal they have a keen interest in and monitor the QUESTT/IRI program.
Important also is QUESTT's work with nine community radio stations, three of which werevisited. QUESTT provided equipment and training for producers and station managers,
expanded the stations' coverage, helped them rebroadcast IRI lessons, and enabled them topresent "Education for All" programs, which include dramas, public service announcements
and interviews of local interest. Six of these local radio stations (in Eastern, Southern,
16Northern, Western and Luapula Provinces) have QUESTT assistance for producing andairing the OVC Life Skills Program titled Our Family, which foster life skills development of
OVCs and aid communities to support OVCs. The three weekly radio programs are a 15
minute soap-opera-like drama, a 15 minute topic reinforcement IRI program for children, anda third program involving 30 minutes of feedback from the community. At present, there areapproximately 900 adult listening groups in Zambia. Evaluation team members met in focus
group discussions with representatives of 10 of these listening groups.

Challenges and Limitations:
Various limitations, challenges and problems have affected IRI and QUESTT during the
2004-09 period. The Zambia National Broadcasting Corporation (ZNBC) solicits high fees to
air the IRI broadcasts, now totaling about K 320,000,000 annually or approximately 90% of
DODE's low budget (0.02% of the total MOE budget). Radio reception is poor in many rural
areas, and the ZNBC signal is increasingly weaker in numerous zones. A key Thursday
morning nationwide time slot for IRI was eliminated in 2008, as well as the Grade 1 Tuesday
morning broadcast. DODE officials attest that staffing levels are quite low; the 10 POCs
added were a much appreciated, partial remedy. DODE suffers from a great lack of funds forprinting and distributing mentor guides when these are needed, which is often. Moreover,supplementary materials for the IRI pupils and adults are almost totally lacking.
The retention of voluntary mentors is also a big challenge. These community members,
many of them talented and dedicated, must sooner or later seek jobs where they makemoney, rather than receive at times a modest allotment of food from the community or othersmall in-kind contribution. The high turnover and attrition rate are one of the principal
problems for the CSs and IRI, the evaluators learned. The need for more radios and therepair and replacement of radios are big problems. The windup radios, produced and sold by
the FreePlay Foundation, include solar panels as an optional power source.
The biggest challenge to IRI and CSs is the lack of policy, organizational, legal and financial
recognition and support from the MOE. The bill to provide legal status for CSs is part of alarger and more complex education-restructuring bill that is currently being debated. The
team was informed that various high-level MOE officials maintain that because IRI cannot be"time-tabled" (or treated as a separate, distinct subject) it does not belong in the curriculum
or deserve to be broadcast. As one long-time educator said:"The attitude in the MOE was that IRI could never compete with non-IRI schools,
and that community schools could never come up with qualified teachers. But
they have. There are many CSTs who are qualified to be regular teachers, butcannot afford the costs. There are easily over 2,000 of these qualified teachers.
They have practical experience, commitment and enthusiasm. There is no
reason why they should not be smoothly incorporated into the MOE and regularteaching profession."

Overall QUESTT Findings:
Visits and meetings with PEO and DEBS offices, schools of all types (and varying from lowperforming to high-performing), PRCs and DRCs, NGOs and parents, guardians and local
citizens in Eastern, Southern and Lusaka provinces offered ample data and opinions
17

concerning IRI and QUESTT. Evaluation team members were impressed by how much headteachers, teachers, volunteer mentors, district and provincial MOE officials and parents knew
about IRI and QUESTT and how strongly they support IRI and its various program
components. They praise IRI for providing structure for lessons and students, good learning
practices such as introductory and follow up activities, practice in careful listening,
components that are fun and active, and an important aid and support to teachers. Children
throughout Zambia hear and learn a common, correct form of English. Evaluation teammembers witnessed these benefits when we carefully observed IRI classes at various gradelevels in numerous community and GRZ schools.
Time and again the evaluation team witnessed the value of IRI and QUESTT in poor ruraland urban neighborhoods where IRI was the principal and sometimes only means ofeducation. That children in these conditions are performing as well or better in school thantheir peers in more comparatively advantaged areas and schools is remarkable. For
example, the team visited an orphanage in the poorest urban neighborhood in Lusaka where
IRI was the only educational tool available to the pupils. These orphan children werelearning in every IRI class at different grade levels we observed. The volunteer teachers had
no texts or manuals, and the pupils had no texts or even supplementary materials of any
kind. Yet, last year on the national examination to enter 8th grade, 46 out of 48 pupils fromthis orphanage/neighborhood taking the examination passed, the highest pass rate teammembers heard recounted during the evaluation.
Zonal, district and provincial MOE personnel praised the IRI mentors' guides, though many
schools lacked guides for most of the grade levels, although guides had been promised by
MOE and had not been received. Evaluation team members noted how well the lessons
integrated awareness and knowledge across subject areas, such as social studies andmathematics. We further saw how many SHN concepts – such as proper hygiene,
importance of healthy drinking water, preventing malaria, and the proper disposal of waste –were effectively woven into the IRI lessons at various grade levels.
The principal challenges and shortcomings of IRI and various QUESTT elements havenearly all been mentioned above, and are largely beyond their control. Inability to receive aradio signal or a reliable radio signal was a common problem. Enough radios and keeping
them functioning well were noted. The acute lack of mentors' guides and student materials
were frequently stated as problems, threatening the very existence of IRI in many locales.
Teachers, head teachers, district and provincial MOE officials and others lamented the factthat the MOE does not give sufficient attention and policy, administrative and financial
support to IRI.
Anonymous questionnaires provided valuable information, with more than half of the
respondents to the questions about QUESTT being district, zonal and provincial MOE
officials (see table of results in Annex D). Respondents gave the highest ratings (4.4 out of5) to IRI and QUESTT for providing increased access to education for children in Zambia,
and the usefulness of the small grants to support IRI programs. Next most valued are theextent to which QUESTT has helped strengthen DODE and the quality and expertise of
QUESTT's technical staff, and the usefulness of the community sensitization workshops
carried out by the POCs. The district, zonal and provincial MOE personnel and the CSTs all
rated these workshops highly. To a person, they praised the professionalism, energy,
dedication, and practical assistance demonstrated by these three POCs. Also receiving
18

strong, high ratings (at 4.0) are respondents' opinions of the quality of QUESTT's IRI
programs, and the value of IRI programs for OVCs.
The lowest collective ranking (2.8) was given to the question concerning the level of
effectiveness the MOE and QUESTT had in mobilizing communities to support volunteer
teachers. Discussions before and after questionnaires were completed revealed that most
respondents blame the MOE and GRZ for not helping communities to recruit, train and pay
for teachers for their local schools. The usefulness of Fastele! Fastele!, the series of 15minute radio programs implemented by QUESTT for teachers and volunteer teachers
received a ranking of 3.0. However, teacher training college personnel gave it a much higher
ranking. Written comments on the questionnaires and later discussions showed that manypotential listeners could not receive the program due to inadequate radio signals, and others
because it was aired during an inconvenient time slot.

EQUIP2
Background:
EQUIP2 works with the MOE to define policies, establish processes and procedures, deploy
tools and data systems, and build capacity to improve the MOE's ability to deliver quality
education. EQUIP2 began March 11, 2004 and will conclude September 30, 2010. The
original grant from USAID for these six and one-half years was estimated at $13,973,991
and was then revised to $26,473,991 in 2007. The program has received $ 20.5 million in
funding thus far, including $3.3 million from PEPFAR. EQUIP2 evolved from a previous
USAID-financed project titled Ed*Assist and EMIS; AED and USAID have been working with
the MOE's EMIS since 2000. AED is the prime grantee for EQUIP2; sub-grantees include
AIR, Comprehensive HIV/AIDS Management Programme (CHAMP), Society for Family
Health (SFH), and the Research Triangle Institute (RTI). AIR, a sub-grantee to AED on
EQUIP2, carries out the Continuous Assessment (CA) component. Approximately $4.8
million of USAID funds have gone to CA in the last four years, according to EQUIP2 figures.
The program focuses on providing high quality technical expertise in a wide range of areas,
adapting to the changing priorities of the MOE. Between 2004 and 2006 the projectcontinued work on the existing EMIS and expanded to include policy research, institutional
management, continuous assessment, and integrated information management. In addition,an HIV/AIDS workplace program was added, using PEPFAR funding. Moreover, in 2007 theproject began work on an institutional response to the Community Schools. Even more
responsibilities were added in 2008 with school health and nutrition, budgets, and a humanresources information system.
The current work focuses on the following components: (1) Policy and Research; (2)Monitoring and Evaluation; (3) Integrated Information Management; (4) ICT Infrastructure
and Network; (5) Continuous Assessment; (6) School Management and Leadership Training:(7) HIV/AIDS Work Place Program; (8) School Health and Nutrition; (9) Institutional
Management and Development (financial management); (10) Provincial Component
Activities; and (11) Project Administration and Management.
The project is physically located within the Planning Directorate of the Ministry of Educationin order to maximize its capacity building role. As such, its work and contributions are likely
19to be identified by schools as MOE, rather than EQUIP2 efforts. Much of the work involves
internal policy, procedures, guidelines, and infrastructure to support the MOE. However,
EQUIP2 also works on the Human Resources Information System (HRIS), Community
School Guidelines, IT development and networks, and on HIV/AIDS and SHN issues,
extending its impact to people beyond schools and MOE headquarters. The successes of
EQUIP2 are successes of MOE, and in fact their work with data collection and training ofprovincial and district personnel affects nine provinces, 72 districts, nearly 9,000 schools,
and some 78,000 teachers and administrative personnel.
EQUIP2 currently has a staff of 22 who work within the MOE structure. Twelve were addedin the last eight months to respond to MOE requests for Technical Assistance (TA) to fill
identified gaps as it moves forward in several key areas, some of which included M&E,
budgeting, and school leadership training. To reach out from headquarters and build
capacity at district and provincial levels, four Provincial Education Advisors (PEA), each withdriver and vehicle, were assigned to Western, Eastern, Northern, and North Western
Provinces. EQUIP2 works in a political environment and cannot move forward independently
on its own timeframe but must respect the speed and direction of the MOE. That direction isoften subject to external and uncontrolled factors, such as governmental financing, high level
position appointments, and passage of bills that affect education. Although EQUIP2 began in
a context that was not welcoming to projects, trust has been built over the years as EQUIP2
supported endeavors increased the prestige and credibility of the MOE, and their assistanceis now requested in many areas. When processes are understood and implemented at MOE
headquarters level, budgeting and using technology for communication for instance, thosebenefits filter down through the MOE structures at provincial and district levels.
The EQUIP2 personnel are highly qualified for the broad range of tasks they have beenrequested to provide. They have deepened the institutional capacity of MOE’s central officewith timely data, strategies for development of more sound administrative practices, andprocedures and policy development. EQUIP2 has been able to adapt relatively quickly to
MOE priorities, and assist with their day-to-day issues. It is very clear that its personnel arevalued by MOE officials for their expertise and competence in responding to the need fordevelopment of Ministry policies for HIV/AIDS in the workplace, guidelines for Community
Schools, improving the quality of the EMIS data, providing critical and timely information to
the Ministry management, and upgrading the Education Statistical Profiles. They have used
the updated information and expertise to assist the MOE with budgeting and planning,developing schemes for M&E, expanding the information technology infrastructure
throughout the MOE, working to provide ICT to the Ministry components in the provinces,and moving toward evidence based decision-making across the system.

Overall EQUIP2 Findings:
EQUIP2 responsibilities are complex and its tendrils reach into multiple corners of the MOE.
Some of the components are related to components in other USAID-funded projects, andalthough the activities may not overlap, they benefit from coordination of efforts to maximize
the dissemination of the work, such as with HIV/AIDS, SHN, and Community School
Guidelines. The work of EQUIP2, by its very nature, requires collaboration and working witha broad variety of partners that extend beyond the MOE and other USAID programs.

20

The EQUIP2 work plans note that the shortage of MOE human resources may beproblematic and that EQUIP2 would respond with short-term TA and collaboration with
provincial and district personnel to give them more responsibility and freedom to implement
the activities. Their stated goals for 2009 indicate they will focus on providing some of thesame capacity building to schools, districts, and provinces as they have provided at the
Ministry level, but many of the activities with districts and provinces are not recognized as
separate from MOE initiatives.

Information management and IT infrastructure:
Because of the improvements in data collection, the MOE can now respond quickly to many
political demands due to more timely reporting procedures for the EMIS system. The EMIS
system has been evolving for years and consists mainly of the data gathered on the school
census forms, with some changing components over the years. It is now collected fromnearly all schools and has assisted with understanding the magnitude of the CS issues. The
data are input at the provincial level and compiled at the national level for the annual
Educational Statistical Bulletin. More important than the bulletins themselves is the timely
availability of data to advise MOE decisions. The improved flow of information back and forthprovides more timely data than in the past and allows for quick responses from MOE. Highlevel decisions about where to deploy teachers and where to build schools were guided by
MOE data, instead of past practices which were arbitrary and subject to personal
preferences. The transparency of published data provides rationale for those who makedecisions, and oversight by citizens who know what was supposed to take place in theirregions. In addition, MOE can document progress toward their goals because of the data,and external organizations are accessing the data for their own use.
The activities of EQUIP2 are not as visible in the field as the work of QUESTT and
CHANGES2, so fewer anonymous questionnaires about their activities were distributed onfield visits. Only people who indicated they were familiar with some of the components
completed the forms, and they were asked to leave items blank if the topic was unfamiliar.
The majority of the respondents were MOE personnel at headquarters, province, and districtlevels. EQUIP2 activities were ranked highest in the areas of information management andinformation technology infrastructure. Their assistance with EMIS is recognized by many andidentified by the respondents as having a high level of quality and usefulness. The next
highest was the area of IT equipment and training. The category that ranked the lowest was
the extent to which the MOE can sustain the valued EQUIP2 programs of assistance. When
asked which areas of assistance were most important, information management and IT
infrastructure were again ranked at the top (see Appendix D).
Written comments on the anonymous questionnaires and interviews with personnel at theMOE, provincial, and district levels indicated a concern that capacity and sustainability for
EQUIP2 services are not being built – either in the Ministry or in the field. The most prevalent
responses expressed concern that the support and capacity building are confined toheadquarters, but several at headquarters voiced concerns that the role of EQUIP2 often
appeared to be leadership rather than TA. The respondents recognized the need for highlevel technical support and EQUIP2 expertise, particularly in the area of IT infrastructure, but
indicated they want to have capacity built with Ministry employees, not just structures, inorder to carry out and sustain the work. Provincial and district personnel also want more
involvement, more input, and more training in information and IT management.
21

The desire for more training and involvement in leadership roles may arise from a variety offactors, but the fact that MOE staff want to obtain more skills and responsibility is notnecessarily a negative perspective on the work of EQUIP2 nor is the type of training required
within their purview. Several EQUIP2 staff who were interviewed by the evaluation team arehighly skilled, many with technical expertise that few possess. Some of those skills are
acquired over a long period of time, not through short training courses.
Training is and has taken place in several instances. Recognition of the need to build M&E
capacity in the MOE was addressed with TA from EQUIP2 and training for 14 MOE staff at
the University of Zambia. This area continues to develop and will expand their training by
working with provincial offices to acquire M&E skills.

Policies, procedures, and guidelines:
EQUIP2 has provided guidance and TA in the development of policies, guidelines andprocedures in a variety of areas. They assisted with the development of the “Community
School Guidelines” which provide enabling policies for communities, and a framework for thedevelopment of legislation to extend legal status to CSs. Support was provided to
institutionalize MOE management teams that provide forums for planning and to move thenational education plan forward. Committees that include cooperating partners and otherministries have been established to help coordinate activities.
On the anonymous questionnaires, the EQUIP2 work with HIV/AIDS workplace policy andimplementation is rated as the third most valuable assistance. The guidelines and processes
that were developed for HIV/AIDS in the workplace and SHN brought together a variety oforganizations to coordinate and expand the work. The HIV/AIDS Workplace programprovided Voluntary Counseling and Treatment (VCT) services and HIV awareness forteachers at schools, union events and Teachers’ Health Days. MOE reports that 35,105 staff
and family members accessed VCT and nearly 41,000 have been sensitized. Working withpartners, SHN policies have been updated and will continue to be expanded to improve thehealth and sanitation environment.

District and school level capacity building:
EQUIP2 is guiding three important and far-reaching efforts that will build capacity at school
and district levels.
Human Resources Information System (HRIS): Efforts are currently underway to bringorganization to the Human Resources records for teachers and all MOE staff and connect tofunctional systems that currently exist. Simply meeting with staff in the human resource
offices highlighted the need – stacks of thick files were scattered throughout. It will requiretime, continued technical assistance, and a good deal of patience to develop a streamlined
data system and ministry expertise to capture and manage the information from those paper
files which require multiple approvals, signatures, and movement from office to office.
This is not a speedy process. Existing practices, procedures, and inter-connections must firstbe identified, systematized, and implemented before an electronic system is put in place.
Otherwise the system will merely reflect the current chaos. Presently, files are easily lost,22

misplaced, decisions delayed, pieces missing or altered, with teachers leaving school tocome to Lusaka in an attempt to straighten out their files. Personnel get to carry their own
files. Successful implementation of the HRIS system will build institutional structures at the
MOE and be felt throughout the system, all the way down to the schools.
School Management and Leadership Training: EQUIP2 became involved in school
management training when the MOE recognized a need for enhanced records managementand data use at the school level. In the process, a design for a new approach to training
school leaders took shape. No systematic training was in place in Zambia at the time.
Working with several directorates and a variety of political factors, a broader approach totraining school managers was designed that involved other entities from prior school
management training. There was a desire to assure that this new training program included
educational leadership, recognizing that research on effective schools indicates that a strong
educational leader is a key contributor to high levels of student achievement in a school.
Although some modules are still being designed, one of the five is identified as Instructional
Leadership. This approach indicates a major shift from past practices. Even the title of thecourse now places Leadership first – School Leadership and Management Course.
The program has been launched and applications are being solicited for training to begin in
the fall of 2009. It is expected that 1,200 school principals will be trained in the coming
months and that the program will eventually be extended so that in the future, all schoolleaders will be required to have training. Officers from DEBS offices will be included in the
training. This effort may need additional financial support to get underway, but once
established, can provide a tremendous vehicle for increasing student achievement.
Evidence-based Decision Making: One member of the evaluation team was privileged to be
involved in the final day of a five day EQUIP2/MOE Evidence-based Decision Making
workshop. Four districts from two provinces were chosen to attend, based on identified
characteristics of the districts – one struggling to achieve targets, the other performing well.
Another week-long session will involve another set of four districts. From six to eight District
Officers attended from each district and were given the EMIS data from their districts. They
worked as a team to identify indicators, use software to design district profiles, do secondary
analyses, and understand how the information can inform policy formation andimplementation. They were given follow up assignments to complete upon return to theirdistricts.
Each team presented some findings, possible reasons for concerns, and ideas foraddressing the findings. Those presentations were informed, energized, and well done. They
exchanged ideas and strategies with each other. When some participants were asked
privately how they felt about the workshop and information they had received, they were
thoughtful, and then responded with “empowered.” They further indicated they felt someownership of the data and a sense of direction. They dialogued with each other in a mannerin which they may not have communicated in district offices while fulfilling their separate
duties. Moving responsibility and capacity to district level and closer to schools positions
them to also be the primary channels for in-service training and new methodologies, like
continuous assessment.

23

Continuous Assessment:
The continuous assessment (CA) component of the EQUIP2 program, sub-contracted toAIR, has two objectives: (1) provide support for classroom instruction; and (2) make betterhigh stakes testing decisions, particularly at the national level. The first objective hastremendous potential for improving the quality of instruction in Zambian schools. The statedpurpose for CA is commendable: "to improve the teacher’s capacity to identify what learners
know and can do, to improve instructional practices, provide feedback to stakeholders, andreduce stress associated with examinations." CA is further defined as an on-going,
diagnostic, classroom-based process that uses a variety of assessment tools to measurelearner performance. CA is a formative evaluation tool utilized during the teaching andlearning process with the aim of influencing and informing the overall instructional process.
The second potential is fraught with pitfalls – chaos, high costs, abuse, and loss of potential
benefits from the first objective.
The Examinations Council of Zambia (ECZ) was given the responsibility for the EQUIP2Continuous Assessment initiative, and piloted it on behalf of the MOE. The Council is not apolicy or advocacy council, but operates as a quasi-independent agency that implements thedecisions of an advisory steering committee composed of key people from several agencies.
For the Continuous Assessment project, direction was also provided by an implementationcommittee that included technical support from AIR, piloted under EQUIP2 on behalf of the
Ministry. The ECZ oversees the formal assessments in Zambia public education - thenational exams (at the end of grades 7, 9 and 12), as well as for teacher training andvocational education. It also directs the National Assessment which was administered to arandom sample of grade 5 students in 1999, 2001, 2003, and 2006 to measure literacy andnumeracy skills.
The IOB Impact Evaluation of 2008 and the Review of the Ministry of Education Sector Plan
in 2007 both address the need for improved assessment practices, and mention CA.
“Educating Our Future”, the document that sets out a national policy for Zambian education,
addressed the issue of continuous assessment in 1996:“The Ministry will introduce procedures that will enable teachers tostandardize their assessment methods, thereby ensuring both transparency
and the availability of properly validated information on pupil
achievement…school-based assessment would perform the evaluative anddiagnostic functions currently associated with the Grade 7 composite
examination. Each school would have a comprehensive system for theevaluation of individual pupils....and grade 7 examinations would play a
progressively reduced role in the education system and eventually would be
abolished. (p. 12)”
In meetings with the members of the ECZ, it was clear that they conscientiously addressed
this responsibility, approaching the work cautiously and professionally. It was also clear thatthey recognize the value of CA as an instructional strategy that will provide valid
assessments to improve student learning, but without the specter of high stakes selection.
They even developed a customized teachers’ manual and trained 180 CSTs in CA.
The pilot began with grade 5 in 2006, grade 6 was added in 2007, and grade 7 in 2008. The
teachers were trained to give assessments throughout the year and to keep records. The
24

current structure directs teachers to assess their students throughout the year in six subject
areas using a variety of assessment tasks – 33 at the end of the first and second terms, and
34 in the third month. Common tests will be used throughout a school and administered to allclasses of the same grade at the same time. Committees will be formed at either the schoolor zonal level, coordinated by the district to prepare the end of term tests. Validity andreliability will be maintained by formulated questions from the prescribed syllabus, having auniform marking scheme, and moderating the marks of all pupils. Standard record cards will
be used, and retained at the school level, but end of year averages for grades 5, 6, and 7 will
be sent to ECZ, DEBS and PEO. The score of the 7th grade national assessment will countas 50 percent and the average of the reported scores for grades 5, 6, and 7 will count as 50
percent for Grade 7 final certification and selection.
A study by the ECZ, “Continuous Assessment Phase 1 Assessment Report 2009,” indicatedno statistically significant differences between CA pilot schools and control schools onscores in any of the five subject areas in the grade 7 national examination. The report states:“It is difficult to explain why no differences were observed between the twogroups on the national examinations. More information needs to be collected
about the test quality indicators (e.g., discrimination, reliability, etc.) of theexaminations. However, it is widely accepted that the malpractices (e.g.,
student cheating on the test, teachers provide answers to the students, etc.)
on the final examination is one of the serious threats to fair testing in Zambia.
This could have led to simular [sic] scores between pilot and control schools.”
Clearly that statement should raise red flags about the potential for more serious abuseswith a plan that requires scores to be collected and records for all students in grades 5, 6,
and 7 maintained by schools, averaged and sent up to three different governmental levels,
based on tests administered at thousands of schools, designed at district/zonal levels over a
period of years, moderated at some level with some unknown process, and then combinedwith the grade 7 national exam. It should also anticipate that such a process will require amajor data structure, another bureaucratic layer and staffing to manage and maintain thesystem, and continuous training of all staff who will be designing high stakes examinations inevery school and district.
An earlier study in 2007 found a strong correlation between the results of the continuous
assessment and the final assessment scores, which is not an unexpected result. Students
who did well on the tests throughout the year did well on the final exam. That illustrates thesignificance of CA as defined in the first objective of this initiative: its value as an
instructional tool. When the teacher is regularly assessing what she is teaching and what
students are expected to learn, the students will know how they are doing and they are much
more likely to be able to do well on the final exam.
In visits with teachers involved in the CA pilot project, there was clear evidence that theirwork with CA has transformed their teaching. They see it as something they do during theteaching process to determine if their students are learning. They described it as diagnostic– both of the pupil’s performance and their own teaching. They have learned to use a variety
of methods to measure learner performance and unique learning styles. These teachers saythey work harder because they have to keep on track and ahead of their students, and25

students attendance is much better because they know they will held accountable forlearning.
Even more interesting was the discovery in visits to the Eastern Province that two basic
schools, not part of the pilot project, understood CA as a valuable instructional tool practiced
by all teachers in those schools. They were able to define it, give examples, and elaborate
on it. When asked how they adopted the practice, they indicated it was part of the Primary
Reading Program and it had spread throughout their schools because it worked. Throughteacher meetings, discussions, and head teacher commitments, many schools in that area
also practice CA as part of what teachers do.
All the interviewed teachers who use continuous assessment as a regular part of theirteaching can describe the changes in themselves, their students, and in the learning that istaking place. Integrating an understanding of CA as an instructional strategy into both preservice and post-service training for teachers and head teachers could transform learning.
But that momentum and potential are likely to be abandoned or buried in controversy andcosts if it continues to be seen as a high stakes test replacement.
The respondents to the questionnaires gave their highest ratings to the quality of EQUIP2’s
work with EMIS and IT equipment and training, and also ranked those two categories as themost important of eight strategic areas. The work with continuous assessment was in the
middle in both the quality and importance ratings. The most consistent written comment wasthat work in all the strategic areas should be retained. Some concerns were evident,
however. The lowest rating (3.0 out of 5.0) was given to the extent to which MOE could
sustain the valued EQUIP2 programs. The open-ended questions indicated common
concerns about the EQUIP2 support being concentrated at the MOE headquarters level, andabout the importance of the EQUIP2 TA becoming more focused on capacity building of
MOE staff at all levels, rather than on doing the work themselves. See Annex D for a table of
the results of the questionnaire.

6. Sector Analysis
In addition to assessing the overall work of each of the three USAID-funded education
programs, the team also assessed work in six sectors (areas of investment) that, in many
instances, involved all three programs (see Figure 1 in Evaluation Methodology). Thisapproach allowed the Evaluation Team to assess the synergies among USAID-fundedinterventions and understand the overall impact of USAID/Zambia Education Program on
Zambia’s education sector. To do this, the Evaluation Team answered key questions about
interventions in the following targeted sub-sectors.

Institutional Development
How have the USAID supported programs promoted institutional development in the MOE?
The MOE has responsibilities for some degree of oversight or relationship with nearly 9,000
schools (including CSs), colleges of education, universities, and other boards or councils
that have duties in the chain of education delivery. Its functions are delivered and distributedin layers of bureaucratic structures at provincial, district, and zone levels.
26

Some components of institutional management and change must be top down, but somecan bubble up through the system. When chaos exists at the MOE headquarters level inadministrative and management functions, like the process for personnel records, itcascades down through the system. Improvement in the process will cascade in the same
fashion. Other improvements in institutional functions – like training for district officers toprovide support and assistance (rather than checklists and monitoring) for schools, teachers,
and instruction, can successfully be strategized at headquarters but built from the bottom up.
EQUIP2 has contributed to institutional development and the image of MOE by assistingfrom both directions.
When EQUIP2 began working to improve the EMIS school census reporting, only about 70%
of schools were included and the reports took months to prepare. By 2008, the reporting had
captured about 99% of the schools, including CSs which had previously been of unknown
quantity. Several other improvements to the EMIS school census reporting provided MOEwith a timely and credible administrative tool for budgeting, planning, analyzing, andassessing the scope of their work. The data were particularly useful in the development ofinitiatives related to community schools, recognition of school construction needs, andallocation of teachers to schools. Other partners have contributed to this work and some of
the cost of EMIS is now a line item in MOE’s budget. Approximately one-third of MOE’sbudget for all operating levels – headquarters, provinces, districts, schools, and colleges
comes from the pooled funds of the European Union. The data credibility has led to internaland external recognition of a stronger MOE than existed a few years ago. The extension of
the MOE reporting network out to the provinces has improved communications and reporting
timelines. The development of consistent and written policies and procedures, such as
Community School Guidelines and HIV/AIDS Workplace Policies, has improved institutional
structure. Much of the work of EQUIP2 has been within the Planning Directorate, which is
now seen as perhaps the strongest directorate in the ministry, but several of their efforts
have reached across directorates.
A major initiative to strengthen the institutional capacity of the MOE is underway that will
cascade across and down the system. EQUIP2 is assisting the Directorate of Human
Resources and Administration with the development of efficient management andadministrative systems for human resources. There is need to develop a comprehensive
system for MOE personnel at all levels – teachers employed in the schools and civil servants
throughout the system. Many functions need to be included or interface with that system:
teacher recruitment and placement, accreditation records for providers of courses andcertificates, teachers currently employed, teacher upgrades, housing requirements, students
completing teaching programs, tracking vacancies, and available teacher candidates, as well
as several other components. Current systems are mostly manual and rely on paper records
stored and stacked through various offices at all levels. Institutional chaos results in delays,
lost files, illegal requests, and incredible teacher frustration and disappointment. The
planned system will be designed to clarify practices, streamline the processes, and work withexisting functional systems in other ministries that involve financial reporting and payroll
management.
The Education Bill to give legal status to CSs has been delayed in Parliament and contains amajor administrative restructure of the governance of schools. That bill includes a proposal
to move responsibilities for early childhood and basic schools from MOE and place them27

under local government, and move high schools under provincial government. Those issues
are now being debated, and if that shift takes place, many other educational and institutional
issues will surely arise.
The teacher and community/school partnership training that was delivered by QUESTT and
CHANGES2 has built community “institutional” structures. Much of that training and teacher
training was provided in collaboration with district personnel, strengthening institutional
capacity at the district level. EQUIP2 has begun working with teams from district offices touse data from the annual school census to identify and address school needs. The colleges
have been aided by the donation of many CHANGES2 publications that update their
woefully inadequate library offerings. EQUIP2 and other donors are providing someassistance for colleges to set up computers and build internal networks. USAID-funded
activities working through MOE structures – provinces, districts, and colleges – build
institutional capacity and credibility for those entities to work with teachers and schools toimprove educational quality, although much more work is necessary in this area.
Unfortunately, many of the current MOE assigned tasks for the provinces and districts are to
support the MOE’s headquarter administrative needs, rather than the needs of the schools
and teachers.
Has Technical Assistance (TA) enhanced the MOE institutional structures particularly inpolicy and research areas?
USAID-funded activities appear to have built some capacity for and attention to the need forconsistent policies and procedures within the MOE. The TA provided with the developmentof Community School Guidelines and HIV/AIDS Workplace Policies created procedures
within MOE headquarters that continue to filter down to the provincial, district, and schoollevels, and even to other ministries. Recognition of the value of consistent practices led to
the request for TA to examine the human resource record-keeping procedures andprocesses. MOE institutional structures will be greatly enhanced if there is successful
implementation of the work currently underway to develop an HRIS system that responds tothe identified needs.
The research area appears to be under-developed and presents challenges. If the intent is
to institute academic research for the sake of research, the value will be questionable. The
current focus on evidence-based decision making using available data has the potential tocreate a research environment that grows from the bottom up, and informs decision makers.
As current data are examined by those who gather and use the data, issues will emerge that
require greater attention, additional information, and researched answers rather than
speculation.
How have USAID programs enhanced the roles/functions of key directorates?
Both DODE and the Planning Directorate have been elevated in status and position becauseof USAID-funded programs. DODE has benefited by the expansion of its role throughout theprovinces because of popularity of IRI and Learning at Taonga Market, as well as Our
Family and Fastele! Fastele! The average questionnaire response rating the degree to which
QUESTT helped strengthen DODE was 4.2 (with 5.0 the highest), an extremely positive
response. But although DODE is seen as strengthened externally, its role within the MOE
appears isolated and unsupported.
28

The Planning Directorate has become perhaps the most influential directorate in the MOE.
Information is power, and with the assistance of EQUIP2 positioned within its midst, they
have timely information and policies and guidelines to support their efforts. Budgeting
requests are based on hard data. That Directorate leads the others in development anddissemination of IT equipment and systems, M&E, and policy development. The confidence,expertise, and success of activities in the Planning Directorate were recognized andexpanded to create regular forums for discussion among the directorates, as well as
between cooperating partners. The work that is underway with the HRIS will greatly enhance
the role and functions of the Directorate of Human Resources and Administration. If the
School Leadership and Management Course is successfully implemented, the role of TESS
may also be enhanced.
To what extent are the headquarters, provincial, district and zonal MOE structures andstaffing able to sustain the implementation of USAID supported programs in community and
GRZ schools?
That question was asked of personnel in district, provincial and MOE offices concerning thework of EQUIP2: To what extent will the MOE at all levels be able to sustain and continue
the more valued programs, if any, in which EQUIP2 has provided assistance. It received anaverage rating of 3.0 on a scale of 1 to 5, the lowest response of the 11 listed activities. The
most valued services appeared to be the work with EMIS and IT equipment and training (4.1
and 3.8 respectively). In open ended responses the only explanation for the low rating forsustainability seemed to be the need and desire for more transmission of skills and moretraining for MOE, provincial and district personnel. Much of the work of the EQUIP2
personnel requires high level skills which are not easily transmitted in short training courses.
The work on teacher training and support, curriculum, and materials by CHANGES2 and
QUEST have been more concentrated with the districts and some with the provinces andtheir contributions are not as evident to the MOE headquarters. The staffing at the districtlevels may be able to sustain the work, but even now their budgets do not provide resources
to do the administrative work, to say nothing of adding the rich dimension of sustainedteacher training and follow up which requires transport costs, materials development andprinting, and workshop expenses.

ICT and Information Management
Have radio programs been effective in mobilizing schools and communities to provide
enhanced support for OVC?
The USAID funded radio programs have effectively reached resource poor communities
where the most vulnerable reside: orphans, girls, poor, people who are ill, elderly. Our
Family, a series of professionally designed programs radio programs, is broadcast threetimes weekly to approximately 900 listening groups in numerous villages. The groups listen
to a drama in soap opera form, an informational broadcast reinforces the drama topic forcommunity and children, and the third broadcast provides feedback from the discussion
groups. Printed guides provide tips for expending the information, and sometimes facilitators
assist discussions or dramas are presented by local groups. Listeners provided personal29

testimony regarding the impact of the broadcasts and indicated the programs provide
reliable information and advice to counteract “bad” information and superstition. The
programs are specifically targeted to aid the most vulnerable Zambians: the OVCs and theircaregivers.
The program modules include: Fostering Self-esteem, Substance Abuse, Early Marriages,
Starting a Business, among many others. A local clinic director who serves 72 villages was astrong advocate for the programs, indicating that she did not have the ability to provide that
information to all those villages, and communities were now able to assist with care andprevention. Another listener said she now realizes she has worth and decides what to dowith her own body. Others provided testimony about how the programs have changed
people and villages: making sure property is not taken from orphans, encouraging chiefs tounderstand why young girls should not be forced into early marriages, understanding thevalue of good nutrition, caring for the elderly, raising gardens to help those who need food,
not going to witch doctors. In Southern Province, one lady listener said, "We learned how towrite wills and retain our property, and also learned to take a copy to the court." Anotherman said, "These radio programs helped him and his family know where to go when they getsick. There was no other place to get this information."
The IRI instruction is a teaching methodology that provides educational access for poor andremote populations, promotes gender equity, and serves mainly vulnerable children.
QUESTT reports indicated that IRI programs were being broadcast to 1,880 IRI centers andcommunity schools with 92,569 IRI learners, but the total enrollment in those schools is361,709. The proportion of orphans in grades 1 through 7 in those schools is nearly 30
percent compared to 18 percent in GRZ basic schools, and female students in CSs accountfor 49.6 percent of the enrollment. Evaluation team members heard reports from GRZ
schools, CSs and orphanages that IRI children are more disciplined and are better listeners
than pupils who have been in GRZ schools without IRI.
The CS access to the LTM programs is not universal. In many schools, the only teacher
materials available are IRI guides for grades 1 and 2, so many grade levels are not served.
Some have no materials at all. For those who can only receive community radio stationtransmissions, only grades 1 or grades 1 and 2 are broadcast. ZNBC no longer broadcasts
LTM on Thursday or Grade 1 on Tuesday, and some signals are faint. Most of the IRI served
schools have no other educational materials and untrained mentors. IRI provides
professional educational guidance for the untrained, a common spoken English for thesecond language learners, and a recognition and support of the community efforts to
educate their children. Over the years several assessments of student learning have shownthat IRI learners achieve as well as and sometimes better than students in GRZ schools,
which is a remarkable achievement considering IRI is often the only educational vehicle andthe volunteer mentors are not qualified teachers.
Fastele! Fastele! radio programs are intended to provide continuous professional
development of teachers and volunteer teachers on a number of teaching topics, including
the teaching of literacy. It was difficult to determine how far this program reaches and its
usefulness was rated as the second lowest overall of the listed components on the QUESTT
questionnaires. Written comments on the questionnaires and later discussions indicateddifficulties with the time slot and the reception

30

What has been the impact of the education management information system on education
development?
The educational impact of the information management system on education development isindirect and more difficult to gauge, but the impact on the recognition of CSs is evident.
Before 2005, few CSs were included in the annual school census. Now, because of focuseddata collection efforts, the response rate for the annual school census grew from 70% to
99% in 2008, with most of the previously non-reporting entities having been community
schools. The MOE received more timely and complete information about all schools, butparticularly about the magnitude of community schools – 34% of schools and 16% ofenrollment in 2007, and 37% of schools and 17 percent of enrollment in 2008 (Educational
Statistical Bulletin, 2007 and 2008). Because the MOE had reliable information, they could
step into the void created by the demise of the Zambia Community Schools Secretariat, andwith the assistance of EQUIP2, “Guidelines for Community Schools” was developed.
Although CSs are still dependent on the resources of the local communities and external
supporting organizations, there is now a bill before Parliament to legalize their status but it ispart of a larger, more comprehension education bill that is being debated.
The existence of timely and trusted education data also provided the MOE with information
to support requests for funding for targeted purposes, and strengthened its overall credibility.
They now have a marker to indicate progress toward Education for All (EFA). Restricting thevision of education management information systems to the EMIS of the annual school
census reports is limiting. That is a snapshot in time and a small component within a modern
education information system. Work is underway building linkages between the school
census reports and other data, such as pupil exam results, the teacher information via theenvisioned MOE HRIS, and appropriate linkages to the payroll management system and thefinancial system that will provide for longitudinal, integrated information.
There are cost effective ways to assist schools to know more about their data, provide moreaccurate reporting, and help build capacity at the local level. Schools would know much
more about their own schools, without a school data sheet or whatever school profiles might
be developed, if the annual school census form would allow them to see their own data.
Unfortunately, the forms schools complete, and a copy retained, are actually data input
forms which provide no totals – not by grade, age, sex, boarders, refugees, orphans,boarding, bursary, nationality, dropouts – the list goes on. Even after a school has completed
the form, there is no local information – just lots of boxes with numbers in them. They are
dependent upon another layer of compiling records or higher levels to send something backbefore a picture emerges. The schools know the information on the forms is important, butthat the forms are for headquarters’ requirements and purposes.

Teacher Training
What has been the impact of teacher education programs on pedagogy in GRZ andcommunity schools?
The most dramatic, measurable effect has been the use of IRI and Learning at Taonga
Market to provide a means and a methodology for reaching thousands of students in CSs.
Many mentors are able to provide quality classroom instruction to children as a result of the
training and of the guided nature of the programs. CHANGES2 worked through MOE
31structures at the provincial, district, and zonal levels to train thousands more pre-service, inservice, and trainers of teachers in both GRZ and CSs. Workshops for head teachers wereconducted by both CHANGES2 and EQUIP2. CA training has been provided to teachers inschools in the pilot. Positive comments were heard from all who received any of the USAIDfunded training. The impact is deep and wide. Teachers are eager for any opportunity for
additional training. Institutional capacity to provide quality training was provided to provincial,
district and zonal personnel.
The ZATEC distance teacher education program was instituted to provide mentors with aprogram to become qualified teachers. Although many consider it to have been
unsuccessful, the lessons learned could assist with future distance education programs. The
mentors in CSs were new to this type of process, as were the teachers. Little on-site support
or visits took place due to lack of time and transportation. Evidently a high proportion of
students failed the exam the first time, but several took the failed portions later and passed.
In one instance 16 of 56 passed the exam the first time, and at a later date 16 passed. Inthat setting 57% passed the exam.
However, research and experience show that real changes in pedagogy do not occur without
sustained support to implement and extend the training. Otherwise the workshops andcourses become events and fade away. Capacity and funding are needed for the institutional
structures to support the positive impact of the USAID-funded activities – and to extend thetraining to provinces that have been on the periphery of the training. Although teachers
colleges have been reluctant to integrate some of the teacher training into their programs,
they value and need current and useful references and materials. To impact the future, realchanges in educational practices must happen at the pre-service level.
What strategies work best in improving teaching skills of community school teachers?
Community school teachers welcome any assistance that is provided to them. However,most CSTs are volunteers and unable to afford to leave their communities and families to
upgrade their skills to become qualified teachers. Observing the mentors presenting LTM
demonstrated the effectiveness of the training to use IRI, the value of the LTM teacher
guides, and the ability of the structured lessons to provide the CSTs with additional teaching
techniques. They benefit from delivery of training close to their communities with regularsupport and follow up. Although other training strategies may have improved the CSTteaching skills, the evaluation did not observe activities that would show those skills.
Particularly praised was the 20-day teacher-training course for CSTs developed by
CHANGES2 and Zambian colleagues. The sequencing of the course and the materials
account for much of its success. The trainees attended 10 intensive days and returned totheir communities to put into practice what they had learned. They returned later for 10 day
follow-up to discuss and expand upon their experiences. The Handbook for Training ofCommunity School Teachers was the manual provided for these teachers.
CSTs would be well served by efforts to upgrade their status to become diploma level
teachers with distance learning courses. The CSTs are mostly place-committed and likely toremain in their villages to teach if they become qualified teachers, unlike those who might beassigned to their villages but have no commitment or desire to serve those communities.

32

How have teaching tools (IRI and continuous assessment) improved teaching practice andlearning in community and government schools?
Both IRI and continuous assessment illustrate best practices of instructional methodologies.
Given that most CSs are served by volunteer mentors, the need for guided teachingpractices will be a primary consideration until such time as qualified and trained teachers areavailable to serve those schools. The IRI lessons, Learning at Taonga Market, provide that
guidance and an alternative delivery method that assists distant and materials-poor schools.
The lessons are designed to incorporate the curriculum learning objectives, age appropriate
activities, a variety of teaching methodologies for different learning styles, clear purposes forthe lesson, and structure and assistance for teachers. The pupils build strong listening skills
and for those who speak a Zambian language, a daily common and clear presentation in
English. At the same time pupils are learning, teachers can also be learning how to best
engage students. Many of the volunteers are better prepared to become qualified teachers
because of the training and practice they have with LTM. The results of a variety of studies
comparing IRI learners with non-IRI learners confirm that IRI learners achieve nearly as well,as well, and better. But the level of achievement isn’t good enough. The bar needs to beraised for all Zambian children.
Continuous assessment could help raise that bar if introduced across the system as a
generally practiced instructional strategy rather than a high stakes assessment tool. The
ECZ has already developed a customized teachers’ manual for CSTs. Teachers who are
involved in the pilot project described how CA had transformed their teaching. With noprompting they could define CA, identify the value, describe the difference it made withstudents, and highlight changes in their teaching. Because there is regular assessment oflearning, students are more attentive and involved, and teachers are more prepared.
At a CS utilizing CA visited in Southern Province, where 106 of the 184 pupils are orphans,
one of the teachers remarked, "Continuous assessment has created a healthy competition
among students. Children now ask when the next assessment will be. It was once a month;
now it is more often." The teacher in charge, a veteran educator, then said, "Parents andguardians come periodically for meetings on their children's performance. Also, the school
has meetings with individual guardians." The teachers in this school meet with the teachersof approximately six nearby schools two or three times at year to discuss teaching andlearning, professional development and related topics, including continuous assessment.
By sheer serendipity, two basic schools not involved in the CA were visited and it was
discovered they were using CA across all grades. Without hesitation they also described thevery same benefits for teaching and learning that were heard from teachers in the pilot
project – but without the implications of direct high stakes results. The relationship to highstakes testing was that by the end of grade 7, pupils were well prepared to take the grade 7
exams. CA has the potential to improve both teaching and learning if it can be delivered
across the system, through both pre-service and in-service programs, without the incredibly
complex and expensive high stakes testing design that is emerging. Concerns have existed
for years about the unfairness of the current grade 7 exam. That will pale by comparison to
the complexity and costs of implementing CA across the system for high stakes purposes.

33

Teaching and Learning Materials
How have education materials impacted the quality of education:
Toolkits, Basic Skills Training Manual, Grade 1-7 Syllabus
The materials developed by CHANGES2 are very well designed and praised by those whohave been trained in the use of the materials. In several schools visited, the teachers’
toolkits were used and much appreciated, particularly since the teachers had no such basic
supplies from the MOE. However, the team saw very few materials in any of the CSs.
Because many schools are materials poor, teachers appreciate any materials that provide
some support for them and for their students, but scarcity often leads to keeping materials in
secret locations so no one will take them. Most CSs had no secure locations to keep thematerials – sometimes no roof. The toolkits, Basic Skills Training Manuals from theCHANGES2 20-day basic teaching skills course, and teaching materials were observed inonly a few locations, and then mostly in basic schools. Many teachers indicated they had nosyllabus and no other available materials.
iPods, mentor guides, radios, wall charts;
The mentor guides and wall charts developed for LTM are necessary to appropriately teachthe lessons, but many schools have guides for only a few grades and some have no guides.
The demand outstripped the supply and no additional materials were printed or delivered.
Although the wall charts were noticed in some schools, they were not available in mostclassrooms. The radios or iPods are absolute necessities for delivery of the IRI programs,
but when the radios break down they don’t seem to be sent for repairs. The iPods allow
teachers to stop the lessons when the pace is too fast or when they want to emphasize a
portion of the lesson so add flexibility to the lesson delivery. However, iPods are expensive,
can malfunction and need maintenance. MP3 players and other newer technology are lessexpensive and hold more promise for common usage. For optimum delivery of LTM, allthese items should be available in a school as a complete package.
Modules, activity books, sample forms and tests?
The continuous assessment materials provide a teacher’s guide and tasks for six subjects inthree languages. Teachers in the pilot schools indicated that the model assessments do notmatch the learning outcomes in the guides. A review of the materials seems to reinforce that.
Although the learning outcomes do seem to match the Zambia Basic Education Syllabi, in
most instances the specific learning outcomes in the guides repeat exactly the same words
as the performance indicators and exactly the same words as the suggested assessment
tasks.

34

School Health and Nutrition
What has been the impact of the school health and nutrition programs on education delivery
systems and learning?
Anecdotal information and common sense both indicate that healthier children are more
likely to attend school and more likely to be able to learn. Materials and teacher training forthe health and nutrition programs are well designed, many teachers have been trained touse them, and they are broadly disseminated. Teachers and their families have received
information and assistance and are better able to assist students. But the link to learning iselusive. The connection might be made if attendance and exam information were available
prior to implementation of the programs and those data linked to program data indicating
which schools received de-worming or Vitamin A supplements or other focused SHN
activities. Many schools indicated that school feeding programs markedly improved school
attendance.
What has been the benefit of the HIVAIDS prevention activities to education quality/system
improvement?
Again the link is elusive, but the prevention activities contribute to removing some of thestigma and providing a more accepting atmosphere for those who are affected directly or
indirectly by HIV/AIDS.

Grants and Scholarships
What benefits have accrued to schools and communities as a result of implementing the
sub-grants? How has this support improved the governance of schools?
The USAID-supported sub-grants to schools and communities for infrastructure
improvement and income generating activities have had dramatic impact on thosecommunities and may be among the most cost effective USAID-supported components ofthe education programs. The value of the grants was highly rated by respondents to thequestionnaires – third highest at 4.2 (of 5.0) for CHANGES2 and highest at 4.4 for QUESTT.
The benefits appear to be four-fold. First, training was done with communities to develop
skills to apply for, plan and utilize the grants. Nearly all grants were administeredcompetently, with community members witnessing to the value and application of those
grant management skills both personally and for the community. Second, combining a small
amount of grant funding (QUESTT up to $2,000 and CHANGES up to $5,000) with local
initiative and supplies, 238 QUESTT grants and 398 CHANGES2 grants resulted in thecompletion of partially constructed buildings by adding roofs, windows, floors, doors; orfurniture, blackboards, tables; or sewing machines and tools to provide uniforms or income
for schools and volunteers; or latrines, kitchens, teachers’ residences, or refurbishing.

35

Table 4. Small Grants Under QUESTT and CHANGES2
Description
Small infrastructure grants related to education (up to $2,000) - QUESTT
Small infrastructure grants related to education (up to $5,000) - CHANGES2
Mid-level grants to COEs and resource centers (up to $10,000)
Special grants to NGOs in four provinces ($65,000 each province)
HIV/OVC support grants (up to $6,000 per school)
Psycho-Social Support grants (support from $500-$700)

Grants
238
398
12
4
120
130
Total Grants 902

The total number of small grants is considerably higher than 902 since there were dozens of
sub-grants from the $260,000 total given to the lead NGOs in four provinces. QUESTT
administered 238 of these small grants, while CHANGES2, which received much more funds
for small grants, administered the remainder. The total investment in schools, communities
and OVCs through these grants is $ 2,805,120.80. The average size of the QUESTTadministered grants is $ 1,355.14.
In community after community we saw the results of this partnership that brought ownershipof their school to a community, a better place for education, and a cost savings for the MOE.
The value of the school-community partnership was illustrated by a visit to one sullen
community school that was constructed by a donor agency without local involvement andcommitment to the process. The staff complained about the lack of community support, otherschools recruiting their students away, no replacement of damaged desks, no feeding
program, and that nobody helped them out. Of all the CSs visited by the team, that was the
only one indicating a complete lack of community ownership and commitment.
A third benefit accrued to the children and the educational process itself. Education now
occupies a more permanent position in the community and children had a safer and more
attractive place to be, prompting better attendance. Fourth was the establishment of working
relationships between community schools and district officials.
Have scholarships increased equity?
The scholarship program managed by CHANGES2 is a prime example of increased equity.
For only K600,000 (US$120) per school year, 20,654 OVCs have attended high school
because of these scholarships. It is unlikely that any of them would have found other means
to be able to continue their education. Per term the schools receive K150,000 for tuition
costs and the pupils receive a minimal amount of K50,000 (US$10) for personal expenses.
Other support is also provided – the comfort kits for girls, HIV/AIDS peer educator training,and psychosocial support. Of the 8,000 scholarships supported in 2009, 70 percent are girls.
Interviews with several scholarship recipients gave voice and faces to the program. The levelof appreciation was tremendous and the degree of need evident. But they expressed
concerns that were shared by their teachers – that they are not prepared to be independentwhen they complete high school, and they aren’t aware of opportunities to continue theireducation beyond that point. But they dream of possibilities, something that would havebeen impossible without a high school education. One of the many girl scholarship pupils to
speak to the team said, "Once you are orphaned, there are many challenges. I want very36

much to go to the university after this year, but it will be very difficult. I want to be a social
worker with an NGO working in rural areas."

7. Sustainability
A new Minister of Education was appointed just as the evaluation team was leaving Zambia.
When the administrative leadership of the MOE changes often, as it has in Zambia in recent
years, it is difficult to predict what activities and programs will be sustained and what will not.
Some of the more deeply embedded activities are more likely to be sustained even if thepolitical will is not present.
By its nature, in-service training of teachers, or any other profession, is continuous and will
always be necessary, like updating doctors, engineers, or scientists. But in-service is often
an event – not sustained, not revisited, not followed up, and often not remembered orutilized. It does not substitute for the sustainability of structured in-service systems or for preservice training where teachers are recruited and trained for the future. Lessons learned
from the ZATEC training for CSTs could contribute to the development of a more sustainable
method of training teachers who are committed to their villages and regions and will remain
there.
Nutrition and health education are critical to the citizens of nations faced with the poverty,living conditions, corruption, and disease that impact their regions. Those who are impacted
by the information obtained through these activities will have received lifetime benefits. But
the basics of education – reading and writing and numeracy – are the sustainableeducational benefits that are critical to rising above the poverty so that conditions can bealtered by the citizens of their own countries. The teachers need to have the training
necessary to teach the core educational components. They also need incentives to be
attracted to being teachers who impact the future of their countries.
Many excellent materials have been developed by these three projects and their continued
use will be sustainable for a long period of time, particularly if they are reproduced inquantities that allow them to be distributed more broadly than currently possible. Thosevalued products are identified in the Recommendations section.
Sustainability of USAID-funded efforts to support community schools depends on the GRZ
assuming responsibility for CSs. Until the GRZ does this, continued support for the volunteer
efforts is necessary to sustain the momentum of communities who are growing weary. The
USAID-funded activities – small infrastructure or income-generating grants, teacher training,
IRI and materials – provide hope that one day they will have government support.
Assistance with the development of governmental implementation structures for to support
community schools will promote sustainability.
In combination, five of the current USAID-funded activities have the potential to contribute toa sustainable educational structure that will improve teacher recruitment and quality, andultimately, quality learning:
1. MOE human resources information system (HRIS);
2. Education Leadership and Management Training course for heads of schools;
3. Workshops supporting districts in the use of data for decision making;37

4. Support for COEs with technology development; and5. Integration of CA into the education system via items 2, 3, and 4.
1. HRIS: The development of a functioning human resources record-keeping system within
the MOE will positively affect the employment status of all Ministry personnel, fromheadquarters all the way down to the schools. Supposedly more teachers graduate fromcolleges than enter the teaching profession. The current record keeping system is so chaotic
that everything is affected: deployment, housing allowances, finances, retirement benefits,salary increases, new teacher support, even who is still employed or alive. This might deter
new graduates from electing to participate in the education sector.
The sustained benefits to a structured system would include transparency (less opportunity
for fraud when records are available and finances can be traced to the source), credibility(accuracy, projections, documentation of needs), and more attractive working conditions(much of the chaos eliminated). Such systems always need to be updated, but even an outof-date system would be better than none at all.
2. Education Leadership and Management Course: The course, as envisioned, has thepotential to bring a major change in school quality. Repeatedly, research on effective
schools demonstrates that school administrators, trained for management and educational
leadership, are a key to improved school quality and student achievement. The newly
designed course introduces an educational component that has not been present in past
government efforts. Even the title was changed in just the past few weeks – Education
Leadership was placed before Management. It is an uphill climb, but there exists a desire toactually train educational leaders, not just mechanical managers, and the former Minister
indicated that future school principals and responsible district personnel will be required togain training.
3. Evidence-based Decision Making District Workshops: If the limited training that wasdone in June 2009 can be expanded to reach all districts and include educational support, it
could empower district personnel to understand and assist schools, rather than “monitor”
and use checklists to serve headquarters requirements. This could become a criticalsustainability piece by having built district and zonal support for the isolated schools, everchanging teachers, and provide a conduit for identified needs from both directions.
4. Support for colleges of education: Colleges of education desperately need updated
technology as well as current publications and instructional materials. Teacher in-service
training is not a substitute for the pre-service training where teachers are trained for the
future. It should serve as an update for new information or ideas. If inroads can be made in
pre-service, this will be sustainable. But if the new teachers are trained in methodologies
favored by the 1970s and 80s, as appears in the outline for the proposed new three-year
program, there will be a perpetual need for USAID-funded in-service – for the very basics of
education, not just health and nutrition.
Certainly post-secondary education is a difficult entity to influence, but the lure of technology
can be an incentive along with enriching the library offerings. Because the college teachers’
access to current materials – printed or electronic – is archaic, the teachers are severely
restricted. Simply providing them with access to information could have a sustained andpositive impact.
38

5. Continuous Assessment: The current CA effort is validating the benefits of CA as an
instructional methodology that can improve both student and teacher performance. Someschools have integrated it into their classrooms based on prior experience with it in the
primary reading initiative. As a high stakes measure, it will be non-sustainable and easily
abused. If it can be integrated as an instructional methodology into items 2, 3, and 4, it could
have a dramatic affect on education in Zambia.

8. Recommendations
Recommendations have been grouped according to key areas requiring support from the
USAID/Zambia Education Office and ranked in order of importance to highlight and prioritize
where future programming might be focused.
1. Teacher Training and Developmenta. Continue and expand the training of CSTs and teachers in charge. The training which
CHANGES2 and QUESTT gave these volunteer community citizens appears wellreceived, high quality, immediately used and something that should be constantly
offered, particularly given the totally understandable frequent turnover rate of thesepeople. Community support for education increases when local residents know theteachers are solidly prepared.
b. Continuous assessment as an instructional methodology has tremendous potential for
improving the quality of education in Zambia. To retain that positive aspect and extend
the potential, assist the MOE and ECZ to devolve training and management of CA to the
district level, extending the classroom methodology and “report cards” across all schools
and grades, maintaining district involvement in the training and management of thereport cards. The grade 7 report cards could become the credential for entrance into
grade 8, and the grade 7 national assessment could be phased out as envisioned in the1996 publication “Educating our Future”.
c.

A distance education program for CSTs leading to qualification has potential to motivate
teachers and enable the MOE to attain its goal of having all CSTs qualified by 2015.
USAID should support such a program.

d. Among the 34 U.S. Peace Corps Volunteers working in education in Zambia, explore
using some of the most apt and gifted for locally-based teacher training. They can bevaluable source personnel for interactive teaching and learning and critical thinking skills.
They can abet USAID's decentralized approach, as these PCVs live in small, largely
poor rural communities, work with clusters of community and basic schools, andcoordinate closely with zonal, district and DEBS personnel.
e. High-quality, relevant training is essential for all head teachers in GRZ basic andsecondary schools. This training should be a combination of, first, educational andpedagogical leadership and, second, school management and administration. For theplanned Education Leadership and Management Course for head teachers and MOE
heads of departments and sections, the best practices and experiences of CHANGES2
39

and QUESTT training sessions should be examined to incorporate into the training
program.
f.

USAID/Zambia is well advised to begin giving increased support to secondary education.
Practical, pertinent training for secondary school head teachers is a good, logical place
to start, especially with the pressures in the country to both increase the quality of
secondary education and better accommodate some of the burgeoning enrollment
coming out of the upper basic grades.

g. The training and upgrading of teachers, both in-service and pre-service, have rightly
been a focus of USAID assistance. To increase teacher training effectiveness, USAID
must take into account issues related to teacher supply such as certification
requirements, deployment constraints and numbers of teacher college graduates. Doingso will allow USAID to make deliberate programming decisions that will increase training
efficiency and quality. MOE and other entities have a wealth of information that could becompiled and examined in order to plan and project needs into the future as to how best
to coordinate the variety of activities just underway. There are many unansweredquestions, but the larger one is “How many teachers will be needed by 2015 to meet thestated goals and how will Zambia get there?”h. Progress in technology infrastructure for colleges could greatly affect the quality of preservice training for teachers. To sustain prior teaching training efforts and affect preservice programs, continue to support IT infrastructure for colleges of education to allow
access to Internet resources that will shore up their severely outdated and inadequate
library resources and entice them to be involved to cooperate with such activities.
Networking advice and some equipment have been provided to colleges by EQUIP2 andseveral partners, but most computers sit idle at the teacher education colleges becauseof the lack of electrical infrastructure, technical personnel, user knowledge, and thebudgets to get them up and running. Students and staff need exposure to new
methodologies and updated information for content areas.
2. Teaching and Learning Materials
a. IRI is a vital educational element USAID/Zambia is currently helping fund, andcontributes to educational quality, access and equity, human rights, and as a valuable
teaching aid for both GRZ and CS schools. It will remain the only interim educational
avenue open to tens of thousands of poor Zambian youth and adults until such time ascommunity schools receive government support and are supplied with certified teachers.
USAID, MOE, and other partners should continue to support IRI and provide access tothe programs, training, and materials to those schools rely on it.
b. Many publications and materials USAID has already sponsored and paid for – especiallyteachers' textbooks, guides and manuals and student texts and supplementary materials– should continue to be published, distributed and used widely. The entire country at the
primary and secondary school levels has a crying need for materials, the team observed.
The highly-valued SPRINT manual for in-service teaching and especially the valuable
SPRINT Teacher’s Guide developed via CHANGES2 should be continued in basic andcommunity schools and COEs, and a version geared for secondary school teachers

40

should be developed. “The Handbook for Training of Community School Teachers” isinvaluable for CSTs.
The OVC life skills “Our Family Teacher’s Guide” developed under QUESTT should becontinued as well as LTM Mentor’s Guide and Training Manual for grades 1 through 7,
including supplementary training manuals. The Science Manual and DVD for grades 5
through 7 should be replicated for future programs. Many SHN materials are too valuable
to have their publication, distribution and usage discontinued. The QUESTT-developed
guides and attendant IRI information and methodology should continue as part ofaspirant teachers' preparation in COEs.
c. Special education students often require specialized materials – Braille, recordings, large
print. The educational materials sponsored by USAID would be valuable for thesestudents if organizations that provide the specialized materials could transfer them into
the required medium.
d. Materials development, field-testing, distribution, related training and M&E should
continue to promote capacity building and "legacy corps" within the MOE, beginning at
the district levels. CHANGES2 and QUESTT fostered broad collaboration among allvertical levels of MOE personnel on these tasks, incorporating NGOs when useful.
These people and new "recruits" can maintain this spirit.
e. Augment the support for and expertise in community radio stations, and attempt to add
to the nine existing such stations that air USAID-supported programs such as IRI and the
OVC Life Skills programs featuring the Our Family series. These stations are asignificant source of community discussion and debate, inter-generational education,
advocacy and civil society strengthening.
f.

Continue and enhance further the OVC Life Skills radio program series for communities
titled Our Family. It is good, provocative and immediately useful and practical for theentire village of listeners with whom evaluation team members spoke.

g. Expand the focused work of CHANGES2 into the other provinces.
3. Grants and Scholarships for Community Schools
a. USAID/Zambia should considerably expand its support to CSs because they represent
the only means of education for hundreds of thousands of Zambian youth, to strengthen
their quality, and to help crystallize the community backing, enthusiasm, pride and spirit
CSs entail. GRZ and MOE advocacy, respect and financial help should be enlisted forCSs, balanced with community participation and ownership. A stronger civil society andhealthy CBOs evolve, moreover.
b. The total amount and number of small grants should be increased. Small grants tocommunities have been well-administered and used and are widely popular, based onwhat evaluation team members heard and saw. They frequently boost formal educationin poor areas, provide cost-effective school construction, are a community action andstrengthening medium, and are positive non-formal education activities (sewing andtailoring, carpentry, poultry-raising, etc.). The IGAs usually produce a modest profit which41

goes to the CS teacher in charge. The community development and esprit d' corps theyengender are admirable.
c. Scholarships to OVCs in secondary schools should be not only maintained but also
expanded. They are needed and widely admired for access and equity in education,
human rights, humanitarian ends, and for multiple other reasons. The returns on thesemodest investments are incalculable.
d. Useful, practical training for the guidance counselors in secondary schools who workwith OVCs would be another wise though modest contribution to vulnerable youth andsecondary schools.
e. Even if CSs are given legal status in the near future, the transition into the Zambian
education system will be lengthy and difficult. In addition to retaining and expanding thepast successful supportive activities with community schools, consider including
assistance with the development of a strategic plan to bring community schools into thegovernment system in a systemic way.
4. School Health and Nutrition
a. De-worming should go nationwide and administered in all community and GRZ basic
schools, especially those in poor, rural areas. Pharmaceutical firms reportedly haveoffered the tablets free of charge for Zambia-wide distribution, and now supposedly $1.5
million are needed to train teachers and for a modicum of logistical support.
b. Vitamin A supplements and bilharzia treatments should be administered to basic school
children in all districts where USAID-assisted comprehensive projects operate in the
coming three years. Explore having the MOH take over these treatments thereafter.
c. The Health Promoting Schools Initiative, instituted by CHANGES2, should be continued
in the appropriate new USAID-funded project. This includes training for teachers andhead teachers, and continued use of the School Health Card for every student.
d. The health and hygiene-related information and messages delivered through IRI should
be continued. Classroom observations of IRI lessons indicated they are important.
e. Most SHN activities should be expanded into new provinces and districts.
5. Institutional Developmenta. Teams that have effectively developed high-quality training courses, teachers' andstudent materials, community guides and the like should be kept intact, adding new
blood and quality participants when the opportunity arises. Different MOE directorates,
particularly TESS, and the COEs have the major responsibility to maintain these corps ofexpertise. USAID and other cooperating partners can assist to a marked degree.
b. District personnel are closer to the work of the schools and therefore more aware ofneeds of the schools and teachers, including deployment of teachers and otherpersonnel. MOE should be encouraged to continue the activities designed to devolve42

responsibility to district and zonal levels using evidence-based decision making tosupport schools and teachers.
c. To foster further MOE institutional development EQUIP2 should determine areas of
responsibilities to be turned over to MOE staff. For example, those areas might include
responsibility for the annual school census data collection and the resulting Educational
Statistical Bulletin. Additionally, EQUIP2 should provide training courses for theircounterparts at MOE headquarters and selected provincial and district counterparts.
These steps are likely to have positive substantive and symbolic consequences, andlessen the feeling among some MOE employees that EQUIP2 staff are doing their workfor them rather than training them to do the work.

6. ICT and Information managementa. Ownership of some of the information management functions could be vested in MOE
and their employees, including the school census and reporting of that information, with
the TA more focused on developing broader education information systems, integrated
and linked to other critical national level information components such as exam results,finances, and teacher records.
b. The development of the teacher HRIS system within the MOE is critical. That work will
require continued technical assistance to bring order out of chaos, and integrate thissystem with the functioning systems of the Ministry and the other affiliated agencies.
c. The work now underway to extend IT capacity to the provinces and colleges of educationneeds to be continued and expanded in order to improve communications, datacollection and management, and availability of up-to-date educational information.
d. The Evidence-based Decision Making Workshops could be accelerated and expanded
across the provinces to build district capacity to improve the quality of schools. Theenergy and empowerment that were evident in the workshop can be harnessed toinclude an emphasis on in-service training and instructional methodologies as well as
monitoring, analysis, and reporting.

43

This page left intentionally blankANNEX A
Key Documents Reviewed

44

Key Documents Reviewed
1. Educating our Future, National Policy on Education, Ministry of Education, Republicof Zambia, May 1996
2. Primary Education in Zambia, IOB Impact Evaluation no. 312, Ministry of Foreign
Affairs, The Hague, the Netherlands, April, 2008
3. USAID/Zambia Assistance to Strengthen Ministry of Education Support toCommunity Schools, DevTech Systems, Inc., May, 2008
4. National Implementation Framework 2008-2010, Implementing the Fifth National
Development Plan, Ministry of Education, Republic of Zambia, October 2007
5. Vision 2030, ‘A prosperous Middle-income Nation by 2030’, Republic of Zambia,
December 2006
6. Zambia Millenium Development Goals, Progress Report 2008, Ministry of Financeand National Planning, Republic of Zambia 2008
7. Making Basic Education Services More Effective in Zambia: Improving Delivery,
Tracking School Level Expenditures, Human Development, Africa Region, The World
Bank, Washington, D.C., June 25, 2008
8. Operational Guidelines for Community Schools, Ministry of Education, Republic ofZambia, November 2007
9. Education Statistical Bulletin, Ministry of Education, Republic of Zambia, Directorate
of Planning and Information (for the years 2004, 2005, 2006, 2007, 2008)
10. Infrastructure Operational Plan 2008, Ministry of Education, Republic of Zambia,
Updated April, 2008
11. Tuned in to Student Success: Assessing the Impact of Interactive Radio Instruction
for the Hardest-to-Reach, Education Development Center, Inc., Washington, D.C.,
February, 2009
12. Zambia Basic Education Syllabi, Grades 1-7, Curriculum Development Centre,
Ministry of Education, Republic of Zambia, 2003
13. Interactive Radio Instruction: Twenty-Three Years of Improving Educational Quality,
Andres Bosch, Education and Technology Technical Notes Series, Volume 2,
Number 1, 1997, The World Bank, Washington, D.C.
14. Interactive Radio Instruction: Impact, Sustainability, and Future Directions, Education
and Technology Notes Series, Volume 4, Number 1, 1999, edited by Alan Dock andJohn Helwig, The World Bank, Washington, D.C.

45

15. School-Community Partnership Evaluation Report, USAID/CHANGES2, Beyant
Kakbwe and William Sam Phiri, April, 2009
16. Quarterly Reports Nos. 13 and 14 (for July-September, 2008 and OctoberDecember, 2008, respectively), USAID/Zambia CHANGES2 Program, Lusaka,
Zambia
17. Quarterly Report No. 15 (for January-March, 2009), USAID/Zambia CHANGES2
Program, Lusaka, Zambia
18. Our Family Teacher's Guide, 2nd Edition, Year 2, Education Development Centre,
QUESTT Project, Lusaka, Zambia, 2009
19. “QUESTT Quarterly Report, January 1st to March 31st, 2009,” Education
Development Center, Inc., Lusaka, Zambia
20. “QUESTT Quarterly Report October 1st to December 31st, 2008,” Education
Development Center, Inc., Lusaka, Zambia
21. “Africa KidSAFE and Zambia Interactive Radio Instruction Program, Zambia: Case
Studies,” draft-version 1, Mobile Task Team on the Impact of HIV/AIDS on
Education, Wendy Heard, August, 2005
22. “IRI Centre Monitoring Report: An Overview and Analysis of Centre and Learner
Performance,” QUESTT M & E, Lusaka, Zambia, October, 2007
23. “IRI Monitoring Report for the 1st Half of 2005,” QUESTT, Educational Development
Center, Inc., Lusaka, Zambia, September, 2005
24. “IRI GRZ Schools Monitoring Report: An Overview and Analysis of Teacher andLearner Performance in GRZ Schools using the IRI Methodology,” QUESTT M & E,
December, 2008
25. “IRI School Monitoring Report: An Overview and Analysis of IRI Centre andCommunity School Status and Performance,” QUESTT M & E, December, 2008
26. “2009 Annual Work Plan & Budget,” Working Draft, EQUIP2 Zambia, February 8,
2009
27. “Quarterly Report, October-December, 2008,” EQUIP2 Zambia, Lusaka, Zambia,
March 2009
28. “Quarterly Report, July-September, 2008,” EQUIP2 Zambia, Lusaka, Zambia,
October 30, 2008
29. Performance of Community Schools in Zambia: A Research Report, Peggy N.
Chakufyali, Ministry of Education, 2008
30. USAID/Zambia Strategy Statement, USAID/Zambia, January, 2006
46

31. FY 2008 Performance Plan and Report: Education, USAID/Zambia, Lusaka, Zambia,
November 26, 2008
32. Strengthening MOE Capacity to Reach National and International Education Goals,
EQUIP2 Zambia, Lusaka, Zambia, 2009
33. Infrastructure Improvement Grants for Community Schools: Operational Guidelines,
MOE and CHANGES2, Lusaka, Zambia, March, 2007
34. Sample HIV Prevention Activities for Integration into Grades 1-9 Subjects, MOE andCHANGES2 Program, Lusaka, Zambia, 2008
35. Peer Educator Activity Manual, Volume 1, Ministry of Education and CHANGES2
Program, Lusaka, Zambia, 2008
36. Peer Educator Activity Manual, Volume 2, Ministry of Education and CHANGES2
Program, Lusaka, Zambia, 2008
37. Grade 5 Continuous Assessment Tasks in Six Learning Areas, Version 1, Ministry of
Education, Republic of Zambia, 2008
38. Grade 6 Continuous Assessment Tasks in Six Learning Areas, Version 2, Ministry of
Education, Republic of Zambia, 2008
39. Teacher’s Guide on Continuous Assessment at the Basic School Level, Version 2,
Ministry of Education, Republic of Zambia, 2008
40. “2009 Budget Preparation Guidelines, Ministry of Education,” Republic of Zambia,
October 2008
41. Learning Achievement at the Middle Basic Level, The Examinations Council ofZambia on behalf of the Ministry of Education, edited by Dr. Sichalwe M. Kasanda
and Chekani T. Sakala, 2006
42. Ministry of Education Strategic Framework for Community Schools, Ministry of
Education, Republic of Zambia, Draft Blue Print Policy Instrument, no date
43. Training Manual for IRI Mentors and Teachers, Ministry of Education, Republic ofZambia, no date
44. Review of the Ministry of Education Sector Plan, Independent Review 2006, Final
Report – May 2007, Copenhagen Development Consulting
45. “Learning at Taonga Market” in Government Schools, An evaluation of the 2005 Pilot
of Interactive Radio Instruction in Grade One, TED in collaboration with QUESTT,
Ministry of Education, Republic of Zambia, 2005

47

46. “Learning at Taonga Market” at Grade 2: An evaluation of Interactive Radio
Instruction in GRZ schools in 2008, QUESTT M&E, March 2009
47. "Learning at Taonga Market: Luapula Province," Directorate of Open and Distance
Education (DODE), MOE, Provincial Education Office, compiled by Wettan Mambwe,
March, 2009
48. Handbook for Training of Community School Teachers, CHANGES2 Program,
Ministry of Education, Zambia 2007
49. Training Manual for Community School Head Teachers, CHANGES2 Program,
Ministry of Education, Zambia 2007
50. School-Community Partnerships, Training Manual for Teachers and Communities,
Republic of Zambia, Ministry of Zambia, March 2007
51. “Financial Review of the Academy for Educational Development (AED) Agreement
No. 690-A-00-04-00095-00,” USAID, Financial Analysis Section Controller’s Office,December 2006
52. “QUESTT Work Plan,” October 2008 – September 2009
53. “A Literature Review of Community Schools in Africa,” Cashen, et al (2001) in Y.
Miller, Washington D.C., USAID, 2002
54. “The “How” Practices and Strategies of Perceived Best ad Progressive Community
Schools in Central and Lusaka Provinces of Zambia,” P. N. Chakufyali, Lusaka,
VVOB, 2007
55. “Situation Analysis of Community Schools in Central Province of Zambia,” Y. A.
Chondoka, 2006
56. “Meeting EFA: Success of Community Schools in Zambia,” J Destefano, et al, 2006
http://literacyhub.org/documents/e2-zambiacasestudy
57. “EFA 2000 Assessment: Country Reports – Zambia,” Ministry of Education, 1999
58. “EFA 2000 Assessment: Country Reports – Zambia,” Ministry of Education, M.J.
Kelly, H.J. Msango, and C. M. Subulwa, 1998
59. “EFA 2000 Assessment: Country Reports – Zambia,” Ministry of Education, Lusaka,(no date)
60. “Policy Framework of Zambia Community Schools Secretariat in Light of Free
Education Policy,’ Sikwibele, ZCSS, Lusaka, 2002

48

ANNEX B
Places Visited and Persons Met and/or Interviewed

49

Places Visited and Persons Met and/or Interviewed
Name

Designation

USAID Zambia
Melissa Williams
Sheila Lutjens
Jim N. Barnhart
Fredrick Henning
Patricia Sitimela
Felix Chanda
Cornelius Chipoma
Beatrice Mweene
Elizabeth Chisala
Joy Mulenga

Mission Director
Deputy Mission Director
General Development Officer
Education Advisor
Program Development Specialist
Financial Analyst
Project Management Specialist
Education Project Management Specialist
Administrative Assistant
Administrative AssistantMinistry of Education Headquarters
Lilian Kapulu
Permanent Secretary Ministry of Education
James M. Silwimba
Acting Chief Education Officer, Teacher Education and Specialized
Services
Victor Muzatwa
Director Distance Education
Foster Lubinda
Controller, Education Broadcasting Services
Frank M. Musunga
Human Resources Development Officer
Susan Mwanza
Human Resources Development Officer
CHANGES2 Staff
Edward Graybill
Chief of Party
Joan Woods
HIV AIDS AdvisorPeter C. Sampa
Provincial Manager, Teacher Education Coordinator
Josias Zulu
Subgrants, Community Outreach and Capacity Building Advisor
Sitwala Mungunda
M & E Specialist, Field Operations Manager
Joy du Plessis
Senior Project Specialist, Teacher Education Advisor
Audrey Mwansa
Scholarships and OVC Advisor
Meredith McCormac
Project Associate, Washington, D.C.
Evans Mumba
Provincial Manager, Southern Province
QUESTT Staff
Richard Trewby
Francis Sampa
David Lungu
Francesca Mubiana
Conrad Bwalya
Miriam Shakafuswa
Hitler Muleya
Audrey Chitalu
Peter Mufinda

Chief of Party
Deputy Chief of Party
Monitoring and Evaluation Coordinator
National Outreach Coordinator
Provincial Outreach Coordinator
Community Radio Coordinator
Grants Coordinator
OVC Life Skills Producer
Provincial Outreach Coordinator, Livingstone District
50

Francis Phiri
Raphael Mazonga
Norman Tembo
Cuthbert Banda
Gift Sitenge
Kent Noel
Mike Laflin,
Lisa Easterbrooks

OVC Life Skills Producer (MOSI-O-TUNYA), Livingstone District
Provincial Outreach Coordinator, Chipata District
OVC Life Skills Producer (MARIA), Petauke District
OVC Skills Coordinator ( PASME ), Petauke District
Data Analyst
Director, Eastern and Southern Africa, EDC
Sr. Vice President and Director, International Education
System Division, Washington, D.C.
Project Coordinator, Washington, D.C.

EQUIP2 Staff
Arnold Chengo
Sriyanjit Perera
Alick Siankumo
Benny Mwaanga
Yantombo Mumbula
Saul Banda
Simon Muyambo
Thomas Lent

Chief of Party, Senior Technical Advisor
Deputy Chief of Party, Senior EMIS Specialist
ICT Infrastructure and Systems Coordinator
Monitoring and Evaluation Specialist
Provincial Education Advisor, Mongu Western Province
Policy and Research Advisor
Human Resources Information System Project Advisor
Project Director, Washington, D.C.

Cooperating Partners

Vincent Snijers
Luis A. Benveiste

Program and Training Coordinator, and Deputy Director, U.S. Peace
Corps
First Secretary, Education, Embassy of the Kingdom of theNetherlands
Lead Ed.Specialist, Africa Region, Human Development, World Bank

Eastern Province
Pilila Jere
Weston Siwale
Wendy B. Nyangu
Stephen Zimba
Matthwes Chirwa
Edward Nkoloma
Vincent Chibwe
James Phiri
Peter Ngoma
James Mwanza
Ruth Zulu
Faith Chikonga Masika
Jonathan Miti
Veronica Mwale
Daniel Nkhoma
T. JereSmart Mwale

Provincial Education Officer, Eastern Province
PESO
SEO, Distance, PEO
SPO, PEO
DEBS, Petauke District
DESO, Petauke District
DRC, Petauke District
Principal, Chipata College of Education, Chipata District
Headteacher, Anoya Zulu Secondary School, Chipata District
D/Headteacher, Anoya Zulu Secondary School, Chipata District
Headteacher, Mpezeni Park Basic School, Chipata District
Headteacher, Trinity Basic School, Chipata District
P.T.A Chairperson, Kalungwizi CS, Chipata District
Headmentor, Chunga CS, Chipata District
Headmentor, Mandondo CS, Chipata District
Headmaster, Mwami Basic School, Chipata District
Headmentor, Kafunde CS, Sinda District

Nancy Yuill

51

Philemon Mwale
Grace Mumba
Paul Lupenga
Rodgers Chilongozi
Ida Daka
Benard Sakala
Kelson Banda
Lukas Mwale

Headteacher, Lutwazi Basic School, Petauke District
Deputy Headteacher, Lutwazi Basic School, Petauke District
Senior Teacher, Lutwazi Basic School, Petauke District
Headteacher, Nyamphande Basic School, Petauke District
D/Headteacher, Nyamphande Basic School, Petauke District
Board Chairperson ( PASME ), Petauke District
Secretary ( PASME ), Petauke District
Station Manager ( PASME ), Petauke District

Southern Province
Fetsus H. Mungo
P.E.O, Southern Province
Cecilia N. Sakala
A/PESO, PEO
Kombe Mulando
SESO - ODL, PEO
Muchele A. Sialwindi
SPO, PEO
Litia Nyahwa
SESO - Sp.Ed, PEO
Fr. Fredrick K. Kabwe
A. Principal, Charles Lwanga College of Education, Monze District
Emmanuel Mweetwa
A/HOS science, Monze District
Silambe B Muyambango
A/HOS section, Monze District
Siyameja T. Namoomba
M.E, Monze District
Alfred Shanzie
Maths Ed., Monze District
Lyn L. Mwiinga
L.L.E, Monze District
C. S. Sitali
Es ( Library ), Monze District
E. C. Ng`andu
Education Studies, Monze District
Malambo T. Hanguwa
DEBS, Livingstone District
Muvwanga Gabriel
DESO, Livingstone District
Sichela Alice Manyepa
ESO, Livingstone District
Gladys Siyawayawa
SHRO, Livingstone District
Grace Kabwe Banda
G & C, Livingstone District
Dennis Hameja
PO, Livingstone District
Silvia Mulenga
A/DRCC, Livingstone District
Lillian Muvwanga
PRCC, Southern Province
Sichela Alice Manyepa
ESO - GI, Livingstone District
Chairperson- PCSC, Mwandi CS, Livingstone District
Likando Mukutulu
Chairperson - Support Group, Livingstone District
Concillia Mundia
P.T.A Chairperson, Mwandi CS, Livingstone District
Alex Kapanga
Josephine Shamamputa
DEBS, Monze District
Shirly Mbozi
A/DESO, Monze District
Seddy Kachabe
ESO - ODL, Monze District
M.K. Chibanda
HRMO, Monze District
Peter M. Sililo
A/SO, Monze District
Louis Kalulu
Siampazanga
PO, Monze DistrictFerdinand Hampongo
DRCC, Monze District
Anna Masila
DEBS, Choma District
R. I. Mubita
DRCC, Choma District
W. Mwiinga
DESO, Gwembe District
52

C. C. Nzala
J. M. N. Sohati
W. S. Sakala
R. I. Mubita
Martha Phiri
Suzane Malikama
Mrs. Mudenda
Christine Sikwela
J. M. Muchoka
Cynthia Musamu
Vincent Mulobela
Kwalombota Tebuho
Racheal Tebuho
Alex Banda
Jones Chitelela
Dominic M. Simusuka
Doreen Moonga Matope
Jonathan Kasolo
Catherine Chilambe
Bernadette M. Kamoto
Titus Phiri
Mary Theresa Kamoto
Tadeus Swiderski
Immeculata Nisililo
Innonge Simtaa

PO, Gwembe District
Headteacher, Choma Day High School, Choma District
D/Headteacher, Choma Day High School, Choma District
DRCC, Choma District
Accountant, Choma Day High School, Choma District
Scholarship Coordinator, Choma Day High School, Choma District
Teacher in Charge, Zambezi Sawmills CS, Livingstone District
Headteacher, Mulwani Basic School, Livingstone District
Deputy Headteacher, Mulwani Basic School, Livingstone District
Focal Point Person, Mulwani Basic School, Livingstone District
Community Chairman, Mulwani Basic School, Livingstone District
Community Member, Mulwani Basic School, Livingstone District
SIC, Mulwani Basic School, Livingstone District
Tailoring Instructor, Mulwani Basic School, Livingstone District
Carpentry Instructor, Mulwani Basic School, Livingstone District
Headteacher, Musokotwane Secondary School, Livingstone District
D/Headteacher, Musokotwane Secondary School, Livingstone
District
ZIC, Musokotwane Secondary School, Livingstone District
Headteacher, Linda CS, Livingstone District
Co-ordinator, CHIKUNI Radio Station, Monze District
Taonga Field Monitor, CHIKUNI Radio Station, Monze District
Producer, CHIKUNI Radio Station, Monze District
Director, CHIKUNI Radio Station, Monze District
Director, MOSI-O-TUNYA Radio Station, Livingstone District
Programmes Assistant, MOSI-O-TUNYA, Livingstone District

Lusaka Province
Alice M. Nzala
P.E.O, Lusaka Province
Henry C. Kalwani
PESO, Lusaka Province
Martha M. Sitali
SEO - ODL, Lusaka Province
Salome Sakala
SESO - ODL, Lusaka Province
Neroh Mwanapabu
SEO - G&C, Lusaka Province
Pamela Banda
EO - TED, Lusaka Province
Shadrick Sikaonga
A/Statistician, Lusaka Province
Lisulo Liholosi
A/Senior Planning Officer, Lusaka Province
Agripa Simatimbe
DRCC, Lusaka Province
Henry J. Msango
University of Zambia, Sch. of Ed., Head - EAPS Dept, Lusaka District
Plyson Muzumara
Principal, NISTCOL, Chongwe District
Hector Swazi
Head, Distance Education, NISTICOL, Chongwe District
Ngonya H. Mwanga
A. DEBS, Chongwe District
Joseph Msoni
A. DESO, Chongwe District
Batra Kasimbo
PO, Chongwe District
Anne W. Kembo
ESO, Chongwe District
D. K. C. Matashi
S/Teacher, Lusakasa Basic School, Lusaka District
Jeniffer M. Sichinga
A/Senior teacher, Lusakasa Basic School, Lusaka District53

E. C. Malama
Evaristo Sichilima
Nelson Zimba
Stanford Mudenda
John Musukuma
Lister Kaonga
Jonson Nyirenda
Arnott Sakanya
Foster Musukuma
Tabo Maheritona
Happiness Hachileka
Yamikani Brian BandaJohn Shakafuswa
Caroline K. K. Serwadda
I. N. Mayey
M. L. Siakavuba
Amos Banda
Dennis Chikompa
Philip Phiri
Iness Yamba
Juliana Chisha
C. Monde
Claire Lungu
Gerry Musoni
Stephania Matambo
Betty SigandeDavid Mumba
Patrick Mpala
Rhodia S. Mudenda
Richard K. Mulenga
Harrison Mwaanga

SIC, Lusakasa Basic School, Lusaka District
Headteacher, Kalikiliki CS, Lusaka District
Teacher, Kalikiliki CS, Lusaka District
Teacher, Kalikiliki CS, Lusaka District
Teacher, Kalikiliki CS, Lusaka District
Teacher, Kalikiliki CS, Lusaka District
Parent, Kalikiliki CS, Lusaka District
Parent, Kalikiliki CS, Lusaka District
Instructor - Tailoring, Kalikiliki CS, Lusaka District
Coordinator, St Vincent Community
Teacher, St Vincent Community
Teacher St Vincent Community
D/Headteacher, Kabulonga Girls HighSchool Councellor, Kabulonga Girls High
Headteacher, Kabulonga Boys High School
Deputy Headteacher, Kabulonga Boys High School, Lusaka District
Accountant, Kabulonga Boys High School
Scholarship Coodinator, Kabulonga Boys High School
Teacher in Charge, Pillar Orphanage, Lusaka District
Headteacher, St Lawrence Basic School, Lusaka District
First Grade Teacher, St Lawrence Basic School, Lusaka District
Teacher, St Lawrence Basic School, Lusaka District
Teacher for Grades 8 and 9, St Lawrence Basic School, Lusaka
District
Teacher for Grade 6, Spec. Ed., St Lawrence Basic School, Lusaka
District
Headteacher, Mutendwa CS, Lusaka District
SIC, Mutendwa Basic School, Lusaka District
School Manager, Nangombe CS, Chongwe District
School Manager, Musolokeni CS, Chongwe District
DRCC, Chongwe District
ZIC, Chongwe District
Zonal Teacher, Ngwerere Zone, Chongwe District

Examinations Council of Zambia
Chekani T. Sakala
Director, Lusaka
Joe Kanyika
Assistant Director, Lusaka
Teza Msakanya
Principle Research Officer, Lusaka
Angel Kaliminwa
Senior Research Office, Lusaka
Shadreck Nkoya
Senior Research Officer, Lusaka
William Kapambwe
Senior Research Officer, Lusaka
Japhet Chanda
Senior Research Officer, Lusaka

54

ANNEX C
Data Collection Instruments

55

CHANGES2
Thank you very much for completing this anonymous questionnaire. We appreciate thetime, courtesy and valuable information you are giving in filling it out.
Please leave a question blank if you are you are not sufficiently familiar with the topic.
1. What is the degree of usefulness for children of the health, nutrition and HIV/AIDS
information and awareness in the curriculum of schools and teacher training colleges?
Very Low

Low

AverageHigh

Very High2. What is the quality by which health and health concerns are integrated in the curriculum?
Very Low

Low

AverageHigh

Very High

3. In your opinion, what is the level of quality of the training and supplementary materials
developed by the CHANGES2 program?
Very Low

Low

AverageHigh

Very High4. To what extent have teachers' professional skills been strengthened due to CHANGES2?
Not at all

A Little

Moderately

Much

Very Much

5. At the MOE's request, CHANGES2 has provided training in school-community
partnerships for personnel not only in its core provinces but also more recently in allprovinces of Zambia. What are and will be the value and results of this training?
Very Low

Low

ModerateHigh

Very High6. The CHANGES2 program has provided scholarships to orphans and other vulnerable
children (OVCs) for secondary school. To what extent have these scholarships promoted
access to school and equity for Zambian youth?
Not at all

A Little

Moderately

Much

Very Much

7. To what extent have small grants under the CHANGES2 program fostered participation
by parents and community citizens in support of local schools?
Not at all

A Little

Moderately

56

Much

Very Much

8. To what degree do you feel the MOE is stronger now, due to the CHANGES2 program, in
supporting in-service training and services for teachers?
Not at all

A Little

Moderately

Much

Very Much

9. What is the quality of the training materials for Community School Teachers (CSTs)?
Very Low

Low

AverageHigh

Very High

UsefulVery Useful

10. How useful has the SPRINT program been?
No use at all

A little

Moderate

11. How useful are the provincial and district resource centers?
No use at all

A little

Moderate

UsefulVery Useful

12. Teachers, head teachers and other education personnel have been trained in 31
districts through the CHANGES2 program. What has been the quality of this training?
Very Low

Low

AverageHigh

Very High13. What is the degree of impact or usefulness of bringing health workers and teachers
together in the same schools so that they integrate their efforts?
Very Low

Low

AverageHigh

Very High

14. In your opinion, rank order from 1 to 9 (with 1 being the most useful and 9 the least
useful), the value or merit of the following nine aspects of CHANGES2. Please use thesame number only once.
______ scholarships for OVCs______ teacher education and professional development______ school health and nutrition______ small grants to communities and schools______ school-community partnerships and outreach______ HIV/AIDS prevention and mitigation______ teacher group meetings______ training courses for teachers and head teachers______ capacity building and organizational strengthening of the MOE

57

15. Please list briefly the most valuable elements of CHANGES2:

16. Please list briefly the least valuable or useful elements of the CHANGES2 program:

17. Is there anything else you wish to mention about CHANGES2 and related aspects?

Thanks again, and best wishes to you in your important work.

58

QUESTTThank you very much for completing this anonymous questionnaire. We appreciate thetime, courtesy and valuable information you are giving in filling it out.
Please leave a question blank if you are you are not sufficiently familiar with the topic.
1. What in your opinion is the quality of QUESTT's interactive radio instruction (IRI)
programs?
Very Low

Low

AverageHigh

Very High2. How would you rate the quality of QUESTT's work with teacher training?
Very Low

Low

AverageHigh

Very High3. What is the degree of success of using volunteer teachers with the IRI programs?
Very Low

Low

AverageHigh

Very High4. What is the value of IRI programs for orphans and vulnerable children (OVCs)?
Very Low

Low

AverageHigh

Very High5. What is the level of quality and expertise of the technical personnel who work in the
QUESTT program?
Very Low

Low

AverageHigh

Very High6. What is the degree of relevance of the IRI methodology incorporated into the teacher
education curriculum at colleges offering the Zambia Teacher Education Course(ZATEC)?
Very Low

Low

AverageHigh

Very High7. To what extent has QUESTT helped strengthen the MOE's Directorate of Distance
Education (DODE)?
Very Little

LittleAverage

Much

Very Much

8. What is the degree of usefulness of "Fastele, Fastele," the series of 15-minute radio
programs implemented by QUESTT for teachers and volunteer teachers?
Very Low

Low

Average

59

High

Very High9. What is the extent of usefulness of ZATEC, including distance learning for Community
School Teachers (CSTs) so they can stay in their villages and schools?
Very Low

Low

AverageHigh

Very High10. How useful or beneficial are the small grants to support IRI programs?
No use at all

A little

Moderate

UsefulVery Useful

11. To what extent has the MOE strengthened itself to more adequately support IRI-basedlearning for children and adults (mainly teachers, including CSTs) in Zambia?
Very Little

LittleAverage

Much

Very Much

12. What is the quality of work carried out by the MOE and QUESTT to facilitate community
participation and partnerships with community radio stations in providing IRI?
Very Low

Low

AverageHigh

Very High13. How useful have been the community sensitization workshops conducted by theProvincial Outreach Coordinators?
No use at all

A little

Moderate

UsefulVery Useful

14. To what extent have the IRI programs and QUESTT increased access to education forchildren in Zambia?
Very Little

LittleAverage

Much

Very Much

15. What is the level of effectiveness the MOE and QUESTT have had in mobilizing
communities to support volunteer teachers?
Very Low

Low

AverageHigh

Very High

16. Please tell us two or three things you like or dislike about IRI programs.

17. If QUESTT had never been established, what would have occurred?

60

18. What difference, if any, has QUESTT made to the Directorate of Distance Education(DODE)?

19. Please list briefly the most valuable elements of QUESTT.

20. Please list briefly the least valuable or useful elements of the QUESTT program.

21. What elements or components, if any, should be added to a future MOE program such
as QUESTT and what elements should be eliminated?

22. Is there anything else you wish to mention about QUESTT and related matters?

Thanks again, and best wishes to you in your very important work.
61

EQUIP 2
Thank you very much for completing this anonymous questionnaire. We appreciate thetime, courtesy and valuable information you are giving in filling it out.
Please leave a question blank if you are you are not sufficiently familiar with the topic.
1. What has been the quality of technical assistance from EQUIP2 to improve the MOE's
ability to carry out research, policy development and management improvement?
Very poor

Poor

AverageGood

Very Good

2. What has been the degree of usefulness of EQUIP2's assistance with the EMIS?
Very Low

Low

ModerateHigh

Very High3. What has been the quality of EQUIP2's work with the Community Schools policy andimplementation guidelines?
Very poor

Poor

AverageGood

Very Good

4. Please rate the worth of EQUIP2's assistance to the MOE with continuous assessment.
Very Low

Low

AverageHigh

Very High5. What is the quality of EQUIP2's assistance with annual work planning and budgeting?
Very Low

Low

AverageHigh

Very High6. What is the level of usefulness of EQUIP2's assistance to the MOE with HIV/AIDS
workplace policy and implementation?
Very Low

Low

AverageHigh

Very High7. Supporting the MOE, what has been the value of EQUIP2's contributions to School
Health and Nutrition policies and guidelines?
Very Low

Low

AverageHigh

Very High8. What has been the quality of EQUIP2's efforts to supply IT equipment and training
services?
Very Low

Low

AverageHigh

Very High

9. To what extent has EQUIP2 helped the MOE improve the quality of basic education?

62

Very Little

Some

Moderately

Much

Very Much

10. EQUIP2 assists the MOE in the following eight strategic areas. How important is
assistance in these areas or categories? On a scale of 1 to 8, with 1 representing
highest value and 8 little value or merit, please rank the value of each area. Please do
not use the same number more than once.
______ school health and nutrition______ policy and research______ school management training______ information management______ continuous assessment______ information and communications technology infrastructure______ HIV/AIDS workplace activities______ capacity building and organizational strengthening of the MOE
11. To what extent will the MOE at all levels be able to sustain and continue the morevalued programs, if any, in which EQUIP2 has provided assistance?
Not at all

SomeModerately

Much

Very Much

12. Please list any problems or weaknesses you have seen with EQUIP2.

13. What elements or components, if any, should be continued in a future MOE program?
What should be added? What elements, if any, should be eliminated?

14. Is there anything else you wish to mention about EQUIP2 and related aspects?

Thanks again, and best wishes to you in your very important work.

63

Classroom Observation

Observer:________________
Province:

District:

Date:

School Name:

Time (Start/End):

School Type: ___government basic___ high school___CS (community school) ___other:______

Project(s):Subject Area/Class:Number of students:
Total present: _______
Total absent: _______

Grade level(s):Length of lesson:
M_____ F______
M_____

F______

Topic/Purpose of lesson:

Setting and Activities:
Briefly describe the classroom setting, environment, resources, any student work displayed.
Classroom arrangement:

Materials displayed:____Traditional____ Circle/semicircle____ Small groups____ Learning centers____ other:_____________________
Student all have desks/seats:____Student produced work____Posters, maps, supportive material____Previous class materials____Bulletin boards____other: _______________________
Yes____ No___

Students all have materials/supplies/books: Yes____ No____Describe content or skills taught.

Resourcesused in lesson available in room

1. Textbooks, other books
2. Chalkboard, chalk
3. Posters or pictorial materials
4. Hand-made materials :Teacher or Student (circle)
5. Teacher Toolkits (CS basic schools)
6. Supplies for student use (drawing/writing materials)
7. Other: (describe)

Formatcheck if present % of time

1. Lecture
2. Discussion to engage students
3. Students working together
4. Songs, games, rhymes
5. Questioning or assessing to determine if students understand6. Other: (describe)

64

Observation
Use the chronology to rate these observations AFTER the lesson. Take 5-10 minutes to use
your chronology to complete the ratings and fill in any details that will help you remember
the category later on.
Ratings:
0
1
2
3

Did not occur at all
Occurred very little
Occurred somewhat
Very descriptive of this lesson

Teacher Organization
Item

Observation
RatingComments

1. Time is used effectively, time not wasted,transitions smooth, lesson completed

0

1

2

3

2. Lesson is planned and has structure –introduction, connection to previous learning,
instruction, related activities, review/recall for
retention

0

1

2

3

3. Teacher present at all times and manages
the classroom

0

1

2

3

4. Environment and available resources areused to best advantage

0

1

2

3

Teaching Learning Process
Item
5. Teacher interacts with students

Observation
Rating0
1
2
3Comments

6. Students are clearly engaged in the lesson

0

1

2

3

7. Students are involved in their own learning– in groups, searching for information, creating

0

1

2

3

8. The purpose of the lesson and related
activities is made clear to students

0

1

2

3

9. Lesson links to students’ experiences andlocal communities

0

1

2

3

10. Teachers have sufficient content
knowledge to address questions as they arise

0

1

2

3

65

11. Teacher questions and assesses todetermine level of student understanding

0

1

2

3

12. Teacher encourages thinking andreflection

0

1

2

3

Additional observations or comments:

Use the following space to record what happens in the classroom chronologically during theobservation session. Include both what the teacher is doing and what students are doing(example: 9:00-9:10 “Teacher introduces lesson, students attentive”). Also, describe transitions
from one activity or location to another. Write specific quotes if pertinent.
Time
Activities
Some items to look for: 1. Elements of the lesson
2. Student behaviors and attention
3. Teaching methods andprocedures

66

Lesson Interruptions: Record any interruptions to the lesson you observed – distractions, noise,
behaviors. Do not count your own entrance into the classroom.
Number of
Describe
Interruptions

67

ANNEX D:
Results of Classroom Observations and Results ofAnonymous Questionnaires

68Types of Classroom Activities

Percent of time observed in theclassroom activities

Classroom activities

Community
Schools

Basic
Schools

Total

14

4

18

Number of classrooms

Percent of time observed in
the classroom activities
Learning in
TaongaOther
Total
Market13

5

18

Lecture

7%

19%

10%

5%

22%

Discussion/interaction with students

42%

38%

41%

42%

37%

Pupils working together

3%

5%

4%

0%

12%

Songs, games, rhymes
Questioning/assessing to determine
understanding
Other: setting up, distractions, notinstructional
Total

11%

7%

10%

12%

5%

29%

17%

27%

30%

17%

8%

14%

10%

11%

7%

100%

100%

100%

100%

100%

Observation ratings 0 to 3:

0 ‐ did not occur, 1 ‐ very little, 2 ‐ somewhat, 3 ‐ very descriptive

Observation of the learning process and Community
classroom activities
Schools

Basic
Schools

Total

Learning at
TaongaMarketOther

10%

Most of the observations were
41% LTM lessons in CS. The guided
4% format focuses more than 70% of
10% the activities on interactions withstudents and questions about the27%
material. Very little time isdevoted to lecture, students
10%move or sing about 11‐12% of the100%time, and an average of 10% of
the time is non‐instructional.
Total

1. Time not wasted, good use of time

2.6

2.8

2.6

2.5

2.8

2.6

2. Lesson appears planned, has structure

2.5

3.0

2.6

2.5

3.0

2.6

3. Teacher present and managing class
4. Available resources and objects in theenvironment used

2.7

3.0

2.8

2.7

3.0

1.9

2.3

2.0

1.9

2.2

5. Teachers interact with pupils

2.6

3.0

2.7

2.6

3.0

6. Pupils clearly engaged in lesson
7. Pupils involved in own learning ‐ groups,
search for information

2.6

2.8

2.6

2.5

2.8

1.6

2.8

1.8

1.6

2.4

8. Purpose of lesson and activities clear

2.9

3.0

2.9

2.8

3.0

9. Lesson links to pupils' experience
10. Teacher has content kowledge to addressquestions
11. Questions to determine level ofunderstanding
12. Encourages thinking and reflection

2.3

1.8

2.2

2.2

2.0

The highest ratings for LTM
were clear purpose and teacher
2.8
managing the class ‐ 2.8 and 2.7
2.0 out of 3.0. Most ratings for LTM
2.7 were close to lessons taught by
2.6 trained teachers. Reflection andpupils directing their own
1.8 learning was lower but linking
2.9 lessons to the pupils' experience
2.2 was higher.

2.4

2.5

2.4

2.4

2.6

2.4

2.5

3.0

2.6

2.5

3.0

2.6

1.6

2.5

1.8

1.5

2.6

1.8

69

Degree of usefulness or level of quality of thefollowing CHANGES2 related activities

T eachers, Colleges,
Community Projects

Number of Respondents
1. Heal th, nutri ti on, HIV/AIDS i nforma ti on and awarenes s i n s chool curri cul um
and tea cher trai ning

11

16

4.5
4.4
4.4
4.5
4.3
4.1
4.9
4.3
4.3
4.9
4.3
4.6
4.3

2. Heal th concerns integrated i n curri cul um
3. Trai ni ng and s uppl ementary materi al s
4. Strengthening of teachers ' profes s i ona l s kil l s
5. School ‐communi ty partners hi p trai ni ng val ue
6. Schol a rs hi ps for OVCs
7. Grants to parents /communi ty pa rti ci pa ti on i n s upport of s chool s
8. MoE s tronger becaus e of CHANGES2 trai ni ng for tea chers
9. Trai ni ng materi als for Communi ty School Teachers
10. SPRINT program
11. Provi nci a l and di s tri ct res ource centers
12. Trai ni ng for teachers , head tea chers , other s chool pers onnel
13. Bri ng heal th workers /teachers together i n s chools to i ntegra te efforts

Province,
District, T OT AL
Zone
32

59

3.9

4.3

4.2

3.5

3.9

3.9

3.6

4.2

4.1

3.6

4.3

4.1

3.7

4.0

4.0

3.9
3.9

4.1
4.1

4.0
4.2

3.3

4.0

3.9

3.8

4.1

4.1

4.1

4.4

4.4

4.1

4.3

4.3

3.8

4.0

4.1

3.9

4.2

4.1

3.8

2.8

5.4

4.2

4.1

4.7

3.9

5.7

Rank the importance of the following nine aspects of CHANGES2 work
3.9
4.3
4.6
4.8
3.8
6.7
4.4
8.6
3.6

Teacher educati on and profes s i ona l devel opment
Trai ni ng cours es for tea chers and head tea chers
School heal th and nutri tion
HIV/AIDS preventi on and mi ti gati on
Teacher group meeti ngs
OVC Schol a rs hips
School‐community partners hi ps and outreach
Capaci ty bui l di ng/organi zati ona l s trengtheni ng of MoE
Smal l grants to communiti es and s chool s

6.0

5.3

4.9

5.1

5.6

5.6

4.8

5.0

6.4

6.3

Rank order
1
4.5
2
4.6
3
5.2
4
5.2
5
5.4
6
5.4
7
5.6
8
5.8
9
3.2

Res pondents are nearly a l l from Lus aka and Southern Provi nce. The Ea s tern Provi nce, and other unders erved provi nces want CHANGES2 opportuni ti es .
Ma jor res pons es to open ended ques ti ons :‐ nearl y every res pondent i denti fi ed tea cher trai ni ng and ca pa ci ty bui l ding as among the mos t val ua bl e contributi ons of CHANGES2‐ next mos t commonl y i dentifi ed valuabl e contri buti ons were provi di ng s ma ll grants to communiti es and the work with SHN‐ recei vi ng many comments were s chool community partners hi p development, s chol a rs hi ps for OVCs , and work wi th HIV/AIDS.
Comments about the future i ncl uded s trong recommenda tions to broaden the work to the other provi nces , more di s tri cts , down tozona l l evel, as well as i mbeddi ng the traini ng and materi al s i n the tea cher educati on col l eges and i n‐s ervi ce programs .
Nothing was identi fi ed to be el iminated.

70Degree of usefulness or level ofquality of the following QUESTT
related activities
Number of respondents
1. IRI program quality
2. Work with teacher training
3. Using volunteers teachers with IRI
4. IRI programs for orphans and vulnerable children
5. Quality and expertise of technical QUESTT personnel
6. IRI methodology incorporated into ZATEC curriculum
7. Helped strengthen DODE at MoE
8. Fastele, Fastele for teachers
9. Usefulness of ZATEC distance learners to keep CST in theirvillages and schools
10. Usefulness of small grants to support IRI
11. MoE strengthened to support IRI based learning
12. Community participation, partnerships with radio
stations to provide IRI
13. Community sensitization by POC
14. IRI increased educational access for children
15. Level of MOE and QUESTT successful in mobilizing
community support for volunteers

College, Radio,
Community,
Projects,
Teachers

District,
Zone,
Provinces

TOTAL

18

22

40

4.3
3.5
3.6
4.1
4.1

3.8
3.8
3.4
4.0
4.1

4.0
3.7
3.5
4.0
4.1

3.0
3.9
2.9

3.7
4.3
3.1

3.5
4.2
3.0

2.9
4.5
3.6

3.4
4.3
3.6

3.2
4.4
3.6

3.6
3.9
4.4

3.8
4.2
4.4

3.7
4.1
4.4

2.4

3.1

2.8

Most positive comments about IRI focused on the motivation for pupils, the educational value, learner
centered, interactive, builds listening skills, improves attendance
Many respondents also indicated they felt without QUESTT and IRI many Zambian children would have nono access to education
If QUESTT and their services had not existed, an overwhelming number of respondents felt many children
would have had minimal or no opportunities for education
Concerns surround poor radio reception, reduction in airtime, non availability of radios and guides thatwere promised have not arrived, and the fast pacing of the lessons

Degree of usefulness or level of qualityof the following EQUIP2 related
activities
Number of Respondents
1. Technical assistance to improve MoE research, policy,
management
2. Assistance with EMIS
3. Work wih Community School policy and guidelines
4. Assistance with continuous assessment
5. Assistance with annual work plans and budgeting
6. Assistance to MoE withHIV/AIDS workplace policy andimplementation
7. Support to School Health and Nutrition policies andguidelines
8. Efforts to supply IT equipment and training
9. Help MoE improve quality of basic education
11. Extent MoE can sustain valued EQUIP2 programs ofassistanceColleges,
Projects

Districts,
Provinces,
Ministry

TOTAL

11

20

31

3.4

3.4

3.9
3.4
3.2
3.4

4.3
3.8
3.7
3.8

3.4
4.1
3.7
3.5
3.7

3.0

3.4

3.3

3.3
3.5
2.9

3.6
3.9
3.4

3.5
3.8
3.2

3.1

3.0

3.0

Rank the importance of the eight strategic areas of
assistance provided to MoE by EQUIP2
Information management
IT and information infrastructure
School health and nutrition
Continuous assessment
Capacity building/organizational strengthening of MoE
Policy and research
HIV/AIDS workplace activities
School management training

Rank
Order
2.1
3.3
5.3
4.2
4.3
4.0
6.3
5.1

2.6
3.8
3.8
4.4
4.5
5.1
4.9
6.2

2.3
3.8
4.1
4.4
4.6
4.8
5.3
6.1

Common concerns identified in responses to open ended questions:‐ support from EQUIP2 concentrated at Ministry ‐ provinces and districts want more involvement, training,
capacity built in information management, as well as IT connectivity to districts and colleges ‐ and soon
‐ more transmission of skills from EQUIP2 officers to MoE employees, more building of capacity via advisory role
‐ input, advice, recommendations should be sought from the field in planning, design, needs
The most common comment about what should be retained ‐ everything ‐ but most important identified as
ICT infrastructure development and management

1
2
3
4
5
6
7
8