metadata: authors: 0: author: Nyagura, Levi Martin
1: author: Riddell, Abby Rubin
count: Zimbabwe
docna: 0: docna: What causes differences in achievement in
            Zimbabwe's secondary schools?
docty: Policy Research Working Paper
subtopic: Public Sector Management and Reform,Educational Sciences,Educational Institutions & Facilities,Educational Populations,Education for Development (superceded),Educational Policy and Planning - Textbook,Effective Schools and Teachers,Education For All
teratopic: Gender
abstracts: cdata!: The authors found that students who
            attended high-fee-paying (trust) schools, elite urban
            governments schools, and mission schools scored better in
            mathematics and English achievement than did students in the
            less-well-endowed government schools and those established
            by local councils.  Much of the variation in the student
            achievement was attributable to the schools the student
            attended.  Examination results were higher in schools with a
            high proportion of trained teachers, with a good supply of
            textbooks, and with a stable faculty (high teacher
            retention).  But once researcher control for these factors,
            contrary to expectations, some underendowed local council
            and government schools are more effective at boosting
            achievement than their counterparts with more resources.
            So, textbooks and teachers are important in raising
            achievement, but more research is needed into what
            characteristics differentiate high-achieving schools from
            low-achieving schools.
geo_reg_key: 517195,517191
repnme: repnme: What causes differences in achievement in
            Zimbabwe's secondary schools?
display_title: What causes differences in
            achievement in Zimbabwe's secondary schools
listing_relative_url: /research/1991/06/699950/causes-differences-achievement-zimbabwes-secondary-schools
topicv3: Teachers Management m1326119 1523,Human Resources m1326304 1211,Textbooks m1326802 2129,Education m1327048 470,Education and Culture m1327747 1472
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Gender
origu: Policy, Research & Ect Affairs
no_of_pages: 62
model_extractions: **Policy, Research, and External Affair**

# **Education and Employment**

WPS 705

- This paper commissioned by the Population and Human Resources Operations Division, Soulthem Africa Department, Africa Regional Office, jointly with the Education and Employment<sup>Division,</sup> Population and Human Resources Department and Zimbabwe's Ministry of Education and<sup>Culture-was</sup> prepared as a background paper for a review of primary and secondary education in Zimbabwc.<sup>Copies</sup> are available free from the World Bank, 1818 H Street NW, Washington DC20433. Piease contact<sup>Cynthia</sup> Cristobal, room S6-214, extension 33640 (55 pages, with tables).

Riddell and Nyagura found that students who attended high-fee-paying (trust) schools, elite urban government schools, and mission schools scored better in mathematics and English achievernent than did students in the Iess-wellendowed government schools and those established by local councils.

Much of the variation in student achievement was attributable to the schools the student attended. Examination results were higher in schools with a high proportion of trained teachers, with a good supply of textbooks, and with a stable faculty (high teacher retention).

But once researcilers control for these factors, contrary to expectations, some underendowed local council and government schools are more effective at boosting achievement than their counterparts with more resources.

So, textbooks and tcachers are important in raising achievement, but more research is needed into what characteristics differentiate highachieving schools from low-achieving schools.

The PRE Working Paper Scries disseminates the findings of work under way in the Banik's Policy.<sup>Research, and Extemal</sup> I Affairs Complex. Anobjective of tie scries is to get these findings out quickly, even if presentations are<sup>less than fully polished.</sup> The findings, interpretations, and conclusions in these papers do not necessarily represent official Bank policy.

Produced by the PRE Dissemination Center
**Tab1e** of Contents

|I<br>.<br>INTRODUCTION**.**<br>**.**<br>.<br>**.**<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>1|
|---|
|Background<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**,**<br>**.**<br>**- *.****<br>edrdt<br>***1 ***|
|<br>Expansion and Quality inSecondary Education**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>2|
|Context of the Study . . . . . . . .<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>. .<br>6|
|**II.**RESEARCH<br>DESIGN**..**<br>***.*..**<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>7<br>|
|Sample........<br>**.**<br>....<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>**8**|
|Variables<br>. .<br>...........<br>**.**<br>**.**<br>**..**<br>**.**<br>**.**<br>**.**<br>.<br>.<br>10|
|Methodology<br>**.......**<br>. . ...<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>12|
|Descriptionof**Sample**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**...**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>13|
|III. THE PROGRESSION TOWARD 'F:NAL' MODELS FOR ZJC ENGLISH AND|
|MATHEMATICS ACHIEVEMENT<br>.<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>.<br>18|
|BACKGROUND MODELSt ADJUSTING FOR INTAKE . .<br>. . . .<br>. . . . . . . . . . .<br>20|
|Model As The Variance Components of Grade 7 Scores ...**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>.<br>20|
|Model1S TheVariance Components of ZJC Scores**.**<br>.<br>**...**<br>**. ..**<br>**.** .<br>20|
|Model 2: Controlling for Grade 7 Scores.<br>. . . . .<br>. . . . . .<br>.<br>21<br>|
|Model 2as Allowing the Grade 7 Intake Variable to be Random at the|
|SchoolLevel (for English).<br>. . . . . . .<br>. . . . . . .<br>22|
|<br>Model 3s Schooltype Differentiation.<br>**.**<br>**.**<br>**.**<br>**..**<br>**.**<br>**...**<br>**.**<br>**...**<br>**.**<br>.<br>22|
|Models 4 and 5: Controlling for the Sex of the Pupils .<br>.<br>.<br>.<br>. . .<br>26|
|WHICHVARIABLES TO CHOOSE?<br>**...**<br>**,...**<br>.<br>.........<br>.<br>.<br>**.........**<br>.<br>26|
|Testing Class and School Level Variables for Possible Inclusion.<br>**.**<br>26|
|Unsuccessful Models<br>.... . .<br>. . . . .<br>.<br>**...................**<br>**....**<br>28|
|TEXTBOOKS, TRAINED TEACHERSAND TEACHER TURNOVER.<br>. . . . . . . . .<br>. .<br>29|
|Model<br>6: The Availability of Textbooks**.**<br>. . .<br>**. .**<br>**..**<br>.<br>.<br>.<br>.<br>.<br>.<br>30|
|Model 7t The Percentage of Trained Teachers in Secondary Schools.<br>.<br>33|
|Model8sTeacher TurnoverinSecondary Schools.<br>.<br>.<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>.<br>.<br>.<br>34|
|THE EFFECT OF DIFFERENT COMBINATIONS OF THE SCHOOL LEVEL VARIABLES:<br> <br>|
|THESEARCH FOR THE 'BEST' MODEL.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>**.**<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>34|
|Models 9,10and11.<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>.<br>.<br>35|
|THE'FINAL' MODELS FOR ZJC MATHEMATICS AND ENGLISH ACHIEVEMENT:|
|COMPARISON WITH THE RESULTS OF THE 1985 STUDY<br>36|
|<br>.<br>. . . . . . . .**.**<br>. . .<br>|
|IV. THE MOST ANDTHELEAST EFFECTIVE SCHOOLS AND CLASSES IN ZJC|
|<br> <br>40|
|MATHEMATICS AND ENGLISH<br>..9.9.9......................... . . . **.9**.<br>|
|**V.**CONCLUSION<br>**.**<br>**.**<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>.<br>**50**|
|REFERENCES<br>.<br>.<br>.<br>.<br>**.**<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>.<br>**.**<br>.<br>.<br>.<br>53|

Abby Riddell **is** an zducation Researcher at the Harvard InstitutS for International DovelopPMOnt; Levi Nyagura is a professor of Education at **the** University of Zimbabwe.
TABLES

|TABLE1.1?Grade 7/Form 1 Trans3.tion Rates(%),1970/71-<br>1988/89 . . . .|1|
|---|---|
|TABLE 1.2: Number of Secondary Schools and Enrolments, 1979-<br>1989 . . .|3|
|TABLE 1.3: Numbers of Teachers in Secondary Schools, 1979-1989 . . . . .|4|
|TABLS 1.4: Student Pass Rates in Mathematics and English at the ZJC Level,<br>1984-<br>1988 . .<br>. . . . . . . . . .<br>.<br>. . . .<br>.<br>. .<br>. .<br>. . .|6|
|TABLE 2.1: Breakdown of Secondary Schools by Schooltyp- in Sample and<br>Guesstimates _..n<br>Total Population, 1989 . . . . . . . . . . .|10|
|TABLE 2.2: Description of Final Sample: No. of Schools by Schooltyps, No.<br>Classes, No. of Pupils(<br>**.**)<br>. . ....<br>**.**<br>**. .**<br>**.**<br>.<br>**.**<br>.|of<br>14|
|TABLE 2.3 Missing Data from Original Sample: No. Schools by Schooltype (%<br>Missing Data from Final Sample by Subject: No. Pupilo . . . .|)<br>14|
|TABLE2.4: Average Values of Variables by Schooltype (S.D.) . . . . **.** . .|15|
|TABLE 3.1: Fixed and Random Parameters (Standard Errors) of Initial Varian<br>Components Models for Mathematics (Pupil Level Variable.) . .|ce<br>24|
|TABLE 3.2: Fixed and Random Parameters (Standard Errors) of Initial Varian<br>Components Models for English (Pupil Level Variable.) . . . .|ce<br>25|
|TABLE 3.3: Fixed and Random Parameters (Standard Errors) of Further Models<br>forZJC Mathematics<br>.......<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>**.**<br>.|31|
|TABLE 3.4: Fixed and Random Parameters (Standard Errors) of Further Models<br>for ZJC English .<br>. . . . .<br>. . . . . . . . . . . .<br>. . . . .|32|
|TABLE 3.5: Fixed and Random Parameters (Standard Errors) of Final Models<br>for ZJC English and Mathematics . . . . . . . . . . . . . . .|37|
|TABLE 4.1: Positive and Negative Outliers: Standardized School Level<br>Residuals, Actual and Predicted Average School Grades by<br>Subject . . . . . . . . . . . . . . . . . . . . . . . . . . .|43|
|TABLE4.2: Ranking of Schools by School Level Residuals: English and<br>Mathematics<br>.<br>. . . . . . . . . . . .<br>.<br>.<br>.|48|
|TABLE 4.3: Ranking of Schools by Actual and Predicted ZJC Grades: English<br>Mathematics......<br>.<br>**.**<br>**.**<br>**.**<br>**.**..|and<br>49|
# GRAPHS

GRAPH Is STANDARDIZED SCHOOL LEVEL RESIDUALS (Y) PLOTTED AGAXNST PREDICTED AVERAGE SCHOOL ZJC RESULTS IN MATHEMATICS (X) . . . . . . . . . . 41 GRAPH 2: STANDARDIZED SCHOOL LEVEL RESIDUALS (Y) PLOTTED AGAINST PREDICTED AV&RAGE SCHOOL ZJC RESULTS IN ENGLISH (X) . . . . . . . . . . . . 42 APPENDIX 1: List of Variables . .. . . .54 APPENDIX 2: School Coding . . . . . . . . . . . . . . . . . . . . . . 55 This Study was commissioned by the Southern Africa PopUlation and Human Resources Division of the World Bank as a background paper for a review of primary and secondary education in Zimbabwe.
# I. INTRODUCTION

Historically Zimbabwe inherited at Independence in 1980 an educational system which had denied educational opportunities to the majority African (black) population. The most diutinguishing feature of the pre-independence educational system was its racial bias, characterized by limited provision of educational resources for the African population, especially in rural areas.

Before independence access to secondary education by African pupils was governed by strict sOlection policies not found in the European education sector. As a result of these harsh politically motivated selectLon **policies,** Grade 7/Form 1 transition rates before independence wore below 35%. (Table 1.1)

- TABLE 1.1: Grade 7/Form 1 Transition Rates (%), 1970/71 1988/89

Year: 70/1 2112 223 73/4 74_5 _75L6 7617 18_ LZL/2 L9/89 Transition Rate: 30 33 31 30 27 25 23 22 22 27 Year: fiQ/1 01/2 02/3 83/4 _84L5 85/6i_ 86/7 _82/8 8A/_ Transition Rates 86 70 74 82 82 _78_ 70 _65_ 66 Source: Calculated from Reay-O for African Educatign, and Reort2 of ehe Secret=far fr u druati2on, various year., Government Printers,Salisbury and Harare, and 0T*achers' Colleges and Schoolss Staffing and Enrolment Statistics, vArious years, Ministry of Education, mimeo.

In addition to unequal access to education, there were political, social and economic disparities under colonial rule. Swift and decisive political action by an independent Zimbabwean government was, therefore, expected by the
black majority in order to correct and reverse the Imbalances and the gross inequaili*e prevailing in the society. Such swift actioA was taken in the education **sector** within six months of obtaining indepondence.

# gxyans 4n and Oualitv in SecondarX,yBMS&tAM

The dramatic changes in educational provision for the African population soon after independence were to a large extent motivated by the desire of ZANU (PF), the ruling part,!, to meet the hiVh demand for education and the aspirations of thousands of black youths denied access to education by the colonial governments. Notably, education, rather than being viewed **as a** privilege, ***.as** declared to be a basic right. The Zimbabwe government com-tted itself to a number of ob4ectives:

- the provision of universal primary education

- the abolition of a racial education system.

- **the** establishment .f a uniform national system of education.

- the establishment of at least one government school in each of **the** rural administratlve districts

- the training of more teachers for both primary and seconodary schools.

- the expansion of secondary educitional facilities especially in those areas previously neglected by the pre-independen7e regimes.

The reality of commitment is amply demonstrated by the statistics in Tables 1.2 and 1.3.

2
TABLE 1.2: Number **oa** Secondary schools and Enrolments, 1979 - 1989

|**Year**|sehool|Schoo Ind2x|Enrolment|Enrolment|
|---|---|---|---|---|
||__|__________________<br>_SIndex+|_||
|1979|17'|100|66215|100|
|1980|197|111|74321|112|
|1981|694|392|148690|225|
|1982|738|417|227647|344|
|1983|790|446|316438|478|
|1984|1182|668|416413|628|
|1985|'215|686|482000|728|
|1986|1276|721|537427|812|
|1987|1395|788|604652|913|
|1988|1484|838|653353|987|
|1989|1502|849|698828|1OS0|

* School index baseline Ls the 1979 figure.

+ Enrolment index baseline is the 1979 enrolment figure. Sourcee **See** Table 1.1 The data in Table 1.2 demonstrate the magnitude of expansion in both numbers of secondary schools and in enrolments. While there were only 177 secondary schools in 1979, the number increased to 1,502 in 1989 reflecting a phenomenal increase of 749%. In terms of enrolment, the relatively small figure of 66,215 students in 1979 jumped to 695,882 Ln 1989, an increase of 951%.

3
TABLX 1.3t Nuw are of Teachers in Secondary Schools, 1979-1989

|Year|3T<br>PIndexw||LlIntrainad|l**_ndercralIfi**<br>**_d+_**|
|---|---|---|---|---|
|1979|35.1)|100|n.a|6|
|i980|3736|106|3|7|
|1981|6112|173|5|7|
|1982|8349|236|12|12|
|1983|1119;|317|22|46|
|1984|14718|416|41|18|
|1985|17315|489|45|19|
|1986|19487|551|47|17|
|1987|21981|622|49|16|
|1988|23899|676|n.a.|n.a.|
|1989|24856|703|50|n.a.|

n.a. not available.

_*_ The teacher index baseline is the 1979 figure.

+ Underqualifisd is defined as Standard 6 plus 2 year. teachor training, Junior Certificate (J.C.) plus 2 or 3 years teacher training, and Cambridge School Certificate (C.S.C.) plus 2 or 3 or 4 yeare primary teacher training.

Source: See Table 1.1 The rapid increase in student enrolments was matched by a rapid growt<sup>t</sup> :n the number of teachers. As can be seen from Table 1.3, the teaching force which stood at 3,534 in 1979 rose to 24,856 in 1989. However, the steady increase in numbers of teachers had another side to it, namely the increase in the number of untrained and underqualified teachers employed in secondary schools. Whilst only 3% of secondary school teachers were untrained in 1979 the percentage rose to 50% in 1989. The percentage of underqualified teachers has fluctuated between 7 - 46% in the period 1980 - 87. Government efforts to train more secondary school teachero have been significant. While there were only two secondary teacher training colleges in 1979, the number had increased to four by 1985 and a fifth college is due to opr. shortly in Chinhoyi. The data in Tables 1.2 and 1.3 clearly demonstrate that the Zimbabwean government 4
**has** porformed outotanding4y ln increaaing the quantity of education - that is building more **schools,** hiring more teachers, and enrolling more pupils. However, the quality of teaching and learning in secondary schools as measured by academic achievement _rates_ in public examinations show- a steady decline over the ten years of Independence. ?or example, in 1979 63% of those students sitting for the CambriAse '0' levels passed in at least **fLve** subjects whereas only 13% attaLned the same standards in 1985. _The_ situatlon in 1989 has not improved. Of about 169,000 studento sitting for the Cambridge 1o' levels (in 1989) only about 22,000 (13%) passed in at least five subjects. Of courso, whilst the percentages passing have decreased since Independence, **the** actual numbers of pupils sitting the examinatione have increased.

This increase in access to secondary education has allowed pupils of low academic ability to enter secondary schools whilst the academically oriented curriculum of the pro-independence period has remained almost uncha!nqed. Furthermore, these schools lack adequate resources such as g-alified and experieneod teachers, textbooks, library books, and other instructional materials.

A stmilar decline in educational quality is illustrated by the achievement rates of students in the ZLdbabwe Junior Certificate (ZJC) English and Mathematics public exam,;^ations for the period 1984 - 88.

5
TABLE 1.4: StAdent **Pau** Rates In Mathenatics **and** English **at** the ZIC<sup>Lavul,</sup> 1984- 1988

||~~|EN5L-ISRH<br>No!a~Nnn|<br>_1 PA_uAO, W|No. §igir|MATHEMA-TICS<br>Ng. ap|-<br>%Psa1t|
|---|---|---|---|---|---|---|
|1985|148670|80832|<br>54|<br>122461|<br>15770|13|
|1986|129909|62660|48|130266|23865|IS|
|1987|146674|62377|3|146241|13672|9|
|1988|169913|58252|34|169009|26716|16|

Source: See Table 1.1 Table 1.4 provides data on pupile reaching a passing standard in the Zimbabwe Junior Certificate national examinations in Mathematics and English. Of 101,086 students who wrote tho ZJC English examination in 1984 on&ly 29 percent attained a passing standard, and the number of students<sup>successfully</sup> completing the ZJC English program has declined steadily since the<sup>leap in</sup> 1985. For Mathematics, the rate of ettlent academic achievement ia disappointingly low (9-18%).

# Context of the Study

These declining levelr of attainmment signal the erosion of educational quality through pressures to expand the educational system in the face of diminishing resources. Within Zimbabw3 such scarce resources include salaries, tuition expenditures, physical facilities, and instructional materials. What appears an appropriate strategy to curb the decline in education,l quality in Zimbabwe secondary schools is to identify school, and class characteristics which boost student academic achievement. The principal aim of this study was to identify schools which produce the best and worst ZJC results in English and Mathematics after controlling for the students' academic background on intake and , determine the characteristics of effective and ineffective secondary schools. The results of the study were an important input into a joint Zimbabwe Ministry of Education and Culture and 6
World Bank education aector atudyy carried out in May-June _l990,_ Major ioeues addressud in the sector study includes low levels of learning achievements, especially iL the recently established aecondary echcola, the unattractive prospects for many school-leavers due to the high level of youth unemployment, whether the Government will be able to continue increasing its financ4al support to the education sector, whether exiating esources are being used optimally, and what additional resources can be utilized for education.

# It. RESEARCH DESIGN

Many countrie, produce 'league, tables of school examination results which ars meant to Identify the beat and the worst schools. Such league' tables do, of course, tell us which schools produce the top grades, but as &very parent and teacher knows, the schools which attract the best students, i.e. those having already proven academic ability, A-e bound to produce the best examination results. Such league tablea are more indicative of the intake of schools than the actual output, for they do not account for the students' prior ability, and therefore they do not measure what has actually been achieved at school. Of far greater interest than a table showing which schools produce the top grades is a rather different sort of 'league' table, which shows the increment in achievement after controlling for schoolel intake of students.

7
Thli study usec tho exam results on **the** Lnternally set ZLmbabwe Junior Certificate (ZJC)V.The cholce of English and Mathematles, besides boeng two compulsory subjects whose examLnation results span dLfferent types of academia abillty, r **u** predLcated on our being able to control for prlor achievement in these subjects, their being the two subjects examLned in Grade 7, the flnal year of primary school.

SAm2l1 It was Lmportant that the sample of schools chosen for ths analysi be representative of the natlonal populatlon of schools. Slx schooltypes adequately reflect the most Lmportant, different kinds of schools operatlng in the country, namely: (1) former Group A government schools, which had catered for **the** European population before Independence, (2) former Group B urban government schools, (3) former Group B rural government schools, both of whlch had catered solely for the Afrlcan population before Independence, (4) high fee-paying (trust) schools , similar to AmerLean private schools' or BrLtLih 'public schools', (5) miosLon schools and (6) the new local-authorlty run dLitrict councll schools.

In order that the students sampled in these six schooltypes also be representatlve of the major othnlc groups Ln Zimbabwe, but at the same time not too dLipersed for follow-uvp school visits, four reglons were selected for further sampling# Harare, the capital, Mashonaland West, the Midlands and /Although lt would have been preferable to uso '0' level examinations taken ln what is, for most students, their final year of secondary school, the sector mLssLon would not have had the opportunlty to intervLiw the students, who, had they **been** ln Form IV last year, would already have left school.

_8_
Matabeleland North. A fair spread of **Ndeb-le** and Shona children attending both urban and rural schools could be obtained from these four regions, without, in addition, posing any security problems. A stratified random sample was drawn, proportional to the size of the districts in the chosen regions. The districts selected we-es Mhondoro and SanyatL (Mashonaland West), Shurugwi and Takawira (Midlands) and Binga and Nkayi (Matabeleland North). Further sampling, proportional to the size of the secondary schools by the six schooltypes, was carried out in those districts.

Ideally, one would want to have the number of schools in each schooltype reflect the total school population. This was attempted, but the actual breakdown of secondary schools by schooltype, could not be obtained. In addition, there was the further constraint of having sufficient numbers of schools in each schooltype to make a reasonable analysis. rt **was** decided that four schools per schooltype would suffice. Table 2.1 extrapolates from the known breakdown of achools for 1985, and includes "guesstimates" for 1989, given the known total school population.

9
# TABLE 2.1: Breakdown of Secondary Schools by Schooltype in Sample and Guesstimates in Total Population, 1989

||No. in<br>sample||Estimat<br>1989|e|
|---|---|---|---|---|
|Former Group A|4|8|34+|<br>2|
|Former GroupB<br>(urban)<br>Former Group B (rural)|4<br>6|8<br>13|)<br>**_I_**<br>)158@|)<br> <br>)11|
|High Fee-paying|4|8|38|3|
|Hission|6|13|172|11|
|District Council|24|50|1006|67|
|Other|-|-|94|6|
|TOTAL|48|100|1502|+100|

+ Actual Figures

# Variables

The time frame wLthin which the study was conducted produced certain constraints. A decision had to be made as to which variables (from the ideal set of variables one would like to include in a study such as this) could realistically be obtained in the time from mid-February, when the sampling was carried out and the data collection begun, and mid-April, when the schools closed for Baster and the analysis began.

The main prerequisites of the study were that the ZJC English and Mathematics results as well as the Grade 7 English and Mathematics examination scores **were** reported for each student sampled. (All Form I students in 1989 10
**whose** names were on tho ZJC entry forms for each of the 48 sampled schools were included.) As most secondary schools have a record of their students' Grade 7 scores, we sent the ZJC entry form print-out to each of the head teachers of the 48 schools sampled and requested that they fill in each 1989 Form II studentle Grade 7 marks. The Grade 7 examination scores allow a control to be made for intake ability ln each of the subjects, enabling us to model the increment in achievement that could be attributed to the secondary school, rather than the child's previous schooling. Whilst we would have liked to have had a measure of each student's socio-economic background, something which we could have obtained through a detailed questionnaire, along with other, student-level information, it was thought unrealistic to expect the head teachers to administer such questionnaires in the time available. Besides the four grades for each pupll, the two subjects for the two examinations - the other pupil-level variables which we were able to Lnclude in the study were the sex and age of the pupils, which were obtained from the ZJC entry forms.

It was also possible to match each pupil with her/his Form II English and Math teachers as well as classmates by using information from the school heads on the deployment of the Form It English and Math teachers in each class. This enabled us to carry out a three-level analysis, including the class as well as the school and pupil levels.

?/in a 1985 study of 'O' level results in English and Mathematics (Riddell 1988), both an intake variable and a pupil-level background index variable were included, but the influence of the intake variable was much larger than the socio-economic variable. In fact, while modelled separately, it was questionable to what extent the intake variable covered for some of the influence of the background variable.

11
> **Aside** from **one** further variable collected from the head teachers, namely, how many pupils **had** to share each textbook in Math and in English, the remaining **clas** and school level information was collected from statistical returns to the Ministry of Education and Culture, on the ED.46 (Part II) form. This narrowed down the optimal list of variables to include the following, additional **class** level variablecs the qualifications and years of experience of the Form II Math and Englieh teachers and thG class size, and the following, additlonal school level var$ables: the percentage of teachers Ln the four different qualLficatlon bands, ranging from untrained to cortlficated graduate, the average years of teachlng experience at the school, the average years of teaching at the partlcular school, whether the school **was** day or boardlng, the size of the school, the overall teacher pupi ratio and the percentage of Africans at the school. A lLst of all the varlables Ln the study can be found in Appendix 1.

# ,Msthodgl=o

The choLce of a multilevel model was an obvious one, given the objectLve of the study. Whllst ordinary least squares regression could control for the intake ability of the students, it would not have been possible to infer at the class and school levels the implications of this control and other parameters. Similarly, if one focussed on the school level, it would not have been possLble to adequately account for the prior ability of the students. This would be rather like seeking explanations for the development of a hybrid species without knowing the constituent breeds. The arguments concerning the appropriateness of multilevel modelling for nested, hLerarchical data found in an education system in which pupils are nested in classes, which in turn are 12
**nested** ln schools, have been put forward ln **several** contexts, e.g. (Burstein 1980a, 1980b, Goldsteln 1987, RLddell 1988, 1989). One of **the** main advantages of usLng **a** multllevel model for a study such an thli is **the** ability to portray sLmultaneously the Lnfluence of different factors at all the levels **one** chooses to specLfy, so there is no need to be limlted by _'a'_ unlt of analysls. The rLchness of interprotation made possLble by multllevel modelling will become clear ln the dlicussion of the results. The multilevel package used for the analysis was L3 (Rasbash, Prosser, Goldstein 1989).

# Description of Sample

Of the 48 schools orlglnally sampled, sone 33 (69%) returned sufflelent data by the second week of April, enabling us to lnclude them ln **the** study. Tho final numbers and percentage breakdowns by schooltype, by school, by **class** and by pupil can be found in Table 2.2. The final sample consisted of 5,293 Form II pupils ln 138 classes Ln 33 schools. Table 2.3 lllustrates the data that are mlisLng from the orlglnal sample, and from the flnal **sample,** by numbers of puplls and by Pubject. Pupils were ellmlnated from the sample if either of the two grades for each subject was missing. Table 2.4 reports the means and the standard devLations of all the computable variables ln the study, by schooltype. Both the _ZJC_ and the Grade 7 examlnatLon are graded on a scale of 1 to 9, 1-6, beLng passLng grades wLth 1 the top grade, and 7-9 being failLng grades. In order for the results of the regression analysis to be Lnterpreted eaLly, these grades were transposed, so that 9 is now the hLghest grade and 1 the lowest.

13
TABLE 2.2

**DescrLption** of rinal Sample: No. of Schools by Schooltype, No. of Classes, No. of Pupils (%)

|No.<br>SC9hol1<br>_(%1_|No.Classes 1%|No. PURil i%)|
|---|---|---|
|Former Op A<br>3(9)|22(16)|735(14)|
|Former Gp B (urb.)<br>4(12)|48(35)|2008(38)|
|Former Op B (rur.)<br>3(9)|15(11)|537(10)|
|High Fee-paying<br>2(6)|6(4)|155(3)|
|Mission<br>6(18)|19(14)|801(15)|
|District Council<br>15(46)|28(20)|1057(20)|
|TOTAL<br>33(100)|138(100)|5293(100)|

TABLE 2.3 MiLeing Data from Original Sample: No. Schools by Schooltype (%); Misosing Data from Final Sample by Subject: No. Pupils

||No. Schools<br>Orig.Sanpl|No. Schools<br>**Final**Sapb|_No.upails_<br>**Fil Sanl.**|No.Pupib<br>**Ebglis**|**No.Puplb**<br>Ma4b|
|---|---|---|---|---|---|
|Former OpA|4|3 (75)|735|584 (79)|611(53)|
|FormerOp B (urb.)|4|4 (100)|2008|1432 (71)|1456 (73)|
|Forner Gp B (rur.)|6|3 (50)|537|433 (81)|431(80)|
|HigherFec-paying|4|2(50)|_155_|134 (86)|133(86)|
|Mision|6|6 (100)|801|735 (92)|733(92)|
|Dierict Council|24|15(63)|1057|829(78)|826 (78)|
|TOTAL|45|33 (69)|5293|4147|4190|

14
TABLE 2.4 Aveage Values of Vauiables by Schooltype (S.D.)

|AHl<br>Scho|<br>ob|**Pormtr**<br>Op A|**Fonner**<br>Gp D(uub)|Fonmer<br>Gp D(ru)|High**Fee**<br>Paying|Minion|District<br>Council|
|---|---|---|---|---|---|---|---|
|ZICE|3.8(1.2)|5.9(2.1)|3.0(1.6)|3.3(1.5)|7.1(2.1)|5.57(1.9)|2.4(1.4)|
|ZJCM|2.S(1.9)|2.7(1.9)|1.8(1.1)|2.1(1.3)|5.0(2.4)|4.5(2.5)|1.7(t.0)|
|0R.7!|6.4(1.7)|7.7(1.2)|5.8(1.4)|6.2(1.4)|8.6(.7)|7.6(1.1)|5.3(l.5)|
|GR7M|7.0(1.8)|7.5(1.6)|6.6(1.7)|7.1(1.9)|8.3(.9)|8.2(1.3)|6.2(1.9)|
|AGE|16.1(1.6)|15.2(.9)|16.2(1.3)|16.S(1.9)|14.6(.6)|15.8(1.1)|16.9(2.3)|
|ITL|2.2(1.4)|1.S(.7)|1.9(1.6)|2.5(1.1)|1.0(0)|1.6(.5)|3.6(.8)|
|MQUAL|2.6(1.5)|1.1(1.6)|2.6(1.7)|1.0(1.0)|1.4(.5)|2.8(1.2)|3.3(1.1)|
|EXPR|6.9(t9.5)|3.3(8.7)|11.5(30)|2.3(2.0)|2.2(.8)|4.9(3.2)|1.9(1.0)|
|MEXPU|3.9(13.0)|1.7(4.3)|5.1(20)|1.9(2.3)|3.6(2.2)|1.9(1.3)|2.0(1.3)|
|CISIZE|33.2(13.5)|34.6(4.6)|29.4(17)|35.8(2.5)|27.0(5.3)|42.9(4.5)|39.3(7.2)|
|SchoollIe<br>TCHQUAL||||||||
|1|21.9(23)|53.6(11.4)|9.3(3.4)|7.4(4.7)|64.8(1.5)|50.2(14.7)|33(6.5)|
|2|38.3(16.7)|34.7(12.0)|39.8(12.4)|59.0(11.6)|29.8(5.9)|_3S.0(10.6)_|31.1(23.1)|
|3|5.9(9.8)|0.9(1.9)|9.6(13.9)|6.8a.7)|o(0)|73(5.0)|1.7(5.1)|
|4|33.5(26.6)|10.8(1.9)|41.3(16.2)|22.1(17.4)|5.4(4.4)|7.6(7.7)|63.8(27.5)|
|TCHEXPU|3.5C2.5)|7.1(2.5)|2.0(0)|3.8(1.3)|5.2(1.0)|5.3(3.2)|2.1(.8)|
|TCIITHJS|2.6(1.6)|3.7(1.0)|1.7(.4)|2.7(.S)|5.2(1.0)|4.4A.4)|1.7(.l)|
|ETEXT|.53(.3)|.79(.2)|.38(.2)|.42(.1)|1.0(0)|.S4W.23)|.4(.1)|
|MTKXT|53(.3)|.79(.2)|.38(2)|.42(.1)|1.0(0)|.84(.23)|.4(-1)|
|SIZJ|1029(558)|1296(407)|1503(319)|995(424)|600(125)|644(178)|314(86)|
|TlR|26.4(3.9)|26.6(3.3)|28.5(1.5)|21.4(4.4)|14(2.5)|26.6(3.3)|26.3(1.8)|
|PEtCTAP|96.1(12.9)|87.3(4.0)|100(0)|100(0)|26.6(4.4)|100(0)|100(0)|

15
The **average** ZJC English grade across all **the** schooltypes war 3.8, wlth the lwoet **average,** 2.4 reported for dLstrict councll schools and the hlghest, 7.1. for high foe-paying schools. For ZJC Mathematics, the average grade. were much lower, 2.5 for all schooltyras, and a low of 1.8 for the former Group S urban schools and 5.0 for hlgh fee-paying schools. The grades on the Grade 7 examlnatlon were much hlgher across the board, the averages for Englioh and Mathematics boLng 6.4 and 7.0, respectLvely, wlth the lowest averages reported for the dLstrLct council schools ln both subjects, 5.3 and 6.2, respectively, and the hLghest reported ln the hlgh fee-payLng schools, 8.6 and 8.3, respectively. The average age of the pupils in the sample was 16.1. The lowest, 14.6, was found, not surprisingly, in the high foe-paying schools, **where** the chlIdrens education is least likely to have been interrupted, and the highest, 16.9, was found, in the dLitrict council schools, where the opposite would be the case.

The teacher qualLficatLon codes, used for the EQUAL and MQUAL varlables, denote the qualifieatLon bands of the Form 1I Engllsh and Math teachers, but the same codlng also applies to the school level varLables, TCHQUAL1-4, boeng the percentage of teachers in the whole school ln the four bands. Bands 1 and 2 comprise fully traLned teachers, band 1 belng certifLcated and uncertLfLcated graduates, and teachers holdlng unrecognLzed degrees, and band 2 beLng 5/4 years of teacher traLnLng or '0' level plus 3/2 years teacher traLning. Bands 3 and 4 comprise underqualLfLed and untrained staff, namely band 3 beLng the old teacher tralnlng qualLfLcatLons of J.C. plus 2/3 years teacher tralnlng and Standard 6 plus 2 years teacher traLnLng, and band 4 comprLisng either untrained teachers or teacher traLne-es At the class levol, 16
the actual Form II English and Math teachers averaged on the trained side across all the schooltypes, but ranged from averages of all fully qualified at the hlgh fee-paying and former Group B (urban) schools to the underqualified and untrained side of the scale for district council schools in both subjeuts. A contrast is found for the years of teaching experience in both subjects. The averages **across** all schooltypes were 6.9 and 3.9 for English and Mathematics respectively, but ranging from less than two years for district council and former Group A urban schools to 11.5 and 5.1 **years** of experience **in the** former Group B urban schools for both subjects, respectively. Class **sizes were smllest** in the high fee-payLng schoolq, **averaging** 27 pupils per class, whereas **the** largest average class **sizes were** found **in** the **mission** schools, with 42.9 being the average for that schooltype.

Grouping the teacher qualLfication bands 1 and 2 together to obtain **the** percentage of trained teachers results Ln an average of 60.2% trained teachers across all the schooltypes, but wlth a range from 34.4% in the district council schools to 94.6% in the high fee-paying schools. Average years of teaching experience is highest in the former Group A schools, at 7.1 **years,** and the lowest, at the former Group 8 urban schools, at 2.0 years, with an average of 3.5 years across all schooltypes. The number of years teachers have **spent at the** particular school in question ranged from 1.7 years in both the former Group B urban and district council schools to 5.2 years in the hlgh fee-paying schools, **the** average across all schooltypes being 2.6 years. The variables BTEXT and MTEXT repres6nt the availability of textbooks ln the two subjects as judged by the head teachers. If each child has her/his own textbook, the variable's value **is** 1.0, if **shared between** two pupils, then 0.50 17
> and **so** forth. The range for the averages of these variables was from .38 for

> both subjsct5 at former Group B urban school to 1.0 at the high fee-paying schools, witR an average for all schooltypes of 0.53. The smallest schools on average were the district council schools, averaging 314 pupil. in sLze, and the largest, not surprliingly the former Group B urban schools, averaglng 1503

> pupils in **size.** Teacher pupll ratLos ranged from 14 ln the hlgh fee-paying schools to 28.5 in the former Group B urban schools, with an average across all the schooltypes of 26.4. The percentage Africans compriLsing the student bodies of the different schooltypes ranged from 26.6% in the high fee-paying schools to 100% or a predominant majority Ln most of the other schooltypes. A list of schooltype categories can be found in Appendlx 2.

# 111. THE PROGRESSION TOWA 'FINAL' MODELS FOR ZJC ENGLISH AND MATHEMATICS ACHIVMENT

Plugging ln varLables into a sophisticated regreesion model can assume a life apart from realLty. Endless testLng of different fixed and random parameters can easLly remove one from the subject under study. The aim of thLs sectlon, however, will be to lnterpret the meaning behind a web of complLcated statistics and hopefully to brlng to life aifferent explanatLons of Mathematics and English achievement at Form II ln Zimbabwe's secondary schools in **1989.** Not only L one's subject matter in the social sciences of the real world, but so are the methods which one uses for analysis. This study is certainly no exceptlon, and whilst multilevel models are in the process of **supersedLng** ordinary least squares (OLS) regression analysLs when applied to 18
nested, hierarchLcal data such as aro found ln educatLonal systems, the programme faLliLtating such modelling have not gone through the years of testing that **single** level programes have undergone.

A prefatory caveat ia needed to axplain why some analysts were carried out in thl study and not others. Numerous problems were encountered in usLng ML3, **the** multilovel regression programme. The size of the data **met** and the numbers of variables collected, thoagh in fact, qult. limLted, as dLicussed above, were at tLmes too great for the capacity of the programme. Certain combinatLons of varlables overran the workspace ceortaln deletLons of miLsing data seemingly occupLed more, not less workspace, frustrating the testing of potentLal models; and reading data Lnto the workspace was not always a straLghtforward qxercise. Nonetheless, lt was still thought preferable to continue usLng hL3, aven with its remaining 'bugs', for a picture is paLnted which is far closer to reality than any single level model can paint.i in single level regression models, there is a single residual term illustrating the varLance whlch has not been explained. In multi)evel models, residual terms at each level are specified. For example, ln the three-levol models descrtebd below, three rooldual terms reflect the differing effects of the explanatory varlables at the pupil, the class and the school levels, Zn other **words, instead** of beLng 'throw-away', error terms, the residual V in faLrness, many of these bugs' have **been** eliminated slnce the first writLng of this report, and in addLtLon, an extended memory versLon of ML3 ls up and running, facilitating **the** LnclusLon of **more** variables. However, **the** limitod number of schools Ln the study still constralns the number of school level varLiables which can be fit successfully ln any one model.

19
parameters **are a separate** focal point of **the** analysis. (See Goldstain 1987 for tochnical baokground.) In each of the models, together wlth the slgnlfLcance of the flxed parameters whlch **ar-** tosted, we are Lnterested ln the roductions made to the overall variance by tho Lncluslon of dLfferent explanatory varLablis, as well **as the** changes ln the dLitribution and size of the components of thli overall variance at each of the three levels.

## BACKGROUND MODELSs ADJUSTING FOR INTAKE

# **Modl Al** Thu **Variance** gq=onentso rad 7 scores

Tables 3.1 and 3.2 report the results of the initlal **models** for Mathematics and EnglIsh respectively. Model A, shows the variation in Grade 7 scoroe between schools and **between** students. Model A L a two level **model because we** do not know the classes Ln which the 1989 Z33 candidates **sat** theLr Grade 7 examinations. For Mathematics, 40% of the variatLon Ln achievement is betwen schoolsi for English, the figure is 53%. The average **Grade** 7 marks for both subjects across all schooltypes are 6.4 for English and 7.0 for Mathmatics.

_godl 1:1 _The_ yriancek C=2nantg of 23C gcores Model 1 **is** roally **the** starting point of thL analyLsi. It illustrates **the** total variance to **be** explained Ln subo-quent models and how it _li_ partitLoned between the throe different levels. For Mathematics achievement, 47% of the varLance Lo between schools, 14% between **claoses** and 39% between 20
students. ror Englioh achievemeet, **the** figures are comparable, wLth 47% a1lo between schools, 18% between **classes and** 34% between students.

# Model 2a Controlling for Qgade 7 Scargs

Model 2 lllustrates **the** effect of controlling for the Grado 7 **scores** ln Engllsh and MathematLcs of each Lndividual Form II pupll. In Mathematics onethlrd of **the** overall variance **in** accounted for by the prLor **Grade** 7 achievement, and one-half ln the **case** of Englsh. The between school variances in both **subjects** are signLfLcantly reduced by the inclusion of this varLable, one-thlrd in the case of Mathematics and two-thirds ln the **case** of Engllih. This means that **some** of the observed between school differences have nothing to do wlth differences in :chool or classroom practLceo but rathaL the sealectLon lnto the schools on the basis of the vtUdents' prlor achLivement. in a single level model, because it **is** not possLble to distinguish between these dLfferent components of variance, **thes** celectLon dlfferenceo **are** confounded wlth 'roal' school dLfferences. The class level variances in Engllsh were reduced by two-thLrds wlth the inclusion of the Grade 7 score, but only by 12% for MathematLcs, indLcating that **classes** are more differentiated according to Grada 7 achievement levels ln English than they are for Mathematics.

The relationship between prior achievement in English and subsequent achiv eme nt at rorx X1 **is** stronger than for MathenatLcs, judgLng by the coeffLcLents of the fixed **parameters.** Vor **every** poLnt Lncreaso in Grade 7 mark, there is more ttan a thlrd of a poLnt increase at ZJC for Mathematics, but more than two-thirds for English.

21
Mode 2amallwino the Ga. **Levl** (for Esalishi 7 Intake Variable to be Random at tho School Whereas Model 2 assumes that tho relationship between Grade 7 achievemont and Form I achiovement is the **sme** in all schools, Model 2a allows this relationship to vary, so that some schools may produce better results than others, gLven partlcular levels of student achieveont on lntake. This model proved to be significant only in the case of English, and not Mathematics, for which a constant slope - or constant relationship - **was evident.** For English, however, taken at the average Grade 7 score of 6.4085, this model resulted in a reduction in the between school variance of 5S, to 28% of **the overall** variance. The total variance itself was reduced by 10%, when compared with Model 2 /

# _Modl_ **3_t** schoo_nu Differentiation

Dummy variables were created for the dlfferent schooltypes under investigation in the study, the Group A schools s*rving as **the reference** point. Tho inclusion of those 5 schooltype variables elirilnated the between school variance (it **was** no longek statistically significant at **the** 5% confidence level) and wubstantiall; reduced the total variance. For English, VA further model **was tested** which allowed the Grade 7 **score** to be random at **the** class _level_ **a** well. This means that different classes have dlfferent relatLonships between Lntake and outcome. Taken at the average GR73, the total variance *xplained was less than for Model 2a, but variance remained to be explained at all thres levels after the inclusion of th@es random paramoters. **This** model produced the following statistics fixed coefficLents: CONS - .7024 (.12) GR7U .6736 (.04)1 random **parameters:** school **levels** CONS .05875 (.09) GR7I .0244S (.01) covarianco -.03657 (.028); **class level:** **_CONS_** .408 (.17) GR7U .03067 (.007) covariance -.1104 (.04)i pupil levels 1.349 (.03) 22
compared with Model 2, the total variance was reduced by 19%, and for Mathematics, by 26%. Although taken separately, the fixed coofficient. for these schooltype variables are not significant, the chi-square statistic for their simultaneous contrast showed that the schooltypes were significantly different from zero, for both subjects. For English and Mathematice, the greatest contrast is between the hlgh fee-paylng and mLioLon schools, on the one hand, and the Group B and dLtriet couneil schools, on the other. ThL was also the case Ln 1985 for English, though theme schooltype differences were not signlfLcant Ln the came of MathematLcs in the previous study)' The 6 schooltypes whlch formed the basis of the stratlfled sampling were collapsed into three for further lnvestigatLont Government (the reference poLnt), high fee-paying and mLssLon, and dLstrict council schools. Although the dLfferentiatLon _whlch_ does exlit - as can be seen from Model 3 - between the Group A and Group B schools L lost by such a regrouping, the contrasts **are** otherwise morc. striking between the remaLning classificatLons. The fixed coeffieLonts for thls model for EnglLsh aret CONS -.15 (.23), GR7U .6345 (.02), HICHNISS 1.059 (.33), DC -.7904 (.29); and the random parameters are: school level CONS .3804 (.13), class level CONS .3543 (.06) and pupil level CONS 1.393 (.03).

The fixed coefficients for this model for Mathematics arat CONS -.4986 (.24), GR7M .384 (.01), HIGCHISS 1.871 (.34), DC -.201 (.30); and the random parameters aret school level CONS .4299 (.14), class level CONS .3022 (.OS), and pupil level CONS 1.377 (.03).

23
TABLE 3.1 Fund an Random Parmstar (Sandad Error) of Initi Variane Confonui Modeb for Madwnuatic (Pupi Lovel Variables)

||A|1|2|3|4|5|
|---|---|---|---|---|---|---|
|Rupo<br>Vauabs|ORGM|ZICM|ZiCM|ZjCM|ZJCM|ZICM|
|Find<br>cwS|6.98<br>(.017)|2.517<br>(.2529)|-.1297<br>(.216)|-.109<br>(1.293)|2.2S9<br>(.2508)|-.2058<br>(.2359)|
|GRM|||.38S2<br>(.0128)|.384<br>(.0406)||.3738<br>(.0128)|
|SEX"|||||.4383<br>(.0449)|.2891<br>(.0415)|
|GP3URD||||-.579<br>(2.25)|||
|aPBRtu||||-.51<br>(2.2s)|||
|HKIGII5||||1.571<br>(3.25)|||
|MISION||||1.4S4<br>(1.443)|||
|DC||||-.59<br>(1.358)|||
|Rano|||||||
|Scbool(%)|1.329 (40%)<br>.1699)|1.873 (47%)<br>(.5196)|1.146 (40)<br>(.3153)|.406<br>(.512)|1.801(45%)<br>(.62)|1.141 (41%)<br>(3149)|
|Class(S)|.|.5774 (14%)<br>(.0858)|.3045 (11%)<br>(.0419)|.302<br>(.157)|.6201(15%)<br>(.M)|.3148(110)<br>(.0|
|Pupil()|2.023 (0%)<br>(.0439)|1.557 (390)<br>(.0318)|1.377 (49%)<br>(.0306)|1.377<br>(.0899)|1.607 (40%)<br>(.0357)|.36(48%)<br>(.0302)|
|TOTAL|_33S2_|4.0074|2.827S|2.085|4.0282|2.8158|
|No._Pul_<br>No. Cl_<br>No. Scboo&|438S<br>-<br>33|4933<br>139<br>33|4189<br>138<br>33|4189<br>138<br>33|4189<br>138<br>33|4189<br>138<br>33|

> _Not:_ Fam ge may no6 mm to 100 due to muwnd.

24
TABLE 3.2 Fixed end Randomn Parameers (Staudard Erron) of bhtia VaIance Componers Modeb for Englidi (PUpU Level Variabl.)

|Modiebls<br>|L|IA|2a||3|4|
|---|---|---|---|---|---|---|
|Re"~na<br>Variable|OR7B|ZJCE|ZJCE|ZJCE|ZCR|ZICR|
|CONS<br>|6.384<br>(.1066)|3.709<br>(.2983)|-.2395<br>(.2018)|-.4588<br>(.1441)|1.009<br>(1.08)|3.675<br>(.2998)|
|GRVB<br>|||.6375<br>(.0161)|.6483<br>(.0357)|.631<br>(.0634)||
|SEX(M)||||||.06429+<br>(.0442)|
|OPE3JRB|||||-1.61<br>(1.168)||
|PUBRUR|||||.1.605<br>(1.25)||
|IUOHFEE|||||.44<br>(1.386)||
|MISSION|||||.238<br>(1.169)||
|DC|||||-1.93<br>(1.114)||
|School(%XCONS)<br>|1.491(53%)<br>(.1867)|2.547(47%)<br>(.7224)|.8568(33%)<br>(.2483)|.1971<br>(.1466)|.139+<br>(.257)|2d539(47%)<br>(.0725|
|GVR7||||.03119<br>(.0102)|||
|COVAR.<br>||||-.06352<br>(.03512)|||
|Ca.(||.9816(18%)<br>(.1426)|.3611(14%)<br>(.0568)|.316<br>(.03)|362<br>(.171)|.913(18)<br>(.1426)|
|Pupl (%)|1.304(47%)<br>(.0283)|1.85(34%)<br>(.0379)|1.3453%)<br>(.0311)|1.369<br>(0.307)|1.393<br>(.090|1.80<br>(.037|
|TOTAL|2.795|5.3786|2.6119|2.348V|1.89|_S.3891_|
|No. Pupi<br>_No. Cm_<br>_No._Scoob|4380<br>.<br>33|4900<br>139<br>33|4147<br>138<br>33|4147<br>133<br>33|4147<br>138<br>33|489<br>139<br>33|

_+o._ js-wo't _i big_X_ **_Ub i_** _ 25
gols 4 and St Conr na for the Box gf the Pupils Model 4 examlnes the effect of sex on Mathematics and English achivement without lncludlng Grade 7 achievement. The model tests for boys, wlth girls as the reference point. The sex of the pupils was found to be a significant variable for Mathematics, accounting for not quite an extra half-polnt for boys. It **was** not found to be signifLcant in the case of English achievement.

Model 5 **is** constructed for Mathematics, alone, given the significance of sex **as** an explanatory variable. Both Grade 7 and sex are included. Whilst the cooffLeLant for **sex** is reduced, lt remains slgnificant. The inclusLon of this variable has a marginal effect in explalning the total variance and Lts different components, comparing the results for Model 5 with Model 2 for Mathematics.

# WHICH VARXABLES TO CHOOSE?

Testina glans and S **_Col_** Level V-arables for Possible inglui Due to aome of the practical problems experLenced in using ML3, and the time constraLnts of this report, lt was not possible to test all the variables using a multilevel **model.** Single level regressions were employed as a means of weeding out varLiables with less explanatory power than others. Although the standard errors are underestimated in the single level regressLons, it was thought that s*eo variables might reasonably be eliminated by puttlng the information **gained** from such an exercise together with the results of a similar study conducted in 1985 (Rlddell 1988).

26
class size, the ago of the puplIs,<sup>**the**</sup> average yoars of teachlng experience of **all the** teachers and the teacher pupil ratio<sup>were excluded from</sup> the analyaLs _on_ the above grounds. All of these variables were found to be statistLcally Lnsignificant, with the exception<sup>of class slze for Mathematics</sup> achLovement, but wlth little explanatory value (below<sup>10% R),</sup> and with the exception of the age of the pupils for<sup>the regression on English achievement.</sup> Even **this** latter variable, however, had<sup>an anomalous, if understandable</sup> coefficient. As was shown in the prevlous<sup>section, older puplIs on average</sup> were found ln the less well-provided schools,<sup>e.g. the dLstrLct council</sup> schools. A reasonable explanation for<sup>the negatLve coefficient found on all</sup> the age parameters is that one is<sup>really accountlng foraschooltype</sup> difference, describing the school, rather<sup>than the relationshLp of</sup><sup>**age**to</sup> achievement in a more straLghtforward sense.W Due to multicollnearity, it **was** not possLble to fit many **single** varLables at each level, and lndox variables had to **be** constructed for the three levels under study. For example,<sup>once thepercentage</sup> of trained teachers _in_ a school was found to be a significant<sup>variable when testod wlth</sup> ML3, other variables such as the average year.<sup>of teachLng experlence at a</sup> school were unlikely to be fitted successfully.<sup>Judgements were</sup><sup>**made,**</sup> clearly, from **the** standpoLnt of educatlonal signlfLcance, if<sup>not solely from</sup> the standpoint of statintLcal slgnlfLcance, regarding<sup>the inclusLon of</sup> different varLables. The variables schoolize,<sup>the percentage of Afrlcans in</sup> Wzndeed, **the** coeffiieLnt of age ln the multilevel modol whlch<sup>regresseda</sup> constant, **the Grade** 7 intake score and age<sup>on ZJCE, was -.135 (.06),</sup> explaLnLng a further 88 of the overall varLance,<sup>when compared wlth Model 2</sup> but eliminating the hetween school variance.

27
the student body, and whether the school was day or boarding were not specLfLcally tested, being relatively non-manipulable variables.

BesLdes the models reported in Tables 3.3, 3.4 and 3.5 below, the following variables were tested using ML3, with no significant results: EQUAL, MQUAL, EXPER, MEXPER. It is interesting that school, rather than class level variables, ostensibly reporting **the same** type of informatLon, were found to be significant. For Lnstance, whether or not a particular class is taught by a trained and experienced teacher was not found to be significant, whereas the percentage of trained teachers overall in a school, was found to **be** a significant varLable. Except for the fLnal throe English models, the school level variance in both subjects is greator than the class **level** varianeo, and it may **be** that as was the **case** with the 1985 study, the class level variance is more intractable in the facea of **these** more readLly obtainable data and that classroom observations alone will provlde the clue to explaining class **level** differences in achievement.

A sex and Grade 7 interaction term was tested for Mathematics achievement. Whilst statistically significant, the explanatory power of the model was hardly improved and so further modelling with this variable was discontinued. A variable comprising the top three grades in the Grade 7 examlnation **was** composed for both subjects and found to be significant in the case of gnglish but not of Mathematics. The coefficient of this variable 'HIGH' for Engllsh was 1.47 (.05). Thli means that a stronger relationship exists botween high achievement in Grade 7 English and ZJC results than for 28
**the** whole **range** of **achievement, whose flxed** coeffiLient **was** .64. It **was** not possible to combine this variable with other variables found to **be** of Lmportance la **subsequent** analyses, wLthout the loss of statistical significance, however. The variable 'HIGH' for Engllih was tested for a random effect at the school level, but none wau found, meaning that the same relationship betweon high prior ability in English and subsequent Form rI achievement was prevalent across different schools.

Although as reportad above, a random effect of Grade 7 for further English achlevement was found, Lt was not **possible** to bulld upon this model without the loss of further explanatory power of other significant varlables as dLscussod below. Nor was it possible to buLld substantlally on the schooltype differences uncovered ln Model _3Y_ Given the mall number of schools Ln the sample (33), there were only so many explanatory varLables whlch one could pack Lnto a model, and as the aim was to explain as much of the varLance an posoible for the purpose of ranklng the secondary schools, one had to be diLcornLag in the choice of variables for any potontlal 'final' model. Nonetheless, the LndivLdual variables of slgnifLcance ln intermediate **models** provide a background tapestry.

# TEXTBOOKS, TRAINED TEACHERS AND TEACHER TURNOVER

Textbooks, trained teachers, and the continulty of employment at any partlcular school were found to be three key varlables in this study. All three contributed significantly to explaLning between school dLfferonces ln

**F** or further Lnvestlgation of contextual effects on achlevement, see Lockheed, et. al., 1991.

29
academic achievement in English and Mathematics. Models 6, 7 and 8, **each** of which regresses ZJC scores in the two subjects on these variables are presented in Tables 3.3 and 3.4 below.

# Model 6: The Availability of Textbooks

Head teachers ware asked to report for their Form I1 English and Mathematics classes, how many students had to _share_ a textbook. Whlle such data may not be an reliable as an actual textbook count, undoubtedly, the head teachers have a good grasp of the scarceness of such resources and their replies have legitimacy. For Mathematics, we shall be comparing Model 5 with Model 6 because it was felt that both sox and Grade 7 scores should be included in the basic background model for that subject. For English, in whlch the regression on sex was not significant, the comparison is botween Models 2 and 6.

The inclusion of the textbook variable for Mathematics **has** the effect of reducing the total variance by 17% but the school level variance in particular, by 59%, the school level variance comprising some 29% of the total variance by this stage. The importance of the textbook coefficient can be loosely interpreted as follows: if a student has a textbook to her/himself, rather than having to share it, it could result In an increment in her/his achievement **at** ZJC by as much as two and a half grade points. Even if s/he had to share it with only one other student, the increment could be by as much as a full grade point.

30
TABLE 3.3

||Fi|xed**and**Random Paa<br>Further Model fo|menrta (Standard Em<br>r ZIC Mathematics|n) of||
|---|---|---|---|---|---|
|Models|6|7|8|9|10|
|_ESW_||||||
|CONS|1.674<br>(.3729)|-1.317<br>(.3643)|-1.121<br>(.3211)|-1.437<br>(.366)|-1.683<br>(.3804)|
|OR7M|.3731<br>(.0128)|.3734<br>(.0129)|.3734<br>(.0128)|.3846<br>(.0128)|.3844<br>(.0128)|
|SIXM|.2891<br>(.0415)|.2898<br>(.0415)|.291S<br>(.0415)|||
|MTECI'|2.599<br>(.5827)||||2.053<br>(.7t39)|
|STRDTCH||.01947<br>(.0055)||.01304<br>(.0061)|.0069<br>(.0069)|
|TCWHJM|||.3344<br>(.0953)|.206<br>(.1051)||
|Rano||||||
|School (1)|.6771 (29%)<br>(.1987)|.7927 (32%)<br>(.228)|.8013 (32%)<br>(.2305)|.71965 30%)<br>(.2091)|.6623 (28%)<br>(.1942)|
|Clas(S)|.3113 (13%)<br>(.0497)|.3138 (13%)<br>(.0504)|.3137 (13%)<br>(.05)|.3020 (13%)<br>(.0486)|.3014 (13%)<br>(.0484)|
|Pupi (%)|1.36(58%)<br>(.0302)|1.36 (55%)<br>(.0302)|1.36 (55%)<br>(.0302)|1.377(57%)<br>(.0306)|1.377 (59%)<br>(.0306)|
|TOTAL|2.3484|2.4665|2.475|2.3986|2.3407|
|No. Pupil<br>No. Clat<br>No. Schools|4189<br>138<br>33|4189<br>138<br>33|4189<br>138<br>33|4190<br>138<br>33|4190<br>138<br>33|

_Not uisMicant_ 31
TABLE 3.4 Fixed sod Random Pernmone (Stundard EStom) of Fufhut Modeb (o ZIC Engib

|Mo"el|6|I|8|9|10|
|---|---|---|---|---|---|
|CONS|-1.617<br>(.3392)|-1.542<br>(.2841)|-1.292<br>(.2515)|-1.813<br>(.2563)|-1.824<br>(.3089)|
|OR78|.6361<br>(.0161)|.6361<br>(.0161)|.6373<br>(.0160)|.6359<br>(.0160)|.6356<br>(.0161)|
|EnrE>:r|2.42<br>(.5228)|||1.165+<br>(.6142)||
|**%TRDTCIH**||.02295<br>(.0042)||.0156<br>(.0042)|.0162<br>(.0054)|
|TCKHlIIS|||.3594<br>(.0727)|.2551<br>(.0701)||
|School(%)|.4736 (21%)<br>(.1516)|.3781 (18%)<br>(1284)|.3968(13%)<br>(1324)|.2351 (12%)<br>(.0914)|.3331 (16%)<br>(.1168)|
|Cs.. (9%)|.3575 (16%)<br>(.0558)|.3627 (17%)<br>(.0567)|.358(17%)<br>(.0559)|.36 (18%)<br>(.0561)|.361 (17%)<br>(.0564)|
|Pupil (%)|1.393 (63%)<br>(.0311)|1.393 (65%)<br>(.0311)|1.393 (65%)<br>(.0311)|1.393 (70%)<br>(.0331)|1.393 (67%)<br>(.0311)|
|TOTAL|2.2241|2.1338|2.1478|1.9881|2.072|
|No. Puwbs<br>No. Cbuse<br>No.**Sc9oob**|4147<br>138<br>33|4147<br>135<br>33|4147<br>138<br>33|4147<br>138<br>33|4147<br>138<br>33|

_+Not_ sigficag 32
SLmilar results hold for English achievement as well, the coefficients being clos, in value, and the effect of the inclusion of this explanatory variable on the total variance also being similar, a reduction of aome 15%, the reduction at the school level being 45%. For English, the school levol variance of thls model is 21% of the total variance.

# Mgodl 7: The Percentace of Trained Teachers in Secondary Schtool.

The percontage of trained teachers in the sampled schools was obtained from the schools' statistical returns, combining **the** first two teaching bands TCHQUAL1 and TCHQUAL2 which comprise the trained teacher segment, in order to get an overall percentage figure. For both subj-cts, **the** inclusion of this variable was signiflcant, but it had a larger impact on the English model than the Mathematics one. The reduction in the total variance for Mathematics **was** 13%, **the** impact on the school level variance being a roduction by nearly a third; for Engllih **the** reduction ln tho total variance from **the** inclusion of this variable was also 18%, but on tho school level variance, a reduction of more than half the previous variance. The coefficients of the tlxed parameters for the two subjects ware similar, implying, again, to be taken loosely, about an increase of 2 grade points if the teaching force is completely trained. As can bo seen from Tables 3.3 and 3.4, the variance components for Models 6, 7 and 8 are similar within each subject, the school level variance for Mathematics ranging from 29-32% of the total variance across **these three** models and for English, from 18-21% across these three **models.** 33
# Modal 82 **Teacher** Turnovar in segondary Sbhool.

The varlable for thL **analysis was** also taken from the statistical return and consLsts of **the** years ea-h teacher has been at the particular ascondary school in which s/he was teaching in 1989. These figures were then averaged for the wholo teaching force in each school. The values of the coefficients of this variable (loosely) imply an increase of about one grado point if the **teachers** at the school have **been** teaching **there** for three or four years, for Mathematics and English, **respectlvely.** The effect of this variable on the components of variance and the total variance for each subject is about the **acme as** for Model 7, a reduction in the school level variance of 30% for Mathematics and 54% for English, and reductions of 12% and 18% _on_ the t-tal varianc¢r, respectively.

THE EFFECT O DIFFERENT COMBINATIONS O THE SCHOOL LEVEL VARIABLES: THE SEARCH FOR THE 'BEST' MODEL Different combinations of the above three variables **were tested** to see which produced a model with **the** groatest explanatory power. The 'final' Model 11 for each subject is reported in Table 3.5. Besides comparing **the** different candidate 'final' models in this section, comparisons need to be made with Model 1 for each subject, for Model 1 illustrates the initial total observed variance and how this is broken down between the three different levels. How much of .his observed variance is explained by the 'final' models can then be analyzed.

34
Modeln **9** 10 **and** L1s For Mathematies, it was not possible to test combinatLons of school level variables whllst keepLng sex an a background variable, so it was removed from the flnal models tested as the explanatory power of the combinations of achool level variables was greater than wlth only ono of theme plus the pupill' sex. Models 9 and 11 were possible candidates for 'final' model. Model 10 unsuccessfully combined MTEXT and %TIRTCHt %TRDTCH lout lts statLstical sLgnLfLcance ln combination wlth the othor variable. Model 11 both explaLoed more of the total variance than Model 9, and the coeffLeLonts of lts flxed parameters lost **less** in combination than was the case for the alternative model. in Model 11 44% of the total variance of Model I has been *xplained, 70% of the botween school dLffeorences and 48% of the b isen **class** differenco, but 88% of the betwen pupil differences remaLn.

For 3ngllsh, also, Models 9 and 11 were **the** possible candldates for 'final' model. Model 10 unsuccossfully comblned TZEXT and %TRDTCH, but wlth the opposite effect to that for Mathematics: ETEXT lost sLgnLfLcance. There wasn't much in the choice between the two remaining models, thelr explanatory power beLng virtually identlcal but perhaps the strength of the coefficLents of the fixed parameters Ln Model 11 beLng sllghtly greater. Model 11 tells us more about Znglish achLevement than the similar model for MathematLcs. Slxtythreo percent of the total variance observed in Model 1 has been explained, 91% of **the** between school dLfferences and 64% _of_ the between class dLfferences, btt 75% of between pupll dLfferences are left unexplained.

35
THS 'FINAL' MODELS FOR ZJC MATHEMATICS AND ENGLISH ACHIEVEMENTs COMPARISON WITH THE RESULTS OF THE 1985 STUDY Models 11 for both subjects are the 'final' models, not in any definitive sense, such as that no other variable combinacLons are possible or, lndeed, that all possLble explanatory variables have been tested. Rather, they are adequate final models glven the purpose of this study, to identlfy some of the most effective and the least effective secondary schools in Zimbabwe, and also that **the models** include signifLcant explanatory variables and indeed go a long way towards explaining school differences.

Comparison with the final models used ln the 1985 study shows that a similar percentage of the total varlatlon ln secondary school achievement **has been** explained by the final equation for Mathematlce achlevement, but in **fact** consLderably more for English achlivement. Whereas 48% of the overall variation ln Mathematics achlevment (at '0' level) was explaLned by the fLnal model ln the 1985 study, 44% of the overall varlatlon ln MathematLcs achLevement (at ZJC) was explained by the final model in the current study. Despito **the** time constraLnts and the limited varLables able to be consLdered for possible inclusLon in the current study, 63% of the total variation in Englsh achievement at ZJC was explaLned, whereas the much mo-e comprehenslve data collectlon carried out in 1985 culminated ln a model for Englih '0' level achlevement which only explained 38% of the total varLation.

36
TABLE 3.5

# Fixed and Random Parameters (Standard 3rrors) of FLnal **Models** for ZJC English and Mathematics

|Model|11|11|
|---|---|---|
|Response Variables|MathematLcs|Englih|
|CONS|-1.909<br>(.3709)|-2.052<br>(.2844)|
|GR7M/3|.3841<br>(.0128)|.6355<br>(.0160)|
|M/ATEXT|2.113<br>(.5728)|1.73<br>(.4264)|
|-TCTHis<br>Ran^d|.2134<br>(.087)|.3067<br>(.0636)|
|School (%)|.5696 (25%)<br>(.171)|.2364 (12%)<br>(.0909)|
|Class (%)|.3001 (14%)<br>(.0482)|.351 (18%)<br>(.0551)|
|Pupil (%)|1.377 (61%)<br>(.0306)|1.393 (70%)<br>(.0311)|
|TOTAL|2.2467|1.9804|
|No. Pupils|4190|4147|
|No. Classes<br>|138<br>|138<br>|
|No. Schools|33|33|

'Not sLgnlflcant 37
Another notable difference between the two studies is that signifieant school level variation ln MathematLcs achiovement was not uncovered ln the 1985 study once Grade 7 lntake ability was controlled for. Yet, in the current study, nearly a third of the between school variance is left unexplaLned even with the LnclusLon of Grade 7 and the two school level variables, textbooks and percentage of traLned teachers. The 1985 result was questionable because of the even smaller number of schools in the final sampla - 29 - so lt is likely that we have uncovered school level dLfferentLation ln Mathematics achlevement which requLres further explanation.

It **_Ls_** also interesting to note that even with tho detaLled student questionnaires administered Ln the 1985 study which produced a varLety of student background factors able to be controlled for in the final model, 75% of the between student differences remaLned unexplained Ln the current study of Engllsh achievement, whlch controlled only for Grade 7 intake, whilst 87% of the between student differences in English achievement remained unexplaLned in the 1985 study. (The pupll background index varLable for English achLevement lncluded coeffLcients for the following variables: sex, ethnic group, father's occupatLon, father's education, and the presence of electricity at home.) (88% of between student differences ln Mathematics achievement remained unexplaLned in the current study, as opposed to 78% in the 1985 study.) The appropriateness of the model for English achievement must surely be reflected in the relatively small percentage of class level variance remainlng, 36%, considerlng the fact that no class level variables were 38
included in **the** current study, but few **were even** tested, due to **the** time constraints on data collection. It **is** unlikely, however, that it would havo **been** possible to include both class and school level variables in the same model, even wlthout the difficulties encountered Ln ML3. _This_ was overwholmingly the case in the 1985 study, not surprLLngly, due to the high correlations betwen factors at the two different levels. For Mathematics, as has boen noted above, the final model explaLned less of the total varLatLon Ln achiovement than was the case for the EnglLsh model, and a higher percentage of class level differences also remaln to be explaLned, 52%.

Flnally, as noted Ln the 1985 study, there seems to be an Lntractable class level varLance, Ln both subjects, at least Ln the face of the explanatory varLables considered ln either study. Comparison of the class level varLances, looking across the models 6-11, uncovers very similar statLitLs, .30 to .31 for Mathematics and .35 to .36 for English. The conclusLon of the 1985 study was that this class level variance would remaLn unexplaLned unless varLablos relating to what actually goes on Lnside the classroom are Lncluded Ln the models.

39
# sV. T5 MOST AND THU LEAST EFFECTXVE SCHOOLS AND CLASSES IN ZJC MATHiMATICS AND ENGLISH

Having **chosen** what **we** feel are the best models describing between school differences ln ZJC Mathematics and English achievemnt, it is now possLble to use the random parameters or residuals estimated for each school to detect outliers and to rank the schools in their 'adjusted' performance.

The scatterplots of the standardized school level residuals against the predicted values of the ZJC grades averaged by school enable one to identify any outliers, or exceptLonal schools, in the final **models** Graph 1 is such a scatterplot for Mathematics, and outside the clustering in the middle range of values for the standardized school level rosiduals, one can see four schools above, say the 3.0 mark, and four schools below, may, the -1.5 mark. Table 4.1 identifies these oight schools and lists their standardized school level residuals as well **as** their actual and predicted grades on the 1989 ZJC Mathematlcs. SimLlarly, for EnglILh, Graph 2 depicts the four positive outliers above, say, the 1.5 mark and the three negative outliers from about the -1.5 mark and below. Once more these schools **are** identlfied in Table 4.1.

40
GRAPH 1 2

# STANDARDIZED SCHOOL LEVEL RESIDUALS (Y) PLOTTED AGAINST PREDICTED AVERAGE SCHOOL ZJC RESULTS IN MATHEMATICs (X)

<!-- Start of picture text -->
*.  . . +*++++++ ............4*........... . . ..<br>+  +<br>+  +<br>+  *  +.<br>6.<br>3.  4<br>+  +.<br>+  +.<br>+  *  +<br>+  *  *  +<br>3.<br>+  44<br>+  *  *  +<br>+  *  *  +<br>+  ***  *  +<br>0.*  *  **  *  *<br>+~~~  2 *  *  +<br>v  ***  +<br>+  *  *  4<br>+3  *  +<br>*++++++*++.  ++++*++++.**+++++++++.******++++++*+**+*.+++++****<br>0.4  1.2  2.0  2.8  3.6  4.4<br><!-- End of picture text -->

41
GRAPH 2

# STANDARDIZED SCHOOL LEVEL RESIDUALS (Y) PLOTTED AGAINST PREDICTED AVERAGE SCHOOL ZJC RESULTS IN ENGLISH (X)

<!-- Start of picture text --> .44.4..44......4..+.4.4..4..4..4..4.4.....4..4..4.4...4.4..4..4.4...+..4..+.4....<br>+. +<br>4. * +<br>4.+<br>2.5. *<br>+ * +<br>1.0. *<br>4. * * 4.<br>+. *2 +.<br>4. ** * +.<br>+. * ** 4.<br>-0.5. * 2<br>4. aa a * * +.<br>+. * +.<br>-2 0.+ *<br>1.5 2.5 3.5 4.5 5.5 6.5<br><!-- End of picture text --> 42
TABLE 4.1

# Positlve and Negatlve OutIliers StandardLzed School Level Reeiduals, Actual *nd Predictgd Average school Grades by subject

||||MATH<br>||
|---|---|---|---|---|
||School ID|Stand.<br>**Rep**|Actual<br>zWcM|Predicted<br>ZJCM|
|Positive|||||
|Outliers|2|3.7|3.7|2.6|
||1S|4.1|6.4|4.8|
||19<br>|3.8<br>|5.5<br>|4.2<br>|
||21|6.3|6.8|4.4|
|**NegatLve**|||||
|**Outliers**|1|-1.7|**2.7**|**3.2**|
||13<br>|-1.7<br>|2.5<br>|2.9<br>|
||16<br>|-3.0<br>|2.9<br>|4.1<br>|
||22|-2.4|1.9|2.6|
||||ENGLISH||
||<br>|Stand.<br>|Actual<br>|Predicted<br>|
||School<br>ZD|Rm|zJC3|_zJC_|
|Positive|||||
|Outllers|1|2.6|6.4<br>|S.3<br>|
||2|3.3|5.6|4.5|
||15|3.0|8.5|7.2|
||24|1.8|6.2|5.5|
|Negative|||||
|Outliers|16|-2.1|5.2|6.2|
||22|-1.4|3.6|4.1|
||23|-1.6|5.9|6.6|

43
What do **we** know about theze schools that can give **us** any lndlcatlon of why they have performed exceptLonally well or exceptLonally poorly? We shall take **the** positlve outllero for Mathematics and English first. The poitLve outliers **are** drawn from three different schooltypess there are two former Group A schools, one high fee-paying school and three mLssion schools. Schools number 2 and number 15 **were** ldentlfled as positlve outllers for both subjects. **FLve** out of the six schools **are** boarding schools, and four out of the **slx are** sLngle **sex** schools, wlth two of each type. There is a high percentage of trained teachers ln all the **six** schools, between 86 and 92% of the teachers belng traLned. The teacher pupil ratios at the schools range from 12 to 33 pupils per teacher. The teachers at all the schools have on average at least two years of teaching experience and have been at **the** schools in questLon at least two years. Four of the six sehools have sufficLent textbooks for each chlld to have her/hli own, the remaLnlng two schools have thelr students sharlng between two students. Five of the **six** schools **are** at least 90% Afrlcan, one havLng only 23% Afrlcans in the student body. School sizes vary from 528 to 929 _pupilo._ FLnally, class sLzes vary from between 29 pupils to a class up to 40 per class.

Aside from wantlng to Lnvestlgate further the fact that the positive outllers **are** primarily boarding and sLngle sex schools, there is nothlng overwhelmlngly indLcatLve of factors llkely to set these schools apart from the rest Ln the sample, particularly as one of the other two sLgnLfLcant features has already been controlled for in the final model, namely, the ratio of textbooks to students.

44
The negative outliors comprise one former Group A school, in fact, curiously enough, the **same** one which features as a posLtLve outlier for Englhsh, school number one, one former Group S (rural) school, one high feepaylng school and two mLisLon schools. School number 22, one of the miLsLon schools, is a negatLve outlier for both subjects, and schoch number 16, a high fee-paylng school similarly features for both subjects. What characterLzes **these** negatLve outllers? Two out of **the** flvo are boardLng schools; fLve out of **the** slx are cooducatLonal; the percentage of trained teachers ln some of them ia lower than for the posLtive outllers, the range being from 67 to 100% trained. **The** teacher pupll ratloo at the schools range from 17 to 30 pupils per teacher. The teachers at all the schools have on average at least four years of teaching *xperience and have been at the particular schools in questlon for at least three years. There are fewer textbooks at **these** schools than for the posLtive outllers, four out of fLve of the schools have two students sharlng the same textbook between them. The school aLzes vary from between 448 pupils and 1419. All are betweon 90 and 100% African, wLth **the** exception of the hLgh fee-paying school which is 32% African. Finally, class sizes vary from between 18 and 40 puplls per class.

InterestLngly, the set of negatLve outlying schools have more teaching experience on average than the posLtive outliers and their teachers have stayed longer at the particular schools. OtherwLse, notable L the lower percentage of trained teachers and the number of textbooks, the latter, of course, havLng already been controlled for in the final model.

45
**Thre** is not an obvious set of c4 -%cteristics that differentiates eLther **the positive** or the negative outliers fom the remaining school population. Given that those outliers emerge from the analysis with several major controls already being exercised, namely, the previous ability of the Form 11 students, the numbers of textbooks available and the number of years on average that the teaching force has been at the particular school, answers will have to be sought elsewhere as to their performance. It is our hunch that moms of these **answers** can be found in the schools and classrooms themselves.

Finally, two further tables are produced here. Table 4.2 presents the ranking by school level residuals. This ranking represents how much better or worse a particular school has performed after the inclusion of the Lnfluences of all the variables entered into the final models. This ranking by school level residuals can be compared with the ranking by actual and predicted scores, presented in Table 4.3. The ranking by the predicted scores is entirely dependent on the fitted variables and therefore does not represent a ranking based on the full variance but only the proportion which is explained by the model. Although this sort of ranking is an improvement on the ranking of raw, unadjusted mean scores, the ranking based on the residual variances is of greater interest, for it is the ranking Afgt adjusting for the explanatory variables, **L..** a4SWt tho predicted values, and it reflects the total variance.

if one looks only at the top ten schools according to the actual and the school residuals ranking, in Mathematics the top three schools according to the residuals ranking are in fact the top three schools in terms of their 46
actual mean scorep, schools 21, 15 and 19. However, going further down **the** list, other schools which are earmarked as being particularly effective on the school residuals ranking don't even appear on the ranking by actual mean scores, e.g. schools 2, 25, 38, 10, 28 and 40. A similar case provalle for English achievement. Of the top four schools on the residuals rt king, three appear at the top of **the** ranking by actual ZJC English scores. However, schools 25, 10, 27, and 45 don't even appear on the ranking by actual scores, yet it can be assumed that they are more effective than some of the other schools lower down on the resLduals ranking which happen to appear high up on the ranking by actual scores. For instance, school 23 **is** seventh on the ranking by actual ZJC Engllsh scores, yet is next to the bottom on the school residuals ranking. ThLi illustrates how misleading lt is to rank schools by achievement that is not adjusted for differences in intake. Some of the most 'effective' schools wlll never be uncovered.

47
**Table 4.2**

**Ranking of Schols by School Level Residualcs English** and Kathesatics*

|||ENGLISH|||MATH||
|---|---|---|---|---|---|---|
|2nnk|ScheolIR|School<br>-XPL|Residual|_School_<br>12|School<br>_Tv23_|Reuidgal|
|1|15|HF|.87|21|M|1.93|
|2|2|A|.81|1S|HP|1.28|
|3|1|A|.69|19|M|1.16|
|4|24|M|.48|2|A|.91|
|5|21|M|.27|25|DC|.59|
|6|19|M|.21|24|M|.52|
|7|25|DC|.17|38|DC|.45|
|8|10|8(r)|.16|10|8(r)|.33|
|9|27|DC|.16|28|DC|.26|
|10|4S|DC|.11|40|DC|.26|
|11|46|DC|.02|27|DC|.19|
|12|39|DC|.01|39|DC|.14|
|13|36|DC|.00|41|DC|.01|
|14|33|DC|-.06|36|DC|.00|
|15|40|DC|-.07|45|DC|-.02|
|16|48|DC|-.07|48|DC|-.04|
|17|28|DC|-.10|20|M|-.06|
|18|38|DC|-.15|30|DC|-.19|
|19|41|DC|-.15|23|M|-.19|
|20|42|DC|-.16|33|DC|-.26|
|21|13|B(r)|-.19|42|DC|-.26|
|22|14|B(r)|-.20|13|8(r)|-.37|
|23|20|M|-.22|14|B(r)|-.38|
|24|32|DC|-.28|46|DC|-.38|
|25|30|DC|-.35|32|DC|-.43|
|26|22|M|-.37|1|A|-.46|
|27|23|K|-.45|22|_M_|-. 66|
|28|16|HF|-.63|16|HF|-.98|

* The four Group B (urban) schools and one Group A (urban) school are not included in those rankings due to tho limitations of ML3. Schools having >255 students were excluded from the residuals calculations and there was insufficLent time to compose a sub-sample from these schools Ln order to have them included.

48
**Table** 4.3 **Ranking** of **Schools by** Actual **and Predicted ZJC Gradess anglish and Katheaatics (Rankings)**

||||_ EQL|ISH|||MATH|||
|---|---|---|---|---|---|---|---|---|---|
|School<br>In|School<br>zm29L_|Predi<br>_cr_|cted<br>_ror_|Actu<br>Scor|al<br>|Pred<br>Sc|icted<br>ore|Actu<br>EQQL|al<br>r_|
|1)||5.34|(8)|6.41|(3)|3.17|(10)|2.69|(9)|
|2)|CpA|4.51|(10)|5.60|(8)|2.59|(13)|3.68|(7)|
|3)||5.65|(6)|5.99|(6)|3.89|(8)|2.35|(11)|
|S)||3.09|(17)|2.96|(17)|2.17|(14)|1.79|(20)|
|6)|GpB(u)|2.62|j24)|3.30|(13)|1.86|(23)|1.75|(22)|
|7)||3.22|(14)|3.05|(16)|2.00|(16)|1.97|(15)|
|8)||4.0S|(12)|2.89|(19)|3.23|(9)|1.85|(18)|
|10)||2.93|(20)|3.16|_(15)_|1.51|(28)|1.90|(16?|
|13)|GpB(r)|3.97|(13)|3.76|(11)|2.89|(11)|2.51|(10)|
|14)||3.00|(18)|2.72|(22)|1.97|(17)|1.53|(29)|
|15)|HP|7.17|(1)|8.53|(1)|4.78|(1)|6.36|(2)|
|16)||6.17|(4)|5.20|(9)|4.08|(5)|2.92|(8)|
|19)||5.73|(5)|6.05|(5)|4.16|(4)|5.54|(3)|
|20)||5.16|(9)|4.67|;10)|4.06|(6)|4.03|(6)|
|21)|mission|6.23|(3)|6.65|(2)|4.45|(3)|6.75|(1)|
|22)||4.11|(11)|3.58|(12)|2.62|(12)|1.87|(17)|
|23)||6.65|(2)|5.95|(7)|4.48|(2)|4.24|(5)|
|24)||5.49|(7)|6.16|(4)|4.04|(7)|4.64|(4)|
|25)||2.08|(32)|2.39|(27)|1.49|(29)|2.25|(13)|
|27)||2.18|(31)|2.51|(24)|1.33|(32)|1.56|(28)|
|28)||2.97|(19)|2.79|(20)|1.94|(20)|2.29|(12)|
|30).||2.66|(23)|1.73|(32)|2.09|(15)|1.78|(21)|
|32)||2.56|(25)|2.05|(31)|1.92|(21)|1.36|(33)|
|33)||3.09|(16)|2.92|(18)|1.94|(19)|1.61|(26)|
|36)|D.C.|3.12|(15)|3.22|(14)|1.95|(18)|1.98|(14)|
|38)||1.75|(33)|1.36|(33)|0.73|(33)|1.45|(30)|
|39)||2.31|(29)|2.31|(28)|1.55|(27)|<br>1.73|<br>(23)|
|40)||<br>2.91|<br>(21)|<br>2.73|<br>(21)|<br>1.41|<br>(31)|<br>1.82|<br>(19)|
|41)||2.49|(27)|2.21|(30)|1.67|(26)|1.68|(25)|
|42)||2.52|(26)|2.23|(29)|<br>1.92|<br>(22)|<br>1.57|<br>(27)|
|45)||2.25|(30)|2.44|(26)|1.43|(30)|1.39|(31)|
|46)||2.44|(28)|2.45|(25)|1.84|(24)|1.37|(32)|
|48)||2.68|(22)|2.57|(23)|1.75|(25)|1.69|(24)|

49
# V. SUAKhRY AND CONCLUSION

This school survey and multi-level analysis were undertaken as part of a larger joint Ministry of Education and Culture-World Bank study of the education sector in Zimbabwe. The study had two purposes. First, it was designed to contribute to a policy dialogue on how to improve student achievement in secondary schools by identifying factors that explain variation in Form II exam scores. Second, it was designed to identify a sample of schools that were outliers in terms of student achievement on national exams, i.e. they performed better or worse than what would be expected given students prior academic achievement and certain basic school resource inputs such as textbooks and teachersV This guided the selection of a sample of schools for **site visits** to g&ther additional information on high and low performing schools (such as school management and expenditures) that may explain the differences Ln achievement.

The development and implementation of the survey instruments and the analysis of the data were completed within a very short period of time (only three months) which provided information rapidly for the sector study. However, the time constraint limLted the range of data collected; multicollLnearLty among the measured varLables further reduced the number of varLables Lncluded Ln the statistical models to assess thair lnfluence on student achievement. These limitations notwithstandLng, the study produced several noteworthy findings and id.ntLfied areas for further research.

The results of the sample survey showed, first, that students Ln high-fee paylng (trust) schools, former group A (elite government) schools, and mLsLon schools have higher levels of English and Maths achLevement than students Ln government groups B (less well-endowed government) schools and those in district council schools. Second, a substantial share of the variation in student achievement among students was attributable to the school school ls considered an outlier if the standardized residual is two or **sA** more standard devLatLons from the mean of 0.

50
**the** student attended and that failure to control for the selectLon of students into **schools** on the basis of prLor academLc achLevement results in an overestimate of the variatLon in achievement attrlbutable to the school. Third, the **results** of the multLvariate analysas showed that student achievement _li_ higher when schools have a greater avaLlability of textbooks, a larger proport.zon of trained teachers and teachers who have taught at that school for a longer period of time. The availablity of textbooks and the length of teacher tenure at a particular school together accounted for 50 percent and 72 percent of the variation attributable to the school in Math and English, respectively, after accounting for the variation due to the academic intake of the students. This suggests that ra$sing the proportlon of trained teachers and, more importantly, improvLng the provision of textbooks and providing incentives for teachers to remaln in the same schools for a reasonable period of time are promising investment options to boost student achievement.

The analysis of school rankings shows that rankings based on raw results of achievement tests aggregated at whe school level are mieleading. some of the most effectlve schools w$1i never be detected if the prior ac&demic achLevement of students is not taken Lnto account. After controlling statistically for prior academic achievement and for basic resource inputs (textbooks and teacher tenure), some dietrLct council schools are more effectLve than what would be expected given their base level of resources and some high fee and missLon schools are less effective than what would be predicted given thelr resource base. Additionally, the interesting finding that there were more experienced teachers in the set of less effective schools as compared with the more effective schools suggests that newly trained recruLts have much to offer and that longer experience may erode teacher effectiven **es. ThiL** ie a common finding in developed countrLes. Typically the learning curve among teachers is steeper and levels of effort are higher durlng the beginning years of teaching. Research into these and other characteristice of the newly trained teachers in Zimbabwe should be conducted in order to assist policy makers in the development of strategies directed at maintainLng high levels of teacher effectiveness.

51
**The** outlier analysis pointed to the need for further<sup>rLIearch to</sup> identify whlch **aspects** of boardlng and oinglo-sox<sup>education explain the high</sup> **levels** of hlgb academic achievement found<sup>in these schools. Although boarding</sup> schools **are** a high-cost alternative to day<sup>schools and cannot be a standard,</sup> it aseem important to investLgate if boarding schools<sup>have certain</sup> characteristics, other than boardlng, that<sup>can be replicated in non-boarding</sup> schools. For example, it may be that students are assigned<sup>more homework</sup> because they have fewer demands on their tlme<sup>outside school. Boardlng school.</sup> also allocate certain periods of the day for<sup>study sessions which students are</sup> required to attend and durLng which they receive assistance<sup>from a resident</sup> teacher. Perhaps these study sessions contribute<sup>to hlgher levels of</sup> achlevement.

Equally, LnvestlgatLng whlch factors explain<sup>the comparatlve</sup> advantage of boys over girls in Mathematlcs<sup>and</sup><sup>**the**advantage associated wlth</sup> sLngle-sex schools could also ldentlfy strategies<sup>that lmprove school</sup> effectLveness. In particular, observational<sup>studLes would be helpful in</sup> determLning Lf teaching practlces contrLbute<sup>to the dloadvantage of girls in</sup> Maths and of students at coeducational schcols, more generally.

52