metadata: authors: 0: author: Sakellariou,Chris
count: Ghana
docna: 0: docna: Decomposing the increase in TIMSS Scores in
            Ghana : 2003-2007
docty: Policy Research Working Paper
subtopic: Primary Education,Educational Sciences,Educational Institutions & Facilities,Gender and Development,Effective Schools and Teachers,Inequality
teratopic: Education
abstracts: cdata!: This paper attempts to explore certain
            aspects underlying the substantial improvement in 8th grade
            student performance in Ghana on the Trends in International
            Mathematics and Science Study fro
geo_reg_key: 1927320,517191
repnme: repnme: Decomposing the increase in TIMSS Scores in
            Ghana : 2003-2007
display_title: Decomposing the increase in TIMSS
            Scores in Ghana : 2003-2007
listing_relative_url: /research/2012/06/16357931/decomposing-increase-timss-scores-ghana-2003-2007
topicv3: Inequality under Law m1326002 925,Teachers Management m1326119 1523,Primary Education m1326139 480,Education m1327048 470,Gender m1327621 615
publishtoextweb_dt: 2012-06-01T00:00:00Z
historic_topic: Education
no_of_pages: 62
author_keys: 0: author_key: 428386
model_extractions: WPS6084 Policy Research Working Paper 6084 Decomposing the Increase in TIMSS Scores in Ghana 2003–2007 _Chris Sakellariou_ The World Bank Human Development Network Education Unit June 2012
Policy Research Working Paper 6084

# **Abstract**

This paper attempts to explore certain aspects underlying the substantial improvement in 8th grade student performance in Ghana on the Trends in International Mathematics and Science Study from 2003 to 2007. The improvement was largely heterogeneous; in mathematics, performance improved more for students already performing well, while the opposite was the case for science, where students at the bottom of the score distribution experienced a spectacular increase in science scores. Most of the increase in scores for both mathematics and science is explained by overtime changes in coefficients (and a smaller part by improvements in characteristics). Contributors not accounted for (and therefore captured by changes in the constant) dominate the effects of the coefficients. One potentially important piece of information missing from the Ghana data is whether a school is private or public; this could potentially explain part of the overtime improvement. This is because over the short period between the two surveys, there was a large increase in the number of private schools in Ghana (by 36 percent between 2005/6 and 2007/8). Finally, an analysis of the over-time change in the test score gap by location (between large and small communities) revealed that the gap became more heterogeneous, narrowing for worse performing students and widening for better performing students.

This paper is a product of the Education Unit, Human Development Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at acsake@ntu.edu.sg.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Decomposing the Increase in TIMSS Scores in Ghana: 2003-2007**<sup>*****</sup>

## **Chris Sakellariou**<sup>**†**</sup>

JEL classification: I22, J3, J45 Keywords: Quantile regression, Oaxaca decomposition, learning outcomes, Ghana Sector Board: Education

> * I am grateful for helpful comments to Peter Darvas, Kevin Macdonald and Harry Anthony Patrinos. All remaining errors are mine.  Research funded by the Bank-Netherlands Partnership Program.

> † Division of Economics, Humanities and Social Sciences, Nanyang Technological University, Singapore (email: <u>acsake@ntu.edu.sg).</u>
# **1. Introduction**

# _1.1. Background_

African countries are among countries with less than 50% of children achieving literacy, numeracy and life skills mastery (Schaefer 2000). A large proportion of children do not have basic functional skills to read, write and enumerate after year four of their educational experience. The UNESCO (2005) report confirms these findings and shows that too many pupils are leaving school without mastering a minimum set of cognitive and non-cognitive skills.

Ghana participated in the Trends in International Mathematics and Science Study (TIMSS) in 2003 and 2007. In 2003 Ghana’s score for mathematics at 276 was higher only compared to that of South Africa among participating countries and much lower than the international average score for mathematics (467), as well as the score for other African participating countries such as Botswana (366) and Tunisia (410). In 2007, Ghana exhibited substantial improvement in the mathematics score, which increased to 309 (a 12% increase), which is still one of the lowest among the participating countries, ranking 47<sup>th</sup> among the 48 participants and lower than all the African participating countries (South Africa did not participate in 2007). Similar improvements were registered for science. The science score increased from 255 in 2003 to 303 in 2007, a 48 point increase (19%). Ghana is one of the countries in which boys perform significantly better than girls. While the score increased significantly over time for both boys and girls, the gender gap in 2007 stood at 22 points in mathematics and 28 points in science.

Table 1: Selected mathematics and science TIMSS scores

||Mathe|matics|Sci|ence|
|---|---|---|---|---|
|Countries|2003|2007|2003|3007|
|Singapore|605|593|578|567|
|Tunisia|410|420|404|445|
|Botswana|366|364|365|355|
|Ghana|276|309|255|303|
|International Average|467|500|474|500|

Source: Anamuah-Mensah et al. (2008)

# _1.2. Challenges and problems_

Ampiah (2010) outlined the problems and challenges that Ghana is faced in improving quality of education. One challenge relates to trade-offs between rapid expansion of basic education (towards universal access) and quality of education. In particular, if children attending are not enabled to improve learning outcomes, especially in literacy and numeracy, access to education is not meaningful. The authors feel that in the case of Ghana, quality of education seems to have become subordinated to quantity of education as a priority and that in many cases, educational expansion has come at the expense of quality.

The poor quality of teaching affects student learning, as most teachers lack the necessary supervision and feedback on their instructional practices. In particular, supervision is weak and 2
school principals and teachers are present in school when they see fit, resulting in serious problems of lateness and absenteeism and low teacher-pupil contact hours (MOESS, 2008).

As is the case in some other developing countries, the public perception is that private schools have higher educational standards compared to public schools in both urban and rural areas; this perception is reinforced by the better performing private school pupils at various assessments such as the School Education Assessment (SEA), the National Education Assessment (NEA) and the Basic Education Certificate Examination (BECE). In 2007/8, private school enrolment stood at 24% of the total national enrolment, while the number of private schools increased by 36% between 2005/6 and 2007/8 (from 2,990 to 4,068. The main advantage of private schools is that they facilitate better English language skills; however, they also provide more access to extra classes and special tuition to students (Ampiah, 2008; 2010).

Practices relating to language of instruction are also important. In Ghana, there are two languages involved in basic education: the child’s own vernacular as well as English language, which is more widely used in education. From primary 4 onwards, the English language becomes the medium of instruction (as well as a school subject). A relatively recent decision by the Ministry of Education has been to develop quality bilingual literacy instruction in all primary schools. However, implementation challenges remain, as some teachers are not functional in local languages, most classes in urban areas are multilingual and textbooks, while written in English are taught in the local language (Ampiah, 2010). Based on the outcome of TIMSS 2007, the majority of the Ghanaian 8<sup>th</sup> grade students (66%) rarely spoke English at home – if ever – and the home language was found to be associated with lower achievement in both mathematics and science (Anamuah-Mensah, Mereku and Ampiah 2008).

Finally, there are problems resulting from high proportions of untrained teachers in basic education. Low retention and high attrition rates resulted in a discrepancy between trained teachers in basic education and the output of the Teacher Training Colleges in Ghana. This resulted in large numbers of untrained teachers filling teacher vacancies. In 2007, the proportion of untrained teachers in Junior high school stood at 33% (MoESS 2007).

# **2. Methodology**

# _2.1 General_

Decomposition methods are frequently used in an attempt to answer important questions, such as what accounts for pay differences between men and women, what is behind increases in wage inequality in recent times, among others. They have also been applied in a variety of other settings, including the study of gaps in test scores between schools (Krieg and Storer 2006), men and women (Sohn 2008) and countries (McEwan and Marshal 2004; Ammermueller 2007).

In the last 10-15 years there has been an evolution and refinement of techniques used in examining distributional issues, specifically in evaluating wage differentials between sub- groups (and more generally the impacts of various programs) over the entire range of the earnings distribution. These new techniques were first used to analyze gender earnings gaps (for example, 3
Albrecht et. al. 2003) and later to examine changes in wage distributions over time, where the focal point is what contributes to the change in these distributions.

There are two types of decomposition which can be implemented using one of an increasing number of methodological approaches. One is the _aggregate decomposition_ of the outcome variable into its two components – the part due to differences in characteristics (composition component) and that due to differences in coefficients. Going beyond Oaxaca-Blinder (1973) decompositions at the mean, computing an aggregate decomposition of more general distributional statistics is relatively straightforward, as there are several implementable estimators with good asymptotics. Such parametric or semi-parametric approaches include those proposed by Juhn, Murphy and Pierce (1993), Machado and Mata (2005) and Melly (2005). There are also non-parametric approaches (such as the one by DiNardo, Fortin and Lemieux, 1996). Aggregate decomposition methods are closely linked to the treatment effects literature and the identification assumptions involved generally carry over from this literature.

In an aggregate decomposition, the components of interest of the overall difference Δov in a distributional statistic, v, are : 1) the component associated with differences in the distribution of observable characteristics (what if everything except the distribution of X was the same); 2) the component associated with the distribution of unobservable characteristics (what if everything except the distribution of unobservables was the same); 3) the component of differences associated with the return to observable characteristics, X, contained in the structural functions (what if everything except the return to X was the same for the two groups); 4) the component associated with differences in the return to unobservable characteristics in the structural functions (what if everything except the return to unobservable characteristics was the same). In practice, because of the presence of the components involving unobservables, one cannot compute all 4 components, at least without additional assumptions.

_Detailed decomposition_ methods, on the other hand, aim at deriving the effect of individual covariates for a distributional statistic, and are more recent, less established and still evolving. Machado and Mata’s (2005) quantile regression and simulation based method can estimate only the subcomponents of the coefficients structure component. DiNardo, Fortin and Lemieux (1996) suggested a reweighting approach to derive the subcomponents of the composition effect, while Altonji, Bharadwaj and Lange (2007) suggested a generalization of this approach. A weakness of these methods is that they are path dependent - the decomposition depends on the order it is performed (Fortin, Lemieux and Firpo 2010).

One recent method of implementing detailed decompositions which is gaining popularity because it is close to the spirit of the Oaxaca-Blinder (OB) decomposition is that by Firpo, Fortin and Lemieux (2009). This method, which is path independent, replaces the original left hand side variable with the recentered influence function (RIF) of the distributional statistic and estimates a RIF regression to derive coefficients which are in turn used to implement a detailed decomposition. As suggested in Firpo, Fortin and Lemieux (2007), the best way to implement this method is in two steps (a “hybrid” approach), whereby in the first step one derives the aggregate components of the gap; for example one can use reweighting with appropriate weighting functions. Since the major weakness of this method is its questionable accuracy (the 4
derived coefficients are only local approximations), the two-step approach helps assess how accurate the approximation is.

In this study, we first choose an appropriate method to implement an aggregate decomposition of the over-time change in the test score distribution in Ghana. The method used is an extension of the Machado and Mata (2005) decomposition method, and decomposes differences in distribution into three components – characteristics, median coefficients and residuals – as outlined in Melly (2005). In the first step, the distribution of the dependent variable, TS (TIMSS test score) conditional on the independent variables, X, is estimated using linear quantile regression (see Koenker 2005). The conditional distribution of test scores (TS) is then integrated over the independent variables to obtain the unconditional distribution. This procedure allows us to estimate more precisely the unconditional distribution of TS by using the information contained in the regressors. More importantly, this procedure allows for the estimation of counterfactual unconditional distributions. This estimator is a special case of the class of estimators discussed in Chernozhukov, Fernandez-Val and Melly (2009) and its statistical properties as well as the validity of the bootstrap follow from their results. Additional advantages of this methodology are that it accounts for heteroscedasticity, and is more efficient compared to the Machado and Mata (2005) method.

Subsequently, we compute a detailed decomposition of the over-time change in test score distributions in Ghana, using the Firpo, Fortin and Lemieux (2009) method (hereafter, FFL). As is the case with other methods, the FFL method is not without problems. To estimate the effect of changes in the distribution of X (with a functional of interest in mind), one uses a first order linear approximation of the effect. The procedure results in two terms: the first order linear approximation term and the remaining approximation error. The first order term is estimated using a mean regression method which exploits the law of iterated expectations and the assumption of linearity of the term. However, the estimand is still an approximation of the effect and the approximation error does not vanish. Recently, Rothe (2010a) showed that the effect of counterfactual changes (both fixed and infinitesimal) in the unconditional distribution of a single covariate on the unconditional distribution of an outcome variable of interest are point identified if the covariate affected is continuously distributed; on the other hand, if the distribution of the covariate is discreet, the effects are only partially identified.

For the purposes of this study, the FFL method is an attractive option in deriving estimates of the contribution of grouped characteristics (i.e., household, school, etc.) to supplement the results of the aggregate decomposition, especially when implemented in its two-step (“hybrid”) version. By comparing the estimates from aggregate decomposition to those from the RIF regression-OB decomposition components, one can get a sense of the extent of the problem.

# _2.2   Aggregate decomposition of test score increase at percentiles_

Starting with an independent sample from a population, where X is a vector of characteristics and the τ<sup>th</sup> quantile of TS conditional on Xi given by: , while assuming a linear relationship between the dependent variable and the covariates (household, school and teacher characteristics), the quantile regression coefficients can be 5
interpreted as the return to different characteristics (or more generally the effects on student performance) at different points of the conditional distribution. Following Koenker and Bassett (1978), the estimate of β(τ) is given by: β(τ) is estimated separately for each τ, and the vector of all quantile regression coefficients is: β β τ . Given that this is a model for the conditional quantiles of TS, while the intention is to estimate the unconditional quantiles of TS, one needs to integrate the conditional distribution over the entire range of the distribution of Xs. Melly (2005) shows that one can overcome the potential lack of monotonicity to derive the sample analogue of the θ<sup>th</sup> quantile as: where the θ<sup>th</sup> quantile of the sample is estimated by weighing each observation by ; under standard restrictions, is is a consistent and asymptotically normally distributed estimator. In practice, if the sample is large (as in our case), a smaller number of quantile regressions are estimated. Statistical inference can be conducted using bootstrapping.

Now consider distributions of two samples, 1 and 0, in our case one group in year 2007 and the other in year 2003 (TIMSS 2007 and 2003). The aggregate decomposition is as follows: where, _m,_ and _r_ refer to median coefficients and residuals respectively; , calculated , and , is the quantile estimate of the distribution that would have prevailed if the median return to characteristics had been the same as in sample 1 but residuals had been distributed as in sample 0. Therefore, the first component measures the residual effect, the second the coefficients structure effect and the last the composition effect.

# _2.3   Detailed decomposition: A “hybrid” approach_

In implementing the detailed decomposition at percentiles, in a first step, a reweighting approach is employed to derive estimates of the 2 components of the over-time changes in test score distributions. Using weighting functions, one can transform features of the marginal distribution of test scores into features of the conditional distribution of TS1 given and TS0 given , as well as the features of the counterfactual distribution of TS0 given . In deriving the re-weighting functions, the probability that a person belongs in group 1 conditional on X (“propensity score”) is derived from a logit regression.

In the second step, we use FFL’s unconditional quantile approach to derive the detailed decomposition, by deriving the recentered influence function (RIF) of the dependent variable at quantile τ, where can be estimated by the sample quantile and can be estimated using Kernel density. If the specification of the unconditional quantile regression is linear, i.e., 6
, then the OLS estimate of β (namely, RIF-OLS estimator) provides a consistent estimator of the marginal effect on the unconditional quantile of a small location shift in the distribution of X, holding else constant. So, one can decompose any two distributions at quantiles in a similar spirit as Oaxaca-Blinder approach. It is worth noting, however, that the assumption of linearity used in implementing this detailed decomposition implies that the outcome distribution depends on the marginal distribution of covariates only through their mean (Rothe 2010b).

The decomposition of test score change over-time can be rewritten as<sup>3</sup> : where and are approximation errors that appear because of the first order approximations and the assumption of linear specification. They are calculated as differences between the aggregate explained effect and the total composition effect: , and that between unexplained component and the total coefficients structure effect: .

An identification problem exists in this sort of detailed decompositions as pointed out by Oaxaca and Ransom (1999), as the coefficients effect is not invariant to the choice of reference group; this problem has no easy solution. One approach is to derive results using more than one alternative reference group. The other is to use one of the solutions suggested to deal with this problem. We used the averaging approach proposed by Yun (2005).

# **3. Data and Summary Statistics**

# _3.1. Data_

We used the Trends in International Mathematics and Science Study (TIMSS) data for 8<sup>th</sup> grade Ghanaian students. TIMSS is an international assessment conducted every four years in order to generate information about trends in mathematics and science achievement in each country and over time. The age of tested student population is 9 to10 years (4<sup>th</sup> grade) and 13 to 14 years (8<sup>th</sup> grade) in more than 60 countries; the outcome is an extensive pool of data about the quantity, quality, and content of teaching, which can be used by researchers to assess individual country progress as well as to make comparisons among participating countries. Ghana participated in the TIMSS assessment in 2003 and 2007, but only with 8<sup>th</sup> grade (JHS2) students.

Table A1 in the appendix gives the summary statistics for the student, household, school and teacher characteristics used in the analysis. Notable over-time changes include, from student and household characteristics: the substantial decline in the proportion of children with paid employment (from 55% to 41%), an improvement of parents’ education qualifications (55% with secondary or tertiary qualifications in 2007 compared to 43% in 2003), the decline in the proportion of students who spent no time doing homework; from school characteristics: the increase in average 8<sup>th</sup> grade student enrolment numbers, the large increase in the proportion of students in small communities of less than 3,000 people (from 20% to 35%) and the substantial decline in the proportion of schools with shortages in buildings and teachers; and from the

> 3 Since and , taking expectations on both sides yields . So, .

7
teacher characteristics: the decline in the proportion of certified teachers (from 94% to 82%) and the increase in the proportion of schools giving pay incentives to teachers (from 12% to 21%).

Further to the reported increase in the TIMSS mathematics and science scores from 2003 to 2007 (by 12% and 19% respectively), graphs A1 and A2 show the over-time changes in Kernel density functions. For mathematics, the increase in scores is fairly similar across the score distribution; for science, on the other hand, over-time the variance of the distribution decreased and the increase in scores is smaller at higher points in the distribution of scores.

The above changes in distributions (especially for science) suggest that over-time changes in the degree of inequality in student performance have taken place. Table 2 gives measures of student performance inequality over-time. All estimates are consistent with a decrease in performance inequality for both mathematics and science, but much more for science. Based on the Gini (as well as the relative mean deviation and the coefficient of variation), inequality declined by about 8% for mathematics and by more than 22% for science. The decline in inequality is substantially higher when based on Theil’s entropy measure. In the case of science, percentile scores suggest that the decline was more pronounced at the lower part of the distribution of scores.

Table 2: Inequality measures for Mathematics and Science scores and their over-time changes

||**200**|**3**|**2007 **||**Change**|**(%)**|
|---|---|---|---|---|---|---|
|Statistic|**Mathemati**<br>**cs**|**Science**|**Mathemati**<br>**cs**|**Science**|**Mathemati**<br>**cs**|**Science**|
|Relative mean deviation|0.115|0.173|0.105|0.131|-8.7|-24.3|
|Coefficient of variation|0.288|0.419|0.264|0.326|-8.3|-22.2|
|Gini coefficient|0.162|0.239|0.149|0.185|-8.0|-22.6|
|Theil entropy measure|0.042|0.094|0.035|0.056|-16.6|-40.4|
|p90/p10|2.14|3.50|2.04|2.58|-4.7|-26.3|
|p90/p50|1.42|1.60|1.38|1.47|-2.8|-8.1|
|p75/p25|1.50|1.95|1.43|1.60|-4.7|-17.9|
|p50/p10|1.51|2.19|1.47|1.75|-2.6|-20.1|

Source: author’s calculations.

Prior to model estimation and derivation of results, we deal with the problem of missing information. Several variables have missing values, as some students and school principals did not answer parts of the questionnaires. The extent of missing values in the variables used in the model ranges from 0 to about 10%. Data for missing responses were imputed using the method of relating observations in the original data to a set of “fundamental” variables (see for example Woessmann 2004). Such variables are: student age, student sex, parent education dummies and dummies for community location of schools. As some of the “fundamental” variables had a very small number of missing values, these values were imputed by the median category.

8
# **4. Results**

# _4.1. The model_

First, a cognitive achievement production function is specified. The model specification is as follows: TSija = TSa(Fija, Sija, TRija)+εija, where TSija is the observed test score of student _i_ in household _j_ at time _a_ (TIMSS year), Fija is a vector of student and family inputs, Sija is a vector of school-related inputs, TRija is a vector of teacher characteristics and εija is an additive error, which includes all of the omitted variables including those that relate to the history of past inputs, and measurement error.

Issues of self-selection into schools are non-trivial, as they may have consequences on the estimated relationships between various student, family, school and teacher characteristics and student outcomes. One possible source of self-selection may result from households choosing specific schools based on observed differences, such as cost and location. We control for a wide range of student background characteristics in order to reduce the extent of such a potential bias. Another source has to do with unobserved and hard to measure differences that can affect choice of schools (Vegas 2002). If for example, more educated or more motivated parents systematically choose certain type of schools, the results will overestimate the positive effects associated with such schools. Yet another source of bias is possible, if families choose schools based on their resources; however, in the case of Ghana one does not expect substantial differences in resources between schools (compared to, say, the United States).

Since family income (or a similar measure) is not available in the data, the interpretation of certain coefficients, such as those pertaining to the availability household resources (such as number of books, having a desk, etc.) as well as combining paid work and study is less than straightforward; such information is expected to at least partially capture the effect of family wealth.

Another potentially important information which is not available in the Ghana data is the identification of schools as public vs. private. As reported earlier, in 2007/8, private school enrolment stood at 24% of the total national enrolment, while the number of private schools increased by 36% between 2005/6 and 2007/8 (from 2,990 to 4,068). The main advantage of private schools in Ghana is that they facilitate better English language skills; however, they also provide more access to extra classes and special tuition to students.

# _4.2. Counterfactual decompositions_

The decomposition of changes in mathematics and science scores can reveal whether the overtime improvement in student performance in Ghana, as summarized by the increase in the mean score applies to all 8<sup>th</sup> grade students, as opposed to the existence of significant heterogeneity in performance improvement between high and low scoring pupils, as well as the relative contribution of improved characteristics vs. changes in the effect of these characteristics on performance across the score distribution. The decomposition results are given in tables 3a and 3b.

9
For both mathematics and science, the overall picture is not adequately described by the reported changes at the mean. The improvement in performance is highly heterogeneous across the distribution of scores and more so for science; however, the pattern is very different between mathematics and science. For mathematics, the grade improvement increases at higher points in the distribution, so that the increase for already well performing students is higher compared to students who performed poorly. The p90/p10 ratio of score increase in mathematics is 1.35.

The opposite pattern is observed for science. The largest improvement applies to students at the bottom decile of the score distribution - 62 points; this is an impressive increase at it amounts to an improvement of about 50% within 4 years. The pattern of performance improvement declines monotonically as one goes to higher points in the distribution and at the 90<sup>th</sup> percentile is approximately 38 points (or by about 10%).

For mathematics as well as for science, the contribution of improved characteristics to the overall increase in scores is rather small. At the median they contribute 17% and 21% to the overall increase in scores in mathematics and science respectively. The contribution of improved characteristics varies from 13% (19%) at the bottom decile to about 25% for both mathematics and science at the top. Across the distribution, therefore, characteristics contribute more to improved performance in the case of science compared to mathematics.

Table 3a: Decompositions of over-time change in Mathematics score at percentiles: Ghana, 2003-07

|Percentile|Total change|Characteristics|Median<br>Coefficients|Residuals|
|---|---|---|---|---|
|P10|30.85|4.36|32.96|-6.47|
||(1.56)|(1.74)|(2.18)|(1.85)|
|P20|33.28|4.83|31.69|-3.24|
||(1.63)|(1.79)|(1.82)|(1.53)|
|P30|34.49|5.28|30.44|-1.24|
||(1.61)|(1.74)|(1.77)|(1.32)|
|P40|35.47|5.83|29.67|-0.03|
||(1.61)|(1.74)|(1.74)|(1.15)|
|P50|36.14|6.29|29.35|0.49|
||(1.48)|(1.66)|(1.67)|(1.19)|
|P60|36.82|6.70|29.03|1.09|
||(1.36)|(1.73)|(1.70)|(1.38)|
|P70|38.00|7.27|28.97|1.76|
||(1.37)|(1.79)|(1.76)|(1.56)|
|P80|39.99|8.46|28.89|2.64|
||(1.58)|(1.84)|(2.25)|(1.74)|
|P90|41.59|10.22|28.30|3.07|
||(2.07)|(2.09)|(2.98)|(2.09)|

Standard errors in parentheses.

10
Table 3b: Decompositions of over-time change in Science score at percentiles: Ghana, 2003-2007

|Percentile|Total change|Characteristics|Median<br>Coefficients|Residuals|
|---|---|---|---|---|
|P10|62.36|11.56|46.50|4.30|
||(2.48)|(1.85)|(2.68)|(2.54)|
|P20|59.79|11.57|43.89|4.33|
||(2.73)|(1.66)|(2.55)|(1.90)|
|P30|56.81|10.90|42.31|3.59|
||(2.77)|(1.68)|(2.25)|(1.72)|
|P40|53.67|10.65|40.85|2.17|
||(2.81)|(1.76)|(2.23)|(1.67)|
|P50|49.98|10.39|39.49|0.10|
||(2.90)|(1.95)|(2.19)|(1.87|
|P60|46.49|10.10|38.03|-1.64|
||(2.83)|(2.00)|(2.35)|(2.31)|
|P70|43.16|10.12|35.94|-2.90|
||(1.78)|(2.17)|(2.53)|(2.87)|
|P80|39.98|9.74|33.34|-3.11|
||(2.59)|(2.50)|(2.89)|(3.49)|
|P90|38.61|9.84|29.80|-1.03|
||(2.18)|(3.03)|(3.53)|(3.95)|

Standard errors in parentheses.

The effect of included characteristics on test scores (median coefficients effect) is the largest component of the over-time increase in tests scores in Ghana. This component contributed more to the improvement in the performance of students at lower parts of the score distribution for mathematics, while it generally had a uniformly positive effect on the improvement of science scores. It accounted for more than 100% of the gross mathematics score improvement at the 10<sup>th</sup> percentile and 68% at the 90<sup>th</sup> percentile. Similarly, it accounted for 75% and 77% of the gross science score improvement at the 10<sup>th</sup> and 90<sup>th</sup> percentiles, respectively.

The second component of the increase in scores (due to improvement in characteristics) is generally small for mathematics, but increases to about one-quarter of the total change at higher points in the distribution of scores. In science the characteristics effect is larger, accounting for 20-25% of the total. The third component (residuals) had a small and generally not statistically significant effect, especially around the middle of the distribution. Below the 20<sup>th</sup> percentile it contributed negatively to the over-time change in the mathematics test scores, while it accounted for a small part of the increase in science scores, only below the 30<sup>th</sup> percentile.

11
**Graph 1: Mathematics: Components of the change in score over-time**

<!-- Start of picture text --> Decomposition of differences in distribution<br>0 .2 .4 .6 .8 1<br>Quantile<br>Total differential Effects of residuals<br>Effects of median coefficients Effects of characteristics<br>40<br>30<br>20<br>10<br>0<br>-10<br><!-- End of picture text -->

**Graph 2: Science: Components of the change in score over-time**

12
<!-- Start of picture text --> Decomposition of differences in distribution<br>0 .2 .4 .6 .8 1<br>Quantile<br>Total differential Effects of residuals<br>Effects of median coefficients Effects of characteristics<br>60<br>40<br>20<br>0<br><!-- End of picture text --> _4.3. Decompositions by gender and location_

# <u>Gender gap</u>

The gender test score gap for 8<sup>th</sup> grade Ghanaian students is generally moderate and in favor of boys, in both mathematics and science. In 2003, the gender gap in the mathematics score at the mean was 17 points or 6% of the mean male score and was little changed at about 21 points or 6.4% of the male score in 2007. On the other hand, the gender gap in the science score declined from 34.5 points or 12.7% of the mean male score to 29 points or 9.5% of the mean male score.

Tables A2a and A2b in the appendix give the decomposition results for the gender gap in mathematics and science scores over the entire score distribution (see also charts 3a-3c). In 2003 the gender gap in the mathematics score, measured in points, was higher towards higher points in the distribution, while remaining approximately constant as a proportion of the male score at any particular percentile. In 2007, the gender gap generally increased (and more so below the median), except at the top of the distribution where we observe a decline in the gender score gap. Both the characteristics and the coefficients components contribute to the overtime change in the gender gap.

Gender gaps in science scores are higher compared to mathematics, at about 30-35 points at the median. This gap declined over time, and drastically so at the top quartile of the science score distribution. The main contributor to this decline is associated with changes in the coefficients component which favored of girls.

13
<!-- Start of picture text --> 35<br>Chart 3a: Gender gap in Mathematics score at percentiles - 2003<br>30 vs. 2007<br>25<br>G<br>20<br>e<br>n<br>15 Math-2007<br>d<br>e Math-2003<br>10<br>r<br>5<br>G<br>a 0<br>p 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>Percentile<br><!-- End of picture text --> 14
<!-- Start of picture text --> 10<br>Chart 3b: Characteristics component of the gender gap in<br>8<br>Mathematics score: 2003 vs. 2007<br>6<br>4<br>G<br>2<br>e<br>n 0<br>d Math-2007<br>5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>-2<br>e Math-2003<br>r -4<br>-6<br>G<br>a -8<br>p -10<br>Percentile<br>35<br>Chart 3c: Coefficients component of the gender gap in<br>30 Mathematics score: 2003 vs. 2007<br>25<br>G<br>20<br>e<br>n<br>15 Math-2007<br>d<br>e Math-2003<br>10<br>r<br>5<br>G<br>a 0<br>p 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>Percentile<br><!-- End of picture text --> 15
<!-- Start of picture text --> 45<br>Chart 4a: Gender gap in Science score at percentiles - 2003 vs.<br>40 2007<br>35<br>30<br>G<br>e 25<br>n 20<br>Sience-2007<br>d<br>15 Science-2003<br>e<br>r 10<br>5<br>G<br>0<br>a<br>5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>p<br>Percentile<br>10<br>Chart 4b: Characteristics component of the gender gap in<br>Science score: 2003 vs. 2007<br>5<br>G<br>0<br>e Sience-2007<br>5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>n<br>Science-2003<br>d<br>-5<br>e<br>r<br>-10<br>G<br>a<br>-15<br>p Percentile<br><!-- End of picture text --> 16
<!-- Start of picture text --> 55<br>Chart 4c: Coefficients component of the gender gap in Science<br>score: 2003 vs. 2007<br>45<br>35<br>G<br>e<br>n 25<br>Sience-2007<br>d<br>e 15 Science-2003<br>r<br>5<br>G<br>a<br>-5 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>p<br>Percentile<br><!-- End of picture text -->

# <u>Location gap</u>

In the absence of information on urban vs. rural school location, information on the size of community was used. In the data, size of community is grouped in four categories: under 3,000, 3,000-15,000, 15,000 to 500,000 and over 500,000. A dichotomous variable was created whereby large communities are defined as those with population over 15,000 and small communities those with population under 15,000. Approximately 46% of schools were located in large communities in 2003 and 42% in 2007. Over time, mean mathematics scores increased by 12% in schools in large communities, vs. 14% in small communities; similarly, science scores increased by 17% in large communities and 22% in small communities.

Estimates of the overtime change in the location gap over the entire distribution of scores are a lot more revealing. They suggest that while in 2003 the location gap was of similar magnitude across the distribution of scores (that is for both low and high performing students) at about 35 points for mathematics and between 30 and 50 points for science, in 2007 the location gap in both mathematics and science increased monotonically as one goes to higher points in the distribution of scores; for worse performing students it decreased drastically, while for better performing students correspondingly increased.

Furthermore, the decomposition results reveal that the structure of the components of the location gap have changed drastically for both mathematics and science. In 2003, most of the gap in favor of large communities was due to better returns to characteristics in large compared to small communities for the entire distribution of scores. In 2007, the narrowing of the gap among worse performing pupils is due to the improvement in the return to the characteristics of pupils in small communities compared to pupils in large communities. However, as one goes to higher points in the distribution, a build-up of characteristics in favor of large communities appears to have taken place over time. Above the median, the substantial increase in the characteristics component, along with an increasing coefficients component led to a widening of the location gap among well performing pupils.

17
Over time, therefore, the location gap became more heterogeneous, decreasing among worse performing pupils and increasing among better performing students. The increase in the characteristics component of the location gap, especially at the upper part of the distribution was crucial in this development. One could speculate that the apparent improvement in quality of characteristics for pupils in schools in large communities compared to small communities may be related to the overtime growth in the proportion of private schools, given that most such schools tend to be located in urban areas.

18
<!-- Start of picture text --> 65<br>Chart 5a: Mathematics: Test score gap at percentiles between<br>students in large vs. small communities - 2003 vs. 2007<br>55<br>L 45<br>o<br>c 35<br>a<br>t 25 Math-2007<br>i<br>Math-2003<br>o 15<br>n<br>5<br>G<br>a<br>-5 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>p<br>Percentile<br>40<br>Chart 5b: Mathematics: Characteristics component of the test<br>35 score gap between students in large vs. small communities -<br>2003 vs. 2007<br>30<br>L<br>o 25<br>c<br>a 20<br>t Math-2007<br>15<br>i Math-2003<br>o 10<br>n<br>5<br>G<br>0<br>a<br>5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>p Percentile<br><!-- End of picture text --> 19
<!-- Start of picture text --> 40<br>Chart 5c: Mathematics: Coefficients component of the test score<br>35 gap between students in large vs. small communities - 2003 vs.<br>2007<br>30<br>L<br>o 25<br>c<br>a 20<br>t Math-2007<br>i 15<br>Math-2003<br>o<br>10<br>n<br>5<br>G<br>a 0<br>p 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>Percentile<br>70<br>Chart 6a: Science: Test score gap at percentiles between students<br>in large vs. small communities - 2003 vs. 2007<br>60<br>L<br>50<br>o<br>c 40<br>a<br>t<br>30 Science-2007<br>i<br>Science-2003<br>o 20<br>n<br>10<br>G<br>a 0<br>p 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>Percentile<br><!-- End of picture text --> 20
<!-- Start of picture text --> 40<br>Chart 6b: Science: Characteristics component of the test score<br>35 gap between students in large vs. small communities - 2003 vs.<br>2007<br>30<br>L<br>o 25<br>c<br>20<br>a<br>t 15 Science-2007<br>i Science-2003<br>10<br>o<br>n 5<br>G 0<br>a 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>-5<br>p<br>Percentile<br>50<br>Chart 6c: Science: Coefficients component of the test score gap<br>between students in large vs. small communities - 2003 vs.<br>40 2007<br>L<br>o 30<br>c<br>G<br>a 20 a Science-2007<br>t<br>Science-2003<br>p<br>i<br>10<br>o<br>n<br>0<br>5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95<br>Percentile<br><!-- End of picture text -->

# _4.4. Detailed decompositions_

Tables 4a and 4b present the results of decomposing the over-time change in scores at the mean, using a standard Oaxaca decomposition methodology; the coefficients used in the decomposition were transformed using the _deviation contrast transform_ procedure (see Yun 2005)<sup>4</sup> in order to deal with the well-known “identification” problem associated with Oaxaca-type decompositions; that is, the problem arising from the dependence of the coefficients effect on the choice of reference group, when various groups of a categorical variable are involved.

Improved characteristics account for one-third of the increase in the mean mathematics score and one-quarter of the increase in the science score. Main contributors include the lower incidence of

> 4 The basic idea of Yun’s solution is to derive estimates for all possible reference groups and then derive the averaging ones; fortunately the implementation doesn’t necessarily involve tedious and repeated regressions for varying reference groups. The STATA command _Devcon_ is used for this purpose.

21
paid work, more homework time, increase in grade 8 enrolment<sup>5</sup> and more pay incentives for teachers. The coefficients component for the covariates used in the model is towards an overtime decrease in both the mathematics and science scores; however, a large effect associated with changes in the constants is the main contributor to the over-time increase in TIMSS scores in Ghana.

Table 4a: Detailed decomposition of change in the 8<sup>th</sup> <u>grade Mathematics score at the Mean</u>

|Mean prediction for 2003: 281.2<br>Mean prediction for 2007: 316.6|**Characteristics**|**Coefficients**|**Total Change**|
|---|---|---|---|
|**Total change**|**12.27**<br>**(9.6)**|**23.19**<br>**(14.5)**|**35.46**<br>**(23.1)**|
|**Characteristics component**||||
|Student and household characteristics||||
|Age|-0.99|-0.28|-1.27|
|Gender of student|-0.02|0.03|0.01|
|Speaks/not speak test language at home|0.05|0.29|0.34|
|Calculator at home|0.20|-0.04|0.16|
|Desk at home|0.00|-1.80|-1.80|
|Works/Not in paid job<br>|2.90|0.18<br>|3.08|
|Homework time|1.22|1.85|3.07|
|Highest education of parent|-0.45|-1.62|-2.07|
|Both parents born/not born in country<br>|0.46|0.43<br>|0.89<br>|
|Books at home|0.10|-0.16|-0.06|
|**Total**|**3.47**|**-1.12**|**2.35**|
|School characteristics||||
|Total enrolment, grade 8|3.29|-5.79|-2.50|
|Community size|0.06|-1.76|-1.70|
|Proportion of disadvantaged students|-0.16|2.18|2.02|
|Ability grouping yes/no|-0.07|0.86|0.79|
|Remedial lesson in Math Yes/no|0.04|0.91|0.95|
|Severe absenteeism in school yes/no|0.68|4.50|5.18|
|Shortage of buildings yeas/no|0.53|-0.14|0.39|
|Shortage of teachers yeas/no|0.00|0.17|0.17|
|Index of good attendance|-1.12|-3.18|-4.30|
|**Total**|**3.25**|**-2.25**|**1.00**|
|Teacher characteristics||||
|Teacher age|0.41|-2.55|-2.96|
|Teacher gender|0.66|-0.40|0.26|
|Teacher years of experience|0.08|-11.02|-10.94|
|<br>Teacher education|0.40|-3.25|-2.85|
|Teacher certificate yes/no|1.60|-0.26|1.34|
|Teacher feels safe/not safe|0.09|6.10|6.19|
|Teacher satisfaction|0.15|-1.98|-1.83|
|Pay incentives for teachers yes/no|2.16|-5.87|-3.71|
|<br>**Total**|**5.55**|**-19.23**|**-14.50**|
|**Constant**|**-**|**45.80**|**45.80**|
|**N**||11,892||
|**2003:**||5,100||
|**2007:**||6,792||

z-values in parentheses.

> 5 Results show that higher enrolment is associated with higher scores in Ghana.

22
Table 4b: Detailed Oaxaca decomposition of change in the 8<sup>th</sup> <u>grade Science score at the Mean</u>

|Mean prediction for 2003: 262.6<br>Meanprediction for 2007: 311.5|**Characteristics**|**Coefficients**|**Total Change **|
|---|---|---|---|
|**Total change**|**11.71**<br>**(7.4)**|**37.13**<br>**(18.2)**|**48.83**<br>**(24.5)**|
|**Characteristics component**||||
|Student and household characteristics||||
|Age|-1.22|1.43|0.21|
|Gender of student|-0.03|-0.34|-0.37|
|Speaks/not speak test language at home|0.00|1.12|1.12|
|Calculator at home|-0.12|-0.12|-0.24|
|Desk at home|-0.01|-1.61|-1.62|
|Works/Not in paid job|2.87|0.00|2.87|
|Homework time|1.66|1.95|3.61|
|Highest education of parent|-0.67|-2.04|-2.71|
|Both parents born/not born in country|0.52|-5.62|-5.10|
|Books at home|0.11|0.01|0.12|
|**Total**|**3.11**|**-5.22**|**-2.11**|
|School characteristics||||
|Total enrolment, grade 8|3.95|-9.03|-5.08|
|Community size|-0.62|-1.21|-1.83|
|Proportion of disadvantaged students|-0.25|3.05|2.80|
|Ability grouping yes/no|-0.54|2.62|2.08|
|Remedial lesson in Math Yes/no|-0.20|-1.12|-1.32|
|Severe absenteeism in school yes/no|0.90|5.13|6.03|
|Shortage of buildings yeas/no|0.50|-0.17|0.33|
|Shortage of teachers yeas/no|-0.01|0.38|0.37|
|Index of good attendance|-1.38|-1.04|-2.42|
|<br>**Total**|**2.35**|**-1.39**|**0.96**|
|Teacher characteristics||||
|Teacher age|0.61|-2.94|-2.33|
|Teacher gender|0.55|1.04|1.59|
|Teacher years of experience|0.03|-14.51|-14.48|
|Teacher education|0.66|-4.64|-3.98|
|Teacher certificate yes/no|1.41|3.00|4.41|
|Teacher feels safe/not safe|0.11|6.53|6.64|
|Teacher satisfaction|-0.39|2.48|2.09|
|Pay incentives for teachers yes/no|2.62|-8.16|-5.54|
|<br>**Total**|**5.6**|**-17.2**|**-11.6**|
|**Constant**|**-**|**60.95**|**60.95**|
|**N**||11,892||
|**2003:**||5,100||
|**2007:**||6,792||

z-values in parentheses.

23
# _4.5. Detailed decompositions at percentiles using a “hybrid” approach_

Table A4a and A4b in the appendix contain the (transformed) RIF regression coefficients at selected quantiles (10<sup>th,</sup> 50<sup>th</sup> and 90<sup>th</sup> ). Students in the right age for 8<sup>th</sup> grade are associated with a higher score, while the effect of being an overage student reduces scores and more so at higher point in the score distribution. Speaking the test language at home seems beneficial to high scoring students, but the opposite is the case below the median. The effect of books at home is complex; high scoring student with the most books at home are associated with a higher score in science; however, the pattern for mathematics is different, with students below the median associated with a higher score in 2003, but a lower score in 2007. The effect of having a parent with tertiary education is small for poorly performing students, but increases substantially at higher points in the distribution, especially in science. Having both parents born in Ghana has a strong positive association with scores in 2003 and less so in 2007. Two of the most powerful regressors have to do with paid work and time spent on homework. At the median, a child in combining paid work with study results in a decline in mathematics (science) score by 10 points (18 points); likewise, spending no time on homework decreases scores by 13 (17) points.

Larger 8th grade enrolment is associated with higher scores.<sup>6</sup> The association between community size and performance is strong but heterogeneous. For example, in science there a strong positive association between high performing students in large cities (>500,000) and performance in 2003; for students below the median the positive association is strongest for medium-size communities (15,000-500,000). However, small communities (<3,000) are consistently associated with lower performance in both mathematics and science. In 2003, students in schools with low proportion of disadvantaged students performed better; however, this association seems to have disappeared or reversed in 2007. There is some evidence that ability grouping by schools is negatively associated with performance, especially I science. Students in schools with a severe absenteeism problem performed worse, while high performing students in schools with high index of good attendance performed better.

Teacher age is negatively associated with student performance, while the opposite is the case for teacher years of experience. Having a teacher with university education (ISCED 5A) results in a significant increase in student scores (but mostly in science), and this effect increased in 2007 compared to 2003. Finally, the practice of pay incentives to teachers has a moderate by consistently positive association with performance, and more so at higher points in the distribution.

Decompositions results (given in tables 5a and 5b, with detailed results in tables A5a and A5b in the appendix) are derived from the 2-step procedure outlined in the methodology section. In the first step, the changes in score and its two constituent components are derived using reweighting, without imposing specific functional forms or parametric distributions when deriving estimates of the total gap. In deriving the re-weighting functions, the probability that a person belongs in the “year 2003” group conditional on (propensity score) is estimated from a logit regression. In the second step, estimates of the effects of each individual variable on the test score gap are generated. The estimation of unconditional quantile regression involves deriving

> 6 The Ministry of Education (2004) of Ghana in its summary of its key findings on TIMSS 2003 found a positive correlation between class size and student performance.

24
the Recentered Influence Function of the dependent variable and estimating an OLS regression of the generated RIF variable on covariates. The estimated coefficients are in fact unconditional partial effects of small location shifts of the covariates; therefore, the decomposition becomes similar to decomposing the gap at the mean using the Oaxaca-Blinder method.

In tables 5a and 5b the contributors to the characteristics and coefficients effects are combined into one group relating to student and family characteristics, another to school characteristics and a third to teachers characteristics. Within the coefficients component, the effect associated with changes in the constant is given separately. Finally, an approximation error/residual is involved; this is due to the linearity assumption involved in estimating the RIF-OLS regressions and because these estimated regression coefficients is only first-order approximations. In practice, the error in the coefficients/characteristics effect can be estimated as the difference between the estimate of the through re-weighting and the estimate obtained from the RIF-regression procedure.

25
Table 5a: Decomposition of change in 8<sup>th</sup> <u>grade Mathematics score at percentiles</u>

||**Characteristics**|**Coefficients**|**Total change**|
|---|---|---|---|
|**P10**||||
|Student and household<br>|1.01<br>|-1.03<br>|-0.02<br>|
|Schools<br>Teachers|-0.50<br>039|4.35<br>-391|3.85<br>-352|
|<br>Constant|.<br>-|.<br>26.16|.<br>26.16|
|Approximation error<br>|2.03<br>|-1.82<br>|0.21<br>|
|**Total**<br>**P20**|**2.94**|**23.75**|**26.69**|
|Student and household|0.99|6.56|7.55|
|Schools<br>Teachers|-0.02<br>0.89|13.63<br>-8.10|13.61<br>-7.21|
|Constant|-|17.47|17.47|
|Approximation error|1.35|-1.10|0.15|
|**Total**<br>|**3.20**|**28.45**|**31.65**|
|**P30**||||
|Student and household|1.32|17.67|18.99|
|Schools|-1.35|21.50|20.15|
|Teachers|2.25|-12.64|-10.39|
|Constant|-|5.22|5.22|
|Approximation error|2.07|-1.86|0.21|
|<br>**Total**<br>**P40**|**4.28**|**29.89**|**34.17**|
|Student and household|0.50|-11.37|-10.87|
|Schools|-1.08|7.29|6.21|
|Teachers<br>|1.32|-5.25<br>|-3.93<br>|
|Constant|-|42.11|41.11|
|Approximation error<br>|2.72<br>|-2.19<br>|0.53<br>|
|**Total **<br>**P50**|**3.46**|**30.60**|**34.06**|
|Student and household|0.88|7.96|8.84|
|Schools<br>|-2.67<br>|18.55<br>|15.88<br>|
|Teachers|3.44|-19.21|-15.77|
|Constant<br>|-<br>|24.58<br>|24.58<br>|
|Approximation error<br>**Total**<br>|0.69<br>**2.33**|-0.43<br>**31.45**|0.26<br>**33.78**|
|**P60**||||
|Student and household|0.23|17.40|17.63|
|Schools|-2.16|3.94|1.78|
|Teachers<br>|2.53|-13.86<br>|-11.33<br>|
|Constant|-|57.93|57.93|
|Approximation error|3.23|-2.73|0.50|
|<br>**Total**|**3.84**|**27.87**|**31.71**|
|**P70**<br>Student and household|-0.06|-8.05|-8.11|
|Schools|-1.45|4.94|3.49|
|Teachers<br>|1.83|-5.84<br>|-4.01<br>|
|Constant<br>Aiti|-<br>372|39.49<br>348|39.49<br>024|
|pproxmaon error<br>**Total**|.<br>**4.04**|-.<br>**27.05**|.<br>**31.09**|
|**P80**<br>||||
|Student and household|-0.35|-18.85|-18.50|
|Schools<br>|-1.03<br>|-2.90<br>|-3.93<br>|
|Teachers<br>Constant|3.46<br>-|-9.21<br>6365|-5.75<br>6365|
|<br>|<br>|.<br>|.<br>|
|Approximation error<br>**Total**|5.24<br>**731**|-5.72<br>**2697**|-0.48<br>**3428**|
|<br>**P90**|**.**|**.**|**.**|
|Student and household|028|-928|-900|
|<br>Schools|.<br>-0.52|.<br>-6.80|.<br>-7.32|
|Teachers<br>|1.41|1.09<br>|2.50<br>|
|Constant|-|51.40|51.40|
|Approximation error|3.88|-4.35|-0.47|
|<br>**Total **|<br>**5.05**|<br>**32.06**|<br>**37.11**|
|N: 2003<br>||5,100<br>||
|N: 2007||6,792||

26
<!-- Start of picture text --> 40 Graph 7a: Mathematics: components of total score change<br>35<br>30<br>Total change<br>25<br>Characteristics<br>20 Coefficients<br>15<br>10<br>5<br>0<br>P10 P20 P30 P40 P50 P60 P70 P80 P90<br>Graph 7b: Mathematics: Major contributors to score increase<br>70<br>60<br>50<br>Student/<br>40 household<br>School<br>30<br>20 Teacher<br>10<br>Constant<br>0<br>P10 P20 P30 P40 P50 P60 P70 P80 P90<br>-10<br>-20<br>-30<br><!-- End of picture text --> The estimates of the total score change and the two components, while qualitatively similar to those obtained using the aggregate decomposition (based on a different methodology), differ substantially with respect to the size of the characteristics component (given the different approaches used), and more so in the case of science. In the aggregate decomposition the characteristics component constituted 15-25% of the total gap. Using Firpo et.al (2009) the characteristics component is around zero or slightly negative.

While changes hidden behind the constant shaped the increase in scores between 2003 and 2007, for students performing at the median and below in mathematics certain characteristics controlled for contributed significantly, in particular school and student/family characteristics . On the other hand, changes associated with teacher characteristics are generally towards 27
reducing scores. In science, changes in the constant overwhelmingly determine the observed increase in scores, with the exception of the median, where school characteristics contributed as much as changes in forces behind the constant.

28
Table 5b: Decomposition of change in 8<sup>th</sup> <u>grade Science score at percentiles</u>

||**Characteristics**|**Coefficients**|**Total change**|
|---|---|---|---|
|**P10**||||
|Student and household|1.10|-3.72|-2.62|
|Schools|-0.52|4.91|4.39|
|Teachers<br>|-0.34|11.78<br>|11.44<br>|
|Constant|-|37.83|37.83|
|Approximation error<br>|0.22<br>|0.61<br>|0.83<br>|
|**Total **<br>**P20**|**0.46**|**51.40**|**51.86**|
|Student and household|1.18|-16.58|-15.40|
|Schools<br>|-2.46<br>|6.57<br>|4.11<br>|
|Teachers|0.48|5.65|6.13|
|Constant||60.33|60.33|
|Approximation error|--0.66|1.40|0.74|
|**Total **<br>|**-1.47**|**57.37 **|**55.90**|
|**P30**<br>Student and household|0.55|0.31|0.86|
|Schools|-3.06|7.42|4.36|
|Teachers|0.84|-3.96|-3.12|
|Constant<br>|-|51.89|51.89<br>|
|Approximation error|-1.37|1.90|0.53|
|<br>**Total**|**-3.04**|**57.56**|**54.52**|
|**P40**||||
|Student and household|-0.12|-14.70|-14.82|
|Schools|-4.06|10.64|6.58|
|Teachers<br>|0.89|-0.24<br>|0.65<br>|
|Constant|-|54.50|54.50|
|Approximation error<br>|-0.71<br>|1.68<br>|0.97<br>|
|**Total**|**-4.00**|**51.87**|**47.87**|
|**P50**<br>Student and household|-1.07|4.81|3.74|
|Schools<br>|-6.25<br>|31.22<br>|24.97<br>|
|Teachers|1.07|-5.47|-4.40|
|Constant<br>|-<br>|15.10<br>|15.10<br>|
|Approximation error<br>**Total**<br>|1.04<br>**-5.21**|-0.36<br>**45.29**|0.68<br>**40.08**|
|**P60**||||
|Student and household|-0.69|-32.79|-32.10|
|Schools|-1.98|-2.89|-4.87|
|Teachers<br>|1.01|4.19<br>|5.20<br>|
|Constant|-|67.73|67.73|
|Approximation error|-3.09|4.24|1.15|
|<br>**Total **<br>|**-4.75**|**40.49**|**35.74**|
|**P70**<br>Student and household|-0.31|-0.08|-0.39|
|Schools|-1.76|10.45|8.69|
|Teachers|2.51|-20.05<br>|-17.54<br>|
|Constant|-|42.65|42.65|
|Approximation error<br>|-1.10<br>|1.78<br>|0.68<br>|
|**Total**|**-0.66**|**34.75**|**34.09**|
|**P80**<br>||||
|Student and household|-0.47|-6.60|-7.07|
|Schools<br>|-0.11<br>|1.20<br>|1.09<br>|
|Teachers|1.17|-10.87|-9.70|
|Constant<br>|-<br>|44.62<br>|44.62<br>|
|Approximation error|-0.30|0.55|0.25|
|**Total**<br>|**0.29**|**28.90**|**29.19**|
|**P90**||||
|Student and household|-0.76|-21.75|-22.51|
|<br>Schools|<br>0.52|<br>-15.53|<br>-15.01|
|Teachers|2.51|-26.43|-23.92|
|<br>Constant|<br>-|<br>87.42|<br>87.42|
|Approximation error|0.58|-0.11|0.47|
|<br>**Total**|**2.84**|**23.61**|**26.45**|
|N: 2003<br>||5,100<br>||
|N: 2007||6,792||

29
<!-- Start of picture text --> 70 Graph 7c: Science: components of total score change<br>60<br>50<br>Total change<br>40<br>30 Characteristics<br>20 Coefficients<br>10<br>0<br>P10 P20 P30 P40 P50 P60 P70 P80 P90<br>-10<br>Graph 7d: Science: Major contributors to score increase<br>80<br>60<br>Student/<br>household<br>40<br>School<br>20 Teacher<br>0 Constant<br>P10 P20 P30 P40 P50 P60 P70 P80 P90<br>-20<br>-40<br><!-- End of picture text --> From the detailed decomposition results (see tables A5a and A5b), besides the dominant effect of unaccounted forces behind the constant, one can single out certain individual contributors which fairly consistently contribute either positively or negatively to the change in mathematics and science scores. From the positive but small characteristics component, a moderate positive effect is associated with students involved less in paid work and spending more time doing homework.

From the coefficients component, and starting with science, a performance improving effect is associated with changes in coefficients associated with the proportion of disadvantaged students in the school, variables relating to absenteeism, proportion of certified teachers and satisfied teachers. On the other hand, a negative effect on scores is associated with changes in coefficients 30
relating to both parents born in Ghana, teacher experience (and occasionally teacher education level) and larger enrolment in 8<sup>th</sup> grade. In mathematics, the most important contributors are generally the same, along with a score increasing effect of the coefficients associated with more homework for students below the median.

# **5. Conclusion**

This paper brings to light certain factors underlying the substantial over-time performance improvement in mathematics and science in Ghana. Performance improved more for better performing students in mathematics, while the opposite was the case for science where students at the bottom of the score distribution experienced a spectacular increase in science scores. More specifically, inequality in scores decreased over time, mostly in science.<sup>7</sup> Most of the increase in scores is accounted by changes in coefficients (and a smaller part by improvements in characteristics). Within the coefficients component, measured or unmeasured characteristics not accounted for (and therefore captured by changes in the constant) dominate this effect. However, for students below the median the effect associated with school and student characteristics is substantial.

One could speculate that among the possible unaccounted characteristics, a school being private vs. public could be of importance in accounting for the over-time increase in scores. This is because a substantial proportion of grade 8 students attended private schools in 2007, private schools are generally perceived to produce better performing students, and most importantly, over the short period between the two TIMSS surveys there was a large increase in the number of private schools in Ghana (by 36% only between 2005/6 and 2007/8). Evidence is also given suggesting that the structure of the gap related to large vs. small communities changes drastically; characteristics improved substantially for pupils in schools in large communities relative to smaller communities.

> 7 On the other hand, standard inequality measures suggest a light decline in inequality in scores in mathematics as well.

31