# **8. Annexes**

## **_A.1. School Productivity or Learning Gains (Grade Effect)_**

OECD (2010) suggests that “while it is possible to estimate the typical performance difference among students in two adjacent grades net of the effects of selection and contextual factors, this difference cannot automatically be equated with the progress that students have made over the last school year but should be interpreted as a lower boundary of the progress achieved. This is not only because different students were assessed but also because the content of the PISA assessment was not expressly designed to match what students had learned in the preceding school year but more broadly to assess the cumulative outcome of learning in school up to age 15. For example, if the curriculum of the grades in which 15-year-olds are enrolled mainly includes material other than that assessed by PISA (which, in turn, may have been included in earlier school years) then the observed performance difference will underestimate student progress”. In 2009, this grade effect estimate was approximately 39 PISA points for reading (see OECD (2010) Annex A1, Table A1.2), in 2012, the same exercise results in 41 points (see OECD (2014) Annex A1, Table A1.2). We have estimated using PISA 2018 data, an average effect for all participating countries of approximately 37 PISA points. For the purpose of our Global exercise we use 40 PISA points as the baseline scenario of our simulations for upper middle-income countries. We assume that learning gains will vary from 20 to 50 PISA points depending on the country’s income level.

A detailed description of the approach used by OECD to estimate this grade effects follows: _“Data on the student’s grade are obtained both from the student questionnaire and from the student tracking form. As with all variables that are on both the tracking form and the questionnaire, inconsistencies between the two sources are reviewed and resolved during datacleaning. In order to capture between-country variation, the relative grade index (GRADE) indicates whether students are at the modal grade in a country (value of 0), or whether they are below or above the modal grade level (+ x grades, - x grades)._ _The relationship between the grade and student performance was estimated through a multilevel model accounting for the following background variables: i) the PISA index of economic, social and cultural status; ii) the PISA index of economic, social, and cultural status squared; iii) the school mean of the PISA index of economic, social, and cultural status; iv) an indicator as to whether students were foreign-born first-generation students; v) the percentage of first-generation students in the school; and vi) students’ gender._ _Table A1.2 presents the results of the multilevel model. Column 1 in Table A1.2 estimates the score-point difference that is associated with one grade level (or school year). This difference can be estimated for the 32 OECD countries in which a sizeable number of 15-year-olds in the PISA samples were enrolled in at least two different grades. Since 15-year-olds cannot be assumed to be distributed at random across the grade levels, adjustments had to be made for the above-mentioned contextual factors that may relate to the assignment of students to the different grade levels.” (OECD, 2014 pg. 261)_ Country specific values can be found in Annex A1, tables A1.2 of both OECD reports (OECD, 2010 and 2014).

41