_<u>Reform led to improvement</u> – through delay of vocational, more education-relevant_ _<u>inputs (hours, motivation, better teachers, etc.)</u>_ <u>. This paper concentrates on Poland. It tries to</u> recognize differences between the country’s samples collected in 2000, 2003 and 2006. Unadjusted score distribution is compared across years. After this purely descriptive analysis some adjustments to make samples more comparable are proposed and semi-parametric methods are employed to produce equivalent score distributions. Finally, the differences-in-differences method is applied to test whether extension of general comprehensive education was the main reason for score improvement over time.

_Students are more accustomed to taking tests and teachers are preparing students for tests_ . Rigorous academic testing was not the norm prior to the 1999 reforms. Soon after the reforms tests became more important and regular. This exposure to assessments may have prepared students, thus making them better “test takers.”

# **5. Empirical Methods and Data**

We test whether the reform – more specifically the change in the structure of the school system – led to the improvement in test scores, through the delay of vocational education. Our main approach is based on propensity score matching and reweighting. Assume that one wants to compare survey results that are directly non-comparable because of differences in the distribution of observable characteristics. Then one can calculate conditional expectations based on these characteristics and use them to calculate the difference of interest. However, when the number of distinct values of important covariates is high or some of them are continuous, then any comparisons of this kind become problematic. This is known as the “curse of dimensionality.” To resolve this problem propensity score matching methods were proposed by Rosenbaum and Rubin (1983). They showed that instead of using multiple covariates one can use the _propensity score_ that reflects the probability of being sampled to one of the groups conditional on covariates. Originally, propensity score matching methods were applied to solve selection problems but in recent applications they were also used to adjust statistics across datasets (see Tarozzi 2007). Similar methods were also applied earlier to compare whole outcome distributions before and after reweighting based on observable individual characteristics (DiNardo, Fort and Lemieux 1996). In this paper, when comparing whole distributions of student achievement, we use simple propensity score weight adjustment. The counterfactual outcome distribution is obtained using kernel density estimators with weights given by: Tarozzi (2007) argues that such reweighting produces comparable outcome distributions. _Depvar_ =1 is defined as being in a sample of interest (“target” sample), which in our case means the sample of PISA students in 2000. _Depvar_ equals 0 for students sampled in 2003 or 2006, depending on a comparison made. Conditional probabilities are estimated using logit regression with a set of student and family characteristics defined in the same way in all waves of the PISA survey (recoded to have similar categories). Additionally, we considered sample weights that are of importance when one wants to make inferences about population effects. PISA survey design was accounted for by multiplying propensity score weights and survey weights.

10