# **Description of the Sample**

The means for the variables that we used to analyze the determinants of learning are presented in Table 1. The PISA data contain missing values among the family background characteristic variables, and in. Table 1, we show where those missing values occur. While some might choose to impute the missing values, we decided not to do so in this case. Therefore, if a variable had any missing values, we dropped the observation in its entirety from our analysis.

We realize that deleting cases with missing values can have dangers, as demonstrated by Little and Rubin (1987). Deleting cases is based on the assumption that the deleted cases occur at random and are a relatively small representative proportion of the entire dataset. However, this may not necessarily be the case. The missing data may be indicative of some pattern and cannot safely be assumed to reflect randomness. In such circumstances, deletion can introduce substantial bias into the study. Moreover, the loss in sample size can appreciably diminish the statistical power of the analysis.

As a rule of thumb, if a variable has more than 5 percent missing values, it is advisable not to delete cases, and many researchers are much more stringent than this (Little and Rubin, 1987). Deleting incomplete cases has its attractions, mostly the virtue of simplicity, but one loses information in doing so. This approach also ignores the possible systematic difference between the complete cases and incomplete cases, and the resulting inference may not be applicable to the population associated with all cases, especially with a smaller number of complete cases to take into account. Some techniques exist to impute missing values, ranging from correlations, single imputation, and a multiple imputation procedure (Rubin, 1987). However, very few of our variables had missing values that made up more than 5 percent of the total. Overall, the sample for 2003 dropped from 10,761 students to 8,828, and in 2006, the sample went down from 10,647 to 9,293.

The mean scores associated with each characteristic increased over time (Table 2). The scores for students whose mothers had a university education were much higher in 2006 than in 2003, at more than half a standard deviation. Speaking the same language at home that is used in school increased scores by more in 2006 than in 2003. The largest increase was for children with at least one computer at home ‒ a 66 point increase or the equivalent of two years of learning.

Another important change is the score associated with the school autonomy variable titled ―the school determines pedagogy.‖ In 2003, those schools that did not determine pedagogy scored higher than those that had autonomy over their own pedagogy, but by 2006, the opposite was true. Also, the association between gender and math scores changed over time. In 2003, there was little difference in overall scores between boys and girls, but by 2006, boys scored 17 points higher girls in math. Given that we did not impute, we knew there was a possibility that our analysis would be biased. To minimize this risk **,** we examined mean scores by variable for two samples in each year (see Annex Table 1). One was the regression sample, which did not include observations with any missing value, and the other was the full PISA sample. The regression sample, despite its (small) number of missing values, was not very different from the full PISA sample in terms of outcomes. The differences in math scores by characteristic did not vary appreciably, by as little as 1 point in some cases and by no more than 10 points in others. Overall, the scores differed by an average of only 4 points. On a scale with a mean of 500 and a 8