Programme for International Student Assessment (2000 to 2012)
The Programme for International Student Assessment has had several runs before the most recent one in 2012. The first PISA assessment was carried out in 2000. The results of each period of assessment take about one year and a half to be analysed. First results were published in November 2001. The release of raw data and the publication of technical report and data handbook only took place in spring 2002. The triennial repeats follow a similar schedule; the process of seeing through a single PISA cycle, start-to-finish, always takes over  four years. 470,000 15-year-old students representing 65 nations and territories participated in PISA 2009. An additional 50,000 students representing nine nations were tested in 2010. Every period of assessment focuses on one of the three competence fields of reading, math, science; but the two others are tested as well. After nine years, a full cycle is completed: after 2000, reading was again the main domain in 2009. Results. PISA 2012. PISA 2012 was presented on 3 December 2013, with results for around 510,000 participating students in all 34 OECD member countries and 31 partner countries. This testing cycle  had a particular focus on mathematics, where the mean score was 494. A sample of 1,688 students from Puerto Rico took the assessment, scoring 379 in math, 404 in reading and 401 in science. A subgroup of 44 countries and economies with about 85 000 students also took part in an optional computer-based assessment of problem solving. Shanghai had the highest score in all three subjects. It was followed by Singapore, Hong Kong, Chinese Taipei and Korea in mathematics; Hong Kong, Singapore, Japan and Korea in reading and Hong Kong, Singapore, Japan and Finland in science. They were a  sample of about 28 million in the same age group in 65 countries and economies, including the OECD countries, several Chinese cities, Vietnam, Indonesia and several countries in South America. The test lasted two hours, was paper-based and included both open-ended and multiple-choice questions. The students and school staff also answered a questionnaire to provide background information about the students and the schools. PISA 2012 was presented on 3 December 2013, with results for around 510,000 participating students in all 34 OECD member countries and 31 partner countries. This testing cycle had a particular focus on mathematics , where the mean score was 494. The mean score in reading was 496 and in science 501. The results show distinct groups of high-performers in mathematics: the East Asian countries, with Shanghai, scoring the best result of 613, followed closely by Hong Kong, Japan, Chinese Taipei and South Korea. Among the Europeans, Liechtenstein and Switzerland performed best, with Netherlands, Estonia, Finland, Poland, Belgium, Germany, Austria all posting mathematics scores "not significantly statistically different from" one another. The United Kingdom, Ireland, Australia and New Zealand were similarly clustered around the OECD average of 494, with the USA trailing this group  at 481. Qatar, Kazakhstan and Malaysia were the countries which showed the greatest improvement in mathematics. The USA and the United Kingdom showed no significant change. Sweden had the greatest fall in mathematics performance over the last ten years, with a similar falling trend also in the two other subjects, and leading politicians in Sweden expressed great worry over the results. On average boys scored better than girls in mathematics, girls scored better than boys in reading and the two sexes had quite similar scores in science. Indonesia, Albania, Peru, Thailand and Colombia were the countries where most students reported  being happy at school, while students in Korea, the Czech Republic, the Slovak Republic, Estonia and Finland reported least happiness. PISA 2009. The PISA 2009 cycle included results in mathematics, science and reading for all 36 OECD member countries and 37 partner countries. Of the partner countries, only selected areas of three countries—India, Venezuela and China—were assessed. PISA 2009+, released in December 2011, included data from 10 additional partner countries which had testing delayed from 2009 to 2010 because of scheduling constraints. PISA 2003. The results for PISA 2003 were released on 14 December 2004. This PISA  cycle tested 275,000 15 year-olds on mathematics, science, reading and problem solving and involved schools from 30 OECD member countries and 11 partner countries. Note that for Science and Reading, the means displayed are for "All Students", but for these two subjects (domains), not all of the students answered questions in these domains. In the 2003 OECD Technical Report (pages 208, 209), there are different country means (different than those displayed below) available for students who had exposure to these domains. PISA 2000. The results for the first cycle of the PISA survey were released on 14 November 2001 . 265,000 15 year-olds were tested in 28 OECD countries and 4 partner countries on mathematics, science and reading. An additional 11 countries were tested later in 2002. Comparison with other studies. The correlation between PISA 2003 and TIMSS 2003 grade 8 country means is 0.84 in mathematics, 0.95 in science. The values go down to 0.66 and 0.79 if the two worst performing developing countries are excluded. Correlations between different scales and studies are around 0.80. The high correlations between different scales and studies indicate common causes of country differences (e.g. educational quality, culture, wealth or genes) or a  homogenous underlying factor of cognitive competence. European Economic Area countries perform slightly better in PISA; the Commonwealth of Independent States and Asian countries in TIMSS. Content balance and years of schooling explain most of the variation. Reception. The results from PISA 2003 and PISA 2006 were featured in the 2010 documentary Waiting for "Superman". China. Education professor Yong Zhao has noted that PISA 2009 did not receive much attention in the Chinese media, and that the high scores in China are due to excessive workload and testing, adding that it's "no news that the Chinese education system is excellent  in preparing outstanding test takers, just like other education systems within the Confucian cultural circle: Singapore, Korea, Japan, and Hong Kong." Students from Shanghai, China, had the top scores of every category (Mathematics, Reading and Science) in PISA 2009. In discussing these results, PISA spokesman Andreas Schleicher, Deputy Director for Education and head of the analysis division at the OECD’s directorate for education, described Shanghai as a pioneer of educational reform in which "there has been a sea change in pedagogy". Schleicher stated that Shanghai abandoned its "focus on educating a small elite, and instead worked to construct a  more inclusive system. They also significantly increased teacher pay and training, reducing the emphasis on rote learning and focusing classroom activities on problem solving." Schleicher also states that PISA tests administered in rural China have produced some results approaching the OECD average: Citing further, as-yet-unpublished OECD research, Schleicher said, "We have actually done Pisa in 12 of the provinces in China. Even in some of the very poor areas you get performance close to the OECD average." Schleicher says that for a developing country, China's 99.4% enrollment in primary education is "the envy of many countries". He  maintains that junior secondary school participation rates in China are now 99%; and in Shanghai, not only has senior secondary school enrollment attained 98%, but admissions into higher education have achieved 80% of the relevant age group. Schleicher believes that this growth reflects quality, not just quantity, which he contends the top PISA ranking of Shanghai's secondary education confirms. Schleicher believes that China has also expanded school access and has moved away from learning by rote. According to Schleicher, Russia performs well in rote-based assessments, but not in PISA, whereas China does well in both rote-based and  broader assessments. Denmark. University of Copenhagen Professor Svend Kreiner, who examined in detail PISA's 2006 reading results, noted that in 2006 only about ten percent of the students who took part in PISA were tested on all 28 reading questions. "This in itself is ridiculous," Kreiner told Stewart. "Most people don't know that half of the students taking part in PISA (2006) do not respond to any reading item at all. Despite that, PISA assigns reading scores to these children." Finland. The stable, high marks of Finnish students have attracted a lot of attention. According to Hannu Simola  the results reflect a paradoxical mix of progressive policies implemented through a rather conservative pedagogic setting, where the high levels of teachers' academic preparation, social status, professionalism and motivation for the job are concomitant with the adherence to traditional roles and methods by both teachers and pupils in Finland's changing, but still quite paternalistic culture. Others advance Finland's low poverty rate as a reason for its success. Finnish education reformer Pasi Sahlberg attributes Finland's high educational achievements to its emphasis on social and educational equality and stress on cooperation and collaboration, as opposed to the competition among  teachers and schools that prevails in other nations. India. Of the 74 countries tested in the PISA 2009 cycle including the "+" nations, the two Indian states came up 72nd and 73rd out of 74 in both reading and mathematics, and 73rd and 74th in science. India's poor performance may not be linguistic as some suggested. 12.87% of US students, for example, indicated that the language of the test differed from the language spoken at home. while 30.77% of Himachal Pradesh students indicated that the language of the test differed from the language spoken at home, a significantly higher  percent However, unlike American students, those Indian students with a different language at home did better on the PISA test than those with the same language. India's poor performance on the PISA test is consistent with India's poor performance in the only other instance when India's government allowed an international organization to test its students and consistent with India's own testing of its elite students in a study titled "Student Learning in the Metros 2006." These studies were conducted using TIMSS questions. The poor result in PISA was greeted with dismay in the Indian media. The  BBC reported that as of 2008, only 15% of India's students reach high school. Italy / South Tyrol. In 2003 South Tyrol (Provincia Autonoma di Bolzano / Autonome Provinz Bozen), a predominantly German-speaking province in the north of Italy, took part in the PISA project for the first time in order to have a regional result as an "adjudicated region". In the rest of Italy PISA is conducted by INVALSI (Istituto nazionale per la valutazione del sistema educativo di istruzione e di formazione), a formally independent research institution affiliated to the Ministry of Education, whereas in South Tyrol PISA was  carried out by the regional Education Authority itself ("Intendenza scolastica" / "Schulamt", since 2018 renamed into "Bildungsdirektion"), which is part of the South Tyrolean regional government. At the end of 2004, in the months prior to the announcement of the test results, the regional Education Authority in Bolzano / Bozen downplayed the validity of the PISA assessment and commissioned alternative school evaluations, preparing the public for a mediocre test result. According to the official PISA report 2003, however, South Tyrol seemed to even beat the PISA world champion Finland. Critique Right from the beginning, there was scepticism as to how South Tyrol  succeeded in outdoing the neighbouring Italian and Austrian provinces. On the front page of its weekend edition for 29/30 January 2005, the South Tyrolean newspaper "Neue Südtiroler Tageszeitung" published a harsh critique and revealed that the South Tyrolean Education Authority had secretly eliminated more than 300 students from the 1500 students officially drawn as South Tyrolean test sample by the PISA Consortium, and soon more inconsistencies were to surface: Comparison with similar assessments The stunning South Tyrolean 2003 PISA results can hardly be reconciled with similar high school evaluations, which were not conducted or influenced by the South Tyrolean Education  Authority itself. Three international or national large scale assessment projects painted a gloomy picture of the South Tyrolean students’ performance. United States. Two studies have compared high achievers in mathematics on the PISA and those on the U.S. National Assessment of Educational Progress (NAEP). Comparisons were made between those scoring at the "advanced" and "proficient" levels in mathematics on the NAEP with the corresponding performance on the PISA. Overall, 30 nations had higher percentages than the U.S. of students at the "advanced" level of mathematics. The only OECD countries with worse results were Portugal, Greece, Turkey, and Mexico. Six percent  of U.S. students were "advanced" in mathematics compared to 28 percent in Taiwan. The highest ranked state in the U.S. (Massachusetts) was just 15th in the world if it was compared with the nations participating in the PISA. 31 nations had higher percentages of "proficient" students than the U.S. Massachusetts was again the best U.S. state, but it ranked just ninth in the world if compared with the nations participating in the PISA. Comparisons with results for the Trends in International Mathematics and Science Study (TIMSS) appear to give different results—suggesting that the U.S. states actually do better in  world rankings. This can likely be traced to the different material being covered and the United States teaching mathematics in a style less harmonious with the "Realistic Mathematics Education" which forms the basis of the exam. Countries that commonly use this teaching method score higher on PISA, and less highly on TIMSS and other assessments. Poverty. Stephen Krassen, professor emeritus at the University of Southern California, and Mel Riddile of the NASSP attributed the relatively low performance of students in the United States to the country's high rate of child poverty, which exceeds that of other OECD countries. However , individual US schools with poverty rates comparable to Finland's (below 10%), as measured by reduced-price school lunch participation, outperform Finland; and US schools in the 10–24% reduced-price lunch range are not far behind. Reduced school lunch participation is the only available intra-poverty indicator for US schoolchildren. In the United States, schools in locations in which less than 10% of the students qualified for free or reduced-price lunch averaged PISA scores of 551 (higher than any other OECD country). This can be compared with the other OECD countries (which have tabled figures on children living in  relative poverty): Sampling errors. In 2013 Martin Carnoy of the Stanford University Graduate School of Education and Richard Rothstein of the Economic Policy Institute released a report, "What do international tests really show about U.S. student performance?", analyzing the 2009 PISA data base. Their report found that U.S. PISA test scores had been lowered by a sampling error that over-represented adolescents from the most disadvantaged American schools in the test-taking sample. The authors cautioned that international test scores are often "interpreted to show that American students perform poorly when compared to students internationally" and that school reformers then  conclude that "U.S. public education is failing." Such inferences, made before the data has been carefully analyzed, they say, "are too glib" and "may lead policymakers to pursue inappropriate and even harmful reforms." Carnoy and Rothstein observe that in all countries, students from disadvantaged backgrounds perform worse than those from advantaged backgrounds, and the US has a greater percentage of students from disadvantaged backgrounds. The sampling error on the PISA results lowered U.S. scores for 15-year-olds even further, they say. The authors add, however, that in countries such as Finland, the scores of disadvantaged students tends to be  stagnant, whereas in the U.S the scores of disadvantaged students have been steadily rising over time, albeit still lagging behind their those of their more advantaged peers. When the figures are adjusted for social class, the PISA scores of all US students would still remain behind those of the highest scoring countries, nevertheless, the scores of US students of all social backgrounds have shown a trajectory of improvement over time, notably in mathematics, a circumstance PISA's report fails to take into account. Carnoy and Rothstein write that PISA spokesman Schleicher has been quoted saying that "international education benchmarks make  disappointing reading for the U.S." and that "in the U.S. in particular, poverty was destiny. Low-income American students did (and still do) much worse than high-income ones on PISA. But poor kids in Finland and Canada do far better relative to their more privileged peers, despite their disadvantages" (Ripley 2011). Carnoy and Rothstein state that their report's analysis shows Schleicher and Ripley's claims to be untrue. They further fault the way PISA's results have persistently been released to the press before experts have time to evaluate them; and they charge the OECD reports with inconsistency  in explaining such factors as the role of parental education. Carnoy and Rothstein also note with alarm that the US secretary of education Arne Duncan regularly consults with PISA's Andreas Schleicher in formulating educational policy before other experts have been given a chance to analyze the results. Carnoy and Rothstein's report (written before the release of the 2011 database) concludes:We are most certain of this: To make judgments only on the basis of national average scores, on only one test, at only one point in time, without comparing trends on different tests that purport to measure the  same thing, and without disaggregation by social class groups, is the worst possible choice. But, unfortunately, this is how most policymakers and analysts approach the field. The most recent test for which an international database is presently available is PISA, administered in 2009. A database for TIMSS 2011 is scheduled for release in mid-January 2013. In December 2013, PISA will announce results and make data available from its 2012 test administration. Scholars will then be able to dig into TIMSS 2011 and PISA 2012 databases so they can place the publicly promoted average national results in proper context. The  analyses we have presented in this report should caution policymakers to await understanding of this context before drawing conclusions about lessons from TIMSS or PISA assessments.