Source document: https://docs-lawep.s3.us-east-2.amazonaws.com/1697631570497.pdf (Australia)
Professor Patrick Griffin Executive Director, Assessment and Teaching 21st‐Century Skills Project, Associate Dean Director, Assessment Research Centre Melbourne Graduate School of Education.

Committee Secretary Senate Education, Employment and Workplace Relations Committees PO Box 6100 Parliament House Canberra ACT 2600 Australia.
Dear Secretary, Re: The effectiveness of the National Assessment Program ‐ Literacy and Numeracy I am writing this personal submission and as Director of the Assessment Research Centre, Melbourne Graduate School of Education to address the terms of reference (c) through (f).
c) NAPLAN's impact on teaching and student learning practices; d) the impact on teaching and student learning practices of publishing NAPLAN test results on the MySchool website e) potential improvements to the program, to improve student learning and assessment; f) international best practice for standardised testing, and international case studies about the introduction of standardised testing; Australia occupies a unique space in the assessment of student learning outcomes because of the recent project “Assessment and Teaching of Twenty First Century Skills (ATC21S)”.
The project has developed methods of online assessment of collaborative problem‐solving and learning through social media or digital networks that as important implications for ACARA and for NAPLAN.
The project directorate has enjoyed collaborations with peak organisations such as Education Services Australia, the Australian Institute of Teaching and School Leadership, the Australian Curriculum and Assessment and Reporting Authority, in order to build a collaborative approach to the assessment of the general capabilities of the Australian curriculum through the context of the Key learning areas of science and mathematics and a humanity subject.
The ground breaking research offers a unique opportunity to assess a broader range of the general capabilities than literacy and Melbourne Graduate School of Education The University of Melbourne Victoria 3010 Australia T: +61 3 8344 8285 F: +61 3 8344 8529 W: www.education.unimelb.edu.au The effectiveness of the National Assessment Program - Literacy and NumeracySubmission 20 numeracy and at the same time gain an estimate of performance and growth in the key learning areas which contextualise the assessed general capability.
If Australia is to realise the prime minister’s goal of the top five position in PISA of 2025 a generation of school students needs to be developed in the skills of the 21st century.

We have been influential in convincing the OECD to include collaborative problem‐solving in Pisa of 2015 and to consider a high‐technology option of this in 2018.

Australia has had opportunity because of its unique possession of world leading expertise in these areas.
The project directorate at the University of Melbourne have been approached by research organisations in the United States, Singapore, South Korea, Japan and Russia as well as Finland, South Africa and a province in China.

These countries and research organisations are anxious to know how they can take the work forward, as are the three companies that helped to lead and fund the development of the new assessments ‐ Microsoft, Intel and Cisco.
The materials we have developed are the only materials of a kind in the world.

The project has helped to develop a process for interpreting learning analytics which are pertinent for NAPLAN online.

This includes procedures that allow background activity to be monitored while students are solving problems online.

This work has been presented in several national and international conferences and will soon be published in international journals and research volumes.
Informal discussions have been held with ACARA about the background activity log of student process data and the possibility of a sample study to determine whether or not such analytics would be appropriate for background monitoring in the NAPLAN testing.

It would enable the collection of much more information than correct/incorrect responses and the interpretation of those particular skills involved.
It would enable an understanding of the thinking process that students follow while solving problems and to provide this additional data in reports via Myschool and elsewhere to inform both teachers and students regarding their performance in NAPLAN.

I emphasise that this is only available once the NAPLAN goes online in 2016.

However the intervening period provides an opportunity to explore the possibility of this method of interpretation prior to 2016.
Australia would be the only country able to undertake this form of monitoring and to benefit from the way it changes the nature of online testing– the general approach of NAPLAN – in a permanent and exciting way.

It enables test items to be used to monitor more comprehensively the way in which a student is thinking, acting and solving problems than a multiple‐choice test does on its own.
c) NAPLAN's impact on teaching and student learning practices; d) the impact on teaching and student learning practices of publishing NAPLAN test results on the MySchool website; In a second and related area of research regarding the general capabilities of literacy, numeracy and critical thinking our research has uncovered an interesting and disturbing outcome.
In a sample of almost 37,000 students from the Catholic, State and Independent schools we have explored the growth of students in reading comprehension, mathematics and problem‐solving.
The data was analysed to compare the growth rates of lower and Melbourne Graduate School of Education The University of Melbourne Victoria 3010 Australia T: +61 3 8344 8285 F: +61 3 8344 8529 W: www.education.unimelb.edu.au The effectiveness of the National Assessment Program - Literacy and NumeracySubmission 20 upper quartile groups of students.

Students of low capacity in the lower quartile provide almost all of the evidence of growth.
Students of high capacity in the upper quartile produced almost flat line graphs over periods of 6 to 12 months of instruction.

This effect is identified in both reading and mathematics and has been replicated among primary school NAPLAN data from high performing catholic primary schools in the northern region of the Melbourne arch diocese.

When this is considered in the light of the goal that the Prime Minister has set for the top five in 2025 there is an urgent need to redress the situation in schools.
Our results mirror a disturbing trend in Australian PISA data.

The top 10% of students in Australia illustrate a decline in performance over the period of 2003 to 2009 and there is reason to believe that this will continue in the 2012 results.

Australian performance in international study of reading supports the conclusion that standards of reading comprehension in Australia are in decline.

In addition the national testing (NAPLAN) is showing an alarming lack of growth over the 4 to 5 years of operation.
Flat line charts are prevalent.

The Catholic Education Office of Melbourne has attempted to replicate our results in high performing Catholic schools.

They have found a similar decline in the performance of high‐capacity students.

The evidence of a failure at a national level to realise the learning potential of a high‐ capacity students is becoming overwhelming.

No country can afford the situation to remain unchallenged.

We have several hypotheses regarding the possible explanations for this effect.
It is possible that teachers are not concerned about high performing students.

Their energies and efforts focus on students who are at risk of failure.

We hope and expect that this attention given to the low achieving students should continue and be rewarded amongst teachers.

The National School partnerships program has helped students and teachers to realise greater gains of learning amongst the lower quartile students.
It is imperative that this impact is maintained and that these students’ results are reinforced and schools are encouraged to continue.

However the lack of growth amongst the upper quartile or high‐capacity students is a problem that this nation cannot afford.

Logically high‐capacity students should be able to improve at rates higher than those achieved by the low capacity students.

Economically the country is not realising one of its greatest assets.
There needs to be a systematic and national approach to redressing such an issue.

We’ve also accumulated evidence that suggests teachers are reluctant to teach students at high levels of performance.

In addition it appears that teachers are unaware of strategies and resources that they can use to teach students at high levels of performance in both mathematics and reading comprehension.

It also appears (surprisingly ) that many teachers lack the discipline skills in reading comprehension and mathematics.
The combination of these issues suggests that there is a need to intervene with teacher education at both pre‐ service and in‐service levels if NAPLAN results are to improve.

The emphasis on improving low‐achieving students and the reluctance or inability to help students at high levels of competence suggests that a deficit model is being promulgated at schools and teacher education institutions.
Melbourne Graduate School of Education The University of Melbourne Victoria 3010 Australia T: +61 3 8344 8285 F: +61 3 8344 8529 W: www.education.unimelb.edu.au The effectiveness of the National Assessment Program - Literacy and NumeracySubmission 20 This evidence points to an urgent need to redress the problem of student performance in general capabilities of the Australian curriculum in particular those that are related to literacy, numeracy, ICT literacy, critical thinking, personal and social competence which are all relevant to the measures that we have been developing and could be incorporated into NAPLAN.
Recommendations: 1.

ACARA to explore the possibility of in‐depth analysis of students thinking while solving NAPLAN style interactive items when the test is delivered online.

2.

That reporting on the MySchool website examine and display the growth rates of both upper and lower quartiles.

3.

That financial encouragement be given to schools on the basis of NAPLAN results where growth rates exceed an effect size of 0.4 for both upper and lower quartile students.
This will encourage all students’ growth to be monitored and that growth rates of schools will not be exaggerated on the basis of the lower quartile gains.

4.

That teacher impact on students in the upper quartile be given increased emphasis.

It may be necessary to further explore pre‐service teacher education, in‐service teacher education and the production of materials and resources that enable teachers to have a direct impact on all students through the use of assessment data.

5.
5.

Additional work needs to be taken through the states and systems of education to ensure that teacher education and professional development programs address the issue of unrealised potential amongst high‐capacity students.

This in effect is a national economic situation that cannot be tolerated if Australia is to maximise the use of the intellectual capacity of our students.

6.
6.

Teachers should be encouraged to maintain their outstanding efforts and results that they have achieved with the lower quartile students.

These students are identified by teachers as being at risk and their development will be beneficial to the Australian economy and social life if they have the basic skills.

We need to guard against pendulum swing where the lowest capacity students are left behind.

7.

The rhetoric of “close the gap” has been misinterpreted.
It is true that there is a big gap between the performance levels of students of different socio‐economic, racial, gender and language backgrounds.

This should not be taken for granted as permanent situations.

Teachers’ education must include the development of professional judgement about varying teaching strategies and student activities to take into account these background characteristics of students.

The rhetoric should be “improvement for all”.


Melbourne Graduate School of Education The University of Melbourne Victoria 3010 Australia T: +61 3 8344 8285 F: +61 3 8344 8529 W: www.education.unimelb.edu.au The effectiveness of the National Assessment Program - Literacy and NumeracySubmission 20 Patrick Griffin Executive Director, Assessment and Teaching 21st‐Century Skills Project, Associate Dean Director, Assessment Research Centre Melbourne Graduate School of Education.
Melbourne Graduate School of Education The University of Melbourne Victoria 3010 Australia T: +61 3 8344 8285 F: +61 3 8344 8529 W: www.education.unimelb.edu.au The effectiveness of the National Assessment Program - Literacy and NumeracySubmission 20