How Do we Know What to do? The Importance of Experimentation in Designing Programmes and Policies to Improve South African Schooling

The quality of schooling in South Africa is a continuing cause for concern. There is broad consensus around the goals of reform, but not about how these goals might or should be achieved. This is because there is little evidence on the effects of alternative reforms allowing assessment of their efficacy and utility. This paper argues for much more experimentation in educational reform, i.e. running experimental reforms and monitoring closely their effects. Some evidence is available from projects and programmes funded by foreign donors, business donors or occasionally provincial education departments. This paper examines some of these, focusing especially on the design and evaluation methodology of the Imbewu programme run in the Eastern Cape funded through British aid. Unfortunately, little attempt has been made to standardise or collate research on the effects of these initiatives. How Do We Learn About Improving Educational Outcomes? 1 There is a substantial body of work documenting the poor performance of South Africa’s public schools, and a similarly substantial body of evidence on the main causes of this poor performance. Too many South African children receive a poor education, and hence poor opportunities in life, not so much because of a shortage of resources invested in schooling, but rather because the considerable resources that are invested are invested very inefficiently. There are too few textbooks in classrooms, teachers are poorly motivated and sometimes incompetent, and management is all too often deficient. Many children bring considerable disadvantages with them to school, on account of their family background and neighbourhood environment. The schooling they receive is 1 This Working Paper comprises Chapter 6 of a Report commissioned by the Standing Committee on Public Accounts, in the South African Parliament, on Making An Informed Investment: Improving the Value of Public Expenditure in Primary and Secondary Schooling in South Africa. Research for the report was funded by the European Commission. 2 insufficient to overcome these disadvantages, ensuring that inequalities in society are reproduced. Disadvantaged children get the worst opportunities. All of this is widely recognised within national and provincial education departments. The various education departments have identified what they aim to achieve. Now, as the Joint Education Trust has suggested, the question is not so much ‘what to do? ’ as ‘how to do it? ’ ‘During the first five years of South Africa’s first democratic government the Department of Education has established a coherent policy framework for the reform of the public schooling system. The priority now is to implement these policies in order to improve the effectiveness of schools in delivering quality education. The problem is no longer what to do – this has been established by much consultation, research and parliamentary debate, and regulated and legislated in numerous policy statements, White Papers and Government Gazettes. The problem now is how to implement these policies. ’ (JET et al., 1999: 8) The activities that attract most attention in the media (in large part because the politicians involved seek media coverage) are likely to be of secondary importance. The Minister of Education visits schools and admonishes teachers and students for negligence. Former President Nelson Mandela presides over the opening of new school buildings in rural areas, paid for by big business. For sure, both the culture of schooling and physical infrastructure are important, but it is far from clear what effect admonitions and new buildings have on what actually happens inside the classroom, and whether student performance is significantly improved. Elected political leaders and appointed officials have to make decisions on how to allocate scarce resources, with the goal (we hope) of maximising the benefit to children in terms of the quality of education. They do so without adequate information on the costs and benefits of alternative uses of resources. There is a pressing need to generate such data. This working paper is concerned with developing the procedures required to identify the benefits and costs of different policies. How can we know which policies ‘work’, and which do not, in terms of building students’ skills? What can we do – and what should we be doing – so that we are confident that public resources are being put to the best possible use?