How are teachers' understandings and practices positioned in discourses of assessment?
Recent curriculum developments in a number of countries have led to the inclusion in formal assessment regimes (leading to 'high stakes' consequences for students and teachers) of a wider variety of methods of assessment, including 'performance' and 'authentic' assessment (e.g. Romberg 1995, DOE 2002, 2005, Broadfoot 1996). There has been a move away from traditional tests and examinations towards tasks that allow more varied, complex responses. This move has simultaneously increased the complexity of the task of assessing student responses-a task generally undertaken by teachers, who must interpret the texts (written, spoken or behavioural) produced by their students in order to evaluate them. This paper reports on a research in progress undertaken with a cohort of B. Ed Hons students (52 students) enrolled at the University of KwaZulu-Natal for a module titled: Assessment in Education. The module is highly theoretical and includes an analysis of assessment policy documents. An initial phase of the study (April 2006) examines teachers' understandings of key discourses in the policy documents through a questionnaire. The second phase of the study (June 2006) examines a selection of teachers' (8 teachers) assessment practices through an analysis of assessment tasks, assessment of student responses and interviews.
What is the dominant assessment discourses (for example a standards-based discourse) in South African policy documents?
How are teachers' understandings and practices positioned in these discourses of assessment?
What kinds of new discourses do teachers' engage with in the module: Assessment in Education?
Do teachers' practices and positioning shift after engagement with the module/ policies?
For the purpose of this presentation this paper will respond to the first two questions as the data in the second phase of the study has not yet been analysed. In this paper I want to argue that progressivism has left teachers with a disabling legacy when it comes to assessment. Teachers' need access to an assessment meta-language which is sensitive to variations in learning contexts and individual differences. Unless teachers' understand and internalize the language and discourses of assessment, the progressive ideals of the new assessment reform will not achieve its intended goals.
Recent curriculum developments in a number of countries have led to the inclusion in formal assessment regimes (leading to 'high stakes' consequences for students and teachers) of a wider variety of methods of assessment, including 'performance' and 'authentic' assessment (e.g. Romberg 1995, DOE 1995, Broadfoot 1996). There has been a move away from traditional tests and examinations towards tasks that allow more varied, complex responses. This move has simultaneously increased the complexity of the task of assessing student responses-a task generally undertaken by teachers, who must interpret the texts (written, spoken or behavioural) produced by their students in order to evaluate them. South Africa has gone through two periods of major curriculum reform producing curriculum documents called: Curriculum 2005 (C2005) in 1997 and The National Curriculum Statements (NCS) in 2002 for the GET phase and in 2003 for the FET phase.
According to Muller (2004), "assessment during the apartheid era was norm-referenced, summative and aggregative" and was characterised by paper-and pencil tests that emphasised academic exercises and the recall of textbook knowledge (Vandeyar and Killen 2003). In Bernsteinian terms assessment was characterised by a strong collection code, which is exemplified by strong boundaries among subjects and between teacher and learner. Assessment of subjects was done independently of each other. Assessment was generally separated from instruction and largely took the form of assessing isolated or fragmented knowledge and skills.
Progressivist pedagogies underpinning C2005 have sought to operate with a 'contentless curriculum'-celebrating individual differences whilst excluding most students from the powerful discourses necessary for success in the wider socio-cultural environment. In Curriculum 2005, the progressive classroom is conceived as operating with a weak classification of subject disciplines through its integrated, 'child centred' curriculum, and an apparently weak classification of the boundaries between teacher (or transmitter of knowledge) and pupil (as acquirer of knowledge)(Bourne 2003: 498).
The NCS released in May 2002 introduced several important changes to proposed assessment practices in schools. For each grade of schooling, there is a set of assessment standards that define the levels of knowledge, skills and attitudes that learners will be required to demonstrate as evidence that they have achieved each phase outcome to an appropriate depth and breadth (conceptual coherence). This means that in each Phase the outcomes remain the same from grade to grade while assessment standards change from grade to grade. The NCS-a streamlined and strengthened version of C2005 has retained its progressive ideals for example a shift from assessing product to process; norm to criterion referenced; summative to diagnostic assessment.
How have teachers responded to the assessment reform In an international comparative perspective on outcomes-based assessment, Combrink (2003) qualitatively assessed the realities and problems related to outcomes-based assessment from an international perspective by interviewing a sample of officials in Australia, New Zealand and America and his findings indicate that a major problem in all three countries was a lack of in-service training or inadequate training which made teachers feel incompetent. Teachers have been reported to lack guidance and /or skills -real or perceived-to implement classroom assessment (Pennycuick 1990, Brown 1991, Bazzini 1993, Emery 1997 as cited in Lubisi, 2000). Another theme that has emerged from literature is that the new assessment increases the workload of teacher (Combrink 2003, Jansen 1999) and that teachers practices may be influenced by time -real or perceived- it takes to design, grade and /or record results of classroom assessment (Pennycuick 1990, Broadfoot et al 1991, Emery 1 997 as cited in Lubisi 2000). According to Torrance (1995), considerable resentment was expressed by teachers in the UK at the extra workload involved in assessment. This study intends to understand how teachers position themselves in the discourses of assessment and argue that teachers need access to the language and discourses in assessment for the progressive ideals of the new curriculum to be realized?
The analysis of data is framed by a post-structural notion of discourse. Discourse is defined as a 'relatively bounded area (sic) of social knowledge' that both constrains and enables how we think about a particular social object or practice (McHoul and Grace, 1993:31). Thus, discourses allow individuals to interpret particular social situations, or phenomena (Pacini-Ketchabaw and Schecter, 2002). Discourses are about what can be said, and thought, but also about who can speak, when, where and with what authority (Ball, 1990). Discourses embody the meaning and use of propositions and words. Thus certain possibilities for thought are constructed. Words are ordered and combined in particular ways and other combinations are displaced or excluded. Discourses get things done, accomplish real tasks, gather authority' (Said 1986:152). Discourse may have the effect of re-distributing 'voice', so that it does not matter what people say or think, only certain voices can be heard as meaningful or authoritative (Ball, 1994:23). Applications of discourse theory allow valuable fine-grained analyses of documents and texts to be undertaken within a broader structural analysis. This approach is also useful in highlighting values and teasing out competing discourses.
Discourse is the key concept of the relationship between power and knowledge (Foucault, 1977). Discourses are, therefore, about what can be said and thought, but also about who can speak, when, where and with what authority. The effects of power narrow the possibilities of discourse and/or obscure its boundaries. Any particular discourse will generate resistance as it encounters competing discourses. Dominant discursive constructions should be given due weight for the ways in which they circulate and the power that they enjoy (Parker, 1997). They exist within what Foucault referred to as regimes of truth, which make any challenges to the 'realities' to which they refer very difficult indeed (Foucault, 1972). A discourse may gain a dominant position over others in the way in which it is strengthened and launched by individuals and institutions. Teachers in the educational process are in a powerful position to do just this. Schools are sites where discourses are reinforced and challenged (Corson, 1998). This paper reports on how teachers' understandings and practices of assessment are positioned in the official policy discourses and what new discourses are created in teachers' understandings and assessment practices. This study reports mainly on two data collection techniques: policy document analysis and questionnaire data. The policy documents for the NCS (GET and FET phases) were analysed for dominant discourses and the questionnaires were analysed for teachers' understandings are positioned in these discourses. 49 out of the 52 questionnaires distributed were returned showing a response rate of 94%. The teachers are from five learning sites in KwaZulu Natal (Empangeni -6 Durban-8; Kokstad-4; Pietermaritzburg-23; and Dundee-8). 33 teachers are Level 1; 8 level 2; 2 deputy principals and 6 principals. 16 of the participants were male and 33 female. Most of the participants had under 15 years of experience of teaching. The majority of teachers teach in the foundation or intermediate phase. 28 out of the 49 teachers have indicated that their school has an assessment policy that has been adapted from the National Policy guidelines. In the main teachers were minimally or not involved at all in developing the school assessment policy.
Among teachers, there is a growing recognition that the lack of explicitness in curriculum policy documents is causing problems in producing satisfactory assessment procedures. The problem is exacerbated by the one week workshops held by the Department of Education to orientate teachers to the new NCS. 16 of the 49 teachers attended these workshops. Teachers' comments on the positive outcomes of the workshop centred around technical aspects of developing learning programmes and lesson plans, recording, emphasis on changing numerical scores by replacing it with rubrics and checklists, methods of assessment, rating learners achievements.
too much in a short pace of time; needed more time and follow up workshops; facilitators not well prepared; some people left the workshop more confused; facilitators not clear about some aspects hence they work-shopped for one week only took 3 days- questions were left unanswered; workshop was too fast paced and superficial, no practical opportunity, increased teacher apathy, does not take context into account; assessment tools and rubrics were not clarified etc. The comments help explain why teachers' understanding of the discourses in assessment are positioned in particular ways. Teachers cannot expect to apply assessment principles they do not understand (Vandeyar and Killan, 2003).
The NCS introduced several important changes to the proposed assessment practices in schools. The most significant change was a shift from the criterion-referenced assessment that was recommended in C2005 to a form of standards-referenced assessment. In each Learning Area in each Phase of schooling there is a set of outcomes that define what learners are expected to achieve for each grade of schooling, there is a set of assessment standards that define the levels of knowledge, skills and attitudes that learners will be required to demonstrate as evidence that they have achieved each phase outcome to an appropriate depth and breadth. This means that in each Phase the outcomes remain the same from grade to grade while assessment standards change from grade to grade. Because these standards are grade specific, they describe how conceptual understanding is meant to progress in each Learning Area (Vandeyar and Killan, 2003).
For the first time in South Africa, teachers are being encouraged to think of outcome attainment as a continuum of possibilities, rather than as a dichotomy. They are no longer being asked to put learners into categories of achieved/not achieved for each outcome. Instead they have to think about how well each learner has achieved each outcome (Vandeyar and Killan, 2003). This is the approach to assessment that had been advocated by Killen (20 00). The National Codes for reporting learner achievement introduced in the NCS for the GET phase still require learners' performance of each outcome for each Grade to be recorded on a four-point scale (exceeded, satisfied, partially satisfied or not satisfied).
This reluctance to move to a fully standards-referenced approach to assessment is also reflected in the requirement that reporting should also be normative and " contain comments on the learners' performance in relation to peers" (Department of Education, 2000, 100). In addition, the tension between progressivism and accountability has negatively facilitated the imposition of external standardized tests for e.g. the grade 9 CTA's which are incompatible both with the classroom experiences of the vast majority of teachers and their learners and the goals of the curriculum. From the questionnaire data, 17 of the 49 teachers did not respond to the question on what they understood by standards based assessment, 3 of these were principals of schools. Multiple understandings, some of which were flawed, emerged from the questionnaire data for example, to assess a learner that can be equal to other learners level at which assessment should be practised process of identifying, gathering and interpreting info about the learners achievements based on certain judgement like when a learner is able to do something setting an assessment according to level of learners- from their own background or an assessment based on what they are familiar with assessment designed by the dept of ed for more than one school e.g.
assess learners on what you intended to assess on supervised and controlled tests task is assessed according to certain pre-determined criteria.
Embedded in teachers' understandings of standards are fairness, context validity; and maintaining standards and accountability of the system through common tests and standardised testing. In all the definition above, none of the teachers alluded to assessment standards that define the levels of knowledge, skills and attitudes that learners will be required to demonstrate as evidence that they have achieved each phase outcome to an appropriate depth and breadth.
clearly indicates that some teachers are positioned in a criterion referenced discourse and have not internalised the policy shift made from criterion referenced to standard referenced assessment.
assessment based on standards assessment that is standardised to achieve outcomes that are set, i.e.
assessment that has standards that move from easy to difficult.
When teachers' understandings are positioned differently from policy then it is unlikely that teachers will be able to apply these assessment principles embedded in the standards based discourse to their practices. Another stark disparity is that at the level of policy documents subject assessment guidelines (DOE, 2005 for example in the Life Sciences) do not mention the word 'assessment standards' throughout the document but is implicit in the description of competence and skills. The official policy discourse of assessment standards are not reinforced in guidelines that develop from the curriculum statements.
C2005 and the RNCS showed a shift from the performance model of education (outgoing curriculum under apartheid) to a competence based model. According to Bernstein (1996) performance model serves primarily economic goals and are considered instrumental. Assessment in this model aims to identify what learners have not acquired, i.e., 'what is missing in the product' (p60) and teacher professionalism is grounded in grading procedures. By contrast, in competence models time is not explicitly punctuated (sequencing and pacing is more flexible), assessment emphasizes what is present in the acquirer's products.
Competence modes moves away from the concept of deficit (assessment in terms of what is absent rather than what is present). It emphasizes empowerment, for example in the RNCS statements there exist a dominant discourse of liberal/progressive empowerment in advocating learner-centredness; cultural empowerment through redress and political empowerment through the development of a democratic citizenry . In the assessment section of the RNCS policy there exist explicit competence descriptions that embed the notion of empowerment, for example, 'develop justifiable and responsible positions on the influences of different beliefs, attitudes and values in various communities' (DOE, 2005- Life Sciences subject statements, p51).
Data from the questionnaire reveals that teachers' understandings of competence are embedded in the performance model- with a strong norm-referenced discourse emerging, for example, to describe some-one comparing with others to see whether learners meet the required standard in order to progress to the next grade when a teacher is able to describe learners competence against each other in a class e.g., when answering questions asked by the teacher levels of performance compared to other people e.g.
What are the theoretical tensions that exist in locating assessment in the RNCS within the competence model Where do predetermined outcomes fit within a competence model of education The NCS explicitly state?
Since the outcomes based approach involves setting benchmarks, it inevitably incorporates the concept of deficit, which is contradictory to competence models (Harley and Parker, 1999). With the specification of explicit assessment standards and competence levels, the benchmark has been set and learners are assessed on what they are able to acquire - a deficit notion.
The process whereby the information obtained through assessment is interpreted to make judgements about a learner's level of competence. It includes a consideration of learner's attitudes and values. (Assessment Policy Act, 1998).
The official policy discourse of evaluation includes collection of adequate evidence, interpretation, judgement compared a benchmark: assessment standards and competence description and a summative assessment of a learner's worth. It considers assessment of attitudes and behaviors, not only cognitive dimensions of learning. What does this discourse do It suggests to teachers (or attempts to impose on them) practices and criteria of assessment to be measured against competence descriptions and assessment standards, and provides them with arguments to make sense, justify and explain their practices to pupils, parents, official moderators, and interested others. Following Bernstein (1999), we can say that this discourse attempts to regulate teachers' practices of assessment?
The discourse of evaluation is not unitary but consists of an official discourse and other unofficial discourses (Lerman, 2001). What are other discourses that teachers' draw on in their understanding of evaluation A strong regulatory discourse linked to monitoring and accountability is evident?
In this position, the teachers speak with the voice of an unofficial discourse, adopting the values of a traditional pedagogic discourse (performance model).
The data reveals that the standards based discourse, competency based discourse and the discourse of evaluation as progressive ideals carry inherent tensions in the way it is articulated in policy and teachers' understandings and practices thereof. Contradictory demands and alternate deficit discourses emerge when the teacher for example, uses the resources of a competence model within assessment structures that impose strategies that are more consistent with a performance model. Unless teachers engage with the official policy discourse, have access to such a discourse (through workshops and in-service training) and develop a meta-language around the discourses embedded in the assessment policy documents the progressive ideals of the curriculum will not be realized in practice. It require teachers to engage with analysis of the construction of the discourse; attempting to explain what interests such a discourse serves (or what changes in society give rise to such a discourse) and how such a discourse resonates with their present practice. Questions that need to be grounded in such an engagement are: what social factors promote dominant deficient discourses, what class assumptions underpin these discourses, how can these discourses be de-constructed to see whether it serves to empower or disempower teachers and pupils.
Ball, S.J. (Ed) (1990) Foucault and Education. Disciplines and knowledge. London: Routledge. -Ball, S.J. (1990) Politics and Policy Making in Education: Explorations in Policy Sociology. London: Routledge.
Bernstein, B. (1996) Pedagogy, Symbolic Control and Identity. London: Taylor and Francis.
Broadfoot, P.M. (1996) Education, Assessment and Society. A Sociological Analysis. Buckingham: Open University Press.
Bourne, J. (2003) "Vertical Discourse: the role of the teacher in the transmission and acquisition of decontextualised language." In European Educational Research Journal, Volume 2, Number 4, 2003.
Implications for South Africa. In Perspectives in Education, Volume 21(1), March 2003.
Corson, D. (1998) Changing Education for Diversity. Philadelphia: Open University Press.
Department of Education. (2000) Report of the Review Committee - Executive summary, 31 May. Pretoria: Department of Education.
Department of Education. (2002) Revised National Curriculum Statement R-9 (Schools). Pretoria: Department of Education.
Department of Education. (2005) National Curriculum Statement FET (Schools).
Foucault, M. (1972) The archaeology of knowledge (A.M. Sheridan Smith, Trans.).
Foucault, M. (1977) Discipline and Punish: the birth of the prison. New York: Vintage Books.
Harley, K. & Parker, B. (1999) "Integrating Differences: Implications of an Outcomes-based National Qualifications Framework for Roles and Competencies of Teachers." In Jansen, J and Christie P, (Eds), Changing Curriculum: Studies on Outcomes-based Education in South Africa. Kenwyn: Juta & Co Ltd.
Killen, R. (2000) 'Standards-referenced assessment: Linking outcomes, assessment and reporting'. Keynote address at the Annual Conference of the Association for the Study of Evaluation in Education in Southern Africa, Port Elizabeth, South Africa, 26-29 September.
Lerman, S. (2001) A review of research perspectives on mathematics teacher education. In: F-L Lin & T. Cooney (Eds) Making Sense of Mathematics Teacher Education. Dordrecht, Kluwer.
Lubisi, R.C. (2000) "An Investigation Into Mathematics Teachers' Perceptions and Practices of Classroom Assessment in South African Lower Secondary Schools." PhD Thesis, School of Education: University of Nottingham.
McHoul, A. & Grace, W. (1993) A Foucault Primer: discourse, power and the subject. London: UCL Press.
Muller, J. (2004) "Assessment, qualifications and the National Qualifications Framework in South African Schooling." In Chisholm, L (Ed) Changing Class, Education and Social Change in Post - Apartheid South Africa. Cape Town: HSRC Press.
Pacini-Ketchabaw, V. & Schecter, S. (2002) 'Engaging the discourse of diversity: Educator's frameworks for working with linguistic and cultural difference'. In Contemporary Issues in Early Childhood, 3(3), 400-414.
Parker, I. (Ed.) (1997). Social constructionism, discourse and realism. Sage: London.
Republic of South Africa: Assessment Policy Act (1998). Assessment Policy in the general education and training band, GRADES R TO 9 AND ABET Government Gazette Vol. 402, No.
Romberg, T. A. (Ed). (1995) Reform in School Mathematics and Authentic Assessment (New York, SUNY Press).
Said, E. (1986). Foucault and the imagination of power. In D. Hoy, (ed), Foucault: A Critical Reader. Oxford: Blackwell.
Vandeyar, S. and Killen, R. (2003) "Has curriculum reform in South Africa really changed assessment practices, and what promise does the revised National Curriculum Statement hold" In Perspectives in Education, Volume 21(1), March 2003?