Valuing teacher judgement

The public needs a better understanding of the role teachers play in assessment practice, says Val Klenowski.

Teacher and students with microscopeInternational developments in the use of standards for assessment and accountability have placed new demands on teachers, students and parents. As a result, how measures of quality are communicated in policy, when they are represented as standards, how they are promulgated, and how they are used in practice, by whom and for what purposes, are now central to an understanding of judgement in current assessment practice.

This means that the role of judgement in assessment – who controls this field, and whose judgement counts – has gained critical attention in the testing domain.

It is important to improve public understanding of the implications of these standards-driven reforms for teachers who are making judgements about the quality of student work. Quality processes are intended to improve the dependability of teacher judgement, and should therefore build trust in teacher judgement among teachers, parents, students and the public. Ironically, teacher judgement has remained fundamental even when standards-driven reforms have been initiated, yet research in this area continues to be limited (Gill & Bramley, 2013). The research that has been conducted on teacher judgement has focused more on inter-rater reliability in relation to the training of examiners (Baird, Greatorex & Bell, 2004; Johnson, Penny & Gordon, 2001; Suto & Greatorex, 2008). It is therefore important to raise awareness, and to inform the public of the significance of teacher judgement in the use of standards for assessment. The consequences of adopting a standards-driven approach to educational change are often under-estimated, and their unintended effects are often not fully understood by policy makers and the public, including parents.

I maintain that the teacher’s role remains central to policy for the improvement of the quality of education and educational standards.

Today, governments are hungry for information. They are propelled to implement standards-driven reforms by perceived declining scores in international analyses such as PISA and TIMSS. Governments have responded to the PISA shock with centralised curriculum development, increased testing, and a loss of faith in teachers’ judgements. The media too has seized on ‘numbers’, especially declining scores in international tables, to present a view of education that is distorted or in a state of crisis. The belief that tests and examinations are the only objective and reliable representations of student achievement has in turn become accepted by the public and politicians. These reforms have occurred quickly. According to Broadfoot and Black (2004, p.9), “decisions about assessment procedures – particularly those concerning ‘high-stakes’ testing of various kinds – are as often based on perceived political appeal as they are on a systematic knowledge of the scientific evidence concerning fitness for purpose”. If national testing programs are to improve outcomes, as distinct from reporting them, then the importance of the teacher rather than the test as the primary agent of change should be acknowledged. The preoccupation in the media with results, and the comparative analysis of school performance, reduce public understanding of the contextual conditions and sociocultural influences that impact on students’ learning and achievement.

Emerging issues in a standards-driven context

Written standards require interpretation and application by a community of practice. Using them for the first time is challenging for teachers, as their confidence about using them develops over time. The dependability of teachers’ judgements is sustained through moderation (Harlen, 2005) and it is in the conversations that teachers have during this process that their understanding is developed.

When teachers make a judgement of student work, they draw on a variety of intellectual and experiential resources, such as a superior knowledge of subject content, an understanding of the criteria and standards appropriate to the task, evaluative skills in making judgements about students’ efforts, dispositions towards teaching and towards learners to help them develop and improve, and a concern for feedback on their own judgements (Sadler, 1998, p. 80–82). This means that the provision of standards, and annotated samples of student work, is necessary but insufficient for teachers to develop consistency in their judgements of student work and in their standards-referenced judgements (Klenowski & Wyatt-Smith, 2013). As teachers come together in moderation meetings and articulate their interpretations of the evidence in the student work in relation to the standards, they develop confidence, and understanding of how the standards are used.

Given the growing global trend for using standards not just for accountability but also for the purpose of improving learning, clarity about how standards might be used to achieve these purposes requires policy and research support (Newton, 2007).

The increased cognitive demands of a futures-oriented curriculum, and the more open-ended nature of contemporary assessment task, suggests that a more holistic approach to judgement is called for, involving a representation of standards as continua. Ensuring that standards are applied in a valid and reliable way when adopting a holistic approach will entail teachers themselves engaging in moderation.

If a standards-driven approach to reform is to achieve the intended goals of improvement towards excellence in learning and teaching, then investment in the teacher workforce is required to support a more balanced approach to assessment that recognizes and supports teacher judgement. This approach would regard teachers as the primary change agents, and accept that their judgement practices are integral to assessment and to expectations of quality performance.  Teachers are best placed to identify important steps for students to improve in their learning, and to develop useful insights about changing pedagogy to meet a student’s learning needs.

Judgement is at the very heart of assessment. It is time for policy officers, and those outside the specialist assessment community, particularly parents, to be more informed of the value and importance of the teacher’s role in judgement practice, and what this implies for policy and resource support to achieve improved learning and teaching through the effective use of standards.

Val Klenowski is a professor in the Faculty of Education at Queensland University of Technology, Brisbane, Australia.

References: 
  1. Baird, J., Greatorex, J., & Bell, J. F. (2004). What makes marking reliable? Experiments with UK examinations. Assessment in Education: Principles, Policy and Practice, 11(3), 331–348.
  2. Broadfoot, P., & Black, P. (2004). Refining assessment? The first ten years of assessment in education. Assessment in Education: Principles, Policy and Practice, 11(1), 7–27.
  3. Cooksey, R. W., Freebody, P., & Wyatt-Smith, C. (2007). Assessment as judgment-in-context: Analysing how teachers evaluate students’ writing. Educational Research and Evaluation, 13(5), 401–34.
  4. Gill, T., & Bramley, T. (2013). How accurate are examiners’ holistic judgements of script quality? Assessment in Education: Principles, Policy and Practice, 20(3), 308–324.
  5. Harlen, W. (2005). Trusting teachers’ judgement: research evidence of the reliability and validity of teachers’ assessment used for summative purposes. Research Papers in Education, 20(3), 245–270.
  6. Johnson, R. L., Penny, J., & Gordon, B. (2001). Score resolution and the interrater reliability of holistic scores in rating essays. Written Communication, 18(2), 229–249.
  7. Klenowski, V., & Wyatt-Smith, C. (2013). Assessment for Education: Standards, Judgement and Moderation. London: Sage.
  8. Newton, P. E. (2007). Clarifying the purposes of educational assessment. Assessment in Education: Principles, Policy and Practice, 14(2), 149–170.
  9. Sadler, D. R. (1998). Formative assessment: revisiting the territory. Assessment in Education: Principles, Policy and Practice, 5(1), 77–84.
  10. Suto, W. M. I., & Greatorex, J. (2008). A quantitative analysis of cognitive strategy usage in the marking of two GCSE examinations. Assessment in Education: Principles, Policy and Practice, 15(1), 73–89.

Share this page