Abolishing marksism

Why use an 18th Century approach to assessing exam answers when we have a 21st Century alternative, asks Alastair Pollitt

MarksDo we have to mark exams? Not necessarily: there is an alternative that relies on the professional knowledge of teachers rather than the technical precision of trained markers, yet can assess pupils’ work far more reliably than marking.

Marking has not been around forever. It was invented in Cambridge University in 1792 to solve a specific problem: resolving disagreement amongst multiple examiners and eliminating favouritism when the examiners were judging their own students. It spread rapidly, initially in the new American colleges, because it also allowed examiners to divide up the marking chore. But the price was high: to make a test 'fair' when only one or two examiners looked at each pupil's work demanded objectivity, more and more strict mark schemes and, eventually, multiple choice exams in everything from Maths to History and even English Writing.

ACJ for assessment

Let’s not go any further down that road. Consider using ACJ instead - Adaptive Comparative Judgement. The examiners (who need to be experienced teachers) are asked to judge pieces of work by two pupils – not absolutely, against complex written criteria, but comparatively, by looking at them and deciding which is the ‘better’ one. This decision takes much less time than detailed marking does; that means we can collect far more than just one decision from one examiner on each pupil’s work, and by cleverly picking which pairs to offer the judges we can generate a very reliable set of scores quite quickly – that cleverness is the adaptive bit of the name.

It’s like sorting the pupils into a rank order, but supported by a rigorous mathematical theory of measurement. Nothing seems any different to students when they take the test, but the examiners would need a computer as the two ‘scripts’ are presented to them online. However, with holistic ACJ, we free ourselves from the yoke imposed by the demand for traditional marking and can design exams to be far more authentic tests of the skills and behaviour we want to develop in pupils. We can set real design tasks in Design and Technology, real projects in Geography, real investigations in Physics – and always score them quite quickly and reliably.

Every teacher in the country could take part. If they entered 20 candidates for an exam, each teacher would need to make about 120 comparisons of work from other schools, amounting to perhaps a day’s work. Assessment would once more belong to the profession.

Which is the better drawing?

Peer assessment is formative

Furthermore, what if we let the pupils make the judgements themselves? I can’t imagine a better way to develop the skills of narrative writing, or of planning and pursuing scientific or historical research, or of acting or singing, than by training pupils to recognise better quality when they see or hear it. Let pupils debate and defend their decisions: many teachers know the power of this implicit learning of what it means to be better. From this kind of peer assessment grows the critical self-awareness that is essential for high quality work. It’s not ‘assessment for learning’ – it’s assessment that is learning.

We have proved ACJ can be effective in Design and Technology, Geography, Science, Mathematics, Modern Foreign Languages, and in several types of English writing. A number of countries including Australia, Sweden, Israel and the USA are exploring it, with students from primary school through to university.

Examiner feedback

Finally, what do markers and teachers think of it? When we asked the participants in our study of primary school writing if they would like to use ACJ or marking in future:

  • 19 preferred ACJ
  • 4 preferred ACJ but with ‘marker training’
  • 2 preferred ACJ but expressed some reservations
  • 2 stated that both should be used
  • 0 chose marking

Yes, even National Curriculum English could be scored with very high reliability with ACJ – and the teachers liked it!

Alastair Pollitt is an independent consultant in the field of educational assessment, and in 2007 established CamExam, which offers research and training on formal assessment. He is also a member of CERP’s Advisory Group.

References: 
  1. Pollitt, A. (2012). The method of Adaptive Comparative Judgement, Assessment in Education: Principles, Policy & Practice, Vol.19, Iss. 2.

Share this page