School league tables: Revealing or misleading?

Each year parents are encouraged to use school league tables to help choose a secondary school for their children. But George Leckie and Harvey Goldstein argue that such comparisons are crude and ultimately misleading.

League tables: good or bad?Choosing a secondary school can be a daunting prospect for parents. A ‘wrong’ decision can have a lasting impact on a child’s future, and parents often go to great lengths to make sure their son or daughter ends up in the best school they can access. One of the most trumpeted measures of a school’s success, and an indicator parents are encouraged to pay close attention to, is a school’s position in various performance tables.

"There is a seven-year gap between the available information and what parents want to know...many schools performing in the top quarter of schools seven years ago perform in the bottom half today"

The most high profile secondary school league tables are those publishing the percentage of children getting five A* to C grades at GCSE. However, this performance measure is an unfair way to compare schools as it says more about differences in schools’ intakes than it does about differences in their quality. Such league tables are also an unreliable way to compare schools as, with only around 200 students per school sitting GCSE exams each year, it is a ‘noisy’ measure of how well schools are truly performing.

The so-called ‘contextual value-added’ (CVA) measure, published by the Government until it was abandoned in 2011*, was considered to be a better measure for comparing schools as it adjusted for differences in schools’ intakes and was published with error bars to communicate the statistical uncertainty in schools’ performances.

Past, present, and future

Our research3-6 has focused on a fundamental problem in using secondary school league tables, value-added or otherwise, for school choice: school league tables report the past performance of schools, based on children who have just taken their GCSE exams, whereas what parents want to know is how schools will perform in the future when their own children take the exams.

Consider parents who chose a secondary school for their child in autumn 2012. Their child will enter school in autumn 2013 and will take their GCSE exams in 2018. Thus, the information parents need when choosing is how schools are predicted to perform in 2018. However, the most recent information available to them is the school league table for how schools performed in 2011.

There is therefore a seven-year gap between the available information and what parents want to know. Clearly, the more schools’ performances change over a seven-year period, the less reliable league tables will be as a guide to schools’ future performances.

The evidence

To examine how serious a problem this issue is, we first examined the official CVA school league table data. We found that many schools which were performing in the top quarter of schools seven years ago perform in the bottom half today. We then predicted schools’ current performances based only on data from seven years previously. We found that these predictions were so imprecise that almost no schools could be distinguished reliably from one another.

This means that, for choosing a school, the league tables carry very little useful information and, by not communicating this fundamental problem to parents, they are very likely to mislead. More should be done by the Government and the media to communicate this important limitation to parents.

George Leckie is a Lecturer in Social Statistics at the Graduate School of Education, University of Bristol. Harvey Goldstein is a Professor in Social Statistics at the Graduate School of Education, University of Bristol.

*UPDATE 15.30 Thursday 24 January: This article was updated to clarify that from 2011 CVA was no longer used, though was in place when this research was originally carried out.

References: 
  1. Department for Education (2012). Performance tables. Retrieved 21 December 2012.
  2. Department for Education (2010). ‘Key Stage 2 to Key Stage 4 (KS2-KS4) Contextual Value Added Measure (CVA) Including English and Maths’ in 2005 Key Stage 4 Contextual Value Added (CVA) Pilot. Retrieved 21 December 2012.
  3. Leckie, G. and Goldstein, H. (2009a). School league tables: Are they any good for choosing schools? Research in Public Policy, Bulletin of the Centre for Market and Public Organisation, 8, 6-9.
  4.  Leckie, G. and Goldstein, H. (2009b) The limitations of using school league tables to inform school choice. Journal of the Royal Statistical Society: Series A, 172, 835-851.
  5. Leckie, G. and Goldstein, H. (2011). Understanding uncertainty in school league tables. Fiscal Studies, 32, 207-224.
  6. Goldstein, H. and Leckie, G. (2008). School league tables: what can they really tell us? Significance, 5, 67-69.

Share this page