Death metal and the English Bacc

When interpreting education research findings, it is important to exercise caution.

Students workingIn truth, this blog post is born out of anxiety. I have just completed a research paper about the English Baccalaureate and I’m worried. I’m concerned about how my findings may be interpreted and how they may be used. The problem is that research in general, and statistics in particular, can be quite seductive. Even if there are significant limitations to how the results can be applied, they bask in a glimmering sheen of scientific respectability that can be very persuasive. This means that making research available for public consumption can be a surprisingly nerve-racking experience.

Once bitten…

This might all sound a bit egotistical, as if I believe the world is waiting with bated breath for my next pearl of wisdom, but, to be fair, I have previous.

A few years ago, some of my research was blown out of all proportion by the press: ‘How death metal is the soundtrack of child prodigies’ (it must have been a slow news day). This was the work of an MA student still learning his trade, and I still have nightmares about whether I got the figures right and whether any of it made any sense. This type of thing goes to show that the agenda of the interpreter (in this case a newspaper in need of a lighter story to complement a day of gloomy news about the economy) can remove the nuance and ambiguity of a well-intentioned investigation. I never asserted, either in my paper or when interviewed about it, that death metal is the soundtrack of child prodigies, but the newspaper still used this headline. In fairness, the article itself is more balanced than the headline…but I suppose that’s my point.

What’s the English Bacc? Wasn’t it scrapped?

The English Bacc is an accountability measure (not to be confused with a different Government proposal, English Baccalaureate Certificates, which was to be a new type of qualification until the idea was dropped following consultation).

The English Bacc measures how many pupils get a grade C or above in five subject areas, which must include English, maths, at least two sciences, a foreign language, and history or geography.

A new points-based system, outlined in February 2013, may also be introduced for 2014. The proposal is that it will measure pupils’ average grade across eight subjects: English, maths, three other English Bacc subjects (sciences, languages, history, geography or computer science), and three more subjects (assuming they are deemed eligible by the DfE, details are to be confirmed).

This new measure will be in addition to the English Bacc measure.

Bacc and forth

When the English Baccalaureate was first introduced, the government cited evidence from a national longitudinal study to support it.¹ The research compared pupils who would have successfully ‘achieved’ an English Bacc (grade C or above in English, mathematics, the sciences, a language, and history or geography) with those who had achieved five A* to C grades across any subjects (but including English and Maths), and those who had not achieved either benchmark.

Those who achieved the English Bacc were found to be at an advantage compared to their peers. They were less likely to find themselves not in employment, education or training (NEET), and more likely to be studying in a higher education institution. The research also showed that they were less likely to have sex before the age of 19 - presumably there is something about studying a combination of English, maths, science, history or geography and a language that makes young people more cautious about such things… 

On the basis of this research alone, it would be easy to make the assumption that the act of taking the English Bacc subjects somehow improves a pupil’s life prospects. However, just because two variables are related does not mean that one caused the other. One thing to look at is whether those who take the English Bacc subjects are generally stronger academically to start with. If they are getting better grades anyway, then surely they are more likely to go to university and avoid becoming NEET, regardless of which subjects they choose?

Let’s take a closer look

My research is a slightly more nuanced interrogation of the data, one which addresses this issue of student ability prior to taking GCSEs. Data from the exam boards certainly suggests that there is a link between choosing to study English Bacc subjects and academic ability. However, even after controlling for this (and other factors, such as school type, which are known to be associated with attainment), there was some support for the notion that taking a combination of subjects that fulfilled the English Bacc requirements boosted overall academic attainment. Those who take the English Bacc subjects do better than similar pupils who do not. The figure below is a slight simplification, but illustrates my point.

Relationship between average prior attainment at KS2 and average GCSE attainment by subject choice

Graph showing the relationship between average prior attainment at KS2 and average GCSE attainment by subject choice

The all-important ‘But...’

So why am I nervous? Surely this is good news, a bit of evidence that supports a reform which is currently under way.

Well, there is another major factor which my analysis cannot account for, and that is pupil preference and choice. The data cited by the government was gathered before the English Bacc was introduced. So it presumably encompasses those who actively chose to study the relevant subjects. With the English Bacc embedded in school performance tables, it may be that there is more pressure on pupils to take English Bacc subjects, and choice therefore becomes more constricted. Pupils may be less engaged because they have been steered towards subjects which do not necessarily chime with their interests. How will these new students perform? Will taking the English Bacc subjects improve their overall academic performance? Will it reduce the likelihood of them ending up NEET, or increase their likelihood of attending university?

This type of data set is fundamentally limited. We simply cannot know what the impact of channelling our young people down the English Bacc route will be until we do it. My study may suggest that there is some sort of gestalt effect at play, that taking these subjects together boosts a pupil’s overall performance, but this is conjecture and must be precisely framed within the limitations of the data.

Evidence-based policy is a fine thing and should be applauded, but is open to misinterpretation if the subtleties of the evidence aren’t acknowledged. Of course, the limitation I’ve noted above is thoroughly detailed in my research paper. But I am still anxious about how my findings could be interpreted and applied by others, perhaps less able or willing to appreciate the nuances. Or maybe I still have heavy metal ringing in my ears.

Stuart Cadwallader

References

  1. Department for Education. (2011a). Youth Cohort Study and Longitudinal Study of Young People in England: The Activities and Experiences of 19 year olds: England 2010, Statistical Bulletin (No. B01/2011). London: DfE. Retrieved October 26, 2012, from http://www.education.gov.uk/rsgateway/DB/SBU/b001014/index.shtml

Share this page