Thursday, September 30^th, 2010

If there is anything certain about the education white paper which is going to be published by the Government by the end of the year, it would seem that a long-standing loophole in league tables is about to be closed.

The system of “equivalences”, whereby non-GCSE qualifications are given weightings worth several GCSEs for the purposes of the tables and of all official statistics, is about to change radically, it would appear.

For years, Intermediate General National Vocational Qualifications were counted in the rankings as “worth” four GCSEs. When schools realised that getting pupils to take them could see the school’s published statistics improve dramatically, numbers taking GNVQs rocketed accordingly.

GNVQs were phased out a few years ago, and the loophole also seemed to have been partially closed by the stipulation that schools were to be judged on the proportion of their pupils gaining five or more GCSEs (or equivalent), including English and maths. The latter part of that formulation made it impossible for schools to do well in the table solely on account of “equivalent” qualifications.

However, in recent years, other courses have been growing in popularity, seemingly because schools observed that these courses had high pass rates and that, if they could get pupils entered and then passing one of them, say, they only needed to get these students to achieve C grades in English and maths and at most one other qualification and the school’s statistical goal would be reached.

As I reported for the TES last month, entries for BTEC First qualifications (worth two or four GCSEs, according to the course), and OCR Nationals (worth 1, 2, 3 or 4 GCSEs) have shot up.

Michael Gove, education secretary, told the Edge Foundation charity earlier this month: “Some of these qualifications badged as vocational enjoy a ranking in league tables worth two or more GCSEs, making them attractive to schools anxious to boost their league table rankings. That has to be changed.”

Having covered some of the peculiarities of the exam system for years now, I still find myself surprised at times. The latest example came a few days ago, in relation to these equivalencies.

Thanks to the efforts of Roger Titcombe, a former head teacher with whom I’ve worked before investigating school results, I obtained the full GCSE – and equivalent – results for 2009 of a school which has won national recognition for improving its grades.

And the patterns unveiled in the grades would be astonishing , I think, to anyone who came to this information afresh, knowing only that GCSEs and other qualifications were deemed to be equivalent for official purposes.

The pattern was as follows. For almost all of the 20 GCSE courses offered in the school, almost the full range of grades had been awarded. In other words, children had scored grades ranging from A*s (although only a few of these were achieved in this school) down to Fs, Gs and even the occasional U. For maths, for which there were 165 entries, every grade – A*, A, B, C, D, E, F, G and U – was awarded to at least two students, while for English, which had the same number of entries, the picture was similar.

For the vocational qualifications, though, the picture was very different. Amazingly, of the 303 entries, in eight different BTEC First or OCR National qualifications, not a single one failed. All achieved starred distinctions, distinctions, merits or pass grades. The school also had pupils entered for 15 different assorted vocational qualifications, categorised simply as “Vocationally Related Qualifications” or basic skills tests. Again, all entries passed. This high pass rate, and the multiple-GCSE equivalences of some of these courses, meant that the school, which has been lauded for its success in rising up the “GCSE” league tables, actually chalked up more A*-C passes through non-GCSE courses than through GCSEs. Confusing, isn’t it?

Now, this may not come as a surprise to anyone who has experience of how this works in schools. It is well known that BTEC Firsts have had high pass rates, and indeed, I reported on the national rates in that TES article. More than 99 per cent of grades awarded nationally in the four-GCSE equivalent BTEC First Diploma last year were at pass or better. For the two-GCSE BTEC First Certificate, the figure was 97 per cent. This is despite Jerry Jarvis, former managing director of Edexcel, which runs the qualification, describing BTEC Firsts as qualifications schools could offer “If you think a student will fail to realise their potential by doing GCSEs”. And, for the OCR Nationals, the line from the OCR board which I received when writing that piece seemed to be that they could not give pass/fail rates for the qualification partly because pupils were only entered for it if they were ready to pass it.

All these points may strike some readers as not new. My latest thought, though, on seeing the relative grade profiles in this one school is this: how were these “equivalent” courses ever given equivalence with GCSEs, when the way they use the grade profile is so different?

Since GCSEs were introduced in 1988, GCSE examiners have used the full range of grades down to G to indicate a pass. Thus, officially at least, D and E grades were seen as worth something for pupils in that they were not deemed failures. The idea was that most children would come away from the exam with at least some credit for their work. The concept of meaningful D-G grades survives to this day, at least officially, despite the huge emphasis now given by the government and schools to achieving a C or better, making it the unofficial pass grade. And a glance at the national grade profiles for GCSEs confirms this, because the full range of grades are awarded.

The vocational qualifications also, clearly, have a pass grade. But the equivalence sets this as being the same not as a grade G, which is the official pass grade for GCSE, but a grade C. And because the GCSE grading system still uses the full A*-G range to reward pupils’ efforts, pupils at GCSE can easily find themselves missing out on a C (the crucial grade for schools, for accountability purposes). Yet the vocational qualifications system is, unsurprisingly perhaps, reluctant to see pupils coming away from a course with nothing. But there is no option for the system, under the equivalences, to give them less than a grade C (actually, often it will not give them less than two Cs, or four Cs), without actually failing them, leaving them with nothing. So almost all are awarded at least a pass grade. To put it another way, and to use the OCR National argument cited above, it could be argued that a child would only be entered for certification for an OCR National when they had a chance of a pass grade. But the fact that this pass grade has been set at equivalent of a GCSE grade C means there is no chance for them to gain credit for their work by earning a GCSE grade at less than C. So the entry system may more or less guarantee that if they are entered, they are going to get at least the equivalent of a GCSE grade C. Yet that is not how GCSEs work.The grading systems are different, so the equivalence is weird.

The confusion also reflects, I think, the politicisation of this system, and a bodged attempt at compromise by politicians. The fact that the GCSE allowed pupils to pass with a G grade may well have rankled traditionalists, who compared it to the old O-level structure in which a D was a fail. But rather than either sticking to the official position, in school accountability, that all grades from G upwards constitute a pass, or changing it so that the C grade was actually made the official pass mark, those in charge kept the G pass officially, but unofficially made the old O-level pass grade (a C) the benchmark by making this the centrepiece of league tables and top-down results pressure on schools. This is confusing: accountability says a C grade is the pass threshold, but the exam grading system says a G grade marks the boundary.

Some will counter that employers also have long regarded the C as the cut-off for success, so it is right for the focus to be put at this boundary. But I have to say I don’t completely buy that argument: a detailed report from the CBI in 2006 provides evidence that D grades were being accepted by employers then, while clearly higher grades than a C are also often required. So the situation is a lot more nuanced than the focus on C/D borderline pupils in schools would suggest. Grades confer relative, rather than absolute, advantage to students: employers (and higher education) tend to want students with the best grades, not always simply those above a somewhat arbitrary cut-off point.

Returning to the equivalences issue, the system really is comparing apples with bananas. It has to go, because the inflated value of non-GCSE courses has skewed schools’ decisions and has certainly meant some pupils have been pushed towards qualifications which are not as valued, in the reality of the employment and post-16 education worlds, as GCSEs, mainly because of the worth of the qualification to the school. That does not mean that these qualifications have not had value to individual pupils achieving them, by the way, just that the setting of the equivalences has clearly influenced schools’ decisions over who should take them. If the equivalence goes, it will be interesting to see how take-up of these popular non-GCSE qualifications fares; I would expect it to fall.

The detail of exactly how the Government sorts out this problem, though, is going to be interesting.

- Warwick Mansell

Monday, 27^th September

Michael Gove, the Education Secretary, today gave more details on the content of the forthcoming review of Key Stage 2 Sats.

In a letter to Russell Hobby, general secretary of the National Association of Head Teachers, Mr Gove said the review would “consider a number of key issues”, including:

-“How best to ensure that schools are properly accountable to pupils, parents and the taxpayer for the achievement and progress of every child, on the basis of objective and accurate assessments.” 

-“How to avoid, as far as possible, the risk of perverse incentives, over-rehearsal and reduced focus on productive learning.”

-“How to ensure that tests are as valid and reliable as possible, within an overall system of assessment (including teacher assessment) which provides the best possible picture of every child’s progress.”

-“How to ensure that performance information is used and interpreted appropriately within the accountability system by other agencies, increasing transparency and preserving accountability to parents, pupils and the taxpayer, while avoiding the risk of crude and narrow judgements being made.”

Mr Gove writes of sharing the NAHT’s concern that “too many schools [are] spending too much time on test preparation in year 6 at the expense of…productive teaching and learning”. This, he writes, is “clearly undesirable”.

He says that “raising standards and narrowing gaps are the central goals of the Government’s education policy.” These goals are “best achieved through ensuring that schools and teachers are free to set their own direction, trusted to exercise their professional discretion and accountable for the progress of the children in their care”.

He adds: “Mindful that the OECD concludes that external accountability is a key driver of improvement in education and particularly important for the least advantaged, the Government continues to view a system of objectively measuring pupil progress and holding schools to account as vital.”

Some system of key stage 2 testing, league tables and the use of the data generated by the tests to hold schools to account for their performance, then, looks to be a non-negotiable part of Mr Gove’s agenda.

The key question in all of this is whether it is possible to have a high-stakes system of school accountability which avoids the problems which Mr Gove acknowledges and seeks to mitigate.

I wonder, in particular, about the notion that schools will be “free to set their own direction” and “trusted to exercise their professional discretion” and yet remain accountable for their pupils’ progress.

If high-stakes accountability is realised in the way it has been in the past, in this and other countries, then the result will be that schools are judged on pupils’ performance on a set of fairly narrow indicators. If large consequences follow, for schools, on the basis of what a narrow set of indicators reveal, then expect a tendency to narrow teaching towards what is measured in those indicators to continue, no matter how much the Government talks of giving teachers freedom over teaching methods and even curricula.

In other words, if ministers talk about handing freedom to the profession, but retain tight control over the statistical measures by which the success and failure of schools is judged, and punish and reward them accordingly, real freedom will be limited.

It may be that real pressure from Government on schools to improve their results will only be exerted on those who rank towards the bottom on the chosen performance indicators, as the letter says “the accountability system…must be able to identify and tackle cases of sustained under-performance”. But if this is the case, it risks setting up a gap between schools at the bottom on these rankings, where teaching is likely to be very narrowly focused on what is needed to improve the indicators, and the rest.

When Mr Gove talks about “raising standards and narrowing gaps” being his over-riding goals, I also take that as translating to raising test scores, and narrowing the performance gaps between particular groups of pupils, as measured by test scores. This, at least, is what the terms have meant under Labour, and I get no sense here that some of the problems inherent in this approach, around, again, the narrowing of focus on specific indicators, are being acknowledged.

Being hyper-sceptical, I also wonder whether holding schools to account for the “achievement and progress of every child”, which is a continuation of the policy under Labour, is not a step too far, in that it takes away the child’s own responsibility to try to do better. That is, if every time a child fails to make progress, the school is blamed, where is the child’s place in achieving a good result?

 Of the four bullet points, I think the fourth (“How to ensure that performance information is used and interpreted appropriately within the accountability system by other agencies, increasing transparency and preserving accountability to parents, pupils and the taxpayer, while avoiding the risk of crude and narrow judgements being made.”) represents the best hope of genuine improvements over the current system.

If the new government were genuinely committed to trying to educate parents and other users of test result data what can be read into the statistics, and what cannot, this would be a step forward.

Overall, though, it looks as if what is being proposed will still contain many of the ingredients of the current system, with weight being put on a limited range of statistical indicators, at least for some aspects of school accountability.

Mitigating some of the problems of high-stakes accountability is important. And it may be that Mr Gove, possibly working with the NAHT, will uncover new ways of trying to square the circle of having a system of high-consequence accountability without some of the downsides which have dogged it in the past. But there is no detail as of yet of how that can happen, so I remain to be convinced.

 While I understand the pressure on politicians over accountability, I still find it disappointing that some kind of statistics-based system seems to be non-negotiable, with the quality of the learning experience for children seemingly something which must be worked in around it.

- Warwick Mansell

Friday, September 17^th

An explosive – in education policy terms, at least – new book published today offers a series of highly newsworthy insights into the political process.

“Reinventing schools, reforming teaching” by John Bangs, who until this summer was head of education at the National Union of Teachers, and the Cambridge professors John MacBeath and Maurice Galton, comes full of quotations from some of the leading movers-and-shakers of the New Labour years.

Anyone reading these extracts will end up distinctly unimpressed with the way education and politics have intersected over the last 15 years.

Among the book’s revelations, some of which I comment on below, are:

-A claim by  Mick Waters, former director of curriculum at the Qualifications and Curriculum Authority, that the exams system is “diseased, almost corrupt.” Exam boards, competing in a market place for schools’ business, sometimes implied to their customers that their exam was easier than their rivals’, he said. He also criticised the boards for endorsing/publishing textbooks on their own exams.

Mr Waters says: “Before I went for this job, I used to think that all this criticism of exams that they were being dumbed-down was unfair. You know, the old argument, more people passed than ever before. Since I’ve been there, I think the system is diseased, almost corrupt. I don’t mean QCA or Ofqual or anybody. We’ve got a set of Awarding Bodies who are in a market place. In previous jobs, I had seen people from Awarding Bodies talk to headteachers implying that their examinations are easier. Not only that, we provide the textbooks to help you through it.”

I think whether you describe the system whereby competing exam boards seek the business of schools, who are working to the demands of politicians, all of whom have some interest in results rising or at least not falling as “corrupt” or not, it is not healthy. In my book, I quote the board Edexcel, which in 2006 advertised a range of new science GCSEs to teachers in the following way: “More chances to succeed. Our curriculum enables all students to perform to their best…students can be assessed at any time, allowing them to be tested on material when it’s fresh, and can take multiple tests before submitting their best performance”.

Last year, I reported how Edexcel, in a teachers’ guide to an engineering GCSE says: “Find out why Edexcel is your best choice for better results.”

While, as I say in the book, in many ways England’s exam system is admirably well administered, bringing enviable technical knowledge to the process, it is crazy that this has been allowed to happen. It is akin, perhaps, to a football referee saying “play in my games, and you’ll score lots of goals”. To understate, it should not be exam boards’ role.

Mr Bangs suggested at the press conference to launch the book yesterday that the solution was to move to a single exam board. I think this would also have downsides, however. The structural problem with the exams system is that those with influence both on the “demand” side (schools) and “supply” side (exam boards and the government) all have an interest in seeing grades rise (or in the case of competing boards, at least not to have their exams seen as harder than their rivals’).

For me, as has been said before, a better solution would be to ensure that voices outside of the chase for grades – a strong regulator, and those who will use the qualifications as checks on what young people understand, such as employers and universities – are given more influence. The Government should also be taken out of the process by using sample tests as the main mechanism by which national standards are checked, so that the chase for better grades – which does not benefit pupils when it happens nationwide – becomes less of an end in itself.

Right, back to the book…

-Sandy Adamson, former head of the standards division in Labour’s Standards and Effectiveness Unit, criticises primary test targets as “stupid”.

He says: “It was the delivery mechanism that was the problem and the stupidity of targets, unobtainable targets, simply pulled from the air and then applied to every school in the country. Twenty thousand primary schools and the majority of reaction was ‘this is unachievable’ and for a substantial minority it was never achieved.”

Mr Adamson also claims Labour lost the confidence of the teaching profession “in the first 12 to 18 months”. “Talked, talked, talked at them. Command and control. Told them what had to be done and the way it was going to be done.”

-Estelle Morris, the former education secretary, justifies Labour’s policy of “naming and shaming” struggling schools in political terms.

She says: “I think naming and shaming of schools gave two clear messages; in the eyes of the public politically it put it on the side of the users of the services, not the producers of the services; and secondly it gave the message to the teaching profession that we weren’t the same Labour party as last time we came into power but we would have a different focus.”

To this observer, she seems to be saying: schools were used to serve politicians’ needs. Am I alone in finding this, put in those terms, as outrageous?

It is, though, in line with a statement in the 2007 book by Sir Michael Barber, the former head of Tony Blair’s “Delivery Unit”. He cites the “name and shame” plan as evidence that the party would be “hard as nails” [with teachers].

- Former Downing Street speechwriter Peter Hyman said that Tony Blair believed that Alastair Campbell’s “bog standard comprehensive” remark had helped give the party “some definition”.

-Sir Mike Tomlinson, who led the two-year review of secondary qualifications whose central recommendation was rejected by Tony Blair on the eve of the 2005 general election, says there is “nothing rational about decision-making and policy-making at all”.

- Kevan Collins, former head of the Primary National Strategy, says the introduction of academies in the early 2000s “segmented the profession”, undermining an attempt to focus on “universal language of teaching and learning” as the focus moved towards “creating certain types of schools”.

- Stephen Byers, another former school standards minister, is also reported as having written to Professor Robin Alexander, the Cambridge academic, “expressing his welcome for the co-operation of academics but attended by the caveat – only to the degree that they supported government policies.”

- Mr Bangs also told the press conference yesterday that the book shows none of his interviewees, from across the political spectrum have confidence in Labour’s Every Child Matters/Children’s Services agenda.

He did praise aspects of New Labour’s record on education, including its school buildings programme and, possibly, its literacy and numeracy strategies. But…well, there is a lot to chew over here. I hope to post a review of the book here in the next few weeks.

- Warwick Mansell