Wednesday, October 31st

A response from Ofqual to a Freedom of Information request, published last week, offers fresh insights into this year’s GCSE English grading controversy.

Followers of my regular blog on the NAHT’s website will need no reminding that I’ve been taking quite a close interest, having posted several lengthy pieces on this and the related issue of Ofqual’s “comparable outcomes” policy for controlling apparent grade inflation.

Ofqual itself is due to publish its final report on this year’s problems on Friday (November 2^nd).

This latest set of correspondence has been released under FOI to blogger and tweeter Antony Carpen, following an earlier request by him which also generated a lengthy correspondence trail and which was covered in my last two blogs on the NAHT site.

This one focuses on correspondence between Ofqual for the Department for Education as the controversy was developing. And it is interesting, right from the start.

In the first set of emails, which date from August 17^th – six days before national GCSE results would be announced at a press conference –   an unnamed Ofqual official tells the DfE that:

“On GCSE English, there is a potential story because in order to make sure the overall subject grades are right/comparable with last year and across the boards, some of the Controlled Assessment units sat in the summer have higher grade boundaries than the units sat in January.”

The email goes on: “Policy colleagues have been talking to DfE at your place and are due to talk again early next week.”

“Controlled Assessment” units are tasks set by the boards for pupils to take  in-class, with the work marked by teachers but with the boards deciding later how many marks are needed for each grade.

For the AQA board, which has by far the highest number of pupil entries for English, the number of marks needed for a C grade was changed from 43 marks out of 80 for pupils submitting work in January for 45 in June. This has proved extremely controversial, since the tasks set for the pupils did not change over this period.

Other boards, however, also changed CA boundaries between January and June, according to data provided in this Association for School and College Leaders document, matched to data provided at the end of Mr Carpen’s FOI.

Edexcel changed the grade boundaries for two controlled assessment papers between January and June: one from 55 marks out of 96 to 65 and another from 60 to 64 marks out of 96, the ASCL document suggests.

A third board, OCR, changed the marks on five CA papers between January and June, in all cases, again, moving the boundary upwards, according to the ASCL document. The two other boards included in our regulatory system – the Welsh and Northern Irish boards – seem not to have allowed early entries for controlled assessment units. (For more on the data, see note below)

The significance of the quotation from Ofqual above, I think, is that it is the clearest statement I have seen yet that what the boards, under supervision and in at least one case pressure from Ofqual, did in moving grade boundaries in this way was driven very much by the need to get the overall pass rate “right” in the end.

This can and is justified by Ofqual in terms of it being necessary to combat “grade inflation”. But it does raise some problems, such as the clear risk that boards raise grade boundaries not, in reality, driven by the need to ensure fairness to all candidates taking different modules of a course – or the same module at different times – but by the need to produce overall headline statistics which are comparable to the previous year’s. Again, this may seem unproblematic, for me, it deserves further debate and scrutiny.

So, if one set of candidates one year is advantaged and another disadvantaged by the setting of grade boundaries, but their overall effect is to produce a total number of grades which is similar to previous years – ie the results of the two groups cancel each other out – it may be deemed satisfactory under this policy but is it really being fair to each group of pupils?

In an earlier NAHT blog, I referred to comments from a senior exam board official in a paper dated July 30^th, unveiled through Mr Carpen’s previous FOI request.

The official said: “If asked by [schools and colleges] and the press to explain the rise in controlled assessment [grade] boundaries, the rationale has to be based on [examiners’] qualitative judgements of work seen, not on a statistical fix.”

That quotation above suggests to me that we  may be more in the territory of “statistical fix” than qualitative judgement.

-There is plenty more in this latest set of FOI correspondence, as might be expected given that it runs to 148 pages. I don’t have time to blog any more now, though, except to note that it does show Ofqual and the DfE co-ordinating their communication very closely. For example, on 22^nd August, a Department for Education official emails Ofqual about a story in the Independent saying “it would be really helpful to have your line on this issue so that we can craft our own around that”.

Similarly, in an email the same day from Ofqual to the DfE, Ofqual indicates it is trying to match its message to that put out by the Joint Council for Qualifications at the GCSE press conference, where results would be announced the following day.

The email from Ofqual says: “We have refined the line on GCSE pass rates to make sure it is a bit closer to the message JCQ will be issuing at the press conference.”

-Note: Candidate entry data provided at the end of this FOI appear to indicate, if I have understood it right, that while changes in controlled assessment grade boundaries were in some cases quite dramatic, relatively few pupils will have benefited from what Ofqual says were “generous” pre-June grade boundaries by submitting controlled assessments before June.

The data show that all the grade boundary changes to controlled assessment discussed above occurred in exam specifications where low proportions of candidates submitted in January 2012 or previously. The only instance in which large numbers of pupils submitted CA entries early was for an Edexcel course for the controlled assessment grade boundaries did not change between January and June.

The significance of this, I think, is that relatively generous grade boundaries for CA in January 2012 are unlikely, in themselves, to have pushed up overall pass rates by much.

However, there were more conventional externally-assessed papers – including a much-discussed foundation paper set by AQA – where there were substantial changes in the grade boundaries between January and June and where substantial numbers of candidates were entered before June.

- Warwick Mansell

Friday, October 12th, 2012

I’ve just caught up with a very interesting Radio 4 documentary on “free schools”, which aired last night.

The piece prompted quite a few thoughts, but I was particularly taken by comments by, I think, Jeremy Rowe, the head of Sir John Leman school in Beccles, Suffolk, about the possible long-term impact of a new school – a “free school” – which opened in the town last month to make two secondaries there.

Mr Rowe’s concern was that although Beccles Free School has so far struggled for pupil numbers, in time it might take pupils away from Sir John Leman and that, in time, this might lead to serious disadvantages: specifically duplication of provision and, if I have remembered this correctly, the fact that Sir John Leman might no longer be able to operate, for example, courses in minority interest subjects.

This could lead to a situation where “there is less of the added value [his school currently provides] and you end up with two schools offering the same restricted diet”.

In other words, there were benefits to having pupils concentrated in one institution rather than split between two.

The counter-argument, put by supporters of free schools, is that competition between institutions, to use the standard and too-often-repeated cliché, forces each to “up their game”, or improve.

The to- and fro-of this is fairly routine in this debate. But what occurred to me is that the competition argument being used by free schools supporters seems to run directly counter to the argument used in much of the debate around healthcare.

There, a prevailing view – though often challenged, it has to be said, at a local level – is that sometimes hospitals need to close in order to focus specialist provision in single institutions. This is seen to be economically more efficient and also to promote the concept of centres of excellence. It also avoids duplication of services, I think the argument runs.

As I said, this argument seems to be well-balanced in the health sector, but the opposite of the prevailing view among current policy-makers in education.

If many people do, understandably, want to defend their local hospital from closure, I’m not aware of any argument that what we need to do, to improve health standards, is to open lots of new hospitals close to each other in order to force existing providers to “up their game”.

Someone might say I’m wrong there, and that this has been advocated.

But I guess the prevailing view would be that – to put it mildly – this would not be the best use of resources. You can argue,I guess, that schools are cheaper institutions than hospitals, but questions about the efficiency of the comparable project in the education sector will not go away, I feel.

The above represent just some very quick thoughts, but I would be fascinated to hear any responses.

- Warwick Mansell