Saturday, October 10th

The Daily Telegraph carries a preview today of the Cambridge University-based Primary Review’s final report. This is said to call for an end to Sats tests in primary schools.

Thursday, October 1st

You may have noticed that I had a piece in the Guardian’s education section over the summer about an online discussion among history teachers as to which board offered the “easiest” exams.

Teachers were debating with each other, over several years, on the schoolhistory.co.uk website, which version of the modern world history GCSE offered the easiest,  most predictable questions.

One teacher, who started the debate, talked at length about the benefits of moving from AQA to OCR. He said of OCR’s exams: “The questions are very straightforward and at least 40% easier!” He said that the coursework requirements were also less exacting for OCR, and that less ground had to be covered with the teaching. This was very much a good thing.

He added: “I hate the fact that we have to shop around and play the system and find the easiest exam board/paper. Wouldn’t it be better if everyone was playing on a level playing field?”

I just come back to it now because what I didn’t find space to include in the piece was that, near the end of the discussion, the verdict from the teacher who started it was that, after the 2008 exams, his statistics had improved sharply.

He said: ”My OCR pass rate was 64% and our FFT [Fischer Family Trust] Type ‘B’ residual was a positive 0.3!”

This was taken as complete vindication of his decision to switch from AQA to OCR.

I’ll try not to sound too pompous or moralising here, but  is this really what education now comes to? A teacher’s quality can be summed up, accurately, of course,  in a number between 0 and 1.

And, if  this system encourages teachers to search out nicely “predictable” exams and to teach remorselessly to the test, so be it.

Remember, the data at the end is all that matters.

Truly sad.

Wednesday, September 23rd, 2009

I gave a talk yesterday on why, in my view, the accountability system now in place in schools is dysfunctional, counterproductive, based on highly questionable assumptions about what motivates teachers and frequently damaging to pupils’ long-term interests. The link is here: 

Apologies, as ever, for some slight glitches in the text.

I also should have put up, a while back, a link to a pamphlet I worked  on about assessment in the early months of this year. “Assessment in schools. Fit for purpose?” is a commentary on the assessment system of which I was one of the authors with the Assessment Reform Group. The product was part of the Teaching and Learning Research Programme, funded by the Economic and Social Research Council. The link for it is here.

Thursday, September 17th

Well, not one but two stories today taking issue at statistics-driven schooling.

Here is one in the Daily Telegraph, reporting on a document from the AQA exam board setting out the malign impact of league tables on the teaching experience for pupils: http://www.telegraph.co.uk/education/6193787/Scrap-school-league-tables-say-experts.html.

And another one, also from the Telegraph but reported elsewhere, too: http://www.telegraph.co.uk/education/6198789/School-exams-taking-joy-out-of-childhood.html. This sees Mick Waters, former head of curriculum at the Qualifications and Curriculum Authority, criticising the effects of test-driven schooling in primary schools.

Monday, September 14th

The Guardian published an article over the weekend on students being pushed out of some schools and colleges half-way through their A-level courses.

Read it here.

Monday, September 7th

Anyone who wants to investigate whether the at-face-value impressive rise in national curriculum test and GCSE results over the past 20 years is genuine is always on the look-out for alternative measures of education standards.

In other words, while the official data might point to seemingly staggering improvements, there could be other explanations than that children are simply becoming better educated, not least in the phenomenon of teaching to the test, whereby instruction becomes very focused on a particular exam. If the gains suggested by the improvements in official results are truly useful to the pupil, they should be capable of being measured through other tests.

A study which has just been presented at the British Educational Research Association’s annual conference in Manchester was particularly useful in this sense, as 3,000 secondary school pupils were presented, last year, with almost identical questions to those a similar sample of children had faced back in 1976 and 1977.  

And the results uncovered by this team of highly experienced academics from King’s College, London, and Durham University, were astonishing when one considers the transformation of secondary exam results over the same period, that there has been little change.

The questions – taken by 11- to 14-year-olds in both eras, were divided into three, testing either algebra, ratio or mastery of the manipulation of decimals. In the last of these three categories, pupils appeared to have improved since the 1970s, perhaps reflecting the greater use of calculators and computers where decimal notation is to the fore, suggested the researchers.

But the 2008 pupils fared roughly the same on the algebra questions as they had in the 1970s. And on the fractions questions, they came off slightly worse than their predecessors of a generation ago.

Looking at the results as a whole, lower ability pupils appeared overall to have performed worse in 2008 than they did in 1976/7. That is, the tests suggested a longer tail of underachievement. Today’s higher-achieving pupils, though, fared slightly better than their forebears.

Overall, though, the message of these test results was largely “no change”. Yet the proportion of pupils achieving O-level grade C or better was only  22 per cent in the early 1980s, compared to more than 55 per cent last year. This is potentially devastating stuff for a government which has made exam results the key indicator of national education performance.

GCSE results, like national tests, suffer as measures of national standards because they do not retain questions from year to year. Hence it is impossible to do the sort of direct comparisons which are made possible by the King’s and Durham study*.

The latest research had  a slight caveat, in that it is not until a further set of tests are carried out this year that the sample of modern-day pupils can be said to be truly representative. However, the indication so far is that the 2008 pupils slightly over-represented the higher end of the ability range, suggesting that if anything, the findings so far make today’s pupils look slightly better at maths than they actually are.

The authors conclude:

“There is no evidence for significant improvement, or significant deterioration, of standards between 1976/7 and 2008.

“Although performance in some areas has improved it looks as if, when all the results are analysed, there will be little evidence for the sort of step-change in mathematical attainment which might be suggested by the claimed improvements in examination results.”

Secondary students’ understanding of mathematics 30 years on”, by Jeremy Hodgen, Dietmar Küchemann, Margaret Brown (all King’s College London) and Robert Coe (University of Durham), was presented at the British Educational Research Association conference today (September 5).

* Because the GCSE papers have to change every year, examiners have to decide where to set grade boundaries in order to maintain standards from year to year. And there have been some suggestions, by very experienced people in this field, that this can lead to a gradual reduction in standards over the years. (For more on this, see chapter 13 of Education by Numbers).

Thursday, August 27th

Do today’s GCSE results, which show another improvement to a new record, amount to testimony that our students and teachers are working harder than ever before and that education standards are rising, or that the papers are getting easier?

It is the routine question at this time of year. And the honest answer is that there is no answer. The statistics, the principal means by which the Government is supposedly being held to account for its education performance, provide no good way of answering the question as to whether underlying standards of teaching and learning are rising or falling.

Put together with immense care and technical expertise by exam board professionals who are among the most sophisticated in the world at this business, the stats do a pretty good job of sorting out the A grade student from the C grade, and the C grade from the F grade.

But they cannot tell us whether education standards are rising or falling. Why not? Because too much about the exams system has changed over the years. As some experts in this field say: “If you want to measure standards, don’t change the measure.”

Among the alterations in the past five years have been the scrapping of compulsory language study to 16 – which might have been expected to improve grades, as pupils utilise more choice over which subjects they study – a move towards more modular courses, and, as today’s figures show, increased “gaming” of the system by schools in encouraging pupils to take subjects including maths early, with the chance to resit later on in the course.

As Mike Cresswell, the director general of the AQA board who led the GCSE results press conference this  morning, argued in a paper a few years ago, “The statistics of public examinations cannot, and do not, provide objective or unequivocal quantitative measures of temporal changes in educational standards.”

You could say, then, that it is a mistake even to attempt to answer the question with which this blog began. This could be argued to be true particularly because curricula changes mean that the content of what is learnt in schools from one era to the next is not strictly comparable. And shouldn’t we just stand back at this time of year and allow the students to receive their grades in peace?

This is, however, never going to happen so long as the Government makes the grades themselves a judgement not only of what pupils have gained from their education, but of its own performance. To put out national figures at the same time that students receive their results and then ask the media just simply to go along with Government claims that they are unequivocal testament to rising standards is to ask the media not to do its job.

If those in charge were really committed to addressing claims that this debate undermines pupils’ achievements, they could do two things. First, they could move the publication of national results until some weeks after students had got their grades, which might separate the argument about national standards from the time when teenagers are given their verdicts. Or second, and much the better long-term solution, they could depoliticise this debate by introducing another measure of national education standards, such as getting a small sample of pupils to be assessed every year outside of the GCSE and A-level system.

That second approach go a long way to stopping the production of ever-rising GCSE and A-level grades as being seen as an end in itself, to be aimed for whatever the means taken to achieve them. The system by which ministers set themselves targets, then watch as league tables, performance management and the Ofsted, school improvement partner and Fischer Family Trust regime of statistically-driven bullying all put huge pressure on schools to help them meet them, is hugely damaging to education, with very few powerful people in this regime asking what the results really mean, and the measures being taken to raise them.

The side-effects, of course, are legion, as my book and this website has attempted to document. The most obvious, now, is that the last four years of most students’ lives at school could be seen as one long exercise in exam preparation, as they go through module after module of GCSEs and A-levels. Modularisation, to use a horrible word, may be the right way to go in educational terms – I am to some extent agnostic on it. But it has developed to the degree it has because it seen to be more likely to produce good grades, whatever they mean, and whatever the educational side-effects. It is true that some within the exam boards are concerned about this trend. But anyone trying to resist it is facing an uphill battle, given the pressures on schools over results.

This is particularly unfortunate because not only are the results not useful as a measure of underlying education standards, the grades drive may not even, in its own right, help students, since raising scores may just mean that employers and universities simply increase the grades they demand of new recruits.

Neither does the advent of the results-are-everything culture seem to have achieved much in terms of improving social mobility: while the proportion of A grades for pupils from comprehensives has improved at both GCSE and A-level in recent years,  the rate has improved by at least as much in private schools, from a higher base.

Students should not be too depressed today. Good grades still reflect hard work. But this emphasis on grades as ends in themselves, for the nation as a whole, is a grave national mistake.

Tuesday, August 25th.

A report on mathematics teaching in secondary schools offers some further disturbing insights into how the push for better reported grades for schools (and their pupils) can come at the expense of building genuine understanding of a subject.

The “Evaluating Mathematics Pathways” interim report was carried out for the Qualifications and Curriculum Authority, to whom it reported in April. It was set up to investigate changes to the qualifications structure in maths.

The report says: “One of the most significant challenges to improving learner experiences in mathematics classrooms is the effect of high-stakes external assessment on the experienced curriculum, particularly the ways teachers are compelled to behave in response to performative pressures.”

 The report then sets out some detail. Schools which are under pressure to raise the proportion of their pupils achieving C grades or better have experimented with different strategies for entering these students for the exam. For example, pupils are encouraged to complete the GCSE by the end of year 10 and then entered with the hope of securing a grade C. Or borderline students are entered for the more difficult “higher tier” papers, covering grades A*-C, and then told they only need to worry about completing the easier questions on the paper. (Presumably because through this method it is easier to get a C grade than by attempting the easier set of papers, where more marks are needed for a C).

The report says: “One of the main implications of the above entry strategies is the very likely scenario that fewer students will get sustained and meaningful engagement with those aspects of the programme of study typically assessed at higher tier.” Crucially, for those wanting to study the subject at A-level, this includes algebra, which is the foundation for much success post-16. Students who had seen their results at GCSE boosted in this way could then move on to A-level and find they struggled, because they had not mastered the subject. The report said that this tendency was particularly strong in 11-16 schools, which do not have to face the consequences of these strategies in the sixth form.

It concludes: “QCA should alert DCSF to the risks in maintaining and widening participation in the study of mathematics post-16 associated with the accountability measure of grade C in maths in the achievement and attainment [league] tables. There is evidence of increased early entry [of pupils] in order to ‘bank the grade C’, which may be particularly detrimental to transition issues at age 16.”

The findings were taken up in another independent report, by the curriculum development body Mathematics in Education and Industry. It says: “Some schools are now entering pupils for GCSE at the end of year 10 hoping to obtain a grade C. This practice is new and seems to be, at least in part, a response to the accountability requirement for mathematics.”

The Nottingham report also raises questions about exam boards’ move to offer GCSEs in “modular” form, where papers are taken throughout a course and there can be resits, rather than altogether at the end.

It says: “[Modular GCSEs] have been used by many centres as a means of raising attainment but do not necessarily improve levels of algebraic competence or mathematical understanding. We have also been told that graduated assessment [of this sort] tends to hinder teaching for progression in a topic.”

Looking into this issue, I also came across a joint document from the two leading subject associations for maths to last year’s Rose Review for the Government of the primary curriculum. (Available from here).

This mentions the impact of high-stakes testing on their subject, which Sir Jim Rose was barred by the Government from considering, a move the submission describes as “ridiculous”.

It says: “The high stakes assessment cannot be ignored – it is the most significant factor which limits the improvement of teaching and learning in primary mathematics.”

It adds: “Although the term ‘raising attainment’ is in common parlance, it was felt that the narrow interpretation of this to mean higher test results skews the teaching and learning in schools. The goals should be to improve teaching in order to improve learning.”

Finally, in my round-up of recent evidence, a report last week from Edge, the educational charity, also bemoaned the effects of teaching to the test. See this. The survey results themselves are available here.

My article for today’s Guardian on teachers reporting choosing exam boards on the basis of how ‘easy’ they are is here.

Thursday, August 20th, 2009

I’ve just returned from the annual A-level results conference, where the heads of England’s three major exam boards present the yearly grade statistics.

For the last three years, much of this has been taken up with an elaborate  and detailed attempt to take some of the heat out of the ritual dumbing down debate.

And I have to say that, despite having a great deal of respect for those running the exam boards, I find this exercise in explaining away what are in some cases valid criticisms of the system a tad unconvincing.

In what has now become a well-established pattern at these early-morning August get-togethers in Westminster, Dr Mike Cresswell, director general of the largest board, AQA, takes centre stage. He then presents detailed charts to show that, while results have indeed improved steadily over recent years, different regions of the country, and different types of school, have improved at different rates.

So, for example, while the proportion of A grades in London has risen by more than seven per cent since 2002, in the North East of England it has improved by less than four per cent.

Similarly, the proportion of A grades in the independent sector has risen over the same period by nearly 12 per cent. In state grammars, the rise was nine per cent; in state colleges, five per cent; in comprehensives, also five per cent; and in secondary moderns, two per cent.

Dr Cresswell’s argument is that, because different types of school and different areas of the country have progressed at different rates, any crude or “naive” arguments that improvements in results are a product of uniform dumbing down are dispelled. Since, by implication, if this were true, all parts of the country and all types of school would have progressed by a similar rate.

While the figures unveiled in this analysis are thought-provoking in their own right, I can’t see that they disprove any argument that there may be some underlying dumbing down, or slipping of standards, to use a less loaded term, going on.*

For, surely, the fact that different types of school and area of the country have varying rates of improvement does not negate the sense that some underlying trend is at play nationwide.

To use an analogy, if you looked at UK house price rises over the years leading up to the peak of the property market in 2007, you would undoubtedly see variations in the degree to which prices had risen in different parts of the country.

But this would not disprove the existence of background factors, apparent nationwide, that might help to explain part of that rise in prices in all areas of the country. An example could be, for example, the ready availability of mortgage credit, which had tended to inflate the market nationally.

Similarly, the fact that global warming might lead to differing increases in temperature in different areas of the planet would not, surely, be taken by scientists to disprove the overall tendency that the earth is warming up.

Or, to take Dr Cresswell’s argument to its logical conclusion, the only way we could ever provide convincing evidence of dumbing down would be if we could show that all parts of the country, and in all types of school (and, presumably, for all categories of student) had improved by a uniform rate in recent years.

I have had little statistical education beyond A-level maths, and Dr Cresswell’s knowledge of both statistics and the exam system is vast. It may be that I have not grasped the full ramifications of this explanation. But it does come across to me as a bit of a smokescreen in what is quite a complex debate.

* (For the record, I don’t subscribe to any notion that exam boards are rushing around crudely lowering grade boundaries to cut standards. But I do think that some complex processes are at work which might tend to make it easier for a student who has mastered the subject to a given level to get a better grade now than they would have got a few years back).

Thursday, August 13th

Another defence of high-stakes testing from Conor Ryan, the former education adviser to David Blunkett and Tony Blair, in today’s Independent.

The article, though well-drafted as usual, is weak in several ways.

Apart from leading in on the one percentage point drop in English results, which as pointed out last week really means nothing, Mr Ryan offers an unconvincing explanation for why the test results are higher than they were in the 1990s, and why the data has shown little such improvement in recent years.

He argues:

“Around 175,000 more youngsters still reach the expected level each year than did so 14 years ago, a result of the combination of accountability and pressure that has accompanied the tests, including Labour’s literacy and numeracy strategies.”

And: “The most important feature of the years between 1995 and 2000, when results rose rapidly, was single-minded momentum. Schools were in no doubt that their top priority was the 3Rs.”

Well, the most likely reason for the improvement is a combination of three factors: teachers becoming more familiar with the requirements of what were in 1995 a completely new set of tests;  the introduction of the numeracy strategy into maths lessons; and a slipping in test standards.

The first factor has been well-documented following the introduction of tests around the world. The third, which was investigated comprehensively in research commissed by the Qualifications and Curriculum Authority and belatedly published in 2003, also suggests a slipping of standards in the years 1995 to 2000, at least in English. Since standards were tightened, the results improvements in that subject have indeed flattened off. The same study found that there had not been a corresponding slipping of standards in maths. This, and other evidence, suggests to my mind that the gain in maths has been genuine. But the much more persuave explanation for that fact is the introduction of the numeracy strategy itself (factor two), rather than the vague policy “momentum” suggested by Mr Ryan. 

He also implies that schools are now in doubt that “their top priority [is] the 3Rs”. This is laughable. While it could be argued that some recent policy initiatives, such as the proposed introduction of the school report card, measuring wider aspects of school life than academic results, have not been focused exclusively on 3Rs standards, the reality of school life is that test results remain the be-all-and-end-all for school leaders in particular. In fact, the emphasis on them in the accountability system has undoubtedly increased between the end of Ryan’s time as a policy adviser at the Department for Education and Skills, in 2001, and now. As I reported in a TES article last year, Ofsted inspection judgements in recent years have been driven almost entirely by test results in those three subjects, and league tables and targets remain as influential as they ever have. While Mr Ryan may bemoan a slight change of emphasis in policy discussions since the Brownites took over from his Blairite friends, the reality at primmary school level has not altered much at all.

There are other elements of the article with which to quibble, not least the conclusion, which says that children would not be better taught without independently set and marked tests [of the current type]. One of the best arguments against this has been the move of private schools away from Sats tests in recent years. But I come back to a familiar argument against this: is it really the best use of the time of pupils in year 11 that they should spend months being drilled for a test which, despite the arguements in this latest piece, are  not important in themselves to pupils’ futures. Mr Ryan, and those making similar arguments, need to be more imaginative when it comes to thinking out alternatives.