Just as a postscript to my last blog, I attended the annual general meeting and prize-giving of the English Association last night. Got chatting with a couple of teachers, who confirmed the worries indicated in my last posting about formulaic examining. The feeling was that the gap between pupils’ educational experiences at school and what would be expected of them at university was growing. This was because, wheras in school pupils are now taught towards the exam, and guided very strongly by their teachers towards strategies which will help them accrue marks, at university independent thinking is prized. Therefore, even those with top grades can struggle as undergraduates. Bearing this in mind, this article in the Guardian this morning, setting out possible Tory plans to give universities a renewed say in school exams, were interesting and, for me, quite persuasive.

- Warwick Mansell

Despite having covered this subject for more than four years now, I am constantly surprised by the extent to which teaching to the test appears to have become a defining feature of our education system.

More evidence came my way during a recent seminar at the annual conference of the Chartered Institute of Educational Assessors in London. Academics from the AQA exam board presented the findings of interviews they had conducted during the last year with 39 first year undergraduate students at Bristol and Manchester who had gained A grades in psychology and biology in last year’s A-levels.

The central finding was that the students said that doing well in the exam was more about knowing the content of the mark scheme than understanding and mastering the subject. Many students, said the researchers, had spent a long time analysing the mark schemes, trying to assess what the examiner was looking for and then not wasting their time with answers which did not fit this precise formula.

One said: “I literally learnt the mark scheme. I was like well there’s no point in trying to go into the details of why this [biological process] works. I knew exactly what wording they wanted.”

Another said: “I would tell anyone just to focus on the mark schemes and the past papers and go through them rather than the content because they teach you more.”

Another said: “You start to realise that A-levels aren’t about how good you are at a subject.  it’s about how good you are at understanding the mark scheme. You need to understand how you might answer the question and what the examiners are looking for.”

Suzanne Chamberlain, who presented the findings, said: “It’s not about learning for learning’s sake. These candidates were absolutely focused on one end goal, and they knew what they had to do to reach that goal.

“They made a great deal of reference to assessment objectives and they made it clear that they knew about those.  They could identify what assessment objectives were targeting, and they knew how to allocate the marks required for different assessment objectives.

“They would say: ‘I would do my description paragraph, and my evaluation paragraph, so many points in each paragraph to make so that I make sure that I can access these marks.’”

 

Students did not just look at mark schemes once, or do one past paper, said Dr Chamberlain. Instead, they took as many past papers and looked at as many past papers “as they could get their hands on”.

 

 

Perhaps most damningly, Dr Chamberlain added: “In some cases, this examination training undermined the need to understand content by focusing on the mark schemed; they could just pick up the phrases that they needed to score points, without necessarily understanding the content.

“They were unprepared for anything that was different. If they went into an examination and there was a bit of extra space on the examination paper, the students said it would throw them out. What was the extra space for, they asked.”

The research was carried out against the background of planned changes to A-levels being introduced in courses which began last year. These are seeing the introduction of the A* grade and, it is said, more challenging questions. The reforms are also supposed to tackle specifically the problem of “formulaic” examining and teaching. But it was unclear from the seminar how this would happen in practice.

There is certainly no move, as far as I am aware, to ban the publication of mark schemes. As someone who went through secondary education in the 1980s, this would make sense to me. Beyond working hard to make sure we understood the question and made our answers address it directly, we had no more detail on what examiners were looking for and I think our education was the better for that. In a results-are-everything culture though, when the grades are seen to be more important than the education which underlies them, this view may struggle for attention. Some may argue that it would not be right to keep mark schemes secret, when examiners have access to them. But not all elements of exams are open, the most obvious example being the fact that the papers themselves are not revealed in advance. If someone can give me a good educational reason why pupils need to be shown mark schemes, I would like to hear it.

More evidence on the short-termism evident in this system is available in the form of an article in the Daily Telegraph today, based on a press release from the Chartered Institute of Educational Assessors.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

- Warwick Mansell

I thought this leader, in today’s Guardian, was interesting, despite the withering criticism it receives from some readers in the comments published below the piece. It alludes to different approaches to public sector reform in different parts of the UK.

- Warwick Mansell

What to make of this latest report?

On the one hand, the recommendation that key stage 2 science tests are removed means that the number of national tests has been cut from six to two in the past 12 months. Although I’m not a critic of testing per se, clearly the tests backed by hyper-accountability are not supporting education in their current form, so this is clearly to be welcomed. In addition, there is heaps of evidence that the format of the KS2 science tests, which consist entirely of one- and two-mark questions, is a particular problem for science teaching. With pupils devoting months to question practice, they have been wasting the time they could have been spending on, for example, experimental work, as the report acknowledges. And the narrowness of the science tests means the results have never been that good an assessment of pupils’ overall understanding of the subject, or the schools’ ability to teach it well.

Further feelings of equivocation come from reading Ed Balls’s reaction to the report, where he lays into conventional league tables along fairly similar lines to his speech at the NAHT’s annual conference, which was the subject of my last blog posting.

In his reaction to the expert group in the House of Commons, Mr Balls said:

“As the Expert Group says:

 

’some schools, teachers and educational organisations are concerned that the use of the outcomes of external tests for purposes which are ‘high stakes’ for schools can lead to unequal attention to all pupils’ needs, and to pupils being put under undue pressure. ’

 

“That is why the Expert Group have asked us to provide new guidance to schools to ensure that preparation for Key Stage 2 tests is proportionate and educationally appropriate.

 

“But the Expert Group also concludes that:

 

‘Many concerns about testing arise not from the tests themselves but from the uses to which the test data is put, and the impact this can have on school and teacher behaviour.”

 

I agree that this wider issue of accountability is at the heart of concerns about Key Stage 2 tests.”

The schools secretary, then, is accepting some of the concerns about hyper-accountability that have been an obsession of mine in recent years. But his prescriptions: simply telling schools not to teach to the test and the introduction of a new report card, are inadequate. The former is laughable. The latter, while having the potential to mitigate some of the pressure on schools to teach to the test and “game” the system to improve test scores, if it includes wider measures of school quality than the current set, will just force them to play the statistical game in other ways. Furthermore, there is nothing in the consultation or in Mr Balls’s response to suggest that league tables will not continue to exist alongside the report cards. And another, little-discussed, point worth mentioning is that it is unclear whether other powerful levers on school behaviour which are heavily influenced by test scores – such as Ofsted inspections and the ability of local authorities and the Government to intervene where test scores are low – will be changed to reflect the “wider” goals of the school report card. In the absence of any information to the contrary, the assumption must be that they will remain intact, suggesting again that any hope that teaching to the maths and English tests will be reduced by these reforms may be forlorn.

In terms of the Expert Group’s report itself, one of the most disappointing things is that it does not have the feel of an independent piece of work: it looks just like a Government document, and some statements sound like DCSF-speak. A scurillous rumour has it that a DCSF official actually wrote it. Given the worries set out by Ken Boston last month about the independence of organisations which supposedly operate at arms length from Government, I think this is a serious concern. The fact that all of the recommendations which were made, supposedly, independently by this group, were accepted by the DCSF does raise further questions, in my mind. Do the “experts” and the Government always agree, I wonder, or is this just an exercise in pseudo-independence in which ministers and civil servants pull all the strings?

On the report’s contents itself,  the abolition of the KS2 science tests (this week’s will be the last) is the most eye-catching element, although the extension of the single level test trial is interesting. The review says that the trial of these possible alternatives to the Sats, which has been going on in more than 400 schools for nearly two years, will continue, but with one crucial change: the schools involved will actually be held accountable for their results in the single level tests. This appears an implicit acknowledgement of probably the biggest flaw in the design of the single level test trial: the fact that, with little hanging on the results for the schools involved, there was no way of investigating any unintended consequences should they choose to prioritise their own need to raise the test scores over their pupils’ long-term learning requirements.

Other than that, the fairly bizarre recommendation to put back the Sats tests to the middle of June is worthy of comment. Not only does this mean that the pupils won’t find out their scores until they’ve started secondary school. It also stands to deprive them of some of the activities – the extended projects, the history and geography, the visits – that most teachers routinely promise them as a sweetener as these 10-year-olds face month after month of test preparation. The report says this recommendation was put forward because of the dip in pupil performance between the Sats results in year six and the start of secondary school. The report says that one reason put forward as to why children’s performances are not as good in early secondary as in the key stage 2 tests is the issue of teaching to the test (ie the KS2 scores are increased artificially by test drilling, so when the pupils get to secondary school and face tests for which they have not been prepared, they do worse). However, the report rejects this argument, and says that the reduction of teaching time devoted to English and maths after the tests in May – ie the children regress in their learning without regular lessons in English and maths – is the more likely factor. This is odd, given that elsewhere in the report, teaching to the test is documented as a factor in year six. So is it not an influence in artificially boosting the results? And, in advising the Government to delay the tests to ensure English and maths remain taught for another six weeks or so, the group effectively admits that testing is what drives teaching in primary schools. If this were not the case, why not just advise schools to teach English and maths until the end of the year? If that advice would not, in reality, work, then why is the report advising ministers to combat teaching to the test by…producing guidance to schools not to do it? This is a big contradiction, which I may reflect on in the future.

The biggest blind spot of the whole report, however, is its failure, in the pattern of many other papers for the Government, to look at the question of teaching to the test head-on, objectively and honestly. It could have asked what the extent of it is and how much damage it is doing. To be fair, the decision on the science tests is backed by an acknowledgement that there has been too much test cramming. But as far as English and maths are concerned, this is skated over, presumably because some form of national testing in these subjects for all pupils is seen as an essential part of school-by-school accountability and, when push comes to shove, the demands of school-by-school accountability always trump any educational concerns.

A final point. The report adds, laughably simplistically, that: “It is a great strength of this country’s education system that we have the level of accountability and public transparency that we do”, and that this benefits everyone in the education system. Well, the MPs who ultimately control school accountability do not seem too keen on the idea of transparency when it comes to their own affairs, do they?

- Warwick Mansell

The National Association of Head Teachers’ annual conference in Brighton captured headlines at the weekend, as heads voted overwhelmingly to ballot for a boycott of next year’s Sats tests if the Government fails to meet its demands on testing.

But just as interesting, in its way, was what Ed Balls told the conference about conventional school league tables.

These tables, of course, have played a large part in school life since the early 1990s. But, in a speech which was designed to placate the heads before they voted on the boycott plan, Mr Balls made a string of criticisms of the rankings.

He said: “The problem at the moment with our system of league tables for accountability is that whilst they may be easy for parents to understand, they measure only a narrow view of what the school actually achieves. The [current accountability system] measures only the progress of the average pupil; it does not measure the wider issues around wellbeing and around health and support. Nor does it give any credit for the role schools play in collaboration.  That’s why we need to reform our accountability system.”

He then highlighted Qualifications and Curriculum Authority research, first revealed in Education by Numbers, that schools spend an average of 10 hours a week in the months January to May in the run-up to the tests in year six, on test practice.

He said: “If it’s the case that schools are spending 10 hours a week preparing for Sats, that’s not good teaching, it’s not good practice, it’s not what I want. At the same time, we cannot go back to the days when schools and parents and governors did not have information on what schools were doing, and that’s not what parents want, either.  I have said consistently I do not think the right thing to do is to remove the key stage 2 tests, but I have already said the system is not set in stone.”

 Then, Mr Balls responded to a question from a head teacher, Gail Larkin from Auriol Junior School in Epsom, Surrey, who asked if he could explain the educational value in continuing with the annual “public humiliation” of heads based on the publication of league tables.

Mr Balls replied: “I think you are right: school league tables give you a narrow view of what a school does. It does not look at the progress of the brightest children…or at the full range of activities which head teachers do every day. It does not look at context  or the wider things about pupils which mean they are happy and doing well.”

He added: “This [is] a system where the way to improve your performance is only to focus on the two to three children who are performing just below level four. That’s not what education is about, and that’s why we need to improve accountability.”

Mr Balls’s comments must be put in context. He was arguing against league tables in favour of a new “report card” system for schools, which is likely to be introduced to give parents a wider range of information on each institution than they currently receive. Alongside conventional exam results data will come measures of pupils’ “well-being” in each school and how it helps the most disadvantaged groups it educates. It will be, it seems to me, far from immune from some of the criticisms which have been levelled at league tables over the years.

But Mr Balls’s comments on those rankings remain extraordinary. Here was the man in charge of our schools admitting that league tables have encouraged them to focus extra effort on the small group of pupils around whom the rankings revolve – that is, to “game” the system - and that they may lead to excessive teaching to the test. Many will wonder why it has taken the Government this long to reach this view, point to the fact that this practice continues to go on, often with endorsement from some organisations very close to the Government, and argue that report cards are likely to replace one set of problems with others; they are effectively just another type of league table. That said, it was amazing to hear the criticisms from a leading member of a government which has long seemed wedded to the rankings, and other forms of statistics-based accountability which exist alongside them, as a centrepiece of its education policies.

 

 

 

 

- Warwick Mansell