Tuesday, February 23^rd.

I loved maths at school. I studied it to A-level and enjoyed the more mathematical aspects of my degree.  I think that, as a country, it would be better if there were many more people who enjoyed the subject and felt confident with it.

However, I was deeply uncomfortable on watching Channel 4’s two-part Dispatches programme, “Kids Can’t Count”, which finished last night. My unhappiness centred not on what was revealed – the problems/challenges facing maths teaching have been known for a long time – but with the nature of the reporting.

Having been prepared to give the programme the benefit of the doubt in its first week, I came to the view after watching yesterday’s instalment that this was –I know this might be stating the obvious, but bear with me – just sensationalism, offering little context and serving up a message of easy teacher-bashing which will have offered few insights for the audience other than “our schools are rubbish”. I think the general public deserves a lot better and that if this is the standard of debate and insight into what is really going on in our public services, we really are struggling.

The overall tone was of doom-mongering about the state of maths understanding in our schools system, which, as I said, is always a serious issue. But the analysis of the problem, and what to do about it, was shallow and completely uncurious.

For instance, although the programmes wheeled out statistics for the proportion of 11-year-olds who do not reach the Government’s “expected” level in maths tests, they failed to put them in context. There was a brief mention that the figures for those achieving to the Government’s expectations are higher than they were in the mid-1990s, not mentioning that this suggests, if taken at face value, a transformation in maths standards that few other countries have achieved. Having stated the fact that there had been an improvement, it then simply moved on to report that progress on these statistics has stalled in recent years, which is true. There was no mention that the Government’s “expected” level in the tests was originally defined as what the average child should achieve.

One perspective was also missing: results in one well-known set of international tests show that England’s 10-year-olds had the most improved scores of any of the countries taking these tests in the years 1995 to 2007. Clearly, I don’t think results from any testing programme should be held up as anywhere near infallible. But I think this fact should have been included, in the interests of balance.

Lack of any attempt to get a countervailing view to the sense that things were going to the dogs was apparent throughout. Justin King, the chief executive of Sainsbury’s who was probably the most prominent interviewee and complained about his recruits’ lack of mathematical understanding, was allowed to make the point that, in his day, ‘O’-level maths papers featured a non-calculator paper, implying it might be a good idea to return to this. The programme failed to tell viewers that non-calculator papers are a feature of current GCSEs, and that national tests for 11-year-olds have a “mental maths” test, in which children have to do calculations in their heads.

The programme also included unchallenged complaints that England suffers compared to our competitors in that young people are not required to take maths beyond the age of 16. This is a valid argument, of course, but again, it needs context. No subjects are compulsory beyond the age of 16 in this country, and freedom of choice has been arguably the defining feature of our post-16 system, argued for very strongly by its defenders; certainly attempts at changing the A-level system towards more compulsion have been fraught with difficulty, not least because of tradition: youngsters have had freedom at 16 whether or not to stay in the education system and changing that is difficult. This is not to say that it is impossible to reform, just that it’s not quite as simple as the programme implied. Why was nobody allowed to make this point, or offer a counter-view?

Overall, I watched the programme with a sense of growing incredulity that it had not sought to investigate its subject matter properly. Bashing the education system is easy, but why was Dispatches so reluctant to find out precisely what was going wrong, on a national scale, if it is? What was the best evidence as to why a pupil could go through school without a better mathematical understanding, a phenomenon which seems to have been going on for a very long time, and not in this country alone? I would submit that no-one, be it politicians or educators, are going out deliberately to harm or undermine pupils’ mathematical understanding.  So if their strategies were not working, why were they not?

We were offered few glimpses of any answers, beyond a survey of 150 teachers which suggested that their mathematical knowledge might not be up to scratch, (which I will admit, if the survey was representative, was an interesting and newsworthy finding). But could this really be solved at a stroke by increasing the maths qualification requirements for new teachers?

The programmes included extensive sections with a maths consultant working in what looked like a skilled way with a school to make the subject more lively. Last night, Carol Vorderman’s replacement on Countdown was featured trying again to liven things up. The pupils seemed enthused and there seemed to be an impact on test results. But it is a well-known phenomenon of education research that experts going into small groups of schools and pouring expertise and attention towards them can have an effect. The problem is that it is often difficult to replicate these approaches on a large, country-wide scale. Again, I wanted to know if research had been tried “livening up” lessons on a large scale in England, and if so, what the results had been.

In fact, the Government did introduce a national numeracy strategy into state schools in 1998. It seems to have had an effect on results. There seemed scant reference to this;  if there was, I missed it. If it had been a failure, in what way, I wondered.

There was, then, very little discussion about the detail of what was wrong with maths teaching in England, apart from the vague sense that things were not exciting enough. I did also spot one glancing reference to what could be seen to be a problem. A child was pictured doing an arithmetic exercise, and Rachel Riley, from Countdown, remarked that they seemed to being led to the answer by the calculating aid they were using, and that this was a problem. Again, if it was a problem, was this a one-off with this particular resource, or a nationwide issue? The implication was the latter, but we were not actually told.

The first programme did feature a discussion of the problems of teaching to Sats, with comments from teachers and the maths consultant. But, again, there was no real investigation into this, perhaps because the figures on which the programme were hung – the proportion of pupils not reaching level four in the tests – rely on using Sats as a valid measure of underlying mathematical ability. What if pupils, either fed up or stressed out by the weeks of revision, were underperforming in the tests?  

You could argue that I’m being ridiculous. This was a prime-time documentary, not an academic paper, and all this nuance could just have swamped viewers. But I think there is a real problem, here. And I think all of us deserve better: this is not a national newspaper with a political viewpoint which is clear to its readers. It is a broadcaster which is, if I understand this correctly (I’m no media specialist), ultimately a public body.

As I say, I don’t think the documentary enlightens viewers in any way in searching out answers to questions such as: what is the true scale of the problem, if there is one, why have things been going wrong and what – in a sense which goes beyond the effects in a single classroom – can be done about it? That’s a real shame, because there is no doubt that this is a serious subject and a better analysis could have been really powerful in shining light on precisely what needs to improve. But the only reaction you were supposed to have was that our schools are terrible, most teachers don’t know what they’re doing and why can’t employers get the education service they need? It’s probably a version of “Oh Dearism”, see http://www.youtube.com/watch?v=8moePxHpvok, a term coined by the filmmaker Adam Curtis in relation to foreign affairs, but, I think, relevant here. Instead  of enlightening viewers, we are instead encouraged to just look at a problem and think “that’s terrible, isn’t it?”. It’s completely disempowering.

There is something else going on. There is something arguably quite sinister, and authoritarian, in arguments which take a simple, seemingly uncontestable proposition, such as the notion that our children need better maths skills, and then move from that to say that things have to change, and have to change in the way that the person arguing for them – in this case, the documentary-makers, in advocating more lively maths teaching – says.

Because the premise would appear to be almost inarguable, people can be encouraged to go along with the particular conclusion being put forward. Arguably, this is education’s central problem. Standards are never high enough. Teachers are never good enough. Therefore, we must change the system, is the argument. And anyone who argues against this is just defending the status quo and does not care that standards are not high enough.

And hence we have endless change and reform, with all its attendant problems. But it is not a rational debate when people are not taking time to really understand what is going wrong, to work out what has worked and what hasn’t, and then to move on.

Far easier simply just to moan: “Oh dear. Something must be done.”

- Warwick Mansell

Thursday, February 4^th, 2010

Another misleading defence of national testing from Conor Ryan in The Independent today.

Mr Ryan, the former political adviser to David Blunkett, is entitled to his point of view. It’s just a shame his arguments are shot through with inaccuracies and false assumptions.

He sets up the piece by arguing that National Union of Teachers and National Association of Head Teachers, which want to replace the tests with teacher assessment, started to ballot their members last week on a planned boycott of this year’s tests. In fact, they merely announced plans to ballot for a boycott last week. A date when the ballott will begin has not yet been set.

More substantively, he says “The NUT and NAHT argue that the tests impose an excessive workload on their members, and force teachers to drill pupils in English and maths”. The implication of the first part of the sentence is that the unions are acting self-interestedly in advocating boycotting the tests: effectively teachers don’t want to support the current national testing system because it is hard work for them, whatever the impact on the pupils.

In fact, workload is not the main issue on which this campaign is being fought. The press release announcing the ballot listed four ways the union believed the test results should not be used: “To construct meaningless league tables of school  results”; “By inspectors to pre-judge schools based on proxy data provided by the SATs”; “To humiliate and demean the work of colleagues working in our toughest communities”; and “To force teachers to spend endless hours rehearsing past papers.”

Only the last could be said to have an impact on workload, and, even here, it could be argued that what is really being objected to is the fact that the workload involved is educationally repetitive and counterproductive, for pupils. I have also checked the NAHT’s proposals for an alternative form of accountability. It is clear from that that the argument is mainly being fought on the grounds that the tests have educationally undesirable side-effects and subject schools in disadvantaged areas to ritual “humiliation” which does the pupils more harm than good.

The one union which is explicitly supporting Sats tests, the National Association of Schoolmasters Union of Women Teachers, has actually done so on workload grounds: it fears that any alternative assessment system would add to teachers’ working responsibility.

Mr Ryan then argues that the Government’s “independent” expert group on assessment – whose independence is a moot point, given that it was appointed by ministers, worked to a remit set down by them and operated with only the most indirect input from this country’s impressive array of academic experts on assessment – “found these tests to be ‘educationally beneficial’.”

In fact, the group’s report said that “externally marked tests in English and maths at the end of key stage 2 can be [my italics] educationally beneficial as well as necessary for accountability purposes”. It then adds, in a heavy caveat not acknowledged in Mr Ryan’s piece: “However, we cannot ignore the risk that tests whose results are used for high-stakes accountability purposes can adversely lead to narrowing of the curriculum, ‘teaching to the test’ and undue pupil stress. We do not support drilling or narrow test preparation”.

Of course, in citing this one report, Mr Ryan also overlooked the wealth of evidence to have emerged in recent years on the use of test data backed by hyper-accountability. This website attempts to chart this evidence (see here, and my blogs), but it is, of course, worth highlighting some examples again.

Advisory Committee on Mathematics Education: “The continual testing and practising for tests has resulted in a narrow and impoverished mathematics curriculum, and poor quality teaching of that curriculum.”

Institute of Educational Assessors: “We may be churning out individuals who can pass tests and who can achieve good results to a given, known test but who cannot necessarily apply their knowledge and skills to other situations.”

National Association of Primary Education: “In a great many schools, coaching for test performance has replaced education.”

The (truly independent) Cambridge Primary Review: “The narrow focus of Sats…should be replaced” and “There is an urgent need for a thorough reform of all aspects of the assessment system in England.”

A Children, Schools and Families Select Committee inquiry into assessment, while – to be fair…- backing the concept of some form of national testing, concluded: “We consider that the current national testing system is being applied to serve too  many purposes.”

Well, no doubt Mr Ryan will dismiss this evidence as the self-interested outpourings of a teaching profession which does not like the outside scrutiny which high-stakes testing brings. But many of the organisations making complaints have no axe to grind: they do not exist to serve the needs of teachers, but education more generally.

Mr Ryan also cites a poll by IPSOS Mori which showed that “75 per cent of parents think information on the performance of primary schools should be made public”. Well, there’s a surprise. This doesn’t mean it has to be test-based information of the current kind, on which schools’ futures also hang. It  would not, of course, invalidate the provision of information based on other assessment methods, such as in-class or teacher assessment. And he says “70 per cent of parents place value on the tests in providing information about how their child’s school is performing”. Well, this is one of the purposes of the tests so it’s not surprising that parents would think it important. The question is whether another system could do it better, or without the current educational downsides.

I could keep going, but the fundamental point that Mr Ryan makes in this article is that because our education system is functioning in a particular way – in this piece, he highlights educational inequality and the fact that “one in four pupils fails to reach the expected [formerly, the average] standards in English and maths” – we need the current testing system or things will be worse.

It’s a specious argument, as if Mr Ryan is saying: “The education system isn’t working well enough, effectively letting down poorer pupils. Only the testing system  I support[which has been in use for years, by the way] can address this. Therefore, not to use the testing system I support is to let down poor pupils.”

Behind the piece is the view that the only form of accountability for schools is one which goes through Whitehall, with a testing system in which the goal for every school is to please those monitoring the test results at the DCSF.

As has been argued many times, there are other ways of providing information on the national quality of our education system, such as assessing a small sample of pupils every year on a much broader range of skills than are measured in the tests. (One of two tests last year on which all 11-year-olds’ writing – effectively their progress in the last four years – was assessed centred on their ability to write a piece about a pair of trainers. Impressed?)  There are other ways of providing information to parents, such as teacher judgements on the performance of a child over a number of years. There are other ways of holding schools to account, such as inspections and moderated teacher assessment, possibly checked by testing a small sample of pupils in different aspects of the curriculum.

Given the demonstrable educational downsides of the current test-based accountability regime in many schools, should we not be looking properly at these alternatives, rather than seeking to mislead people into sticking with the status quo?

-          I posted last year on Conor Ryan’s reaction to the Cambridge Primary Review’s report on the curriculum. View it here.

- Warwick Mansell

Wednesday, February 3rd, 2010

Well, there’s been a bit of a gap in these postings, caused by the serious distraction that is moving house, followed by the rather-less-stressful distraction of a holiday to recover from that. But I’m getting back on track now, and hope to be posting regularly.

That said, this first one of the year is just a link to a blog I’ve written on another site. I’ve written a piece for the NAHT on why publishing school test scores seems so attractive for politicians, including those in other countries, despite the problems that often follow for pupils. Read it here.

I’ve also been reading a fascinating book, not directly related to education, which I think has some interesting implications for the way schools (and other public services I guess) are now run. Watch this space for my next post on that.

Warwick

- Warwick Mansell