Thursday, February 4^th, 2010

Another misleading defence of national testing from Conor Ryan in The Independent today.

Mr Ryan, the former political adviser to David Blunkett, is entitled to his point of view. It’s just a shame his arguments are shot through with inaccuracies and false assumptions.

He sets up the piece by arguing that National Union of Teachers and National Association of Head Teachers, which want to replace the tests with teacher assessment, started to ballot their members last week on a planned boycott of this year’s tests. In fact, they merely announced plans to ballot for a boycott last week. A date when the ballott will begin has not yet been set.

More substantively, he says “The NUT and NAHT argue that the tests impose an excessive workload on their members, and force teachers to drill pupils in English and maths”. The implication of the first part of the sentence is that the unions are acting self-interestedly in advocating boycotting the tests: effectively teachers don’t want to support the current national testing system because it is hard work for them, whatever the impact on the pupils.

In fact, workload is not the main issue on which this campaign is being fought. The press release announcing the ballot listed four ways the union believed the test results should not be used: “To construct meaningless league tables of school  results”; “By inspectors to pre-judge schools based on proxy data provided by the SATs”; “To humiliate and demean the work of colleagues working in our toughest communities”; and “To force teachers to spend endless hours rehearsing past papers.”

Only the last could be said to have an impact on workload, and, even here, it could be argued that what is really being objected to is the fact that the workload involved is educationally repetitive and counterproductive, for pupils. I have also checked the NAHT’s proposals for an alternative form of accountability. It is clear from that that the argument is mainly being fought on the grounds that the tests have educationally undesirable side-effects and subject schools in disadvantaged areas to ritual “humiliation” which does the pupils more harm than good.

The one union which is explicitly supporting Sats tests, the National Association of Schoolmasters Union of Women Teachers, has actually done so on workload grounds: it fears that any alternative assessment system would add to teachers’ working responsibility.

Mr Ryan then argues that the Government’s “independent” expert group on assessment – whose independence is a moot point, given that it was appointed by ministers, worked to a remit set down by them and operated with only the most indirect input from this country’s impressive array of academic experts on assessment – “found these tests to be ‘educationally beneficial’.”

In fact, the group’s report said that “externally marked tests in English and maths at the end of key stage 2 can be [my italics] educationally beneficial as well as necessary for accountability purposes”. It then adds, in a heavy caveat not acknowledged in Mr Ryan’s piece: “However, we cannot ignore the risk that tests whose results are used for high-stakes accountability purposes can adversely lead to narrowing of the curriculum, ‘teaching to the test’ and undue pupil stress. We do not support drilling or narrow test preparation”.

Of course, in citing this one report, Mr Ryan also overlooked the wealth of evidence to have emerged in recent years on the use of test data backed by hyper-accountability. This website attempts to chart this evidence (see here, and my blogs), but it is, of course, worth highlighting some examples again.

Advisory Committee on Mathematics Education: “The continual testing and practising for tests has resulted in a narrow and impoverished mathematics curriculum, and poor quality teaching of that curriculum.”

Institute of Educational Assessors: “We may be churning out individuals who can pass tests and who can achieve good results to a given, known test but who cannot necessarily apply their knowledge and skills to other situations.”

National Association of Primary Education: “In a great many schools, coaching for test performance has replaced education.”

The (truly independent) Cambridge Primary Review: “The narrow focus of Sats…should be replaced” and “There is an urgent need for a thorough reform of all aspects of the assessment system in England.”

A Children, Schools and Families Select Committee inquiry into assessment, while – to be fair…- backing the concept of some form of national testing, concluded: “We consider that the current national testing system is being applied to serve too  many purposes.”

Well, no doubt Mr Ryan will dismiss this evidence as the self-interested outpourings of a teaching profession which does not like the outside scrutiny which high-stakes testing brings. But many of the organisations making complaints have no axe to grind: they do not exist to serve the needs of teachers, but education more generally.

Mr Ryan also cites a poll by IPSOS Mori which showed that “75 per cent of parents think information on the performance of primary schools should be made public”. Well, there’s a surprise. This doesn’t mean it has to be test-based information of the current kind, on which schools’ futures also hang. It  would not, of course, invalidate the provision of information based on other assessment methods, such as in-class or teacher assessment. And he says “70 per cent of parents place value on the tests in providing information about how their child’s school is performing”. Well, this is one of the purposes of the tests so it’s not surprising that parents would think it important. The question is whether another system could do it better, or without the current educational downsides.

I could keep going, but the fundamental point that Mr Ryan makes in this article is that because our education system is functioning in a particular way – in this piece, he highlights educational inequality and the fact that “one in four pupils fails to reach the expected [formerly, the average] standards in English and maths” – we need the current testing system or things will be worse.

It’s a specious argument, as if Mr Ryan is saying: “The education system isn’t working well enough, effectively letting down poorer pupils. Only the testing system  I support[which has been in use for years, by the way] can address this. Therefore, not to use the testing system I support is to let down poor pupils.”

Behind the piece is the view that the only form of accountability for schools is one which goes through Whitehall, with a testing system in which the goal for every school is to please those monitoring the test results at the DCSF.

As has been argued many times, there are other ways of providing information on the national quality of our education system, such as assessing a small sample of pupils every year on a much broader range of skills than are measured in the tests. (One of two tests last year on which all 11-year-olds’ writing – effectively their progress in the last four years – was assessed centred on their ability to write a piece about a pair of trainers. Impressed?)  There are other ways of providing information to parents, such as teacher judgements on the performance of a child over a number of years. There are other ways of holding schools to account, such as inspections and moderated teacher assessment, possibly checked by testing a small sample of pupils in different aspects of the curriculum.

Given the demonstrable educational downsides of the current test-based accountability regime in many schools, should we not be looking properly at these alternatives, rather than seeking to mislead people into sticking with the status quo?

-          I posted last year on Conor Ryan’s reaction to the Cambridge Primary Review’s report on the curriculum. View it here.

Wednesday, February 3rd, 2010

Well, there’s been a bit of a gap in these postings, caused by the serious distraction that is moving house, followed by the rather-less-stressful distraction of a holiday to recover from that. But I’m getting back on track now, and hope to be posting regularly.

That said, this first one of the year is just a link to a blog I’ve written on another site. I’ve written a piece for the NAHT on why publishing school test scores seems so attractive for politicians, including those in other countries, despite the problems that often follow for pupils. Read it here.

I’ve also been reading a fascinating book, not directly related to education, which I think has some interesting implications for the way schools (and other public services I guess) are now run. Watch this space for my next post on that.

Warwick

Thursday, December 10th 2009

I’ve not done this before, but I’m posting an exclusive news story here. In the New Year, I intend to have a personal website/blog not directly related to the book. This might feature some articles which I’ve not had published. In the meantime, this seemed the best place for the story below.

Warwick Mansell

Teachers preparing to mark next year’s Sats exams for 11-year-olds have been banned from speaking to the media or discussing any problems with the tests in online discussion forums.

Contracts sent this week to markers by the test marking firm Edexcel include a gagging clause forbidding them from disclosing any information about the marking process to reporters or, most contentiously, among themselves on “online forums or social networking sites”.

The contracts, themselves marked “confidential”, say: “You must not disclose any information to the media or any other forum in the public domain, including posting on online forums and social networking sites, about any aspect of your work with Edexcel without the express permission of the company.” The stipulation about online forums and networking sites did not appear in last year’s contracts. Examiners have always been banned from sharing publicly information about the content of tests in advance of them being taken, but the new clause goes far further than this, to extend to all aspects of the examining process.

A statement from the company suggested that the ban also applies to GCSE and A-level examiners.

In recent years, Sats examiners, either retired teachers or serving professionals who mark in their spare time at home, have used prominent websites which are open to the public to post comments about the marking process, including many complaints.

Thousands of messages of unhappiness, including complaints of papers going astray and glitches in the computerised systems used to check on marker quality, were registered on the Times Educational Supplement’s website during the 2008 marking fiasco which led to the late announcement of 1.2 million pupils’ results and the termination of the contract of the previous company responsible for the marking, ETS Europe.

This helped to trigger media coverage of the problems. In 2009, a lesser number of complaints on the forums also led to reports of difficulties with the marking process. Examiners say the forums are also vital in helping them to address perceived anomalies in the tests’ mark scheme.

One experienced marker of the English tests said: “I feel very uncomfortable about being effectively barred from any communication with other markers.  Being able to discuss concerns and share opinions was the one thing that made the process manageable this year.”

A second English marker said: “Other than the contact we have with our supervisor, markers are in total isolation. The forums are a very major source of information and support.  For example, it is only by being able to exchange information with other markers that we can confidently get clarification on the accepted and agreed marks relating to answers to particular questions.

“Perhaps Edexcel are concerned about the discussion forums. But they are not making our job any easier.”

The Sats tests in English and maths are not guaranteed to go ahead next year, with two major teachers’ unions threatening a boycott of the exams scheduled for next May. Some 76 per cent of National Union of Teachers members taking part in an indicative ballot on a boycott of next year’s tests said they would support a ballot, it was announced today (Thursday). The turnout was just under 25 per cent.

An Edexcel spokesperson appeared to confirm today that none of its markers and examiners would be allowed to post discussions of the marking process on public discussion forums such as those of the TES.

She said:  “The clause is included in all Edexcel marker contracts (regardless of which qualification they are marking for) to ensure the integrity and confidentiality of our tests and examinations and of student information and exam responses. Support is available to national curriculum test markers through their supervisors, a dedicated website and the Edexcel subject team.”

By the way, you can now follow me on twitter @warwickmansell.

Wednesday, December 2^nd

I woke up the other morning, sadly enough, thinking about school accountability. Maybe I need to get away…

No, seriously, I was just having a think about the repercussions of some documents I came across which had been produced by the communications team at Ofsted, and about which I wrote an article for the TES on Friday.

The piece made the point that much of the inspectorate’s effort – or at least that of its communications team – is spent defending and trying to promote Ofsted’s reputation. To judge from these papers, the inspectorate tends to view public announcements on the state of schools as an opportunity to showcase the Ofsted “brand”. It even, on one occasion, appeared to misuse evidence from a survey of teachers on how they felt about the inspection process to present a more positive picture  (for Ofsted) than the evidence would support. Simple public interest considerations – just telling the truth about schools, and being straightforward with evidence, and letting organisational reputation take care of itself  – appeared to have gone by the board.

What are the implications of this? Well, I think it has interesting repercussions for a debate (in my head, at least) about which of a variety of public sector organisations, and those within them, actually serve the public well.

Teachers are now very heavily regulated, through Ofsted and through the performance indicators/league table system of institutional accountability. Implicit in this system is that such a structure is necessary to ensure that the professionals being monitored act in the public interest. We need an inspection system of the current type, for example, would be the argument, partly to guard against schools letting their pupils down. We need statistics-based accountability to remind teachers that their job is to produce good exam results for their charges, and that they will be held to account if they fail to do so.

If you read “Instruction to Deliver”, the 2007 book by Sir Michael Barber, who set up and ran Tony Blair’s public services “Delivery Unit”, the view clearly espoused there is that the road to success is to have civil servants intensely and ceaselessly monitoring public sector “outcome” statistics, and then seeking to cajole and prod others into making those figures rise.

But the paradox, for me, is this. The system vests a lot of trust in those working for or within institutions such as Ofsted or the Department for Children, Schools and Families, and effectively says that those looking at the statistics in Whitehall are seeing the true picture. Because, the implication is, we cannot trust teachers always to act in the public interest, implicitly we trust those nearer the political centre, working in these organisations, and further away from pupil-teacher interactions in the classroom, to do so.

But the Ofsted revelations suggest that sometimes loyalty to that organisation takes precedence, as far as I can see, over loyalty to any general public interest. And, I think the same is often true for the DCSF, having observed how it operates.

For example, despite several informed official requests to do so, the DCSF has never conducted a detailed investigation into the scale and impact of teaching to the test in schools. Why? After all, there is an awful lot of evidence that it is doing harm to at least a proportion of pupils. Nevertheless, the picture is still not complete: we need to know more about, for example, the proportion of primary schools which go in for very extensive teaching to the test, and their characteristics. But, to repeat, the Government has never looked at this in any detail. Why not? The only conclusion can be that it would not be in the department’s interest to do so – it would be embarrassing and politically inconvenient- whatever the educational effects on pupils.

The DCSF has also recently rejected without consideration a major inquiry into primary education which drew on thousands of items of evidence, including extensive public consultation. Why? Again, the only conclusion to draw is that the Cambridge Review was politically inconvenient for the government. If this analysis is true, government interest again trumps the public interest.

Now consider again those public servants who are at the sharp end of the drive to improve education, the teachers .They are far from perfect, of course, and are also subject to institutional reputational pressures from their schools, which are covered extensively in my book.  But there is one thing we should remember. Teachers are actually far less removed from those whom the policy/political process should be all about helping – the pupils – than those who are regulating them. For, if ever teachers had any doubt as to what constitutes serving the public, they are reminded of it every day of their working lives, in the form of the pupils they educate. If they let those pupils down, they are likely to see the results in human form. They may also see that the published statistics, so crucial, politically, to those at the centre, often do not tell the whole story.

The great shame is that we seem to put so much trust in those regulating this regime, who are more detached from the pupils it is meant to be serving, rather than in those whose job it is to help them, on a day-to-day basis. There should never be absolute trust in teachers. But it does seem to me to be perverse that institutions which are shown to act cynically in pursuit of their own interests can have such power over those on the front line many of whose first thought, surely, will be to do what they can to help those they are meant to be educating.

I may have articulated better similar ideas to this in a recent piece on my NAHT blog, which is available here.

Sunday, November 15th

What to make of England’s politically-driven education debate? Well, often it seems to verge on the surreal.

The latest example came my way today, courtesy of two articles in the Sunday Telegraph. One was a piece by Ed Balls, the schools secretary, setting out his plans to “intervene” in schools where results are said not be good enough. The other was the news story in which further flesh was put on this announcement. Both talk about a new bill, to be announced in the Queen’s Speech this week, which will include these capabilities.

The first sense I got, on reading the pieces, was one of bewilderment. For the latest piece of education legislation, the Apprenticheships, Skills, Children and Learning Act, only became law days ago. This already gives Mr Balls and his successors sweeping powers to intervene in schools where performance is said not to be good enough.

That Act gives the Secretary of State the right to intervene – to direct local authorities to take action – in schools which are not just deemed to be underperforming as measured, presumably, by national test and exam results. He will also have these powers in respect to schools which “may in future be low-performing”, suggesting Nostradamus-like capabilities reside within Whitehall. Even this was not the first time similar intervention capabilities have been enshrined in law: legislation passed in 2007 allowed councils to step in to put pressure on the managements of schools which were said by exam stats to be “coasting”.(See story here) Thus, the 2009 Act was itself a ramping up of powers which are now, a few days later, deemed insufficient.

So what is the point of the new bill? How could it possibly go further than the powers already, now, on the statute book? Well, the news story puts forward some other elements to be included, such as forcing schools to conduct annual surveys of parents, and to draw up an “action plan” to tackle problems. The story adds: “The bill will make it easier for offending (my italics) institutions to be taken into partnership with successful schools, be run by outside education providers or even, in extreme cases, be closed by ministers.”

Again, leaving aside the potentially offensive - to those working in these schools - language,  I am struggling to see much that is new in all of this. And if these powers were needed, why were they not included in the previous bill? The obvious, cynical perhaps but then watching what goes on closely encourages that kind of viewpoint, explanation is party politics: it make sense to have a new bill, and a new set of headlines, which purport to show a government which is tough on ”underperformance”. A governing party in need of votes needs to present afresh its policies to voters. Another law helps it to do that. So we get another law.*

Being charitable to the Government, it could simply be that local authorities, who have had these intervention powers for a while now, have not been keen enough to use them, and that pupils and their parents are the losers. Mr Balls accused some local authorities in his article of “dragging their feet” on intervention, and refusing to react to poor results by forming plans to turn the school in question into an academy, or to form a partnership with another high-performing school.

This, for me, is the key to the whole debate. When a local authority, or a school, decides to take action which might be seen from afar as not doing enough to tackle underperformance, who is best positioned to make the judgement as to whether this is right or not? For the Government, looking at the statistics of a school’s results from Whitehall, this position may look like complacency. But local people can have all kinds of reasons for deciding to take this course of action: they may not like the form of an academy proposal; they may have reservations about the fact that academies often replace clear local accountability and control by a community with handing power to a sponsor and to the Department for Children, Schools and Families; or they may believe that the school’s management is best placed to lead it under the current arrangements. Although institutional changes will give the impression that something is being done, many will argue that the key to good schooling is the quality of teacher-pupil interaction in the classroom, and simply changing institutional structures, which can be traumatic for pupils, may not be answer.

They may, then, have legitimate reasons for hesitancy. But our democratic system operates, now, from the political centre with statistics – rather than the day-to-day local engagement with an institution which one might get from, say, visiting it – the prime means by which one “sees” the quality of any school, hospital or whatever. Under this structure, local views can effectively be written off as “complacency”, and Mr Balls’s view is the one that must prevail. The true wisdom of what to do in any given situation, then, comes from Whitehall and Westminster, backed by results spreadsheets. But crucially, of course, as argued in my book, the accountability that occurs here is at the at-times superficial level of the official stats; it cannot see the prices that are sometimes paid by pupils in the drive to raise the numbers, such as the focus on particular courses, particular pupils or through teaching to the test.

As should be clear, these arrangements now vest a lot of trust in the Secretary of State, the one person through which democratic accountability effectively now travels. Under this line of argument, he is the guard against complacency within the public sector: by teachers, heads and local authorities. And, in taking action to intervene, he will always act in the public interest because…the public interest is for our schools to improve, as measured by the Secretary of State’s performance measures. Anyone questioning this is, by extension, simply another of those looking to excuse decades of underperformance.

There are a huge number of questions behind all of this. Perhaps the most pressing, though, is whether the exam numbers themselves, which have improved over recent decades, represent true progress, or something else. If it’s the latter – and, as my book argues, “gaming” of the system to generate the results it demands is likely to be a factor in some of the gains – big prices may be being paid.

 *The Conservatives will also be in on the act if elected: Michael Gove, Mr Balls’s shadow, promised in his speech to the party’s annual conference to take tough action in struggling schools as soon as/if they are elected. I am told that this will use Ofsted inspections as a key judging mechanism, rather than only relying on stats.

Tuesday, November 10th

I was at the Guardian’s Innovation in Education conference in London yesterday, and noticed several comments arguing that the current accountability regime stifles fresh thinking in schools.

One of the keynote speakers, Larry Rosenstock, the chief executive of a group of American semi-independent charter schools in San Diego, certainly appeared to be making this argument, although I think he was probably the fastest speaker I’ve ever heard at a conference, so it was hard to keep up.

But he appeared to be arguing that centralised standards-setting and regulation by, for example, high-stakes testing and regulation, could drain the life out of a school system.

He said:  “If setting standards translates as expectations and challenge for students, then I am totally for that. But what tends to get practised in schools is standardisation of what is provided, which tends to suck the oxygen of innovation out.”

The director of the Finnish National Board of Education also argued that his country’s high-trust system, which lacks an English-style inspection system, league tables and national tests, helped promote fresh thinking.

But for me perhaps most powerful were the comments I received from those on the floor of the conference. A local authority adviser told me that innovation in education was all very well, but it wouldn’t happen while schools are under such huge pressure to demonstrate immediate, short-term results. The epitome of this was the Government’s National Challenge programme, under which schools are threatened with closure or take-over if their results remain below 30 per cent achieving five A*-Cs at GCSE, including English and maths. In this environment, it must be difficult for any other thinking than desperate back-covering in the drive for better scores take precedence.

This adviser said that those well above this threshold were reluctant to change what they were doing, with an “if ain’t broke, don’t fix it” view of what they did. Those just above the line thought they better not risk doing anything differently, for fear of falling below it. Ironically, it was only those who were well below the cut-off who thought they might as well try something alternative, as they had nothing to lose.

He added that the targets culture undermined true education. He said: “The way that schools hit their targets is by narrowing the curriculum and manipulating things. That’s not education.”

An employee of a national advisory body said: “There are many significant barriers to innovation at the moment [in the form of ]accountability for short-term improvement and targets. It’s really frightening for teachers: who is prepared to put their career on the line to do something different?”

A teacher from the floor said: “The prevailing conditions are probably against innovation right now. We have got a potential change of government, so people do not know what the situation will be next year. And Ofsted is increasingly focusing on data such as contextual value added scores, and performance especially in maths and English, so I do not think the conditions are particularly favourable for innovation.”

To be fair, the view of the conference overall was more optimistic: two thirds of the 200 or so there agreed or strongly agreed with the statement “the current conditions present great opportunities for radical innovation in education”. But the influence of the accountability regime was a common theme throughout the day, and “fear” was top of the reasons why conference guests worried that innovation might be stifled.

To be clear, and this may be slightly heretic to those attending the conference, I don’t personally see innovation as a good in itself. it could be argued that experiments with learning that do not work – which are surely a likely risk for truly radical innovation – are too high a price to pay when each student only gets one chance at full-time compulsory study. Therefore, it could be argued that a truly radical free-for-all in terms of new approaches to learning is a step too far. But, as ever with this subject, it is a question of balance. The ability of a system to promote sufficient engagement from teachers to want to change practice for the better – their intrinsic motivation to do a better job – does seem to me to be something that we undermine at our peril.

As another aside,  it is interesting that it is the accountability system, not other apparently significant changes which appear to suggest more freedom to teachers such as the recently reformed and stripped down secondary curriculum, which are perceived as having more influnce on whether true innovation flourishes.

All in all, the effect on innovation of the current standards straitjacket could be another disadvantage to put on the seemingly ever-growing list.

Monday, October 26th

A collection of insightful views on the Cambridge Primary Review featured in last Friday’s TES letters column.

See: http://www.tes.co.uk/article.aspx?storycode=6025758

 http://www.tes.co.uk/article.aspx?storycode=6025759

 http://www.tes.co.uk/article.aspx?storycode=6025760

and: http://www.tes.co.uk/article.aspx?storycode=6025761

Wednesday, October 21st

The Archbishop of Canterbury, Rowan Williams, attacked the school accountability system as “oppressive” and “mechanistic” today.

See: http://www.telegraph.co.uk/education/6398384/Archbishop-of-Canterbury-attacks-oppressive-education-system.html#

Monday, October 19th

It was pleasing to see Education by Numbers mentioned in the final report of the Cambridge Primary Review. I will post more on the contents of the near-600 page review book when I have had time to digest some more of its findings. But I will just note for now how impressive it is to read something which is so careful with the evidence, and so independent in its conclusions.

Saturday, October 10th

The Daily Telegraph carries a preview today of the Cambridge University-based Primary Review’s final report. This is said to call for an end to Sats tests in primary schools.