Most journalists –I like to think, at least – try to probe beneath the surface a little bit in seeking to get some insight into what is going on. Sometimes, this process can be frustrating, especially when evidence comes in which does not fit a pre-conception, hypothesis…or strong news line.

Late last week, I had some experience of this when doing some number-crunching on this year’s national test results.

I wanted to find out if the thousands of schools boycotting this year’s tests – some 26 per cent of the total – would have any effect on the national data generated at the end of the process.

My thinking went along the following lines. I knew that strenuous efforts were made by the testing authorities – principally, the Qualifications and Curriculum Development Agency (QCDA) – to hold the standard of the tests constant every year.

That is: the QCDA oversees a complex process, including “pre-testing”, whereby it is supposed to ensure that it is just as easy – or difficult – for a child of a certain ability and understanding of their subject to achieve a certain level one year, as it would be the next.

When entire national cohorts of 11-year-olds take the tests every year, if the difficulty* of the tests is held constant, then the numbers passing them should give a good idea of the overall national standard of understanding of that particular cohort, in the tested parts of each subject at least, it is argued.

However, what if, in the case of a boycott, the average ability** levels of pupils taking the tests in one year changed? What if pupils from the boycotting schools tended to come overwhelmingly from those who would have been expected to achieve the Government’s target level very easily? In that case, assuming there was no major change in the overall national ability profile compared to the previous year, the sample of pupils taking the tests would have been skewed, because the number of high-achieving pupils taking the tests would have been reduced because many would have been part of the boycott.

If the tests were just as hard as in previous years, results would then fall because of the number of likely high achievers taken out of the test-taking “sample”. But this would say more about the effect of the boycott than it would reflect any fall in national standards overall.

Conversely, if the boycotting schools tended to have a pupil population of lower-than-average ability in the tested subjects, then results might rise as the proportion of high-ability youngsters taking the tests went up.

A couple of weeks ago, after getting hold of the list of schools which boycotted the tests, I began to wonder if the second scenario might hold true. I wrote an analysis piece which ranked the regions of England on take-up for the boycott.

Although finding precise patterns was tricky, it did seem that regions with higher proportions of pupils eligible for free school meals tended to have higher support for the boycott, while those with lower free school meal eligibility tended to have higher numbers of pupils taking Sats as normal.

With pupils eligible for free school meals established, by a great deal of research, as less likely to do well in tests, I thought this might translate to mean that the boycott disproportionately took out from the test-taking sample children who might have been expected, on average, not to do well in the tests.

If this were the case, I thought, even if the results of those taking the tests rose, this might say less about national standards rising, and more about the effects of the boycott.

But this was only a hypothesis. Fortunately, it was testable, if only in a rough way. I got hold of the national test results for 2009 of all primary-age schools in England. I then checked to see if the schools which boycotted the tests this year had higher-than-average results in 2009, or lower-than-average.

And the answer? Well, to my surprise their pupils’ performance, at least in terms of the numbers achieving the Government’s “expected” level four, was almost spot on the national average last year by my calculations.

Nationally, 80 per cent of pupils achieved level four in English, 79 per cent in maths and 88 per cent in science in 2009. Among the schools which boycotted the tests this year, the respective percentages were 79.6, 79.2 and 88.5. The average points score, which the Government calculates by simply adding up the percentage of level fours across all three subjects in each school, was 247 nationally last year and 247.4 in the 2010 boycott schools, I found.

So, there I was expecting to write about how the sample had been skewed massively by the effect of the boycott, and how we should be able to read even less into the national scores than ever this year. Yet the numbers appear not to support such a conclusion.

That said, this may just be luck on the part of the Government. I have checked with the QCDA, whose answer suggests to me that it did not investigate the characteristics of pupils in schools boycotting the tests this year.

It said that its job was to ensure the tests were kept at a constant difficulty level each year and that this process was not affected by the industrial action.

In other words, if for any reason only a proportion of the cohort took the tests in any year and this affected what could be read into the results as a guide to overall national standards, this was not a matter for the QCDA because it was concerned with the difficulty of the tests themselves, rather than the nature of the cohort taking them.

I find that answer strengthening my belief in the merits of national sample tests/assessments as measures of overall standards for England as a whole. Under this system, by design every year the testing authorities would select only a proportion of pupils to take tests, but they would do so in such a way as to ensure it was nationally representative. In other words, they would have to control the sample of pupils taking the tests each year to ensure the results could be read as standing for England as a whole.

By contrast, if the Government sticks with the current testing model, any future boycott action could have an effect on what could be read into the results as guides to national standards.

I think this is more evidence that the problem with the current system is that there are multiple purposes for these tests: they are not designed solely as a check on national standards, of course, but to check on schools, teachers and pupils, as well as providing evidence for many other aspects of education. So the QCDA seeks to ensure that the tests are comparable to previous years for pupils and schools, but sees it as not its job to investigate the possibility of any effect on national standards caused by the sample of pupils taking the tests changing.

The Department for Education told me: “Results from the 2010 Key Stage 2 National Curriculum Tests will be published on 3 August. This publication will include any relevant commentary about the impact of industrial action on the results, including issues of representativeness.”

That looks to me as if there will be some disclaimer offered by the Government next week about being cautious about how much can be read into this year’s results as indicators of national standards, because of the boycott. My number-crunching suggests one should be cautious, but perhaps not as cautious as I thought before looking a bit more closely.

*By “difficulty”, I mean not just how hard the questions are, but how easy or hard it is for a pupil of a given ability to achieve a certain level on a paper each year.

** I am using the word “ability” loosely here, to mean the overall likely capability of an 11-year-old to do well in the national tests. I realise that there are debates over whether ability is fixed from a young age, or, of course, whether test-taking ability reflects overall understanding of a subject.

Sunday, July 25^th

An article in today’s Observer, setting out a teacher’s observations about how schools have changed during his near 40-year career, includes as “lows” the advent of Ofsted, targets and league tables.

The teacher, Alan Hemsworth, says: “I hate league tables – I think they are so destructive – and it has a spin-off in the classroom, because then everything becomes focused on results, results, results.”

The highs included German exchanges, in my view an example of an educational experience which surely has value aside from the grades it may or may not have generated for each pupil at the end.

Well worth a read:

http://www.guardian.co.uk/education/2010/jul/25/teacher-remembers-40-years

A report in today’s Telegraph quoted university admissions tutors saying they found it difficult to choose between A-level candidates partly because of “teaching to the test”:

http://www.telegraph.co.uk/education/educationnews/7872573/Universities-criticise-exam-grade-inflation.html

What to make of yesterday’s announcement that just over one in four primary and junior schools took part in the boycott of this year’s key stage 2 tests?

One head I spoke to on Monday, before the final figure was announced, said he thought that overall, take-up of the boycott had been disappointing. The head, who backed it strongly, said this had been the profession’s one chance to really stand up to government on the issue, and it had bottled it.

I guess many of those sympathetic to the boycott might hold this view about the one-in-four figure, and clearly, I am an outsider to this process who has only come to be familiar with the intricacies of this debate in the last few years. Hands up here, too: I also write a regular blog, often on assessment issues, for the National Association of Head Teachers.

But I’m not sure that a figure of 26 per cent non-compliance with the testing regime as it stands is insignificant. The fact that one in four heads were so unhappy, they were prepared to take a decision which looks to me to be quite brave, in the face of reports of pressure from local authorities – and, no doubt, some governing bodies – to go through with the tests, I think says something about the level of dissatisfaction out there. School leaders, it has been argued persuasively I think, are not naturally disposed to industrial action of this sort. And the figure comes about even despite some heads’ reservations about the timing of the action: for legal reasons, the unions were unable to ballot for a boycott earlier in the academic year, meaning that many schools had already gone in for months of test preparation before being asked to join the boycott. I think this will have put off some heads who do not like the tests from refusing to administer them.

In fact, the position that we find ourselves in, in 2010, with the testing system now having been established for a decade-and-a-half, and schools having had to work with that structure since then, is quite remarkable, I think. Having seen this system close-up for all this time, one in four heads are not only still very unhappy with it, they are so unhappy, they are prepared to boycott high-stakes testing altogether.

A government which looked at these figures dispassionately could hardly conclude that the system as it currently exists (that is, not just the tests, but the tests backed by hyper-accountability) commands much confidence from those who know it best.

-          Interestingly, for those against the boycott who might think that heads took decisions on avoiding the tests against the wishes of parents, the head I spoke to said all his year six parents were asked whether they favoured administering the tests as normal and sending the results off to the government, (ie not taking part in the boycott) or teacher-controlled assessment.

They opted, he said, overwhelmingly for the latter, as had parents at other local schools. The school assessed the pupils itself, including using some Sats tests from previous years, and used teachers from another school, which was also taking part in the boycott, to conduct moderation on the marks its staff gave to pupils for some of the tests.

-          An analysis of the socio-economic backgrounds and prior achievements of pupils in boycotting schools would also be interesting. The testing authorities will need to have looked at this data to ensure that the sample of pupils taking the tests this year is not markedly different from that of previous years. If not, national results, due out in early August, could be skewed.

 I bought today’s Sunday Times intrigued by one of the stories on its front page, under the headline: “Gove plans A-level exam revolution”.

Michael Gove, the education secretary, it said, has announced plans to make the A-level more “rigorous”, by scrapping AS levels.

Universities, it said, would be invited to design new A-levels, which would be “modelled on the new Cambridge Pre-U qualification, taken by a number of leading state and independent schools in preference to A-levels”.

Gove said: “We will see fewer modules and more exams at the end of two years of sixth form and, as a result, a revival of the art of deep thought.”

This, though likely to be far from uncontroversial among teachers and sixth formers, sounded fair enough, I thought. Of course, pupils spend far too long preparing for and taking exams in their last years at school – the last four years, for teenagers taking GCSEs and A-levels, are now dominated by them. Addressing that problem is important, I think, for all the arguments that were advanced for the benefits of the AS system during the Curriculum 2000 reforms.

However, one sentence in this report had me shaking my head in disbelief. “The existing exam boards…could continue to offer the AS/AS combination, but Gove believes schools will abandon these exams as it becomes clear that they do not meet university requirements,” said the report.

Well maybe, but expecting Gove’s suggested return to traditional A-levels to thrive and attract candidates in serious numbers alongside the current incarnation of the exam looks very optimistic to me.

Why? Well, any exam that looks like it could be harder than what is on offer at the moment will struggle to win favour from students, and certainly from their schools, if it is plunged into a marketplace which still features these existing qualifications, and where results pressures are huge.

 If teachers and schools think they can get an A grade from something which is called an A-level now, and which gives them a luxury of a half-way result which will help them both to gauge their likely overall success in the qualification, and to have the potential to resit early papers, why abandon it for a system without these qualities?

Gove would no doubt respond that schools and students would do so because his exams would be  more highly regarded by university tutors. But I’m not sure. I think the Cambridge Pre-U itself, which sounds very like the Gove A-level, has faced a battle to convince even leading independent schools to abandon the A-level because, however much some might like the Cambridge qualification educationally, A-levels as currently constituted would be perceived by many to give them more control over achieving good results.

Don’t get me wrong: as argued, on the face of it I think there are strong grounds for thinking that Gove’s A-levels would lead to a better learning experience for the pupil. But, as my book argued, schools and pupils are not necessarily using this criterion to select courses, given the results pressures they face.

Launching this exam and then expecting it to win in the A-level “marketplace” looks optimistic at best, and naive at worst. If Gove really wants this reform to succeed, given the huge pressures on schools and students to achieve results, I think he is going to have to go further and abolish the route that he clearly thinks stands to be an “easier” option: the current AS/A2 A-level.

How long did Department for Education officials and ministers spend cobbling together the official document they have published assessing the likely impact of their new academies policy?

Not nearly long enough, to judge from what looks like the thrown-together character of this paper, and some truly heroic assumptions on which some of the calculations it comes up with appear to be based.

The paper is an “impact assessment” of the academies bill, which is currently going through Parliament. The idea behind this sounds sensible: when policies come out, the government is supposed to show that it has thought through the implications of them, and considered alternatives.

So far, so good, then. But problems begin to appear for this document in the first paragraph. It opens with the questions: “What is the problem under consideration? Why is government intervention necessary?”

The answer begins: “As part of their [sic] pre-election commitments, the Government said that outstanding maintained schools would be allowed to adopt Academy status by September 2010.”

Erm…this wouldn’t be both parties within the coalition government, would it? In fact, the commitment appears to be a reference to the Conservative manifesto, which said “all existing schools will have the chance to achieve academy status, with outstanding schools pre-approved”. There is no mention, in the Conservative manifesto, however, of the September 2010 date, so the document seems to have got that factually wrong. Leaving that aspect of detail aside, clearly, the Liberal Democrat manifesto is not relevant here, as it says conventional academies should be replaced by “sponsor-managed schools”, accountable to local authorities. The equation of coalition government policy with what appear to be Conservative manifesto commitments occurs several times in the document, which states later on page 1 that the policy “is a Government manifesto pledge”.

Later on page 1, the question is posed: “When will the policy be reviewed to establish its impact and the extent to which the policy objectives have been achieved?” The answer? “It will be reviewed.” So that’s that sorted out, then. An annex at the back of the paper leaves space for the government to fill in a “Post Implementation Review Plan”, with information to be added under seven categories. All have been left blank.

There are other strange aspects of this 11-page paper. Its date is listed as effective from 01/01/2010, rather than the period on or after the coalition government’s formation, which is bizarre. (The document actually bears a signature which could be that of Michael Gove, education secretary, which itself is dated 26/5/2010, which was the day after the Queen’s Speech, when the Academies Bill began its passage through Parliament.)

Against a box in which the department is asked to sum up the policy’s impact on equalities  – a note says that “race, disability and gender impact assessments are statutory requirements for relevant policies” -  the document says there are no implications. Again, it seems bizarrely, and worryingly, to this observer, counter-intuitive to have reached this position with no evidence offered: what if “outstanding” schools being offered academy status first disproportionately educated children from one ethnic group, for example? To take this document at face value, the government deems this as not worthy of consideration. The assessment also ticks “no” in a section asking the government to identify nine other possible categories of effects, including on small firms, greenhouse gas emissions and human rights.

Under “evidence base”, the document says that academies are freer from regulation than conventional state schools, are independent of local authorities and thus have “far greater autonomy than maintained schools”. They are able to take decisions on priorities “to suit the pupils in the Academy”; are not constrained by national teacher pay structures; and can form partnerships that work for them and their pupils “without constraint”.

“These as part of a broader package of freedom flowing from autonomy, mean that Academies are freer than other schools to focus their efforts on teaching and learning and push up attainment as a consequence.”

Hmm, whatever the details of how particular freedoms may benefit academies, I don’t think it’s true in any general sense that academies have more freedom to “concentrate on teaching and learning” than other schools.

But it is the figures on the likely results improvements at academies, and the highly convoluted arguments about economic benefits that could follow that I want most to focus on here.

The paper attempts to come up with a cost-benefit analysis for the policy. On the debit side, it reckons that “total annual economic costs” flowing from setting up and running another 200 academies a year in 2010-14 will be £462 million. “Total economic costs” (I confess after reading the document I cannot tell the difference between these two terms) are put at £530 million.

But on the credit side, the document estimates that the academies will produce economic benefits  – to those individuals educated within these institutions, I think – of  £1.072 billion.

The chain of reasoning seems to go as follows.

First, academies have been shown to improve GCSE results, the document says. The paper assumes that a school changing its status from mainstream to academy will generate a 1.5 percentage point improvement in the proportion of its pupils achieving five or more GCSEs, including English and maths.

I cannot quite see, from the paper itself, where this figure has come from. Earlier in the document, it states that 63 academies which had results for both 2008 and 2009 had been found to have improved their five-or-more GCSE including E and M rate by five percentage points, which was twice the national average.

Explaining why the 1.5 percentage point figure had been chosen, the paper says that the figure had been “down-rated by half” to allow for the fact that new academies would be different from those covered by the 2008 and 2009 figures. 1.5 per cent, of course, is less than a third of the five percentage point increase shown by the academies in 2008 and 2009, so this is confusing-or have they taken the 2.5 percentage points that academies improved above the national average, and then more-or-less halved that? Having read the document several times, I don’t know.

The paper then attempts to assign an economic effect to an individual achieving five or more good GCSEs (there is no mention of this having to include English and maths in this section), as opposed, presumably, to not achieving this benchmark. It puts a figure on this as an extra £100,000 in “lifetime productivity” (earnings, I guess), for men, and £85,000 for women.

This, says a note, is based on an “internal DfE analysis” of three existing studies, which seem to focus mainly on the returns for individuals of gaining vocational qualifications.

It appears that this figure is then multiplied by the number of extra pupils that might be expected to gain five good GCSEs, including English and maths, as a result of the academies policy (the document assumes there will be 200 new academies each year to 2014), to produce an overall economic benefit arising from academies of £282 million per year, or £1.072 billion over four years, after taking into account inflation.

It is difficult to know where to start with all of this, without denouncing it as pie-in-the-sky rubbish.

The previous government’s own, largely sympathetic, evaluation of the academies policy, by PricewaterhouseCoopers, found, in November 2008, that “there is insufficient evidence to make a definitive judgement about academies as a model of school improvement”. How the new government can therefore assume that changing schools’ status to academies, in itself, would bring about a 1.5 percentage point improvement above what would have been achieved without it is puzzling.

And trying to predict improvements in academic results in the “new” academies – which seem likely to be drawn from among the more successful schools with, probably, better results as a starting point – on the basis of what predecessors which tended to come from the lower end of league tables achieved, seems especially perilous.

To be fair to the document, it does say towards the end that the figures “may be an overestimate” of the likely results gain, since the new academies “will have less scope for improvement than existing Academies” (presumably because their results are already higher) and “will receive much less start-up funding”.

Fair enough, except that on page 2, it stresses that: “Future benefits are assessed as only half those that have been achieved in past academies. Benefits are therefore likely to be underestimated.” (my italics)

Really, reading this document, you wonder why they even bother doing these impact assessments. I wonder if it has been done in a terrible rush in the early days of the new government, to try to get some political momentum behind the new policy. Whatever, if this is the level of analysis that has gone into this policy, there could be trouble ahead.

You can read the impact assessment in all its glory at: http://www.education.gov.uk/~/media/Files/lacuna/academiesbillimpactassessment.ashx

The “free schools” policy discussed on the radio this morning, and being launched in detail today by Michael Gove, is very interesting.

It does pose legitimate questions about the influence of the state over what goes on in classrooms, and about the balance between national and local government power over what goes on in schools, professional freedom over decision-making, and parental decision-making. The most immediate questions centre on how the schools will be funded.

I want to blog more on this subject in the coming weeks.

However, I thought now I would just post on one particular aspect of the free schools policy which Mr Gove alluded to again on the Today programme this morning: its inspiration, at least in part, from the American charter schools policy, and particularly one set of charter schools, the Knowledge is Power Program (KIPP), set up by two teachers.

KIPP schools are seen as among the most successful charter schools in the US, and have been referred to enthusiastically by Mr Gove. He may be right; they certainly have a good record. But there are other debates around these schools. Having read Diane Ravitch’s fascinating book on the US education system (see my last blog), I thought I would simply quote at length her section on KIPP schools. I have not checked her facts myself; but it is another perspective that is worthy of consideration, I think, in this debate. Ravitch is a former assistant secretary of state for education  in the US, and now a research professor of education at New York University and a columnist for Education Week newspaper.

Ravitch writes (page 135): “The charter schools with the most impressive record of success are the KIPP (Knowledge Is Power Program) schools, which have been called culture-changing schools, because they aim to teach students not just academics but also self-discipline and good behavior. KIPP was launched in 1994 by two teachers, David Levin and Michael Feinberg, after they completed their two-year assignment in the Teach for America program in Houston. Feinberg opened a KIPP school in Houston, and Levin opened one in the South Bronx in New York City. Both schools achieved exceptional results. Generously funded by foundations, Levin and Feinberg opened dozens more KIPP schools across the nation, specifically to prepare poor minority students for college. Fifteen years after the organization was founded, there were eighty-two KIPP schools with approximately 20,000 students.

Almost every KIPP school is a charter school, and most are middle schools (grades five through eight). In contrast to regular public schools, KIPP schools have longer days (nine and a half hours), some Saturday classes, and three weeks of summer school; typically, a KIPP school provides 60 percent more time in school than a regular public school. Students, parents, and teachers sign a contract agreeing to fulfill specific responsibilities. The central organization does not define KIPP’s pedagogy and curriculum; it leaves these decisions to individual school leaders.

In the demands they make on students, teachers, and parents, the KIPP schools are reminiscent of the American public schools of the 1940s, or even the 1920s, before the onset of class-action lawsuits and union contracts. In those days, it was not unusual to encounter schools with strict disciplinary codes and long working hours (though not nine-and-a-half-hour days).

Despite its successes, KIPP has its detractors. Critics question the applicability of the KIPP model to public education in general. One persistent question is whether KIPP enrolls all kinds of students, as regular public schools must. Like other successful charter schools, KIPP admits students by lottery; by definition, only the most motivated families apply for a slot. Charters with lotteries tend to attract the best students in poor neighborhoods, leaving the public schools in the same neighborhood worse off because they have lost some of their top-performing students. They also tend to enrol fewer of the students with high needs—English-language learners and those needing special education.

 The students who remain in KIPP schools for four or more years tend to achieve large test score gains. Most KIPP schools consistently outperform traditional public schools in the same neighborhood. But KIPP schools often have a high attrition rate. Apparently many students and their parents are unable or unwilling to comply with KIPP’s stringent demands. A 2008 study of KIPP schools in San Francisco’s Bay Area found that 60 percent of the students who started in fifth grade were gone by the end of eighth grade. The students who quit tended to be lower-performing students. The exit of such a large proportion of low-performing students—for whatever reasons—makes it difficult to analyze the performance of KIPP students in higher grades. In addition, teacher turnover is high at KIPP schools, as well as other charter schools, no doubt because of the unusually long hours. Thus, while the KIPP schools obtain impressive results for the students who remain enrolled for four years, the high levels of student attrition and teacher turnover raise questions about the applicability of the KIPP model to the regular public schools.

KIPP has demonstrated that youngsters from some of the toughest neighborhoods in the nation can succeed in a safe and structured environment, if they have supportive parents and are willing to work hard, spend long days in school, and comply with the school’s expectations. Thus far, public schools have not copied their methods. Regular public schools must accept everyone who applies, including the students who leave KIPP schools. They can’t throw out the kids who do not work hard or the kids who have many absences or the kids who are disrespectful or the kids whose parents are absent or inattentive. They have to find ways to educate even those students who don’t want to be there. That’s the dilemma of public education.

The theory of the charter movement is that competition with the regular public schools will lead to improvements in both sectors, and that choice is a rising tide that lifts all boats. But in reality, the regular public schools are at a huge disadvantage in competition with charter schools. It is not only because charter schools may attract the most motivated students, may discharge laggards, and may enforce a tough disciplinary code, but also because the charters often get additional financial resources from their corporate sponsors, enabling them to offer smaller classes, after-school and enrichment activities, and laptop computers for every student. Many charter schools enforce discipline codes that would likely be challenged in court if they were adopted in regular public schools; and because charter schools are schools of choice, they find it easier to avoid, eliminate, or counsel out low-performing and disruptive students.

Yet, even with their advantages, charter schools—like all new schools—face daunting challenges. Reformers declare their intention to open new schools as though this would solve the problems of low performing schools. But new schools cannot be mass-produced or turned out with a cookie-cutter design. Opening a new school is difficult. It involves starting with or recruiting a strong leader and a capable faculty, obtaining facilities, developing a program, assembling a student body, creating an effective administrative structure, and building a culture. Getting a new school up and running may take as many as five years. Some will succeed, some will be no different from the schools they replaced, and others will fail.”

Ravitch was initially an enthusiast for a concept of charter schools which saw them as giving teachers the chance to take more professional control over what happened in the classroom. However, she now also has reservations about how this has turned out in practice in the American context. This is another aspect of the coalition’s free schools policy which was heavily discussed this morning, which I want to write about in the coming days.

I’m not used to starting an article with the kind of headline you see above. Education journalists are (or at least, should be, in my view…) accustomed to dealing in nuance, caveat and complexity, and not in the unequivocal enthusiasm which those six words suggest.

But I feel they are justified in the case of Diane Ravitch’s “The Death and Life of the Great American School System”. For me, her book has performed an invaluable task, showing up the connections between debates in the US about school accountability, choice, testing and corporate involvement in education, and what has been going on over here. With positions probably more polarised (and therefore rival stances more starkly visible) in the US, arguably this makes the debate over here easier to understand.

For readers in England in particular, and especially as we ponder the possibility of a Conservative-led government which seems enthusiastic for at least one element of US policy (charter schools), the book shows up just how heavily the US debate has influenced, and continues to shape, arguments on this side of the Atlantic. It suggests that much of the heat in the US is generated by ideology, espoused by business-orientated advocates of “reform” who are ranged against defenders of state education as it has traditionally been understood. The strength of Ravitch’s book has been that she tries to see beyond the politics of this row to ask what is really in education’s best interest, as she sees it, rather than what best fits a politicised agenda. In this, I think she is quite brave: big business and many in the US media have lined up to support the reform movement, as if to question it is to side with lower ambitions for schools. It can be quite difficult, then, as Ravitch acknowledges, to make the counter-argument. But she succeeds.

The book has certainly not been short of influence, either: last time I looked, it was in the top 200 of any book on the bestseller list of the US Amazon website, a staggering achievement for a work on education policy.

Part of the interest, for readers coming to it afresh, must come from Ravitch’s background. The full title of her book is “The Death and Life of the Great American School System: How Testing and Choice are Undermining Education”. Anyone reading that apparent rejection of two ideas which have traditionally been associated more with the political right than with the left – or, indeed, anyone perusing her regular blog on these issues on the website of Education Week newspaper – might be surprised to learn that Ravitch describes herself as a conservative, having served as Assistant Secretary of Education for the elder President Bush in the early 1990s.

On the first page, she describes how, in the autumn of 2007, she had an “intellectual crisis”. “Where once I had been hopeful, even enthusiastic, about the potential benefits of testing, accountability, choice, and markets, I now found myself experiencing profound doubts about these same ideas,” she writes. The next 241 pages see Ravitch set out the evidence that made her change her mind.

The argument is really quite straightforward. Ravitch posits that, in recent years, big business and ambitious politicians have put forward the idea that education should be transformed to be placed on a more business-like footing. Schools and teachers should be held to account through test scores, and they should compete for parents’ custom. Those which are successful should be rewarded, while schools which fail to improve their numbers should close and unsuccessful teachers should lose their tenure. Unions were seen as a block to reform and their influence should be curbed, it is argued, since schools should be run like businesses and be freer to sack teachers for poor performance and pay staff what they choose.

Giving parents choice over which school their child attended, then, was seen both as a good in itself and as a vehicle for pushing up school standards through competition.

This model was initially attractive, Ravitch writes. At first, she says,“I got caught up in the wave of enthusiasm for choice in education. I began to wonder why families should not be able to choose their children’s schools the way they choose their place of residence, their line of work, their shoes or their car….there was an undeniable appeal to the values associated with choice: freedom, personal empowerment, deregulation, the ability to chart one’s own course. The anti-choice side was saddled with defending regulation, bureaucracy and poor academic results.

“How much easier it was to promise (and hope for) the accomplishments, successes and rewards that had not yet been achieved and could not yet be demonstrated, but were surely out there the other side of the mountain.”

I’m not against choice in education as a principle, either. My own book does not set out to criticise it, as a concept, or challenge its existence in the English context. But the question is whether these policies actually work to improve our schools. Ravitch ends up hugely frustrated, and anxious, about the attempt to frame education in a corporate image.

“The problem with the marketplace is that it dissolves communities and replaces them with consumers,” she writes. “Going to school is not the same as going shopping.”

Along the way, she tells some powerful stories, identifying what look to this observer like gaping holes in the US accountability structure in particular. She deems it an “accounting strategy” – “measure, then punish or reward”, which has little to do with education: education by numbers indeed.

In terms of these “holes”, did you know, for example, that under No Child Left Behind, George Bush’s law from the year 2001 which required states to set children tests in reading and maths, all states had to publish timelines showing that 100 per cent of their pupils would achieve “proficiency” in these subjects by 2014?

This was, of course, an impossible goal, Ravitch writes, quoting two academics who wrote that it was akin to Congress declaring that “every last molecule of water or air pollution would vanish by 2014, or that all American cities would be crime-free by that date.” Yet it was not just an aspiration: the threat was that schools would be closed and teachers sacked if these targets were not met. I understand that Barack Obama has now scrapped the targets.

She also tells stories about how states managed to improve their results on the tests by seemingly lowering standards. While the federal government said that all states should have testing systems for reading and maths, and held states to account for results in those tests, it appears to have left unregulated the content of the tests (partly following a political row over history teaching dating to the 1990s, after which states decided it would be too controversial to try to reach a view on what children should actually be taught; what mattered was simply that they were tested), or how standards are kept constant. Ravitch cites a 2009 research study’s view that seemingly impressive gains in Chicago were down to “changes in the tests and testing procedures, not real student improvement”. President Obama praised the apparent gains in that city when he appointed the man who oversaw the Chicago schools system, Arne Duncan, as his secretary of education. Duncan is now busy pushing forward the choice/accountability agenda.

Of the difficulties with the US version of high-stakes testing, Ravitch writes: “Students may be able to pass the state test, yet unable to pass a test of precisely the same subject for which they did not practice. They master test-taking methods, but not the subject itself. In the new world of accountability, students’ acquisition of the skills and knowledge they need for further education and for the workplace is secondary. What matters most is for the school, the district and the state to be able to say that more students have reached ‘proficiency’. This sort of fraud ignores the students’ interests while promoting the interests of adults who take credit for non-existent improvements.” She also writes how holding schools and teachers to account in this way takes away pupils’ responsibility for doing well.

Closing schools, she writes, is an admission of failure and should be a matter of absolute last resort. “The goal of accountability should be to support and improve schools, not the heedless destruction of careers, reputations, lives, communities and institutions.”

Looking at reforms of the schools system in New York City under the mayor, Mike Bloomberg, the “media mogul”, and his chancellor, Joel Klein, Ravitch also documents problems with the report card system which became a vehicle for school accountability in that city and has been suggested by Labour for introduction here. Last year, she says, 84 per cent of elementary and middle schools received an A on their report cards, the top grade in an A-F grading system, compared to only 23 per cent two years previously. The rise, which lacked credibility, was partly driven by the state’s “secret decision to lower the points needed to advance on state tests”, the test results contributing to schools’ overall grades.

Ravitch also takes aim at the charter schools movement, including assessing the evidence on the success of charters, which is mixed. One chapter is entitled the “Billionaire Boys’ Club”. This shows how some of the richest families in America, including Bill and Melinda Gates, the Walton family, which is behind WalMart, and Eli Broad, a billionaire from home building and life insurance, have pumped funding into schools reform, including charter schools. But these donations, Ravitch writes, have been very much on their own terms, with the successful entrepreneurs tending to insist on market-driven policies. Although they promote accountability for schools, there seems to be little accountability surrounding their own decision-making policies, argues Ravitch. She has written on her blog that this amounts to a “privatisation” movement for state education.

The Broad Foundation seems particularly influential. It invested millions in charter schools; in training school board members and administrators; in advocacy, think tanks and other organisations such as the Center for Education Reform and the National Governors Association; and in public relations operations for New York City’s Department of Education and underwrote coverage of education reform issues by a public television network. During the 2008 presidential election campaign, the Gates and Broad foundations jointly invested $60 million to “make education reform a national campaign issue”.

Although some will no doubt welcome this largesse from people who could spend their money in less challenging fields than school reform, and engagement from the business community in the education field in theory could be great if managed well, for me it raises serious questions. The key thing is where the power resides in these relationships. If business people were willing to hand over their cash and allow others – perhaps those with professional expertise whose views might not accord with their own – to take decisions on how to spend it wisely, maybe they would have a better case to be viewed as an unequivocal positive force for a nation’s schools, and its democracy. But, if Ravitch’s account is correct, this is very much charity on the giver’s terms. To put it another way, why should a rich individual exert more influence over the direction of schools policy in furtherance of a particular viewpoint simply because he or she is wealthy?

I have gone on far too long, but must make a few more points. It could be argued that there is no straight “read across” from US education policy to the situation in English schools. In particular, our academies programme has shown how extracting sponsorship of schools from rich individuals is sometimes not as simple as it might appear to be in the American context. And our own New Schools Network, which has close links to the Conservatives, says it wants to take only the best of the charter schools movement and replicate it here.

Nevertheless, there are many echoes of the debates around test-based accountability in English schools. And I find myself agreeing with Ravitch’s overall conclusion, that, for all the talk about the structures of schooling, and the need to “incentivise” teachers to do their jobs well, in reality what really matters is simply what pupils are taught, and how. Ravitch makes a strong case that it is the curriculum, above all, which matters. She thinks every child should have the right to a broad and balanced learning experience right through high school, offering the best that liberal education can provide, including all of the following subjects: “history, geography, literature, the arts, the sciences, civics, foreign languages, health and physical education”, alongside the ubiquitous tested subjects of English (language) and maths. Schools should be rooted within their communities, and accountable to them. Teachers should also have the right to be represented by unions.

Although elements of this might sound a tad traditionalist for some, I have to say, it is not far from my own opinion (although, let’s be clear: I am not, and have never been, a teacher, so others may be better qualified to take a view on what should be taught). Particularly powerful is the humanity of what she has to say: it is better to treat teachers, parents, and children, as individuals worth respecting than to see them essentially as abstract elements within an ideological argument, the book implies. Perhaps my only surprise is that Ravitch, now a research professor at New York University and described on the back cover as “one of the most important public intellectuals of our time”, should have been so swept up by the reform movement in the first place. But no matter.

The title comes from Ravitch’s belief that the forces she takes on are helping to undermine that ideal of what good public schools should be about, with little evidence that the changes being pushed, including currently by Barack Obama, really will promote better education in the long run.

Perhaps the most urgent quote in the book comes at the end of the chapter on choice. It reads: “In barely 20 years, the idea of school choice rapidly advanced in the public arena and captivated elite opinion. Given the accumulating evidence of its uneven results, this was surprising. Even more surprising was how few voices were raised on behalf of the democratic vision of public education.”

This book gives much on which to ponder, I think.

Tuesday, May 4th

The more time I spend in education journalism, the more regularly I find myself surprised by the amount of misinformation which flies around in support of particular views.

The latest example can be found in yesterday’s Guardian, in an article by Peter Preston, its former editor, in which he criticised members of the National Association of Head Teachers for the  union’s forthcoming boycott of this year’s key stage 2 tests. I have already written a reply to a previous piece by Mr Preston, in which he pledged support for the national tests in a piece defending targets across the public sector. This was published in the Guardian’s Response column last October.

But I wanted to write something, also, on this latest piece, as it again features some misconceptions, as well as some basic factual errors.

First, he says that, because of the refusal of probably thousands of heads to take part in the tests in English and maths this year, there will be no “useful league tables available” to parents wanting to choose a school.

Well, it is a moot point whether the current league tables are “useful”, at least in performing the function they are supposed to serve: helping families guage the quality of a school. As anyone connected with education must surely know, the biggest influences on schools’ league table positions are the characteristics of the pupils, rather than anything the school does. Unadjusted data say little about teaching quality and their use to suggest simple comparisons can be made is highly misleading. Also, of course, it will still be possible to construct league tables this year, as teachers’ own judgements of pupils’ performance are being published.

Second, he says that “no reliable guide on national performance” will be available, with Sats not taken in a proportion of schools. That statement should not be true, this year or in the future. This year, even if only half of pupils were able to take the tests, it should not be beyond the wit of government statisticians to use this large sample to generate an overall picture of national standards.(They will be able to work out whether those not taking the tests are from, on average, high- or low-performing schools, and adjust the total results from those who do submit scores accordingly.) In future, of course, it would not be necessary to have every pupil take a test in order to generate national data; in fact, setting only a sample of pupils a test, with no school-by-school results generated, is precisely the approach being taken with regard to generating national data on science performance this year. It has also been a feature of other countries’ measurements of national standards for decades.

Third, he writes, “the only victims of this melee are 11-year-olds who’ve worked hard at reading, writing and maths for years – and now won’t know where they stand”. This, I would submit, is utter nonsense unless one holds that, without the current system of national testing and monitoring, teachers would be unable to decide for themselves what level each of their pupils has reached. While schools whose heads are taking part in the NAHT action will not generate test results to send to the government, it is far from true to say that this means that pupils will not know their levels. Schools which want to put their pupils through a test can set them a past paper. This result could be used to generate a level for a pupil. Or it could inform teacher assessment. Or the school could choose not to set a child a test, but still give them a level based on their teacher’s own judgement of their progress over four years. I would be surprised if any schools respond to the boycott by not telling parents how their children are doing. Mr Preston may feel this information is “tainted”, because it does not go through an external marking process overseen by the government. If that is so, he should just say that he doesn’t trust teachers to reach their own judgements on how pupils are doing, or to form a professional view on what kind of assessment experience is best for the child.

The fourth point is more basic. Mr Preston says that “standards in Scotland and Wales have slipped since testing there stopped”. Erm, Peter, there never has been an English-style testing system in Scotland, so I’m not sure where you’re getting this information. In Wales, it is also unclear where your information comes from. The Principality did indeed fall slightly in the last major testing survey in which it took part. (The Trends in International Mathematics and Science Study, or TIMSS). But that performance can say nothing about the effects of Wales’s decision to scrap English-style national tests in 2005 TIMSS tested two cohorts of children: 10-year-olds and 15-year-olds. The 10-year-olds had yet to go through year six, which is the year most affected by pre-test preparation under the old, English-style, system. The 15-year-olds went through the old, English-style tests so their performance can say nothing about Wales’s decision to adopt a different model.

Really, this debate has to be better informed than this.

The on-line debate in which I took part yesterday, for Schoolgate, The Times’s education blog, can be viewed here:

An interesting exchange of views, I think…