Archive for the ‘Data Watch: FFT’ Category

 

Wednesday, March 9th, 2011

Ok, I’ve decided to do something slightly different, here, in the form of a blog largely not written by me, but based on two emails I’ve received in recent months on the vexed and often technical issue of data analysis systems and target-setting.

This may be overly technical for some non-teacher readers of this blog, but I thought I’d put it up here as I get occasional inquiries about the Fischer Family Trust system in particular, and am interested in the implications of how these systems work in the classroom.

What follows are the more-or-less verbatim contents of two emails (reproduced here anonymously but with the authors’ consent) I received re data analysis systems, one from a teacher who seems reasonably positive/pragmatic about the whole experience, and the second who, as you will see, has concerns.

So here is the first teacher, who is a senior leader.

“I have always liked my schools to use two data sources, past performance and CEM Centre (MiDYIS, YELLIS and ALIS), although my current school uses CATs. 

“Raw data informs me as the teacher, but I adjust the targets that I give to students (no student in my GCSE classes is told that they will achieve less than a C, because all can easily achieve that and most can surpass it).  Data, as I tell staff, only provides questions and never answers.  It informs good teaching, but doesn’t make a good teacher.

“I then also tell staff the most important analysis of exam performance is comparing how students did in your class in comparison to other subjects in school. Did they do better with you or elsewhere? Then if they are below the data targets you need to take the mirror test. Do you feel that you did everything to help that student do better (look yourself in the mirror). If you are happy with what you did, move on, but ask ‘can we make adjustments to next years interventions?’

“I do recognise that these sets of data are not perfect and they can only ever be an indicator.  For FFT the worry is because of the inflation or deflation of scores at KS2 because of brilliant or poor teaching. In CEM and CATs students can do worse than they are capable of because of all the factors that can suppress test performance.  However, overall they do produce part of a useful guide and highlight possible underperformance to all staff.

“I’m happy to discuss any of this.  I am no way a zealot, just want all my students to progress so constantly looking for things to improve what I do. I think teachers’ fear of data comes from poor leadership as to how to use it.”

Here is the second email, reproduced verbatim from the start:

“Dear Mr Mansell,

“Thank you very much for your work on testing, hyper-accountability and the many problems in education today. I found your book Education By Numbers to be very thought-provoking, my copy is full of highlights where I was almost shouting out in agreement with many of the points you made.

“I have been teaching maths in the same high school for thirty years, and I find the current obsession with getting the best results for the school very dispiriting.

“I have tried to talk to my head of department and Head Master about improving learning and understanding, but it is a waste of time. They want to meet the targets, so pressure staff and pupils, force pupils to attend extra classes after school or instead of attending morning tutor meetings, but do not consider real educational improvements; too risky?

“Also, the pupils who will never achieve a C are effectively written off by the RAP(raising achievement process)which only targets D to C or C to B, and some troublesome pupils who the FFT say should achieve seem to move on elsewhere so they do not drag the results down.

“The RAP system is being extended to years 7, 8 and 9, not to improve education, but to achieve the magical FFT 5th percentile. Regular testing, split levels i.e. 3a, 3b, 3c etc, when it is doubtful if any teacher can reliably say ‘Jonnie is working at level 3 in algebra’. Levels may be estimated plus or minus one, sublevels are a nonsense, also the use of numbers as labels for ‘levels’ is misleading as the levels are descriptive, categorical data not measurements on a scale.

“One thing that I want to say to you is that the message that is regularly given about 5 or more grade Cs at GCSE being ‘good’ is a disaster for some bright pupils. I have had a few say ‘as long as I get 5 Cs I am doing well’.

“For able pupils C is poor, to get the message across to year 10 and 11 pupils I have bluntly said that for them ‘C means crap’, what they should be getting are 7 or more A*, A, B grades.

“The obsession with grades and levels for the benefit of the institution, instead of a focus on helping pupils to achieve the best for themselves, is a cancer in the education system”.

- Warwick Mansell

2 Comments
posted on March 9th, 2011

 

Thursday, October 1st

You may have noticed that I had a piece in the Guardian’s education section over the summer about an online discussion among history teachers as to which board offered the “easiest” exams.

Teachers were debating with each other, over several years, on the schoolhistory.co.uk website, which version of the modern world history GCSE offered the easiest,  most predictable questions.

One teacher, who started the debate, talked at length about the benefits of moving from AQA to OCR. He said of OCR’s exams: “The questions are very straightforward and at least 40% easier!” He said that the coursework requirements were also less exacting for OCR, and that less ground had to be covered with the teaching. This was very much a good thing.

He added: “I hate the fact that we have to shop around and play the system and find the easiest exam board/paper. Wouldn’t it be better if everyone was playing on a level playing field?”

I just come back to it now because what I didn’t find space to include in the piece was that, near the end of the discussion, the verdict from the teacher who started it was that, after the 2008 exams, his statistics had improved sharply.

He said: ”My OCR pass rate was 64% and our FFT [Fischer Family Trust] Type ‘B’ residual was a positive 0.3!”

This was taken as complete vindication of his decision to switch from AQA to OCR.

I’ll try not to sound too pompous or moralising here, but  is this really what education now comes to? A teacher’s quality can be summed up, accurately, of course,  in a number between 0 and 1.

And, if  this system encourages teachers to search out nicely “predictable” exams and to teach remorselessly to the test, so be it.

Remember, the data at the end is all that matters.

Truly sad.

- Warwick Mansell

No Comments
posted on October 1st, 2009

 

Right, this blog is for a new page I’m setting up on the site which will be concerned with the use of Fischer Family Trust data in schools.

This is an area that I’ve been trying to get to grips with over the last year, amid tales from teachers and others which point to concerns about the workings of this data system, which makes estimates of pupils’ future exam performance based on their previous test scores and other information.

Many of the underlying issues with the use of FFT data in schools seem to have links with wider concerns about the implications of England’s test-driven education system. These include the dangers of using information for purposes for which it was never designed, the problems with over-interpreting data and the inherent conflict between using information on pupils’ exam performance to help that child improve, and also to hold their teacher to account.

I wrote a piece for the TES on this shortly before I left the paper, which would be a good starting point, I guess, for those interested in this subject. It’s available here.

Since then, I’ve had contact from a teacher and, separately, a parent, who both have made the interesting observation that FFT data can actually act to demotivate children.

The teacher said that she teaches in a school where many children are from poorer backgrounds and arrive with low predictions under the Fischer Family Trust’s “D” indicator – often F or G grades at GCSE.

The teacher adds:  “Firstly, I don’t have to do much to meet my targets for these groups of children and wonder if FFT data therefore helps to maintain the status quo – ie we are not expected to achieve much with the most socially deprived pupils we teach.  Secondly, my experience shows that pupils at this end of the spectrum often have far more ability than is predicted by FFT.  In my first year of using FFT data, a pupil predicted grade F achieved grade B and pupils have regularly exceeded targets in these groups – quite often pupils achieve two or three grades higher than predictions.  This tells me that poverty is not necessarily correlated with ability and we do pupils a disservice if we use data based on this assumption (FFT use of post code and FSM).”

The teacher goes on: “My school (and I suspect many others) uses FFT data not only with teachers but in setting pupil targets.  On target setting day, an individual pupil’s FFT data is published for them and parent’s/carers to see and this is expected to be their aspirational target.  At the top end of ability that may be appropriate but at the bottom end – is an E, F or G at GCSE meant to be aspirational – it is certainly demotivating for the pupil. I find myself telling pupils and parents to ignore it, that the data is irrelevant and not applicable to the child – as indeed it often is not.”

Recently, a parent got in contact about the way FFT data was being used in her son’s school. Her son is in year 10. She said:

“He achieved level 4’s at KS2 and level 6’s at KS3, and he has just had the result of the first part of his GCSE double science award in which he achieved a grade ‘A.’ [Yet] his target grades based on Fischer Family Trust analysis done by the school are for grade ‘D’s in every single subject. I have asked the school how this can be so given that he managed level 4’s and 6’s in key stage tests. I was always led to believe that a level 6 would equate to a grade B at GCSE. He is currently attaining grade B level grades in most subjects but he has not had his target grades increased to match his performance and hard work. I am incensed that his diligence and the good teaching he is having are not being recognised by the school as an institution.”

I would welcome any comments on this, particularly as it seems to me that Fischer Family Trust data is increasingly influential in terms of how teachers and even pupils are judged.

I will also shortly be introducing another new category, called Data watch: Ofsted, which will look at the use of statistics in inspections.

 

- Warwick Mansell

7 Comments
posted on March 31st, 2009