Down the assessment rabbit hole

There a great scene in the Matrix where Morpheus offers Neo the chance to see for himself what the matrix is.

The continued references to Alice in Wonderland throughout the films are interesting and I like the way that Morpheus describes the rabbit hole analogy – it certainly fits with our relationship with assessment over recent years.

Morpheus offers Neo the choice to see the truth by taking the red pill and that’s the spirit of this blog post – a series of assessment provocations designed to do the same thing. To open our eyes to truth.

Teacher assessment

Teacher assessments (termly data entry of working towards / expected / greater depth) are not valid judgements of what children have learned. They are too broad and cannot capture meaningful information.

‘Greater depth’ teacher assessments can only mean that children have learned more of what was intended than those at ‘expected’. Simply having greater depth as an available grade lowers expectations of what all children are expected to learn.

The future of primary assessment is probably based around binary outcomes (no exceeding grade at EYFS / phonics screening / no KS1 SATs / multiplication tables check). A teacher assessment system of 3 or 4 grades is not necessary.

Judging writing attainment against criteria is too unreliable. Use comparative judgement for summative assessment.

Standardised testing

The main reason to use standardised tests are to report something even remotely meaningful to those in charge of governance.

Standardised test scores don’t tell teachers or leaders about the extent that children have learned the curriculum.

Using a standardised test suite that takes up teacher time in marking, photocopying, data entry or question level analysis is an inexcusable waste of time.

Standardised test scores going slightly up or down is more likely statistical noise than progress or whatever the opposite of progress is.

Question level analysis is pointless. If you’re going to do it, use class work and not a random set of test questions that don’t match what’s been taught.

Other testing

Don’t pretend that teachers can write tests that are rigorous enough to make valid judgements of learning.

Writing tests that are rigorous enough would take too much time and would be the wrong focus of attention.

Progress

Progress cannot be measured so do not spend time trying to do it, certainly not based on teacher assessments.

Pupil progress meetings based on standardised test scores skew our attention to the wrong things – they don’t assess our curriculum.

Pupil progress meetings based on teacher assessments are too noisy and invalid to be of any use.

Data collection

Schools collect too much data that is not used to improve teaching.

Often, teachers input data into tracking systems and then are told that same information back. Teachers already know – don’t waste their time.

Judging the quality of teaching

Nobody can reliably judge the quality of a lesson.

Evaluative lesson observations are only useful to spot practice that is clearly a concern.

We’re probably looking for the things we like rather than what’s effective because we cannot know in that moment what is effective.

Workbook scrutiny

Workbook scrutiny does not tell us what children have learned. It can only tell us the standard of work, effort and presentation that teachers accept.

Looking at children’s books without teachers or children is not worth the time.

---

Building knowledge

You might agree or disagree with these provocations but there is a general theme of invalid judgement whether we’re seeking to assess children’s learning, the quality of teaching or indeed how good our school is. Instead of judging or evaluating, we might just be better off seeking to build knowledge. But the complexity of school life is such that seeking to build knowledge of the teaching of reading, or writing, or maths, for example, is just too big. We need to build knowledge around specific problems, for example:

Children’s learning

How well do children in year 3 know their 7 times table?

Can every child in year 2 read at 90 words per minute?

Can every child in year 5 write a paragraph without run-on sentences and without incomplete fragments?

Quality of teaching

How clearly do teachers model the concept of a sentence in year 1?

Do teachers in year 4 provide enough purposeful practice for children to learn the 12 times table to automaticity?

How do teachers in year 5 give feedback to children that write run on sentences?

---

Holding to account

The assessment provocations, whether you disagree with them or not, should call into question that way that leaders hold others to account and the way that we are held to account.

This is where a shift is needed. If the assessments that we carry out are unreliable and the judgements invalid, perhaps holding teachers and leaders to account needs to be moved upstream a bit. Yes, we need to be held to account on the impact of our work but if we’re paying attention to the wrong things, that impact is less likely anyway. Maybe we need to be held to account on what we decide to pay attention to first.

We need to justify our priorities and should be able to answer the questions:

• What is the specific problem?
• How do you know it is a problem?

We need to justify our strategies and should be able to answer the questions:

• What strategies have you chosen to solve the problem?
• How do you know that those strategies are sensible choices?
• What potential strategies have you discarded and why?

We need to justify our plans for implementation and should be bale to answer the questions:

• How are you giving the strategies the best chance of success?
• How have you ensuring clarity of purpose for staff?
• What are you stopping staff from doing to make this happen?

---

Assessment, quality assurance and school improvement. Take the blue pill and you’ll wake up in your bed and believe whatever you want to believe. Take the red pill and see how deep the rabbit hole goes.

• Coherence, not consistency | Developing shared teaching approaches
• Not everyone needs an action plan
• Can a school improvement plan embrace complexity?
• 6 questions for school leaders
• A rising tide | What we might attend to most to improve our school