A fundamental problem with the book scrutiny is that it’s impossible to judge whether learning has taken place. A book scrutiny can tell us a lot – but it can’t tell us what – or how much – has been learned. I appreciate that some schools don’t claim to use them to judge learning, but some do, and it’s problematic.

A book scrutiny can tell us whether the child wrote in pen or pencil that day. We can see whether the child underlined the date, the title, and other such ostensible markers of attitudes to learning. We can see whether that child managed to maintain his/her standards of presentation that day. We can see how much s/he wrote. We can see whether the pupil is producing responses of a similar standard to his/her peers – although that ‘standard’ is a minefield too – and we can’t compare what was learnt from one book to the next. The child who didn’t pick up a pen may well have learnt more than your diligent note-taker. I’m not condoning a lack of writing, if that’s the expectation in the lesson. I’m just saying that you can’t judge learning from how much writing took place.

A book scrutiny can often indicate to us what the teacher taught that day – although a lesson involving extensive reading or speaking will not be evident in a book. Apparent absences in exercise books could be easily misinterpreted as a lack of learning. Pick up an exercise book instead of the anthology, and a scrutineer could easily think that nothing’s happening in a series of lessons, because none of the poetry notes have been seen. Once again, sweeping assessments about how much has (and hasn’t) been learnt are made.

A book scrutiny can tell us whether the pupils doodle. We’ll see whether they write graffiti in their books. We’ll glean whether they have an emerging autograph that requires much practice at the back of the book. I’m a doodler – always have been. I quickly learnt, at secondary school, when and where I could and shouldn’t doodle. Telling the doodlers about the time and the place is important – judging them as being poor learners because they doodle is daft. If you notice doodling in an exercise book during a book scrutiny, then the one thing that you know is that you have identified a doodler. If you tell the child not to do it, and then see doodling subsequently, then you have a different issue – but it’s not one that can tell you anything about the learning.

My favourite piece of ‘evidence’ – I’ve saved it until the end – is the good old written comment at the start of a lesson: “I would judge my understanding of Elizabethan audiences as 1/10” and then – wow! – a comment at the end of the lesson: “I would judge my understanding of Elizabethan audiences as 8/10”. You smashed it! The whole class have improved by an average of 7 in just one lesson. Remarkable! Ask them about Elizabethan audiences in 24 hours, then in a week, then in a month, and then in a year. And then we’ll talk.

Why can’t we judge learning from a book scrutiny? Because learning is invisible (David Didau writes about this). ‘Learning’ is about long-term retention. It’s impossible to gauge this properly through an exercise book. One of Coe’s poor proxies for learning includes ‘students are busy: lots of work is done (especially written work)’ (This presentation works well for a quick tour). We shouldn’t be assessing what’s been learned just in a book scrutiny.

We need to promote the strategies that aid retention. We need to consider CLT. We need to undertake retrieval practice. We need to work on improving our students’ crystallised intelligence – on widening the schemata across the curriculum. Here are some links. Try this for a starter. Also, Adam Boxer has written about it here. Tom Needham’s series of blogs is phenomenal – here’s the first of several – this one is on the worked example effect. and Greg Ashman has written about it here. Rebecca Foster discusses desirable difficulties here. and Ruth writes about schema here – I love this one; the concept map really got me thinking about links in my own subject area. I’ve written about the importance of repetition and modelling, using I-We-You, here. All of these blogs link to further sources. It’s an enjoyable, daunting, rabbit warren. I would recommend Peps Macrae’s Memorable Teaching. This was definitely the first book that got me to fully reconsider much of my classroom practice.

The last paragraph, although a series of links, is what underpins the thrust of this. Looking for evidence of learning is detracting from the things that schools should be pushing forward on – strategies that will actually result in effective, sustained learning.  Some schools’ senior and middle leadership teams are looking for evidence of something that is almost impossible to evidence. Before a book scrutiny, if the answer to the ‘what are we looking for?’ is ‘learning’, then books are not going to yield the goods.

Blake’s Ancient of Days found somewhere on Google Images, labelled for reuse.