01.20.15On Annual Testing and Other Kinds of Spinach
First, a shout out to Martin West and Matthew Chingos for a really sensible and balanced assessment of the why and how of annual testing: why it matters so much and how it works, which in turn tells us why we need to do it every year.
The net is this: It’s like spinach.
Part of a balanced diet
We don’t always want to eat it. But it’s good for us so we do it anyway because in the long run it’s better. And just maybe over time we come to like it. It’s an acquired taste but done well it has a certain appeal. It grows on ya.
So too with testing. Yeah, it can be tense. Yes, we really need to implement it better and more sensibly. But done well it helps you get smarter and honor your commitment to the work by making sure you win. If it’s done right.
What’s doing it right?
Definitely:
- Making sure the data on teachers goes to principals to use and manage rather and that public data is about schools–which is what parents choose anyway. If schools are accountable for results over the long term (not one year) they will make smart long-term decisions about teachers. But it also allows principals to say: “You know what? This is a really good teacher; even this year’s low scores are not enough data to judge her.” Or, “Ok, we got it wrong. Let’s see how fast we can fix it.”
- Making better use of the data we get to learn and grow- that means using it to learn what’s working and what’s not and finding out who the people are who we need to study and learn from to solve teaching’s endemic problems. This means getting data to schools and teachers in better formats faster.
- Rigorous tests
- Thinking about the data positively more than just negatively. In other words more important than identifying what’s not working, quality assessment helps us identify honor and learn from what is working- usually that’s people, unacknowledged heroes who have developed solutions to teaching challenges but toil in obscurity. To me it’s very important that teachers get to shape the intellectual foundations of their work. THEY and not outsiders should decide what teachers should do. To make that happen we need to be able to measure and find the folks who do it best.
As the Atlantic Monthly recently pointed out, breakthroughs in innovation have almost always been preceded, historically, by breakthroughs in measurement. So let’s use it wisely and get the most out of what we invest in it that we can.
Actually it’s not like this at all.
Think about this: I spent the fall in the UK. They don’t have annual testing there. They have GCSEs. Students take them in all subjects at about age 16 and they determine preparedness for university. And they help schools see how effective they are. They’re not the only data on school quality–they get inspections by OFSTED–but they matter a lot. My friend Max Haimendorf was in the news because his school King Solomon Academy, blew the GCSE’s away. His kids demonstrated a level of success rarely seen among high poverty kids and that changed the conversation in England. Now think about what Max had to do to achieve that. He had to develop and implement an academic program and then steer it, guide it, shape it, STICK WITH IT, for FIVE YEARS without any data to show for sure that he was on the right track. He had to fly blind. What an act of skill and faith! But, dear lord, no wonder he’s an outlier. Who sustains an effort like that without wavering for five years?
I am sure teachers in the UK would be up in arms if they added annual testing but it would also help people without Max’s heroic helmsmanship also achieve great results. In fact, not coincidentally, Max would probably tell you that he succeeded by modeling his school on a school that DID have annual testing- North Star Academy in Newark, NJ. Part of what he did was to piggyback on their data set. I know teachers in the UK wouldn’t like to add annual testing to their lives. At least at first. But if it were done well I suspect they’d change their minds about it with time. To a large degree I think that’s happened here. Most of the responsible backlash is about how the results have been hastily applied to teachers, not about whether testing can make us smarter. Anyway, the net is that annual testing, while not usually our favorite thing, is something we need to keep.
Annual testing isn’t the bad guy. It is the high stakes attached to it that has had a negative impact.
agree. though i think “high stakes” is a bit of a loaded term. There have to be stakes. they just need to be thoughtfully deigned and executed. that’s very hard to do at scale.
Doug, I couldn’t agree more with your statement about stakes. “Thoughtfully designed and executed” takes time, collaboration, and a common language from which to approach a variety of content areas. There is also a certain level of humility that comes with this kind of design, and accepting that others, in the US or abroad, may have a better methodology that has impacted kids in more meaningful ways… I read an interesting report from The American Productivity and Quality Center that stated, “Benchmarking is the practice of being humble enough to admit that someone else has a better process and wise enough to learn how to match or even surpass them.” I continually come back to this quote when considering curriculum or assessment design. Who emerges as a leader in these areas, and how can we, at my school, humble ourselves to learn from these innovative leaders?
Just some food for thought!
Well said, I think so much of the bluster about testing comes from the misuse of the data, or fears about its misuse. But my question is always, if you’re not using testing data to inform your practice, what are you using? I think college graduation rates are a far superior measure, but they’re so far in the future that they’re practically useless for real planning.
well said. the point is not so much that we spend too much time testing as it is that we tend ot get so little out of it in terms of knowledge. and this is n;t necessarily an indictment of teachers and school leaders. state testing regimes often make it all but impossible to use data optimally. what i am supposed to do when i get my scores from last year in october? when i can’t see the questions and how my kids did on them? etc
Probably worth saying that every school I have ever worked in here in England (Scotland has a different curriculum and system of exams) has organised annual exams in almost every subject. They just haven’t been externally marked exams (i.e. the teachers set and mark them) so there is little consistency between schools.
Probably also worth saying that there is constant debate in this country about whether we have the right amount of exams, and even after what has been about a decade of reducing the number of written exams that are required (although not always reducing the amount which are sat in practice, resitting has been a problem), there is still a widespread opinion among many in education that there are too many exams. This probably shows that there are always some people who will complain about any system of exams, but yearly external exams would never go down well here.
Finally, is there a trade off here between frequency and quality in exams? How many of the yearly exams in the US are multiple-choice tests marked by machines? Exams here are rarely multiple choice, and we struggle as it is to get enough suitably qulaified and able people to mark the exams.
Great questions. first: yes, clearly a tradeoff between rigor and efficiency. if you want kids to write open ended responses –i am in favor of this–you are implying more time to administer, more time to score and more delays in acting on data. beyond just pointing out that it’s a tradeoff, i guess i’d also observe that many of the people who decry the lack of rigor in tests are the first to get upset when the tests get longer. to me the important ratio is how much you learn from the test given the amount of time you spend on it. so if you build a test that’s more rigorous and therefore longer your first question should be: how can i ensure that teachers can learn an immense amount from the results?
I think the response in the UK to annual testing would probably be about like it was here. in the first year or two lots and lots of people said “annual testing is clearly an evil conspiracy”; now i think that’s a fringe opinon and most people would say “annual testing has plusses and minuses. i get that we need to assess i just wish we could execute on it better.” and to the degree that people really learn from the data and get better at their jobs because of it they often come to like it more than they thought they would over time.
anyway thanks for your comment, Andrew. i get so much out of reading your stuff on twitter and the echo chamber. 🙂 Surprised you had time to write this, given coventry city’s plight. that hovering just above relegation business takes a lot of time to fret over.
I can see validity for.testing beginning at a certain level but my 1st graders are being tested 3 times a year using 3 different tests. That works out to 9 tests and many lost hours of instruction in a year and these aren’t aligned to each other so we have 3 different sets of results each time. Perhaps its time to think about what’s appropriate for children in primary grades and focus on teaching rather than testing.