Doug Lemov's field notes

 

I recently joined Kate Jones, teacher of history in Abu Dhabi, and author of five books including the Retrieval Practice collection and Five Formative Assessment Strategies In Action, on her show on Teachers Talk Radio. In talking about Retrieval Practice and other topics we hashed out an idea for ‘hybrid” retrieval practice, using questions that harnessed some of the benefits of multiple choice questions with some of the benefits of open recall questions. We then decided to collaborate on this post to develop the idea further.

 

Typically there are two formats in which students can be asked to use retrieval practice in writing: free recall questions (often called “open response” in the US) and multiple choice questions (MCQ).

 

Free recall questions generally require more effort—more ‘desirable difficulty’ from students to answer. This is because recall—remembering something without a cue—is harder than recognition—which requires knowing, selecting and identifying the right answer when you see it. It involves more challenge and mental effort too but it is the increased effort that makes it effective strengthening the information in long term memory. A free recall task enables students to elaborate and extend their responses, taking retrieval practice to a deeper level that isn’t possible with the limitations of MCQs. These are all benefits of the free response approach.

 

But free recall questions can be more time consuming for teachers to grade and students to complete. MCQs by contrast can allow for faster retrieval practice. Designed well they can allow teachers to gather a lot of information quickly and easily, and by inserting potential misconceptions into answer choices, make sure students aren’t fooled by them. MCQs can provide a nice change of pace for students and tend to be preferred by students because of their simplicity in contrast to free recall although some students can find the limitations of MCQ frustrating, as they restrict students from demonstrating what they can recall. MCQs require careful design in terms of the questions and, especially, potential answers- the plausibility, nuance and specificity of them will dictate the level of challenge of a question; this is important to ensure MCQs are desirably difficult.

 

When using retrieval practice teachers often seek to ‘lower the stakes’—to get the retrieval practice benefits without making things feel like a formal assessment. To do that it can often be helpful to add an “I don’t know yet” choice among multiple choice responses. This makes it safe to say I don’t remember and it’s preferable for students to tell teachers they don’t know than for them to possibly guess right and have the teacher assume an understanding that isn’t there. This identifies in a gap in knowledge, for both the student and teacher to be aware of. However, adding an “I don’t know” answer choice to MCQs can also create an unintended consequence. It can be an easy option that in some cases might cause students not to struggle to recall things they aren’t sure they remember.

 

In a recent conversation the two of us came up with a possible alternative to address these challenges. Instead of adding a fourth of fifth choice to the MCQ that reads, “I don’t know yet …” include an option on MCQs for students who aren’t sure but with language that asks students to practice recalling what they DO know.

 

And so choosing the option isn’t a short cut for students who seek to avoid full effort.  We liked the idea of adding choice that reads,  “I know that …” with students then asked to fill in the blank describing as much as they could recall on the topic.

 

We liked it because students might not know the answer to the MCQ but it is highly likely they will have some prior associated background knowledge that is connected to the question. The option “I know that …” allows the student to share what they can recall from memory. Therefore they are not missing out on a retrieval opportunity but also from a confidence perspective, this allows the student to recognise and reflect that they can recall relevant information even if they can’t answer that specific question.

 

Unlike, “I don’t know” asking students to write what they can recall is actually the opposite of the ‘easy option’ approach because it is the only response that requires an extended answer and elaboration. Combining MCQs with “ I know that …” is also combining possibilities for cued recall and free recall, taking a hybrid approach to retrieval practice.

 

A common problem with MCQs is that unless we require further elaboration, such as an explanation as to why the selected option is correct or incorrect, we often don’t actually know if students can recall this information correctly? Is it an educated guess, involving some recall, as knowledge of the other distractors can lead to power of elimination, therefore there is an element of recall still. Or could a guess simply be completely random and potluck? The correct answer is in front of students for them to identify and select or take a chance, with a likely 1 in ¾ chance of it being correct.

 

As there are pros and cons to using MCQs and free recall, we can combine them via ‘hybrid retrieval tasks’ to gain the benefits of each. A MCQ about the causes of WW1 could ask; Who killed the Archduke Franz Ferdinand in 1914?

 

• Kaiser Wilhelm
• Gavrilo Princip
• Nedjelko Cabrinovic
• I know that …

 

There is an opportunity for further recall even for students who are unsure of the answer.

 

However, even if a student answers the question correctly ( option 2; Gavrilo Princip ) this still doesn’t show if the students understands or can recall why the assassination of Franz Ferdinand contributed to the outbreak of WW1 in 1914, which is essential in the study of causes of WW1. This caused us to think about another adaptation: an MCQ with two follow ups:

 

Who killed the Archduke Franz Ferdinand in 1914?

Part 1:

• Kaiser Wilhelm
• Gavrilo Princip
• Nedjelko Cabrinovic
• Not sure but I know that …

 

Part 2: This led to the outbreak of war because…

 

The factual recall of who killed Franz Ferdinand is important, however it simply isn’t enough. This approach enables the teacher to maximize the benefits of both MCQs and open recall. By combining a hybrid model we are creating desirable difficulties but also creating opportunities for retrieval success and retrieval challenge in the classroom.

 

Thanks to Kate for co-writing this. For more of her insights you can follow her on Twitter: @KateJones_teach