Research School Network: Designing Good Multiple Choice Questions In this new age of distance learning, how can teachers create effective multiple-choice questions to assess from afar?
—
Designing Good Multiple Choice Questions
In this new age of distance learning, how can teachers create effective multiple-choice questions to assess from afar?
Share on:
by Durrington Research School
on the
By Andy Tharby
Multiple choice questions have the potential to provide teachers and schools with high-quality, time-efficient information about their students’ learning. When designed well, they are a particularly powerful and effective assessment tool. Now that teachers are setting more online work than ever before, many are experimenting with designing and using multiple-choice quizzes to check the depth and quality of their students’ distant learning.
It is tricky to design a good multiple-choice question and there are some common traps to avoid. Luckily Professor Stuart Kime of Evidence Based Education used his session at Research Ed Loom Durrington, April 4th2020, to deliver a deliciously practical tour of the evidence surrounding the design of effective multiple-choice questions. This blog will provide some of the top tips from that session, supported by straightforward examples and non-examples (courtesy of this writer).
1. Always phrase the stem as a question rather than as a statement.
Good example:
What is the capital city of India?
a. New Delhi
b. Mumbai
c. Kolkata.
Non-example:
New Delhi is the capital city of India.
a. True
b. False
c. There are two capital cities in India.
2. Use vertical formatting and letters for answers/distractors.
Good example:
Who won the 1966 World Cup?
a. West Germany
b. Wales
c. England
Non-example:
Who won the 1966 World Cup?
1. West Germany 2. Wales 3. England
3. The distractors (wrong answers) should be similar to the key (the correct answer). Ideally, these should tap into common misconceptions and errors that are associated with the topic in hand. Where possible, they should provide some information into why the child got the answer wrong.
Good example:
Which sentence includes an apostrophe mistake?
a. The boy’s hair was blonde.
b. The wind blow’s through the forest.
c. My garden’s beautiful in the spring.
Non-example:
Which of these is a word to describe a feeling of overwhelming excitement?
a. Depression
b. Cabbages
c. Ecstasy
4. Avoid unnecessarily difficult vocabulary and grammar (unless, of course, this is what you are testing for). Instead, focus on the construct of interest using clear, simple language. Naturally, this will depend on the age and proficiency of the learners.
Good example:
What is a dialect?
a. A form of a language that people speak in a particular part of a country, containing some different words and grammar.
b. The form of a language that people in a particular social group speak.
c. The form of a language that a particular person speaks
Non-example
What is a dialect?
a. A regional variety of language distinguished by features of vocabulary, grammar, and pronunciation from other regional varieties and constituting together with them a single language.
b. A form of language (non-standard, restricted register) or a set of lexical items used by a socioeconomic class, a profession, an age group or other social group.
c. An individual’s distinctive and unique use of language, including speech. This unique usage encompasses vocabulary, grammar, and pronunciation.
5. Avoid opinion-based subjective questions.
Good example:
Which character could be described as an archetypal Edwardian capitalist?
a. Mr Birling
b. Eva Smith
c. Inspector Goole
Non-example:
Which character is the most sympathetic in the play?
a. Mr Birling
b. Eva Smith
c. Inspector Goole
6. Avoid negatively phrased stems. There is also no need for more than two distractors, especially as they can be so hard to write. The phrase ‘none of the above’ gives the teacher no useful information and should be avoided.
Good example:
Which day of the year is Valentine’s Day?
a. February 14th
b. February 15th
c. February 16th
Non-example:
Which is not an example of a religion?
a. Christianity
b. Islam
c. Buddhism
d. Sikhism
e. Capitalism
f. Jainism
g. Zoroastrianism
h. None of the above
Perhaps the best advice to putting together multiple-choice questions is that there should only be one clearly right answer for those who truly know it. For any others who do not know it, the question should provide three possible alternatives.
While multiple-choice questions offer many opportunities for assessment, especially through remote, digital learning, there are some drawbacks to keep in mind. First, they are very time-consuming to put together and hard to write well. Second, teachers should be very cautious when making inferences based on multiple choice tests that have been administered in non-standardised conditions – for example, students can easily look up answers at home. Third, if multiple choice questions are to be used effectively as a form of retrieval practice then they must cause children to think hard about subject content – sometimes direct recall without options is a better alternative for retrieval practice.
If you would like to know more about effective multiple-choice questions, then take a look at the resources on the Evidence Based Education website.
More from the Durrington Research School
Show all newsThe Evidence Base behind Attendance Interventions
The importance of attendance means that there is a growing demand for a review of the research into attendance interventions.
Metacognition and self-regulation in geography
Head of geography Sam Atkins explains how he has been helping students develop their metacognitive regulation .