- Overall Formatting and Face Validity
- Composing Question Stems
- Composing Response Options
- Final Point for Consideration
It’s often assumed that administering multiple-choices tests is solely an issue of convenience: testing large numbers of students simultaneously with minimal time spent grading or assessing students’ passive recognition of key concepts. There is no denying that these are particular advantages, but this doesn’t mean that multiple-choice tests can’t be developed to promote and assess deep student engagement with course content. Below you’ll find some useful tips about formatting and composing effective multiple-choice tests, including question items and response options.
Overall Formatting and Face Validity
- Use the one-best response format; avoid true/false, multiple-correct and complex K type formats that test logic and reading skills rather than content knowledge
- Present questions and options vertically instead of horizontally to make the break between responses explicit, thereby ensuring the readability of individual questions
- Ensure that all of the options for a particular question appear on the same page as their corresponding question; do not split items across two pages
- If referencing media (e.g. illustrations or charts), ensure that its location is explicit and obvious; whenever possible place that media on the same page as, and directly above, the question
- Avoid overly specific and overly general content; keep questions and options short and concise
- Keep vocabulary appropriate for the group being tested; avoid the use of acronyms
- Although three options may be adequate, four options can help maintain the validity of a question stem and overall test. Five options increases work effort (e.g. reading time) without providing a significant difference in the ability to discriminate between strong and weak performers
- Use relevant material to test higher level learning, such as inclusion of typical settings; application questions (versus simple recall) can increase validity of exam
- Proof and edit – and have someone else proof and edit – each question stem and response option for proper and consistent grammar, punctuation, capitalization and spelling
Composing Question Stems
- Avoid trick questions which test neither mastery of content nor achievement of learning objectives; they erode students’ confidence, making them second-guess themselves (and you)
- Focus on a single topic in each question so that if a student chooses an incorrect response, it is easy to identify which content they have not mastered
- Keep the content of each question independent from that of other items on the test, this way difficulty with one question does not mean a student is unable to complete other test questions
- Phrase the stem as a question; students should be able to come up with a reasonable potential answer prior to looking at the choices
- Frame the stem positively; avoid negatives such as NOT and EXCEPT; if negative words are used, ensure that they are CAPITALIZED and boldfaced
- Present stems in such a way as to question FACTS rather than personal opinions or preferences (e.g. avoid using the pronoun “you” in the stem of the question)
Composing Response Options
- Make all distractors plausible yet definitively incorrect; silly or implausible distractors increase students’ chances of guessing the correct answer, even if they have not studied
- Use familiar yet incorrect phrases or typical student errors as distractors to ensure that students cannot guess the answer based on the familiarity of only one of the choices
- Keep length of choices about equal to avoid guessing based on common assumptions that the longest answer is always the correct answer (i.e. because the professor is careful to make it precisely correct and defensible)
- “None of the above” should be used carefully as it increases question difficulty; if this option is not used, students know that the correct answer is included in the offered list and, thus, may be able to logic through the answer
- “All of the above” should always be avoided; it’s difficult to create a valid question in this format and it’s an easy one for test-wise people to figure out
- Avoid giving clues to correct responses by using either specific determiners (e.g. “always”, “never”, “completely”, “absolutely”) or choices identical to or resembling words in the stem
- Avoid providing clues to the right answer via, for example: grammatical inconsistencies; conspicuous correct choice; pairs or triplets of options; blatantly absurd options
Final Point for Consideration
Consider the overall difficulty of the test in light of knowledge that university exams are supposed to assess mastery of course materials as taught. So, some questions should be designed to test items that most people should know based on the course materials, while other items should allow for discernments to be made between highly competent and less competent students.
Was this page helpful?
1118
9e0b4f0fb0
3 people found this useful