Ten Rules for Writing Effective Test Questions

As an instructional designer, you’ll frequently have the task of assessing whether your courses have met their instructional objectives. Typically, trainers consider several parameters to judge the outcome of a course – course completion, engagement, interaction with fellow learners, assignment quality and timeliness, etc. However, a learners’ performance on tests is often an important benchmark for determining course effectiveness. How trainers and course designers develop those test questions is, therefore, critical.

Testing a Broad Spectrum of Intellectual Accomplishments

The key to writing effective test questions is in developing them so they assess a broad spectrum of intellectual understanding of the course content. It is important to test all six levels of that understanding, as espoused by Bloom’s Taxonomy:

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis, and
• Evaluation

…with knowledge being the lowest level of understanding, and evaluation demonstrating the highest form of learning.  At a technical level, effective test questions have seven key characteristics, including cognitive complexity, high-quality content, appropriate language, meaningfulness, reliability, transfer and generalizability, and fairness.

While it’s tempting (and often simple) to develop tests containing a single type of question, an effective testing strategy involves the use of multiple test types. These include Essays, Multiple-choice, Matching, True-False, and Completion.

Test-writing Basics

You’ll accomplish a more comprehensive test of a learner’s understanding if you include test questions from two general categories of tests:

1. Objective questions: When writing these questions, make learners choose a correct response from possible alternative answers, or require them to provide a phrase or word to complete a sentence or statement.

Examples of objective questions include: True/False, Multiple-choice, Completion, or Matching questions.

1. Essay-type or Subjective questions: If you need learners to demonstrate “original” thought, then subjective questions are the best way to accomplish that. These questions allow learners to plan, organize and articulate their answers “unscripted” – i.e., without the aid of multiple choices or hint phrases or statements.

Examples of subjective questions include: Problem-solving and short and long-form essay-type answers.

Now that we’ve covered the basics, it’s time to talk specifics.

10 Rules You Should Never Ignore

Writing effective test questions is part art and part science. However, when building their test banks, most trainers and L&D professionals instinctively gravitate towards testing that is easy to measure, while leaving out what’s essential. If you are among those looking to improve your test-writing skills, then here are 10 rules to help you do just that:

• When writing exceptionally lengthy tests, make sure to divide them into different sections to maintain learner interest
• Where the question is (understandably!) lengthy and/or the answering requirements complex, providing some concise answering guidelines and a worked-out example can help
• Resist the temptation to lay out questions sequentially based on types of items – all T/F first, then a whole set of MC questions; or test types – all Objective, followed by subjective. Instead, mix the questions up throughout the test
• Don’t chain-link test questions, where answers to one question depend on answers to a prior question. You’ll place learners at a disadvantage if they get one wrong answer initially
• Questions to test for knowledge and recollection should ask to identify facts, figures, names, terms, or events:

1. Name the person who….?
2. What did the Founding Fathers mean by….?
3. Who discovered…., and when…?

• Questions to test for comprehension must ask learners to interpret, paraphrase, explain, recount or summarize key concepts:

1. In your own words, explain what is meant by……?
2. What does the Theory of Gravity help explain?
3. What 4-reasons does the text highlight for…?

• When writing multiple-choice questions, include at least four possible choices to prevent learners from guessing the correct item
• Sometimes, especially with time-limited online tests, in their haste to meet completion deadlines, learners confuse choices presented in lower case {a) b) c) d)}: Depending on fonts used, “a” and “d” seem similar. Instead, use capitals: A) B) C) D) to avoid confusion
• To avoid a similar “interpretation challenge”, pre-populate “T” and “F” beside each T/F question, and request test-takers to circle the appropriate answer. This will relieve test graders of the difficulty in interpreting hastily written (or illegible) responses
• When writing matching questions, make sure the items to be matched share common characteristics:

Not ideal		Preferable
___1. Ammonia ___2. Penicillin ___3. Sodium Chloride ___4. Apollo 11	A.      Fleming B.      Armstrong C.      NaCI D.     NH3	___1. Sulfuric Acid ___2. Water ___3. Sodium Chloride ___4. Ammonia	A.      NaCI B.      H2SO4 C.      NH3 D.     H2O

Subterfuge and Clues Don’t Help

It’s an often held (incorrect!) belief that deliberately confusing and tricking learners in tests is a good way to determine whether they have truly understood the material. Wrong! The best way to assess understanding and knowledge is by testing for application and transference.

Trainers must therefore avoid deliberately lengthy, convoluted questions, and unnecessarily complex instructions. Focus, instead, on testing the 6-levels of understanding (Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation) by sticking to the 10 rules of writing effective questions presented above.

On the other end of the spectrum is the view that embedding “clues” within the questions doesn’t impact the outcome of the test. Once again, wrong! Where the answer is predicated on an indefinite article, the use of a(n) helps address this challenge:

Not ideal	Preferable
Fill in the blanks: Benjamin Franklin is known for his famous axiom that: An ______  of ________ is worth a ______ of _______	Fill in the blanks: Benjamin Franklin is known for his famous axiom that: A(n) ______  of ________ is worth a(n) ______ of _______
“An ounce of prevention is worth a pound of cure”

Why is constructing such questions preferable? Because, in the real world, learners seldom encounter “hints” and “suggestions” that lead them to arrive at the preferable solution. Writing test questions devoid of hints/clues of the possible correct answer helps learners demonstrate understanding and application of the subject matter.

Want to save $100? I’m discounting my most popular course, the Instructional Design for ELearning program, over half off for summer! This comprehensive program is feature-rich and will give you the skillset to create amazing online courses. Save now, because the price will definitely go back up!