Unnecessarily
Complex Questions and/or Directions
Unnecessarily complexity may be due to the amount of information
included in the item, to poor grammar, or both.
Not
Specifying Basis for Sequencing/Ordering
The basis for ordering things should be specified in the assessment
item. For example, putting planets in a certain order can be done
in several ways: by distance from the sun, by size, or by alphabetical
order.
Not
Indicating the Nature of the Desired Description
Most things can be described in at least two ways: by their physical
features or by their functions. Test items should indicate both
what is to be described and what is to be included in the description.
Using
Absolutes
The use of absolutes (e.g., always, never, only, no) should be
avoided. Not only absolute statements are usually incorrect, but
also some student will know an exception to the keyed answer and
get confused or panic.
Using
Implausible Alternatives
When multiple-choice items contain alternatives that are obviously
incorrect, students have a greater chance of selecting the correct
choice.
Using Equal
Numbers of Items for Matching Items
When number of items to be paired in matching items is the same
it increases the chance of students to guess correctly on items
not previously learned.
Including
Grammatical Clues
Articles such as “a” and “an”, plural word
forms, and gender forms may provide clues to correctly answering
without having learned the content.
Having
to Reference to Alternatives
The students should be able to read the question and answer the
question without reference to the alternatives -- and their answer
should be among the alternatives when they get there... In other
words, the stem asks a question that has a definite answer.
Not
Emphasizing Key Words
Anytime you have a key word that would change the answer if the
student misses it, boldface it, CAPITALIZE it, underline it
or all of the PREVIOUS.
Not
Specifying What Best Answer Is
When best-answer items are used, the qualifier in the stem needs
to be emphasized and also you must specify in what way an answer
is the
"best".
Negatively
Worded Questions
Negatively worded stems should be avoided.
Lengthy
Information in Alternatives
Include as much information in the stem and as little in the options
as possible. For example, if the point of an item is to associate
a term with its definition, the preferred format would be to present
the definition in the stem and several terms as options rather
than to present the term in the stem and several definitions as
options.
Using “All
of The Above”
Don't use "all of the above." Recognition of one wrong
option eliminates "all of the above," and recognition
of two right options identifies it as the answer, even if the other
options are completely unknown to the student. All of the above
can also be unfair, because it means answer (A) is correct, so
student anticipates (A), looks down, sees (A), chooses it, then
moves on. Gets it wrong, even though knew (A) was the correct answer
-- gets answer right but question wrong.
Using
Trickery
The use of trickery must be avoided. There is no point in finding
out if you can trick the students -- you can.
Templates for Patient Vignettes
Templates can be very helpful when creating questions. The following
information about question templates is an excerpt from the National
Board of Medical Examiners (NBME)’s “Constructing Written
Test Questions for the Basic and Clinical Sciences” manual.
You can download the full version
of the manual, as a PDF file.
Item Templates
The overall structure of an item can be depicted by
an item template. You can typically generate many items using the
same template.
For example, the following template could be used to generate a
series of questions related to gross anatomy:
A (patient description)
is unable to (functional disability). Which of the following is most
likely to have been injured?
This is a question that could be written using this template:
A 65-year-old
man has difficulty rising from a seated position and straightening
his trunk, but he has no difficulty flexing his leg. Which of the
following muscles is most likely to have been injured?
- Gluteus minimus
- Hamstrings
- Iliopsoas
- Obturator internus
Patient Vignettes Components
Many basic science questions can be presented within the context
of a patient vignette. The patient vignettes may include some or
all of the following components:
- Age, gender (e.g. a 45-year-old man)
- Site of care (e.g. comes
to the emergency department)
- Presenting complaint (e.g. because
of a headache)
- Duration (e.g. that has continued for 2 days)
- Patient history
(with family history ?)
- Physical findings
- +/- Results of diagnostic studies
- +/- Initial treatment, subsequent
findings, etc.
Additional Templates
-
A (patient description) has a (type of injury and location).
Which of the following structures is most likely to be affected?
-
A
(patient description) has (history findings) and is taking (medications).
Which of the following medications is the most likely cause of
his (one history, PE or lab finding)
-
A (patient description) has
(abnormal findings). Which [additional] finding would suggest/suggests
a diagnosis of (disease 1) rather than (disease 2)?
-
A (patient description)
has (symptoms and signs). These observations suggest that the disease
is a result of the (absence or presence) of which of the following
(enzymes, mechanisms)?
-
A (patient description) follows a (specific
dietary regime). Which of the following conditions is most likely
to occur?
-
A (patient description) has (symptoms, signs, or specific
disease) and is being treated with (drug or drug class). The drug
acts by inhibiting which of the following (functions, processes)?
-
A
(patient description) has (abnormal findings). Which of the following
(positive laboratory results) would be expected?
-
(time period) after
a (event such as trip or meal with certain foods), a (patient or
group description) became ill with (symptoms and signs). Which
of the following (organisms, agents) is most likely to be found
on analysis of (food )?
-
Following (procedure), a (patient description)
develops (symptoms and signs). Laboratory findings show (findings).
Which of the following is the most likely cause?
-
A (patient description)
dies of (disease). Which of the following is the most likely finding
on autopsy?
-
A patient has (symptoms and signs). Which of the following
is the most likely explanation for the (findings)?
-
A (patient description)
has (symptoms and signs). Exposure to which of the (toxic agents)
is the most likely cause?
-
Which of the following is the most likely
mechanism of the therapeutic effect of this (drug class) in patients
with (disease)?
-
A patient has (abnormal findings), but (normal findings).
Which of the following is the most likely diagnosis?
Types of Questions
- Guess my drug
- Guess my toxic exposure
- Guess my diet
- Guess my mood
- Predict physical findings
- Predict lab findings
- Predict sequelae
- Identify underlying cause/diagnosis
- Identify cause of drug responses
- Identify drug to administer
Sample Lead-ins and Option Lists
Which of the following is abnormal?
sites of lesions;
list of nerves;
list of muscles;
list of enzymes;
list of hormones;
types of cells;
list of neurotransmitters;
list of toxins, molecules, vessels, or spinal segments.
Which of the following findings is most likely?
laboratory results;
list of additional physical signs;
autopsy results;
results of microscopic examination of fluids, muscle or joint tissue;
DNA analysis results;
serum levels.
Which of the following is the most likely cause?
underlying mechanisms of the disease;
medications that might cause side effects;
drugs or drug classes;
toxic agents;
hemodynamic mechanisms, viruses, or metabolic defects.
Which of the following should be administered?
drugs;
vitamins;
amino acids;
enzymes;
hormones.
Which of the following is defective/deficient/nonfunctioning?
enzymes;
feedback mechanisms;
endocrine structures;
dietary elements;
vitamins.
Copyright © 1996, 1998 National Board of Medical Examiners® (NBME®)
Copyright © 2001, 2002 National Board of Medical Examiners® (NBME®) |
