Practice: Evaluating Exam Questions

This section of the online workshop gives you an opportunity to practice evaluating some multiple choice questions. For each one, consider what attributes of the question might decrease its validity or reliability. You can click the "Show More" to see a revised (better) version of each question along with a rationale for the revision.

These examples are drawn from pharmacy, but the principles apply to any multiple choice questions, and in many cases, more broadly to various types of exam questions. The correct answer for each question is in bold font.

  1. All of the following are common adverse effects associated with drug overdoses of thyroid hormone replacement therapy except --
    1. Cardiac palpitation
    2. Arrhythmias
    3. Tachycardia
    4. Weight gain 
    5. Heat intolerance

Show more

Revised question:
  1. Which of the following adverse effects would be associated with a drug overdose of thyroid hormone replacement therapy?
    1. Cardiac palpitation 
    2. Dry skin
    3. Weight gain 
    4. Dehydration
    5. Unusual odor
Rationale: Questions should ask students to identify the correct answer, rather than an incorrect answer. Identifying what is wrong does not mean the student knows what is correct, so asking students to find an incorrect answer decreases reliability.
  1. A 55 year-old patient has been referred to you. She complains about a skin rash and a cough. In the course of history taking, she tells you that she takes high blood pressure medication but doesn’t remember the name. You suspect --
    1. Captopril 
    2. Nifedipine
    3. Prazosin
    4. Propanolol
    5. Clonidine

Show more

Revised Question:
  1. A 55 year-old patient has been referred to you. She complains about a skin rash and a cough. In the course of history taking, she tells you that she takes high blood pressure medication but doesn’t remember the name. What antihypertensive agent could be causing these drug toxicity symptoms?
    1. Captopril 
    2. Nifedipine
    3. Prazosin
    4. Propanolol
    5. Clonidine
Rationale: Base each item on a specific problem stated in the question. Students should not have to infer what problem is being presented. The question should test specific course-based knowledge or skills, not the ability of the student to make an inference. Reliability and validity decrease if the problems tests inference about the question rather than course material. 
  1. Your PharmD project involves plotting dose response curves for immunosuppressants. You get approval from your advisor to test three currently used products for your project. After an extensive literature review you decide on a methodology and choose rabbits as your test species. When applying for IRB approval you learn that you must provide adequate disposal of all waste including all animal feces and urine, any tissue samples, and all unused immunosuppressants.  You meet with your advisor to discuss your methodology and data collection procedures. She asks, “What would be the best parameter to measure to evaluate efficacy of rabbit antithymocyte globulin and basiliximab, respectively?”
    1. Total lymphocyte counts, B lymphocyte counts
    2. CD3+ cell counts, CD25 
    3. CD4+ cell counts, IL-2 concentration
    4. IL-2 concentration, CD4+ cell counts

Show more

Revised Question: 
  1. Your PharmD project involves plotting dose response curves for immunosuppressants. What would be the best parameter to measure to evaluate efficacy of rabbit antithymocyte globulin and basiliximab, respectively?
    1. Total lymphocyte counts, B lymphocyte counts
    2. CD3+ cell counts, CD25 
    3. CD4+ cell counts, IL-2 concentration
    4. IL-2 concentration, CD4+ cell counts
Rationale: Remove excess information in the question. Test learning objectives, not the student’s reading speed or ability to pinpoint the relevant information in a question (unless of course you're deliberately testing the student's ability to do such sorting and prioritizing -- in which case, the question should pertain directly to identifying such information).
 
  1. Gastric acid secretion is not stimulated by the presence of:
    1. Gastrin
    2. Histamine
    3. Acetylcholine
    4. Norepinephrine 

Show more

Revised Question:
  1. Gastric acid secretion is stimulated by the presence of:
    1. Cimetidine
    2. Histamine 
    3. Pepsin
    4. Norepinephrine 
Rationale: The question should be stated as positive instead of negative, unless testing an exception to a rule (e.g., a question about which drugs should not be administered in a given situation due to safety concerns). If using negatives, use clear wording, emphasize that the student is looking for the wrong answer, and avoid double negatives to prevent students from making mistakes due to the question format rather than lack of knowledge.
 
Alternative revised question:
  1. A 20 year-old woman goes to the emergency department stating that within the past hour she ingested “a handful of sleeping pills.” She is still awake. Which of the following drugs SHOULD NOT be given to induce vomiting?
    1. Metoclopramide 
    2. Ipecac
    3. Apomorphine
Rationale: Here, safety issues are involved. They key point is which drug should be avoided, so use of the negative is justified. Note the signaling of a negative response using all capital letters.
 
  1. Carbonic anhydrase inhibitors act on the enzyme responsible for the conversion of carbon dioxide to which of the following?
    1. Bicarbonate and calcium ions
    2. Bicarbonate and sodium ions
    3. Bicarbonate and hydrogen ions 
    4. All of the above.

Show more

Revised question:
  1. Carbonic anhydrase inhibitors act on the enzyme responsible for the conversion of carbon dioxide to produce bicarbonate and which of the following?
    1. calcium ions
    2. sodium ions
    3. hydrogen ions 
    4. lithium ions
Rationale: Alternative responses should not overlap. Each answer should be mutually exclusive to be able to discriminate what part of the answer the student understands. For these reasons, "all of the above" is generally a bad choice for multiple choice exams. 
 
  1. A 52 year-old woman with a history of eczema and heavy alcohol use begins taking ibuprofen to control hip and knee pain due to osteoarthritis. Over the course of six months, as the pain worsens, she increases her dosage to a high level (600 mg four times daily). What toxicity is most likely to occur?
    1. Abnormal heart rhythms
    2. Necrotizing fasciitis
    3. Gastric ulceration - heavy alcohol use increases the susceptibility of an individual to ibuprofen induced GI toxicity. 
    4. Confusion and ataxia
    5. Eosinophilia

Show more

Revised question: 
  1. A 52 year-old woman with a history of eczema and heavy alcohol use begins taking ibuprofen to control hip and knee pain due to osteoarthritis. Over the course of six months, as the pain worsens, she increases her dosage to a high level (600 mg four times daily). What toxicity is most likely to occur?
    1. Abnormal heart rhythms
    2. Necrotizing fasciitis
    3. Gastric ulceration 
    4. Confusion and ataxia
    5. Eosinophilia
Rationale: Keep answer options consistent in length and structure to avoid providing clues to the correct answer.
  1. The ability of chemotherapy to kill cancer cells depends on which of the following factors?
    1. Its ability to destroy specific tissues
    2. Its ability to stop cell division 
    3. Its ability to cause necrosis of tissues
    4. All of the above
    5. None of the above

Show more

Revised question:
  1. The ability of chemotherapy to kill cancer cells depends on which of the following factors? Its ability to
    1. destroy specific tissues
    2. stop cell division 
    3. cause necrosis of tissues
    4. initiate cell regeneration
Rationale: Avoid use of both “All of the Above” and “None of the Above.” The second decreases reliability because it allows the student to eliminate "all of the above" if one wrong answer is identified. Identifying none of the above does not indicate that the student knows the correct answer.
 

Next Section

Click to go to the next section: Building Exams