Building Exams

This page uses our four-part framework to identify some exam design goals that can guide your choices about the types of questions you include and the ways you organize them. After each goal, we have included some relevant strategies. You are invited to use the Comments section to discuss additional goals or strategies.

Click on the links to explore resources that provide more detailed guidance on exam design. These will open in new windows so you don't lose your place on this page.


Goal I: Create a valid exam, one that provides accurate information about students' learning.

Unfortunately, it's relatively easy to build an inadvertently invalid exam -- one that tests students' reading speed, skill at deciphering complex directions, or ability to maintain focus in stressful situations as much or more than your course concepts or skills. Some useful strategies for avoiding this include:

1. Minimize potential confusion by limiting and focusing the cognitive demands that are not related to answering the questions.

  • You want students to use most of their time and brain power producing good anwers, not puzzling through complex directions or strategizing about how to maximize points. Unless there is a good reason related to your learning goals to make the navigation of the exam itself challenging, keep the non-content-related tasks as simple as possible. Or consider distributing the instructions ahead of time so students can take whatever time they need beforehand to understand what the exam will ask them to do. 
  • As research on multitasking emphasizes, it takes significant cognitive energy and time to transition from one kind of thinking work to  another. You can limit the amount of task-switching required of students by grouping similar items or tasks (e.g., multiple choice questions or short answers) together. Note that it's actually useful for learning and assessment purposes to mix up the content--course concepts, key terms, topics, etc.; you just don't want to burden students with having to repeatedly figure out what a question is asking them to do. 
  • If you give similar exams periodically, alert students to any change in format or tasks so they don't make errors based simply on a shift in your instructions.

2. Design the exam to help minimize potential stress or anxiety, which can detract from student performance.  

  • Begin the exam with topics that students are likely to feel some mastery over. From a confidence-building perspective, the initial questions should be easier, lower on Bloom's taxonomy.
  • Build in opportunities for students to feel some agency over the exam by making informed choices about their approach. E.g., clearly label the points awarded for each answer as a way of signaling how to prioritize or spend time. Keep sections independent of one another so students can choose to start with the material they feel greatest mastery over--or want to get out of the way early.
  • Again, consider distributing exam instructions ahead of time, or at least provide clear information beforehand about what kinds of questions the exam will include and what concepts or skills they are designed to assess. 

3. Beware of building in 'dependencies'--i.e., items that require success on earlier items. You don't want to design an exam where students fail on Part B because they made errors on Part A. (For instance, a question that one can only answer accurately if they know a key term which the exam also requires students to define; or a graphing task that requires utilization of accurate calculations from an earlier exam section). Such dependencies make it difficult to distinguish whether students lacked Part B's content/skills, or struggled because they did not have Part A's content/skills. These also make exams challenging to grade!
 


Goal II: Create a reliable exam that provides consistent information about student learning and allows you to distinguish between levels of achievement. 

1. For consistency's sake, beware of building in options that mean different students will be answering questions that pose different degrees of challenge. See p. 7 of the CRLT Occasional Paper on exam design for more on the limits of giving choices on essay questions in particular.

2. Include questions/tasks that pose varying levels of challenge: some that all students who have learned basic course content can gain full points on, some that require more thorough skill or mastery, some that allow students to demonstrate an exceptional degree of mastery of course concepts or skills. This document outlines one model for building an exam with carefully proportioned numbers of such 'C,' 'B,' and 'A'-level questions.
 


Goal III: Build a recognizable exam that aligns with the instruction you have offered.

1. Perhaps most obviously, you can work on alignment by reviewing the assigned materials and your lesson plans to ensure you are covering only concepts and skills you have given students opportunities to learn.

2. Consider engaging students in the process of designing an exam. For instance, you can ask them to brainstorm key concepts in class, and then you can choose from those to create an identification or definition section. Or in groups, have them identify real-world problems that require utilization of a particular formula or model, and then use the best examples in an application question on the exam.

3. Once you have drafted an exam, review it to tally the points allotted to various skills, concepts, or content areas. Have you distributed points in a way that accords with your sense of what's most important for students to learn in your course? Have you given equally important concepts similar weight? 


Goal IV: Create a realistic exam that allows students to demonstrate their learning given the constraints you've created.

1. ​The first time you use a given exam or format, try taking it yourself, writing out all of the answers completely (or ask a generous friend or colleague to do the same). Considering the time advantage your expertise gives you, is it reasonable to expect your students can complete all of the tasks in the time allowed? 

2. Provide students guidance about the proportion of their time and energy that should be spent on various sections. Point values communicate much about how exam-takers should allot their time. Space on a page can do so as well. 

3. Consider carefully whether time limits or resource restrictions (e.g., no dictionaries, lists of formulas, or calculators) are useful for assessing student learning, or necessary given other constraints on your time and resources--and offer a self-paced or open-book exam if not. An unrealistic exam can easily become an invalid exam. Consider: "Speededness refers to the situation where the time limits on a standardized test do not allow substantial numbers of examinees to fully consider all test items.  When tests are not intended to measure speed of responding, speededness introduces a severe threat to the validity of interpretations based on test scores" (Lu and Sireci, 2007).

It bears repeating that, depending on the assumptions you make about matters such as reading and calculation pace, some of your students may require accommodations for documented disabilities. To create an inclusive learning environment for all students, it's good practice to imagine a broad range of student abilities when designing your exam as well as to normalize (rather than stigmatize) the need for accommodations for disabilities. A welcoming syllabus statement about accommodations, spoken invitation on the first day of class to contact you about possible accommodations needs, and reminder around exam prep time can communicate your commitment to supporting the learning of all students, regardless of their disability status. For more, see this SSD webpage for faculty.   

It is also important to note that academic integrity concerns can shape your choices about the resources you permit students to use during exams. For guidance on designing, proctoring, and grading exams with such matters in mind, see this CRLT Occasional Paper on "Promoting Academic Integrity in the Classroom."   

 


Next Section

Click to go to the next section on Grading Exams.