Writing Better Quizzes and Exams: Reasoning Questions and Asking “Why?”

Quizzes and exams often make one crucial pedagogical mistake: they do not ask enough reasoning questions. Reasoning questions, simply put, are questions that ask about the reason why an answer to another question is correct. For any given quiz or exam question, there is always an implicit chain of reasoning from the given information to the correct answer. It is the ability to correctly follow this chain of reasoning that should allow students to obtain the correct answer if they genuinely understand the material and can follow that implicit reasoning process.

Let’s look at a simple example of how reasoning questions can be used to make formative assessments, quizzes, and exams more effective. Consider the following example of a typical quiz question that you might find in any college-level critical thinking course:

Example 1:

Consider the following argument:

Because Colorado is west of Virginia, and because Nevada is west of Colorado, it follows that Nevada is west of Virginia.

This argument is deductively (sound / unsound).

In this simple example, the correct answer is that the argument is deductively sound. But there is an implicit chain of reasoning from the given information in this example to the correct answer. In order for an argument to be deductively sound, the argument must first be deductively valid (namely, the conclusion must follow by necessity from the premises), and the premises must actually be true.

The problem with the quiz question in Example 1 is that it does nothing to draw out the reasoning steps involved in obtaining the correct answer. It would be much more effective to have a series of questions that require the student to explicitly state the correct answer to a separate question about every step of the reasoning process, as shown here in Example 2:

Example 2:

Consider the following argument:

Because Colorado is west of Virginia, and because Nevada is west of Colorado, it follows that Nevada is west of Virginia.

This argument is deductively (valid / invalid) because the conclusion (follows / does not follow) by necessity from the premises.

The two premises of this argument (are / are not) both true.

Therefore, this argument is deductively (sound / unsound).

Example 2 has several advantages over Example 1:

  • In Example 2, the implicit reasoning steps to obtain the correct answer from Example 1 are made explicit, serving to reinforce the conceptual connections and the normative reasoning process that a student should go through in order to obtain the correct answer for the right reasons. In completing Example 2, students undergo a type of Socratic training process, not just learning the concepts, but also learning to think through the overall reasoning process in the right way and learning to ask themselves the right questions.
  • Because the reasoning steps involved in obtaining the overall correct answer have been separated out into separate questions in Example 2, an instructor can more easily locate the source of a student’s misunderstanding and provide the appropriate remediation. For example, did a student obtain the incorrect overall answer because he or she didn’t understand what it means for an argument to be valid, or because he or she didn’t know whether Nevada is west of Virginia?
  • Students are much less likely to guess their way through Example 1 (i.e., to game the system) than they are Example 2. In Example 1, because there is only one graded question with two possible answers, a student has a 50 percent chance of obtaining the correct answer purely by guessing without understanding a single thing about what it means for an argument to be deductively sound. In contrast, because Example 2 contains four separate questions strung together in a chain of reasoning, a student has only a 6.25 percent chance of answering all four questions correctly purely by guessing (0.5 × 0.5 × 0.5 × 0.5 = 0.0625 = 6.25%). This means that Example 2 is a much more reliable and accurate testing mechanism for gauging student comprehension than Example 1.

Almost any simple quiz or exam question, no matter the subject matter, can benefit from being split into at least a two-part question, with the second part asking students to provide the reason(s) why the overall answer to the first part is correct. Or, in the case of a more complex reasoning process, as shown in Example 2, the overall answer can be treated as the last step in a chain of questions, each of which holds students accountable for a separate step in the reasoning process to obtain that correct answer.

The reason that this approach to more sophisticated quiz and exam questions is so broadly extensible is that there is always an implicit reasoning process, or a conceptual gap, however large or small, between the given information in a quiz or exam question and the correct answer. So there is always room to find at least one additional question holding students accountable for the intermediary reasoning process linking the given information to the overall correct answer.

My rule of thumb for increasing the use of reasoning questions in objectively graded quizzes or exams is to avoid single-instance questions that are not explicitly linked to at least one other question, whether as a follow-up question asking about the reasoning behind the correct answer or as a series of questions to hold students accountable for every step in the reasoning process to obtain the correct answer. Quizzes and exams written in this way are more pedagogically helpful for students, as the quizzes themselves become a teaching tool reinforcing the correct or normative reasoning processes students should be mastering. They also do a better job of gauging student comprehension and localizing the source of student errors or conceptual misunderstandings. And they are mathematically more difficult for students to guess their way through without genuine mastery.

Zachary Fruhling is a senior instructional designer at HotChalk Inc. where he is responsible for the design and implementation of graduate-level online course materials. Zachary has over 15 years of experience in higher education, both in the classroom as a university-level philosophy instructor and in educational technology as a digital content author, instructional designer, and creator of interactive online course materials and media for a variety of disciplines.

Leave a Reply

Your email address will not be published. Required fields are marked *

Writing Better Quizzes and Exams: Reasoning Questions and Asking “Why?”

by Zachary Fruhling
0