But What is the Right Answer?

by Kellen Riley, Doctor of Pharmacy Candidate, University of Maryland School of Pharmacy

As pharmacy students advance through their didactic curriculum, a fundamental switch in learning occurs. While the first few years of classes consist largely of rote memorization - of brand-generic pairings, therapeutic classes, and anatomy – sometime during the second or third year, students are expected to begin synthesizing the knowledge previously gained to arrive at the “most appropriate” clinical decision. Aptly so, as these students are preparing themselves for careers as pharmacists in a world where most answers are not black and white.

When students begin to migrate away from memorizing facts toward synthesizing clinical decisions, they realize that the multiple-choice exam becomes less and less appropriate as an assessment tool. This phenomenon becomes most apparent to me when my classmates and I are engaged in case-based discussions in a course that uses multiple choice exams:

Imagine that you are the student. You value the discussion taking place and know that it is strengthening your understanding of the material.  But in the back of your mind, unavoidably looming, is the exam. You know that in a few short weeks, you will sit for a test in which there is literally only one best answer and you will be forced to sift through all the information you can recall about this topic and choose just one. Even though you understand that there isn’t one right answer, you also want to be able to do well on that test. Finally, one student raises his or her hand to ask the question that is on every student’s mind: “OK, but what is the right answer?”

Now imagine that you are the professor leading the discussion. You have just spent the past five minutes having a lively and engaged discussion.  Students are furiously typing notes and asking intelligent questions. After all the discourse in which you have obviously stated that this particular scenario is not quite clear-cut, to your dismay, a student raises his or her hand and asks, “OK, but what is the right answer?”

This question can be posed many different ways (“So, if we were to get this question on the exam…” or “If we had to pick only one answer…”), but in each case the theme is the same: in courses where multiple-choice examinations represent the majority of a student’s grade, the students’ priority is ensuring maximal success on these exams, even if the methods of achieving that goal seem to oppose the “critical thinking” atmosphere created by the instructor.

I know from personal experience that my classmates and I do not perform as well on multiple choice exams in case-based courses.  Despite the desires of the professor, many students prioritize exam preparedness over learning objectives and lively discourse, if they believe their grades will suffer. Some researchers have examined the effect of multiple choice exams on the learning environment.  They found not only that students’ perception of their learning was strongly correlated with the type of examinations used but also found that multiple choice exams can hinder critical thinking skills.^1,2  Some argue that multiple choice exams are not inherently bad assessment techniques, but that the most important consideration when evaluating students’ learning is that there is an alignment between the way the material is presented and the way it is assessed.

Thomas Reeves, a professor in the Department of Educational Psychology and Instructional Technology at The University of Georgia, examined the alignment between eight critical factors (goals, content, instructional design, learner tasks, instructor roles, student roles, technological affordances, and assessment), and their effects the learning environment.³ According to his research, the factor most often out of alignment was assessment, and he believes the blame falls on the instructor:

“Simply put, instructors may have lofty goals, share high-quality content and even utilise advanced instructional designs, but most assessment strategies tend to focus on what is easy to measure rather than what is important.”³

Reeves’ characterization of multiple choice exams might be a little unfair because multiple choice exams may in fact be the only feasible method for instructors to provide timely feedback to a large group of students. While every student wants the assessment to be “completely fair,” the second most predictable desire of students regarding exams is that they want to receive feedback “like, yesterday.”

While good grades are nice, students place importance on examinations that are an accurate reflection of their knowledge and skill.  In courses that require critical thinking and synthesis, multiple-choice examinations are probably not the best option. A meta-analysis performed by Kenneth Feldman reveals that when examining the relative importance of various instructional dimensions, both students and teachers rate the importance of “quality of examinations” equally.⁴ These data suggest that instructors do realize the importance of assessment, and that the logical next step is figuring out what methods an instructor can use to achieve better alignment between the information and skills to be learned and the assessment.

Although much depends on the material being presented, possible options within the health professional education include practical examinations, small-group interactions, debates, peer assessments, or one-on-one standardized patient interactions like the Objective Structured Clinical Examination (OSCE). One interesting avenue of assessment includes the multiple choice item development assignment (MCIDA), which requires students to “develop multiple choice items, write justifications for both correct and incorrect answer options and determine the highest cognitive level that the item is testing.”⁵ It is an interesting twist on the traditional multiple choice exam in that the students write the test instead of sitting for one. Although the author acknowledges that this assignment can be time-consuming to grade, using the MCIDA enhanced learning outcomes for students.

When it comes to higher learning, some classrooms feature a battle between a professor’s desire to lead a discussion in which students focus on the critical thinking process and students’ desire to perform well on an upcoming examination by focusing on “the bottom line.” If instructors can design their course in a manner that promotes alignment between the critical thinking activities of the classroom and the assessments used to test their students’ knowledge of that material, they help ensure student success – not only on exams, but in life.

References:

1.  Stehle, S, Spinath, B, Kadmon, M. Measuring Teaching Effectiveness: CorrespondenceBetween Students’ Evaluations of Teaching and Different Measures of StudentLearning. Res High Educ 2012;53:888–904.

2.  Stanger-Hall, KF. Multiple-Choice Exams: An Obstacle for Higher-Level Thinkingin Introductory Science Classes. CBE Life Sci Educ. 2012;11:294-306.

3.  Reeves, TC. How do you know theyare learning?: the importance of alignment in higher education. Int. J. Learning Technology 2006;2:294-309.

4.  Feldman, KA. Effective College Teaching from the Students’ and Faculty’s View: Matchedor Mismatched Priorities? Res High Educ 1988;28:291-344.

5.  Fellenz, MR. Using assessment to support higher level learning: the multiple choice item development assignment. Assessment & Evaluation in Higher Education. 2004;29:703-719.