Should Modified Essay Questions Be the Primary Method to Assessing Performance?

By Kim-Ngan Tran, Pharm.D.,  PGY1 Pharmacy Practice Resident, Medstar Union Memorial Hospital

Multiple-choice questions (MCQs) are widely used as the primary means to assess student performance in most pharmacy and medical programs because they are a standardized method that can yield an objective score for a large number of test takers in a short period of time. Medical/pharmacy programs have changed their curricula to include more problem-based learning, which is an essential skill for a clinical practitioner. However, it is quite challenging for teachers to construct good MCQs that evaluate a student’s problem-solving skills.

Let’s discuss the advantages and disadvantages of MCQ and essay exams. MCQ exams are cheap and easy to score.  They can test a breadth of knowledge within a short examination time.  Moreover, it is the most commonly used as an evaluation method for licensure and board examinations.  Therefore, student must get practice with the MCQ format in order to confidently take licensure and board exams.  Another positive effect is that students’ final exam scores often improve because they have been previous tested on similar material during the semester.¹ Despite those advantages, MCQs have some weaknesses. There is a risk of students guessing answers and thus the results are not a true reflection of what the student knows. Writing MCQs is very time-consuming and fairly difficult because you would want to produce questions that can evaluate students’ ability to understand and apply concepts — not just recall facts based on memorization. Because of these challenges, many teachers do not return an exam or discuss exam answer keys with students in order to reuse the questions in the future. Thus students are not able to review missed or wrongly answered questions … and thus failing to learn important concepts.¹  In contrast to MCQ exam, an essay exam is relatively easier to write and less time consuming for teachers to create. This method provides teachers a better away to assess their students’ thought process, critical thinking, and writing skills. By providing an answer and not selecting choices, guessing is minimized. Nevertheless, grading essay exams is very time consuming and difficult to score objectively. Longer examination times are needed for students to finish an essay exam when compared to a MCQ exam if you wish to test an equal breadth of knowledge. Thus, essay exams are typically used for small classes and when teachers desire to measure students’ problem-solving skills.

Is there another approach?   Is there another question type that has the advantages of both the MCQ and traditional essay exam technique?   The modified essay question (MEQ) was developed by Hodgkin and Knox as an examination tool for the Royal College of General Practitioners.² MEQ exams include multiple problem-solving questions that are designed to test higher cognitive skills and can be consistently, objectively, and efficiently scored. By using MEQs, teachers can measure the knowledge retained and evaluate the students’ thinking and reasoning skills.^2,3 A typical MEQ provides a clinical scenario and one or more short answer questions based on Bloom’s taxonomy (memorizing, understanding, evaluating, analyzing, and creating).^4,5 For example, instead of using MCQ to test students about chronic obstructive pulmonary disease, a teacher could use MEQ to assess students’ ability to evaluate and provide an appropriate treatment plan for a patient scenario (Table 1). Students have to understand signs and symptoms related to the disease state and how to diagnose that condition. Then, they must apply their knowledge to develop an individualized treatment plan.

Table 1: Example MEQ

A 56 year old African American male presents to the emergency department with a complaint of worsening shortness of breath and cough for two weeks. He is a smoker with 40 pack year history. His past medical history includes hypertension, diabetes, hyperlipidemia, chronic obstructive pulmonary disease (COPD), and obstructive sleep apnea. List the differential diagnoses based on this patient’s symptoms? List the laboratory tests needed to confirm the diagnosis? What is the best initial treatment plan for this patient? After 3 days, the patient is stable and can be discharged. What is the best discharge treatment plan for this patient? State the current guideline that supports your answer.

To ensure the quality of the exam, questions must be relevant, valid, reliable, and objective. Teachers should define intended learning objectives to be assessed by each question. Patient cases should be clearly developed, include the relevant data needed to answer questions, and avoid extraneous information. MEQs should not be long.  Ask a colleague  to review the exam and provide suggestions to improve the questions. The common problem of essay exams is that grading may not be standardized. To address this problem, teachers should have key answers for all questions and assign a specific number of points to each. Then, if a student’s answer matches the keyed answers, he/she will get full credit. Partial credit can be given if a question contains more than one part; therefore, each answer must be assigned points in the key. The duration of the exam and the amount of time allotted to each question is important because if you do not give students enough time to answer questions, they will perform poorly and you can’t evaluate the students’ actual performance.

The MEQ model will work better if a teacher has help from a teaching assistant (or colleague) to grade all the responses. MEQs should only be used in a class that requires students to build higher-order thinking skills. Clearly, Pharmacotherapy and therapeutics course are intended to build critical thinking and decision-making skills.   These would be appropriate environments to use MEQ to evaluate students’ clinical knowledge.

References

1. Roediger H, Marsh E. The positive and negative consequences of multiple-choice testing. J Exp Psychol Learn Mem Cogn. 2005;31: 1155-9.
2. The Boards of Censors of the Royal College of General Practitioners. The modified essay question. J Roy Coll Gen Practit. 1971;21: 373-6.
3. Stratford P, Perice-Fenn H. Modified essay question. Physical Therapy. 1985;65: 1075-9.
4. Khan M, Aljarallah B. Evaluation of modified essay questions (MEQ) and multiple choice questions (MCQ) as a tool for Assessing the Cognitive Skills of Undergraduate Medical Students. International Journal of Health Sciences. 2011;5: 39-43.
5. Palmer E, Devitt P. Assessment of higher order cognitive skills in undergraduate education: Modified essay or multiple choice questions? Research paper. BMC Medical Education. 2007;7:49.