Rethinking Assessment Strategies

by Ben Carroll, PharmD, PGY-1 Pharmacy Practice Resident, North Mississippi Medical Center

Since making the transition from a student pharmacist to pharmacy resident, I have contemplated my four years in pharmacy school and whether the assessment strategies the instructors used were ideal. I graduated with almost two-hundred classmates. With such a large class size, my pharmacy school’s ability to provide individually tailored and personalized learning was limited. I feel that not only the school I attended but likely most others across the country, commonly fall short when it comes to using assessment strategies that accurately reflect the extent of a student’s learning.

Recently, I read the book Grading Smarter, Not Harder: Assessment Strategies That Motivate Kids and Help Them Learn by Myron Dueck.¹ In the book, Dueck contends that many of the assessment strategies teachers implement are detrimental to student motivation and achievement. He discusses ways educators can tailor assessment practices that really determine what matters most: student understanding and application of content.

The book challenges the traditional use of the grading policies related to late assignments and awarding zeroes for assignments not submitted. Dueck argues that the use of a grading scale from one to one-hundred does not make sense mathematically. Ten points separate an A, B, C, and D. However, if a student fails to turn in an assignment and is given a zero then 60 points separate that student’s grade between a D and an F. This scale can make missing a single assignment the “academic death penalty” and is mathematically inaccurate. It might be a better strategy to grade all assignments on the GPA scale of zero through 4 in which A=4, B=3, C=2, D=1, and F=0.

Dueck argues that it is unfair to assign a grade of zero to an assignment that was never submitted because this does not accurately reflect how much the student learned. In the book, he proposes giving uncompleted assignments an incomplete and implementing an intervention rather than handing out zeroes. There is a stepwise approach to this tactic. In the first step, the educator sets a timespan when each assignment should be completed rather than a due date. By communicating to students, that they have a certain window of time to turn in their assignment, they are given more flexibility and choice. The actual due date would be the final day in the timespan, but by setting a window of time for work to be completed, students may be less likely to procrastinate. This technique also helps prevent a flood of assignments from being submitted on a single day – making it more difficult for the teacher to grade them all in a timely manner. In the second step to this approach, Dueck suggests using a late or incomplete assignment form. Using this form, the student has the opportunity to explain why his or her assignment was not turned in. The form could include a section in which the student is asked to select potential interventions that might help him or her to complete the assignment. Dueck described examples of how this approach improved student success in his classrooms. One student in particular completed a form explaining that she failed to complete an assignment because of recent bouts of anxiety and panic attacks. This student was introduced to counseling resources at the school that she was not aware were available. Subsequently, her grades steadily improved.

This strategy of using incompletes and interventions rather than zeroes argues against a one-size-fits-all approach to assessing students. Conversely, when using grading and assessment strategies that are individualized, some students may feel they aren’t being treated fairly and others might take advantage of these lenient grading policies. None-the-less, I believe the benefits outweigh the risks. A fellow classmate of mine in pharmacy school was the mother of a child with a chronic illness that required frequent hospital admissions. I know that our school tried to accommodate her in certain ways, but there were times in which more leniency should have been provided in terms of extending deadlines and excusing absences. I don’t believe anyone in my class would have taken issue with a more flexible approach given her situation.

In the book, Dueck also argues that teachers should be more focused on finding ways to promote learning through creativity. In my opinion, pharmacy schools do not emphasize the importance of creativity enough. We take case-based multiple-choice exams with clear-cut, one correct answers. However, when we practice pharmacy in the real-world, we are frequently called to think creativity to solve complex patient issues. I spent countless hours in pharmacy school using notecards, highlighters, and acronyms attempting to memorize every detail about a medication in order to be prepared for tests. While I believe that a certain amount of this type of learning is necessary, I do think we should spend more time practicing “how to think” rather than memorizing. This can be done by setting up more project-based activities which require students to integrate knowledge across a range of disciplines. Perhaps students should not be assessed on their performance during these activities. Removing assessments for these activities may alleviate some stress and students may feel more freedom to think outside the box. They may also be more inclined to ask questions knowing every word isn’t being critiqued. Creativity promotes curiosity and requires students to think on a deeper level about concepts and ideas.

When thinking back on some of my best teachers, I realize that they all shared one common attribute: the ability to motivate. In the book, Dueck states, the best teachers are “often better coaches than teachers” and highlights the value of applying coaching skills learned on the playing field in the classroom. I believe that, in general, coaches are more likely to develop personal relationships with their players than teachers are with their students. Coaches often share personal stories about how they overcame adversity to improve to motivate their players. I feel many students would benefit from their teacher adopting this approach. Coaches also promote the importance of team unity whereas in the classroom it sometimes feels like every-man-for-himself. Teachers can promote more unity in the classroom by not grading on a curve. Grading on a curve encourages students to constantly think about how their grade compares to their classmates. Studies have shown that students in competitive class environments do not learn or retain information as well as students in more cooperative class environments.² Also, the use of a curve is based on the idea that the aptitudes of a single class represent a sample of the general population which is rarely the case.

I think we should rethink the assessment strategies commonly used in pharmacy and other health professional schools.  By adopting coaching techniques and giving students more flexible opportunities to demonstrate what they have learned, they will be more motivated to achieve success, and would, in turn, begin their careers as better clinicians.

References

1. Dueck M. Grading Smarter, Not Harder. Assessment Strategies That Motivate Kids and Help Them Learn.  ASCD; Alexandria, VA., 2014,
2. Humphreys B, Johnson RT, Johnson DW. Effects of cooperative, competitive, and individualistic learning on students’ achievement in science class. Journal of Research in Science Teaching. 1982;19:351–356.

Using Mnemonics and Spatial Visualization to Retain Critical Knowledge

by Robbie Ross, PharmD, PGY1 Pharmacy Practice Resident, G.V. (Sonny) Montgomery VA Medical Center

A student ruffles through his notes, textbook, and lectures for several hours trying to cram every last bit of information before the upcoming pharmacology exam. He arrives on test day, reads the first question, and his heart sinks; he knows he read about it, but just can’t remember the answer. Memory is an important part of teaching/learning and involves more than just recollection. Memory involves the learning, understanding, and the ability to link information together whether it was taught or experienced/researched on one’s own. When a teacher evaluates a student’s memory through tests, they are assessing more than just recall, but rather a deeper understanding of the information.

Students today live in an age of technology where information is always available at their fingertips and many students come to believe that recalling information really isn’t as important as it used to be. For example, how many of us can recall a friend’s phone number any more? Moreover, we are more likely to remember things we find interesting, which can pose a challenge in some classes. Students are more likely to achieve higher marks in courses related to their major or in electives, and more likely to struggle in general education classes.¹

The responsibility for remembering and retaining information is mutual — both the student and the teacher need to do their part. Educators need to pay attention to the “learning” aspect of “teaching/learning” and help students develop strategies to retain and understand the information. When a student is presented with new information, it is first held in short-term (aka working) memory while it is processed. Short-term memory is limited in capacity and duration — only five to nine items can be held in short-term memory for about ten to twenty seconds.²

This is where memory aids can come in handy. Several strategies have proven effective to facilitate memorization and long-term retention including mnemonics and spatial visualization.

• Mnemonics are strategies for associating words or phrases with a list of facts or principles.
• Spatial visualization involves assigning pieces of information with everyday actions and objects. Then when it is time to recall the information, imagining those actions and objects associated with the information.

Another technique this is often used to memorize information is repeating.  Repeating is simply that, repeating the information to keep it in short-term memory. Unfortunately, repeating doesn’t move the information into long-term memory. Moreover, short-term memory has a limited capacity, so it’s difficult to retain large amounts of information when we rely on repeating. Cramming for an exam is a common example where students often use the repeating method to pass a test but fail to recall the information days or week later.²

Many of us first learned about mnemonics in childhood. For example, “Every Good Boy Does Fine” was a useful way to remember the major musical notes and “Please Excuse My Dear Aunt Sally” was a mnemonic I learned to remember the order of operations in middle school math class. Other methods used in our early schooling included rhymes to remember what year “Columbus sailed the ocean blue” and the months that have 30 days. While mnemonics were useful for retaining these “elementary school” concepts, can these principles be applied to college-level courses where the content is much more detailed and sophisticated? In short, yes! One study tested the use of mnemonics in college students who were required to match a portrait to the name of its artist. Students who used mnemonics to recall the information significantly outperformed those who did not.³ The technique has been used to teach engineering students.  Eighty percent of the students reporting they benefited from the use of mnemonics and believe it help improved their exam scores.³

Mnemonics are rarely used in pharmacy school and vast amounts of information are presented to students from PowerPoint slides. From my experience, cramming for tests is common and most student use the repeating method, reading and re-reading handouts and notes as many times as possible shortly before an exam. This works well for some types of information and for a short period of time (long enough to remember it for the exam), but it is soon forgotten. A few instructors (but not many) still include memory aids in their presentations.  I still remember “Please Let Grandma Brown Bring Peaches To Your Wedding” as a way of remembering the colors of warfarin tablets (pink/ lavender/ green/ brown/ blue/ peach/ teal/ yellow/ white). However, mnemonics are used far less often in college than in elementary school.

Another memory aid that health professional students could use is SketchyPharm and SketchyMedical.com which uses spatial visualization to help learners understand complex topics. This is a subscription-based service first developed for medical students but has one section specifically related to pharmacy topics. A picture is presented and each item in the picture is associated with a more complex piece of information, and essentially helps the student visualize what they are learning through storytelling. This gives the phrase “a picture is worth a thousand words” a whole new meaning.⁴ While there are no peer reviewed research articles (yet) to support its effectiveness, medical students from the University of North Carolina rated the use SketchyPharm higher than many other common exam prep techniques (like flash cards) and 83% of students claimed they used it to study for their Step 1 medical licensure exam.⁵

The use of memory aids diminishes as a student progress from elementary school, to high school, to college, and beyond. Why?  Memory aids work and instructors should be using mnemonics and encouraging students to use spatial visualization to not only recall information but move important facts out of short-term memory. There are lots of websites to help create mnemonics; so, there no excuse not to incorporate them into lectures in combination with active learning activities. Encouraging students to use SketchyPharm or spatial visualization strategies during and after a lecture can really solidify the information being taught.

References

1. Comtois T. Using Memory to Become a more Effective Advisor [Internet]. Manhattan (KS): NACADA Clearinghouse of Academic Advising Resources; 2006 Nov [cited 2020 Feb 22].
2. Banikowski AK, Mehring TA. Strategies to enhance memory based on brain-research. Focus on Exceptional Children [Internet]. 1999 Oct [cited 2020 Feb 22];32(2).
3. Dave H, Awasthi S. An Investigation of the Role of Mnemonics in Higher Education. Proceedings of International Conference on Digital Pedagogies (ICDP) 2019 [Internet]. 2019 Apr 21 [cited 2020 Feb 22].
4. SketchyMedical [Internet]. Sketchy Group, LLC; 2018 [cited 2020 Feb 22]
5. Keepers B, Oh L. Step 1 Preparation: 2020 Hindsight. University of North Carolina [Internet]. 2018 May [cited 2020 Feb 2022].

The Utility of Debates as an Alternative Instructional Method

by Amanda Bridges, PharmD, PGY1 Pharmacy Practice Resident, University of Mississippi Medical Center 

The landscape of education is ever-changing with educators pioneering fresh ideas to disseminate knowledge and maintain student engagement. While traditional lectures remain a mainstay in many health professional degree programs, students often struggle to make meaningful connections with the subject matter. Healthcare topics are often complex and require motivation and engagement from students to apply content to future experiences. Studies have shown that active involvement in learning results in more favorable outcomes. The success of any teaching strategy is dependent on successful execution by the instructor. Enter debates – an instructional strategy that dates back to the 5th century. Debates provide an opportunity for students to thoroughly research and logically present a topic.  Moreover, debates foster critical thinking and help students build effective communication skills.¹ While not ideal in all didactic settings, debates are a great way to teach controversial topics that have literature to support different stances.

Icons made by Freepik from www.flaticon.com

Some of the most challenging skills to develop as students make the transition to independent practitioners are clinical decision-making and problem-solving.²  Students are often left to develop these skills after graduation in conjunction with learning the intricacies of a new career. Debates foster the development of both of these skills by encouraging critical thinking to formulate a well-reasoned argument. Successful preparation for the debate involves discovery and assessment of literature – a skill that students will continue to use throughout their careers. Additionally, the exercise helps students to develop effective and persuasive communication techniques that will prove useful during both professional and personal interactions. Traditional lectures, I believe, fall short at fostering the development of these skills.³ 

The implementation of debates as a learning strategy occurs in two distinct steps – the pre-debate and the debate. Instructors should prepare for the debate by framing the chosen topic as a debate – that is to say, making a statement regarding an issue rather than asking students to answer a question. Students are then assigned to two opposing sides – a pro side in favor of the statement and a con side who is opposed to the statement. During the pre-debate stage, students are expected to thoroughly research and prepare a brief (typically 10-12-minute) presentation to support their side of the argument as well as collect ample information to support their stance in the event of a rebuttal from the opposing side. During the debate, the two sides present their argument and a rebuttal followed by a discussion by the entire class, facilitated by the instructor, about both sides of the argument. Some teachers had added another opportunity for learning by having the students prepare test questions tho assesses the key points presented during the debate.^1,2 In addition to being a means to assess student understanding, question writing can stimulate interest in academia as a career path.

In contrast to the assessment techniques used after traditional lectures, which focus on knowledge acquisition, debates can be used to evaluate students on concepts other than knowledge. Typically, the desired learning outcomes are assessed using a rubric that is given to the students before they begin their debate assignment. The assessment of individual students should focus on the persuasiveness of the presentation, use of data to support the argument, composure during the debate as well as eye contact and body language. An assessment of the debate team (or group) could include avoidance of redundancy during the formulation of the argument – an indirect way to assess team communication.¹ The evaluation rubric can be modified based on faculty resources and class size.² 

Studies have shown that students may benefit more from a debate when the approach is coupled with a patient case. Without this application, students struggle to make the connection with clinical practice.¹ Using debates repeatedly during a course can help students hone their skills.  When asked, students admitted that the initial debates took much longer to prepare than those occurring later in the semester.² 

While this instructional method has some clear advantages, debates as a teaching strategy also have limitations. As a student who struggled to speak up and express opinions in front of the class, I can see how this approach is intimidating. However, as healthcare professionals on an interdisciplinary team, it is essential we develop the confidence and poise to speak up and make appropriate recommendations. For students who struggle to speak up, a solution might involve grouping similar students on a team so that there isn’t a strong personality to drown them out. Additionally, debates inherently encourage competition which could lead to a trivialization of issues for the sake of winning.³ The goal is not to “win” the argument but rather to thoroughly research and present the argument with clarity using facts.  Once both sides of the argument are clearly understood, it’s important to emphasize consensus-building and compromise. Lastly, debates can be time-consuming and require a lot of faculty manpower. It may be necessary to split up students into subgroups in a large class to allow direct observation during multiple debate sessions.²

As a student I would have been terrified to take a course that required students to debate topics throughout the semester.  But in retrospect, the opportunity to enhance critical thinking and clinical decision-making using a debate format would have been immensely valuable. The greatest utility of debates, in my opinion, lies in the exploration of topics not easily explained in black and white terms during traditional lectures.

References

1. Charrois TL, Appleton M. Online debates to enhance critical thinking in pharmacotherapy. Am J Pharm Educ. 2013; 77 (8): Article 170.
2. Hawkins WA, Fulford M, Phan S V. Using debates as the primary pedagogy to teach critical care in a PharmD curriculum elective course. Curr Pharm Teach Learn. 2019; 11 (9): 943-948.
3. Darby M. Debate: a teaching-learning strategy for developing competence in communication and critical thinking. J Dent Hyg. 2007; 81 (4).

The Airplane Mode Classroom

by E. Ashton Smith, PharmD, PGY1 Pharmacy Practice Resident, Mississippi State Department of Health 

With today’s technology, people spend hours upon hours consumed with a screen every day. Whether it is scrolling on social media, shopping online, texting with friends, playing games, or watching the next episode of a favorite Netflix series, we are rarely disconnected from the internet. We can enjoy this media from our phones, laptops, tablets, televisions, and now even smartwatches. It can be a pleasant distraction.

Icons made by Freepik from www.flaticon.com

But let’s talk about distractions. Media available on the Internet and through our devices can be a good thing; for example, to pass the time during periods of boredom or to partake in a few enjoyable laughs from a friend’s text message. However, the distraction of constant notifications and the temptation to while away hours of time online can be harmful when there are other things at hand that are currently more important.

Devices in the classroom have become second nature to today’s college students. Laptops, cell phones, and tablets are brought into the classroom each day and can be beneficial for things such as note-taking and other class-related activities. These devices, however, can be a major distraction from learning and participation. In a recent study, the investigators surveyed college students in 26 states in the United States.¹ Their sample included college freshmen all the way up to graduate students. The survey focused on the use of digital devices for non-class purposes in the classroom and the effects that usage may have on learning. Almost 97 percent of the participants admitted to using a digital device on a typical school day in the classroom for non-class related activities. The top two reasons (which were both reported by more than 50% of students) for using a device in the classroom were to stay connected and fight boredom. Although the students were choosing to use those devices during class time, it was not without realizing that it could hinder their learning. Nearly 90% of students admitted that the biggest disadvantage of using devices in the classroom was that it causes them to not pay attention.¹ These results are from a survey conducted in 2016.  It seems likely device use would be even greater now in 2020, as the availability of social and video media online continues to skyrocket.

A study published in 2012 found that students who used their laptops during class scored an average of 11 percentage points lower in the course than the students who did not use their laptops.² That is a whole letter grade! Not only does the laptop user become affected by non-class related internet usage during class, but the surrounding peers are also hindered by this distraction. In an experiment conducted in an undergraduate psychology class at McMaster University, half of the participants were instructed to use paper and pencil only to take notes during a lecture.³ The other participants were asked to use a laptop during the class session and were given tasks to complete on their computers during the lecture that was not related to the instruction. The students were given assigned seats and placed strategically around the room such that some students would have a view of other students using laptops, and others would have a distraction-free view. At the conclusion of the lecture, all participants completed a multiple-choice comprehension test with questions evaluating simple knowledge and application of the information presented during the lecture. Not only did the participants using the laptops perform significantly worse on the comprehension test but also the participants who had a view of a student with a laptop scored significantly lower than participants in the distraction-free zones.³ 

Students know the negative consequence and yet still continue to use their devices during class. So, what is the solution? Some schools have explored the idea of disabling WiFi access in classrooms. Cornell University in Ithaca, New York has implemented the use of software to block access to WiFi while inside classrooms. This doesn’t, however, stop cellular signals, but it likely reduces media usage during class. Purdue University in West Lafayette, Indiana has restricted WiFi access throughout campus during class time hours. Streaming sites were completely banned in March 2019. Feedback from faculty has been positive, with one professor saying it has helped students who were previously distracted by other students streaming during class. The ban also challenged teachers at the university to develop class activities that engage students since more were actually paying attention in class!⁴ 

Some classrooms have gone “tech-free.” Teachers in schools throughout the country have experimented with policies that prohibit students from bringing laptops, phones, and tablets into the classroom. A professor in the Department of Psychology and Neuroscience at Duke University tested this policy. She stated that the results were positive and students really loved it. Indeed, students in the tech-free class scored five percent higher on exams than previous semesters.⁵ 

Another way to ease the students into using devices less frequently is to include “tech breaks” during class time. These five-minute breaks are sandwich between lecture segments during which students are allowed to pull out their phones and check social media, text messages, and emails. Students must pay close attention during lecture time to receive this privilege, however. For example, an hour-long class might have 30 minutes of lecturing, then five minutes for a tech break, before the second half of the class begins. This is a good way to not look like the bad guy, eliminating technology completely, and acknowledging how important students’ devices are to them.

Strategies to limit electronic device use in classrooms will continue to rise as the next generation of learners who are even more addicted to their devices enter college. While eliminating WiFI access and banding electronic devices continue to gain traction in schools, more pressure will be placed on teachers to use more engaging instructional strategies during class. Teachers need to explore ways to restructure class time. Ideas include more in-class small group activities, pop quizzes at the end of presentations, and inviting and interacting with guest experts. One of the most useful techniques is to chunk class time into brief activities, each focused on different learning objectives. These brief activities keep the students engaged and involved. Clearly, we need more research about the harms of technology addiction, how it hinders learning, and strategies to address it. 

References

1. McCoy B. Digital distractions in the classroom phase II: student classroom use of digital devices for non-class related purposes. Journal of Media Education [Internet]. 2016; 7: 5-32. [cited 2020 February 10].
2. Duncan D, Hoekstra A, Wilcox B. Digital devices, distraction, and student performance: does in-class cell phone use reduce learning? Astronomy Education Review [Internet] 2012; 11: DOI: 10.3847/AER2012011 [cited 2020 February 10].
3. Sana F, Weston T, Cepeda NJ. Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education [Internet]. 2013; 62: 24-31. [cited 2020 February 18].
4. Smith M. Purdue University bans Netflix, other streaming services for students in academic buildings. The Washington Post [Internet]. 2019 March 15. [cited 2020 February 18].
5. Gaither S. Why you should consider a tech-free classroom. Psychology Today [Internet]. 2019 September 23. [cited 2020 February 18].