Stop Re-Reading … and Try Self-Testing!

by Amanda Schartel, Pharm.D., PGY2 Ambulatory Care Resident, University of Maryland School of Pharmacy

Take a minute and think back on the hundreds of quizzes, tests, and other assessments you have taken during your years in school.  Remember the countless hours and late nights you put in trying to learn everything in time?  Now think about how much of that information you actually remember days, weeks, months, and years later.  If you’re like me, you probably realize just how much of that information you have forgotten.  While you put in a lot of time and effort studying, the information learned was all too soon forgotten.  This is due, in part, to the ineffective and inefficient study strategies that most of us employ.  As students progress on to college and graduate school, poor study habits becomes a bigger problem as the material becomes increasing complex. This problem is particularly important for students training to become healthcare professionals given the fast pace at which medical information changes.¹

Student Perceptions and Use of Study Strategies

In order to help our students succeed, we must first identify what study strategies they currently use and what perceptions they have about effective ways to learn.  A survey of undergraduate students at a top university found that repeated re-reading was the most common study strategy (83.6%) with more than half of the students ranking re-reading as their preferred study strategy.  One of the least common study strategies used was self-testing (10.7%).  And the majority of these students avoided self-testing if they could not go back to re-read the material.²

These findings are similar to a study conducted by Hagemeier and colleagues who surveyed pharmacy students about their study habits.  They found that only 11.6% of students would self-test if they did not have the option of going back to re-read the material.  Half of the students reported that if they self-tested, it was to see what they had learned.  Only a third of the students believed self-testing helped them learn more.  The majority of students stated that spreading out study sessions was beneficial, but only 17% scheduled study sessions in advance and 60.2% reported that they crammed for exams.  

The majority of students state that their study habits are NOT based on formal instruction on how to study.¹ This suggests that most students have never been trained to study in effective ways.  Interventions by educators may give students the tools needed to succeed.

(In)Effective Study Strategies

We now know that many students use re-reading as their preferred study strategy. But is this actually an effective way to study? Are there other study strategies that are more effective?

 

While re-reading is common, it has only modest benefits on learning.³  Re-reading is most effective when there is a short delay between readings.  It is not effective when the material is re-read immediately following the initial reading.  Re-reading is most useful for recall-based tasks, and its effects on comprehension are questionable.  In head-to-head studies, re-reading has been found to be inferior to several other study strategies, including self-testing, elaborative interrogation, and self-explanation.³  Students may be under the false impression that re-reading is an effective strategy because they are committing a great amount of time when using this strategy.  But this may be a “labor-in-vain” phenomenon, whereby spending extra time memorizing information does not promote meaningful learning.² In a comprehensive review of learning techniques by Dunlosky and colleagues, re-reading, summarization, highlighting/underlining, keyword mnemonic, and imagery for text were identified as a low utility learning strategies.³  This was based on research showing that these techniques do not increase performance, lead only to short-term retention of information, and are only helpful for some students and in some contexts.³

So if these commonly used learning strategies are ineffective, which strategies would be more effective for students to use?  Dunlosky and colleagues identified two high utility strategies:  self-testing and distributed practice.³  Both of these strategies benefit learners of different ages and abilities, as well as increase performance across many tasks and contexts.²

Self-Testing

The literature has described the concept of the “testing effect” since the 1600s.  It suggests that the act of retrieving information from memory increases long-term retention.¹  Also, it has been proposed that self-testing may improve performance by enhancing how students mentally organize and process information.²  The testing effect is not just limited to formal assessments, but can also be seen when students practice recall on their own (i.e. complete practice problems, use flashcards).  In a study of first-year medical students by West and colleagues, researchers found that self-testing was a significant predictor of first-semester final grades and practical examination averages.⁴

Distributed Practice

We know that many students typically pack their studying into a short period of time before an exam.  While this may lead to short-term retention of the material, it has been shown that spreading out study activities over time, or distributed practice, has greater benefits and leads to long-term retention.  This may be because distributed study sessions force students to repeatedly retrieve previous knowledge, which is known to reinforce memory and retention.  Furthermore, when study sessions are spaced out, it forces the student to work harder to retrieve the information from memory and allows them to more accurately assess their understanding.³ Similar to self-testing, West and colleagues found that time management—including maintaining a study schedule—was a significant predictor of first-semester final grades and written examination averages.⁴

Help Students Adopt Better Study Habits

Given that many students use study strategies that are less than optimal, educators should provide guidance to students about the value of alternative strategies such as self-testing and distributed practice.  Furthermore, structuring instruction to encourage students to use these strategies, such as creating practice problem sets for self-testing, can help the students to become successful lifelong learners.

References

1. Hagemeier NE, Mason HL. Student pharmacists’ perceptions of testing and study strategies. Am J Pharm Educ. 2011;75:Article 35.
2. Karpicke JD, Butler AC, Roediger HL. Metacognitive strategies in student learning: do students practise retrieval when they study on their own? Memory 2009;17:471-9.
3. Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychol Sci Public Interest 2013; 14:4-58.
4. West C, Sadoski M. Do study strategies predict academic performance in medical school? Med Educ. 2011;45:696-703.

Action Learning through a Medical Mission Trip

by Julie Pauly, Pharm.D. Candidate, Lloyd L. Gregory School of Pharmacy, Palm Beach Atlantic University

Action learning requires individuals to take ownership of their decisions by solving real-life problems with reflection on the results. First described by Reg Revans in 1982, action learning principles were developed for businesses and other organizations as a means for employee development, team-building, problem solving, and quality improvement. There are learning opportunities in encountering the unknown. According to Leonard and Marquardt, action learning has educational implications, where students can enhance not only foundational knowledge, but leadership and team-building skills.¹

Action learning is simply “learning by doing” where an individual within a “set” or group reflects upon her/his role, while gaining the aid or advice not only from a facilitator but also peers.² Each group must establish their own rules and structures that determine norms of behavior and how they will complete complex tasks. Action learning is best applied when standard solutions to problems are unavailable and past experiences can guide decisions. This lack of fixed procedure creates a “highly situational” learning environment, where application of knowledge and strategy must be catered to the environment in which the student is serving. While action learning can be risky because it requires empowering individuals to take on tasks they may feel ill prepared to perform, there also comes a high gain in productivity in this “learning by doing” concept.

A medical mission trip embodies the action learning principles. This type of learning environment fosters the “learning by doing” mentality by providing learners with opportunities to execute and manage patient care, but also use their didactic education to “practice” with patients who have diverse needs.

In May of 2014 the Gregory School of Pharmacy took a group of pharmacy students on a medical mission trip to Honduras for a week of service to the people of Siguatepeque. I served as one of four fourth year clerkship students; each relying on one another for completing tasks and ensuring a smooth trip.  As a collective, we were responsible for managing the medication inventory, packing, and distribution of medications to be taken on the trip, as well as organizing the medications before and after each clinic day.  We were also responsible for scheduling the other student team members and facilitating their activities at each of the clinic stations.  This type of dynamic enhanced our understanding of what “team-building” really meant, learning the worth of those individuals who step up and do their part, as well as empowering those who needed to pull their weight. The preceptors served as “facilitators,” where they guided the clerkship students if questions arose but gave us full autonomy in the execution of the plan. These responsibilities gave the clerkship students an opportunity to develop their administrative, management, and leadership skills — skills that are very important for any working professional.

In addition, the clerkship students, under the supervision of the preceptors, had the privilege of interviewing patients to address their medical complaints. We were encouraged to share our view of the diagnosis and recommend treatment strategies that included lifestyle modification and medication(s) available from our limited formulary. More than 100 individuals lined up each day to be seen by our team.  We got plenty of practice!  Each day our interactions with patients became more refined, and each new experience reinforced our confidence.  According to Gifford, obtaining experience in this manner is superior to traditional teaching methods.  I agree!  Indeed my learning was far beyond what I’ve experienced in any “traditional” classroom or even experiential learning setting.  Perhaps because I was given more autonomy and everyone was relying on me.  My knowledge, critical thinking, and confidence as a clinician was accelerated.

As action learning requires, a degree of uncertainty needs to be present.  Koo describes the concept of uncertainty as “how to ask appropriate questions in conditions of risk, rather than to find the answers to questions that have already been precisely defined by others.”³  As a clerkship student, I was required to consider the difference in culture and norms in Honduras compared to my life in the United States.  Moreover, the medications and tools available to us in Honduras were very limited.  Thus applying my knowledge of standard “guideline” recommendations was not possible. We had to learn how to deliver the best possible care using the limited resources we possessed. Every decision had to carefully weigh what was best for the individual patient in front of us as well as the needs of everyone.  We had to ration our limited supplies.  We lacked extensive diagnostic tools and this made it very challenging when addressing patient complaints; I had to reflect on my knowledge of disease states to make decisions that were still at the highest standard of care. Thinking back on the experience, I am grateful for this uncertainty — the lack of well-defined ways to practice.  I know as a clinician we will be called to think outside the box.  Even in the United States, things don’t always have well-defined answers.

Another facet of action learning is reflecting on what was learned and how you will use your new knowledge and skills in the future.  This requires a personal inquiry regarding what is important when carrying out your role and responsibilities.  It was important for me to reflect upon my interactions with my peers and preceptors to gauge how best to approach any given situation. At the end of each day (and even as I write this), I reflected on the interactions I had with patients. I now have a deeper understanding of what it meant to be invested in a patient and maintaining compassion. Kindness and being available to a patient — truly listening to their concerns — goes a very long way if you want to give your best to a person.

I feel action learning is an ideal for training healthcare professionals. Hands-on practice in situations that do not have well-define answers gives the blossoming healthcare student the skills and experiences needed for their future role as a professional.  This type of “learning by doing” is often achieved through post-graduate residency programs; but this needs to be part of our professional degree programs too.

In order for action learning to work effectively, there must be willing expert facilitators who invest time and energy in their students. The facilitator must make certain that all learners have meaningful “doing verses watching” practice experiences, where the student takes the lead under a watchful eye. A group “set” must also be responsible for working together. However, some individuals may not wish to fully participate in this type of learning environment or accept this level of responsibility. Facilitators must encourage engagement and help manage group dynamics. A facilitator would also have to assist with problem-solving by help students reason through a situation and acknowledge that multiple answers may be available.

Is action learning appropriate for learners at all levels of education? Action learning requires us to generate our own knowledge through action, using our past knowledge as a foundation.  Therefore, it may not be appropriate for younger students, say those in middle or high school. Action learning also requires an internal reflective inquiry and this may be challenging to younger students.

Action learning has limitations too. If all learners in the group are not committed to the learning experience, the group will suffer. Action learning also requires an actively engaged facilitator, which may prove challenging while also managing other job responsibilities. Lastly, there are site-specific limitations and state laws that govern the scope of an intern’s practice responsibilities, thus giving students a high degree of autonomy is not always possible.

In conclusion, the application of action learning principles in higher education is incredibly powerful and this method of teaching should used when educating healthcare professionals. My mission trip experience is a model of action learning put into practice.

References

1. Leonard, H.S. and Marquardt, M.J. The evidence for the effectiveness of action learning. Action learning: Research and practice. 2010. Pg. 7, 2, 121-136.
2. Gifford J. Action Learning: Principles and Issues in Practice. Institute for Employment Series. May 2005.
3. Koo L. Learning Action Learning. Journal of Workplace Learning. 1999. 11(3):89.
4. Marquardt M. Action Learning and Leadership. The Learning Organization. 2000. 7(5): 233-241.

Podcasts: Don’t You Forget about Me

by Kashelle Lockman, Pharm.D., PGY1 Pharmacy Practice Resident, University of Maryland Medical Center

I stumbled upon the joys of podcasts when I was trying to make the DC and Baltimore traffic more bearable. Now I say, “Traffic is backed up for 10 miles due to a car crash? Whatever - I will just sit here and enjoy my podcast.” Of course, I started with podcasts for fun, such as This American Life, Car Talk, Writer’s Almanac, and Wait, Wait Don’t Tell Me. Then, as a fourth year student, I realized I could listen to medically oriented podcasts as a way to keep up with new studies and guidelines. This became especially useful as a PGY-1 resident. I recently found myself short on time to prepare for a topic discussion on JNC8 (the new hypertension guidelines). Well, there’s a podcast for that! After listening to it on my way home, I was able to more quickly review the guideline document and prepare my notes for the discussion. Since I love learning from podcasts so much, I suspect other learners might find them useful too.

A recent study in the Proceedings of the National Academy of Science found that STEM students who sat through traditional lectures were 1.5 times more likely to fail than students who were taught by professors that incorporated active learning techniques.¹ However, incorporating active learning can be a challenge given the large volume of material educators must cover in a pharmacy curriculum. While perhaps not as flashy as video podcasts (aka vodcasts or vidcasts) with their dual audio and visual components, we shouldn’t forget that podcasts are a useful tool to free up time for active learning and flip the classroom. Podcasts are easier to create than vodcasts, take up less computer / device memory, and allow the listener to multitask, to a certain extent. Commuting, dishwashing, and laundry folding can be turned into productive, intellectually stimulating time with a great podcast on your mobile device! Don’t get too carried away with trying to multi-task though, lest you interfere with your ability to process and learn new information! (See The Multi-tasking Myth: Technology Use andInstruction Outcomes by Brent Reed, Pharm.D., BCPS for more information.)

When podcasts were first introduced as a teaching tool in higher education, they were largely viewed as an add-on learning tool to a traditional lecture.  Many studies demonstrated their utility when used by students to review material.² In one recent study, dental students who received supplemental podcast instruction were shown to have statistically significant improvement in scores on a multiple-choice test with 91.3% of the students reporting the podcasts were useful.  However, 63% thought a podcast would be inferior to having audio and video.³ The effectiveness of podcasts versus vodcasts has not been evaluated but a study where lectures were offered through Mediasite (video) and as podcasts, more students accessed Mediasite recordings than listened to podcasts.  Thus, students appear to prefer video to audio only formats. But educational outcomes were not measured in this study.⁴

Some educators have evaluated the use of podcasts as a replacement for traditional lectures. In 2013, a small study (n=35) found that undergraduate nursing students who listened to a 51 minute pharmacology podcast broken into 3 separate segments scored higher on a multiple-choice and case-based assessment than did students who received the same material via a traditional face-to-face lecture or a continuous, non-segmented podcast. The difference in scores did not meet statistical significance (due to the small sample size) but there was a clear trend. It should be noted that the study did not include a student group taught using a series of short, segmented face-to-face lectures – but, lets face it, that’s probably not practical in terms of room scheduling.²

Podcasts are not just used for formal education in degree programs; many professional societies and journals deliver new research and guidelines through podcasts. The Society for Critical Care Medicine (SCCM) was the first to produce a podcast for a national medical society; its podcast iCritical Care features interviews with leaders in the field of critical care medicine. Both the New England Journal of Medicine and the Journal of the American Medical Association offer weekly podcasts summarizing their latest issues. The American Journal of Health-System Pharmacy produces a podcast that features studies it publishes. Other pharmacy news sources with podcasts include Pharmacy Times and Pharmacy Practice News. Assigning students a podcast versus a reading would introduce them to this technology as a potential method for engaging in continuing professional development after graduation. These podcasts can be found in the iTunes store.

If you want to make your own podcast to supplement or replace a classroom activity, it’s both easy and affordable. Audacity is free software that allows the creation and editing of audio files. It’s available for numerous operating systems, including Mac, Windows, and Linux. If you have a Mac, you have built in audio creation and editing software — Garage Band comes on all Macs and can be used to create and export podcasts. If ambient noise interferes with your recording, you can use a USB headset with a microphone.

Podcasts can be delivered through a secure Learning Management System, such as Blackboard, or they can be made freely available on the web. The latter option allows use of Really Simple Syndication (RSS), so students can subscribe and receive updates as new podcasts are uploaded. Rosalind Franklin University College of Pharmacy shares its podcast, Helixtalk, using the latter option. In either case, it is important to review how to access and download podcasts with students as knowing how to use the technology can be a barrier for some students.⁵ It might also be helpful to highlight that podcasts can be downloaded to mobile devices. In a study of nursing students, 70% of students who listened to podcasts played them on a computer, even though 73% of the students had mobile devices capable of MP3 playback.⁶ This indicates students may not know or appreciate the advantage of mobility offered by a podcast.

While podcasts don’t provide visual information like vodcasts, they can be used to augment live classroom activities, video, and readings. Rosalind Franklin College of Pharmacy’s podcasts focus on information relating to the Top 200 drugs and provide students with an additional way to review material. Similarly, short podcasts could be used to highlight essential material from a course, clinical controversies, or provide clarification on those muddier points that baffled students during lecture. By introducing podcasts as an educational resource in the classroom, educators can expose students to a technology they can use to stay abreast of new knowledge for the rest of their careers. I look forward to including podcasts among my learning assignments as I embark on my career in pharmacy education.

References

1. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014;111(23):8410-5. 
2. Abate K. The effect of podcast lectures on nursing students' knowledge retention and application. Nurs Educ Perspect. 2013;34(3):182-5.
3. Kalludi SN, Punja D, Pai KM, Dhar M. Efficacy and perceived utility of podcasts as a supplementary teaching aid among first-year dental students. Australas Med J. 2013;6(9):450-7.
4. Pilarski PD, Johnstone DA, Pettepher CC, Osheroff N. From music to macromolecules: Using rich media/podcast lecture recordings to enhance the preclinical educational experience. Med Teach. 2008;30(6):630-2.
5. Meade O, Bowskill D, Lymn JS. Pharmacology podcasts: A qualitative study of non-medical prescribing students' use, perceptions and impact on learning. BMC Med Educ. 2011;11:2.
6. Mostyn A, Jenkinson CM, McCormick D, Meade O, Lymn JS. An exploration of student experiences of using biology podcasts in nursing training. BMC Med Educ. 2013;13:12.

Finding Success in Productive Failure

by Thao K. Huynh,  PGY2 Oncology Pharmacy Practice Resident, University of Maryland School of Pharmacy

Learning is an activity that requires our undivided attention. Learning a novel or unfamiliar concept often involves struggle, which is a necessary component to critical thinking. A teaching method that causes a greater amount of struggle will lead to more learning as exemplified by productive failure.

According to Manu Kapur, productive failure engages students in differentiating prior knowledge from features of the new concept.¹ In using productive failure as a teaching strategy, there are two phases. The first phase requires students to generate and explore, while the second phase encourages them (with the help of the teacher) to consolidate and assemble new knowledge. In the first phase, teachers must discern what students know about a novel concept that hasn’t yet been formally taught. During this phase, a complex problem is presented collaboratively amongst students. During the second phase, the novel concept must be consolidated and presented with direct instruction in a structured way.²

In order to test this method, Kapur conducted a classroom-based research experiment comparing direct instruction with productive failure. In an all-boys school in Singapore, seventy four ninth grade students were taught about the mathematical concept variance in two different ways. Students were tested prior to the intervention to evaluate their pre-existing knowledge and it was similar at baseline in both groups. In one classroom students were provided direct instruction (DI).  In the other classroom, students in the productive failure (PF) group were provided no instruction or guidance. The two classes differed in that the DI class of 35 students participated in four 55 minute periods of instruction where the teacher explained the concept of variance, modeled the application of the concept by working through problems with the class, and highlighted pitfalls and misconceptions. Students in this class were able to work through problems in triads followed by a discussion of the solutions by the teacher. DI students were assigned homework after each class to reinforce the topic. In contrast, the PF class was not provided direct instruction during the first two periods. Instead, students worked in triads to solve one of the problems on their own. In periods three and four, PF students were provided the same direct instruction but the teacher also compared and contrasted student solutions and explained the solution in a manner similar to the DI class.  The PF class were given and solved fewer problems than did students in the DI class.   Moreover, students in the PF class were not given homework. Following this period of instruction, both groups of students took the same exam.  All students performed similarly overall – however, students in the PF group outperformed students in the DI group on the conceptual insight and data analysis portions of the exam.¹

Although, it can be argued that students in the productive failure group outperformed the direct instruction group due to a greater amount of time devoted to to studying the concepts; their performance is likely related to the amount of time and energy put forth into delving critically into the topic.

Productive failure can be used in the pharmacy curriculum and clinical pharmacy practice to help enhance learning experiences.  Incorporating productive failure may work in a group setting such as Abilities or Practice Lab, case discussions, and problem-based learning courses.  Students in these small learning groups naturally (without direct instruction) have some prior knowledge on the topic.  Productive failure will facilitate development of their critical thinking skills.  Allowing student to struggle is critical to the process.  This sets the stage for the instructor to introduce new concepts to students. This environment, according to Collins, leads “metacognitive scaffolding, contrasting cases, peer-interaction scripts, mixed-ability groups, and perhaps representational scaffolding in the intervention phase will enhance student learning in the [direct] instruction phase.”³ Furthermore, a lack of struggle can lead to detrimental outcomes.  For example, in a study that observed nurses on a patient care unit, when problems arose, instead of analyzing the situation, they opted for quick (but less than optimal) solutions.⁴

I believe productive failure is most useful when there is sufficient time to explore different solutions to a complex problem. If time is critical, productive failure may be detrimental. In addition, for productive failure to have the greatest benefit, learners should have some baseline knowledge on the topic.  In other words, the new concepts must be related in someway to the students prior knowledge.  If the new concepts are too foreign, students might not be able see the connections between their prior knowledge and the new knowledge.  I’ve encountered a recent situation where productive failure would not have worked well. An oncologist wanted to use equine thymoglobulin for a critically ill patient with steroid refractory acute graft-versus-host disease post stem cell transplant.  The drug needed to be administered as soon as possible. Not being an expert in this topic and having never seen horse thymoglobulin used before, I needed to clarify this with my preceptor. I found a review article on the topic but my preceptor needed to provide guidance. In this situation, using productive failure as a strategy to learn a novel concept without any instruction would have caused further delay in patient care as well as the potential for patient harm. On the other hand, productive failure worked with two students I asked to lead a topic discussion. The topic was acute myeloid leukemia, which neither student had any previous knowledge. I began with few patient cases for the students to read and discuss with one another.  After the students had an opportunity to struggled through the patient cases, I provided a formal one-hour interactive presentation. I measured the success of how well the students applied the information to direct patient care situations.  I saw a dramatic increase in knowledge from both students and they were able to provide appropriate recommendations to the patient care team.

Looking for ways to improve the way you teach?  Embracing the productive failure method will facilitate learning by encouraging learners to differentiate new concepts from prior knowledge and to be analytical in applying these new concepts.

References

1. Kapur M. Productive failure. Cognition and Instruction. 2008 Jul;26(3):379-424.
2. Kapur M, Toh PLL. Educational design research – Part B: Illustrative cases. Enschede, the Netherlands: T. Plomp, & N. Nieveen (Eds.); c2013. Chapter 17, Productive failure: From an experimental effect to a learning design; p. 341-355.
3. Collins A. What is the most effective way to teach problem solving? A commentary on productive failure as a method of teaching. Instr Sci 2012;40:731-735.
4. Edmondson AC. Learning from failure in health care: frequent opportunities, pervasive barriers. Qual Saf Health Care 2004;13(Suppl II):ii3-ii9.