Access to Recorded Lectures and Students’ Performance and Memory

by Alicia Rogers, Pharm.D., PGY-1 Pharmacy Resident, Baptist Memorial Hospital-North Mississippi, Oxford, Mississippi

In recent years, there has been lots of discussion about whether to require students to attend class for lectures and then later allow students access to recordings of the lectures. Until recently, I didn’t quite understand why this was such a hot topic or what all the fuss was about. My thoughts were simple. If students wanted to skip class and watch lectures at a later time, let them. However, as I’ve come to understand, there is bigger picture. It’s not simply about convenience or preferences, it’s about student learning. Does allowing students to have remote and later access to recorded lectures affect students’ long- term memory and performance? I decided to take a deeper look.
 

Recently, investigators at the University of North Carolina Eshelman School of Pharmacy published a study entitled “Exploring the Consequences on Memory of Students Who Know They Have Access to Recorded Lectures.”² I found the results very interesting and contrary to my assumptions. The study included to two parallel groups: students with access to a recording of a lecture after the lecture was delivered and students who did not. The goal of this study was to test immediate recall performance and to gauge performance during delayed recall. Students with access to recorded lectures had the option to repeatedly view the material. Students without access would need to find other means to review the material. This study found that there was increased performance in delayed recall with the group with assess to lectures. However, access to recorded lectures was not beneficial from any other standpoint and there was a suggestion that knowledge of assess to recorded lectures could have negative effects on memory and encoding long-term.² In their discussion, the authors explore two educational theories which may explain this finding: The Efficient Encoding Hypothesis and The Desirable Difficulty Hypothesis. When students have access to recorded lectures, taking fewer notes reduces the quality of brainpower to learn because no real work is required to support the commitment to long-term memory. There was also evidence to support The Efficient Encoding Hypothesis whereby students with no access took fewer notes, but in turn, paid more attention to the initial lecture for greater understanding.² These results were completely different than what I thought would be true.

One major problem I have with this study is the conditions under which it was conducted. It was a simulation. Thus, the study lacked "real life" variables and consequences. This might play a major role in students' behaviors and subsequent performance. Performance and memory are dependent on several intrinsic factors, including motivation.

The authors should be applauded for considering some important instructional design principles to help students learn effectively in their study. Microlearning, spaced learning, and retrieval were all used. Microlearning is providing instruction in short snippets such 5-15 minute videos.³ Spaced learning is the act of repeating information to learners over time instead of relaying all of the information in a single session. Spaced learning substantially improves long-term encoding and may boost learning by as much as 200%.³ Lastly, retrieval involves improving retention and performance by asking questions. All three learning strategies were used in both groups. Research suggests that these strategies help learners refresh their knowledge and strengthen their memory of the content.

These instructional design strategies are routed in several theories in education psychological including behaviorism, cognitive learning theory, and constructivism. Behavioral psychology advocates repetition and reinforcement in learning material to create a “behavior” in the learner. Cognitive psychology focuses on engaging the learner’s senses to create a learning process, while constructivism emphasizes the learner’s own experience and personal interpretation.1 There are intrinsic factors that play a role in learning including motivation and how individuals encode information. It would be expected that students that re-watch recorded lectures would have increased retention. Although re-watching recorded lectures is not the only means to increase retention, it does provide an opportunity for repetition.

As a former pharmacy student, access to recorded lectures allowed me to reinforce the information that was initially covered during a face-to-face lecture. There are times when students can be distracted during a lecture — after 1 or 2 hours, who wouldn’t get distracted! Having access to recorded lectures allows student access to materials they may have missed during the live session. Some students use recordings in order to fill in gaps in their notes and to help with a better understanding of the course material. When re-watching lectures, students can slow down the speed and this can be helpful to some students who need a bit more time to fully understand what the teacher is saying (e.g. students whose primarily language is not English).

We are in the technology age. Millennials are accustomed to online learning and having information available at their fingertips 24/7. Many people enjoy the convenience of technology and take advantage of learning on-the-go. As a student, I enjoyed being able to re-watch lectures in the comfort of my own home so that I could focus on understanding the information without interruptions. I believe students can get more out of lectures when they do not have a constant stream of distractions. Distractions are a common problem in the classroom. Thus, not having access to recorded lectures after class would be detrimental from some, if not most, students.

So, while this study is intriguing, I do not think denying access to recorded lectures is the best strategy to enhance recall and long-term memory. Indeed, there was no difference in performance when students re-studied using a recorded lecture when compared to when students retrieved information through questioning.⁴ While the overall group performance was similar, there are likely individual differences and some students would undoubtedly perform better using one reinforcement strategy versus another.

In my opinion, this study was flawed. The study was not conducted under “real life” conditions and the subject matter (astronomy) was unrelated to anything that a pharmacy student would ordinarily study. Moreover, students are expected to learn and retain information from multiple courses and subjects simultaneously. In a pharmacy curriculum, knowledge builds on itself. Students often use previously learned material to help bring together their understanding and “just-in-time” access to recorded lectures can be very helpful. So, while access to recorded lectures might not be helpful in simulated learning environments like this study, it doesn’t appear to be harmful. So, if it’s not broken, why fix it?

References

1. Little B. Principles of instructional design. Retrieved from https://www.mindtools.com/blog/corporate/principles-instructional-design/  Accessed on October 6, 2019.
2. Patel B, Yook G, Mislan S, and Persky A. Exploring the consequences of memory of students who know they have access to recorded lectures. American Journal of Pharmaceutical Education 2019; 83(5): Article 6958. Retrieved from https://www.ajpe.org/doi/full/10.5688/ajpe6958 
3. ShifteLearning. Use these 5 instructional design strategies to create an effective e-learning course. Retrieved from https://www.shiftelearning.com/blog/instructional-design-strategies-effective-elearning  Accessed on October 6, 2019.
4. Palmer S, Chu Y, and Persky A. Comparison of re-watching recording and retrieval practice as post-class learning strategies. American Journal of Pharmaceutical Education 2019; Publication Ahead of Print. Retrieved from https://www.ajpe.org/doi/pdf/10.5688/ajpe7217

Incorporating LGBT Instruction into Pharmacy Curricula

by Elizabeth Hearn, Pharm.D., PGY-1 Community Pharmacy Resident, University of Mississippi School of Pharmacy, Jackson, Mississippi

While only 4.5% of adults in the United States identify themselves as lesbian, gay, bisexual, or transgender (LGBT), this population faces some significant health problems.¹ HealthyPeople2020 reports that LGBT youth are more likely to be homeless and are up to 3 times more likely to attempt suicide when compared to heterosexual, cis-gendered youth. They also have significantly higher rates of illicit drug, tobacco, and alcohol use.² Transgender individuals are more likely to suffer from mental health disorders such as depression, post-traumatic stress disorder, and suicidality as well as HIV/AIDS and other sexually transmitted infections.^2,3 Perhaps one of the most shocking data for healthcare providers to digest is that LGBT individuals report feeling isolated and a lack of social support due to culturally incompetent healthcare providers.²

HealthyPeople2020 set nationwide goals for improving the health, safety, and well-being of the LGBT community. One of the metrics is to provide, “medical students with training to increase provision of culturally competent care.”² However, healthcare providers across multiple healthcare disciplines – not just physicians – have reported limited knowledge and confidence in treating transgender patients.³ Are we doing enough to equip pharmacy students with the knowledge and confidence needed to address the healthcare disparities experienced by the LGBT community? Or are we leaving them ill-prepared?

A recent cross-sectional survey of 142 schools of pharmacy in the United States found a wide variety of transgender-related instructional activities. However, only 53% (35, n=66) of surveyed schools reported purposely implementing instruction regarding transgender care in their curriculum. Of those schools who have transgender-related care coursework, only 14 of 35 provided 2 hours or less of student experience with this patient population. Moreover, only 15 schools had plans to further develop or implement transgender-related care in their curricula within the next 3 years. When schools were asked to rate graduating pharmacy students’ confidence level for providing competent care to transgender patients, only 2 schools felt students would have a high level of confidence.³ With these data in mind, there is room for improvement with regard to transgender-related care in American pharmacy schools.

Instruction regarding LGBT health-related issues can be taught in several ways. In a 2017 study, didactic instruction about transgender-related care improved pharmacy students’ scores on knowledge-based assessments and self-reported confidence in decision making. Third-year Doctor of Pharmacy students received a 2-hour lecture about transgender-related care. When compared to 4th-year students who did not receive the lecture, 3rd-year students scored significantly higher on both a knowledge-based assessment (63.4% vs. 72.5%) and a self-confidence assessment (60.6% vs. 76.8%).⁴

In another study, lecture-based instruction about LGBT healthcare was shown to improve students’ understanding of the role of a pharmacist in transgender care. Pre- and post-lecture surveys showed a marked increase in student pharmacists’ understanding of their role in caring for transgender patients (30.6% vs. 96.5%).⁵ Amongst the schools of pharmacy who have already incorporated instruction about transgender-related care into their curricula, didactic lectures and discussions are by far the most prevalent instructional methods.³

Some schools of pharmacy report offering Introductory and Advanced Pharmacy Practice Experience (IPPE and APPE) opportunities for students to interact with LGBT patients. Utilizing IPPEs and APPEs for this purpose may seem ideal, as educators have reported they perceive that there is not enough time in the didactic curricula to include additional lectures, discussions, or care-based activities. Designated IPPE and APPE sites could be LGBT-specific clinics or locations that have a higher-than-average percent of their patient population who identifies as LGBT (such as an HIV clinic). However, these IPPE and APPE experiences are uncommon, with only 12 (out of 63) schools providing an experience where students can interact with LGBT patients. Due to limited availability and potential site-to-site variability, these types of IPPEs and APPEs may be best suited as elective experiences.³

Researchers at two schools of pharmacy in Washington and New Mexico took a creative approach to their instructional methods. Students were exposed to 3 hours of content about cultural, empathetic, and medical considerations for patients with diverse gender identities. Interestingly, the educators used a variety of mediums, namely: a pre-recorded video lecture; a list of frequently-asked-questions and answers; a game-show-style game; a patient video; a gender identity exploration exercise; role-playing scenarios; and a panel discussion with gender-diverse patients. The gender-exploration activity, which was adapted from Lavender Health’s Gender Role Socialization activity, was rated least helpful by students in the post-class surveys. However, more than half of the students found the remaining exercises to be very informative — the panel discussion with gender-diverse patients was considered by students to be most helpful. A panel session with individuals who identify as LGBT appears to be an effective way to include LGBT instruction in pharmacy curricula.⁶

We should use these examples to establish or update curricula at schools of pharmacy. When developing LGBT-related care education, consider the following learning opportunities for student pharmacists:

1. Understanding sexual and gender development
2. Discussing gender identity and how to document sexual preferences and gender identity in medical records⁴
3. Identifying barriers to access to care by LGBT people
4. Outlining drug and health-related resources for LGBT patients who are uninsured or underinsured
5. Motivational interviewing for smoking, alcohol, or drug cessation in the LGBT community⁷
6. Managing pharmacotherapy in LGBT people experiencing:
1. Mental health disorders
2. HIV and AIDS
3. Sexually transmitted infections
4. Chronic diseases
5. Gender transition ^4,7

Student pharmacists have expressed the desire to learn about this topic.⁵ Multiple exposures to LGBT-care-related issues would result in greater sensitivity, confidence, and competence.⁵ Given that health disparities for the LGBT community clearly exists and pharmacy students are willing to learn how to minimize them, it is our responsibility to step up and provide the opportunity. Incorporating LGBT instruction in our classrooms can transform a pharmacy student from a culturally incompetent healthcare provider to a well-informed, self-confident practitioner.

References

1. Williams Institute: Adult LGBT Population in the U.S. March 2019. Accessed 15 August 2019. Available from: https://bit.ly/2W2AxHB
2. HealthyPeople2020: Lesbian, Gay, Bisexual, and Transgender Health. Accessed 15 August 2019. Available from: https://bit.ly/2H939tF
3. Eckstein MA, Newsome CC, Borrego ME, et al. A Cross-sectional Survey Evaluating Transgender-Related Care Education in United States Pharmacy School Curricula. Currents in Pharmacy Teaching and Learning. 2019; 11(2019): 782-792. DOI: 10.1016/j.cptl.2019.04.005
4. Ostroff JL, Ostroff ML, Billings S, Nemec EC. Integration of Transgender Care into a Pharmacy Therapeutics Curriculum. Currents in Pharmacy Teaching and Learning. 2018; 10(2018): 463-468. DOI:10.1016/j.cptl.2017.12.016
5. Knockel LE, Ray ME, Miller ML. Incorporating LGBTQ Health Into the Curriculum: Assessment of Student Pharmacists’ Knowledge and Comfort Level in Caring for Transgender Patients. Currents in Pharmacy Teaching and Learning. 2019; DOI:10.1016/j.cptl.2019.07.001
6. Newsome C, Chen LW, Conklin J. Addition of Care for Transgender-Related Patient Care into Doctorate of Pharmacy Curriculum Implementation and Preliminary Evaluation. Pharmacy. 2018; 6(4): 107. DOI: 10.3390/pharmacy6040107
7. Mandap M, Carillo S, Youmans SL. An Evaluation of Lesbian, Gay, Bisexual, and Transgender (LGBT) Health Education in Pharmacy School Curricula. Currents in Pharmacy Teaching and Learning. 2014; 6(2014): 752-758.  DOI:10.1016/j.cptl.2014.08.001

Patient Simulation and the Benefits to Student Pharmacists

Darby Pullen, Pharm.D., PGY1 Pharmacy Practice Resident, Baptist Memorial Hospital, North Mississippi

Pharmacy education is continually changing and faculty should be familiar with a variety of models to teach students. During my time in Pharmacy School, patient simulation mannequins were used to demonstrate and practice hands-on techniques that I might later use in practice, such as how to give glucagon when a patient with diabetes is hypoglycemic and not responsive.

Students today are exposed to more stimuli and expect more than a typical PowerPoint presentation. Mannequins can mimic several disease states such as cardiac arrest and allergic reactions. Students can gather a variety of clinical information from a mannequin including a pulse, heartbeat, lung sounds, and other hemodynamic parameters.¹ Virtual patients are another form of simulation that can enhance student learning. Virtual patients are created in digital, computer-based environments.

Simulation is a way to bridge the foundational knowledge that is taught in the classroom to hands-on experience. Moreover, some students prefer hands-on activities in order to fully understand different disease states. Mannequins can benefit the students who are visual, kinesthetic, or auditory learners.¹ This would, in turn, enhance the pharmacy student’s knowledge, confidence, and clinical skills.¹

Simulation can enhance students’ learning process.¹ Using patient simulation requires more active student participation in a low-risk environment.¹ Students are able to have more hands-on activities that simulates real-life scenarios but without risking actual patient harm. Facilitators are usually available to correct student behavior when they are interacting with the mannequins and computer simulators can provide ongoing feedback.¹

According to the ACPE accreditation standards for pharmacy schools, “graduates must possess basic knowledge, skills, and abilities to practice pharmacy independently, at the time of graduation."² ACPE allows up to 20% of the students’ required hours of practice experience to be fulfilled using simulation.² By including more patient simulation in the classroom, it would be a win-win situation. Students would have greater access to semi-authentic learning exercises and would fulfill requirements for ACPE.

Faculty at the Universiti Teknologi MARA in Selangor, Malaysia evaluated the difference in students’ performance when they were taught using a human patient simulator compared to a problem-based learning activity.³ The investigators were teaching students about diabetic ketoacidosis and thyroid storm. This was a crossover study. Students were randomized to receive instruction using either a human patient simulator or a problem-based learning activity for one of the disease states and then crossed-over to receive instruction using the alternative instructional method for the second disease state. After each case, students were assessed using a posttest immediately after the lesson followed by a knowledge retention test 10 weeks later. The knowledge retention test included recall and application questions.

On follow-up surveys, students indicated they were more satisfied learning using the patient simulation rather than problem-based learning.³ The posttest and knowledge retention scores were significantly higher (p < 0.05) in the mannequin group (mean scores: posttest 78.5; knowledge retention 58.5) for the thyroid storm case when compared to problem-based learning group (mean scores: posttest 75.1; knowledge retention 53.5).1 This study suggests that human patient simulation may be more effective for long-term knowledge retention than problem-based learning.

In another study, investigators at Griffith University in Queensland Australia assessed the role of virtual (simulated) patients in pharmacy education. Specifically, they examined how students feel about the experience as well as how well they performance on knowledge tests and assessments of clinical skills.⁴ They performed a meta-analysis and found nine studies that compared virtual patients to traditional teaching methods. Their findings showed that the use of virtual patient to teach about therapeutic topics was not superior to traditional teaching in terms of student performance, but does improve the students’ reaction to the learning experience.⁴

Simulator mannequins are expensive. They can run anywhere from 16 to 90 thousand dollars.³ Plus there are additional costs for maintaining the mannequins and hiring qualified operators. This is a high cost for pharmacy schools to add to their budgets. However, schools can consider collaborating with other health professional schools. Nursing and medical programs often use patient mannequins. By using them for interprofessional activities, this not only improves pharmacy student knowledge but also teaches them about team member roles. By having more interprofessional interactions, medical and nursing student benefit as well.

Using mannequins and virtual patients, students must use critical thinking skills in addition to their baseline pharmacotherapy knowledge.⁴ Students can be placed into a high-stress environment but the stakes are low. You don’t want students having their first interaction with a real patient while still trying to learn about a new disease state. Students can be prepared for the experience by completing an orientation exercise and continue to repeatedly practice with the mannequins. Using virtual patients and mannequins, pharmacy students would have greater confidence when advancing into their advanced practice experiences.

There is some evidence that simulator mannequins improve students’ learning, particularly long-term knowledge retention.^4,5 Students respond favorably to learning activities involving mannequins and virtual patients because they are low risk. Traditional learning methods are still needed but simulator mannequins and virtual patients can help students build their confidence. Simulator mannequins should be considered when there is a hands-on technique that students need to master, such as working as an interprofessional team during a code or administering medications. Virtual patients can further pharmacy student’s knowledge and clinical decision-making skills.

References

1. Vyas D, Wombwell E, Russell E, et. al. High-Fidelity Patient Simulation Series to Supplement Introductory Pharmacy Practice Experiences. American Journal of Pharmaceutical Education. 2010; 74 (9) 169. DOI:10.5688/aj7409169
2. Accreditation Council for Pharmacy Education: Policies and Procedures for ACPE Accreditation of Professional Degree Programs – January 2010. Accessed 30 April 2019.
3. Chin KL, Yap YL, Lee WL, et. al. Comparing Effectiveness of High-Fidelity Human Patient Simulation vs Case-Based Learning in Pharmacy Education. Am J Pharm Educ 2014; 78 (8) Article 153. DOI:10.5688/ajpe788153 
4. Baumann-Birkbeck L, Florentina F, Karatas O, et. al. Appraising the Role of the Virtual Patient for Therapeutics Health Education. Currents in Pharmacy Teaching and Learning. 2017; 9 (5): 934-944. DOI:10.1016/j.cptl.2017.05.012
5. Seybert, AL. Patient Simulation in Pharmacy Education. Am J Pharm Educ 2011; 75 (9) Article 187. DOI:10.5688/ajpe759187

Key Elements to Consider When Developing Interprofessional Education Experiences

by Chase Board, Pharm.D., PGY1 Community Pharmacy Resident, University of Mississippi School of Pharmacy

My pharmacy school’s curriculum required each student to participate in one interprofessional education (IPE) training session and to write a self-reflection on the experience.  My IPE session occurred during my third year.  Working in a group with nursing and osteopathic students, we were tasked to provide an assessment and develop a treatment plan for a patient case scenario.  The instructors prefaced the activity to encourage us to focus our communication and teamwork skills.  For me, it seemed we struggled to find any purpose in learning how to work as a team.

The Joint Accreditation of Interprofessional Continuing Education defines IPE as events “when students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes.”¹  Highly effective interprofessional teams can improve health outcomes by enhancing the quality of patient care, reducing medical errors, reducing the hospital length of stay, and lowering medical costs.²  In a crossover study evaluating the quality of written transition plans for patients within a healthcare delivery system, treatment plans written by healthcare professionals who collaborated as a team correlated to stronger work quality when compared to healthcare professionals who did not actively collaborate together.³

Thanks in part to this growing body of literature, many academic institutions are implementing IPE.  Indeed, the Accreditation Council for Pharmacy Education (ACPE) requires all colleges/schools of pharmacy to implement IPE in their curricula.  In a January 2018 update, the ACPE Board of Directors clarified guidance to schools of pharmacy curriculum to include IPE experiences involving prescribers, specifically physicians and medical students, in didactic and experiential courses.⁴

Preceptors at the University of Wisconsin-Madison School of Pharmacy implemented IPE with a “dual identity” design.5  Their program trains students to become traditional pharmacists, but they integrate interprofessional learning and socialization with other healthcare disciplines.  The program offers elective experiences allowing students to volunteer at a student-run clinic.  These opportunities allow students to learn and apply interprofessional skills such as cooperation, communication, and coaching.  The aim is to prepare students for their fourth-year Advanced Pharmacy Practice Experiences (APPE) and graduate “career-ready, collaborative pharmacists.”⁵

Preceptors should structure their IPE experiences based on the four core competencies outlined by the Interprofessional Education Collaborative (IPEC).  IPEC is composed of 21 national associations representing nearly all health professions educators, including the American Association of Colleges of Pharmacy (AACP).  IPEC serves to promote and advance the efforts of interprofessional learning experiences.  The recommendations provided by IPEC focus on four core competencies:⁶

1.   Values/Ethics for Interprofessional Practice

Team members should have mutual respect and shared values. This is a cornerstone of effective collaborative practice.

2. Roles/Responsibilities

Individuals should use their knowledge in collaboration with the other members of the team to address patient health care needs.

3. Interprofessional Communication

Team members should promote and maintain the health of their patients by engaging in effective team-based communication.  They should direct their dialogue towards the patient, the patient’s family, other members of the health care team, and the community.

4. Teams and Teamwork

Teams should discuss relationship-building values and team dynamic principles to plan, deliver, and evaluate patient care.

 

For example, during a fourth-year APPE, I served as a facilitator in a structured IPE experience.  Groups of pharmacy, nursing, and osteopathic medicine students were assigned to review a patient case, communicate electronically, and attend a face-to-face session.  The face-to-face activity was intended to simulate an actual patient encounter.  My role as an IPE facilitator was to observe each team during the simulation and record comments based on my observations. I was provided a rubric to assess IPEC core competencies such as communication, professionalism, roles/responsibilities assignments, and team cohesiveness.  At the end of the simulation experience, groups received feedback regarding the written progress notes and electronic communication.  They were asked to write self-assessments based on their individual and team-based performance. I believed this activity represents a good example of how to structure an IPE experience.

There are many assessment tools available to evaluate IPE experiences. When structuring an IPE experience, it is important to identify the type of tools available, as well as determine which tools are most effective.  The Committee on Measuring the Impact of Interprofessional Education on Collaborative Practice and Patient Outcomes suggests IPE experiences should utilize a mix of qualitative and quantitative assessment tools to evaluate outcomes.  Implementing only one methodology does not fully explore and capture the data that can be obtained by a mixed-methods approach.  A mixed-methods approach can provide insight into both the “what” and “how” of IPE interventions and outcomes.⁷

When selecting tools to use for IPE assessment, the assessment tools should measure attitude, behaviors, knowledge, and skills.⁸ This aligns with the IPEC four core competencies (values/ethics, roles/responsibilities, interprofessional communication, and teamwork).  The IPE experience I helped facilitate during my fourth-year APPE used three assessment tools to evaluate attitudes, perceptions, and behavior.⁸ Assessment tools like these can be found on the National Center for Interprofessional Practice and Education website.

Preceptors should role model effective interprofessional collaboration as a strategy to teach pharmacy students appropriate behavior to uphold during IPE experiences.  Including activities in a rotation whereby students observe and reflect properties can help them develop conscious behavior.⁹  Some strategies preceptors should use include being self-aware of being a role model, reserving time for discussion with students, facilitating reflection on the experiences, and explicitly communicating what is being modeled for the student.¹⁰

I’ve had the opportunity to role model for students at a local free clinic during my residency training.  A medical student was consulted to counsel a patient who was starting insulin therapy.  The medical student said they were struggling to remember subcutaneous injection technique.  I reviewed the proper insulin injection technique with her. Feeling more competent, the student counseled the patient and returned to me to reflect on her experience.

Functioning effectively on an interprofessional team is important.  Thus, we need to teach these skills to students while they are in school.  IPE experiences should be linked to the four IPEC core competencies. When developing IPE assessment, teachers should use quantitative and qualitative methods to evaluate student performances and experiences.  Role modeling can be used by facilitators and preceptors to demonstrate what effective behavior looks like in collaborative practice.

References

1. Definitions. Joint Accreditation Interprofessional Continuing Education website. 2019. Accessed May 15^th, 2019.
2. Buring SM, Bhushan A, Broeseker A, et al. Interprofessional education: definitions, student competencies, and guidelines for implementation. Am J Pharm Educ. 2009;73(4): Article 59.. Accessed May 15^th, 2019.
3. Farrell T, Supiano K, Wong B, Luptak M, Luther B, et. al. Individual versus interprofessional team performance in formulating care transition plans: A randomised study of trainees from five professional groups. J Interprof Care. 2018;32(3):313-320.
4. Clay Kirtley J., Vlasses P. ACPE Update – 2018. Oral Presentation at: American Pharmacists Association Annual Meeting; March, 2018. Nashville, TN. Accessed May 16^th, 2019.
5. Gerhards K. PharmD Program Strengthens Interprofessional Education. University of Wisconsin-Madison School of Pharmacy’s website. 2018. Accessed Mary 17^th, 2019.
6. Interprofessional Education Collaborative. Core competencies for interprofessional collaborative practice: 2016 update. 2016. Washington, DC: Interprofessional Education Collaborative. Accessed May 19^th, 2019.
7. Chapter 5: Improving Research Methodologies. Measuring the Impact of Interprofessional Education on Collaborative Practice and Patient Outcomes. Washington (DC): National Academies of Science. Published December 2015. Accessed May 19^th, 2019.
8. Shrader S, Farland M, Danielson J, Sicat B, Umland E. A Systematic Review of Assessment Tools Measuring Interprofessional Education Outcomes Relevant to Pharmacy Education. Am J Pharm Educ. 2017;81(6): Article 119. Accessed May 19^th, 2019.
9. Cruess S, Cruess R, Steinert Y. Role Modelling – making the most of a powerful teaching strategy. BMJ. 2008;336(7646):718-721.

Assessing Students on Advanced Pharmacy Practice Experiences

by Taylor Loper, Pharm.D., PGY1 Pharmacy Practice Resident, Baptist Memorial Hospital-North Mississippi

One thousand, four hundred and forty. That’s the minimum number of Advanced Pharmacy Practice Experience (APPE) hours the Accreditation Council for Pharmacy Education (ACPE) mandates a pharmacy student receive before they graduate.¹  Over the course of at least thirty-six weeks (and sometimes more), these students are exposed to a variety of practice settings, institutions, and situations designed to support the development of their knowledge, skills, and abilities.  Essentially, their competence to practice as a pharmacist following graduation. The ACPE Standards state that the APPEs should be designed to “hone practice skills, professional judgment, behaviors, attitudes and values, confidence, and sense of personal and professional responsibility.” Schools are also required to have a formal assessment of the achievement of APPE competencies using validated assessments. Student performance must be documented at key time points throughout these experiences.

https://pharmacy.uiowa.edu/students/academic-programs/doctor-pharmacy/pep/appe

The CAPE educational outcomes developed by the American Association of Colleges of Pharmacy (AACP) are a set of goals that all pharmacy curriculums should be focused on achieving. These educational outcomes are linked to Entrustable Professional Activities (EPAs) that all pharmacy graduates should be able to perform.² (For more on EntrustableProfessional Activities, see the post by Andrew Mays) Students should have ample opportunity to practice these activities become proficient, and demonstrate mastery before becoming licensed pharmacists. This can be problematic in that the heavily didactic nature of the first three years of the pharmacy curriculum results in few opportunities to practice these EPAs.  Too often students are often being assessed before they have an opportunity to master the EPAs during APPE rotations.

Reflecting back on my experiences as a pharmacy student, the assessment of learning on APPE rotations involved a series of assignments that had to be completed by the end of the rotation.  An example of this would be a set of questions asking you to reflect on interprofessional teams and the benefits of working with different professions at your current practice site. While most assignments were site-specific, several (like the one above) were repeated for multiple rotations. Additionally, specific objectives were set forth and students were asked to provide evidence of assignments or activities they completed that enabled them to meet those objectives. An example of this would be to “evaluate and interpret patient data.” A student could then provide details of working patients up, reviewing medical records, or conducting patient interviews. This gave students an opportunity for reflection while providing concrete examples of progress that the APPE preceptors could then base their end-of-rotation evaluations on. However, completing these assignments and documenting these examples was often time-consuming. By the final APPE, they felt cumbersome, especially the assignments that we had to repeatedly do on multiple rotations.

This process of assessing student performance raises several questions. First, how do we ensure each student meets the required competencies for each rotation? With practices settings and sites varying significantly, assessing each student on basic competencies can be difficult. Moreover, different preceptors have different expectations. All this variability makes it very difficult to create a consistent assessment process that is not dependent on the student’s learning experiences. Second, how do we assure an assessment tool can be applied in a variety of APPE rotations without omissions or redundancy? Requiring the same assignments and reassessing the same set of skills for a student who is taking two community rotations puts a strain on the student and preceptor. But we must find a way to ensure the student is developing on each rotation. Finally, how do we measure competency, such as the EPAs?  Should be rated “acceptable”?  Or “completed”? Should a student be required to “complete” them by the end of each APPE or by the end of all APPE experiences?

Several institutions have tried to address these questions. The System of Universal Clinical Competency Evaluation in the Sunshine State (SUCCESS), is an internet-based APPE assessment tool created by the colleges/schools of pharmacy in Florida.³ Under this system, preceptors rate students as “excellent”, “competent”, or “deficient” for each competency at the end of each APPE. They are also allowed to select “no opportunity” if not observed. These ratings were then converted by the school to determine the student’s grade.  There was a correction factor for students that were earlier along during their APPE schedule. It also allowed preceptors to weigh each competency based on importance and frequency in the practice setting / site. This weight provides preceptors the ability to focus on the learning goals that are most relevant. Another such tool was created by faculty at the University of Colorado Skaggs School of Pharmacy after the addition of 14 ability-based outcomes to their curriculum.⁴ By polling current preceptors, they were able to determine which competencies and outcomes were frequently observed and how important they are to the success of students on each APPE. These responses were used to create APPE-specific tools to ensure students met rotation goals that aligned with the ability-based outcomes of the curriculum.

It’s clear that assessing the performance of APPE students is a crucial, yet complex, task. Based on the two methods documented above, implementing an effective evaluation method requires the active participation of preceptors in developing a tool that is specific to each APPE experience. Preceptor evaluations of students need to be specific to the setting and site but must also relate to the overarching ACPE standards and ACCP outcomes.

I believe that monthly preceptor evaluations of students and their progress toward or achievement of learning objectives are necessary to ensure each APPE experience is helping to develop the student’s competence. However, rather than completing a series of monthly (and sometimes redundant) assignments, a series of unique assignments completed over the ENTIRE year coupled with specific ability-based assessments might be a better strategy.  This can reduce assignment fatigue and still provide appropriate documentation that each student can competently perform the EPAs and other educational outcomes before they graduate. It would great to see some research to determine the validity of this approach.

References

1. Accreditation Council for Pharmacy Education. Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (Standards 2016). 2015.
2. Haines ST, Pittenger AL, Stolte SK, et al. Core Entrustable Professional Activities for New Pharmacy Graduates. Am J Pharm Educ. 2017; 81(1): Article S2.
3. Reid LD, Nemire R, Doty R, et al. An Automated Competency-Based Student Performance Assessment Program for Advanced Pharmacy Practice Experiential Programs. Am J Pharm Educ. 2007; 71(6): Article 128.
4. Gilliam EH, Brunner JM, Nuffer W, et al. Design and Content Validation of Setting-Specific Assessment Tools for Advanced Pharmacy Practice Experience Rotations. [published online ahead of print March 6, 2019] Am J Pharm Educ. Article 7067.