Should Standardized Patients Score Student Performance?

by Ashley Miller, PharmD, PGY1 Community Pharmacy Resident, University of Mississippi School of Pharmacy

It's the end of the semester, and the last thing standing between you and your summer break is the objective structured clinical examination (OSCE). You know that you’ll be entering multiple interactive stations that will assess your ability to perform patient care-related activities. Who do you hope will be grading your performance – a teacher you’ve had, or a stranger — a standardized patient (SP)? I know what I preferred when I was the one undergoing these evaluations, but I was curious to learn more about what other professional students, faculty, and researchers had to say about who is the best person to evaluate and score a student’s performance.

OSCEs date back to the 1960s and were first used as assessments in medical schools.  Each OSCE station is intended to represent a realistic clinical scenario during the student interacts with a “patient.”^1,2  At many schools, the patient role is played by an experienced actor known as a standardized patient (SP).  An OSCE allows students to "practice" in an environment safe for both them and patients.¹ OSCEs are reliable and valid assessment tools and predict students' future success.^1,2  Their use was expanded to other health professional programs including dentistry, pharmacy, and nursing.^1,2 They were designed to comprehensively evaluate clinical, interpersonal, and problem-solving skills and consistently portray the clinical scenario so that every student has the same experience (and opportunities).^1,3 While preparing and delivering an OSCE is very time-consuming, both educators and students alike agree that OSCEs are a valuable learning and assessment tool.³

One thing not always agreed upon when considering OSCEs is whether a faculty member or SP should grade performance. In some instances, an SP may interact with the student while a faculty member grades the interaction while observing the encounter either remotely or in the same room. Some argue that having faculty graders introduces additional bias and negatively influences students' performance when compared to a more neutral grader.³ Others claim SPs do not have the skillset or training needed to properly assess students.³ Previous studies involving faculty versus SP graders have not provided a clear answer as to who makes the “best” grader.

Different evaluator factors contribute to variability when scoring OSCE encounters, including lack of defined criteria, lack of training, and the number of items to be assessed.⁴  One study looked at factors that affected student scores during an OSCE when evaluated by faculty versus SPs.⁴ Before grading began, all examiners were first put through a series of training on the OSCE process and the criteria they were to use for scoring students.⁴ The researchers found that the scores given by SPs were higher than those given by faculty members, suggesting that the type of grader does influence scores.⁴ Another interesting finding was that the faculty evaluators assessed technical skills more strictly, yet were not as strict when grading communication skills when compared to SP evaluators.⁴ The technical skills assessed included history-taking, physical examination, and patient education.⁴ Communication skills that were graded include the attitude of the student, active listening, ability to build rapport, and effective questioning.⁴ Notably, faculty members who were scoring items related to their specialty tended to assign lower grades.⁴ The authors hypothesized that these differences are seen because faculty graders are more familiar with assessing the technical skills (particularly if it was relevant in their specialty) and have higher expectations for performance, while SPs are not as comfortable giving lower scores on technical matters.⁴

While some faculty members believe that their presence does not impact student performance, students often report that knowing teachers are grading OSCEs increases testing anxiety.³ The increased stress then impacts performance which, in turn, affects students’ grades.^3,4 In a study conducted using student questionnaires to survey the use of SP versus faculty graders, McLaughlin et al. found that the majority of students felt SPs helped create a less stressful testing environment, were as good at giving feedback as faculty graders, and felt they were adequately equipped to assess their skills.² The findings of this study demonstrate that students generally prefer to be graded by an SP and believe that an SP can competently assess their performance.²

So, who should grade a student’s performance during an OSCE? It likely depends on who you are asking. Overall, it seems that most students feel SPs are equipped for the task, are fair graders, and help them to feel more at ease. Much like how I felt when I was a student, it seems students would prefer an SP in these encounters since it is a more realistic experience — similar to interacting with patients in the “real world.”³  However, some may contend that, while students may not be as comfortable, having professors performing the assessment is in the student's best interest long-term because they can more accurately assess the student’s technical skill. One point made for this argument is that some studies have shown that grades given by faculty are predictive of future performance.² Another point made by researchers and those in academia for having faculty graders is that they are content experts and may be able to identify students who have only surface-level knowledge but appear confident and skillful to a non-expert.² It is also possible to have SPs interact with the students while faculty members observe and grade the encounter synchronously or asynchronously.  In this way, the student performance is scored by both the SP and faculty members.  However, this would cost more time and money as both SPs and faculty would need to be trained. Research shows SPs focus more on communication while faculty focus more on technical skills in an encounter, thus, it may come down to the most important skill being assessed in a particular OSCE station when choosing who should score it.

^References 

1. Alsaid A, Al-Sheikh M. Student and Faculty Perception of Objective Structured Clinical Examination: A Teaching Hospital Experience. Saudi J Med Med Sci [Internet]. 2017;5 (1):49-55.

2. McLaughlin K, Gregor L, Jones A, et al. Can SPs Replace Physicians as OSCE Examiners? BMC Med Educ [Internet]. 2006;6: Article 12. 

3. Salinitri FD, O’Connell MB, Garwood CL, et al. An Objective Structured Clinical Examination to Assess Problem-Based Learning. Am J Pharm Educ [Internet]. 2012;76(3): Article 44. 

4. Park YS, Chun KH, Lee KS, et al. A Study on Evaluator Factors Affecting Physician-Patient Interaction Scores in Clinical Performance Examinations: A Single Medical School Experience. Yeungnam Univ J Med 2021;38(2):118-126.

Facilitating Student Success in Remediation Programs

by Madison Gray, PharmD, PGY1 Pharmacy Practice Resident, University of Mississippi Medical Center

Some colleges of pharmacy and other professional schools have programs in place to identify students early who are underperforming so that they can provide additional help and coaching before they fail a course.¹ Many of these strategies focus on tutoring and supplemental academic assistance to address underperformance and, ultimately, prevent failure.^1,2  Course failure is not only associated with a significant financial burden but also psychological and social consequences.¹ So, what is the most appropriate approach to remediation? Should remediation programs focus on course/content review or strategies for success? Several remediation approaches have been described in the literature for various professional schools and range from course repetition to developing individual remediation plans.² Individualized remediation that focuses on helping students become self-regulated learners is, in my opinion, a more effective approach to ensuring a student’s long-term success.^1-3

Years of research have identified self-regulated learning to be a differentiating factor separating high and low-performing students.³ Three key factors underly the success of self-regulated learning including preparation, performance, and self-reflection.³ Self-regulated learning requires the identification of barriers, appropriate time management skills, motivation, and strategic study habits.³

Many factors affect student performance and individual barriers often exist that impact each student’s ability to develop a self-regulated learning behavior.^1,3 Sansgiry and colleagues evaluated the effect of these factors on both high and low-performing students.  They found that test competence (which includes test anxiety) was one differentiating factor between the two groups.¹ They defined test competence as a “student’s ability to manage and cope with the amount of study material for examinations and/or tests.”¹ Many students (69.3%) reported feeling some type of anxiety during testing and some students even report experiencing physical symptoms.¹ Some other barriers affecting student performance include access to learning materials and technological barriers.¹ Identifying barriers for students is the first step in addressing issues that may hinder their success. Strategies to identify and address these barriers should be a routine part of the remediation process and could include interviews, surveys, and student self-reflection.

Self-regulated learning is a behavior that requires well-developed time management skills.³ Students who try to learn course material in a short amount of time tend not to perform as well as those who develop study plans whereby the learning effort is distributed over an extended period of time.^3,4 Appropriate time management skills that allow for more time for studying may also help decrease test anxiety and increase the student’s confidence in their preparation.^3,4 A study by Hartwig and colleagues assessed the study habits of college students in correlation with their grade point averages (GPAs).⁵ Students who reported scheduling study times over an extended period rather than cramming the day (or two) before an exam trended toward higher GPAs although the results were not statistically significant⁵. Remediation programs that encourage students to set goals, create tasks, and use schedules will help teach time-management skills that can be used in and out of the classroom.⁴ Effective time-management skills are necessary for both academic and career success.⁴ Once a student has good time management skills, learning how to use other study strategies may be helpful.^4,5

Students who use a variety of strategies to study, such as re-reading material, summarizing, note-taking, flashcards, and self-testing, are more likely to be successful.⁵ These different strategies have been studied and some of these strategies are used by high performing students more often when compared to low performing students.^3,5 In the study by Hartwig and colleagues, a survey administered to college students included questions about study strategies, self-testing, and study schedules.⁵ Self-testing correlated with higher GPAs versus other study strategies.⁵ The majority of students who reported using self-testing as a study strategy also reported that they did so to test themselves on how well they learned the material.⁵ Notably, re-reading was also associated with higher GPAs among the surveyed students; however, other studies have not found this association.⁵ These are just a few positive study strategies that can be utilized by college students to promote self-regulated learning.^3,5 Having students reflect back on their study habits to determine what works and what doesn’t work is one strategy that can be utilized during remediation programs.^3,5 Some students are not accustomed to having to study and this creates a challenge when they get to college.¹ They may go from previously not having to study (at all!) to learning how to study.^1,2 An individualized remediation program is an opportunity to address some of these challenges.^1,2

Course repetition is used by many schools and colleges of pharmacy.² This approach to remediation focuses on academic competence by requiring students to simply repeat the course they have failed.^1,2 Course repetition aims to re-expose students to the same course material in the hopes they achieve higher performance.^1,2 Some programs allow students to continue in the program and repeat the course once it is offered again while other programs require that the course be successfully passed prior to moving forward within the program.^1,2 The later strategy often involves students sitting out for a year and this obviously has significant financial implications.^1,2 Course repetition is not individualized and often does not address the underlying issues that contributed to the failure. This approach to remediation fails to address the non-cognitive barriers to students’ success.

Programs that focus on individualized remediation allow students to actively participate in the remediation process and help set them up for success both in the classroom and their careers. In an individualized remediation program, students must identify their barriers (with guidance) and engage in self-reflection. Such programs help students develop plans to address their identified barriers. Additionally, individualized programs should aim to promote self-regulated learning behaviors by giving students experience creating goals, formulating learning tasks, and developing time-management skills. Individualized programs should also focus on helping students develop new study strategies such as self-testing and summarizing. By identifying and addressing the non-cognitive barriers that often cause students to fail, remediation programs can foster the development of self-directed behaviors that enable students to be successful in subsequent coursework … and life.

 

References:

1. David M, Fuller S, Hritcko P, et al. A Review of Remediation Programs in Pharmacy and Other Health Professions. Am J Pharm Educ [Internet]. 2010;74(2): Article 25.
2. Sansgiry S, Bhosle M, Sail K. Factors That Affect Academic Performance Among Pharmacy Students. Am J Pharm Educ [Internet]. 2006;70(5): Article 105.
3. McKeirnan K, Colorafi K, Kim A, et al. Study Behaviors Associated with Student Pharmacists’ Academic Success in an Active Classroom Pharmacy Curriculum. Am J Pharm Educ [Internet]. 2020;84(7):Article 7695
4. Britton B, Tesser A. Effects of Time-Management Practices on College Grades. J Educ Psychol [Internet]. 1991 [cited 2021 Jan 10];83(3):401-10.
5. Hartwig M, Dunlosky. Study strategies of college students: Are self-testing and scheduling related to achievement? Psychon Bull Rev [Internet]. 2012;19:126-34.

Mindfully Teaching and Learning

by Alisha S. Nicks, PharmD, PGY1 Pharmacy Practice Resident, G.V. (Sonny) Montgomery VA Medical Center

Before you clicked on the link to read this blog post, were you thinking about your to-do list? Or perhaps you were reflecting on a decision you made last week that you wish you could change? While our physical bodies are in the present moment, our minds often drift to thinking about mistakes of the past or planning for the future. Mindfulness, as defined by Jon Kabat-Zinn, Ph.D., is purposefully paying attention to the present moment in a non-judgmental manner.¹ Psychological and educational research shows that Dr. Kabat-Zinn’s mindfulness-based stress reduction (MBSR) program teaches individuals the power of paying attention to the present, models a conscious way of living, and provides healthy coping mechanisms for stress.²

MBSR is an eight-week training program centered around formal mindfulness practices, including mindful attention to breathing through sitting meditation and body awareness through Hatha yoga. MBSR also teaches participants to informally practice mindfulness through attentiveness in everyday activities, such as walking, talking, driving, and eating. In a systematic review assessing the efficacy of MBSR on stress management, MBSR interventions positively impacted both psychological and physiological measures of stress. Although an eight-week MBSR course may not be feasible for most schools to provide to their students, faculty, and staff, instructors can create “mindfulness moments” throughout the curriculum. Including mindfulness techniques in one’s instruction can motivate students to become more self-aware as well as build positive relationships.³ The results of one study in an early childhood education program showed that students who participated in an “awareness of the breath” activity at school also engaged in mindfulness-based conflict resolution strategies with friends and family.⁴

What if instead of telling students to focus on some end result, we encouraged them to focus on the actual pursuit itself?
– Amy Burke: Mindfulness in Education, Learning from the Inside Out

Health professional students often struggle with stress from challenging coursework, extracurricular activities, and personal responsibilities. Moreover, they often struggle with internal pressure to meet personal expectations of perfection and a fear of failure.⁵ After a case discussion I facilitated with students at the University of Mississippi School of Pharmacy, a few third-year students asked me for advice on how to prepare for pharmacy licensure exam. They seemed anxious about an exam that is more than 18 months away. After sharing a few recommendations, I encouraged them to focus on the materials they are learning now, taking action now to really understand what they are currently learning, rather than worrying about the exam or life post-graduation. Mindfulness is not turning a blind eye to present difficulties or future responsibilities. It involves conscious acceptance and intentional responses to current circumstances.

Mindfulness begins by observing one’s internal state (emotions and thoughts) as well as external environment (sensations) but without judgment or automatically reacting.  The five facets of mindfulness are described below (see Table 1).

Table 1: Five Facet Mindfulness Questionnaire⁷

Scales	Definition	Sample Item
Observe	Tendency to observe, notice, or attend to internal and external phenomena	I intentionally stay aware of my feelings.
Describe	Tendency to describe or label sensations, perceptions, thoughts, emotions, etc. with words	My natural tendency is to put my experiences into words.
Accept without judgment	Tendency to accept without making judgments or evaluations	I disapprove of myself when I have irrational ideas.
Nonreactive	Tendency not to react to one’s experience	I watch my feelings without getting lost in them.
Act with Awareness	Tendency to focus undivided attention on the current activity or avoiding automatic pilot; concentration	I easily get lost in my thoughts and feelings.

Instructors play a critical role in setting the tone of the learning environment and engaging students in the learning process. In a commentary exploring the lessons that teachers can learn from actors about living in the present moment, Daniel R. Malcom reflects on his lectures as a junior faculty member that focused on information delivery but neglected to fully engage learners.⁶ Dr. Malcom suggests being reliably in the present moment with learners as a key strategy for creating learner-centered environments and strengthening relationships between teacher and learner.

In a study conducted in Ireland regarding stress and stress management, student pharmacists were interviewed in a series of focus groups. Students who did not have strong and regular interactions with instructors felt a lack connectedness and this hindered their access to support systems and resources. Focus group participants recommended integrating mindfulness practices into the curriculum instead of didactic lectures on stress management to encourage active learning. In a clinical trial evaluating the impact of a four-week mindfulness course, student pharmacists experienced improvements in stress and became more skillful at the observing facet of mindfulness.⁷ Although additional evidence is needed to assess the impact of mindfulness on learning outcomes, techniques listed in Table 2 have shown to reduce stress and increase wellbeing. 

Table 2: Mindfulness Strategies and Practices for Educational Settings³

 

Setting/ Audience	Learning Objective	Mindfulness Strategies	Example of Practice	Benefits
Educators	Understand how to treat students with respect, fairness, and non-judgment	Cultivate self-awareness to identify preconceived notions or judgments that may stand in the way of treating students equitably	Practice moment-by-moment awareness of your thoughts and feelings by focusing on bodily sensations or taking deep breaths	Educators become more reflective, thoughtful, and culturally responsive in their language and actions toward students
Educators	Proactively manage the stresses of teaching and leading	Educators recognize triggers of their own stress and emotional reactions	Pause and take several deep breaths whenever a strong emotional reaction is triggered, to cultivate self-awareness rather than reactivity	Educators can manage stress, personal hardship, and trauma
Elementary & Secondary Education Students	Cultivate focus and attention	Integrate a deliberate pause to help students settle their minds and bodies in preparation for learning	Begin class with a “mindful moment” where students are guided through either (1) sitting and noticing their bodily sensations or (2) slow and deliberate movement (e.g., deep breaths with movements such as raising their arms over their heads and down to their sides in repetition)	Students learn to slow down, quiet down their nervous systems, and check in with themselves and any feelings or thoughts that might need attention as they get ready to learn
Elementary & Secondary Education Student	Increase academic confidence, reduce stress, and manage anxiety	Use mindful breathing to help students self-regulate when under stress	Teach students to recognize the symptoms of anxiety in the body (e.g., elevated heart rate, fast-moving thoughts) and to practice taking several slow, deep, and full breaths to signal ease to the body when they feel these symptoms	Students are better able to manage stress and anxiety by practicing a healthy behavior

The highest function of education is to bring about an integrated individual who is capable of dealing with life as a whole.
– Krishnamurti: Education and the Significance of Life

Being more aware of the present moment is a healthy coping mechanism as students navigate virtual learning environments which have been required due to restrictions during the COVID-19 pandemic. Studies show that virtual learning settings create new challenges for students and many feel less confident in their ability to master the material.⁸ Mindfulness can serve as a constructive way to support student and faculty resilience during these unprecedented times. Incorporating techniques such as mindful breathing before exams and mindful appreciation of positive moments during instructional activities can inspire perseverance and reduce burnout. While mindfulness has proven to enhance emotional regulation, some people do not respond to these practices as there is no one-size-fits-all wellness model. Therefore, institutions should provide mindfulness activities and courses as an option to help teachers and learners become skilled in responding intentionally rather than reacting irrationally to life’s circumstances. Mindfulness can create meaningful connections between students and teachers both in and out of the classroom.

References

1. O’Driscoll M, Byrne S, Kelly M, et al. A Thematic Analysis of Pharmacy Students’ Experiences of the Undergraduate Pharmacy Degree in Ireland and the Role of Mindfulness. AJPE [Internet]. 2019;83:6457.
2. Sharma M, Rush SE. Mindfulness-Based Stress Reduction as a Stress Management Intervention for Healthy Individuals. J Evid Based Complementary Altern Med [Internet]. 2014;19:271–286.
3. Browning A. Mindfulness in Education: An Approach to Cultivating Self-Awareness That Can Bolster Kids' Learning. WestEd(2020). Accessed: February 6, 2021.
4. Ager k, Albrecht NJ, and Cohen M. Mindfulness in Schools Research Project: Exploring Students’ Perspectives of Mindfulness—What are students’ perspectives of learning mindfulness practices at school? Psychology 2015; 6 (7): 896-914.
5. Murry LT, Witry MJ. ‘Wasting Time Inside My Mind’: Exploring Student Pharmacists’ Perspectives on Engaging in Mindfulness Oriented Meditation Using Concepts From Education Research. Pharmacy Education [Internet]. 2020;204–214. Accessed: February 6, 2021.
6. Malcom DR. Being Present With Learners and the Deeper Lessons From Acting. Am J Pharm Educ [Internet]. 2020; 84: Article 8062.
7. O’Driscoll M, Sahm LJ, Byrne H, et al. Impact of a Mindfulness-Based Intervention on Undergraduate Pharmacy Students’ Stress and Distress: Quantitative Results of a Mixed-Methods Study. Curr Pharm Teach Learn [Internet]. 2019;11:876–887.
8. Schlesselman LS, Cain J, DiVall M. Improving and Restoring the Well-Being and Resilience of Pharmacy Students During a Pandemic. Am J Pharm Educ [Internet]. 2020; 84: Article 8144.

Preparing Health Professions Students for Telehealth

by Madeline Wolters, PharmD, PGY1 Pharmacy Practice Resident, G.V. (Sonny) Montgomery VA Medical Center

Telehealth is a rapidly growing modality for delivering health care services.  It can improve access to care, enhance quality and frequency of visits, and reduce costs.¹ Telehealth can be administered in many ways but the most common are video conferencing, telephonic communication, and remote patient monitoring.² The World Health Organization defines telehealth as:

The delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies for the exchange of valid information for diagnosis, treatment and prevention of disease and injuries, research and evaluation, and for the continuing education of health care providers, all in the interests of advancing the health of individuals and their communities.¹

As a pharmacist who trained in Mississippi, I did receive some instruction regarding the delivery of teleservices since access to transportation and the geographic availability of providers are common barriers in rural areas. However, with the arrival of the COVID-19 pandemic, we have seen traditional healthcare delivery restructured, moving away from face-to-face visits to telehealth services.

Technology and virtual services are here to stay, so we must provide instruction to students of all health professions about telehealth.  Here are the key elements that should be addressed in the core curriculum:³

1. Professionalism in a virtual age
2. Appropriate use of mobile health information and technology
3. Communication skills in online environments

In my own experience as a pharmacy student, I was taught these skills through a series of lectures, training courses, professional assemblies, and practice-based experiences. There are countless ways to present and teach about telehealth to fit the needs of learners. Because health professionals must earn and maintain the trust of the people we serve, as practitioners and educators, it is our responsibility to uphold professionalism and teach those principles to the next generation. It is also important to remember that what may seem obvious to an “old pro” might not be intuitive for a novice learner.

Most health profession programs begin with instruction that is primarily classroom-based.  Early in the curriculum is the ideal time to introduce the concepts of professionalism, appropriate use of technology and patient information, and the foundations of successful communication. As learners progress and enter into the experiential or clinical portion of their education, these concepts can be expanded and reinforced through hands-on experiences and practice. In an interview on how telehealth is transforming healthcare during the COVID-19 pandemic, Dr. Richard Van Eck of the University of North Dakota School of Medicine and Health Sciences stressed the importance of putting learners in simulated and realistic environments to truly understand telehealth.⁴ He stated,” you can do all the didactics you want, but until you're in the midst of doing it [telehealth], you don't really understand what's involved”.⁴

The American Medical Association released a “Telehealth Visit Etiquette Checklist” which provides valuable tips that can be applied to all patient care interactions.⁵ A telehealth visit should mimic an in-person appointment as close as possible. If the patient will be using video-capable technology during an encounter, the student should be aware to dress in the same level of professional attire, including a white coat, if applicable.⁵ Just as a practitioner would prepare an exam room, the student should prepare their virtual environment.⁵ The necessary technology should be accessible, the electronic health record (EHR) should be open, and the background scenery and lighting should be appropriate for the encounter. For telephone calls, make sure the student is either using a telephone owned by the healthcare facility or an application that scrambles or protects the student’s personal phone number. The teaching institution should be able to provide any space and equipment that the learner should need.

Learners should also be taught how to appropriately use and share health information and technology. The patient’s confidentiality should be ensured by managing the appointment in a private space and following HIPAA (Health Insurance Portability and Accountability Act) requirements. It is also important to teach what health information may or may not be sent through non-secure platforms like email or text messages. If written patient information is to be shared with other providers, students need to be taught how to encrypt emails or navigate messaging through the EHR. The student should know who to contact if he or she feels the patient’s privacy may have been breached. Additionally, students need to know how and when they are permitted to use personal and institutional issued technology. Specific policies may vary from site to site, but ultimately a student should be taught a general guide to appropriate technology use.

Lastly, communication is at the center of all patient interactions. From the very beginning, students should be taught how to engage in motivational interviewing, deliver patient education, and manage patient exams. However, adjustments may be needed when using a virtual platform. Dr. Van Eck noted that many students inexperienced with telehealth report that they have difficulty understanding what the patient is saying and they are not able to read body language.  Also, patients are more likely to say things like “Who are you?”⁴ It may be helpful to run through example scenarios to boost a student’s confidence with operating the technology and practicing how to interact with a patient. Since you are not in control of the patient’s environment, students should be taught strategies to redirect the patient’s attention. For example, if the TV is loud in the background, the student should have the practice and self-assurance to say, “I would like us to have the best visit possible. Would you be able to turn the TV off for now?²” The importance of speaking clearly and deliberately is heightened during a telehealth encounter. Students may need to make frequent pauses to allow for transmission delays.⁵ During video-capable visits, students must be taught how to maintain eye contact and use non-verbal cues as much as possible.⁵ For telephone calls, students need to learn how to introduce the encounter and explain why they are calling the patient. Since hearing will be the only sensory mechanism, students will need to learn how to use inflection with their voice and implement continuous confirmation with empathetic language.⁵ As with all patient encounters, students must know to verbalize and clarify the next steps and patient care plan. The "teach-back method" is still helpful to confirm a patient's understanding.

A recently published systematic review examined the integration of telehealth topics in health professions curricula.¹ The review noted that telehealth concepts are multifaceted and can be overwhelming for students without foundational knowledge and guided experiences.¹ However, instruction provided to students using different modalities (online delivery, clinical experiences, simulations, and face-to-face instruction) all improved student satisfaction and self-confidence with telehealth encounters.¹ Ultimately, multiple exposures to these concepts throughout the curriculum are essential for the next generation of health care works to become competent and confident using telehealth technologies.¹ As telehealth becomes commonplace, telehealth experiences must become a required component of every curriculum.

References

1. Chike-Harris KE, Durham C, Logan A, et al. Integration of Telehealth Education Into the Health Care Provider Curriculum: A Review. Telemedicine and E-Health [Internet], Published online April 3, 2020. https://doi.org/10.1089/tmj.2019.0261
2. Telehealth. American Pharmacists Association. Accessed February 05, 2021.
3. Aungst TD. Integrating mHealth and Mobile Technology Education Into the Pharmacy Curriculum. Am J Pharm Educ 2014;78: Article 78119.
4. Educators discuss integrating telehealth in student curriculum. American Medical Association, January 2021. Accessed February 12, 2021.
5. Telehealth Visit Etiquette Checklist. American Medical Association. April 2020.  Accessed Feb 15, 2020.

Benefits and Concerns with Educational Handovers

by Elizabeth Sykes, PharmD, PGY1 Pharmacy Practice Resident, Baptist Memorial Hospital-North Mississippi

Most health professional are familiar with hand-off or handover reports which occur at the beginning and end of a shift, when a patient is transferred from one unit to another, or when there is a change in a patient’s condition.  Handovers occur between nurse to nurse, nurse to physician, nurse to a pharmacist, and even from pharmacist to pharmacist.  These handovers occur within and across all disciplines so that patients receive the most appropriate care.  Handovers are very important for patient safety and continuity of care.  It has been shown that poor or inaccurate handovers may lead to delayed and inappropriate treatment, medical errors, and inaccurate assessments and diagnoses.  Perhaps an educational handover between teachers and educational programs can have similar benefits?

An educational handover, the sharing of appropriate learner performance information between teachers and preceptors to support the learner’s ongoing training and development, is a potentially valuable way to support learning over time.¹ In this way, feedback about performance problems can be made based on multiple, longitudinal observations.¹  Benefits of educational handovers may include improved learning through more tailored feedback and support, improvements in a supervisor’s ability to assess learners related to specific competencies, increased assessor accountability, and improved patient safety through the early identification of weak or problematic behaviors.² 

However, not everyone agrees that educational handovers are useful, valuable, or appropriate.  Despite the potential benefits of an educational handover, there are some concerns associated with it.  Informing future teachers or preceptors about potential performance problems may introduce bias into the assessment process and it may lead instructors to treat some learners differently or label them. This could then lead to both the learner and the assessor acting in ways and viewing the learner’s performance through a lens that lines up with prior assessments. In addition, an educational handover may violate a learner’s right to confidentiality and privacy.²

A study performed at McGill University examined the potential bias from an educational handover on workplace-based assessment scores in medical education.  When given handover reports mentioning weaknesses, the hypothesis was that supervisors would provide lower assessment scores and more negative comments than those who did not receive learner reports.  This was a mixed-methods randomized, controlled, experimental study.  All participants viewed two videos of a simulated resident-patient encounter and then assessed the residents’ performance using the mini-Clinical Evaluation Exercise (mini-CEX).  The two videos viewed them in the same order.    The participants were randomized into three groups that differed based on the educational handover condition: no education handover report (control group), educational handover report indicating weaknesses in medical expertise, and educational handover report showing weaknesses in communication.  Participants had to complete a questionnaire that included questions about basic demographic variables (age, gender), clinical and educational variables (specialty, years of experience supervising, years of experience assessing), and mindset.  An analysis of variance was used to compare mean scores, percentages of negative comments, comments focusing on medical expertise, and comments focusing on communication across experimental groups.³

Seventy-two supervisors completed the study with 21 participants in the control group, 21 in the educational handover group indicating weakness in medical expertise, and 30 participants in the group receiving the educational handover indicating communication weaknesses.  No differences were detected in demographic characteristics, rater experience, or mindset across the three groups.  There was no effect of the handover report on assessment scores (F(2, 69) = 0.31, P = 0.74) or percentage of negative comments (F(2, 60) = 0.33, P = 0.72).  However, the participants who received a report indicating communication weakness generated a higher percentage of comments about communication skills than the control group (63% vs. 50%), P = 0.03).³ 

	Control Group (no handover report) n = 21	Medical Expertise Weakness Group n = 21	Communication Weakness Group n = 30
Mean score for Video 1	5.6 (4.9-6.2)	5 (4.2-5.8)	4.9 (4.5-5.4)
Mean score for Video 2	4.8 (4.2-5.4)	5 (4.4-5.6)	4.9 (4.4-5.5)
Mean score for both videos	5.2 (4.6-5.7)	5 (4.4-5.6)	4.9 (4.5-5.3)

This study suggests that an educational handover can lead to more targeted feedback without influencing scores.  Further studies are needed to examine the influence of reports of various performance levels, areas of weakness, and learner behaviors.³ 

Competency-based medical education (CBME) is becoming the cornerstone of medical education programs.  But the transition from undergraduate medical education to graduate medical education is not a smooth process.  It has been suggested that an educational handover at the end of medical school might help with this transition and would help students become more prepared to care of patients.  The Medical Student Performance Evaluation (MSPE) is submitted in early October each year, and there is very little information provided about the final year of medical school.  In April 2018, the American Medical Association’s Accelerating Change in Medical Education consortium developed five recommendations for developing an educational handover that would be provided to residency programs at the end of medical school.  The 5 recommendations are: (1) The purpose of the educational handover is to provide performance data to guide continued improvement in the learner’s ability and performance, (2) the process used to create an education handover should be philosophically and practically aligned with the learner’s continuous improvement, (3) the educational handover should be learner-driven with a focus on individualized learning plans that are co-produced by the learner and his/her coach or advisor, (4) the transfer of information within an educational handover should be done in a standardized format, and (5) together, medical schools and residency programs must invest inadequate infrastructure to support learner improvement.

Despite these recommendations, there are still challenges with educational handovers between educational programs.  Medical schools would have to develop a curriculum for educational handovers that focuses on assessing individuals' performance.  These should include authentic workplace-based assessments coupled with a formative feedback process.  The Family Educational Rights and Privacy Act (FERPA) regulations apply to educational handovers.  Even though medical school graduates transfer from one educational setting to another, FERPA protects the privacy of learners in both contexts.  Thus, any educational handover will need to comply with FERPA.  Medical schools would need to develop a standardized process to support meaningful communication.  Lastly, creating an educational handover should support a successful transition.  Gathering assessments and adjusting each trainee’s experience during the first few months of a residency would be challenging.  But to be truly effective, the content of the educational handover should be used to implement learner-specific curricular modifications.⁵

Health profession educational programs should have some form of handover, but many of them don’t, or they may lack the appropriate framework.⁴  I believe educational handovers would help prepare learners (students and residents) for the next step in their career.  It can provide future preceptors and employers with information about the learners’ past performance and how they should tailor experiences to help address potential weaknesses.  Educational handovers may lead to some bias, but I believe with appropriate training, preceptors and residency program directors can learn to appropriately use this information to address learner needs. 

References 

1. Guidelines for Educational Handover in Competence by Design. Royal College Committee on Specialty Education: 2018 May 
2. (Gumuchian ST, Pal NE, Young M, Danoff D, Plotnick LH, Cummings BA, et al. Learner handover: Perspectives and recommendations from the front-line. Perspect Med Educ. 2020;9:294-301. 
3. Dory V, Danoff D, Plotnick LH, Cummings BA, Gomez-Garibello C, Pal NE, et al. Does Education Handover Influence Subsequent Assessment? Acad Med. 2021;96:118-125.
4. Gordon M, Hill E, Stojan JN, Daniel M. Educational Interventions to Improve Handover in Health Care:  An Updated Systematic Review. Acad Med. 2018;93(8):1234-1244. 
5. Morgan HK, Mejicano GC, Skochelak S, Lomis K, Hawkins R, Tunkel AR, et al. A Responsible Educational Handover: Improving Communication to Improve Learning. Acad Med. 2020;95:194-199.