Do NAPLEX or MPJE Preparatory Courses Improve Pass Rates?

by Jacey Gossett, PharmD, PGY1 Pharmacy Practice Resident, Baptist Memorial Hospital North Mississippi

As a recent pharmacy school graduate, I took the National Pharmacy Licensure Exam (NAPLEX) and the Multi-state Pharmacy Jurisprudence Exam (MPJE). A hot topic of conversation among my classmates is the pass rates on these exams.  Being a “good” student who earned A’s and B’s throughout pharmacy school, I expected the NAPLEX to be something that I would be able to easily pass — boy, was I wrong! To my, my friends, and my family’s surprise, I received a “FAIL” on my first NAPLEX attempt. The word FAIL staring me in the face that day last June was a major kick in the gut. Since I passed my MPJE shortly after graduation in May, I thought the NAPLEX would be a similar outcome. Luckily, in July, I received a “PASS” on my second NAPLEX attempt.

It was shocking to me, but I was not the only one from my graduating class that had to retake the NAPLEX.  How in the world had we made our way through pharmacy school just to receive a big fat “FAIL” when taking our boards? Our school and the professors certainly did everything they could to push us toward success on our boards. We had a year-long NAPLEX prep course that ran concurrently with our advanced pharmacy practice experiences (APPE).  We were given various assignments to “ensure” that we were preparing ourselves for these challenging exams. We had multiple practice exams to give us experience. In December of my P4 year, I scored 69 on my first practice NAPLEX. Although this is not a “passing” score, I felt I was on the right track as I was just getting deeper into my studying. In the spring of my P4 year, I took another practice exam and brought my score up to 73.  I was improving but I recognized that had more studying to do.

There have been multiple studies published in recent years examining factors that might help students be successful on their board exams. One cross-sectional study sought to describe the characteristics of NAPLEX preparation programs currently offered by schools of pharmacy and the correlation between program characteristics and first-attempt pass rates. Fifty-eight Pharm D programs completed an online survey about their NAPLEX preparation programs. A majority (86%) of schools indicated they offered a NAPLEX prep program. But offering a NAPLEX prep program was not associated with higher first-attempt pass rates. Some concerns raised by the authors of the paper included student workload (e.g., balancing the demands of a prep program during APPEs) and the faculty workload associated with delivering these programs.¹

In a retrospective study, investigators compared NAPLEX scores (n=150) to several factors that might predict performance. The investigators found that the NAPLEX score was most strongly correlated with pharmacy GPA (r=0.66) and Pre-NAPLEX score (r=0.45) but also race/ethnicity, Pharmacy College Admission Test (PCAT) composite score and section scores, undergraduate GPA, undergraduate science GPA, and on-time graduation.²

In another retrospective study, the correlation between preparatory testing and other factors were compared to performance on the MPJE. This analysis showed that the Pre-MPJE scores failed to predict whether a student would pass the MPJE but a student’s performance in the pharmacy law course did.³

As you can see from the results of these studies, there is no clear correlation between prep courses and the odds of passing either the NAPLEX or MPJE. Therefore, simply having such courses available does not ensure success.  Thus, it is truly up to individuals to ensure that they are using prep courses, practice exams, and other resources to fully prepare themselves. It is difficult to find a quick and easy fix to improving first-time pass rates for board exams. There are many “tips and tricks” on the Internet. These “tips and tricks” seem to be consistent with things students have likely heard during pharmacy school – things like having a study plan, taking practice tests, creating self-testing materials, working with a study group, getting plenty of sleep, and not cramming.

Spaced repetition is a study technique that involves reviewing and recalling information at optimal spacing intervals until that information is deeply learned. This technique has students review materials, repeatedly, over a long period of time. Research has clearly shown that spacing out repeated encounters with material over time provides superior long-term retention.  Self-testing coupled with spaced repetition amplifies the benefits.⁴

As I’ve learned, it’s important to study and prepare for the MPJE and NAPLEX well in advance. I was able to pass my MPJE solely on the material learned during our pharmacy law course and our professor was very clear about how hard and tricky the exam could be. I was able to pass my NAPLEX on the second try by realizing my areas of weakness. I used the same study strategies, but by being more self-aware of my weaknesses I was better prepared for my second attempt. Perhaps the best approach is to help students figure out what they do not know and then encourage them to use effective evidence-based study techniques.

There are several evidence-based learning and studying techniques, like spaced repetition, that can be used to enhance a student’s recall of information, but it is truly up to the student to “take the bull by the horns” and get the studying done. Students need to know very early in their pharmacy school journey just how hard these exams can be.

References

1. Fiano K, Attarabeen O, Augustine J, et al. Association between Naplex Preparation Program Characteristics and First-Time Pass Rates. Am J Pharm Education 2022; 86(6): Article 8760.
2. Chisholm-Burns M, Spivey C, Byrd D, McDonough S, Phelps S. Examining the Association between the NAPLEX, Pre-NAPLEX, and Pre- and Post-admission Factors. Am J Pharm Education 2017; 81(5): Article 86.
3. Havrda D, Hall E, Spivey C, et al. Examining Preparatory Testing and Other Factors Associated With Performance on the Multistate Pharmacy Jurisprudence Examination. Am J Pharm Education 2022; 86(7): Article 8774.
4. Kang S. Spaced Repetition Promotes Efficient and Effective Learning. Policy Insights from the Behavioral and Brain Sciences(PIBBS) 2016; 3 (1): 12-19.

Resilience Training in Healthcare to Meet the Needs of Learners and Practitioners

by Lori Emory, PharmD, PGY1 Pharmacy Practice Resident, Mississippi State Department of Health Pharmacy 

Resilience is the ability to “bounce back” from challenges — to learn and grow from setbacks. An individual’s resilience is built up over the course of their life and is shaped by their actions taken prior to, during, and after difficult and often stressful situations.^1,2  

Stress is a natural response by the body to physical and mental challenges. While stress can be a motivating factor that prompts us to try new things as we face new challenges, too much stress over a prolonged period of time is unhealthy and can lead to anxiety and burnout. Anxiety is an overreaction to stress experienced from performing “ordinary” daily activities which can lead to significant impairment. Burnout is a diminished ability to respond to stressors and can lead to anhedonia and depression.^1,3 

In healthcare, providers are responsible for the well-being of numerous patients. Chronic stress – often unavoidable in healthcare – puts health professionals at a higher risk for developing anxiety or burnout, which, in turn, results in less than optimal care and increases the risk of errors and poor patient outcomes.^1,3,4 Thus, resilience strategies, often techniques learned from previous stressful situations, help providers meet new challenges.⁵ 

There are conflicting feelings about resiliency training in healthcare.  Some have argued that resiliency training focuses too much on individuals by teaching them how to adapt to and cope with the ever-growing demands of the profession while letting institutions off the hook by failing to address poor working conditions, such as chronic understaffing and addressing hindrances.^2,4 While data supporting resilience training programs in healthcare are limited, results show participants generally appreciate learning about the tools they can use to become more resilient and like being part of a community where they can share experiences with fellow health professionals.^1,2 

Although self-care has been around for many years, resiliency training was introduced to me as a formal concept while applying for residency. Many residencies now offer resilience training as a required component of their program structure. Given that most pharmacy jobs provide little to no social support for new practitioners, I believe having resiliency training incorporated into the residency experience can be very beneficial. 

Several studies have examined the elements of resiliency training that participants generally believe are needed for it to be most beneficial:  

1. Focusing on the experience of health professionals and students is vital in a successful program. ^1,2,4,5,6  Healthcare workers are exposed to difficult human experiences while working with serious injuries, illnesses, and even death.^1,2,4,6 Participants in resiliency training programs discussed the need for training to be guided by another health professional with a shared understanding of the many challenges that come with working in healthcare.^1,2,6 

2. Sharing experiences in a positive and non-judgmental manner builds community among healthcare providers who often feel isolated.^1,2  Many providers report feeling isolated and a desire to participate in a community of peer support.¹ Program participants reported small group discussions allowed them to recognize their own behaviors better and learn from others’ real-world experiences.^1,6 Participants often report preferring voluntary attendance at these sessions as they felt it helped ensure that all participants would come in with an open mind about sharing experiences and learning to grow from difficult situations.^2,6  

3. Following up with participants helps turn new skills into daily practiced habits. While the initial training session(s) were often considered to be helpful by participants, programs that included follow-up reflection, allow participants to expand on the skills they learned and think about how they could use those skills in their daily lives. ^1,2,6 

4. All health professionals can benefit from these types of programs, regardless of their current level of experience. Even students at the beginning of their training are exposed to the difficult realities of a career working in healthcare, and serving people who are experiencing some of the greatest stress in their lives.^2,5 Promoting resiliency training programs during early didactic coursework or introductory practice experiences may provide key life-long skills that participants will benefit from long-term as they progress through training and into their professional lives.^2,6 Students particularly report benefits from training programs that are structured to provide insight into stressors that they may experience, during their training and in practice.^2,3,6  

Offering resiliency training led by experienced faculty members should be considered at all educational institutions that are preparing the next generation of health professionals. Resiliency training programs should be voluntary, such as course electives or extracurricular meetings, where all students can participate without feeling the need to give up other interests. Placing participants in small groups of 8-10 with a mix of learners at various levels and faculty with guided discussion prompts can foster natural mentorships and a sense of community within groups. Providing this kind of support early and often throughout the professional degree program can provide a safe space where genuine conversations can take place. Regularly obtaining feedback from participants is important to tailor these programs to meet the needs of students and faculty based on schedules and topics of interest.  

References: 

1. Epstein RM, Krasner MS. Physician Resilience: What It Means, Why It Matters, and How to Promote It. Academic Medicine 2013: 88(3):301-303.
2. Johnson J, Simms-Ellis R, Janes G, et al. Can we prepare healthcare professionals and students for involvement in stressful healthcare events? A mixed-methods evaluation of a resilience training intervention. BMC Health Serv Res 2020; 20: Article number 1094.
3. Fares J, Al Tabosh H, Stress AH, et al. Burnout and coping strategies in Preclinical Medical Students. N Am J Med Sci 2016; 8 (2):75-81.
4. Murthy VH. Confronting health worker burnout and well-being. N Eng J Med 2022;387(7):577–9.
5. Kunzler AM, Helmreich I, König J, et al. Psychological interventions to foster resilience in healthcare students. Cochrane Database of Systematic Reviews 2020, Issue 7. Art. No.: CD013684:7-43.
6. Bird A, Tomescu O, Oyola S, Houpy J, Anderson I, Pincavage A.A curriculum to teach resilience skills to medical students during clinical training. MedEdPORTAL 2020;16:10975.

Teaching Health Professional Students to Manage Patient Bias

by Allison Wadlow, PharmD, PGY1 Pharmacy Practice Resident, G.V. (Sonny) Montgomery VA Medical Center

Health disparities based on race have been a long-standing problem in the United States. Much light was shed on this matter in recent years prompted by numerous murders of minorities and the Black Lives Matter advocacy movement. It is well known that racial disparities exist within the healthcare system of this country. Even when patients have comparable incomes, similar education, and live in the same geographic location, minority populations sadly do not always receive the same standards of care as their white counterparts.¹

Many health professions programs offer training to learners on addressing bias and improving care delivery for all patients. Over the past decade, some popular training programs include Trauma Informed Care, Adverse Childhood Experiences training, and Multicultural Training. These programs have included materials to serve patients better, with the primary focus on patients, not the providers of care.

Bias towards healthcare professionals from underrepresented and marginalized groups is, unfortunately, commonplace. A recent study “Harassment and Discrimination in Medical Training” found that most health professional trainees have experienced bias at some point in their career, most often based on their race, gender, or sexuality.² 

To address this problem, medical students and faculty at multiple institutions in Northern Virginia and Washington, DC created a training program for medical students to manage patient bias.³ The goal of this program was to prepare students for the biases they may encounter in practice.  The investigators created a simulation exercise whereby participants learned, in a safe environment, to navigate these situations.³

The simulations involved encounters with standardized patients. During the encounter, the student was instructed to tell the patient about a follow-up appointment that had been scheduled with a provider who had a foreign-sounding name. The standardized patient was instructed to either accept this appointment or request an alternative provider. The student was then instructed to use motivational interviewing techniques to understand where the request to change providers stemmed from. The possible reasons for the patient’s request were:

• An untreated hearing loss made it difficult for the patient to understand accents
• A belief that “foreigners shouldn’t be taking American jobs”
• A preference to receive treatment from a physician who shares a similar cultural background due to the patient’s previous experiences of intolerance or disrespect

During the encounter, it was up to the student to determine the root cause of the request. In some instances, the request might lead to better patient care, and it should be honored. For example, in the case of a patient with hearing loss, switching to a provider without an accent, the patient might be able to better distinguish words spoken during the encounter. Of course, if the provider did not have an accent, the student should provide reassurance to the patient.  However, in some cases, the request to change providers was racially motivated. Of course, a surname does not always indicate the race of the provider. The point of this training experience was to allow students the opportunity, in a safe and controlled environment, to navigate through a patient request that, on the surface, may be motivated by racial bias. The goal is to develop the skills to uncover the root of these requests, rather than assuming that racism is the problem.

At the beginning and end of the course, students were surveyed to measure their growth. After completing the activities, students reported an increase in their confidence to explore intentions and beliefs, navigate a conversation with a patient exhibiting bias, and use nonverbal skills to demonstrate empathy.³

Addressing bias directed towards providers is not taught in most health professions' curricula or residency training programs. I think health-professional programs should begin to include this type of training so that students and residents are better equipped to manage patient bias.

The first step would be to give students exposure to patients who express a bias toward certain (commonly marginalized) groups. Nearly all students who participated in this simulation exercise stated they were very uncomfortable, but that it was a positive experience to be able to interact in a safe environment. The students stated they felt more prepared to manage these situations with real patients in practice.  Because learners should never be intentionally required to interact with people who have racial animus, they would learn how to assess patients’ requests, how to act in the face of bias, and how to move forward with patients who are explicitly or implicitly biased towards them or others. 

The next step would be to teach learners how to address patients who have explicit biases based on race, gender identity, and sexual preferences. Students must first be taught how to approach patients and uncover the rationale behind their biases. Once the student has uncovered the reasoning for the bias, there will be times when patients are explicitly racist, and students will need to be prepared (mentally) to manage the situation. Do they simply ignore and endure the bias? Do they refer the patient to alternative providers? Do they know when it is safe to speak up and say something to the patient? While there are no easy answers to these questions, the least we could do is provide students the opportunity to experience this type of conversation with patients before having to deal with it in the real world.

As with the implementation of any new training, cost becomes a factor to consider. The cost of hiring standardized patients may not be feasible for many programs. One workaround is to have volunteers serve as standardized patients, which could include faculty, upperclassmen, or alumni. Another way to mitigate the costs could be to have recordings of patient encounters, pausing the recording at various times to discuss with the class what they are observing. This option could open a rich class discussion on the negative consequences of bias toward marginalized patients as well as providers.

Bias in healthcare is not something we can eliminate, but we can build our skills to recognize and mitigate it.  We can prepare our students to uncover and address the biases they may encounter, including the bias that patients may express toward them or their professional colleagues. It is not ethical to know a problem exists, one that can mentally harm our learners, and do nothing to prepare them. It’s time to talk about patient bias and teach health professional students how to manage it.

References:

1. Williams DR, Rucker TD. Understanding and addressing racial disparities in health care. Health Care Financ Rev 2000;21(4):75-90.
2. Fnais N, Soobiah C, Chen MH, et al. Harassment and discrimination in medical training: a systematic review and meta-analysis. Acad Med 2014;89(5):817-27.
3. Zewdie M, Duval M, Liu C, et al. Virtual Communication Across Differences: Development of a Workshop on Managing Patient Bias. Acad Med 2023;98(2):209-213.

Professional Identity Formation (PIF) in Health Professions Education: Doing is Different from Being

by Lauren C. McConnell, PharmD, PGY1 Pharmacy Practice Resident, Baptist North Mississippi Hospital

Professional identity formation, or PIF, is the process through which a person becomes a professional — typically from student to practicing professional. The progression of PIF is uniquely individualized and superimposed on each student’s personal identity, values, morals, and beliefs.¹ The goal of forming a professional identity is to develop a resilient sense of belonging within a health profession.² PIF goes beyond students acquiring knowledge (‘thinking’) and demonstrating professionalism (‘acting’) to support one’s perception of self (‘feeling’).

Professionalism, as defined by The White Paper on Pharmacy Student Professionalism, is “the active demonstration of the traits of a professional”.³ Health professions students are intrinsically and extrinsically motivated to join a professional community and are willing to uphold certain professional expectations, such as wearing a white coat, communicating respectfully, and being accountable.^4,5 However, acting like a professional and being a professional are two different phenomena.

Interrelationship Between Professional Identity and Professionalism

Professionalism and professional identity are distinct yet related concepts, which makes the fluid relationship between the two challenging to describe (see Figure 1). Professionalism is an outward display of the conduct of a professional, while a professional identity is the internal perception of one’s role as a professional.⁶ Professional traits and behaviors are crucial for PIF, as ‘acting’ like a professional encourages assimilation to that role.⁷ Similarly, self-awareness of a professional identity is essential for developing a professional demeanor. Several stepwise models exist that have attempted to describe this relationship. Acts of professionalism are observable signs which indicate the concurrent development of professional identity.⁶ Therefore, my professors at Auburn University and I recently proposed a model to illustrate the infinite and undirected interplay between PIF and professionalism, the Möbius Strip.⁷

Figure 1: Professionalism-Professional Identity Möbius Strip

According to Moseley et al., “as the internalization process of PIF occurs, outward professional behaviors are displayed, and as one chooses to behave as a professional, their sense of identity blossoms”.⁷ This model aligns with the proposal that the end goal of health professions education should not just focus on ‘doing’ but also on ‘being’.⁸ As with all educational goals, methods for teaching and evaluating progress are essential. The conundrum is how this fluid process can be measured and supported.

PIF-Friendly Pedagogy

Obtaining a professional identity is the desired outcome in health professions education, as it is the backbone of all decisions students will make as professionals.⁸ However, many students (and admittedly, myself included) fail to recognize themselves as professionals early in their health education journey. For this reason, PIF has long been an elusive target amongst health professions educators. Furthermore, PIF is a non-linear process, and each student progresses toward their professional identity at a different pace, which makes it challenging to foster and evaluate progress.⁹ For this reason, health professions educators should incorporate PIF-friendly teaching strategies into curricula.

PIF pedagogy is the practice of teaching, facilitating, and coaching students through their PIF journey — teaching methods that support the development of an identity that aligns with the values of their profession. Educators are a fundamental component of the student’s journey. The formation of a professional identity is influenced by external factors, such as curricula, learning environments, expectations, mentorship, and feedback.⁵ I distinctly remember key preceptors who created positive learning environments and served as role models that positively impacted how I perceived myself as a future pharmacist. Therefore, it is important for educators to foster relationships and create experiences that are meaningful to students, as PIF is facilitated, not taught.

Self-assessment and self-reflection are two PIF-friendly strategies that educators can use in curricula to help students become more aware of their professional strengths and weakness.¹⁰ The ability of the student to be self-aware of their presence and growth within a professional community increases PIF and creates a sense of belongingness.⁹ Other meaningful relationships outside the formal education environment (e.g., with preceptors, other health professionals, and patients) play a similar and equally important role. To me, there is no replacing the feeling you get the first time a patient mistakenly refers to you as a pharmacist or when a physician shows appreciation by stating ‘good catch.’ Through these interactions, students gain recognition for their place on the healthcare team. Situated learning theory suggests that “learning should take place in a setting the same as where the knowledge will be used”.¹¹ Therefore, it is no surprise that students report early introduction to their profession, direct interaction with patients, and frequent collaboration with other health professionals as key drivers of identity construction.¹²

Because educators are facilitators of PIF, structured evaluations (e.g., exams or performance-based assessments) are not helpful measures of student progression, particularly given that PIF does not occur at a single point in time. Experts recommend that assessments of PIF should occur longitudinally to ensure that the student’s professional identity is progressively developing over time.¹³ Unfortunately, there are no standardized methods for measuring PIF, and assessments rely on student understanding of who they are within a profession. I remember creating short- and long-term career goals as a first-year student pharmacist, thinking I knew exactly who I was and what pharmacy career path I wanted to pursue. But with each semester, I revisited these goals and was honestly embarrassed by what I thought I knew about who I wanted to be. 

In one study, investigators designed a Professional Self Identity Questionnaire (PSIQ) that attempts to measure the degree to which health professions students identify as a member of their profession.¹⁴ Building on this notion, faculty at Auburn University Harrison College of Pharmacy recently created a PIF instrument to encourage students to reflect on their professional identity. This instrument asks students to self-assess fourteen qualities/behaviors, such as confidence, knowledge, personality, professionalism, and communication.¹⁰ These PIF-friendly exercises, using a combination of self-assessment and self-reflection, attempt to measure what educators cannot see: how students see themselves in relation to their profession.

There are several other activities and instructional strategies that can be used to promote PIF, such as feedback, experiential education, co-curricular activities (e.g., health fairs), mentoring/role modeling, student well-being groups, and white coat ceremonies.^7,15 Of course, most professional curricula already incorporate many of these pedagogical methods, but require active effort by educators to intentionally foster PIF. Reflecting on my time as a student, I now know why I have always appreciated professors who were passionate about what they taught, preceptors who encouraged autonomous work, and mentors who led by example – they intentionally helped create my professional identity. Educators should continue to purposefully use and prioritize PIF-friendly pedagogical methods, particularly early in curricula, to support the process of professional identity formation amongst their students.

References

1. Cruess RL, Cruess SR, Steinert Y. Amending Miller's pyramid to include professional identity formation.Acad Med. 2016;91(2):180-5.
2. Kellar J and Austin Z. The only way round is through: Professional identity in pharmacy education and practice. Can Pharm J (Ott). 2022 Aug 13;155(5):238-240.
3. Roth MT and Zlatic TD. American College of Clinical Pharmacy. Development of student professionalism. Pharmacotherapy. 2009 Jun;29(6):749-756.
4. Holden M, Buck E, Clark M, Szauter K, Trumble J. Professional identity formation in medical education: The convergence of multiple domains. HEC Forum. 2012 Dec;24(4):245-255.
5. Findyartini A, Greviana N, Felaza E, et al. Professional identity formation of medical students: A mixed-methods study in a hierarchical and collectivist culture. BMC Med Educ. 2022 Jun 8;22(1):443.
6. Forouzadeh M, Kiani M, Bazmi S. Professionalism and its role in the formation of medical professional identity. Med J Islam Repub Iran. 2018;32(1):765-8.
7. Moseley LE, McConnell L, Garza KB, Ford CR. Exploring the evolution of professional identity formation in health professions education. New Dir Teach Learn. 2021 Dec 6;168:11-27.
8. Snell R, Fyfe S, Fyfe G, Blackwood D, Itsiopoulos C. Development of professional identity and professional socialisation in allied health students: A scoping review. Focus on Health Prof Educ. 2020 Apr 30;21(1):29-56.
9. Cruess SR, Cruess RL, Steinert Y. Supporting the development of a professional identity: General principles. Med Teach. 2019 Jun;41(6):641-9.
10. Ford CR, Astle KN, Kleppinger EL, Sewell J, Hutchison A, Garza KB.Developing a self-assessment instrument to evaluate practice-readiness among student pharmacists. New Dir Teach Learn. 2021 Dec 6;168:133-145.
11. Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham, ID. Use of communities of practice in business and health care sectors: A systematic review. Implement Sci. 2009 May 17;4:27.
12. Wald HS. Professional identity (trans)formation in medical education: Reflection, relationship, resilience.Acad Med. 2015 Jun;90(6):701–6.
13. Garza KB,Moseley LE, Ford CR.Assessment of professional identity formation: Challenges and opportunities.New Dir Teach Learn. 2021 Dec 6;168:147-151.
14. Crossley J and Vivekananda-Schmidt P. The development and evaluation of a Professional Self Identity Questionnaire to measure evolving professional self-identity in health and social care students. Med Teach. 2021 Dec;31(12):e603-7.
15. Chandran L, Iuli RJ, Strano-Paul L, Post SG. Developing "a Way of Being": Deliberate approaches to professional identity formation in medical education.Acad Psychiatry. 2019 Oct;43(5):521–7.

Achieving the Promise of Authentic Workplace-Based Assessments

by Sophie Durham, PharmD, PGY1 Community Pharmacy Practice Resident, Mississippi State Department of Health Pharmacy

Workplace-based assessments (WBAs) can be intimidating and burdensome for students and evaluators alike; however, these assessments pose an opportunity to use real-time direct observation to provide feedback that supports a learner’s growth and development.¹ Unfortunately, students often fail to see the usefulness of feedback in clinical settings or feel that their grades might be negatively affected by observations reported through workplace-based assessments.

Throughout my Advanced Pharmacy Practice Experiences (APPEs), I craved feedback so that I could develop as a clinician and ensure that I was providing optimal patient care. I valued the feedback that I received at the midpoint and final evaluations; however, these evaluations were used to determine my final grade. As a student, I benefitted from receiving more frequent, informal feedback to improve my performance in real time. By providing students with more timely formative assessments, preceptors allow students to reflect on their experiences and make necessary corrections to improve their practices without the stress of contributing to their grades.

WBAs are used to evaluate trainees’ performance in practice and can be used by learners as relevant feedback to engage in reflection. WBAs encompass a wide range of assessment strategies that require evaluators to move away from merely assigning numbers toward a more structured format of assessment. WBAs can be used to provide feedback on trainee-patient interactions, procedural skills, case-based discussions, and multi-source feedback.²

Lauren Phinney and colleagues at the University of California San Francisco used cultural historical activity theory (CHAT) to identify feedback system elements and tensions among these elements to explore workplace-based assessment used during medical clerkships. The school introduced a WBA tool in 2019 that includes drop-down items describing the clerkship specialty, skills observed, entrustment ratings adapted from the Ottawa scale, and space for narrative comments to encourage formative feedback. Students are required to gather two WBAs per week. The research interviewed first and second-year medical students participating in core clerkship rotations.¹

CHAT allows investigators to examine how tools mediate activities. An activity system is defined as the interaction between learners and tools to achieve an outcome. Tensions among these elements can promote change, create knowledge, and lead to new activity patterns.¹ After interviewing students in a series of focus groups, researchers identified five tensions:

1. Misinterpretation of WBA Feedback as Summative Assessment. Although WBAs were intended to serve formative purposes, first-year students perceived the object, or purpose, of the WBA to be for summative purposes. Formative assessments are intended to monitor student learning to provide ongoing feedback to improve teaching and learning. More specifically, formative assessments help students identify strengths and weaknesses. This allows students to target areas of improvement and help faculty pinpoint areas where students are struggling to provide assistance.³ On the other hand, the goal of summative assessments is to evaluate student learning at the end of a rotation and are often high stakes, resulting in the assignment of a grade or score. Even when second-year students correctly identified the purpose of WBAs as low-stakes feedback, students were still concerned that this feedback would be used to inform summative assessments and strategically chose to use WBAs when they anticipated positive feedback instead of opportunities for constructive feedback. Two ways to enhance the distinction between summative and formative evaluations in WBAs are to use two different platforms to complete WBAs and summative assessments and allow students to self-complete WBAs.¹
2. Cumbersome Tool Design that Delayed Feedback. WBA requests were sent via computer, so many of these requests were sent hours after feedback encounters. Students found that the distribution and completion of WBAs were delayed, which resulted in generic or untimely feedback. Utilizing QR codes on smart phones and improvements in technology facilitated supervisor engagement and rapid feedback.¹
3. Concern About Burdening Supervisors with WBA Tasks. While clerkship leaders encouraged students to seek feedback, students were concerned about interrupting workflow or interfering with patient care. Students found the assessments to be labor-intensive and redundant. Students employed strategies to streamline the process, such as recording and submitting comments that preceptors provided during the encounter with the WBA request form, which made it easier for preceptors to complete the assessments.¹
4. WBA Requirement as Checking Boxes vs. Learning Opportunity. The weekly quota of completion of two WBAs overshadowed the purpose of WBAs as a formative feedback mechanism. The authenticity and usefulness of the feedback could be jeopardized when students and supervisors focus on the rule instead of the opportunity to provide feedback. On the other hand, some students reframed this requirement to benefit them. One benefit of the requirement included the ability for students to direct their learning to meet self-identified goals and receive timely feedback to ensure that they were making progress toward achieving these goals. Another benefit of the rule was to initiate consistent feedback discussions with preceptors who did not volunteer to provide feedback.¹
5. WBA Within Clerkship-Specific Learning Culture. Supervisors’ promotion and acceptance of WBAs ultimately set the tone for WBA encounters. Students found that preceptors that actively facilitated WBA encounters provided more useful feedback, while preceptors that gave pushback created a barrier. In addition to using more convenient platforms to complete WBAs, students identified more convenient situations, logged feedback retrospectively, and bypassed tool discussion to minimize the burden on team members in settings that were not conducive to WBAs.¹

In competitive cultures like medicine, it can be difficult to facilitate formative assessments. The author concluded that by incorporating learner input to make intentional changes, perceptions and utilization of WBAs can be enhanced.¹

The authors provided potential solutions to the perceived problems with WBAs. There is often a disconnect between the intention and interpretation of workplace-based assessments.  Thus, we need to consider structuring their format and delivery by gathering student feedback. Through this collaboration with students, we can strive to achieve authentic workplace-based assessments that accurately reflect learners’ progress and are used to improve future performance.

While this study focused on the benefits of WBAs in student-preceptor interactions at one medical school, WBAs can be used in several ways. WBAs can be applied across multiple settings and can be separated into three different categories: observation of clinical performance, discussion of clinical cases, and feedback from peers, coworkers, and patients. These assessment tools provide insight to the trainee, assessor, and academics alike.²

In addition to getting student feedback, I believe we need to gather feedback from preceptors to determine their perceptions of WBAs. Thus, WBAs could be further improved to meet the needs of both students and preceptors. To ensure that we are providing useful and timely feedback to learners, its important to reduce the barriers to WBA use. By using QR codes, separate platforms to differentiate summative and formative assessments, and platforms that are compatible with smartphones when computers are not available, schools can establish user-friendly and time-efficient processes and ensure that WBAs that are valuable without adding substantial burden that jeopardize feedback quality.¹

References:

1. Phinney, LB, Fluet A, O’Brien BC, Seligman L, Hauer KE. Beyond checking boxes: Exploring tensions with use of a workplace-based assessment tool for formative assessment in clerkships. Acad Med 2022; 97: 1511-1520.
2. Guraya, SY. Workplace-based assessment; Applications and educational impact. Malays J Med Sci 2015; 22: 5-10.
3. Formative vs. Summative Assessment [Internet]. Pittsburgh: Carnegie Mellon University; [cited 2022 Nov 18].