Incorporating OSCEs into the Curriculum

by Jueli Li, PharmD, PGY1 Pharmacy Practice Resident, University of Maryland Medical Center

The objective structured clinical examination (OSCE) is an authentic assessment, where learners are evaluated in a realistic, simulated real-world setting that requires the student to apply their knowledge and skills in a problem-based learning environment.^1-3 Although OSCEs are implemented differently at different health professional schools, they all include a series of stations in which standardized patients portray a part in an interactive clinical case scenarios. Students are graded based on a set of items on a checklist that are intended to assess students’ ability to problem-solve.^2,3  An overall impression rating is also given to the students and knowledge is indirectly measured based on the response to the case scenario.^2,3 At my pharmacy school, students were evaluated using OSCE each spring semester as a part of our pharmacy practice laboratory course. During the first year, students were assessed on one OSCE case and by the time they complete their third year, students are assessed on a series of five case scenarios. Each scenario focused on a different aspect of patient care, including medication reconciliation, patient counseling, documentation, and other pharmacy-related tasks.



The OSCE was initially described in the 1970s as an objective evaluation of clinical competency and has since then been used in the United States Medical Licensing Examination, the Medical Council of Canada Qualifying Examination, as well as the Canadian Pharmacist Qualifying Examination.^4,5 OSCEs are used during the professional education of many health professions including dentistry, optometry, nursing, and more. Starting in 2007, the United States pharmacy school accreditation board, Accreditation Council for Pharmacy Education (ACPE), included OSCE as an option for schools to provide simulated clinical experiences.⁶ Since then, there has been a surge in the number of pharmacy schools who have implemented them.

OSCEs are a valid method to evaluate pharmacy students’ performance. In an observational single-cohort study conducted in the United Kingdom which compared multiple-choice exams to OSCEs, the investigators found that student performance on the multiple-choice exam was only moderately correlated with the outcome of an OSCE (r=0.6). The study showed that OSCE assessed a student’s knowledge and skill, while multiple choice examinations are only able to assess a student’s knowledge.² These results were further corroborated by another observational study in the United States, where the authors found a very weak relationship between OSCE scores and written examination scores (r=0.15; p=0.24).

A survey of students and faculty have found that the OSCE method was view favorably, particularly in terms of realism. Furthermore, more than 75% of students rated OSCEs as an appropriate way to measure their knowledge, communication, and clinical skills. The results showcased the value of utilizing OSCEs to provide a more comprehensive assessment for problem-based learning as well as building critical thinking, communication abilities, and clinical judgment.³

In 2010, a study sampled 108 pharmacy schools in the United States and found a great deal of variability with regard to how pharmacy schools implemented OSCEs. Many schools indicated that cost and increased faculty time commitment were barriers to implement OSCEs. In order to create valid exams, a consortium of faculty are needed to develop authentic case scenarios and validate checklists. It is also important to provide training for standardized patients and examiners, and to have an adequate amount of space or stations.⁵

Fortunately, there is a wealth of information on how to implement OSCEs. The Association of Standardized Patient Educators (ASPE) has an OSCE toolkit.⁷  A paper by the Medical University of Vienna in Austria provides detailed guidance document on how to develop and implement OSCEs.⁸ Both documents recommend adequate planning in the pre-assessment phase to determine available resources, resource allocation, test logistics, and an “examination blueprint” or a test plan that includes the objectives, goals, and competencies that the OSCE cases will assess. Once those points have been discussed, realistic cases and grading checklists need to be developed. Additionally, it is important to provide detailed instructions – not only to the students, but also the standardized patients.  This includes the grading checklists and embedded instructions in the written cases scenarios. To decrease inter-rater variable, the Medical University of Vienna suggests providing standardized patients and graders a frame of reference for a “gold standard” student and identifying standards for pass and fail. The ASPE OSCE toolkit contains a to-do list for developing the cases, instructions on how to implement the OSCEs, how to set the grading checklists, as well as an actual case example, which could be very helpful for any institution.⁷

The integration of OSCEs into the curriculum allows students an opportunity to develop their critical thinking abilities, communication skills, and knowledge base in a more realistic setting. Current literature suggests that OSCEs evaluate more than just knowledge and complement traditional assessment methods. Schools interested in improving or initiating the use of OSCEs at their institution can find ample resources for guidance.

References:

1. Mueller J. The authentic assessment toolbox: Enhancing student learning through online faculty development. J Online Learning and Teaching. 2005 Jul:1(1):1-7.
2. Kirton SB and Kravitz L. Objective structured clinical examinations (OSCEs) compared with traditional assessment methods. Am J Pharm educ. 2011 Aug 10: 75(6): Article 111.
3. Salinitri FD, O’Connell MB, Garwood CL, et al. An objective structured clinical examination to assess problem-based learning. Am J Pharm Educ. 2012 Apr 10: 76(3): Article 44.
4. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. Br Med J. 1975; 1: 447-451.
5. Sturpe DA. Objective structured clinical examinations in doctor of pharmacy programs in the United States. Am J Pharm Educ. 2010 Oct 11: 74(8): Article 148.
6. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. 2007 Jul 1.
7. Association of Standardized Patient Educators. Pharmacy OSCE toolkit. [Internet]. (cited 2015 Oct 29).
8. Preusche I, Schmidts M, Wagner-Menghin M. Twelve tips for designing and implementing a structured rater training in OSCEs. Med Teacher. 2012; 34: 368.

Promoting Diversity, Curiosity, and Creativity in Education

by Stephanie Walters, PharmD, PGY2 Geriatric Pharmacy Pracitce Resident, University of Maryland School of Pharmacy

Standardization seems to be defining our educational system, from the Common Core in grade schools to standardized curriculum in graduate schools. While it is valid and important to have quality standards, standardization risks minimizing individuality, curiosity, and creativity. Sir Ken Robinson, leading creativity expert in education, addresses these three principles for improving our education system in his TED Talk, “How to escape education’s death valley.”¹  Sir Robinson has been the featured speaker in several TED Talks, always highlighting the importance of creativity and how to foster it. While Sir Robinson focuses on childhood education; however, I believe we can relate these same concepts to the adult learner.

Principle 1: Diversity and Individuality

It may seem obvious, but every person is different. While standards in education may be needed to ensure quality between different settings, at the same time it can discount diversity and lessens the emphasis on individuality. I loved the example Sir Robinson explains regarding the educational system in Finland: when Americans asked how the Finnish educational system addressed dropouts they responded, "Well, we don't have one. Why would you drop out? If people are in trouble, we get to them quite quickly and we help and support them."

Per a report from the American Association of Colleges of Pharmacy, the average attrition rate at US Colleges/Schools of Pharmacy is ~10.5%.² This same sentiment used in Finnish grade schools could definitely be utilized for the pharmacy student as well. Pharmacy students come from a variety of backgrounds. Whether it’s a second career choice or coming straight from high school, there is wide diversity within this cohort. Understanding and embracing these differences should create a thriving learning environment. One strategy utilized in the Finnish educational system, is the idea of not focusing on standardization. Instead, they take a broader approach to education (including arts, science, humanities) and there is no standardized tests. This strategy individualizes the learning environment and assigns responsibilities to the school level, as opposed to mandates from the state or federal governments. Thus, providing enough resources to address these diversities at a school level, as the Finns do, is imperative to support the adult learners and provide a quality learning environment.

Principle 2: Curiosity and Learning

Curiosity didn’t kill the cat. Instead, curiosity drives the act of learning. In a study of college students in Hong Kong, China, students with high levels of curiosity were shown to have more intrinsic motivation.  External regulations (i.e. rules and university policy) had no effect on improving students’ motivation.³ These findings are quite striking when we look at the standardized structures and regulations within graduate programs like pharmacy schools. Finding a balance between meeting these standards and cultivating curiosity is a necessity.

Active learning strategies used in a variety of educational settings might promote curiosity among learners. Curiosity is defined as a desire to know or learn. In the large classroom setting, providing interesting, interactive cases or group discussion on hot topics can foster this desire to learn. Within an active, hands-on learning setting, such clinical practice sites, the teacher can foster further curiosity by understanding the student’s interests and providing opportunities for new experiences that align with those interests. This approach empowers students through their own individual interests, and thus fuels their curiosity.

Principle 3: Innate Creativity

Sir Robinson is most widely known for his advocacy of promoting creativity in education. He believes we are all inherently creative, and this creativity fosters diversity as well as inspires curiosity. It’s a cycle that creates flourishing learning environments. For adult learners, a teacher can promote creativity by using a creative problem solving model, like the Osborne-Parnes model.⁴ This model uses a creative thinking process (from mess-finding to acceptance-finding) that can be used in a variety of adult learning experiences and to promote problem solving.

Instruction should be designed to engage the learner using these three important principles. The needs of teachers should not be overlooked! Teachers need continuous professional development and support. Offering courses on educational theory, workshops of active learning techniques, and encouraging the use of technology with adult learners are just a few ways to help translate these principles into practice.  For example, educating teachers about the different approaches and theories of learning provides a variety of avenues (i.e. constructivism, mindset, or andragogy) that can foster diversity, curiosity, and creativity.^5-7

As Albert Einstein once said, “It is the supreme art of the teacher to awaken joy in creative expression and knowledge.”

Bibliography

1. Robinson K. How to escape education’s death valley [Internet Video]. TED Talks Education. 2013 April [cited 2015 Sept 29].
2. Academic Pharmacy’s Vital Statistics [Internet]. American Association of Colleges of Pharmacy. 2015 Oct [updated 2015 Oct 5; cited 2015 Oct 25].
3. Hon-keung Y, Man-shan K, Lai-fong C. The Impact of Curiosity and External Regulation on Intrinsic Motivation: An Empirical Study in Hong Kong Education. Psychology Research. 2012 May;2(5):295-307.
4. Osborne-Parnes Creative Problem Solving (CPS) Model [Internet]. Teaching Creativity. 2009 Jun [updated 25 Jun 2009; cited 2015 Nov 6].
5. Constructivism (philosophy of education) [Internet]. Wikipedia. 2015 Nov [updated 2015 Nov 6; cited 2015 Nov 6].
6. Dweck C. Mindset [Internet]. Mindset Online. 2010 [updated 2010; cited 2015 Nov 6].
7. Pappas C. The Adult Learning Theory – Andragogy [Internet]. eLearning Industry. 2013 May 9 [updated ; cited 6 Nov 2015].

Preventing Academic Burnout in Students

by Mary Afrane, PGY1 Pharmacy Practice Resident, MedStar Union Memorial Hospital

Stress in the day-to-day life of a college student is common and expected. However, if stress becomes unmanageable and interferes with academic performance and school-life balance, the student might be suffering from a more serious condition called academic burnout. Academic burnout is characterized by emotional exhaustion, educational pessimism, and feelings of incompetence.^1-2 Academic burnout is an important topic in higher education because it can impact the relationship between students and faculty and increase dropout rates.

Speaking from personal experience after spending most of my adult life in school training for a career in healthcare, it is easy to become overwhelmed by the constant demands and responsibilities that drain your energy. Students who are in a state of burnout often become cynical and start to lose interest achieving the goals that led them to school in the first place.³ Students experiencing burnout may display signs such as excessive absenteeism, lack of interest in academic activities, and express a sense of meaninglessness toward educational assignments.¹ Students may also display severe negative emotional symptoms such as hopelessness, helplessness, and depression.

Not surprisingly, academic burnout is negatively associated with academic performance and the quality of student’s learning experience.¹ Causes of academic burnout are multifactorial, including environmental and individual reasons. Stress and low self-efficacy are two variables that influence academic engagement.²

Many students will demonstrate signs of burnout towards the end of the academic semester or near the completion of their thesis or dissertation program. This is generally not of major concern, given that a break for recovery is in sight and most students are able to successfully complete the remaining work. However, burnout becomes a concern if it occurs early on in the semester and the student lacks tools for dealing with the chronic exhaustion, depersonalization, and feelings of low personal achievement.

Burnout can be managed by using the “Three R” approach:³

Recognize the warning signs of burnout.

Reverse the damage by managing stress and seeking support.

Resilience towards stress can be built upon by improving physical and emotional health.

There are other practical steps that can be taken by students and teachers to reduce academic burnout. Faculty can help to minimize student cynicism by developing instructional plans that stimulate students’ thinking. Course assignments can be purposefully designed such that the assignments are not perceived as “busy work,” but something that adds meaningful value to the students’ educational journey. College students can mitigate burnout by selecting courses of interest to them. Recognizing one’s personal limitations to handle a heavy academic load can help to prevent chronic exhaustion. In addition, students can take steps to address person matters such as low self-efficacy. Students with high self-efficacy are able to cope with a failure better because they able to attribute the failure to factors such as low effort or gaps in knowledge and skills. By making the proper attributions, those with high self-efficacy are able to bounce back.¹ Addressing issues of low self-efficacy enables students to withstand overwhelming pressures when faced with difficult tasks.

Poor time management can give rise to feelings of being overwhelmed. It is imperative that steps are taken to achieve adequate preparation for exams and completing major tasks. Properly prioritizing assignments and achieving balance between personal and academic life can give students control of stressful situations and reduce feelings of chronic exhausting. Stress coping programs employing cognitive-behavioral therapy have been shown to reduce exhaustion in students, with benefits often lasting 1 to 2 months after program paritcipation.⁴ Stress coping programs are conducted with the primary purpose of highlighting the student’s maladaptive response to stress and developing better stress coping skills. The participants reflect on their thought process and are guided on converting negative thoughts to positive ones. Physical exercise and relaxation can reduce stress levels and can reduce academic burnout.⁴

Having a strong support system and realistic expectations, when facing a heavy academic load can help students deal with the intensive demands. When I was a college student, I attempted to complete two science majors in four years by taking a 21 credit course-load each semester. After listening to close friends and mentors, I realized that my plan was unrealistic and was a set up for burnout. I reassessed my plans and made changes so that I was able to complete both degrees in five years, which allowed extra time to become involved in extracurricular activities — thing that gave more depth and meaning to my college experience.

Academic burnout adversely impacts students’ performance and the student-teacher relationship. Students without good coping skills might succumb to the pressures of a challenging academic workload. Students can take steps to address low self-efficacy and teachers can be more purposeful in designing academic instructional plans.

References

1. Rahmati Z. The study of academic burnout in students with high and low level of self-efficacy. Procedia - Social and Behavioral Sciences 2015;171:49-55.  
2. Charkhabi M, Abarghuei MA, Hayati D. The association of academic burnout with self-efficacy and quality of learning experience among Iranian students. SpringerPlus 2013;2:677.
3. Fallon SJ. Burn out: how to recognize, prevent, or recover. [Internet]. June 2009. Accessed October 31, 2015.
4. Sonmez GY, Capri B, Lisesi SP. The effects of stress coping program on burnout levels of high school students. Int J New Trends in Edu and Their Implications 2013;4:148-64.