Incorporating OSCEs into the Curriculum

by Jueli Li, PharmD, PGY1 Pharmacy Practice Resident, University of Maryland Medical Center

The objective structured clinical examination (OSCE) is an authentic assessment, where learners are evaluated in a realistic, simulated real-world setting that requires the student to apply their knowledge and skills in a problem-based learning environment.^1-3 Although OSCEs are implemented differently at different health professional schools, they all include a series of stations in which standardized patients portray a part in an interactive clinical case scenarios. Students are graded based on a set of items on a checklist that are intended to assess students’ ability to problem-solve.^2,3  An overall impression rating is also given to the students and knowledge is indirectly measured based on the response to the case scenario.^2,3 At my pharmacy school, students were evaluated using OSCE each spring semester as a part of our pharmacy practice laboratory course. During the first year, students were assessed on one OSCE case and by the time they complete their third year, students are assessed on a series of five case scenarios. Each scenario focused on a different aspect of patient care, including medication reconciliation, patient counseling, documentation, and other pharmacy-related tasks.



The OSCE was initially described in the 1970s as an objective evaluation of clinical competency and has since then been used in the United States Medical Licensing Examination, the Medical Council of Canada Qualifying Examination, as well as the Canadian Pharmacist Qualifying Examination.^4,5 OSCEs are used during the professional education of many health professions including dentistry, optometry, nursing, and more. Starting in 2007, the United States pharmacy school accreditation board, Accreditation Council for Pharmacy Education (ACPE), included OSCE as an option for schools to provide simulated clinical experiences.⁶ Since then, there has been a surge in the number of pharmacy schools who have implemented them.

OSCEs are a valid method to evaluate pharmacy students’ performance. In an observational single-cohort study conducted in the United Kingdom which compared multiple-choice exams to OSCEs, the investigators found that student performance on the multiple-choice exam was only moderately correlated with the outcome of an OSCE (r=0.6). The study showed that OSCE assessed a student’s knowledge and skill, while multiple choice examinations are only able to assess a student’s knowledge.² These results were further corroborated by another observational study in the United States, where the authors found a very weak relationship between OSCE scores and written examination scores (r=0.15; p=0.24).

A survey of students and faculty have found that the OSCE method was view favorably, particularly in terms of realism. Furthermore, more than 75% of students rated OSCEs as an appropriate way to measure their knowledge, communication, and clinical skills. The results showcased the value of utilizing OSCEs to provide a more comprehensive assessment for problem-based learning as well as building critical thinking, communication abilities, and clinical judgment.³

In 2010, a study sampled 108 pharmacy schools in the United States and found a great deal of variability with regard to how pharmacy schools implemented OSCEs. Many schools indicated that cost and increased faculty time commitment were barriers to implement OSCEs. In order to create valid exams, a consortium of faculty are needed to develop authentic case scenarios and validate checklists. It is also important to provide training for standardized patients and examiners, and to have an adequate amount of space or stations.⁵

Fortunately, there is a wealth of information on how to implement OSCEs. The Association of Standardized Patient Educators (ASPE) has an OSCE toolkit.⁷  A paper by the Medical University of Vienna in Austria provides detailed guidance document on how to develop and implement OSCEs.⁸ Both documents recommend adequate planning in the pre-assessment phase to determine available resources, resource allocation, test logistics, and an “examination blueprint” or a test plan that includes the objectives, goals, and competencies that the OSCE cases will assess. Once those points have been discussed, realistic cases and grading checklists need to be developed. Additionally, it is important to provide detailed instructions – not only to the students, but also the standardized patients.  This includes the grading checklists and embedded instructions in the written cases scenarios. To decrease inter-rater variable, the Medical University of Vienna suggests providing standardized patients and graders a frame of reference for a “gold standard” student and identifying standards for pass and fail. The ASPE OSCE toolkit contains a to-do list for developing the cases, instructions on how to implement the OSCEs, how to set the grading checklists, as well as an actual case example, which could be very helpful for any institution.⁷

The integration of OSCEs into the curriculum allows students an opportunity to develop their critical thinking abilities, communication skills, and knowledge base in a more realistic setting. Current literature suggests that OSCEs evaluate more than just knowledge and complement traditional assessment methods. Schools interested in improving or initiating the use of OSCEs at their institution can find ample resources for guidance.

References:

1. Mueller J. The authentic assessment toolbox: Enhancing student learning through online faculty development. J Online Learning and Teaching. 2005 Jul:1(1):1-7.
2. Kirton SB and Kravitz L. Objective structured clinical examinations (OSCEs) compared with traditional assessment methods. Am J Pharm educ. 2011 Aug 10: 75(6): Article 111.
3. Salinitri FD, O’Connell MB, Garwood CL, et al. An objective structured clinical examination to assess problem-based learning. Am J Pharm Educ. 2012 Apr 10: 76(3): Article 44.
4. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. Br Med J. 1975; 1: 447-451.
5. Sturpe DA. Objective structured clinical examinations in doctor of pharmacy programs in the United States. Am J Pharm Educ. 2010 Oct 11: 74(8): Article 148.
6. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. 2007 Jul 1.
7. Association of Standardized Patient Educators. Pharmacy OSCE toolkit. [Internet]. (cited 2015 Oct 29).
8. Preusche I, Schmidts M, Wagner-Menghin M. Twelve tips for designing and implementing a structured rater training in OSCEs. Med Teacher. 2012; 34: 368.

Promoting Diversity, Curiosity, and Creativity in Education

by Stephanie Walters, PharmD, PGY2 Geriatric Pharmacy Pracitce Resident, University of Maryland School of Pharmacy

Standardization seems to be defining our educational system, from the Common Core in grade schools to standardized curriculum in graduate schools. While it is valid and important to have quality standards, standardization risks minimizing individuality, curiosity, and creativity. Sir Ken Robinson, leading creativity expert in education, addresses these three principles for improving our education system in his TED Talk, “How to escape education’s death valley.”¹  Sir Robinson has been the featured speaker in several TED Talks, always highlighting the importance of creativity and how to foster it. While Sir Robinson focuses on childhood education; however, I believe we can relate these same concepts to the adult learner.

Principle 1: Diversity and Individuality

It may seem obvious, but every person is different. While standards in education may be needed to ensure quality between different settings, at the same time it can discount diversity and lessens the emphasis on individuality. I loved the example Sir Robinson explains regarding the educational system in Finland: when Americans asked how the Finnish educational system addressed dropouts they responded, "Well, we don't have one. Why would you drop out? If people are in trouble, we get to them quite quickly and we help and support them."

Per a report from the American Association of Colleges of Pharmacy, the average attrition rate at US Colleges/Schools of Pharmacy is ~10.5%.² This same sentiment used in Finnish grade schools could definitely be utilized for the pharmacy student as well. Pharmacy students come from a variety of backgrounds. Whether it’s a second career choice or coming straight from high school, there is wide diversity within this cohort. Understanding and embracing these differences should create a thriving learning environment. One strategy utilized in the Finnish educational system, is the idea of not focusing on standardization. Instead, they take a broader approach to education (including arts, science, humanities) and there is no standardized tests. This strategy individualizes the learning environment and assigns responsibilities to the school level, as opposed to mandates from the state or federal governments. Thus, providing enough resources to address these diversities at a school level, as the Finns do, is imperative to support the adult learners and provide a quality learning environment.

Principle 2: Curiosity and Learning

Curiosity didn’t kill the cat. Instead, curiosity drives the act of learning. In a study of college students in Hong Kong, China, students with high levels of curiosity were shown to have more intrinsic motivation.  External regulations (i.e. rules and university policy) had no effect on improving students’ motivation.³ These findings are quite striking when we look at the standardized structures and regulations within graduate programs like pharmacy schools. Finding a balance between meeting these standards and cultivating curiosity is a necessity.

Active learning strategies used in a variety of educational settings might promote curiosity among learners. Curiosity is defined as a desire to know or learn. In the large classroom setting, providing interesting, interactive cases or group discussion on hot topics can foster this desire to learn. Within an active, hands-on learning setting, such clinical practice sites, the teacher can foster further curiosity by understanding the student’s interests and providing opportunities for new experiences that align with those interests. This approach empowers students through their own individual interests, and thus fuels their curiosity.

Principle 3: Innate Creativity

Sir Robinson is most widely known for his advocacy of promoting creativity in education. He believes we are all inherently creative, and this creativity fosters diversity as well as inspires curiosity. It’s a cycle that creates flourishing learning environments. For adult learners, a teacher can promote creativity by using a creative problem solving model, like the Osborne-Parnes model.⁴ This model uses a creative thinking process (from mess-finding to acceptance-finding) that can be used in a variety of adult learning experiences and to promote problem solving.

Instruction should be designed to engage the learner using these three important principles. The needs of teachers should not be overlooked! Teachers need continuous professional development and support. Offering courses on educational theory, workshops of active learning techniques, and encouraging the use of technology with adult learners are just a few ways to help translate these principles into practice.  For example, educating teachers about the different approaches and theories of learning provides a variety of avenues (i.e. constructivism, mindset, or andragogy) that can foster diversity, curiosity, and creativity.^5-7

As Albert Einstein once said, “It is the supreme art of the teacher to awaken joy in creative expression and knowledge.”

Bibliography

1. Robinson K. How to escape education’s death valley [Internet Video]. TED Talks Education. 2013 April [cited 2015 Sept 29].
2. Academic Pharmacy’s Vital Statistics [Internet]. American Association of Colleges of Pharmacy. 2015 Oct [updated 2015 Oct 5; cited 2015 Oct 25].
3. Hon-keung Y, Man-shan K, Lai-fong C. The Impact of Curiosity and External Regulation on Intrinsic Motivation: An Empirical Study in Hong Kong Education. Psychology Research. 2012 May;2(5):295-307.
4. Osborne-Parnes Creative Problem Solving (CPS) Model [Internet]. Teaching Creativity. 2009 Jun [updated 25 Jun 2009; cited 2015 Nov 6].
5. Constructivism (philosophy of education) [Internet]. Wikipedia. 2015 Nov [updated 2015 Nov 6; cited 2015 Nov 6].
6. Dweck C. Mindset [Internet]. Mindset Online. 2010 [updated 2010; cited 2015 Nov 6].
7. Pappas C. The Adult Learning Theory – Andragogy [Internet]. eLearning Industry. 2013 May 9 [updated ; cited 6 Nov 2015].