Incorporating OSCEs into the Curriculum

by Jueli Li, PharmD, PGY1 Pharmacy Practice Resident, University of Maryland Medical Center

The objective structured clinical examination (OSCE) is an authentic assessment, where learners are evaluated in a realistic, simulated real-world setting that requires the student to apply their knowledge and skills in a problem-based learning environment.^1-3 Although OSCEs are implemented differently at different health professional schools, they all include a series of stations in which standardized patients portray a part in an interactive clinical case scenarios. Students are graded based on a set of items on a checklist that are intended to assess students’ ability to problem-solve.^2,3  An overall impression rating is also given to the students and knowledge is indirectly measured based on the response to the case scenario.^2,3 At my pharmacy school, students were evaluated using OSCE each spring semester as a part of our pharmacy practice laboratory course. During the first year, students were assessed on one OSCE case and by the time they complete their third year, students are assessed on a series of five case scenarios. Each scenario focused on a different aspect of patient care, including medication reconciliation, patient counseling, documentation, and other pharmacy-related tasks.



The OSCE was initially described in the 1970s as an objective evaluation of clinical competency and has since then been used in the United States Medical Licensing Examination, the Medical Council of Canada Qualifying Examination, as well as the Canadian Pharmacist Qualifying Examination.^4,5 OSCEs are used during the professional education of many health professions including dentistry, optometry, nursing, and more. Starting in 2007, the United States pharmacy school accreditation board, Accreditation Council for Pharmacy Education (ACPE), included OSCE as an option for schools to provide simulated clinical experiences.⁶ Since then, there has been a surge in the number of pharmacy schools who have implemented them.

OSCEs are a valid method to evaluate pharmacy students’ performance. In an observational single-cohort study conducted in the United Kingdom which compared multiple-choice exams to OSCEs, the investigators found that student performance on the multiple-choice exam was only moderately correlated with the outcome of an OSCE (r=0.6). The study showed that OSCE assessed a student’s knowledge and skill, while multiple choice examinations are only able to assess a student’s knowledge.² These results were further corroborated by another observational study in the United States, where the authors found a very weak relationship between OSCE scores and written examination scores (r=0.15; p=0.24).

A survey of students and faculty have found that the OSCE method was view favorably, particularly in terms of realism. Furthermore, more than 75% of students rated OSCEs as an appropriate way to measure their knowledge, communication, and clinical skills. The results showcased the value of utilizing OSCEs to provide a more comprehensive assessment for problem-based learning as well as building critical thinking, communication abilities, and clinical judgment.³

In 2010, a study sampled 108 pharmacy schools in the United States and found a great deal of variability with regard to how pharmacy schools implemented OSCEs. Many schools indicated that cost and increased faculty time commitment were barriers to implement OSCEs. In order to create valid exams, a consortium of faculty are needed to develop authentic case scenarios and validate checklists. It is also important to provide training for standardized patients and examiners, and to have an adequate amount of space or stations.⁵

Fortunately, there is a wealth of information on how to implement OSCEs. The Association of Standardized Patient Educators (ASPE) has an OSCE toolkit.⁷  A paper by the Medical University of Vienna in Austria provides detailed guidance document on how to develop and implement OSCEs.⁸ Both documents recommend adequate planning in the pre-assessment phase to determine available resources, resource allocation, test logistics, and an “examination blueprint” or a test plan that includes the objectives, goals, and competencies that the OSCE cases will assess. Once those points have been discussed, realistic cases and grading checklists need to be developed. Additionally, it is important to provide detailed instructions – not only to the students, but also the standardized patients.  This includes the grading checklists and embedded instructions in the written cases scenarios. To decrease inter-rater variable, the Medical University of Vienna suggests providing standardized patients and graders a frame of reference for a “gold standard” student and identifying standards for pass and fail. The ASPE OSCE toolkit contains a to-do list for developing the cases, instructions on how to implement the OSCEs, how to set the grading checklists, as well as an actual case example, which could be very helpful for any institution.⁷

The integration of OSCEs into the curriculum allows students an opportunity to develop their critical thinking abilities, communication skills, and knowledge base in a more realistic setting. Current literature suggests that OSCEs evaluate more than just knowledge and complement traditional assessment methods. Schools interested in improving or initiating the use of OSCEs at their institution can find ample resources for guidance.

References:

1. Mueller J. The authentic assessment toolbox: Enhancing student learning through online faculty development. J Online Learning and Teaching. 2005 Jul:1(1):1-7.
2. Kirton SB and Kravitz L. Objective structured clinical examinations (OSCEs) compared with traditional assessment methods. Am J Pharm educ. 2011 Aug 10: 75(6): Article 111.
3. Salinitri FD, O’Connell MB, Garwood CL, et al. An objective structured clinical examination to assess problem-based learning. Am J Pharm Educ. 2012 Apr 10: 76(3): Article 44.
4. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. Br Med J. 1975; 1: 447-451.
5. Sturpe DA. Objective structured clinical examinations in doctor of pharmacy programs in the United States. Am J Pharm Educ. 2010 Oct 11: 74(8): Article 148.
6. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree. 2007 Jul 1.
7. Association of Standardized Patient Educators. Pharmacy OSCE toolkit. [Internet]. (cited 2015 Oct 29).
8. Preusche I, Schmidts M, Wagner-Menghin M. Twelve tips for designing and implementing a structured rater training in OSCEs. Med Teacher. 2012; 34: 368.