Instructor Feedback

by Nina Cimino, Pharm.D., PGY1 Pharmacy Practice Resident, Suburban Hospital

As a recent pharmacy school graduate, a current student of educational theory and practice, and as an aspiring teacher, I have been doing a lot of thinking about the transition from student to teacher.  As a student, I relied heavily on feedback to improve my performance.  As I continue learning about educational theory and looking for strategies to improve as a teacher in the classroom, I wondered how feedback fits into faculty self-reflection and development.  This topic has been receiving attention in the media, with Bill Gates advocating for more meaningful and frequent feedback to teachers as a tool to improve classroom instruction and educational outcomes.¹  Watch this video of Bill Gates discussing his ideas with CNN’s Fareed Zakaria.

During formal educational activities, students receive feedback from their instructors in a variety of formats:  

• Graded assignments and examinations provide feedback about a student’s mastery of course material
• Verbal feedback from faculty members provides suggestions for improvement
• Guidance during classroom discussions encourages students to engage with course material
• Encouragement from faculty members provides positive reinforcement and motivation
• Re-direction (when necessary) helps student identify areas for improvement

These sources of feedback all help learners determine if they understand course materials, are thinking rationally about concepts, and promotes self-reflection on ways to improve learning and understanding.  Once learners are no longer formal students in a classroom and assume the role of teacher, what are the sources of feedback to help them improve?

While many instructors receive feedback from students in the form of course evaluations, issues other than the effectiveness of a particular instructor may influence student evaluations.²  Students are admittedly biased by their own performance.  Their perceived success can impact their course evaluation, even though factors other than instructor effectiveness contributed to their performance.  While student feedback is one source of valuable information for teachers, other sources can also be valuable in promoting instructor self-reflection and improvement.

Northeastern University School of Pharmacy recently implemented a tool for peer observation and evaluation of faculty members.³  Faculty members received formal training on how to provide peer feedback from the University’s Center for Effective University Teaching and were asked to serve as a peer observer for a colleague’s large-group teaching activity.  The peer observation and evaluation process consisted of four components:

• A pre-observation meeting to discuss the objectives of the class session
• Classroom observation (one observation annually)
• A post-observation meeting to discuss the instructor’s self-reflection of his or her performance and the observer’s positive and constructive feedback (2-3 strengths and 2-3 areas for improvement were required)
• A post-student assessment meeting to assess the students’ achievement after completion of examinations and/or assignments

Surveys of faculty members before and after implementation of the peer observation program indicated that instructors found the program to be beneficial for improving their teaching.³  The majority of instructors who participated in the program (87%) reported making changes to some elements of their teaching (e.g. content, teaching methods, and/or assessment). In addition, 83% of participants agreed or strongly agreed “peer assessment is a better measurement of teaching effectiveness then [sic] student evaluations.”  The majority of participants also agreed that the benefits of participating in the peer observation and evaluation program outweighed the time commitment required.

In addition to student evaluations and peer observation programs, focus groups may also be used to gather information from students regarding the effectiveness of a teacher, course, or an entire educational program.⁴  While focus groups are not as easy to administer as traditional student evaluation surveys, they are particularly useful for exploring patterns or issues that arise during students’ evaluation of a teacher.  In addition to identifying student perceptions of a teacher’s effectiveness, focus groups allow for a facilitator to ask follow up questions in order to gain insight into issues identified by students.³

It is well recognized that feedback helps students identify strengths and areas for improvement, as well as promoting student self-assessment of their learning.  Providing instructors with feedback from multiple sources would arguably have similar benefits.  Student evaluations of courses and instructor effectiveness is undeniably important, but peer observation and evaluation, as well as student focus groups, can provide additional information that student evaluations simply don’t capture.  Focus groups allow for in-depth exploration of issues identified by students.  Feedback from a peer who understands the challenges faced based by instructors and who can offer suggestions for improvement based on personal experiences and successes would be invaluable.  These alternative feedback mechanisms are useful not only for instructors who are new to the profession but also seasoned instructors.

References: 

1.  Miks J. "Bill Gates on What Makes a Good Teacher." Global Public Square RSS. Cable News Network, 1 Feb. 2013. Web. 16 Feb. 2013.

2. The value and limitations of student ratings.  The Ideas Center, Inc.  Accessed on:  March 5, 2013.

3.  DiVall M., Barr J, Gonyeau M., et al.  Follow-up assessment of a faculty peer observation and evaluation program.  Am J Pharm Educ 2012; 76, article 61. 

4.  Eberly Center for Teaching Excellence. Using Focus Groups to Get Student Feedback. Carnegie Mellon: Enhancing Education. Carnegie Mellon University. Web. Accessed on:  Feb 25, 2013.

Why is a Pharmacy Residency Needed?

by Melonie Blake, PharmD, PGY1 Pharmacy Practice Resident, Sibley Memorial Hospital

The profession of pharmacy has evolved considerably over the past few years toward the provision of patient-centered care. It has been imperative to ensure that pharmacists can function in a capacity that will provide safe and effective medication use. In order for this to occur, there has been a lot of advocacy to support the premise that all pharmacists should complete a residency before assuming direct patient care roles. I have heard the arguments in favor of this notion, as well as rebuttals that residencies are not necessary.

Before starting pharmacy school, I knew that I wanted to become a clinical pharmacist and completion of a residency was a path toward that goal. It was apparent that some clinical pharmacists did not complete a residency, particularly those who graduated before the entry-level Doctor of Pharmacy degree became the norm. Unfortunately, in recent years, the job market has taken a turn and it has become more difficult and competitive.  But the professional is also more accountable and pharmacists are more involved with patients by using the expertise that has been gained from pharmacy school, experiential learning, residency, and other post-graduate experiences.

This evoluation has led to greater participation by pharmacists on interprofessional teams, performing as the “drug experts”, and optimizing medication use by ensuring safety and efficacy, tailored to each patient’s need.¹ In order to fulfill this demand and to live up to these new expectations and standards, we have to be well prepared and confident in the decisions that we make. After going through the Introductory and Advanced Pharmacy Practice Experiences (IPPE/ APPE) at my school, I did not feel fully prepared to take on the challenges of being a clinical pharmacist, even though I valued and gained a great deal from those experiences. They provided me the foundation, along with didactic instruction, to continue to develop my clinical skills after graduation and introduced me to the integral role that clinical pharmacists play in healthcare today.

In Securing and Excelling in a Pharmacy Residency, the value of residency training is discussed.  In one of the chapters, the author explains that pharmacy residencies build upon pharmacy school by expanding the new pharmacist’s knowledge base and learning to effectively apply our knowledge and skills to each patient.  Through these experiences it increases the new practitioner’s confidence. The residency program provides a structured process to develop clinical skills with an “effective preceptor.”² By taking an active role to give efficient, individualized care to each patient, pharmacy residents become increasingly capable of delivering patient-centered care.  Like experiential learning, pharmacy residencies are necessary and cannot be replaced.

Organizations such as American College of Clinical Pharmacy (ACCP) and American Society of Health-systems Pharmacists (ASHP) have advocated for requiring pharmacy residencies for pharmacist involved in direct patient care, especially as the role of the pharmacist continues to expand.³ ACCP noted that pharmacy residents contribute to the profession through research and innovation.  This demonstrates the positive impact residency training has on patients, the institution, and the future of the profession.

Pharmacy residencies present an opportunity for residents to perform analyses. Analysis can help to determine the priority of patient needs, leading to safe and effective care based on the patient, clinical guidelines, best practices and the policy of the institution. In addition, analysis allows for residents to self-assess. Being able to analyze their strengths and weaknesses seems to play an important role, as residents continue to develop clinically. It allows the resident a chance to deepen their knowledge and focus in areas that need improvement. A union between experience and analysis can help to bridge the performance gap of residents, as they acquire the tools for continuous professional development, and transition to becoming independent practitioners.

As a resident, I have had the opportunity to be involved in pharmacist-led protocols, provide services in drug information to patients and health care providers, and continue to help shape the profession of pharmacy through my experiences, research and involvement in the profession. As medication therapy management (MTM), ambulatory care and other specialty areas of pharmacy become commonplace, I look forward to being at the fore front of our profession. My experiences have not strayed me away from the view that residencies are needed in order to maximize our fullest potential to provide patient-centered care.

References

1.  Schommer, JC, Planas, LG, Johnson, KA, et al. Pharmacists contribution to the U.S. health care system. Innovations in pharmacy. 2010; 1:1-8. 

2.  Crouch, M. Securing and excelling in a pharmacy residency. Burlington: Jones & Barlett Learning; 2013. Chapter 2, the value of residency training andvision for the future; p. 9-14.

3.  Murphy, JE, Nappi, JM, Bosso, JA, et al. American college of clinical pharmacy’s vision of the future: postgraduate pharmacy residency training as a prerequisite for direct patient care practice. Pharmacotherapy 2006;26:722-733.

But What is the Right Answer?

by Kellen Riley, Doctor of Pharmacy Candidate, University of Maryland School of Pharmacy

As pharmacy students advance through their didactic curriculum, a fundamental switch in learning occurs. While the first few years of classes consist largely of rote memorization - of brand-generic pairings, therapeutic classes, and anatomy – sometime during the second or third year, students are expected to begin synthesizing the knowledge previously gained to arrive at the “most appropriate” clinical decision. Aptly so, as these students are preparing themselves for careers as pharmacists in a world where most answers are not black and white.

When students begin to migrate away from memorizing facts toward synthesizing clinical decisions, they realize that the multiple-choice exam becomes less and less appropriate as an assessment tool. This phenomenon becomes most apparent to me when my classmates and I are engaged in case-based discussions in a course that uses multiple choice exams:

Imagine that you are the student. You value the discussion taking place and know that it is strengthening your understanding of the material.  But in the back of your mind, unavoidably looming, is the exam. You know that in a few short weeks, you will sit for a test in which there is literally only one best answer and you will be forced to sift through all the information you can recall about this topic and choose just one. Even though you understand that there isn’t one right answer, you also want to be able to do well on that test. Finally, one student raises his or her hand to ask the question that is on every student’s mind: “OK, but what is the right answer?”

Now imagine that you are the professor leading the discussion. You have just spent the past five minutes having a lively and engaged discussion.  Students are furiously typing notes and asking intelligent questions. After all the discourse in which you have obviously stated that this particular scenario is not quite clear-cut, to your dismay, a student raises his or her hand and asks, “OK, but what is the right answer?”

This question can be posed many different ways (“So, if we were to get this question on the exam…” or “If we had to pick only one answer…”), but in each case the theme is the same: in courses where multiple-choice examinations represent the majority of a student’s grade, the students’ priority is ensuring maximal success on these exams, even if the methods of achieving that goal seem to oppose the “critical thinking” atmosphere created by the instructor.

I know from personal experience that my classmates and I do not perform as well on multiple choice exams in case-based courses.  Despite the desires of the professor, many students prioritize exam preparedness over learning objectives and lively discourse, if they believe their grades will suffer. Some researchers have examined the effect of multiple choice exams on the learning environment.  They found not only that students’ perception of their learning was strongly correlated with the type of examinations used but also found that multiple choice exams can hinder critical thinking skills.^1,2  Some argue that multiple choice exams are not inherently bad assessment techniques, but that the most important consideration when evaluating students’ learning is that there is an alignment between the way the material is presented and the way it is assessed.

Thomas Reeves, a professor in the Department of Educational Psychology and Instructional Technology at The University of Georgia, examined the alignment between eight critical factors (goals, content, instructional design, learner tasks, instructor roles, student roles, technological affordances, and assessment), and their effects the learning environment.³ According to his research, the factor most often out of alignment was assessment, and he believes the blame falls on the instructor:

“Simply put, instructors may have lofty goals, share high-quality content and even utilise advanced instructional designs, but most assessment strategies tend to focus on what is easy to measure rather than what is important.”³

Reeves’ characterization of multiple choice exams might be a little unfair because multiple choice exams may in fact be the only feasible method for instructors to provide timely feedback to a large group of students. While every student wants the assessment to be “completely fair,” the second most predictable desire of students regarding exams is that they want to receive feedback “like, yesterday.”

While good grades are nice, students place importance on examinations that are an accurate reflection of their knowledge and skill.  In courses that require critical thinking and synthesis, multiple-choice examinations are probably not the best option. A meta-analysis performed by Kenneth Feldman reveals that when examining the relative importance of various instructional dimensions, both students and teachers rate the importance of “quality of examinations” equally.⁴ These data suggest that instructors do realize the importance of assessment, and that the logical next step is figuring out what methods an instructor can use to achieve better alignment between the information and skills to be learned and the assessment.

Although much depends on the material being presented, possible options within the health professional education include practical examinations, small-group interactions, debates, peer assessments, or one-on-one standardized patient interactions like the Objective Structured Clinical Examination (OSCE). One interesting avenue of assessment includes the multiple choice item development assignment (MCIDA), which requires students to “develop multiple choice items, write justifications for both correct and incorrect answer options and determine the highest cognitive level that the item is testing.”⁵ It is an interesting twist on the traditional multiple choice exam in that the students write the test instead of sitting for one. Although the author acknowledges that this assignment can be time-consuming to grade, using the MCIDA enhanced learning outcomes for students.

When it comes to higher learning, some classrooms feature a battle between a professor’s desire to lead a discussion in which students focus on the critical thinking process and students’ desire to perform well on an upcoming examination by focusing on “the bottom line.” If instructors can design their course in a manner that promotes alignment between the critical thinking activities of the classroom and the assessments used to test their students’ knowledge of that material, they help ensure student success – not only on exams, but in life.

References:

1.  Stehle, S, Spinath, B, Kadmon, M. Measuring Teaching Effectiveness: CorrespondenceBetween Students’ Evaluations of Teaching and Different Measures of StudentLearning. Res High Educ 2012;53:888–904.

2.  Stanger-Hall, KF. Multiple-Choice Exams: An Obstacle for Higher-Level Thinkingin Introductory Science Classes. CBE Life Sci Educ. 2012;11:294-306.

3.  Reeves, TC. How do you know theyare learning?: the importance of alignment in higher education. Int. J. Learning Technology 2006;2:294-309.

4.  Feldman, KA. Effective College Teaching from the Students’ and Faculty’s View: Matchedor Mismatched Priorities? Res High Educ 1988;28:291-344.

5.  Fellenz, MR. Using assessment to support higher level learning: the multiple choice item development assignment. Assessment & Evaluation in Higher Education. 2004;29:703-719.