Peer Assessment: More Than Busy Work

By Anh Tran, Pharm.D., PGY1 Pharmacy Practice Resident, Medstar Union Memorial Hospital

Take a moment and think about a time when you were in high school or college and you were asked to assess your peers on their work.  Or vice versa.  I remember a time when I had just turned in a paper in an undergraduate English class.  The professor then informed us that we would be grading each other’s papers!  The first thought that went through my mind was, “This is just busy work!”  Actually, peer assessment can be a very effective learning tool. 

Peer assessment is the process whereby students receive a critical evaluation and feedback of their work from a similarly experienced individual, peer, or colleague.  This practice is commonly used in various settings, including pharmacy education.  For example, peer assessment can be used to evaluate a patient counseling session conducted by a student pharmacist or a pharmacotherapy presentation by a pharmacy resident.  Peer assessment plays a vital role in a pharmacist’s professional development, whether during school, experiential rotations, postgraduate training, or career. Furthermore, the practice of peer assessment promotes active learning, group work, and complex problem solving.

In addition to promoting these great aspects of learning, peer assessment has other distinct advantages.  Peer assessment enables faster and more detailed feedback.¹  How many times have you turned in an assignment and waited for weeks for the professor to grade it and provide feedback?  Most likely, you forgot your thought process through that assignment and thus, the feedback is no longer useful to you.  Instead, having peers grade each other’s assignments provides more timely feedback, which is more useful because the assignment and the students’ thoughts are still fresh in their minds.  In addition, since assignments are being reviewed simultaneously by multiple graders, there is the potential for more detailed and in-depth feedback.

Peer assessment might have some advantages from a teaching and learning point of view, but what are students’ attitudes towards it?  In a study conducted by Wu and colleagues, 91.9% of PharmD students surveyed believed that peer assessment is a skill that they will use in their pharmacy career.  In terms of student-to-student peer evaluation, 80% of students were comfortable providing an honest assessment to their partner and 95.7% of students were comfortable receiving it. Furthermore, only 34.4% of the students believed that the assessment of students is solely the responsibility of faculty and not students.²  In another study, Basheti and colleagues demonstrated that anonymous peer feedback in a pharmacy course is an effective means of providing constructive feedback on performance.  The study found that 78.1% of students felt that their participation in the peer assessment process helped them to deepen their understanding of the course content and 78% of students would endorse the use of this practice in other courses.³  Thus, students felt comfortable with peer assessment and perceived it as a valuable tool in their education.

Peer assessment is consistent with the principles of andragogy.  In other words, peer assessment takes evaluation from “teacher-driven” to “learner-driven”.  By taking assessment out of the teacher’s hands, students have yet another learning opportunity.¹  Peer assessment can lead to a deeper understanding of a topic by evaluating the work of others.³  For example, when I evaluated the English paper of an undergraduate peer, I was pleasantly surprised what I learned just from reading it!  We had written on the same topic, but we had different views and opinions.  By practicing peer assessment, students can discover other perspectives on a topic which can broaden their understanding.

Finally, peer assessment fosters metacognition, which is a knowledge or awareness of one’s own learning processes.¹  By participating in peer assessment, students are in a better position to understand the grading criteria.  Thus, they can then internalize this understanding and apply it to their future work and to improve their own performance.  For example, in a practice patient counseling session, a pharmacy student grading a peer would develop a better understanding of best practices and can then apply these criteria to his/her future counseling sessions.

While peer assessment has many great qualities, there are some concerns.  Can peer assessment truly serve as a substitute for the teacher’s assessment?  Are these assessments valid?  Falchikov and colleagues attempted to answer these questions by performing a meta-analysis comparing peer and teacher assessments in higher education.  The meta-analysis showed a mean correlation over all the studies to be r = 0.69, indicating reasonably good agreement between peer and teacher assessments.⁴  Similarly, Sadler and colleagues conducted a study to determine the agreement between the grades given by a teacher and those given by a peer.  This study showed that peer-grades were highly correlated with teacher grades (r =905)!¹

Assessment and evaluation are essential components of instructional design and peer assessment is a good way of engaging students in the classroom. Studies have identified ways to implement peer assessment by educators.   It’s important to provide training on the evaluation process to students and to provide clear criteria for peer feedback in order to avoid superficial comments.  In addition, professors should blind the reviews in order to reduce bias, since friendships may affect the accuracy of peer assessment.¹

When educators implement structured, unbiased approachs to peer assessment, it can play an exceptional role.  Not only is it an effective learning tool, but peer assessment can foster team work, active learning, and metacognition.  Students realize the importance of peer assessment and are comfortable participating in such a process.  So the next time your professor announces that you’ll be grading your peers, embrace it and further your learning!

References

1.   Sadler PM, Good E. The impact of self- and peer-grading on student learning. Educational Assessment. 2006; 11(1):1-31.

2.   Wu K, Davison L, Sheehan A.  Pharmacy students’ perceptions of and attitudes towards peer assessment within a drug literature evaluation course. Am J Pharm Educ. 2012; 76(4):62.

3.   Basheti IA, Ryan G, Woulfe J, Bartimote-Aufflick K. Anonymous peer assessment of medication management reviews. Am J Pharm Educ. 2010; 74(5):77.  

4.   Falchikov N, Goldfinch J. Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks. Review of Educational Research. 2000; 70(3):287-322.

360-Degree Feedback

by Andrea Passarelli, Pharm.D., PGY1 Pharmacy Practice Resident, The Johns Hopkins Hospital

Preceptors play an integral role in the development of pharmacists.  Most schools of pharmacy provide formalized training and education to their preceptors in order to help them master effective precepting techniques, but literature surrounding this topic is scarce.  Preceptors should use a variety of teaching strategies throughout a rotation and these should be tailored to the learner’s stage or professional development.¹  As learners grow in their ability and confidence, they should be performing more tasks independently, such as medication reconciliation, discharge counseling, and "rounding" with the medical team.  In these situations, the preceptor might use an assessment technique known as 360-degree feedback to evaluate the student’s performance.  The 360-degree feedback technique generally involves the preceptor asking other healthcare professionals as well as patients about the student’s performance.  This evaluation technique is gaining popularity in healthcare and other sectors.  I wanted to research this technique further so that I could use this assessment strategy during my career.

One paper published by Joshi et al. in Academic Medicine described the successful implementation of 360-degree feedback for obstetrics and gynecology residents at Monmouth Medical Center.² Residents were assessed on their interpersonal and communication skills by nurses, faculty members, allied health professional staff, medical students, patients, and co-residents.  In addition, each resident completed a self-assessment.  The researchers found good correlation between evaluations within each group of evaluators as well reasonably strong agreement among evaluators regarding each resident's rank among the peer group. Interestingly, there was a negative correlation between the rankings by faculty, staff, and medical students with the rankings given by peers.  The highest-rated residents (based on faculty, staff, and student evaluations) received low marks from their co-residents and vice versa.  This may perhaps be due to some perceived competition or a desire to “get ahead” by rating high-achieving residents poorly.  On self-assessment, junior residents typically rated themselves highly while senior residents rated themselves average or low.  This may have been because senior residents were setting higher standards for themselves or have increased self-awareness later in the curriculum. A potential advantage of 360-degree feedback employed at Monmouth Medical Center: evaluators were eager participants because their feedback was anonymous. The 360-feedback technique has been widely described in medical education journals, but has been most often been used in residency training rather than student education.^3-6  360-degree feedback can be an extremely effective tool for a preceptor, as learners will often communicate differently in the presence of a preceptor and when independently communicating with peers, staff (who may be perceived to be at different social ranks within the organization), and patients.

When I was a student I had one experience with 360-degree feedback during my first acute care rotation.  On the last day of the rotation, my preceptor allowed me to round independently with the team.  After the team finished rounding, my preceptor asked the medical interns, attending physicians, and nurse case-manager about my performance.  In subsequent advanced practice experiences, both as a student and resident, preceptors have occasionally sought feedback from physicians regarding my performance.  But most based my evaluation solely on their direct observations.

While my experience with the 360-degree technique from the perspective of a learner is limited, I believe there are some important points that should be considered prior to implementation.  First, it is important to inform the learner that this technique will be used.  Although the goal is to assess the learner’s performance without influence from the preceptor, learners might be taken aback if their performance is discussed with other individuals without their knowledge.  Further, it is important for preceptors to remember that in order to effectively implement this technique they must ask a breadth of individuals (with different roles) to participate in the process.  This was done during my first rotation, but not in subsequent rotations as preceptors only asked physicians for their input.  I have never had a preceptor ask one of my patients about their perception of my abilities.  And with the notable exception of my first rotation, my preceptors haven’t asked medical interns, medical students, or nurses for feedback.

360-degree feedback appears to be an incredibly valuable assessment tool that can enhance the quality of evaluations provided to both pharmacy students and residents.  This technique allows preceptors to more accurately assess a learner’s communication skills, especially in the absence of preceptor supervision.  Research has shown that this is an effective and accurate evaluation technique when used in medical residency training, but its use has not (yet) been described in the pharmacy education literature.  Preceptors utilizing this technique should be familiar with its fundamentals, and should ask individuals in multiple roles, including patients, for feedback regarding the learner’s performance.  When used appropriately, 360-degree feedback allows the preceptor a unique opportunity to obtain a complete picture of the learner’s strengths and can help them identify areas for improvement.

References

1.    McDonough RP, Bennett MS. Improving communication skills of pharmacy students through effective precepting. Am J Pharm Educ. 2006 ;70(3): Article 58.

2.    Joshi R, Ling FW, Jaeger J. Assessment of a 360-degree instrument to evaluate residents’ competency ininterpersonal and communication skills. Acad Med 2004;79:458-63.

3.    Meng L, Metro DG, Patel RM.Evaluation professionalism and interpersonal and communication skills:implementing a 360-degree evaluation instrument in an anesthesiology residencyprogram. J Grad Med Educ 2009;1 (2):216-20.

4.    Wood J, Collins J, Burnside ES et al. Patient, faculty, and self-assessment of radiology resident performance: a360-degree method of measuring professionalism and interpersonal/communicationskills. Acad Radiol 2004;11(8):931-9.

5.    Sorg JC, Wilson RD, Perzynski AT et al. Simplifying the 360-degree peer evaluation in a physical medicine andrehabilitation residency program. Am J Phys Med Rehabil 2012; 91(9):797-803.

6.    Chandler N, Henderson G, Park B et al. Use of a 360-degree evaluation in the outpatient setting: the usefulness ofnurse, faculty, patient/family, and resident self-evaluation. J Grad Med Educ 2010;2(3):430-4.

Taking the Fear Out of Feedback

by Connie Yoon, Pharm.D., Assistant Professor, University of Maryland School of Pharmacy

During my two years of residency training, I knew providing feedback to my preceptors regarding the rotation and their teaching behaviors was expected.  Yet for some rotations, this moment was filled with fear.  Fear that the receiver would misunderstand my intentions or that this may harm my relationship with this person.  Fear that what I said would be misused or turned against me.  Despite this fear, I provide them with honest feedback.  Why did I take the risk and perhaps face retribution? Because feedback is necessary for professional development and learning.  I consider feedback as an opportunity for quality improvement, not a reason for retaliation.  I should also clarify that while fear may be a roadblock to giving feedback, it is also a roadblock for those receiving feedback, particularly when the feedback is negative.¹  Feedback is a two way street.  In a field where feedback is critical to facilitating education and patient care, it is important to review methods for delivering and receiving constructive feedback.

To begin, we must define feedback in the context of teaching and learning.  Feedback involves “a process to collect information to determine whether or not successful teaching and learning has occurred.”²  Feedback is “a commitment between teachers and students for academic and professional development.”³   It is an important tool used by successful teachers to make adjustments to the teaching process by identifying areas that are successful and those that require improvement.

Feedback is strongly associated with students’ perceptions of the quality of education.⁴  Therefore, as educators, we must consistently give and request feedback from students.  However, the unfortunately reality is that students are infrequently requested to provide feedback regarding the teacher’s performance (and typically only at the end of a course!).  Conversely, teachers do not provide feedback to students as often as they'd like.⁵  Moreover, teachers tend to have an inflated view of their feedback skills.  This was evident in a study published in 2008 by Sonthisombat.⁵  This study surveyed Pharm.D. students and preceptors to compare the perceptions of preceptors to that of students in regard to the preceptor’s teaching behaviors.  Survey items included rating the teacher’s frequency in giving positive feedback for good work, inviting comments or criticisms of the preceptor, discussing student strengths and limitations, and evaluating and advising students of their progress.  On nine of the 47 items, preceptors’ ratings were significantly higher than that of student ratings.  The majority of the preceptors perceived themselves as “well done and adequate” in terms of providing feedback to students, providing positive feedback, and responding positively to student’s comments and suggestions.  A significantly lower percentage of students rated their preceptors as well done and adequate in these areas, revealing a disparity between preceptors’ and students’ perceptions.  This study identified several areas for preceptor development and providing feedback was clearly a prime component. As educators, if we are committed to our students and the quality education of their education, giving constructive feedback is one thing we must learn to do well.

In order to fulfill this commitment, we must create an environment that encourages giving and receiving feedback.  There are several characteristics of constructive feedback.^1-3  Feedback should be:

• Relevant.  Address the teaching and learning performance objectives identified for the learning activity.
• Objective.  Base feedback on facts or observations not on feelings. Keep comments impersonal as much as possible.
• Individualized.  The goal of the feedback is to meet the needs of the student or teacher and enable the receiver to maximize their potential.
• Helpful or informative. Provide suggestions for improvement.
• Respectful.  Respect the student and/or teacher’s integrity and needs.
• Confidential.  Do not involve intermediaries unless absolutely necessary.  Only the student and teacher should be involved in the feedback session.
• Encouraging.  The student and teacher should be motivated by a desire to improve their learning/teaching methods.  This will can help to reduce the hostile emotions commonly associated with feedback that is negative.

In general, feedback is better received when given in a timely manner so that recipients have a opportunity to improve.  More frequent feedback makes each encounter less emotional and feedback becomes a common, routine part of the learning process.  Learning expectations for the activity should be articulated early to reduce ambiguities in performance objectives and minimize surprises during the evaluation.  Negative feedback should be balanced with motivational statements and positive feedback.  In order to maximize the potential use of feedback as a learning tool, both the deliverer and receiver of feedback should be receptive.  Solicit feedback and express appreciation when receiving it to model an accepting behavior.¹  Most importantly, create an atmosphere of trust and respect when giving and receiving feedback.

Constructive feedback, given or received, plays a critical role in teaching and learning.  We should not allow fear to prevent us from employing feedback in our daily activities.  Providing feedback in a structured, constructive manner can help minimize fear and maximize its utility as an educational tool.

REFERENCES

1.    Hunter TS. Giving and receiving constructive feedback in pharmacy practice. Drug Topics [Internet]. 2009 Jun 1 [cited 2012 Aug 30].

2.    Ovando MN. Constructive feedback: a key to successful teaching and learning. Int J Educ Management 1994;8:19-22

3.    Hamid Y and Mahmood S. Understanding constructive feedback: a commitment between teachers and students for academic and professional development. J Pak Med Assoc 2010;60(3):224-227.

4.    Torres DM, Sebastian JL and Simpson DE. Learning activities and high-quality teaching: perceptions of third-year IM clerkship students. Acad Med 2003;78:812-814.

5.    Sonthisombat P. Pharmacy student and preceptors perceptions of preceptor teaching behaviors.  Am J Pharm Educ 2008;72(5):1-7

The Instructor’s Role in Group Work

by Eun-Young Lee, Pharm.D., PGY1 Pharmacy Practice Resident, Howard County General Hospital

Teamwork is relevant to my pharmacy practice and the skills I acquired as a student in group exercises in the classroom were critical for building the habits necessary for effective teamwork. Instructors, therefore, have an important role to assist students during group exercises.

Many educators agree that group work fosters social development, critical thinking, and problem solving skills.^1,2   These skill sets are desired by many employers and can equip the students for their future career endeavors. There are several ways educators can guide students to work effectively as a team. The instructor should be actively involved in mentoring the groups while providing enough opportunities for autonomy for students to find their own resolutions and answers. Furthermore, there are several things instructors should consider when incorporating group assignments within a course.  Instructors should provide a clear purpose for each assignment and offer strategies to help address non-contributing students.

Providing a Clear Purpose of the Assignment

Instructors should help students understand the purpose of each assignment by clearly stating expectations, goals, and objectives of the group activity.^2,3,5 Identifying learning objectives assists students to focus on what needs to be accomplished.

In addition, group assignments should purposefully cultivate interdependence.  Although the instructor is available to mentor students throughout the learning process, group projects should encourage student learning amongst their peers.⁵ In other words, collaborators of a team can learn to depend on their group members. There are four types of interdependences:⁴

• Goal interdependence: By achieving one common group goal, each individual believes he or she is able to fulfill his or her own goal.
• Reward Interdependence: Good performance of the entire group leads to a reward, such as a good grade.
• Role interdependence:  Each member has a specific role and function to fulfill within the group.
• Resource interdependence: Each member has specific resources or knowledge that must be contributed.  One person does not have all the resources to complete the task independently.

Interdependence in group assignments is critical. Healthcare providers within multidisciplinary teams depend on interdependence to provide the best treatment for patients.  This requires various health care workers to coordinate and contribute knowledge according to their specialty.

Addressing Problems Related to Non-Contributing Students

Ideally, each member should equally participate, but many groups face the problem of social loafers.¹ Educators should be aware that there are two types of non-contributing students, a struggler and a social loafer. Students categorized as strugglers have difficulty participating due to a lack of knowledge or are marginally behind in their understanding.¹ Loafers on the other hand, can be characterize as “free-riders” who make little effort to contribute to the group’s work.¹

Both types of non-contributing students may be mistakenly viewed as loafers.  However, approaches to assisting loafers versus strugglers are different.  Struggling students may benefit from supplementary support from the instructor.¹ Conversely, a loafer needs a different approach. Strategies such as using peer assessments, keeping individual diaries, and retaining logs of online cooperation may be applied to monitor participation.¹ These strategies allow contributing members to successfully complete their assignment without being hindered by the slackers.¹  Social loafers should receive feedback from their group and this may correct their behavior to fully participate.¹ A downside to confronting an inactive student by the other group members may lead to the alienation of the loafer or exclusion altogether. Usually, groups are able to resolve this problem without the instructor’s intervention.¹

Instructors should define loafers and strugglers prior to assigning the project to the class.  This allows team members to identify and address the issues of loafers and strugglers differently.  Also, instructors should provide counseling to remind students that everyone in the group possess different levels of skills and contribute in different ways.¹ This allows the students to recognize that individuals who initially struggle may become highly effective participants once their skills are developed.¹ Additionally, instructors should encourage students to assign roles that best suit their expertise or skill level. Thus, I believe that allowing opportunities for all students to contribute will build confidence and allow students to thrive in group settings.

Conclusion

Group assignments offer students opportunities hands on experience to develop teamwork skills. This begins with providing groups with clear instructions to help them understand the purpose of the assignment. Inevitably, during collaborative work, members of the group may face conflicts and issues.  Instructors should utilize strategies to facilitate both contributing and non-contributing students.  This allows groups to achieve a common goal, while giving them enough autonomy to solve problems on their own.

Overall, I believe there are several advantages of incorporating group work into classroom-based courses. Being able to contribute effectively as a team member is a desirable skill employers value.  As a student, I had many opportunities and I am grateful to have worked within many groups with different dynamics.

References:

1.  Freeman L, Greenacre L. An examination of Socially Destructive Behaviors in Group Work.  The Journal of Marketing Education. 2011; 33: 5-17.

2.  Burke A. Group Work: How to Use Groups Effectively. The Journal of Effective Teaching.  2011; 11: 87-95.

3.  Group Work. Bloomington: Indiana University; 2011. [cited 2012 Sept 7].

4.  Colbeck CL, Campbell SE, Bjorklund SA.  Grouping in the Dark:  What College Students Learn from Group Projects.  The Journal of Higher Education.  2000 Jan/Feb; 71: 60-83.

5.  Atwood A, Klurfeld A, Kotker J. Using Group Work Effectively: Graduate Student Instructors Teaching and Resource Center. [Internet]. Berkley: University of Berkeley; 2012 Jan 30. [cited 2012 Sept 10].