Creating an Effective Feedback Environment to Enhance Students’ Field Experiences

by Clare Olin, PharmD, PGY1 Pharmacy Practice Resident, University of Mississippi Medical Center

Many students in higher education learn through field experiences, like internships or clinical “rotations.” During these experiences, an assigned preceptor helps the student to gain “on the job” experience to better understand their chosen profession. Experimental education is important because it provides students with an opportunity to use skills they initially learning about during didactic instruction. The preceptor reinforces the knowledge the student initially gained in the classroom.  Students also receive feedback from the preceptor who evaluates their performance.  Ultimately, these experiences and the feedback from their preceptors prepare them for their career. 

Feedback given by preceptors is meaningful in guiding a student’s growth. It’s important to create what Steelman and colleagues an effective “feedback environment.”¹ This is how feedback is delivered in both words and actions. Learners are more likely to accept and act on feedback when it is given well. There are seven variables that contribute to the feedback environment and influence whether the recipient will perceive the feedback as useful:¹

• Source credibility
• Feedback quality
• Feedback delivery
• Frequency of favorable feedback
• Frequency of unfavorable feedback
• Source availability  

Source credibility refers to how students perceive their preceptors. Students are more likely to acknowledge and implement the comments provided by people they highly respect.  Thus, a preceptor should be, first and foremost, a good professional role model.  Feedback is considered high quality when it is consistent, relevant, specific, and useful to the learner. Feedback delivery refers to both verbal and non-verbal communication. The reception of both positive and negative comments can be influenced by the tone, volume, word inflection, eye contact, and even body language of the deliverer. The frequency of favorable feedback can influence a student’s behavior and keep them motivated. Thus, positive reinforcement techniques can help engage students throughout their practice experiences and build positive attitudes and relationships. Positive feedback also builds students confidence and helps them develop independence. The frequency of unfavorable feedback is also important.  Constructive feedback is intended to help students improve and, when delivered effectively, can also keep students motivated. When unfavorable feedback is given, it is more likely to be taken positively if preceptors describe specific ways that the students’ performance can improve. Source availability refers to the how accessible the preceptor is – both physically and psychologically. Students are more likely to approach an accessible preceptor with questions and concerns. This can then help build a relationship between the two, and in turn, it may allow the feedback received to be more meaningful. While these seven principles were determined by observing employee relationships with their direct supervisors, the concepts are equally applicable to clinical instruction. In clinical teaching environments, students are preparing for a job and feedback strategies should be similar.² 

One study looked at the feedback strategies of preceptors in relation to student’s perceived feedback value.³ Students (n=132) from multiple states and with a variety of clinical experiences were asked to complete a survey. The survey consisted of three sections: demographic data, feedback environment scale (FES), and the feedback orientation scale (FOS).  The students were asked to rank statements about FES and FOS using a 5-point scale.  The FES section of the survey examined the following aspects of the environment:

• Source credibility
• Feedback quality
• Feedback delivery
• Favorable feedback
• Unfavorable feedback
• Source availability
• Promotes feedback-seeking behaviors

The FES represented how well students respond to the feedback “environment” that the preceptor created. The FOS included the student’s self-reflection on how they best use feedback and included:

• Utility
• Accountability
• Social awareness
• Feedback self-efficacy

The results demonstrated a significant positive correlation between the FOS and FES scores (p = 0.01), meaning that as the student’s perception of the feedback environment improved so did the perceived usefulness of the feedback given. When focusing on each element of the feedback environment, feedback utility was most affected by feedback quality and delivery. There was a significant relationship between accountability and the frequency of favorable feedback (p = 0.05), suggesting that consistency of feedback promotes the application of the preceptor’s assessments to the student’s daily work. Although utility, social awareness, and self-efficacy did not achieve statistical significance, there were positive relationships between each and the FES scores.

In order to implement these principles in practice, preceptors should start by being accessible and available to students. This should include providing phone number(s), email address, and reliable office hours or meeting times.  In addition, it is important to create a psychologically safe environment so that students feel comfortable reaching out. Regularly scheduled times for feedback can help the preceptor and the student stay on track. Preceptors should plan what to say in advance of these meetings and be prepared to give specific examples of both positive and negative observations.⁴ Preceptors should also encourage students to engage in self-reflection by using open-ended questions.  Finally, preceptors should give feedback promptly after a practice experience while the student can recall specific details.⁴

Keeping the seven dimensions of an effective feedback environment in mind can help new preceptors create positive field experiences. Constructive feedback should be specific and provide actionable strategies the student can use to improve their performance. Positive reinforcement should be given whenever a student has shown improvement. Effective feedback enhances students’ confidence and will enhance their ability to their professional responsibilities in the future.

References

1. Steelman LA, Levy PE, Snell AF. The feedback environment scale: construct definition, measurement, and validation. Educ Psychol Meas. 2004;64(1):165-184. doi:10.1177/0013164403258440.
2. Jonsson A. Facilitating productive use of feedback in higher education. Active Learn High Educ. 2013;14(1):63-76. doi:10.1177/1469787412467125.
3. Nolan T, Loubier C. Relating Instructor Feedback and Student Reception in the Clinical Environment. Radiol Technol. 2018 Jan;89(3):238-256.
4. Hardavella G, Aamli-Gaagnat A, Saad N, et al. How to give and receive feedback effectively. Breathe 2017; 13: 327–333.

Situational Judgment Tests (SJTs) to Measure Beyond Recall

by Karmen McMinn, PGY1 Pharmacy Community Practice Resident, Mississippi State Department of Health

In order to become a pharmacist, a student must receive a Doctor of Pharmacy degree. This means that they receive several years of education, but does all of that knowledge mean they will be successful? In addition to having a wide range of factual knowledge, pharmacists must also be able to display empathy towards patients and caregivers, work well in teams, and many other qualities that do not rely on the recall of facts.^1,2 These qualities (empathy, team player, etc) are sometimes be referred to as “soft” skills while being able to recall factual knowledge is often referred to as academic or cognitive skills.³ Some have argued that strong academic skills are inversely related to soft skills.^1,2 Thus, someone who earned straight A’s in those early science classes may struggle during advanced practice experiences.

One common requirement for pharmacy school admission is the Pharmacy College Admission Test (PCAT). The PCAT was designed to measure general academic ability and scientific knowledge. This background knowledge is something all students must have in order to be successful in pharmacy school. The problem with a test like this is that it only looks at a student's general academic knowledge while neglecting to assess other important skills a successful pharmacist must possess. For example, the PCAT does not assess a person’s ethical decision-making ability. This is where situational judgment tests (SJTs) might be useful. They can be an effective tool for assessing soft skills.^2,4

Icons made by Smashicons from www.flaticon.com

SJTs are a type of psychometric test in which learners are presented with a realistic scenario or situation. Examinees are then presented with 4 or more actions they could take and instructed to pick the option they believe is the most appropriate – an action they would take in real life. The test taker is often asked to pick not only the most appropriate response but also the least appropriate response. In some SJTs, they are asked to rank the relative appropriateness of the responses from most appropriate to least appropriate.⁵ SJTs can measure traits not related to a person’s ability to recall factual knowledge.⁶ This can be done by making sure questions ask what the person “would do” instead of what someone “should do.”⁷ Skills that can be measured on an SJT include personality traits like conflict management, interpersonal skills, teamwork, and cultural awareness.^3,5 All of these skills can help identify people who would make amazing pharmacists that have the ability to interact with a wide range of people and work effectively as a part of a healthcare team.¹

In one study, investigators at Monash University in Australia developed an SJT. They used experts to evaluate the tool’s validity, reliability, fairness, and to determine the appropriateness of using an SJT as a formative assessment. This study appears to be the first to report on the development, implementation, and evaluation of SJT as a formative assessment for pharmacy students. They developed the test to help identify students that might need more training to develop the soft skills integral to becoming a successful pharmacist.³

Here is an example SJT scenario and directions³

Scenario
Nikhil, a pharmacy student, is working in a community pharmacy. A customer explains to Nikhil that she came to the pharmacy yesterday to collect some blood pressure tablets. However, when she arrived home, she realized that she had been given double the strength of the tablet that she required and has not taken any of the new medications. Nikhil arranges for the pharmacist to correct the medication and apologizes to the customer for the error. However, the customer looks angry and says, “sorry is not good enough.”

Response Instructions
How appropriate are each of the following responses by Nikhil in this situation?

Options: 
1 = a very appropriate thing to do; 2 = appropriate, but not ideal; 3 = inappropriate, but not awful; 4 = a very inappropriate thing to do

Responses

Inform the customer that he has already apologized to her and that there is nothing more that he can do

1. Tell the customer that he was not working yesterday
2. Tell the customer that she needs to calm down
3. Ask the customer whether she would like compensation
4. Ask the pharmacist to come and speak to the customer
5. Provide the customer with information on the pharmacy’s formal complaints procedure

The potential advantages of using SJTs in health professional curricula include building a student’s understanding of the concept “best” and “better” ways of performance and increases self-assessment skills. Self-assessment skills are an important part of continuing professional development. Providing students with feedback and the opportunity for reflection can help motivate further development of these soft skills. It can also be helpful to students by administering multiple SJTs so that they can see their improvement over time.^3,6

There are a few issues that educators should consider before implementing SJTs. First, it is important to make sure the scenario or situation is well described. There must be enough information for a student to be able to fully visualize the scenario. If a student cannot envision the scenario, it will be difficult for them to pick the “best” answer. Secondly, it is best to develop a scenario that does not force a student to choose an action that would go violate their personal beliefs and values.^3,7

Educators can use SJTs to help develop skills and traits, such as interprofessional skills and cultural sensitivity, that help students become better pharmacists. These tests can be used as a tool to assist with admission decisions but also deployed repeatedly throughout the curriculum in order to document change over time. By using SJTs for formative purposes, an institution can personalize the development of soft skills, focusing the student’s attention on weaknesses as well as uncovering strengths. In the end, every school wants to graduate well-rounded and well-educated pharmacists.^1,3


References:

1. Gilchrist A. Top 5 Pharmacist Personality Traits. Pharmacy Times. 2015 July 23.
2. Jones J, Krass I, Holder GM, Robinson RA. Selecting pharmacy students with appropriate communication skills. [Internet]. Am J Pharm Educ 2000; 64(1): 68-73.
3. Patterson F, Galbraith K, Flaxman C, Kirkpatrick CMJ. Evaluation of A Situational Judgement Test to Develop Non-Academic Skills in Pharmacy Students. Am J Pharm Educ 2019 [Ahead of Print]
4. About the PCAT. [Internet]. Pearson. 2019. Cited 2019 Nov 12.
5. Situational Judgement Test [Internet]. Psychometric Tests. 2019 Jan 9. [cited 2019 Nov 11].
6. Austin Z, Gregory PAM. Evaluating the accuracy of pharmacy students’ self-assessment skills. Am J Pharm Educ. 2007;71(5): Article 89.
7. Assessment & Selection. Other Assessment Methods. Situational Judgment Tests. United States Office of Personnel Management. 2019. Cited 2019 Nov 21.