Patient Simulation and the Benefits to Student Pharmacists

Darby Pullen, Pharm.D., PGY1 Pharmacy Practice Resident, Baptist Memorial Hospital, North Mississippi

Pharmacy education is continually changing and faculty should be familiar with a variety of models to teach students. During my time in Pharmacy School, patient simulation mannequins were used to demonstrate and practice hands-on techniques that I might later use in practice, such as how to give glucagon when a patient with diabetes is hypoglycemic and not responsive.

Students today are exposed to more stimuli and expect more than a typical PowerPoint presentation. Mannequins can mimic several disease states such as cardiac arrest and allergic reactions. Students can gather a variety of clinical information from a mannequin including a pulse, heartbeat, lung sounds, and other hemodynamic parameters.¹ Virtual patients are another form of simulation that can enhance student learning. Virtual patients are created in digital, computer-based environments.

Simulation is a way to bridge the foundational knowledge that is taught in the classroom to hands-on experience. Moreover, some students prefer hands-on activities in order to fully understand different disease states. Mannequins can benefit the students who are visual, kinesthetic, or auditory learners.¹ This would, in turn, enhance the pharmacy student’s knowledge, confidence, and clinical skills.¹

Simulation can enhance students’ learning process.¹ Using patient simulation requires more active student participation in a low-risk environment.¹ Students are able to have more hands-on activities that simulates real-life scenarios but without risking actual patient harm. Facilitators are usually available to correct student behavior when they are interacting with the mannequins and computer simulators can provide ongoing feedback.¹

According to the ACPE accreditation standards for pharmacy schools, “graduates must possess basic knowledge, skills, and abilities to practice pharmacy independently, at the time of graduation."² ACPE allows up to 20% of the students’ required hours of practice experience to be fulfilled using simulation.² By including more patient simulation in the classroom, it would be a win-win situation. Students would have greater access to semi-authentic learning exercises and would fulfill requirements for ACPE.

Faculty at the Universiti Teknologi MARA in Selangor, Malaysia evaluated the difference in students’ performance when they were taught using a human patient simulator compared to a problem-based learning activity.³ The investigators were teaching students about diabetic ketoacidosis and thyroid storm. This was a crossover study. Students were randomized to receive instruction using either a human patient simulator or a problem-based learning activity for one of the disease states and then crossed-over to receive instruction using the alternative instructional method for the second disease state. After each case, students were assessed using a posttest immediately after the lesson followed by a knowledge retention test 10 weeks later. The knowledge retention test included recall and application questions.

On follow-up surveys, students indicated they were more satisfied learning using the patient simulation rather than problem-based learning.³ The posttest and knowledge retention scores were significantly higher (p < 0.05) in the mannequin group (mean scores: posttest 78.5; knowledge retention 58.5) for the thyroid storm case when compared to problem-based learning group (mean scores: posttest 75.1; knowledge retention 53.5).1 This study suggests that human patient simulation may be more effective for long-term knowledge retention than problem-based learning.

In another study, investigators at Griffith University in Queensland Australia assessed the role of virtual (simulated) patients in pharmacy education. Specifically, they examined how students feel about the experience as well as how well they performance on knowledge tests and assessments of clinical skills.⁴ They performed a meta-analysis and found nine studies that compared virtual patients to traditional teaching methods. Their findings showed that the use of virtual patient to teach about therapeutic topics was not superior to traditional teaching in terms of student performance, but does improve the students’ reaction to the learning experience.⁴

Simulator mannequins are expensive. They can run anywhere from 16 to 90 thousand dollars.³ Plus there are additional costs for maintaining the mannequins and hiring qualified operators. This is a high cost for pharmacy schools to add to their budgets. However, schools can consider collaborating with other health professional schools. Nursing and medical programs often use patient mannequins. By using them for interprofessional activities, this not only improves pharmacy student knowledge but also teaches them about team member roles. By having more interprofessional interactions, medical and nursing student benefit as well.

Using mannequins and virtual patients, students must use critical thinking skills in addition to their baseline pharmacotherapy knowledge.⁴ Students can be placed into a high-stress environment but the stakes are low. You don’t want students having their first interaction with a real patient while still trying to learn about a new disease state. Students can be prepared for the experience by completing an orientation exercise and continue to repeatedly practice with the mannequins. Using virtual patients and mannequins, pharmacy students would have greater confidence when advancing into their advanced practice experiences.

There is some evidence that simulator mannequins improve students’ learning, particularly long-term knowledge retention.^4,5 Students respond favorably to learning activities involving mannequins and virtual patients because they are low risk. Traditional learning methods are still needed but simulator mannequins and virtual patients can help students build their confidence. Simulator mannequins should be considered when there is a hands-on technique that students need to master, such as working as an interprofessional team during a code or administering medications. Virtual patients can further pharmacy student’s knowledge and clinical decision-making skills.

References

1. Vyas D, Wombwell E, Russell E, et. al. High-Fidelity Patient Simulation Series to Supplement Introductory Pharmacy Practice Experiences. American Journal of Pharmaceutical Education. 2010; 74 (9) 169. DOI:10.5688/aj7409169
2. Accreditation Council for Pharmacy Education: Policies and Procedures for ACPE Accreditation of Professional Degree Programs – January 2010. Accessed 30 April 2019.
3. Chin KL, Yap YL, Lee WL, et. al. Comparing Effectiveness of High-Fidelity Human Patient Simulation vs Case-Based Learning in Pharmacy Education. Am J Pharm Educ 2014; 78 (8) Article 153. DOI:10.5688/ajpe788153 
4. Baumann-Birkbeck L, Florentina F, Karatas O, et. al. Appraising the Role of the Virtual Patient for Therapeutics Health Education. Currents in Pharmacy Teaching and Learning. 2017; 9 (5): 934-944. DOI:10.1016/j.cptl.2017.05.012
5. Seybert, AL. Patient Simulation in Pharmacy Education. Am J Pharm Educ 2011; 75 (9) Article 187. DOI:10.5688/ajpe759187

Key Elements to Consider When Developing Interprofessional Education Experiences

by Chase Board, Pharm.D., PGY1 Community Pharmacy Resident, University of Mississippi School of Pharmacy

My pharmacy school’s curriculum required each student to participate in one interprofessional education (IPE) training session and to write a self-reflection on the experience.  My IPE session occurred during my third year.  Working in a group with nursing and osteopathic students, we were tasked to provide an assessment and develop a treatment plan for a patient case scenario.  The instructors prefaced the activity to encourage us to focus our communication and teamwork skills.  For me, it seemed we struggled to find any purpose in learning how to work as a team.

The Joint Accreditation of Interprofessional Continuing Education defines IPE as events “when students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes.”¹  Highly effective interprofessional teams can improve health outcomes by enhancing the quality of patient care, reducing medical errors, reducing the hospital length of stay, and lowering medical costs.²  In a crossover study evaluating the quality of written transition plans for patients within a healthcare delivery system, treatment plans written by healthcare professionals who collaborated as a team correlated to stronger work quality when compared to healthcare professionals who did not actively collaborate together.³

Thanks in part to this growing body of literature, many academic institutions are implementing IPE.  Indeed, the Accreditation Council for Pharmacy Education (ACPE) requires all colleges/schools of pharmacy to implement IPE in their curricula.  In a January 2018 update, the ACPE Board of Directors clarified guidance to schools of pharmacy curriculum to include IPE experiences involving prescribers, specifically physicians and medical students, in didactic and experiential courses.⁴

Preceptors at the University of Wisconsin-Madison School of Pharmacy implemented IPE with a “dual identity” design.5  Their program trains students to become traditional pharmacists, but they integrate interprofessional learning and socialization with other healthcare disciplines.  The program offers elective experiences allowing students to volunteer at a student-run clinic.  These opportunities allow students to learn and apply interprofessional skills such as cooperation, communication, and coaching.  The aim is to prepare students for their fourth-year Advanced Pharmacy Practice Experiences (APPE) and graduate “career-ready, collaborative pharmacists.”⁵

Preceptors should structure their IPE experiences based on the four core competencies outlined by the Interprofessional Education Collaborative (IPEC).  IPEC is composed of 21 national associations representing nearly all health professions educators, including the American Association of Colleges of Pharmacy (AACP).  IPEC serves to promote and advance the efforts of interprofessional learning experiences.  The recommendations provided by IPEC focus on four core competencies:⁶

1.   Values/Ethics for Interprofessional Practice

Team members should have mutual respect and shared values. This is a cornerstone of effective collaborative practice.

2. Roles/Responsibilities

Individuals should use their knowledge in collaboration with the other members of the team to address patient health care needs.

3. Interprofessional Communication

Team members should promote and maintain the health of their patients by engaging in effective team-based communication.  They should direct their dialogue towards the patient, the patient’s family, other members of the health care team, and the community.

4. Teams and Teamwork

Teams should discuss relationship-building values and team dynamic principles to plan, deliver, and evaluate patient care.

 

For example, during a fourth-year APPE, I served as a facilitator in a structured IPE experience.  Groups of pharmacy, nursing, and osteopathic medicine students were assigned to review a patient case, communicate electronically, and attend a face-to-face session.  The face-to-face activity was intended to simulate an actual patient encounter.  My role as an IPE facilitator was to observe each team during the simulation and record comments based on my observations. I was provided a rubric to assess IPEC core competencies such as communication, professionalism, roles/responsibilities assignments, and team cohesiveness.  At the end of the simulation experience, groups received feedback regarding the written progress notes and electronic communication.  They were asked to write self-assessments based on their individual and team-based performance. I believed this activity represents a good example of how to structure an IPE experience.

There are many assessment tools available to evaluate IPE experiences. When structuring an IPE experience, it is important to identify the type of tools available, as well as determine which tools are most effective.  The Committee on Measuring the Impact of Interprofessional Education on Collaborative Practice and Patient Outcomes suggests IPE experiences should utilize a mix of qualitative and quantitative assessment tools to evaluate outcomes.  Implementing only one methodology does not fully explore and capture the data that can be obtained by a mixed-methods approach.  A mixed-methods approach can provide insight into both the “what” and “how” of IPE interventions and outcomes.⁷

When selecting tools to use for IPE assessment, the assessment tools should measure attitude, behaviors, knowledge, and skills.⁸ This aligns with the IPEC four core competencies (values/ethics, roles/responsibilities, interprofessional communication, and teamwork).  The IPE experience I helped facilitate during my fourth-year APPE used three assessment tools to evaluate attitudes, perceptions, and behavior.⁸ Assessment tools like these can be found on the National Center for Interprofessional Practice and Education website.

Preceptors should role model effective interprofessional collaboration as a strategy to teach pharmacy students appropriate behavior to uphold during IPE experiences.  Including activities in a rotation whereby students observe and reflect properties can help them develop conscious behavior.⁹  Some strategies preceptors should use include being self-aware of being a role model, reserving time for discussion with students, facilitating reflection on the experiences, and explicitly communicating what is being modeled for the student.¹⁰

I’ve had the opportunity to role model for students at a local free clinic during my residency training.  A medical student was consulted to counsel a patient who was starting insulin therapy.  The medical student said they were struggling to remember subcutaneous injection technique.  I reviewed the proper insulin injection technique with her. Feeling more competent, the student counseled the patient and returned to me to reflect on her experience.

Functioning effectively on an interprofessional team is important.  Thus, we need to teach these skills to students while they are in school.  IPE experiences should be linked to the four IPEC core competencies. When developing IPE assessment, teachers should use quantitative and qualitative methods to evaluate student performances and experiences.  Role modeling can be used by facilitators and preceptors to demonstrate what effective behavior looks like in collaborative practice.

References

1. Definitions. Joint Accreditation Interprofessional Continuing Education website. 2019. Accessed May 15^th, 2019.
2. Buring SM, Bhushan A, Broeseker A, et al. Interprofessional education: definitions, student competencies, and guidelines for implementation. Am J Pharm Educ. 2009;73(4): Article 59.. Accessed May 15^th, 2019.
3. Farrell T, Supiano K, Wong B, Luptak M, Luther B, et. al. Individual versus interprofessional team performance in formulating care transition plans: A randomised study of trainees from five professional groups. J Interprof Care. 2018;32(3):313-320.
4. Clay Kirtley J., Vlasses P. ACPE Update – 2018. Oral Presentation at: American Pharmacists Association Annual Meeting; March, 2018. Nashville, TN. Accessed May 16^th, 2019.
5. Gerhards K. PharmD Program Strengthens Interprofessional Education. University of Wisconsin-Madison School of Pharmacy’s website. 2018. Accessed Mary 17^th, 2019.
6. Interprofessional Education Collaborative. Core competencies for interprofessional collaborative practice: 2016 update. 2016. Washington, DC: Interprofessional Education Collaborative. Accessed May 19^th, 2019.
7. Chapter 5: Improving Research Methodologies. Measuring the Impact of Interprofessional Education on Collaborative Practice and Patient Outcomes. Washington (DC): National Academies of Science. Published December 2015. Accessed May 19^th, 2019.
8. Shrader S, Farland M, Danielson J, Sicat B, Umland E. A Systematic Review of Assessment Tools Measuring Interprofessional Education Outcomes Relevant to Pharmacy Education. Am J Pharm Educ. 2017;81(6): Article 119. Accessed May 19^th, 2019.
9. Cruess S, Cruess R, Steinert Y. Role Modelling – making the most of a powerful teaching strategy. BMJ. 2008;336(7646):718-721.

Assessing Students on Advanced Pharmacy Practice Experiences

by Taylor Loper, Pharm.D., PGY1 Pharmacy Practice Resident, Baptist Memorial Hospital-North Mississippi

One thousand, four hundred and forty. That’s the minimum number of Advanced Pharmacy Practice Experience (APPE) hours the Accreditation Council for Pharmacy Education (ACPE) mandates a pharmacy student receive before they graduate.¹  Over the course of at least thirty-six weeks (and sometimes more), these students are exposed to a variety of practice settings, institutions, and situations designed to support the development of their knowledge, skills, and abilities.  Essentially, their competence to practice as a pharmacist following graduation. The ACPE Standards state that the APPEs should be designed to “hone practice skills, professional judgment, behaviors, attitudes and values, confidence, and sense of personal and professional responsibility.” Schools are also required to have a formal assessment of the achievement of APPE competencies using validated assessments. Student performance must be documented at key time points throughout these experiences.

https://pharmacy.uiowa.edu/students/academic-programs/doctor-pharmacy/pep/appe

The CAPE educational outcomes developed by the American Association of Colleges of Pharmacy (AACP) are a set of goals that all pharmacy curriculums should be focused on achieving. These educational outcomes are linked to Entrustable Professional Activities (EPAs) that all pharmacy graduates should be able to perform.² (For more on EntrustableProfessional Activities, see the post by Andrew Mays) Students should have ample opportunity to practice these activities become proficient, and demonstrate mastery before becoming licensed pharmacists. This can be problematic in that the heavily didactic nature of the first three years of the pharmacy curriculum results in few opportunities to practice these EPAs.  Too often students are often being assessed before they have an opportunity to master the EPAs during APPE rotations.

Reflecting back on my experiences as a pharmacy student, the assessment of learning on APPE rotations involved a series of assignments that had to be completed by the end of the rotation.  An example of this would be a set of questions asking you to reflect on interprofessional teams and the benefits of working with different professions at your current practice site. While most assignments were site-specific, several (like the one above) were repeated for multiple rotations. Additionally, specific objectives were set forth and students were asked to provide evidence of assignments or activities they completed that enabled them to meet those objectives. An example of this would be to “evaluate and interpret patient data.” A student could then provide details of working patients up, reviewing medical records, or conducting patient interviews. This gave students an opportunity for reflection while providing concrete examples of progress that the APPE preceptors could then base their end-of-rotation evaluations on. However, completing these assignments and documenting these examples was often time-consuming. By the final APPE, they felt cumbersome, especially the assignments that we had to repeatedly do on multiple rotations.

This process of assessing student performance raises several questions. First, how do we ensure each student meets the required competencies for each rotation? With practices settings and sites varying significantly, assessing each student on basic competencies can be difficult. Moreover, different preceptors have different expectations. All this variability makes it very difficult to create a consistent assessment process that is not dependent on the student’s learning experiences. Second, how do we assure an assessment tool can be applied in a variety of APPE rotations without omissions or redundancy? Requiring the same assignments and reassessing the same set of skills for a student who is taking two community rotations puts a strain on the student and preceptor. But we must find a way to ensure the student is developing on each rotation. Finally, how do we measure competency, such as the EPAs?  Should be rated “acceptable”?  Or “completed”? Should a student be required to “complete” them by the end of each APPE or by the end of all APPE experiences?

Several institutions have tried to address these questions. The System of Universal Clinical Competency Evaluation in the Sunshine State (SUCCESS), is an internet-based APPE assessment tool created by the colleges/schools of pharmacy in Florida.³ Under this system, preceptors rate students as “excellent”, “competent”, or “deficient” for each competency at the end of each APPE. They are also allowed to select “no opportunity” if not observed. These ratings were then converted by the school to determine the student’s grade.  There was a correction factor for students that were earlier along during their APPE schedule. It also allowed preceptors to weigh each competency based on importance and frequency in the practice setting / site. This weight provides preceptors the ability to focus on the learning goals that are most relevant. Another such tool was created by faculty at the University of Colorado Skaggs School of Pharmacy after the addition of 14 ability-based outcomes to their curriculum.⁴ By polling current preceptors, they were able to determine which competencies and outcomes were frequently observed and how important they are to the success of students on each APPE. These responses were used to create APPE-specific tools to ensure students met rotation goals that aligned with the ability-based outcomes of the curriculum.

It’s clear that assessing the performance of APPE students is a crucial, yet complex, task. Based on the two methods documented above, implementing an effective evaluation method requires the active participation of preceptors in developing a tool that is specific to each APPE experience. Preceptor evaluations of students need to be specific to the setting and site but must also relate to the overarching ACPE standards and ACCP outcomes.

I believe that monthly preceptor evaluations of students and their progress toward or achievement of learning objectives are necessary to ensure each APPE experience is helping to develop the student’s competence. However, rather than completing a series of monthly (and sometimes redundant) assignments, a series of unique assignments completed over the ENTIRE year coupled with specific ability-based assessments might be a better strategy.  This can reduce assignment fatigue and still provide appropriate documentation that each student can competently perform the EPAs and other educational outcomes before they graduate. It would great to see some research to determine the validity of this approach.

References

1. Accreditation Council for Pharmacy Education. Accreditation Standards and Key Elements for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree (Standards 2016). 2015.
2. Haines ST, Pittenger AL, Stolte SK, et al. Core Entrustable Professional Activities for New Pharmacy Graduates. Am J Pharm Educ. 2017; 81(1): Article S2.
3. Reid LD, Nemire R, Doty R, et al. An Automated Competency-Based Student Performance Assessment Program for Advanced Pharmacy Practice Experiential Programs. Am J Pharm Educ. 2007; 71(6): Article 128.
4. Gilliam EH, Brunner JM, Nuffer W, et al. Design and Content Validation of Setting-Specific Assessment Tools for Advanced Pharmacy Practice Experience Rotations. [published online ahead of print March 6, 2019] Am J Pharm Educ. Article 7067.

Social-Class Achievement Gaps in Higher Education: Can Values Affirmation Interventions Help?

by Lily Van Cheng, PharmD, PGY1 Community Pharmacy Practice Resident, University of Mississippi School of Pharmacy

As an underrepresented minority (URM) and first-generation (FG) college student, the psychosocial factors that influence one’s success at the collegiate level of education is both fascinating and frightening. FG college students comprise roughly 15-20% of students in American universities today.¹ FG students are more likely to come from working-class backgrounds and face significant economic and psychosocial barriers that create performance discrepancies called the “social-class achievement gap.”² The performance gap might be the result of poverty, the rigor of high school preparation, parenting practices, and/or cultural mismatches. None-the-less, it is arguable that the gap between FG students and continuing-generation (CG) students are merely the results of differences in baseline academic preparation or readiness.

Martin Leon Barreto for The Chronicle Review

A tool that some educators have used to address these challenges has been the values affirmation (VA) intervention.^3,4 VA interventions are designed to address the students’ perceived “stereotype threat.”  FG students are more likely to be confronted with stereotypes that threaten their identity and self-esteem which affect their academic performance. The VA intervention technique addresses stereotype threat by asking students to reflect and write about their most important values. It is hypothesized that this practice enhances the student’s ability to cope with internal identity threat and reaffirms their core values to reestablish their personal integrity and worth. In one study conducted with middle school students, a VA intervention significantly improved the grades of Latino students. The grades of white students were not impacted. The VA intervention thereby partially closed the achievement gap for URM students.⁵

In a more recent study conducted at the University of Wisconsin – Madison, researchers evaluated the role of a VA intervention comparing the performance of FG versus CG college students in a double-blinded randomized experiment in an introductory biology course.⁶ Outcome measures included confidence in their innate academic abilities and perceived concern about their generational background on academic success.  In addition, the researchers compared final course grades, overall GPA in other courses (excluding the biology course), and rate of continuation in the second-semester biology sequence. Students were randomized in blocks based on a variety of characteristics, including generational and URM status. In both the VA intervention and control groups, there were FG and CG students.  All students in the VA intervention were instructed to identify and write about values that were most important to them.  Students in the control group were instructed to identify values least important to them and write about why these values would be important to someone else.

The results?  The researchers found a significant generational status effect. While FG students obtained lower grades than their CG counterparts in the same biology class (p < 0.01), the VA intervention led to significant improvements in the FG students grades (p < 0.05), resulting in a 50% reduction in the social class achievement gap. In terms of progression into the second-semester biology course, in the control group, CG students (77.7%) were significantly more likely to enroll in the second course in comparison to FG students (66.2%).  Conversely, in the VA intervention group, FG students (85.7%) were more likely to enroll than CG students (74.8%). This represents a 20% increase in enrollment for FG students (p < 0.01) who participated in the VA intervention.  In contrast, CG students were no more likely to enroll regardless of whether they were in the intervention or control group (p = 0.41). The results suggest that a VA intervention can indeed narrow the social class achievement gap, improve the success for FG students in an introductory biology course and other college classes, and help keep them on track to progress in the science sequence.

Factors that threaten a student’s motivation or ability to learn vary from classroom to classroom, but it is vital that educators identify the variables that might influence a student’s success. In addition to the generational differences, other variables such as ethnicity, sex/gender, stress, and cultural mismatch may influence a student’s ability to academically succeed.^7,8 Learners come from different backgrounds and have individual struggles. Some are pretty obvious such as ethnicity and language. But others, like generational differences in educational attainment, are harder to identify and trickier to address. Supporting our learners so they can succeed to the best of their ability starts with acknowledging that barriers exist and doing our best to address those barriers. Whether an achievement gap is the result of stereotype threat or a cultural mismatch, VA interventions can play a positive role in influencing our learners’ success.

As healthcare providers, we strive for ways to bridge the health disparities that exist between people of different social classes. As health professional educators, shouldn’t we be striving for ways to bridge the academic disparities that exist? Taking a 10-minute check-in with our students using a VA intervention could be the difference that a student needs to succeed. I challenge every educator to try this in their classroom. Take 10 minutes at the beginning of class every month to have your students identify and write about what positive traits they value. Is it empathy? Compassion? Athleticism? It doesn’t matter if it’s for a grade or not. But portray it in a way that the students realize it is important to really give it honest thought. We spend so much time teaching what they lack or don’t know. It’s time we start reminding and reaffirming our students that what they currently know or possess is just as important. When we help our students reaffirm interdependent values they perceive as integral to their self-worth, we will see positive improvements in and out of our grade books.

References

1. Saenz, VB.; Hurtado, S.; Barrera, D.; Wolf, D.; Yeung, F. First in my family: A profile of first-generation college students at four-year institutions since 1971. Los Angeles, CA: Higher Education Research Institute; 2007. http://www.heri.ucla.edu/PDFs/pubs/TFS/Special/ Monographs/FirstInMyFamily.pdf
2. Snibbe AC, Markus HR. You can’t always get what you want: Educational attainment, agency, and choice. Journal of Personality and Social Psychology 2005; 88:703–720.
3. Cohen GL, Garcia J, Apfel N, Master A. Reducing the racial achievement gap: A social-psychological intervention. Science 2006; 313:1307–1310.
4. Sherman, DK.; Cohen, GL. The psychology of self-defense: Self-affirmation theory. In: Zanna, MP., editor. Advances in experimental social psychology. Vol. 38. San Diego, CA: Academic Press; 2006. p. 183-242.
5. Sherman DK, Hartson KA, Binning K, Purdie-Vaughns V, Garcia J, Taborsky-Barba S, Tomassetti S, Nussbaum AD, Cohen G. Deflecting the trajectory and changing the narrative: How self- affirmation affects academic performance and motivation under identity threat. Journal of Personality and Social Psychology 2013; 104:591–618.
6. Harachiewicz JM, Canning EA, Tibbetts Y, Giffe CJ, Blair SS, Rouse DI, Hyde JS. Closing the Social Class Achievement Gap for First-Generation Students in Undergraduate Biology. Journal of Educational Psychology 2014; 106(2): 375-389.
7. Smart-Richman L, Leary MR. Reactions to discrimination, stigmatization, ostracism, and other forms of interpersonal rejection: A multimotive model. Psychological Review  2009; 116:365–383.
8. Steele CM, Aronson J. Stereotype threat and the intellectual test performance of African Americans. Journal of Personality And Social Psychology 1995; 69:797–811.

-->