Should TBL Readiness Assurance Tests Be Graded?

by Kaylee Hall, PGY1 Pharmacy Practice Resident, University of Mississippi Medical Center

Team-based learning (TBL) is a learning strategy that requires students to prepare for topics prior to class and be ready to apply that knowledge to solve problems in a group setting. In professional schools, this is typically done by working through patient cases and clinical questions as a team. TBL is structured so the class starts with an Individual Readiness Assurance Test (iRAT), typically consisting of multiple-choice questions designed to test if the student is prepared to work through the cases/questions during that class. Following the iRAT, students then complete the group Readiness Assurance Test (gRAT), which consists of the same questions from the iRAT; however students work together in groups to answer the questions, facilitating group discussion and problem-solving. Students typically receive immediate feedback on the gRATs to promote conversations within the group and a deeper understanding of the subject matter. While the RATs are an important part of the learning process, controversy exists on whether the RATs scores should be counted towards students’ grades or counted as participation (i.e., the student is awarded full credit for being present and completing both the iRAT and gRAT).¹

A recent study conducted at the University of Tennessee College of Pharmacy assessed student performance when RATs were graded versus ungraded in a TBL course. They compared results from an elective course offered in the spring of 2020 and 2021. RAT scores were graded in 2020 and not graded in 2021. After the 2021 course, students were asked to take a survey to assess class preparation and perceived team accountability. The investigators found no significant difference in student exam performance when comparing the ungraded versus graded cohorts and concluded that ungraded RATs did not adversely impact students' examination performance.²

Pros to counting RAT scores

Assessment drives learning. Having RAT scores count towards the students' grades may provide them with extrinsic motivation to complete the pre-class materials which are essential for TBL. Previous investigations have looked at how grading iRATs affected class preparation and performance. They found that when iRATs were graded, students were more likely to download pre-class materials and performed significantly better on iRATs. When they compared the download frequency of preclass materials, it dropped by about 30% for Year 1 and by nearly 50% in Year 2 courses when iRATs were ungraded compared to graded.³

Cons to counting RAT scores

An assessment focus rather than a mastery focus. Assessments may promote cramming of material and superficial learning of the subject in order to pass the quiz or examination without truly understanding the concepts.⁴ Grading RATs may also encourage students to use a performance or performance-avoidance approach to learning, where students focus on the assessment with the goal to perform well or outperform their peers. This orientation toward learning promotes superficial learning of the material without encouraging students to master the subject.⁵

Negative effects on group cohesion. Grading RATs may push students to focus on individual efforts instead of working as a team. Poor group cohesion may promote social loafing where students give less effort because they can rely on other members of the team to do the work.⁶ In contrast, not grading RATs may encourage students to work together to achieve mastery of the subject. Without the need to achieve a grade, group assessments encourage students to prepare for the materials based on their intrinsic motivation and the desire to contribute their ideas. When students are intrinsically motivated, they tend to be more engaged and stay engaged longer than students who are extrinsically motivated.

Indeed, student surveys have found that ungraded cohorts feel more responsible for the team and have a greater desire to contribute to the group’s work. Students also report that they felt their contributions were important, indicating good group involvement and cohesion. Additionally, the majority of students reported that class preparation is necessary to perform well in the course.²

Increased pressure to perform. The pressure to perform well on assessments may encourage academic dishonesty and promote unneeded stress. Without the pressure to perform, students are able to focus on a deeper understanding of the material and are less likely to engage in superficial learning simply to perform well on examinations.

Conclusion

Few studies have looked at student performance when RATs are graded versus ungraded in the Team-Based Learning model.  Professional opinions differ on which approach is best. Historically, we know that assessment drives learning, but graded assessments may not be the most appropriate approach to the team-based learning strategy and may have unintended consequences. Grading readiness assurance tests may promote superficial learning of the material, lead to poor group cohesion and inflict unnecessary stress on students.  At least one study suggests that ungraded RATS does not harm student grades. Removing grades may diminish extrinsic motivation for students to prepare for readiness assessments but allows students to foster their intrinsic motivation — to be motivated more by the desire to contribute to the group and to master the material. More evidence is needed to truly assess the pros and cons of graded versus ungraded readiness assurance tests in professional schools.

Resources:

1. Hrynchak P, Batty H. The educational theory basis of team-based learning. Med Teach. 2012;34(10):796-801.
2. Eudaley ST, Farland MZ, Melton T, et al. Student Performance on Graded Versus Ungraded Readiness Assurance Tests in a Team-Based Learning Elective. Am J Pharm Educ. 2022;86(9): 8851.
3. Koh YYJ, Rotgans JI, Rajalingam P, et al. Effects of graded versus ungraded individual readiness assurance scores in team-based learning: a quasi-experimental study. Adv Health Sci Educ Theory Pract. 2019;24(3):477-488.
4. Epstein RM. Assessment in medical education. N Engl J Med. 2007;356(4):387-96.
5. Meece JL, Anderman EM, Anderman LH. Classroom goal structure, student motivation, and academic achievement. Annu Rev Psychol. 2006;57:487-503.
6. Karau SJ, Williams KD. Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology. 1993;65(4):681–706.