Using Audience Response Systems

by Adrian Hui, Pharm.D., PGY1 Pharmacy Resident, MedStar Union Memorial Hospital

How many of you have used an audience response system? Please use your devices to select an answer. 

A.    Yes

B.    No

C.   Not sure

D.   What is an audience response system?

Many of you may be familiar with this scenario, commonly seen in classroom settings today.  With their increasingly widespread use, audience response systems have made their way into television game shows such as “Who Wants to be a Millionaire” where the audience can be polled to help the contestant answer a question.  If you are not familiar with the term “audience response system,” perhaps one of the following alternatives may ring a bell: personal response system, electronic voting system, or student response system.¹

What exactly are “audience response systems (ARS)?”  ARS are tools used to quickly gather data from people participating in a poll.  ARS made their first appearance in the 1960s at Cornell and Stanford Universities² but were costly, cumbersome, and frequently malfunctioned.¹  Eventually, ARS became commercially available in the 1990s, and with several technological improvements (e.g. infrared capability, portability), it soon became widely used in various educational settings in the early 21^st century.¹  Nowadays, ARS are simple and easy to use.  They only require the following components:³

• Presentation software (i.e., PowerPoint with Turning Point)
• Hardware to transmit information (i.e., “clickers”, mobile computers, mobile phones)
• Wireless receiver or Internet connection to a computer to receive the data

I first experienced ARS as an undergraduate student when I was required to purchase my first “clicker” for an introductory genetics class.  The device was about the size of a small TV remote and had buttons for the numbers 0-9, the letters A-J, and a few other miscellaneous functions.  We were required to bring the “clickers” to every class and use them to answer multiple choice quiz questions interspersed throughout the lectures.  Our answers on these “quizzes” counted towards 10% of our final grade.  This mode of learning was unlike anything I had ever experienced before, and although I initially felt apprehensive about ARS, I eventually came to appreciate its utility. 

There are several benefits to using ARS.  Once a question is posed, responses are anonymous and summarized graphically for the audience.  However, the instructor has the flexibility to track each individual response and can collect the data for a grade or to give credit for attendance.  Anonymity can have a positive impact on learning, often enhancing participation, engaging the students, and promoting greater interactions between the students and their peers/teachers.¹  In fact, a study by Clauson et al.⁴ showed that 93.2% of pharmacy students appreciated the anonymity of ARS.  To gauge how well the students understand the material being covered in class, the instructor can use ARS intermittently throughout a lecture.  Based on the results, the instructor can determine whether it's appropriate to re-explain concepts more in-depth if the class seems to be struggling or continue on to the next topic.  Thus, ARS, if used effectively, can improve the quality of learning and provide both the instructor and students valuable feedback.  ARS may help improve test results as well.  In a study by Cain et al., final grades for a Physiological Chemistry and Molecular Biology (PCMB) II course in 2008 (using ARS) were significantly higher than those in previous years without ARS (p < 0.05 for 2008 vs 2007 and p < 0.001 for 2008 vs 2006).  Also, students taking PCMB II with ARS in 2008 had significantly higher final grades compared to when they took PCMB I the previous semester without ARS (p < 0.001).  Although this study has some limitations, the findings suggest that ARS may be a useful tool to improve student performance on exams.³  See Table 1 for a summary of the benefits of ARS.¹

Table 1: Summary of ARS Benefits

Classroom environment benefits

Attendance: Students go to class more

Attention: Students are more focused in class

Anonymity: All students participate anonymously

Participation: Students participate with peers more in class to solve problems

Engagement: Students are more engaged in class

Learning benefits

Interaction: Students interact more with peers to discuss ideas

Discussion: Students actively discuss misconceptions to build knowledge

Contingent teaching: Instruction can be modified based on feedback

Learning performance: Learning performance increases as a result of using ARS

Quality of learning: Qualitative difference when learning with ARS (e.g., better explanations, thinking about important concepts, resolving misconceptions)

Assessment benefits

Feedback: Students and teachers like getting regular feedback on understanding

Formative: Assessment is done that improves student understanding and quality of teaching

Compare: Students compare their ARS responses to class responses

 Adapted from Kay et al (2009)¹

Although ARS has some benefits, there are some challenges too.  From my experience, the “clickers” were a source of stress when students lost/forgot them or had technical difficulties.  This was especially the case when the “clickers” were used to answer quiz questions that counted toward a student’s grade or attendance.  Cain et al.³ found that the most common complaint about ARS (shared by 82% of pharmacy students) was the cost of the devices.  It should be noted, however, that some schools “cover” the cost of the devices and recirculate them to students in different classes.  But let’s face it, even when the school buys the devices to be shared by students, the cost of this technology is, in the end, bore by students in the form of tuition and fees.  Technical problems with ARS can also be frustrating for instructors that have integrated ARS questions into their lectures.  Furthermore, ARS may be a challenge for relatively inexperienced instructors unable to adapt their teaching to address unexpected responses from the students.  Developing ARS questions can be time consuming and takes skill and experience in order to create truly effective ones.  Effective questioning should not target simple recall of the material being presented but rather should cultivate conceptual, analytical, and problem solving skills.⁵  Such thoughtful questions are more likely to lead to enriching discussions and stimulate higher-level cognitive processes instead of serving as one-dimensional questions simply used to keep the student awake.  For a summary of the challenges with ARS, see Table 2.¹

Table 2: Summary of ARS Challenges

Technology-based challenges

Bringing remotes: Students forgot/lost remotes and could not participate in class

ARS did not work: Remote devices did not function properly

Teacher-based challenges

Responding to student feedback: Less experienced teachers cannot adjust to student feedback

Coverage: Cover less course content if ARS is used

Developing questions: Time consuming to create ARS questions

Student-based challenges

New method: Students find it difficult to shift to a new way of learning

Discussion: Discussion leads to confusion or wasting time

Effort: Too much effort is required by students when using ARS

Summative assessment: Using ARS for tests may not be popular with students

Attendance for grades: Students do not like ARS used for monitoring attendance

Identifying students: Students want to remain anonymous

Negative feedback: Students feel bad when receiving negative feedback

 Adapted from Kay et al (2009)¹

How can we optimize the use of ARS in the classroom?  Since our attention span often fades after about 20 minutes,¹ one strategy is to provide ARS questions every 20 minutes to break up a lecture.  Along with using effective questions, a small incentive to increase participation and attendance may be helpful.  Using ARS responses for 5% of a student’s grade is sufficient to increase motivation and attendance.¹  Lastly, although ARS may take away time to cover material, I would argue that ARS enables the instructor to explore concepts in greater depth and thus provides greater value in terms of student learning.  Overall, the benefits of ARS seem to outweigh its challenges, and several studies have shown that the majority of students favor its use in the classroom.  Yet, more research is needed to better understand the utility of ARS and to hone best practices.¹ 

References:

1. Kay RH and LeSage A. Examining the benefits and challenges of using audience response systems: a review of the literature. Computers and Education. 2009; 53:819-27.

2. Nelson C, Hartling L, Campbell S et al. The effects of audience response systems on learning outcomes in health professions education. A BEME systematic review: BEME Guide No. 21. Med Teach. 2012;34(6):e386-405.

3. Cain J, Black EP, Rohr J. An audience response system strategy to improve student motivation, attention, and feedback. Am J Pharm Educ. 2009; 73(2): Article 21.

4. Clauson KA, Alkhateeb FM, Singh-Franco D. Concurrent use of anaudience response system at a multi-campus college of pharmacy. Am J Pharm Educ. 2012;76(1): Article 6.

5. Dangel HL and Wang CX. Student response systems in higher education: Moving beyond linear teaching and surface learning. Journal of Educational Technology Development and Exchange 2008; 1(1): 93-104.