Creating Valid Multiple-Choice Exams

by Scott Ross, PharmD, PGY1 Pharmacy Practice Resident, Mississippi State Department of Health

With increased class sizes and teacher load, multiple-choice exams have become the primary method for evaluating health professions students. There are many pros and cons to using multiple-choice tests. This article aims to investigate the cons and improper techniques and offer potential solutions to improve the quality of questions and enhance student learning.

Multiple-choice exams remain a popular form of assessment because they have several advantages, including ease of grading, standardization, the objectiveness of scoring, and the ability to test many discrete concepts. And teachers can administer multiple versions of the same test.^1,2 Furthermore, evidence suggests that a well-constructed multiple-choice exam is just as effective as a short-answer test in terms of promoting the retention of material.^3,4 However, a poorly constructed multiple-choice exam will not accurately measure learning and can lead to frustration. There are several best practices that many teachers fail to follow, including using "all of the above" or "none of the above" answer choices, writing "throw-away" answers as potential choices, asking students questions that focus on their ability to memorize and recall trivial details, and authoring stems that are unclear/vague.^1,5

While seen commonly on multiple-choice exams, the "none of the above" answer choice leaves students wondering what the correct answer is. Indeed, it could be argued that the best possible answer is not among the answer choices, and thus "none of the above" would always be the best option. In many cases, students view "none of the above" as a throw-away answer that can be ignored. Similar but different issues arise with the "all of the above" answer choice. Using the "all of the above" answer choice may have the benefit of determining if the student is aware that more than one choice is correct, but this quickly results in guessing and relying on partial knowledge of the material to answer correctly.⁶ When the "all of the above" choice frequently appears on an exam, students will likely pick up on trends and will lean towards selecting this answer even when they lack an understanding of the material. Thus, "none of the above" and "all of the above" answer choices should be avoided.⁶ Instead, consider using "select all that apply" questions because they thwart guessing — but admittedly, they are more difficult. To discourage guessing, some instructors award points for each correct response but take off points if a student selects an incorrect answer or does not select a correct response.¹ While this is undoubtedly more challenging, it is more efficient and less cognitively demanding than asking students to respond to series of open-ended questions.

Another common issue when constructing multiple-choice tests is including "throw-away" answers — answers that are so obviously incorrect that even those who do not know the subject matter can quickly eliminate them. Including these answer choices is harmful because it increases the odds of guessing correctly. It is a best practice to include at least 3 but no more than 5 plausible answer choices.⁷  The key word here is plausible – at least they should seem reasonable to the learner who is not sufficiently knowledgeable about the subject matter.

Some critics of multiple-choice testing state that exam scores using this format do not always correlate to the learner's understanding of the material — the method simply asks students to memorize and recall information.⁵ This is important to keep in mind when forming questions as many instructors rely too heavily on "recall" or knowledge-based type questions. While they are more challenging to write, it is possible to create questions that require critical thinking. Forming thoughtful questions that require students to analyze, apply, and evaluate is vital to ensuring they develop the skills needed in their future careers.

Another common problem is forming misleading or vague question stems or answer choices that lead to confusion or misunderstandings. It is also best to avoid negative phrasing (e.g., "which of the following is not true …") in exams since this can cause students to misread the question. If a question truly cannot be phrased positively, it is best to make the negative wording stand out by using italics, capitalization, or bolding of the word(s). Having clear answer choices is just as important as forming clear question stems. An excellent way to ensure that questions and answers are worded clearly and concisely is to send the material to someone else to review. It is crucial to keep in mind how the answer choices relate to each other. Answer choices should be homogenous in the sense that they relate to the same content and have a similar sentence structure and length. This is to prevent giving clues to students as to what the correct answer is. Another strategy to prevent clues is to always present the choices in numerical or alphabetical order.

Perhaps the biggest concern with multiple-choice tests is the format itself. Most choices will not be provided to the health professional.  Rather they must recall and weigh the potential options themselves.  Thus, multiple-choice exams are not authentic assessments — they do not reflect real life. Real-life decision-making comes from generating choices for ourselves and formulating our own answers by considering multiple pieces of information and then making a judgment. Thus, relying solely on multiple-choice assessments to determine a student's progress does not accurately reflect whether a student is competent.  Other forms of assessment, including objective structured clinical exams (OSCE), evaluations of authentic work products, and observations during field-based activities, must also be used.

Creating valid multiple-choice exams is a vital skill that all teachers should master to ensure their students have mastered the material. However, there are several common problems that should be avoided, and multiple-choice assessments have several limitations. Using a combination of assessment strategies is essential to get a comprehensive view of each student's knowledge, skill, and abilities.

 

References

1. Weimer M. Multiple-choice tests: Revisiting the pros and cons [Internet]. Faculty Focus. 2019 [cited 2021Sep19].
2. Medawela RMS, Ratanayake DRDL, Abesinghe W, et al. Effectiveness of "fill in the blanks" over multiple choice questions in assessing final year dental undergraduates. Educación Médica 2017, 19 (2): 72-76.
3. Khan JS, Mukhtar O, Tabasum S, et al. Relationship of Awards in multiple choice questions and structured answer questions in the undergraduate years and their effectiveness in evaluation. Journal of Ayub Medical College 2010; 22 (2): 191-195.
4. Haynie W. Effects of Multiple-Choice and Short-Answer Tests on Delayed Retention Learning [Internet]. Journal of Technology Education 1994; 6 (1): 32-44.
5. Fors K. Opinion: Multiple choice tests don't prepare students [Internet]. The Utah Statesman. 2020 [cited 2021Sep19].
6. Butler A. Multiple-choice testing: Are the best practices for assessment also good for learning? [Internet]. The Learning Scientists. 2017 [cited 2021Sep19].
7. Butler AC. Multiple-choice testing in education: Are the best practices for assessment also good for learning?. Journal of Applied Research in Memory and Cognition 2018; 7 (3): 323-331.

If You Feel Like an Imposter, Perhaps It Is Time to Change Your Mindset

by Abby Bradley, PharmD, PGY1 Pharmacy Practice Resident, University of Mississippi Medical Center

The brain is theoretically the most powerful organ in the body and is responsible for processing and storing thoughts, memories, and experiences that happen throughout one’s life. It is these three things that shape who we are as a person and create our mindset towards ourselves and others. Our mindset plays a powerful role in what we believe. In Mindset: The New Psychology of Success, Carol S. Dweck describes the difference between those with a fixed mindset, a belief that abilities can’t be changed, and a growth mindset, a belief that abilities can be developed.^1,2  Many pharmacists (and other health professionals) feel like “imposters” which arises from a belief that their success is largely due to luck and timing, rather than their own effort.  The imposter phenomenon might be related to a fixed mindset. With a growth mindset, perhaps new graduates can better manage the inevitable challenges as they enter the workforce?

What is the difference between the person who crumbles versus the person who thrives after receiving negative feedback?  Mindset. A person with a fixed mindset assumes abilities and talents are relatively fixed.  Either someone has the ability to do something well or does not. A fixed mindset can be identified by characteristics such as avoiding challenges, feeling threatened by others, giving up easily, resorting to cheating and deception to get ahead, and focusing on the outcome rather than the process. On the other hand, those with a growth mindset believe that one's abilities, intelligence, and skills can be grown and developed through effort and hard work. Failure is seen as an opportunity for growth and criticism is a tool to better one's self. With a growth mindset, hard work and determination are embraced — there is a passion for learning (the process) rather than a hunger for success (the outcome). It is important to keep in mind that individuals often have different mindsets towards different domains in their lives, and mindsets can change.

	Fixed Mindset	Growth mindset
Belief	Abilities and intelligence is fixed	Abilities and intelligence can be developed
Skills	Fine the way it is, nothing to change	How can I improve?
Effort	Give up easily	Persevere when faced with failure and setbacks
Mistakes	Avoid new experiences due to fear of failure, blames others	Embraces and sees as an opportunity to learn
Feedback	Defensive, takes it personally	Accepts as a way to learn

One intriguing question that has been recently proposed: Is there a correlation between a fixed mindset and the imposter syndrome? The imposter phenomenon (IP) is the official psychological term and it describes a pattern of thinking whereby successful individuals feel unworthy of the success they have achieved.  They don’t feel competent and worry that their lack of skill will be “discovered.” In one study that surveyed medical, dental, nursing, and pharmacy students, significant levels of distress and rates of IP were found.³  Indeed, pharmacy students were at the highest risk for the IP when compared to other health professions.³ A recent study found a significantly higher prevalence of IP among pharmacy residents in comparison to trainees in other healthcare professions.⁴  These data show a worrisome pattern but can we do anything about it?  Although a correlation between IP and a fixed mindset has not been conclusively proven, some researchers believe that adopting a growth mindset could reduce the risk of IP. By implementing techniques that foster a growth mindset early in pharmacy education, students would learn to be better equipped to handle the stress and competitive environment of pharmacy school as well as the workforce.

How can we foster a growth mindset among pharmacists and student pharmacists? The first step begins with educating the educator. To be able to foster a growth mindset, the educator must have good foundational knowledge and demonstrate a growth mindset themselves.  They need to be role models! Simply bringing awareness to the idea of different mindsets has been shown to foster a shift in thinking. This can be done in didactic lectures, small group discussions, and personal experiences.

Ways That Educators Promote Mindsets

Fostering a Fixed Mindset	Fostering a Growth Mindset
·  Multiple-choice exams ·  Praising intelligence, skill, talent ·  Focusing on results ·  Limited, nonspecific formative feedback	·  Socratic questioning ·  Learning experiences ·  Allowing multiple drafts ·  Pre- and post-tests ·  Frequent formative feedback

Feedback is critically important during any learning experience, but to promote a growth mindset, intentional, constructive feedback must be provided so that students are praised for their processes and improvements rather than the grades they achieve.  The teacher should focus on effort and growth. Although feedback is usually given at the end of a course or experience, the foundation should be laid at the beginning of an experience when expectations and goals are established. After having the opportunity to perform and be assessed, trainees should be provided with constructive feedback and opportunities to improve. To be considered constructive, feedback should be specific, based on direct observations (or other evidence), and objective (criterion-referenced) while also providing advice on how to improve. Self-reflection should also be used as a way for trainees to reflect upon the processes they employed when complete tasks and assignments. Thinking and talking about processes provide insight about what went well as well as areas of improvement.

Grades in general do not provide insight into the learning process or growth of a student. A single summative assessment or ranking does not promote the beliefs that foster a growth mindset; however, the use of formal assessments can be beneficial when used appropriately. Introductory and advanced pharmacy practice experiences, as well as residency training, represent ideal environments to implement pre-and post-tests to highlight the amount of growth from the experience. Rather than receiving only a final grade, trainees can tangibly see their growth by comparing their pre-rotation and post-rotation scores.

Self-assessments tools like The Mindset Assessment on The Mindset Works website provide insight into one's mindset and could be beneficial for both educators and trainees. This short assessment is a diagnostic tool that can be used to objectively assess and learn more about one's mindset; it also provides specific recommendations on how to move toward a growth mindset as well as personalized feedback.⁵

Healthcare and pharmacy practice is an everchanging field that can be taxing for students, residents, as well as practitioners. By cultivating a growth mindset, pharmacists can overcome the challenges faced during their training and after entering the workforce. Faculty and preceptors play a key role in educating and promoting a growth mindset during the early stages of their pharmacy careers. Trainees must learn to develop the skills needed to persevere in the face of failure, accept criticism as a learning opportunity, seek out challenges, and, just important, reduce the stress and anxiety from feeling like an imposter. 

References

1. Dweck CS. Self-Theories, Their Role in Motivation, Personality, and Development. Philadelphia, PA: Psychology Press; 2000.
2. Dweck CS. Mindset, The New Psychology of Success. New York: Random House Digital; 2008.
3. Henning K, Ey S, Shaw D. Perfectionism, the imposter phenomenon and psychological adjustment in medical, dental, nursing and pharmacy students. Med Educ. 1998;32(5):456-46.
4. Sullivan JB, Ryba NL. Prevalence of imposter phenomenon and assessment of well-being in pharmacy residents. Am J Health-Syst Pharm. 2020;77:690-696.
5. Burgoyne AP, Macnamara BN. The reliability and validity of the mindset assessment profile tool. PsyArXiv; 2020. doi: 10.31234/osf.io/hx53u

Prioritizing Health Literacy Education

by Bria T. Lewis, Pharm.D, MPH, PGY-1 Community Pharmacy Resident, University of Mississippi School of Pharmacy

Effective communication is an essential skill for healthcare workers. Communication between healthcare professionals and patients is multifaceted and can become complicated by reduced or poor health literacy skills. According to the U.S. Department of Health and Human Services, health literacy is “the degree to which individuals have the capacity to obtain, process, and understand basic health information needed to make appropriate health decisions.”¹ Unlike general literacy, health literacy focuses on specific skills needed to traverse the health care system and enables clear communication between healthcare providers and patients.

Improving health literacy education for health professionals is an essential concept of the U.S. National Action Plan to Improve Health Literacy. This concept must be prioritized by all health professionals who communicate with patients, and, just as importantly, those who educate emerging health professionals. It is important that health professions educators teach ways to both assess health literacy and to strategies to communicate to patients with low health literacy. Educators must take responsibility by recognizing the importance of effectively communicating health information to patients and work to address any deficits that may impede a patient from making the best decisions.² To mitigate such deficits, educators of health professionals must teach students about the common barriers that patients experience including a lack of understanding about disease states, local health guidelines, and the interpretation of test results³. 

While there are currently no widely accepted guidelines on health literacy education for healthcare professionals, healthcare educators should focus the scope of instruction on the following key skills:

1. Identifying Patients with Low Health Literacy: Healthcare professionals may not be able to identify patients with low health literacy. Factors associated with sufficient health literacy levels include higher individual income, advanced education, and greater professional success. In contrast, older adults, minority, or low-income populations are at risk for insufficient health literacy. Low health literacy has been shown to correlate with an increased risk of death and emergency room visits followed by hospitalization.

2. Use of Plain Language: Using non-medical language can enhance understanding between the patient and the provider. Students and health professionals may need to develop alternative language to explain concepts instead of using their acquired medical terminology.  Indeed, many patients, especially those of underserved populations, may not have literacy competency above a 5th-grade level. Thus, curriculums must include teaching students how to simplify complex words and concepts into 5th-grade English terms. This can be achieved by referencing medical terms that may appear during lectures in both the form understood by the medical community and the form understood by the average citizen. 

3. Focus the Message: Limiting the information to focus on one to three key messages is crucial. Focusing the key messages on behavior modifications will help empower and motivate patients. Educators should emphasize lessons that teach students and healthcare professionals to develop short explanations for common treatments and disease states which motivate action.

4. Importance of the Teach-Back method: Reviewing and repeating key information at the end of each visit will help with reinforcement. The ‘Teach Back” method serves as an effective tool to assess understanding and increase retention of information. Educators should introduce and use this tool throughout the curriculum in a fashion that requires students to “Teach Back” health information in laymen’s terms. This can be done by establishing simulation counseling sessions where students are required to translate medical information without using jargon.

Fortunately, there are several readily accessible health literacy education resources that educators can use. The Agency for Healthcare Research and Quality (AHRQ) has developed the Health Literacy Universal Precautions Approach to health literacy. This approach supports simplifying communication and reducing the complexity of healthcare. The toolkit offers twenty-one tools for improving health literacy by addressing spoken communication, written communication, and supportive systems. This guide is available for download at: AHRQ Health Literacy Universal Precautions Toolkit.  The CDC Clear Communication Index is a tool used to develop and assess public communication materials to determine if a message or material will likely match the health literacy skills of your intended audience.

Health literacy affects the health status of patients. Health literacy is a national concern. To provide the best care to our patients, all health professionals need to learn the key concepts and how to communicate complex ideas to patients using simple, clear language. 

References:

1. Health Literacy. Official Web Site of the U.S. Health Resources & Services Administration.” HRSA.gov, 31 Mar. 2017. Accessed 17 June 2021.
2. Bowen D. 5 How To’s for Teaching Health Literacy. Paeaonline.org. Accessed 17 June 2021.
3. Health Literacy. National Library of Medicine. NLM.gov, 2010. Accessed 17 June 2021.

Scary Word of the Day: Advocacy

by Mary Reagan Richardson, PharmD, PGY1 Community Pharmacy Resident, Mississippi State Department of Health

Advocacy is a term that is thrown around with different meanings in different settings.¹ All professional students have, at one point or another, heard the word advocacy mentioned as something they should do. But how do we advocate?  And is it something that can be taught? It seems that everyone agrees that advocacy is necessary and important but defining the competencies needed to “advocate” and measuring it are harder to come by.¹ In fact, when searching for primary literature on the topic, I found only four studies that mention advocacy.

So what is advocacy? One definition states that ”Advocacy is telling or demonstrating something you know to someone in order to improve the quality of life for others.”² The American Medical Association defines physician advocacy as, ”Action by a physician to promote those social, economic, educational, and political changes that ameliorate the suffering and threats to human health and well-being that he or she identifies through his or her professional work and expertise.”¹ There are several other definitions but they all include two central themes: knowledge propagation and a call to action.

There is a distinction between advocacy and self-advocacy. Advocacy is primarily about asking others to take action to benefit another person or a community. Self-advocacy is all about informing someone about what you or your profession can do. It is still a form of advocacy, just not done for the benefit of others. Self-advocacy is often a part of advocacy. If you cannot clearly articulate what it is you do and why only you can do it, how can you ask for an action-oriented change to occur? Self-advocacy fulfills the knowledge propagation step of advocacy.

Pharmacists, nurses, physicians, and other health professionals are all taught what it is their profession does as well as what they can or perhaps should be permitted to do. So, it only makes sense that advocacy is taught during professional degree programs. There is some evidence about the benefits for teaching self-advocacy to elementary school children, however, consensus on ways to teach how to advocate for others is not well developed.³ In my readings, it seems that advocacy can be broken down into three major types; person-to-person communication, using your knowledge to fill a need in the community, and direct, participatory communication with legislators or primary stakeholders.^1,2,3,4

For example, I am a pharmacist in a community, independent pharmacy. A patient comes in talking about how long the wait is and complains “why can’t you just put the pills in the bottle already?” There are several approaches that can be taken here: A) ignore the comment, B) apologize for taking so long C) explain what pharmacists do when filling a prescription. The pharmacy advocate would go with option C. What if that patient knew that you called the prescriber to get the dose adjusted due to an interaction with another medication? It is through these sorts of interactions, which happen every day, that we, as practitioners, educate the public about what goes on behind the scenes to improve their health. This is person-to-person advocacy.²  Calling the physician to get the dose change is also advocacy, because the pharmacist is using his/her knowledge, on the patient’s behalf, and making a call-to-action (e.g. change the dose of the medication). Advocacy in both of these circumstances enhances the public’s understanding of the pharmacist’s professional role.

Another example of advocacy is when you see a need for something in your community and take action. Like implementing COVID-19 vaccination clinics in a community pharmacy. The rules and regulations can be onerous and the additional demands of such a service can be very disruptive to a pharmacy’s workflow. However, pharmacies all over the country are implementing them to address the largest public health crisis in a century. You are advocating for your patients and community by taking on the added cost, stress, and time to administer these vaccines to improve the health of the community.

On a much larger scale, advocacy encompasses talking to local, state, and federal legislators and other policymakers about issues pertaining to your profession. However, many health professionals find it quite daunting to advocate for their profession directly to legislators. Many people are unsure of how to go about talking with legislators about what they do and the problems they encounter. The best way to prepare for these conversations is to stay up-to-date with the latest news about your profession, locally and nationally. The more informed you are as a practitioner, the better your case to a legislator will be. For example, when I spoke with a legislator in 2018 about expanding the scope of pharmacist practice in Mississippi, his first question to me was, “Have other states done this?” If I had not done my homework and known that, in fact, other states had authorized pharmacists to do similar things and the positive impact it had, that conversation would have ended right then and there. Speaking with legislators about a topic doesn’t have to be an in-person discussion. You can always send an email to your local or state senators explaining why and how you see a problem being fixed. Regularly communicating with your legislators and other policymakers is the most active and participatory form of advocacy.⁵

These examples provide a road map on ways in which we can advocate for our professions. In terms of teaching how best to teach students to advocate, there are very few published examples.  There are some residency programs that have advocacy curriculums that have been successful.⁴ Most professional degree programs include something about advocacy in their curriculums but do not have published learning objectives or competencies.⁶

I believe that there should be an advocacy course in the required curriculum or, at the very least, offered as an elective. This course would focus on how to research an issue, how to identify the major stakeholders, and how to make an “ask” (i.e. the call to action) This course should get students thinking critically about the problems their profession and the patient they serve face. A lobbyist could be brought in to discuss speaking strategies and the “how to’s” of talking to policymakers. The learning objectives and competencies should assess whether a student could effectively deliver an “elevator pitch” on an issue. Advocacy is something that becomes more natural as you practice it. For this reason, any advocacy course should have simulation activities and field-based exercises that students need to complete. Teaching students that advocacy can be as easy as talking with your family or calling your United States Senator can help make advocacy more approachable and an inherent expectation of being a professional.⁴

^References

1. RoyeaAJ, Appl DJ. Every voice matters: The importance of advocacy. Early Childhood Educ J 2009; 37: 89-91.
2. Earnest MA, Wong S, Federico SG. Perspective: Physician advocacy: What is it and how do we do it? Acad Med 2010; 85: 63-67.
3. Lee, A. (2021, April 12). The importance of self-advocacy for kids who learn and think differently. Retrieved April 21, 2021, from https://www.understood.org/en/friends-feelings/empowering-your-child/self-advocacy/the-importance-of-self-advocacy
4. Servaes J, Malikhao P. Advocacy strategies for health communication. Public Relations Review 2010; 36: 42-49.
5. Chamberlain LJ, Sanders, LM, Takayama JI. Child advocacy training. Curriculum Outcomes and Resident Satisfaction. Arch Pediatrics & Adoles Med 2005;159: 842-847.
6. Douglas A, Mak D, Bulsara C, Macey D., Samarawickrema I. The teaching and learning of health advocacy in an Australian medical school. Inter J Med Educ 2018; 9, 26-34.

Adapting to Adaptive Learning Technology

by Endya L. Young, PharmD, PGY1 Pharmacy Practice Resident, Magnolia Regional Health Center

While humans tend to be alike in more ways than not, we are also very different. We differ in the ways we think, feel, and act. And we differ in the ways we learn. Students each have their own learning style, strengths, and weaknesses that do not always align with a “one size fits all” approach. What if there was a technology to meet the different needs of students? What if this technology had the potential to foster self-directed learning skills? Such a technology would be a great addition to health professions curricula in combination with other teaching methods. This technology exists today!  Although it is fairly new, the technology has the potential to provide some significant benefits to students in the long run.

What is adaptive learning technology (ALT)? It is computer-based software that provides a personalized learning experience based on how the student performs.¹ It allows the student’s learning experience to be navigated in a way that fits the student’s needs and increases the likelihood that the student will be successful.² How does this work? ALT modifies the student’s learning materials in real-time based on their interaction with the program.³ Based on the student’s responses, ALT identifies the student’s strengths and address weaknesses and then adjusts the instructional materials, changes the pace, modifies the assessments, or provides feedback specific to the learner’s needs. ALT provides an efficient and flexible way to remediate learners who have not yet mastered a lesson while presenting new information to those who have.⁴

The number of studies on the effectiveness of adaptive learning technologies in pharmacy curricula is limited, but I believe it could be of significant benefit to students.  Using adaptive learning technology would prompt students to further develop their self-directed learning and their independence when learning — skills they need after they graduate. It has the potential to reduce gaps in learning and help to identify students who may be struggling.⁴ Use of this technology in pharmacy schools seems promising because of the need to retain the foundational knowledge while acquiring new information from the ever-evolving, ever-changing world of healthcare.

A recent study analyzed changes in self-directed learning when adaptive learning technology was used. This study enrolled first-year pharmacy students who were completing a two-semester Pharmacists’ Patient Care Process (PPCP) course series.³ In the first semester of this course, professors used active learning during traditional lectures, for example requiring students to complete pre-readings and reflections on course content followed by in-class quizzes.³ Adaptive learning technology was then used during the second semester of the course. Students were required to complete midterm and final examinations in both semesters. Surveys were completed by the students to assess the following: assignment management, stress management, procrastination management, seminar (lecture) learning proficiency, comprehension competence, examination management, and time management.³ To measure the students’ experiences and perceptions of ALT, focus groups were also used to gather additional feedback. The investigators report that students appreciated the additional practice and assessments that ALT provided. The study also concluded that using ALT freed up time during class for the instructors and students to engage in more active learning activities.

The themes identified from the student focus groups in this study convinced me that adaptive learning technology is something that should be incorporated into the pharmacy curriculum. The first theme was student learning preferences. Students overall found the assessments in ALT to be helpful, but also stated that the use of this technology made it more difficult to study for examinations.³ The students in this group stated that they would have liked to have some sort of guide such as PowerPoint slides to aid them in identifying the most pertinent information. The second theme mentioned was teaching methods. Students liked the mix of the teaching methods used, such as pre-class activities, mini-lectures to highlight key points in the learning material, and in-class activities to reinforce their learning.³ It is important to note that some students did not engage with ALT as they should have, often only answering the assessment questions and bypassing course material.  This is important because another study that used ALT in a physics course at a South African University found that students who spent more time engaging with the program performed better on examinations.⁵ The third theme was valued. Students seemed to benefit more from hearing their professor’s insight on the material being taught and helped them to apply concepts as they progressed. The fourth theme was technology and the challenges the students encountered with its use.

Although the findings in this particular study showed that the use of ALT was not favored by most students, I think that some of the student’s concerns are due to a lack of familiarity. Their desire to be given notes and the fact that many students struggled with procrastination and time management makes me think they oppose ALT simply because it is not something they have used before and have not yet developed the skills to be self-directed learners. Students may have had difficulties because they the lack skills needed to discern important information on their own.³ They preferred to have all of the information provided to them and to have the teacher point out what is important during class. Some students also did not engage with the ALT as they should have, often prioritized other classes. Using a combination of in-class active learning activities with ALT in between class sessions, in my opinion, gives the students the opportunity to learn from and engage with the professor but also develop life-long learning skills.  This will require some major adjustments for student students (and instructors!). Such a major change should be introduced gradually. Overall, I believe ALT has great potential – helping students who have not yet mastered the material a personalized experience while simultaneously promoting the development of self-directed learning skills. 

References

1. Forsyth B, Kimble C, Birch J, Deel G, Brauer T. Maximizing the Adaptive Learning Technology Experience. Journal of Higher Education Theory and Practice [Internet]. 2016;16(4):80-88.
2. Liu M, Kang J, Zou W, Lee H, Pan Z, Corliss S. Using Data to Understand How to Better Design Adaptive Learning. Technology, Knowledge and Learning. 2017;22(3):271–98. 
3. Toth J, Rosenthal M, Pate K. Use of Adaptive Learning Technology to Promote Self-Directed Learning in a Pharmacists’ Patient Care Process Course. American Journal of Pharmaceutical Education [Internet]. 2020;85(1): Article 7971.
4. Moskal P, Carter D, Johnson D. 7 Things You Should Know About Adaptive Learning [Internet]. EDUCASE 2017.
5. Basitere M, Ivala E. Evaluation of an adaptive learning technology in a first-year extended curriculum programme physics course. South African Computer Journal; 2017; 29 (3):1-15.