But What is the Right Answer?

by Kellen Riley, Doctor of Pharmacy Candidate, University of Maryland School of Pharmacy

As pharmacy students advance through their didactic curriculum, a fundamental switch in learning occurs. While the first few years of classes consist largely of rote memorization - of brand-generic pairings, therapeutic classes, and anatomy – sometime during the second or third year, students are expected to begin synthesizing the knowledge previously gained to arrive at the “most appropriate” clinical decision. Aptly so, as these students are preparing themselves for careers as pharmacists in a world where most answers are not black and white.

When students begin to migrate away from memorizing facts toward synthesizing clinical decisions, they realize that the multiple-choice exam becomes less and less appropriate as an assessment tool. This phenomenon becomes most apparent to me when my classmates and I are engaged in case-based discussions in a course that uses multiple choice exams:

Imagine that you are the student. You value the discussion taking place and know that it is strengthening your understanding of the material.  But in the back of your mind, unavoidably looming, is the exam. You know that in a few short weeks, you will sit for a test in which there is literally only one best answer and you will be forced to sift through all the information you can recall about this topic and choose just one. Even though you understand that there isn’t one right answer, you also want to be able to do well on that test. Finally, one student raises his or her hand to ask the question that is on every student’s mind: “OK, but what is the right answer?”

Now imagine that you are the professor leading the discussion. You have just spent the past five minutes having a lively and engaged discussion.  Students are furiously typing notes and asking intelligent questions. After all the discourse in which you have obviously stated that this particular scenario is not quite clear-cut, to your dismay, a student raises his or her hand and asks, “OK, but what is the right answer?”

This question can be posed many different ways (“So, if we were to get this question on the exam…” or “If we had to pick only one answer…”), but in each case the theme is the same: in courses where multiple-choice examinations represent the majority of a student’s grade, the students’ priority is ensuring maximal success on these exams, even if the methods of achieving that goal seem to oppose the “critical thinking” atmosphere created by the instructor.

I know from personal experience that my classmates and I do not perform as well on multiple choice exams in case-based courses.  Despite the desires of the professor, many students prioritize exam preparedness over learning objectives and lively discourse, if they believe their grades will suffer. Some researchers have examined the effect of multiple choice exams on the learning environment.  They found not only that students’ perception of their learning was strongly correlated with the type of examinations used but also found that multiple choice exams can hinder critical thinking skills.^1,2  Some argue that multiple choice exams are not inherently bad assessment techniques, but that the most important consideration when evaluating students’ learning is that there is an alignment between the way the material is presented and the way it is assessed.

Thomas Reeves, a professor in the Department of Educational Psychology and Instructional Technology at The University of Georgia, examined the alignment between eight critical factors (goals, content, instructional design, learner tasks, instructor roles, student roles, technological affordances, and assessment), and their effects the learning environment.³ According to his research, the factor most often out of alignment was assessment, and he believes the blame falls on the instructor:

“Simply put, instructors may have lofty goals, share high-quality content and even utilise advanced instructional designs, but most assessment strategies tend to focus on what is easy to measure rather than what is important.”³

Reeves’ characterization of multiple choice exams might be a little unfair because multiple choice exams may in fact be the only feasible method for instructors to provide timely feedback to a large group of students. While every student wants the assessment to be “completely fair,” the second most predictable desire of students regarding exams is that they want to receive feedback “like, yesterday.”

While good grades are nice, students place importance on examinations that are an accurate reflection of their knowledge and skill.  In courses that require critical thinking and synthesis, multiple-choice examinations are probably not the best option. A meta-analysis performed by Kenneth Feldman reveals that when examining the relative importance of various instructional dimensions, both students and teachers rate the importance of “quality of examinations” equally.⁴ These data suggest that instructors do realize the importance of assessment, and that the logical next step is figuring out what methods an instructor can use to achieve better alignment between the information and skills to be learned and the assessment.

Although much depends on the material being presented, possible options within the health professional education include practical examinations, small-group interactions, debates, peer assessments, or one-on-one standardized patient interactions like the Objective Structured Clinical Examination (OSCE). One interesting avenue of assessment includes the multiple choice item development assignment (MCIDA), which requires students to “develop multiple choice items, write justifications for both correct and incorrect answer options and determine the highest cognitive level that the item is testing.”⁵ It is an interesting twist on the traditional multiple choice exam in that the students write the test instead of sitting for one. Although the author acknowledges that this assignment can be time-consuming to grade, using the MCIDA enhanced learning outcomes for students.

When it comes to higher learning, some classrooms feature a battle between a professor’s desire to lead a discussion in which students focus on the critical thinking process and students’ desire to perform well on an upcoming examination by focusing on “the bottom line.” If instructors can design their course in a manner that promotes alignment between the critical thinking activities of the classroom and the assessments used to test their students’ knowledge of that material, they help ensure student success – not only on exams, but in life.

References:

1.  Stehle, S, Spinath, B, Kadmon, M. Measuring Teaching Effectiveness: CorrespondenceBetween Students’ Evaluations of Teaching and Different Measures of StudentLearning. Res High Educ 2012;53:888–904.

2.  Stanger-Hall, KF. Multiple-Choice Exams: An Obstacle for Higher-Level Thinkingin Introductory Science Classes. CBE Life Sci Educ. 2012;11:294-306.

3.  Reeves, TC. How do you know theyare learning?: the importance of alignment in higher education. Int. J. Learning Technology 2006;2:294-309.

4.  Feldman, KA. Effective College Teaching from the Students’ and Faculty’s View: Matchedor Mismatched Priorities? Res High Educ 1988;28:291-344.

5.  Fellenz, MR. Using assessment to support higher level learning: the multiple choice item development assignment. Assessment & Evaluation in Higher Education. 2004;29:703-719. 

An “A” for Effort?

by M. Ellen Tsay, PharmD, Clinical Toxicology Fellow, University of Maryland, School of Pharmacy

I have been in school for approximately 21 continuous years.  Now that I am on the “other side” I have had a few opportunities to teach pharmacy students. Teaching has many joys but there are challenges too, such as when students question their grades because they believe they deserved something better.

There have been anecdotal reports of a rise in “academic entitlement” among students in the Millennial generation.  These reports describe students who appear to lack a sense of personal responsibility for one’s own education, a rise in rude behavior, a decline in academic performance, and student expectations that grades should based on effort rather than quality.^1,2,3

What exactly is this “academic entitlement?” This phenomenon appears to be rooted in certain beliefs:¹

1.   Knowledge is a right.

2.   The instructor is the primary source of material/guidance.

3.   The instructor is responsible for the student’s success or failure.

4.   All students should receive equal recognition.

5.   Action should be taken by the instructor and/or administration if expectations are not met.

What is contributing to the rise of these attitudes among students? Some blame consumeristic views, pervasive in our culture, being increasingly applied to education.  If students are paying for their education, they should be viewed as customers who have a right to receive satisfaction and quality service. In pharmacy education, the increased number of pharmacy schools (especially for-profit institutions) may potentially fuel this consumeristic view of education.¹ Other factors that may be contributing to academic entitlement include grade inflation and changes in teaching practices.^1,4,5  With the increase in communication facilitated by technological advances, instructors are now more “available.” As convenient and useful this type of communication may be, it may foster expectations of rapid response for all communication. Further, anonymous evaluations may indirectly shift the instructor’s teaching behavior in order to receive favorable ratings from students.

Are these anecdotal reports backed by data? Perhaps today’s young people are more entitled due to a greater prevalence of narcissism and self-enhancement (e.g. a discrepancy between self-perceived ability to actual ability). Investigators compared the mean Narcissistic Personality Inventory (NPI) scores of students at the University of California Davis and Berkley from 1979-1985,  1996, and 2002-2007. In addition the investigators looked at how the sentiments of personal intelligence of students at the University of California between 2003-2007 compared with their SAT scores and college GPA’s. This discrepancy was then compared to data from the Monitoring the Future project, where high school students (since 1976) were asked to rate their intelligence compared to their peers, which was then compared to high school grades. There was no observable increase in NPI scores in college students from the 1980s to 2000s or changes in self-enhancement.⁶  However, when looking at subscales of the NPI, specifically scores related to entitlement, exploitiveness, and self-sufficiency, there have been significant increases over time.

However, the concept of academic entitlement may be more complex than simply narcissistic tendencies. Students enrolled in a public university were asked to complete a series of questions (including a newly developed scale assessing academic entitlement) in order to delineate key contributors to this phenomenon. What investigators found was that exploitive attitudes correlated strongly to students who were considered to be academically entitled.⁵ Many students have a tendency to expect special treatment without consideration for the circumstances or situation.  Moreover, 66.2% of the participants in this study felt that effort should be taken into consideration when determining the final grade. Academically entitled students tend to have lower self-esteem, display more achievement anxiety, and are driven more by extrinsic rewards.⁵   This type of student may work to earn a good grade but have little interest in mastering the material.

How should educators of today handle academically entitled students? Stephen Lippmann, an assistant professor in sociology at Miami University, gives educators other suggestions in how to handle academically entitled students in the classroom.⁴  For starters, clear and explicit objectives should be set before any lesson is given. Instructional design models should also be utilized when planning out a lesson in order to ensure effective (and efficient) teaching.  In addition:

• Post anonymous examples of quality work to students. The posting of “quality work” will not only help to address the issue of low self esteem, but will make example of what sort of work an instructor is looking for.
• Require students to submit any complaints/concerns in writing before meeting with an instructor. If a student writes out their complaint or concern prior to a meeting with an instructor, both parties can effectively prepare ways to address the situation before coming to a solution.
• Any grade negotiation should include a re-evaluation and therefore the possibility of a lower grade. Some students may feel they have “nothing to lose” when negotiating grades with professors. However, if the possibility of losing points is at stake, students will be forced to self-reflect first.
•  Reframe the classroom dynamic. Lippmann encourages instructors to make clear their teaching (and learning) philosophy to students at the beginning of the course. It should also be made clear to the students they are responsible for their own learning, and therefore the classroom dynamic should be shifted away from a teacher demand-student compliance model.

Academic entitlement likely reflects underlying beliefs about the roles and responsibilities of teachers and students.  Educators should reflect on their personal teaching philosophies and may need to adapt their teaching styles in order to keep up with changing attitudes in education.

References

1. Cain J, Romanelli F, Smith KM. Academic Entitlement in Pharmacy Education. Am J Pharm Educ 2012 ;76: Article 189.

2. Clift E. From Students, a Misplaced Sense of Entitlement. Washington, D.C.: The Chronicle of Higher Education; 2011 March 11. [Cited 2013 Feb 14].

3. Roosevelt M. Student Expectations Seen as Causing Grade Disputes [Internet]. New York: The New York Times; 2009 Feb 17. [Cited 2013 Feb 14].

4. Lippmann S, Bulanda RE, Wagenaar TC. Student Entitlement. College Teaching 2009;57:197-204. 

5. Greenberger E, Lessard J, Chen C, Farruggia SP. Self-entitled College Students” Contributions of Personality, Parenting, and Motivational Factors. J Youth Adolescence 2008;37:1193-1204.

6. Trzesniewski KH, Donnellan MB, Robins RW. Do Today’s Young People Really Think They Are So Extraordinary?. Psychol Sci 2008;19:181-188.

Habits of Mind

by Janessa Smith, Pharm.D., PGY1 Pharmacy Practice Resident, The Johns Hopkins Hospital

How do we learn? What allows us to retain new information and have it available in the future? Is learning driven by external stimuli in the environment or is it dependent on internal processes within the learner? These are the questions that drive theorists to explore the mysterious world of educational psychology.  Some of the fundamental theories of learning include social learning, constructivism, and behaviorism attempt to answer some of these questions.  And new understandings are emerging all the time.  A relatively new concept called “habits of mind” has been described and it’s taking the world of education by storm.

First described by Arthur L. Costa, Ed.D and Bena Kallick, PhD, habits of mind (HOM) are specific behaviors that intelligent humans employ when confronted with problems in which the resolution is not immediately known.¹ These 16 habits (defined in Table 1) attempt to explain how behavior and learning are intertwined and dependent on one another. Costa and Kallick suggests that each time these behaviors are employed, “the effects of their use are reflected upon, evaluated, modified and carried forth to future applications.” This ability is what they define as intelligence in humans, which is a distinct concept from cognitive ability. The pair believes that intelligence is not just about having information but knowing how to use the information to act in specific situations based on previous experience. This concept applies to both academic and non-academic situations.

         Table 1

Habit of Mind	Description
Managing Impulsivity	Taking the time to deliberate before acting.
Listening with Understanding and Empathy	Making the effort to perceive another person’s perspective.
Thinking Flexibly	Considering Options and Changing Perspectives.
Striving for Accuracy	Setting high standards and finding ways to improve.
Persisting	Persevering in a task to completion and not giving up.
Metacognition	Thinking about one’s thinking. Being aware of how thoughts, feelings and actions affect others.
Questioning and Problem Posing	Findings problems to solve, seeking data and answers.
Applying Past Knowledge to New Situations	Accessing prior knowledge and applying that knowledge to new contexts.
Thinking and Communicating with Clarity and Precision	Striving for accuracy in oral and written communications.
Gathering Data Through all Senses	Paying attention to the world around through taste, touch, smell, hearing and sight.
Creating, Imagining and Innovating	Generating new and novel ideas.
Responding with Wonderment and Awe	Being intrigued by the mystery in the world.
Taking Responsible Risks	Living on the edge of one’s competence.
Finding Humor	Enjoying the incongruous and unexpected. Being able to laugh at oneself.
Thinking Interdependently	Being able to work and learn as a team.
Remaining Open to Continuous Learning	Resisting complacency in learning and admitting when one does not know.

  Adapted from Campbell⁴

I find this to be a very intriguing concept of how intelligent people utilize their knowledge. Prior to reading about HOM, I believed, as most people do, that intelligence was driven by how much knowledge one has and the ability of that person to retrieve that knowledge. This new concept suggests that deeper processes are important.  It’s not simply a matter of storage and retrieval. While Costa and Kallick introduced me to the HOM concept, it wasn’t until reading an excerpt from How Children Succeed: Grit, Curiosity, and the Hidden Power of Character by Paul Tough that I began to fully understand that this concept transcends the classroom.² In his book, Tough describes how one’s character is a stronger predictor of life success than cognitive ability. He provides a number of examples to illustrate this point, the most striking being “The Perry Preschool Project”, a sociology experiment that began in the mid-1960s in an industrial town west of Detroit. This study randomly assigned three- and four-year old children into either an intervention or a control group. The intervention group was enrolled in Perry Preschool, a two-year, high-quality preschool program. The control group was not. The initial intent of the project was to evaluate the effect of Perry Preschool on the children’s IQ – a measure of intelligence. The initial results of this experiment showed that those that attended Perry Preschool performed better on cognitive tests but the difference between the groups diminished by the time they reached the third grade. Interestingly, when evaluating the long terms results, those children that attended Perry Preschool were more likely to be “successful” in life. They were more likely to graduate from high school, to be employed at age twenty-seven, and to be earning a higher salary at age forty when compared to the control group.

At its heart, this experiment was an evaluation of two different teaching models.³ Investigators compared the Direct Instruction Model — a traditional teaching model where teachers directly teach students and reward them for correct answers — to the High/Scope model where teachers set up the daily routine but allow children to plan, do, and review their own activities.  Thus students engaged in active learning — individually, in small groups, and as whole-class groups. This model of instruction ties in directly with the HOM. It allows learners to use each of the 16 HOM in their daily learning and demonstrates that this teaching model can improve educational performance at a young age and has long-term impact in terms of success in the learner’s personal life.

Additionally, Tough attributed the difference between the two groups to the development of “noncognitive skills” in the Perry Preschool group. These skills were a sum of behaviors that were observed and recorded over the decades among the two groups. The schools tracked “personal behavior” which included how often the students swore, lied, stole, or were absent or late to school. They also recorded behaviors of “social development” which tracked characteristics such as curiosity and peer relationships.  Students in the intervention group performed much better.  Thus the High/Scope teaching model likely facilitated the development of many of the HOM such as Managing Impulsivity, Responding with Wonderment and Awe and Applying Past Knowledge to New Situations.

The HOM complement the traditional and widely accepted educational theories that explain how people learn. They rely on the same fundamental concepts of these more traditional theories, but provide an explanation of how intelligent people use knowledge and add an element of accountability and responsibility.  John Campbell, faculty at Central Queensland University in Australia, describes the parallels between HOM and other learning theories, including constructivism and social learning theory.⁴ He explains that in order to construct knowledge, learners must reflect, plan and evaluate (i.e. Metacognition) as well as use senses to gather data from their surroundings (i.e. Gathering Data through all Senses). Additionally, he explains that constructivism emphasizes the use of group interaction (i.e. Thinking Interdependently) and active rather than passive learning (i.e. Questioning and Posing Problems, Managing Impulsivity).  Campbell compares the three aspects of social learning (observation, language and self-talk) with HOM, explaining that “self-talk” corresponds with Managing Impulsivity and Metacognition, “language” relates to Thinking and Communicating with Clarity and Precision and “observation” is demonstrated by Gathering Data through all Senses.

While still a relatively new concept, I believe the HOM have the potential to significantly influence the way we educate children and adults. By incorporating the HOM into learning exercises, educators can enhance what is learned and improve its application to other situations in life. After all, the purpose of education is to provide a structured environment where learners develop in all domains of their lives:  academically, personally, and professionally.

References

1.  Costa AL, Kallick B. Describing 16 Habits of Mind. [Internet]. [cited 2013 Feb 3].

2.  Tough P. How Children Succeed: Grit, Curiosity, and the Hidden Power of Character. Boston: Houghton Mifflin Harcourt; 2012.

3.  Schweinhar LL, Montie J, Xiang Z, et al. The High/Scope Perry Preschool Study ThroughAge 40: Summary, Conclusions and Frequently Asked Questions. Ypsilanti, MI. High/Scope Press. 2005 by High/Scope® Educational Research Foundation. [cited 2013 Feb 20].

4.  Campbell J. Theorising Habits of Mind as a Framework for Learning. Proceedings of the Australian Association from Research in Education Conference, Adelaide, South Australia. [Internet].  [cited 2013 Feb 3].