The Connection Between Confidence, Knowledge, and Experience

by Madalyn Van Valkenburg, PharmD, PGY1 Pharmacy Practice Resident, G.V. (Sonny) Montgomery VA Medical Center

During my first hospital pharmacy experience, I remember being awe-struck by the confidence exuding from the pharmacist when she gave her recommendations to the attending physicians and other members of the interprofessional team. She seemed at ease discussing the evidence supporting the recommendations. And when there was uncertainty about the next steps, she asked explicit questions to develop a more accurate assessment. I wanted to have this level of confidence in my clinical decision-making, but I was unsure about how to achieve it. I think every student (and resident) seeks to gain a high level of confidence but how can educators assess and cultivate it?

Before measuring confidence, we need to define it. Therein lies the initial problem. Confidence is tricky to define because it is not concrete – you can’t actually see it. It is a belief the action taken is right, proper, and effective.¹ Clinical confidence is the certainty that a decision or action undertaken in the clinical setting is correct and will lead to the best outcome.

One of the interesting aspects of confidence is that it doesn’t always match with knowledge. This mismatch is known as the Dunning–Kruger effect whereby, based on our perceived knowledge, we overestimate our ability. In other words, some knowledge of the subject matter leads us to conclude we are more competent than we actually are when measured using objective tests.²

In a 2006 study, Valdez and colleagues compared second-year pharmacy students’ self-confidence scores regarding the treatment of dyslipidemia and hypertension to their scores on a multiple-choice exam. Confidence was measured using a 12-item questionnaire and rated on a 5-point Likert scale (1=low confidence and 5=high confidence). Each confidence question was linked to a critical concept on the 21-item multiple-choice test. For example, students were asked to rank their confidence “identifying causes of resistant HTN” and this concept was evaluated on one or more items on the multiple-choice test. The confidence assessment (administered first) and multiple-choice test (administered second) were given immediately after students had received didactic instruction about the treatment of patients with dyslipidemia and hypertension. For most items on the test, there was little or no correlation between the students’ level of confidence (mean scores typically = 3.5 to 4.2) and whether (or not) they correctly answered the question. In other words, students who incorrectly answered questions about a concept were just a likely to rate their confidence as a 4 or 5 (moderate-high or high) as students who correctly answered the question. The same confidence and knowledge assessments were administered 4 months later. Interestingly, student confidence remained relatively high (despite the passage of time); however, their retention of the knowledge decreased significantly, by about one letter grade.³ Since the multiple-choice exam was administered after the confidence assessment, it seems clear that students were not able to accurately judge their knowledge. Moreover, as we all know, in the absence of use, knowledge diminishes over time as the “use it or lose it” phrase implies. And yet, students continued to be quite confident in their knowledge even after doing poorly on an exam and with the passage of time.

While learners may over-estimate their knowledge and skill, is it possible to increase their confidence using novel teaching techniques? In a pharmacotherapy laboratory course, teachers at the University of Wisconsin-Madison School of Pharmacy compared the use of paper-based patient-case narratives to the same cases deployed in a simulated Case Scenario/Critical Reader (CSCR) Builder program. The hypothesis was that the simulated environment would increase student engagement, knowledge, and confidence. Each group – paper-based and simulation— completed a 13-item pre-experience confidence survey (0-39 score) regarding their self-perceived ability to manage a patient with osteoarthritis. The simulated-case students had access to an electronic medical record (EMR), could navigate through a series of multiple-choice questions, and could gather information from the simulated patient and physician. The simulation group reported significantly increased confidence in their ability to assess the medication regimen and document the encounter (p < 0.05) when compared to the paper-based group. However, the mean SOAP scores were not significantly different. So, the instructor’s effort (> 20 hours) put into creating a simulated patient case may have increase student confidence but its impact on skill appears to be marginal.⁴

Similarly, instructors at the University of North Carolina at Chapel Hill School of Pharmacy designed a rigorous third-year pharmacy elective where students gained experience with exercise counseling. The students created pamphlets and monitored a patient over a 4-week period. Students who took the elective were more confident counseling patients about exercise and remained more confident 6 months later when compared to students who did not enroll in the course.⁵ Thus, engaging students in practical, hands-on experiences appear to be an important aspect of developing confidence.⁶

Developing one’s confidence is an important step in becoming an effective clinician. Students may be misled by high exam scores into believing their clinical abilities are well developed. This can be problematic because overestimation may result in students inadvertently practicing beyond their level of competence, resulting in patient harm. However, providing students with opportunities to simultaneously employ their knowledge through concrete, real-life experiences improve their clinical confidence and competence.⁶

Recommendations to help students more accurately assess their confidence and competence:

• Measure confidence before administering knowledge and/or skill assessments
• Provide students with engaging ways to learn and test their skills
• If students overestimate their knowledge or skill, challenge them to identify where their knowledge or skill is lacking
• Personal experience, providing students autonomous practice, can help students grow their confidence and competence

Questions yet to be answered:

• What factors influence students’ perception of confidence?
• What is the relationship between clinical experience and confidence?
• What effect does problem-based learning (and other forms of classroom-based problem-solving) have on clinical confidence?

References

1. Confidence. Merriam-Webster's dictionary. 2019.
2. Kruger J1, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999; 77: 1121-34.
3. Valdez CA, Thompson D, Ulrich H, Bi H, Paulsen S. A Comparison of Pharmacy Students’ Confidence and Test Performance. Am J Pharm Ed. 2006; 70 (4) Article 76.
4. Barnett SG, Gallimore CE, Pitterle M, Morrill J. Impact of a Paper vs Virtual Simulated Patient Case on Student-Perceived Confidence and Engagement. Am J Pharm Ed 2016; 80: Article 16.
5. Persky AM. An Exercise Prescription Course to Improve Pharmacy Students’ Confidence in Patient Counseling. Am J Pharm Ed 2009; 73: Article 118.
6. Jih JS, Shrewsbury RP. Student Self-Analysis of Their Nonsterile Preparations and its Effect on Compounding Confidence. Am J Pharm Ed 2018; 82: Article 6473.

-->

Problem-Based Learning: Is It A Better Way To Learn?

by Brandon Hawkins, Pharm.D., PGY1 Pharmacy Practice Resident, University of Mississippi Medical Center

Howard Barrows, one of the earliest champions of problem-based learning (PBL), once defined it as “the learning that results from the process of working toward the understanding or resolution of a problem.”¹ Since its first use at McMaster University School of Medicine in the 1960s, PBL has been used by many other health disciplines including nursing and pharmacy.^2,3 PBL really took off in pharmacy education in the early 2000’s when the Accreditation Council for Pharmacy Education (ACPE) standards proclaimed that the Doctor of Pharmacy curriculum should promote “lifelong learning through the emphasis on active, self-directed learning and … teaching strategies to ensure the adeptness of critical thinking and problem-solving.”⁴ This theme continues today in the 2016 ACPE accreditation standards.⁵

As an instructional method, PBL is primarily designed to empower learners to solve a problem through the application of knowledge. Given its wide range of implementations, there is no universally agreed definition of problem-based learning. Traditionally, PBL usually involves a small group of students with an elected (or appointed) leader and scribe. A faculty member serves as the facilitator whose primary role is to observe the group dynamics and ensure the intended learning objectives are achieved but does not provide any direct instruction. The groups review “trigger material,” such as a patient case or clinical scenario with no prior exposure and without instruction by the facilitator.⁶ Some institutions like the University of Southern California implemented “assisted” PBL, whereby didactic lectures are used to enhance the students’ background knowledge, but the technique still contains elements common to a traditional PBL experience.⁷ However, despite some differences in their implementation in pharmacy education, there are several common themes: discussions in small groups; hypothetical or real case scenarios; facilitation focused on group progress; and self-direction combined with collaborative learning. PBL uses problems in order to develop problem-solving skills as well as reinforce existing and acquire new knowledge.^8,9 Thus, PBL was an obvious choice for many schools and colleges of pharmacy to meet their need for a self-directed form of active learning. However, despite the level of “deep learning” provided by PBL, a common concern is that it may not lead to the same level of performance on standardized exams, which often focus on knowledge recall and memorization.⁶

Various disciplines and institutions have been experimenting with how best to implement PBL and to what degree this teaching strategy should be used throughout the curriculum. Should it be implemented in a single year as preparation for advanced practice experiences? Or used exclusively throughout the entire curriculum? Or sporadically as a substitute for case-based learning? Several investigators have now published about their experiences with PBL. In pharmacy education, feedback from students and educational performance data provide some insight into the methodology’s successes.

In a comparative study conducted at the University of Southern California, student rotation performance was compared after students participated in either PBL (Class of 1995) or received traditional didactic lectures (Class of 1994) during their third year of the pharmacy curriculum.¹⁰  Both groups received the same instruction in the first year of the curriculum and had similar mean GPAs (2.88 vs. 2.9, p=0.1). However, when comparing the graduating classes of 1994 and 1995’s mean GPA during experiential rotations, the PBL group was had significantly higher GPAs for both elective and required rotations (3.29 and 3.38 vs. 3.09 and 3.11, respectively). The authors concluded that PBL produced positive outcomes during fourth-year advanced practice experiences because it increased students’ ability to engage in self-directed learning, increased their independence, and enhanced their decision-making skills. The authors felt these results were important given that the functions, responsibility, and skillsets required during the fourth year of the curriculum are similar to that of pharmacists providing pharmaceutical care.

A relatively recent meta-analysis analyzed 5 studies conducted in Canada, the US, and UK comparing the outcomes of PBL to conventional didactic instruction in pharmacy courses.¹¹ The primary endpoints were midterm and final grades, as well as subjective evaluations. While both the midterm (OR = 1.46; 1.16–1.89) and final (OR = 1.60; 1.06–2.43) grades were significantly higher in the PBL groups, subjective evaluations between the two did not differ. The authors concluded that PBL yielded superior student performance on assessments, while also promoting clinical reasoning and self-directed learning. However, the authors did note that the relatively small sample size may not be large enough to ensure the generalizability to other pharmacy programs.

Student assessment of PBL seems largely positive as well. In one survey, graduates from the University of Mississippi School of Pharmacy were surveyed regarding PBL and the adequacy of their preparation for Advanced Pharmacy Practice Experiences (APPEs).¹²  In disease state/drug therapy discussions, efficient retrieval of current medical literature, and patient-specific evaluation of drug regimens, 50% or more of graduates believed PBL had provided them with above average preparation in these areas. The authors point out that the success of PBL shouldn’t be solely measured by student success on licensing exams, but also students’ perceptions and self-confidence to enter practice.

Though the impetus for many colleges and schools of pharmacy to move toward PBL may have been, in part, to satisfy accreditation standards, it would seem that the results, at least in pharmacy education, suggest this instructional technique is effective. There are multiple sources of data that suggest the PBL is as good as, or possibly superior to, more passive learning strategies such as didactic instruction. However, while assessments and correlations with academic performance are helpful in gauging its efficacy and benefits, it can difficult to truly assess the student experience.

I recently had my first experiences as a PBL facilitator. When I begin a PBL session, I always ask the students what their expectations of the facilitator are. They frequently asked for clinical pearls. While I believe that PBL provides more robust, “real world” examples and deeper learning as a whole, I do think it is valuable for teachers to share “clinical pearls” with students. Traditional PBL offers little opportunity for teachers to share “pro tips,” instead emphasizing how to learn, apply, and approach a complex problem. Though assessments and surveys may indicate that students are more prepared for practice and are generally satisfied with PBL as a learning method, it does leave me wondering if learners are missing out on “fact-based learning” that more traditional methods of instruction afford.


References

1. Barrows H. A taxonomy of problem-based learning methods. Med Educ. 1986;20(6):481-486.
2. Creating Lifelong Learners. London: English National Board; 1994.
3. Ross L, Crabtree B, Theilman G, Ross B, Cleary J, Byrd H. Implementation and Refinement of a Problem-based Learning Model: A Ten-Year Experience. Am J Pharm Educ. 2007;71: Article 17.
4. American Council on Pharmaceutical Education. Chicago; 2000:52-53.
5. Accreditation Council for Pharmacy Education. 2016. Accreditation standards and key elements for the professional program in pharmacy leading to the Doctor of Pharmacy degree. Available at https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf.
6. Wood D. Problem based learning. BMJ. 2003;326(7384):328–30.
7. Romero R, Eriksen S, Haworth I. Quantitative Assessment of Assisted Problem-based Learning in a Pharmaceutics Course. Am J Pharm Educ. 2010;74(4):Article 66.
8. Savery J. Overview of Problem-based Learning: Definitions and Distinctions. Interdisciplinary Journal of Problem-Based Learning. 2006;1(1):9-20.
9. Barrows H. Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning. 1996;Winter 1996(68):3-12
10. Nii L, Chin A. Notes Comparative Trial of Problem-Based Learning Versus Didactic Lectures on Clerkship Performance. Am J Pharm Educ. 1996;60: 162-164.
11. Galvao T, Silva M, Neiva C, Ribeiro L, Pereira M. Problem-Based Learning in Pharmaceutical Education: A Systematic Review and Meta-Analysis. The Scientific World Journal. 2014; Feb:1-7.
12. Hogan S, Lundquist L. The Impact of Problem-based Learning on Students' Perceptions of Preparedness for Advanced Pharmacy Practice Experiences. Am J Pharm Educ. 2006;70:Article 82.