Clinical trial NCT04130607: A Study to Evaluate Strategies for Teaching Effective Use of Diagnostic Tests
Official title: A Study to Evaluate Strategies for Teaching Effective Use of Diagnostic Tests
Study type: INTERVENTIONAL
Overall status: COMPLETED
Phase: NA
Conditions: Instructional Methods
Keywords: Teaching Strategies, Diagnostic reasoning
Lead sponsor: Sentara Norfolk General Hospital
Sponsor class: OTHER
Collaborators: name: McMaster University
First posted: 2019-10-17
Enrollment: 65
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: RANDOMIZED
Intervention model: PARALLEL
Intervention model description: Randomized trial of teaching methods with 3 arms.
Primary purpose: HEALTH_SERVICES_RESEARCH

Brief summary: A recent Institute of Medicine monograph brought attention to high rates of diagnostic error and called for better educational efforts to improve diagnostic accuracy.1 Educational methods, however, are rarely tested and some educational efforts may be ineffective and wasteful.2 In this study, we plan to examine whether explicit instruction on diagnostic methods will have an effect on diagnostic accuracy of 2nd-year medical students and internal medicine residents.

Detailed description: Research has shown that expert diagnosticians use a two-step process to confirm a diagnosis: hypothesis generation to generate diagnostic possibilities, followed by hypothesis verification to confirm the most likely diagnostic possibility.3-5 The first step appears to be non-analytical, related to pattern recognition. The second step could be calculated using analytical reasoning, however, physicians rarely make an overt calculation of conditional probabilities. Instead, experienced clinicians typically use an implicit habit or heuristic called "anchoring and adjusting" to incorporate diagnostic testing information into their thinking.6,7 Cognitive psychologists have postulated that anchoring and adjusting provides a way that probability estimates can be updated based on additional new evidence. Most of the discussion in the literature focuses on how this heuristic can lead to biased thinking because of base-rate neglect or anchoring.6 Very little discussion is on how this heuristic could be improved to yield more accurate probability estimates and whether proper use of the heuristic could be taught.

The degree to which a diagnostic test should lead to an adjustment of a probability estimate depends on the operating characteristics of a test, that is, the sensitivity and specificity. Likelihood ratios, once understood, are easier to incorporate into one's thinking, and thus could be used to calibrate the anchoring and adjusting heuristic.7

In this randomized trial, we tested whether explicit conceptual instruction on Bayesian reasoning and likelihood ratios would improve Bayesian updating, compared with a second intervention where we provided multiple (27) examples of clinical problem solving. The third arm provided minimal teaching about diagnosis, but no explicit teaching or examples.

Eligibility criteria: Inclusion Criteria:

* Medical Student at McMaster University or Eastern Virginia Medical School
* Completed 18 months of coursework

Primary outcomes:
measure: Accuracy of participants probability revisions were compared to posttest probability revisions that were calculated using Bayes Rule. An effect size was calculated to measure how close students matched the calculated revision.
timeFrame: Post-test was taken within 72 hours of instructional phase completion.
Secondary outcomes:
Other outcomes:
Locations:
city: Norfolk
country: United States
facility: Sentara Norfolk General Hospital
state: Virginia
Study officials:
References:
citation: Brush JE Jr, Lee M, Sherbino J, Taylor-Fishwick JC, Norman G. Effect of Teaching Bayesian Methods Using Learning by Concept vs Learning by Example on Medical Students' Ability to Estimate Probability of a Diagnosis: A Randomized Clinical Trial. JAMA Netw Open. 2019 Dec 2;2(12):e1918023. doi: 10.1001/jamanetworkopen.2019.18023.