# Evaluating script concordance tests (SCTs) through the lens of Bayesian reasoning: Enhancing assessment in medical education.

Luc Dauchet, Raphael Bentegeac, H. Ghauss, Marc Hazzan, Patrick Truffert,
Philippe Amouyel, Victoria Gauthier, Aghiles Hamroun

<figure><img src="image_1.png" /></figure>

## To cite this version:

Luc Dauchet, Raphael Bentegeac, H. Ghauss, Marc Hazzan, Patrick Truffert, et al.. Evaluating script concordance tests (SCTs) through the lens of Bayesian reasoning: Enhancing assessment in medical education.. Journal of Epidemiology and Population Health, 2025, J Epidemiol Popul Health, 73, pp.202804. 10.1016/j.jeph.2024.202804. hal-04941880

HAL Id: hal-04941880

https://lilloa.hal.science/hal-04941880v1

Submitted on 12 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
Special communication

# Evaluating script concordance tests (SCTs) through the lens of Bayesian reasoning: Enhancing assessment in medical education

Luc Dauchet<sup>a,b</sup>, Raphaël Bentegeac<sup>a,b</sup>, Haress Ghauss<sup>a,b</sup>, Marc Hazzan<sup>d</sup>, Patrick Truffert<sup>c</sup>,
Philippe Amouyel<sup>a,b</sup>, Victoria Gauthier<sup>a,b</sup>, Aghiles Hamroun<sup>a,b,*</sup>

<figure><img src="image_3.png" /></figure>

## ARTICLE INFO

**Article History:**

Received 1 August 2024

Accepted 16 December 2024

Available online 22 January 2025

**Keywords:**

Script concordance test

SCT

Clinical reasoning

Probabilistic reasoning

Bayesian reasoning

## ABSTRACT

**Background:** Script Concordance Tests (SCTs) represent an innovative assessment method which have been introduced in the 2024 French National Ranking Examinations (EDN). These tests compare a student's clinical reasoning with that of a panel of experts under conditions of uncertainty. Typically, the question involves the impact of new information on an initially proposed hypothesis, with answers given on a Likert scale.

**Main findings:** This article aims to didactically illustrate how SCTs are consistent with probabilistic reasoning as modeled by Bayes' theorem. In addition, by comparing SCT writing guidelines with Bayesian reasoning concepts, several ambiguities were identified: (1) What stage of clinical reasoning do SCTs evaluate? (2) What are the appropriate labels for Likert scale responses? (3) Does the expert panel provide a relevant reference for SCTs?

**Conclusions:** Currently, many of these questions remain unanswered in the literature, with recent data suggesting that experienced physicians' responses to SCTs are often biased. Beyond their use as an assessment tool in the EDN, SCTs offer a valuable opportunity to develop and deepen the teaching of probabilistic reasoning in medical education and serve as a potential area of research to improve clinical practice.

© 2024 The Author(s). Published by Elsevier Masson SAS. This is an open access article under the CC BY license ([http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/))

## 1. Introduction

In France, the reform of the second cycle of medical studies (R2C) places a competency-based approach at the forefront of medical training, with a strong emphasis on clinical and scientific reasoning [1]. The assessment of clinical reasoning is a key challenge within the R2C framework and requires a variety of assessment tools, including script concordance tests (SCTs), which have been included in the French National Ranking Examination (EDN) in 2024. Developed by Charlin et al in the 1990s, SCTs are based on the observation that healthcare professionals' hypotheses evolve with new information [2]. SCTs help students clarify critical steps in their clinical reasoning by comparing their answers with those of a panel of experts.

SCTs consist of a clinical vignette and a series of questions. Each question features three columns: the initial hypothesis, new clinical or paraclinical data, and a 5-point Likert scale for students to rate the impact of the new information. The Likert scale ranges from -2 to +2, reflecting the influence of the new data. Scoring is based on the degree of concordance between the student's response and those of

the expert panel, with the modal response receiving the highest score. Despite the fact that SCT is being used on a large scale with French medical students, some issues remain controversial. Lineberry et al. [3] discussed the validity of the expert panel and the validity of the statistical analysis used in the literature to evaluate the reliability of SCTs. They also pointed out that students who avoid extreme responses would artificially increase their score.

This article aims to illustrate how SCTs resonate with probabilistic reasoning, as modeled by Bayes' theorem. Consequently, we will detail specific design challenges of SCTs, particularly in terms of question framing and Likert scale interpretation, which are brought to light through this parallel with probabilistic reasoning concepts. By understanding these issues, we could improve the efficacy and fairness of SCTs in evaluating medical students' clinical reasoning skills.

## 2. SCT: A natural bridge between clinical and probabilistic reasoning?

In our experience of drafting and responding to SCTs as panelists, we have observed an obvious link between the structure of SCT and probabilistic reasoning. The latter, corresponding to the mathematical model that allows incorporating new information within the

<footnote><sup>a</sup> Service de Santé Publique, Epidémiologie, Economie de la Santé et Prévention, CHU Lille, 59000 Lille, France</footnote>

<footnote><sup>b</sup> UMR1167 RIDAGE, Institut Pasteur de Lille, Inserm, Univ. Lille, CHU Lille, 59000 Lille, France</footnote>

<footnote><sup>c</sup> Service de Néonatalogie, Hôpital Jeanne de Flandres, Univ. Lille, CHU Lille, 59000 Lille, France</footnote>

<footnote><sup>d</sup> Service de Néphrologie, Dialyse, Transplantation rénale et Aphérèse, Hôpital Claude Huriez, Univ. Lille, CHU Lille, 59000 Lille, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: [aghiles.hamroun@univ-lille.fr](mailto:aghiles.hamroun@univ-lille.fr) (A. Hamroun).</footnote>
framework of a diagnostic hypothesis, is, however, rarely applied in clinical practice. Thus, when an SCT questions the contribution of new information to either confirm or reject a hypothesis, it inherently addresses an approach corresponding to Bayesian reasoning. Although SCTs appear to explore Bayesian reasoning, none of the main guidelines mention probabilistic reasoning or Bayes' theorem [4–9].

## 2.1. What do we mean by Bayesian reasoning?

Probabilistic reasoning, also known as Bayesian reasoning, is a type of inductive reasoning that quantifies uncertainty in terms of probabilities, using Bayes' law of conditional probabilities [10]. More practically, probabilistic reasoning can be considered as the act of changing one's mind, with a prior confidence in a hypothesis and updating that confidence after observing new information. In a clinical setting, the hypothesis formulated is very often that of a disease, and the information can be any element of semiology: a clinical sign, a symptom, the result of a diagnostic test, a history, or others. Relevant new information may therefore modify our confidence in this hypothesis, which then becomes an a posteriori hypothesis.

### 2.1.1. Bayes' theorem and use of odds

Bayes' formula, also known as the theorem of conditional probabilities, allows modeling the evolution of a prior probability into a posterior probability [11], taking into account the contribution of new information (e.g., the result of a diagnostic test).

$$p(M|D) = p(M) \frac{p(D|M)}{p(D|M)p(M) + p(D|\neg M)p(\neg M)}$$

The posterior probability (given the new observed data) p(M|D) is thus equal to the initial probability p(M) (before the new data) multiplied by the quantity: $\frac{p(D|M)}{p(D|M)p(M)+p(D|\neg M)p(\neg M)}$

This formula, hardly usable, and as described previously [12], can be more easily translated using odds. An odd, like a probability, is a way of measuring the chances that an event will occur or measuring a level of confidence in a hypothesis. Indeed, odds and probabilities are related one-to-one by a simple formula where O is the odd.

It is important not to confuse odds with odds ratios: an odds ratio is a ratio of two odds, just as a relative risk is a ratio of two probabilities.

$$O(event) = \frac{p(event)}{1 - p(event)}$$

One can understand this relationship with a simple example. If we have 2 green balls out of 3 in a jar, we have a probability of (2 / 3) -2 chances out of 3 -of getting a green ball at random, but one could also say that we have an odd of (2 / 1) -2 chances to 1- of this happening.

Bayes' formula in the language of odds can thus be simplified. Here, we continue to denote M as the hypothesis "the patient has the disease," $\neg M$ as "the patient does not have the disease," and D as the new observed data.

$$O(M|D) = O(M) \frac{p(D|M)}{p(D|\neg M)}$$

The posterior confidence level (or odds) O(M|D) is equal to the prior confidence level (or odds) O(M) multiplied by a confidence change amount $\frac{p(D|M)}{p(D|\neg M)}$

### 2.1.2. The likelihood ratio: A tool to quantify opinion change regarding the hypothesis likelihood

This quantity of change in the hypothesis confidence is also called the likelihood ratio RV<sub>D</sub> of the data.

$$O(M|D) = O(M) \cdot RV_D$$

If the RV<sub>D</sub> is greater than 1, then the information increases our confidence in the diagnostic hypothesis, and this is referred to as a positive likelihood ratio.

If the RV<sub>D</sub> is less than 1, then the information decreases our confidence in the diagnostic hypothesis, and this is referred to as a negative likelihood ratio.

Thus, likelihood ratios represent measures of belief updating, based on a ratio between two probabilities [13]. The likelihood ratio of new data thus represents the comparison of two thought experiments: comparing the likelihood of the observed data in a world where the patient has the disease with the likelihood of the observed data in a world where the patient does not have the disease. If the new information is much more likely in the world where the patient has the disease, then the likelihood ratio will be greater than 1, indicating a positive likelihood ratio favoring the disease hypothesis. Conversely, if the new information is much less likely in the world where the patient has the disease, then the likelihood ratio will be <1, indicating a negative likelihood ratio that disfavors the disease hypothesis.

### 2.1.3. Bayes' theorem and Fagan's nomogram

The probabilistic reasoning approach can also be carried out using Fagan's nomogram [14]. The construction of this nomogram is based on the same mathematical manipulation of odds without requiring calculations. This nomogram consists of three vertical scales, corresponding to the prior probability (pre-test or pre-information), the likelihood ratio (positive or negative) of the new information (or diagnostic test), and the posterior probability (post-test or post-information, sometimes referred to as the predictive value in the case of a diagnostic test) [14].

For example, if we consider a patient who is coughing without fever or known contacts during an epidemic period, the probability of having COVID is initially low, $p(M) = (1/10)$. However, we learn that the antigen test, with a very high positive likelihood ratio ($RV_+ = 90$) [15], has returned positive. We can estimate the posterior probability that this patient has COVID. To make this estimate using the nomogram, we can simply draw a line from our initial probability on the left vertical scale through the reported likelihood ratio on the central vertical scale. The point of intersection with the right vertical scale is the posterior probability, here 91% (Fig. 1).

It is important to note that the estimation of a posterior probability involves both the likelihood ratio and the prior probability. The frequency argument, our starting point, can therefore strongly influence the posterior probability. This explains why it takes much more evidence to convince us of the hypothesis of a rare disease than it does to convince us of a common pathology.

## 2.2. From Bayes' theorem to SCT: Illustration with Fagan's nomogram

As described previously, SCT attempts to explore medical students' ability to evaluate the influence of new clinical or paraclinical information [2]. By definition, SCT appears to be approaching the assessment of probabilistic reasoning.

Let us look at the structure of an SCT and its breakdown into three columns: on the left, column 1 leads to the formulation "If you thought about......" including a clinical vignette offering elements to approach the probability of a priori hypothesis; in the middle, column 2, "and then you find out......" corresponds to new information that could influence the prior hypothesis; and, on the right, column 3, "this new information makes your diagnosis......" corresponds to the conclusion of the SCT, which aims to evaluate the impact of the new information on the prior probability (Supplementary Table 1). These three columns virtually mirror the three scales presented in Fagan's nomogram.

The classical numerical representation of the final answer in the form of a Likert scale ranging from -2 to +2 (passing through 0) intuitively mimics the range of the likelihood ratio, from negative to
<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Practical Application of Fagan's Nomogram: COVID-19 Antigen Test Example. (adapted from Dauchet et al 2024 [16]).</figcaption></figure>

positive (0 to +∞, passing through 1 corresponding to no contribu-
tion from the new information). Fig. 2 thus illustrates the close link
we just described between the formulation of SCT and probabilistic
reasoning as visualized through Fagan's nomogram.

The objective of this section is to show i) how "the impact of new
information" can lead to different interpretations due to the lack of
explicitness in the response format, and ii) why it is necessary to har-
monize the wording of SCT response scales so that both students and
panelists can have the same understanding.

**3. SCTs' response format: a detail that matters?**

In our experience, we have encountered difficulties related to the
wording of the response modalities recommended in the literature.

3.1. When the likert scale creates a dilemma for the respondent

Let us take an example of an SCT adapted from Fournier et al. [7]
(Table 1). In this example, it is difficult, if not impossible, to choose

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Relationship Between Probabilistic Reasoning and Script Concordance Tests (SCTs): Illustration by Fagan's Nomogram.</figcaption></figure>
<table><caption>**Table 1**
Fictional SCT Example (adapted from Fournier JP et al. [7]).</caption><caption>You enter the room of Mr. X, an 86-year-old patient hospitalized in your unit for heart failure. Upon entering, you notice that he is blue and unconscious.</caption><thead><tr><th>If you were planning to:</th><th>And then you learned:</th><th>This action would become :</th></tr></thead><tbody><tr><td>Check his pulse</td><td>It was his birthday</td><td>-2 -1 0 +1 +2</td></tr></tbody></table><p>2: completely unnecessary<br>-1: unnecessary<br>0: neither more nor less useful<br>+1: more useful<br>+2: very useful</p>

between response modality 0 (the new information does not brings any element likely to change the course of action) and modality +2 (in absolute terms, with or without the new information, the planned action of taking the pulse remains very useful). The margin of interpretation can lead to significant response variability for an identical perception of the clinical situation, even to an inability to respond coherently. This ambiguity in the proposed response format has led us to question the possible interpretations of the impact of new information within an SCT. The analogy with probabilistic reasoning helps to highlight the different possible meanings of the question posed.

## 3.2. Likert scale: Different scenarios for different interpretations

Several interpretations are possible for estimating "the impact of new information" on the prior hypothesis and depend directly on the wording of the Likert scale labels (Fig. 3). We will describe as "relative" a Likert scale that quantifies the change in the probability of the prior hypothesis under the effect of new information. The labels of the scale will then be of the type (-2 much less likely, -1 less likely, 0 neither more nor less likely, +1 more likely, +2 much more likely), and as "absolute" a scale that estimates the revised (posterior) probability of the hypothesis following the new information. The labels of the scale will then be of the type (-2 very unlikely, -1 unlikely, 0 moderately likely, +1 quite likely, +2 very likely). Even in the case where the "relative" Likert scale is used, it can still be interpreted in two different ways: either one evaluates the intrinsic value of the new

information independently of the context, modeled in probabilistic reasoning by the likelihood ratio (scenario A), or one evaluates the variation between the prior and posterior probabilities (scenario B). Although intuitively related, these two ways of evaluating the impact of new information must be distinguished. In scenario A, the response is independent of the prior probability formulated from the elements present in the clinical vignette, which is not the case in scenario B. Thus, these two interpretations can lead to discordant responses, especially in situations of extreme prior probability (very low or very high).

In summary, depending on the formulation and objective of the SCT, it is possible to evaluate three types of responses corresponding to three different scenarios:

* Scenario A: the likelihood ratio of the new information (intrinsic value of the new information)

* Scenario B: the difference between the prior and posterior probabilities of the hypothesis

* Scenario C: the posterior probability of the hypothesis resulting from the new information

These three ways of interpreting the question are all relevant, but they present different levels of difficulty. Scenario A is probably the less complex since it is independent of the correct estimation of the prior probability of the hypothesis. Scenarios B and C, on the other hand, are more difficult. The student must be able to correctly

<figure><img src="image_6.png" /><figcaption>**Fig. 3.** Script Concordance Tests (SCTs), Likert scales, and probabilistic reasoning interrelation, illustrated by Fagan's nomogram.</figcaption></figure>
Likert Scale and Interpretation Challenges: Practical Examples

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>**Fig. 4.** Illustrating Interpretation Difficulties in Script Concordance Tests (SCTs): Practical Examples.</figcaption></figure>

estimate the prior probability of the hypothesis as well as the likelihood ratio of the new information in order to deduce the posterior probability combining these two pieces of information.

In addition to the uncertainty surrounding the question posed, these different Likert scale response formats can lead to variations in the responses given by the students and the expert panel for the same clinical situation (Fig. 4). In Example 1, with a very low prior probability of 0.1 %, a satisfactory positive likelihood ratio of 10 (reflecting a significant intrinsic value of the new information), the posterior probability of the hypothesis would be 1 % (a difference of 0.9 points of percentage between the prior and posterior probabilities). The responses provided depending on the scenario considered can then be very different: from a satisfactory likelihood ratio expressing good intrinsic informative value (+2 on the Likert scale), to a still low posterior probability of 1 % (-2 on the Likert scale), to a small difference between the prior and posterior probabilities (0 on the Likert scale). Fig. 4 provides further examples of how, with identical perceptions of the clinical situation, responses can be very different depending on the wording and understanding of the Likert scale labels.

### 3.3. What does the literature tell us?

We synthesized definitions of the questions posed and proposed Likert scales for script concordance tests (SCT) based on reference articles, books, and writing guides (Supplementary Table S2). Our analysis identified six reviews and methodological guides, mainly from Canadian, French, and Belgian sources [4–9]. The main objective of SCTs is to assess how new information affects the initial hypothesis; however, this goal is often vaguely defined, making it difficult to identify the specific scenario targeted by the exercise.

The labels on the Likert scale should clearly indicate the intended scenario. Many guidelines mix labels for absolute interpretations at extreme values (e.g., "the hypothesis is practically ruled out" to "this can practically only be this hypothesis") with those for relative interpretations at median values (e.g., "the information has no effect on the hypothesis"). For example, the widely cited diagnostic scale by Fournier et al. presents an asymmetric format with labels such as "very unlikely" (-2) and "very likely" (+2), where the extreme ratings refer to absolute scales, while middle ratings refer to relative assessments [7]. This mix creates ambiguity and complicates response selection, as multiple options may appear non-exclusive, obscuring

the original intent of the SCT. To clarify the Likert scale, the first step is to use consistent items. The scale should have items related only to the variation in probability (e.g. -2 much less likely, -1 less likely, 0 neither more nor less likely, +1 more likely, +2 much more likely) or items related only to the final probability (-2 very unlikely, -1 unlikely, 0 neither likely nor unlikely, +1 likely, +2 very likely).

However, this is not sufficient to guarantee that everyone will interpret it the same way. For example, scenarios A and B may use the same scale. Therefore, the scenario to which the question refers should be made explicit. In addition, both the panel and the students should be trained in Bayesian reasoning to ensure that they have a good understanding of the question. However, this may not be enough to ensure the accuracy of the panel's probabilistic reasoning.

### 4. The expert panel may not be the most relevant reference

SCTs assess students' reasoning by comparing their answers to those of panels of experts, with the most common expert response receiving the highest score. This approach assumes that experts are the gold standard because of their skills and knowledge. However, experts also face challenges in estimating conditional probabilities, which are prone to biases [16]. A study in 2020 found systematic deviations in human probabilistic reasoning, suggesting that even experts struggle with optimal reasoning [17]. Consistently, a 2015 review by Whiting et al. found that clinicians often overestimate post-test probabilities due to misunderstandings of diagnostic performance measures [18]. A 2021 study involving 723 US physicians confirmed this trend, showing that even experienced physicians often overestimate post-test probabilities in common scenarios such as pneumonia and myocardial ischemia [19].

These findings question the reliability of expert panels as the gold standard in SCTs [16]. Biases in probabilistic reasoning may disadvantage students who reason more accurately or have been trained to Bayesian methods [20] but whose answers would differ from the expert consensus. A recent French study involving 898 medical trainees and practitioners found that 85 % failed to correctly estimate post-test probabilities despite high confidence, and that more experienced respondents were not more accurate but often more confident in their errors [21].

These findings suggest a need to reevaluate the use of expert panels in SCTs and to explore better methods for teaching and assessing probabilistic reasoning in medical education in order to improve
the validity of SCTs and prepare students clinical decision-making. An alternative to the expert panel could be to estimate pre- and post-test probabilities and likelihood ratios using data extracted from a literature review, as was done in a previous research protocol [19]. However, this requires a lot of research time and can only be done for clinical questions with available data. It is applicable in experimental studies, but it does not seem feasible for academic examinations where a large number of questions are required. This question format may limit the impact of bias on probabilistic reasoning, assuming that the bias is similar for all clinical findings and does not affect ranking. However, this format has not yet been evaluated. Kreiter et al. have proposed an alternative to SCT (see Supplementary Figure 1) [22].

## 5. Conclusions

While SCTs inherently assess probabilistic reasoning, this connection is not emphasized in the reference materials and supporting documentation. Applying the concept of Bayesian reasoning to SCTs reveals several ambiguities for students, instructors, and panelists. Key challenges include:

1. The specific question posed to students via SCTs is unclear: are we assessing their ability to estimate the discriminative value of new information, the posterior probability, or the change in probability from pre- to post-information?

2. The Likert scale introduces significant ambiguity: the question should focus on a single indicator that is explicitly stated in the prompt. Current guidelines mix scales that evaluate both the magnitude of opinion change and the absolute posterior probability estimate.

3. Clinicians typically integrate diverse information, but SCTs confine them to a scenario with limited data, requiring precise estimation of prior probability, likelihood ratio, and posterior probability. The literature questions the reliability of expert panels as an optimal gold standard for this exercise.

Integrating Bayesian reasoning more explicitly into SCTs could enhance their use in evaluating medical students and help educators design clearer assessments. However, current SCT designs, as outlined in guidelines, create ambiguity regarding probabilistic reasoning.

## Conflict of interest

The authors do not have any conflict of interest to declare.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jeph.2024.202804.

## References

[1] Cohen Aubart F, Roux D. Reform of the second cycle of medical studies in France: risk or opportunity? Rev Med Interne 2021;42(3):149-53. doi: 10.1016/j.revmed.2020.12.005.

[2] Charlin B, Brailovsky C, Leduc C, Blouin D. The Diagnosis script questionnaire: a new tool to assess a specific dimension of clinical competence. Adv Health Sci Educ Theory Pract 1998;3(1):51-8. doi: 10.1023/A:1009741430850.

[3] Lineberry M, Kreiter CD, Bordage G. Threats to validity in the use and interpretation of script concordance test scores. Med Educ 2013;47(12):1175-83. doi: 10.1111/medu.12283.

[4] Dory V, Gagnon R, Vanpee D, Charlin B. How to construct and implement script concordance tests: insights from a systematic review. Med Educ 2012;46(6):552-63. doi: 10.1111/j.1365-2923.2011.04211.x.

[5] Sibert L, Fournier JP, Charlin B. ECNi épreuve TCS Test de concordance de script :Le Guide Méthodologique. Paris: Maloine; 2015.

[6] Caire F, Sol JC, Charlin B, Isodiri P, Moreau JJ. Le test de concordance de script (TCS) comme outil d'évaluation formative des internes en neurochirurgie : implantation du test sur Internet à l'échelle nationale. Pédagogie Médicale 2004;5(2):87-94. doi: 10.1051/pmed:2004018.

[7] Fournier JP, Demeester A, Charlin B. Script concordance tests: guidelines for construction. BMC Med Inform Decis Mak 2008;8:18. doi: 10.1186/1472-6947-8-18.

[8] Giet D, Massart V, Gagnon R, Charlin B. Le test de concordance de script en 20 questions. Pédagogie Médicale 2013;14(1):39-48. doi: 10.1051/pmed/2012026.

[9] Lubarsky S, Dory V, Duggan P, Gagnon R, Charlin B. Script concordance testing: from theory to practice: AMEE guide no. 75. Med Teach 2013;35(3):184-93. doi: 10.3109/0142159X.2013.760036.

[10] Hornberger J. Introduction to Bayesian reasoning. Int J Technol Assess Health Care 2001;17(1):9-16. doi: 10.1017/s0266462301104022.

[11] Bours MJ. Bayes' rule in diagnosis. J Clin Epidemiol 2021;131:158-60. doi: 10.1016/j.jclinepi.2020.12.021.

[12] Nixon MP, Momotaz F, Smith C, Smith JS, Sendak M, Polage C, et al. From pre-test and post-test probabilities to medical decision making. BMC Med Inform Decis Mak 2024;24(1):210. doi: 10.1186/s12911-024-02610-3.

[13] Richardson WS, Wilson MC, Keitz SA, Wyer PC. Tips for teachers of evidence-based medicine: making sense of diagnostic test results using likelihood ratios. J Gen Intern Med 2008;23(1):87-92. doi: 10.1007/s11606-007-0330-1.

[14] Fagan TJ. Letter: Nomogram for Bayes's theorem. N Engl J Med 1975;293(5):257. doi: 10.1056/NEJM197507312930513.

[15] Arshadi M, Fardsanei F, Deihim B, Farshadzadeh Z, Nikkhahi F, Khalili F, et al. Diagnostic accuracy of rapid antigen tests for COVID-19 detection: a systematic review with meta-analysis. Front Med 2022;9:870738. doi: 10.3389/fmed.2022.870738.

[16] Dauchet L, Bentegeac R, Ghauss H, Hazzan M, Truffert P, Amouyel P, et al. Le panel d'experts pour les tests de concordance de script : une référence réellement adéquate ? [The expert panel for Script Concordance Tests: A truly adequate reference?]. Rev Med Interne 2024;45(12):776-81 French. doi: 10.1016/j.revmed.2024.05.023.

[17] Dasgupta I, Schulz E, Tenenbaum JB, Gershman SJ. A theory of learning to infer. Psychol Rev 2020;127(3):412-41. doi: 10.1037/rev0000178.

[18] Whiting PF, Davenport C, Jameson C, Burke M, Sterne JA, Hyde C, et al. How well do health professionals interpret diagnostic information? A systematic review. BMJ Open 2015;5(7):e008155. doi: 10.1136/bmjopen-2015-008155.

[19] Morgan DJ, Pineles L, Owczarzak J, Magder L, Scherer L, Brown JP, et al. Accuracy of practitioner estimates of probability of diagnosis before and after testing. JAMA Intern Med 2021;181(6):747-55. doi: 10.1001/jamainternmed.2021.0269.

[20] Brush JE, Lee M, Sherbino J, Taylor-Fishwick JC, Norman G. Effect of teaching Bayesian methods using learning by concept vs learning by example on medical students' ability to estimate probability of a diagnosis: a randomized clinical trial. JAMA Netw Open 2019;2(12):e1918023. doi: 10.1001/jamanetworkopen.2019.18023.

[21] Lakhlii C, Lejeune FX, Rouault M, Khamassi M, Rohaut B. Illusion of knowledge in statistics among clinicians: evaluating the alignment between objective accuracy and subjective confidence, an online survey. Cogn Res Princ Implic 2023;8(1):23. doi: 10.1186/s41235-023-00474-1.

[22] Kreiter CD. A Bayesian perspective on constructing a written assessment of probabilistic clinical reasoning in experienced clinicians. J Eval Clin Pract 2017;23 (1):44-8. doi: 10.1111/jep.12469.