Variability of pre-test probability for obstructive coronary artery disease between referring primary care physicians and estimations from the referral letters
Authors: Stertman E , Gabro F , Sandstedt M , Sysoev O , Sederholm Lawesson S , Engvall E , Nilsson S , Iredahl F 
Published: October 2024

Background

Estimation of Pre-Test Probability (PTP) for obstructive Coronary Artery Disease (CAD), derived from sex, age and characterization of chest pain or dyspnea, is recommended by European guidelines to guide further investigation. When Coronary Computed Tomography Angiography (CCTA) was introduced as first line investigation in patients with PTP &lt;15%, in the primary care of a catchment area of ~470 000 inhabitants in Sweden, attachment of PTP was made compulsory to the referral. The interrater variability, or consistency of agreement, of PTP estimation is not known. The validity of PTP to guide healthcare decisions depends to some extent on the reliability of estimations. Appropriate prioritization of invesigations is reliant on the information included in referral letters.

Purpose

The purpose was to study the interrater variability of estimations of PTP. We hypothesized: 1) the interrater variability of PTP estimations to be less than excellent (ICC &lt; 0.9), and 2) not statistically different among PTP classes; 3) a significant difference between clinical estimations and estimations made from the referral letters.

Methods

We studied the 545 patients referred for CCTA from primary care, 1st of June 2021 to 7th of Dec. 2022, with available PTP estimations. The interrater variability, expressed as Intraclass Correlation Coefficients (ICC), was determined for estimations of PTP by the referring physicians (RP) and three external raters (ER). The latter were physicians practising in primary care who rated PTP based on the referral letters to CCTA, independently and blinded. Each rater assessed multiple patients and each patient was assessed by all raters. ICC with a two-way random consistency model for single raters was calculated and evaluated according to a proposed standard(1).

Results

The ICC for PTP estimations between the RP and the median of three ER was 0.75 (CI 95 % 0.71-0.83, p&lt;0.001). ICC was higher for PTP class &gt; 15% compared to the lower classes: &lt;5% (n = 65, ICC ≈ 0); 5-15% (n = 343, ICC 0.48, CI 95 % [0.39-0.56]); &gt;15% (n = 137, ICC 0.71, CI 95 % [0.62-0.78]) (Figure 1).

The ICC for PTP estimations was significantly higher between the three ER (0.88, SE 0.009, CI 95% [0.86-0.89]) than when the RP was added as a rater (ICC 0.80, SE 0.012, CI 95 % [0.77-0.82]). Adding a fixed effect of the difference in ratings between RP and ERs to the ICC model did not change model output (R = 0.80 (SE 0.012, CI 95 % [0.77-0.82]).

Conclusions

There is a good repeatability, in estimation of PTP based on medical history as described in referral letters, between blinded independent reviewers of CCTA referrals. The repeatability between the external raters and the referring physicians’ compulsory entered PTPs is lower, yet good. The variability of PTP estimations, speculatively from disagreement on main symptom and inconsistency in characterization of chest pain, may impact patient selection and outcomes and is understudied.