# Deep Learning Enhances Radiologists' Detection of Potential Spinal Alignancies in CT Scans
## 3 Results
### Intra-Observer Agreement without and with the Aid of the DLA
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Participants**} \\ \cline{2-8}  & **1** & **2** & **3** & **4** & **5** & **6** \\ \hline
**Experience** (years)** & 6 & 7 & 5 & 1 & 11 & 1 \\ \hline  & \multicolumn{6}{c}{**Phase 1—no support from DLA**} & \multicolumn{6}{c}{**Mean**} \\ \hline
**Suspicious** & & & & & & & \\
**lesions** & 2.00 & 4.00 & 3.00 & 0.00 & 2.00 & 0.00 & 1.83 \\
**reported (n)** & & & & & & & \\ \hline
**False** & & & & & & & \\
**positives (n)** & 0.00 & 0.00 & 0.00 & 0.00 & 0.00 & 0.00 & 0.00 \\ \hline
**Sensitivity** & & & & & & & \\
**(case level)** & 12.50\% & 25.00\% & 18.75\% & 0.00\% & 12.50\% & 0.00\% & 11.46\% \\ \hline
**Sensitivity** & & & & & & & \\
**(object level)** & 7.41\% & 14.81\% & 11.11\% & 0.00\% & 7.41\% & 0.00\% & 6.79\% \\ \hline
**Specificity** & & & & & & & \\
**(case level)** & 100.00\% & 100.00\% & 100.00\% & 100.00\% & 100.00\% & 100.00\% \\ \hline
**Accuracy** & & & & & & & \\
**(case level)** & 56.25\% & 62.50\% & 59.38\% & 50.00\% & 56.25\% & 50.00\% & 55.73\% \\ \hline
**Accuracy** & & & & & & & \\
**(object level)** & 53.70\% & 57.41\% & 55.56\% & 50.00\% & 53.70\% & 50.00\% & 53.40\% \\ \hline
**False positive** & & & & & & & \\
**rate** & 0.00\% & 0.000\% & 0.000\% & 0.000\% & 0.000\% & 0.000\% \\ \hline  & \multicolumn{6}{c}{**Phase 2—with support from DLA**} & \multicolumn{6}{c}{**Mean**} \\ \hline
**Suspicious** & & & & & & & \\
**lesions** & 5.00 & 5.00 & 8.00 & 0.00 & 14.00 & 3.00 & 5.83 \\
**reported (n)** & & & & & & & \\ \hline
**False** & & & & & & & \\
**positives (n)** & 1.00 & 0.00 & 1.00 & 0.00 & 2.00 & 1.00 & 0.83 \\ \hline
**Sensitivity** & & & & & & & \\
**(case level)** & 31.25\% & 31.25\% & 43.75\% & 0.00\% & 75.00\% & 12.50\% & 32.29\% \\ \hline
**Sensitivity** & & & & & & & \\
**(object level)** & 14.81\% & 18.52\% & 25.93\% & 0.00\% & 44.44\% & 7.41\% & 18.52\% \\ \hline
**Specificity** & & & & & & & \\
**(case level)** & 93.75\% & 100.00\% & 93.75\% & 100.00\% & 87.50\% & 93.75\% & 94.79\% \\ \hline
**Accuracy** & & & & & & & \\
**(case level)** & 62.25\% & 65.62\% & 68.75\% & 50.00\% & 81.25\% & 53.12\% & 63.54\% \\ \hline
**Accuracy** & & & & & & & \\
**(object level)** & 55.56\% & 59.26\% & 68.12\% & 50.00\% & 68.52\% & 51.85\% & 58.89\% \\ \hline
**False-positive** & & & & & & & \\
**rate** & 3.70\% & 0.00\% & 3.57\% & 0.00\% & 7.41\% & 3.70\% & 3.06\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Intra-observer agreement depicted for the two study phases according to the gold standard (phase 1—without the DLA and phase 2—with the DLA support). Based on the AI's indication of true-positive spinal conditions, four participants felt requesting further follow-up or diagnostic tests was necessary. These requests were made to investigate potential abnormalities or conditions related to the spine, as suggested by the AI. None of the participants requested additional examinations due to possible FP findings. On a case level, participants' average sensitivity in detecting spinal lesion(s) (which translates into reporting at least one correct lesion in a positive case) increased from 11.5% to 32.3% (20.08 percentage points) when using the DLA tool. The sensitivity increased from 6.8% to 18.5% (11.70 percentage points) on an object level. The average FP rate increased from 0.0% (no primarily reported false spinal findings) to 3.1% on a case and object level. The mean accuracy value increased from 55.7% to 63.5% on a case level and from 53.4% to 58.9% on an object level. There was no clear trend that could link the participants' clinical experience and their responsiveness to the AI predictions, with both junior and more experienced radiologists having heterogeneous behaviors toward the presented algorithm results. It should be noted that, in less-experienced participants, we observed changes in their reports between reading sessions unrelated to the AI findings.