Deep learning improves detection of aortic stenosis in b-mode ultrasound: results of a pivotal multicenter validation study using artificial intelligence
Authors: Poilvert N , Goldstein S , Phelan D , Rubenson D , Mott B , Mutharasan K , Nishtala A , Schultz W , Sedehi D , Thomas Y , Marra A , Martin R , Lang R , Thomas J 
Published: October 2024

Abstract

Background
Aortic stenosis (AS) is a highly prevalent and morbid condition, with moderate or severe AS present in 9% of patients over 75 years. Despite improved diagnostic and therapeutic methods, AS remains underdiagnosed, particularly in Black patients and underserved communities. Echocardiography (echo) is effective for detecting AS but requires expertise to quantify and interpret valve gradients and valve area. We hypothesized that a deep learning (DL) algorithm could be trained to detect AS severity using B-mode (non-Doppler) echo alone, the modality most likely to be available in point-of-care ultrasound (POCUS) studies. Here, we present the results of a multicenter pivotal trial to test whether artificial intelligence (AI) can help increase the detection of AS from B-mode echoes.

Methods
A convolutional neural network (CNN) was trained to classify the severity of aortic stenosis from 3 B-mode echo views of the aortic valve (parasternal long- and short-axis and apical 5-chamber). The training dataset had over 500,000 cine loops from 29,527 echoes in 26,981 patients. A fully crossed multireader multicase study was conducted using 220 echoes never seen during algorithm development. 5 level II-trained cardiologists (referred to as POCUS readers as they only reviewed B-mode clips) were presented with these echoes twice at least a month apart, once with and again without AI interpretation of AS severity. Readers identified 2 dichotomous severity levels (either "suggestive of moderate to severe AS" or not), with ground truth established by 3 level III-trained cardiology echo experts. While experts had access to full studies (including Doppler) to adjudicate severity, POCUS readers had no Doppler or measurements of any kind available. Sensitivity and specificity were compared for the 5 POCUS readers when aided by the AI algorithm vs unaided, with similar tests of inter-reader agreement.

Results
There was a statistically significant improvement in sensitivity (+5.5%, 95% CI [1.5%, 9.5%], p=0.007) with stable specificity (-0.3%, 95% CI [-3.6%, 3.0%], p=NS) when the 5 POCUS readers received the assistance of the AI software. Fig 1 shows 2,500 bootstrapping assessments to better show consistency of these findings. In addition, the POCUS inter-reader agreement (overall percent agreement) improved by approximately 7% (p&lt;0.001), becoming comparable to that of expert readers (Fig 2).

Conclusions
In a comprehensive multireader pivotal trial, an AI algorithm improved the performance of POCUS cardiologist readers to identify significant AS from limited B-mode echoes as might be obtained in screening or outreach settings. Sensitivity for detection of moderate-severe AS increased significantly by 5.5% with stable specificity. Inter-reader agreement improved to resemble that of expert cardiologists given access to full study information, including Doppler. AI may have a role in assisting detection of significant AS from limited B-mode echoes.Fig 1Fig 2