Extensive validation and prospective observation of the impact of an AI-based rapid antibiotics susceptibility prediction platform in multiple medical centers
Authors: Wang Hsin-Yao , Chung Chia-Ru , Tseng Yi-Ju , Yu Jia-Ruei , Chen Chao-Jung , Wu Min-Hsien , Lin Ting-Wei , Huang Wan-Ting , Liu Tsui-Ping , Lee Tzong-Yi , Horng Jorng-Tzong , Lu Jang-Jih 
Published: July 2021

Importance

No medical artificial intelligence (AI) has been robustly validated and deployed in a clinical laboratory in real-world settings, and the clinical impact of the medical AI remains unknown.

Objective

To deploy a medical AI platform for rapid antibiotics susceptibility test (AST) prediction, and evaluate its clinical impacts.

Design

A medical AI platform, XBugHunter, was extensively validated (internal validation, time-wise validation, and independent testing) with data between May 22, 2013 and June 30, 2019. The clinical impact was evaluated based on a prospective observation from February 1 to September 30, 2020 during deployment.

Setting

Data was collected in two tertiary medical centers in Taiwan, and the AI was deployed in a tertiary medical center.

Participants

For validation, 90,064 consecutive cases were included. During the deployment, a prospective observational cohort of 1,490 consecutive cases was collected.

Exposures

AST prediction from XBugHunter

Main outcomes and Measures

Diagnostic performance of XBugHunter was evaluated during validation. The clinical impact was evaluated in terms of the saving of inappropriate antibiotics prescription, AST turn-around-time, and mortality of bacteremia during deployment.

Results

Predictive models consistently performed well in the extensive validations. In the deployment, XBugHunter’s predictive sensitivity and specificity for Staphylococcus aureus (oxacillin) were 0.95 (95% CI, 0.82–0.98) and 0.97 (95% CI, 0.94–0.99), respectively. For Acinetobacter baumannii (multiple drugs), the sensitivity was 0.95 (95% CI, 0.91–0.99) and specificity was 0.93 (95% CI, 0.88–0.98). The turn-around-time reduction on reporting AST of blood cultures was 35.72 h (standard deviation: 15.55 h). Death within 28 days occurred in 28 of 162 S. aureus bacteremia patients (17.28%) in the XBugHunter intervention group, which was lower than the 28 days’ mortality rate (28.06% [55/196]) in the same period of time in 2019, without XBugHunter. The relative risk reduction was 38.4% (relative risk, 0.62; 95% CI, 0.41–0.92). Regarding antibiotic prescriptions, 2723.7 defined daily dose per year of inappropriate antibiotics could be avoided for treating S. aureus by deploying XBugHunter.

Conclusions and Relevance

Among S. aureus bacteremia patients, this study demonstrated that XBugHunter can prevent inappropriate antibiotic use, and the adjustment of antibiotic treatment can yield lower mortality.

Key Points

Question

What is the clinical impact of XBugHunter, a machine learning-based, antibiotic susceptibility test prediction platform?

Findings

In the prospective observational cohort of deploying XBugHunter, the reduction of turn-around-time of reporting antibiotic susceptibility test was 35.72h. The reduction of S. aureus bacteremia mortality rate was 10.78%, and the estimated saving of inappropriate antibiotics uses was 2723.7 defined daily dose per year.

Meaning

Deployment of XBugHunter provides a more rapid report of antibiotic susceptibility test, and thus reduces inappropriate antibiotics prescription and mortality of S. aureus bacteremia.