  Although deep learning models for chest X-ray interpretation are commonly
trained on labels generated by automatic radiology report labelers, the impact
of improvements in report labeling on the performance of chest X-ray
classification models has not been systematically investigated. We first
compare the CheXpert, CheXbert, and VisualCheXbert labelers on the task of
extracting accurate chest X-ray image labels from radiology reports, reporting
that the VisualCheXbert labeler outperforms the CheXpert and CheXbert labelers.
Next, after training image classification models using labels generated from
the different radiology report labelers on one of the largest datasets of chest
X-rays, we show that an image classification model trained on labels from the
VisualCheXbert labeler outperforms image classification models trained on
labels from the CheXpert and CheXbert labelers. Our work suggests that recent
improvements in radiology report labeling can translate to the development of
higher performing chest X-ray classification models.
