  Recently, sound-based COVID-19 detection studies have shown great promise to
achieve scalable and prompt digital pre-screening. However, there are still two
unsolved issues hindering the practice. First, collected datasets for model
training are often imbalanced, with a considerably smaller proportion of users
tested positive, making it harder to learn representative and robust features.
Second, deep learning models are generally overconfident in their predictions.
Clinically, false predictions aggravate healthcare costs. Estimation of the
uncertainty of screening would aid this. To handle these issues, we propose an
ensemble framework where multiple deep learning models for sound-based COVID-19
detection are developed from different but balanced subsets from original data.
As such, data are utilized more effectively compared to traditional up-sampling
and down-sampling approaches: an AUC of 0.74 with a sensitivity of 0.68 and a
specificity of 0.69 is achieved. Simultaneously, we estimate uncertainty from
the disagreement across multiple models. It is shown that false predictions
often yield higher uncertainty, enabling us to suggest the users with certainty
higher than a threshold to repeat the audio test on their phones or to take
clinical tests if digital diagnosis still fails. This study paves the way for a
more robust sound-based COVID-19 automated screening system.
