  Reliability of machine learning (ML) systems is crucial in safety-critical
applications such as healthcare, and uncertainty estimation is a widely
researched method to highlight the confidence of ML systems in deployment.
Sequential and parallel ensemble techniques have shown improved performance of
ML systems in multi-modal settings by leveraging the feature sets together. We
propose an uncertainty-aware boosting technique for multi-modal ensembling in
order to focus on the data points with higher associated uncertainty estimates,
rather than the ones with higher loss values. We evaluate this method on
healthcare tasks related to Dementia and Parkinson's disease which involve
real-world multi-modal speech and text data, wherein our method shows an
improved performance. Additional analysis suggests that introducing
uncertainty-awareness into the boosted ensembles decreases the overall entropy
of the system, making it more robust to heteroscedasticity in the data, as well
as better calibrating each of the modalities along with high quality prediction
intervals. We open-source our entire codebase at
https://github.com/usarawgi911/Uncertainty-aware-boosting
