Transparency of algorithmic systems entails exposing system properties to various stakeholders for purposes that include understanding, improving, and /or contesting predictions. The machine learning (ML) community has mostly considered explainabilityas a proxy for transparency. With this work, we seek to encourage researchers to study uncertainty as a form of transparency and practitioners to communicate uncertainty estimates to stakeholders . First, we discuss methods for assessing uncertainty. Then, we describe the utility of uncertainty for mitigating model unfairness, augmenting decision-making, and building trustworthy systems. We also review methods for displaying uncertainty to stakeholders and discuss how to collect information required for incorporating uncertainty into existing ML pipelines. Our contribution is an interdisciplinary review to inform how to measure, communicate, and use uncertainty as a form of transparency.
Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions . First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, decision-making and fairness. We aim to encourage researchers and practitioners to measure, communicate, and use uncertainty as a form of transparency.
R
Transparency of algorithmic systems
Algorithmic transparency
D
/or
R
The machine learning (ML) community has mostly considered explainabilityas a proxy for transparency. With this work, we seek to encourage researchers to study uncertainty as a
Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary
R
and practitioners to communicate uncertainty estimates to stakeholders
by estimating and communicating the uncertainty associated with model predictions
R
describe the utility of uncertainty for mitigating
characterize how uncertainty can be used to mitigate
R
augmenting
augment
R
building
build
R
We also review
Finally, we outline
R
discuss
recommend
R
Our contribution is
This work constitutes
R
to inform how
drawn from literature spanning machine learning, visualization/HCI, decision-making and fairness. We aim to encourage researchers and practitioners