  Beyond assigning the correct class, an activity recognition model should also
be able to determine, how certain it is in its predictions. We present the
first study of how welthe confidence values of modern action recognition
architectures indeed reflect the probability of the correct outcome and propose
a learning-based approach for improving it. First, we extend two popular action
recognition datasets with a reliability benchmark in form of the expected
calibration error and reliability diagrams. Since our evaluation highlights
that confidence values of standard action recognition architectures do not
represent the uncertainty well, we introduce a new approach which learns to
transform the model output into realistic confidence estimates through an
additional calibration network. The main idea of our Calibrated Action
Recognition with Input Guidance (CARING) model is to learn an optimal scaling
parameter depending on the video representation. We compare our model with the
native action recognition networks and the temperature scaling approach - a
wide spread calibration method utilized in image classification. While
temperature scaling alone drastically improves the reliability of the
confidence values, our CARING method consistently leads to the best uncertainty
estimates in all benchmark settings.
