  With recent progress in the field of Explainable Artificial Intelligence
(XAI) and increasing use in practice, the need for an evaluation of different
XAI methods and their explanation quality in practical usage scenarios arises.
For this purpose, we present VitrAI, which is a web-based service with the goal
of uniformly demonstrating four different XAI algorithms in the context of
three real life scenarios and evaluating their performance and
comprehensibility for humans. This work reveals practical obstacles when
adopting XAI methods and gives qualitative estimates on how well different
approaches perform in said scenarios.
