  We present xRAI an approach for extracting symbolic representations of the
mathematical function a neural network was supposed to learn from the trained
network. The approach is based on the idea of training a so-called
interpretation network that receives the weights and biases of the trained
network as input and outputs the numerical representation of the function the
network was supposed to learn that can be directly translated into a symbolic
representation. We show that interpretation nets for different classes of
functions can be trained on synthetic data offline using Boolean functions and
low-order polynomials as examples. We show that the training is rather
efficient and the quality of the results are promising. Our work aims to
provide a contribution to the problem of better understanding neural decision
making by making the target function explicit
