## 2 Materials and Methods
### Direct, Inverse Neural Network-Based Material Parameter Identification Process
#### 2.2.3 Bayesian Neural Networks  
In most MPI methods, including the direct inverse methods using standard NNs, the output consists of the deterministic material parameters, providing no information about their accuracy and the model's confidence in the prediction. For example, predictions outside the parameter range applied for the training or a too-small training dataset size can result in highly deviating MP predictions, which can lead to a significantly reduced informative quality of subsequent numerical simulations. In this case, information about the existing uncertainties can significantly support the engineer in the assessment of the results and contribute to obtaining correct conclusions from them, e.g., in order to initially increase the prediction accuracy of the method. Furthermore, methods based on NNs often tend to overfit, which limits their generalization ability. One opportunity to overcome these limitations and to maintain prediction accuracy is to use Bayesian neural networks [16], which have the same advantages as the standard neural networks but possess the benefit of providing additional information about the persisting uncertainties. In the literature, no uniform BNN definition exists, although they can be understood as stochastic artificial neural networks (SANN) which are trained by Bayesian inference [16]. SANN are built by applying stochastic components such as stochastic activation or stochastic weights to the network, to simulate multiple possible models \(\theta\) with their associated probability distribution \(p(\theta)\)[17]. \(\theta\) are the parameters of the network such as weights and biases, which are determined during the learning process depending on the training data \(D\). Depending on the chosen NN architecture, different BNNs are achievable, such as a Bayesian convolution neural network (BCNN) or Bayesian recurrent neural network (BRNN), whereas in the present work the underlying architecture is an MLP. For BNN model generation, a prior distribution over the possible model parametrization \(p(\theta)\) and a prior confidence in the predictive power of the model \(p(y|x,\theta)\) has to be chosen, with \(x\) being the input and \(y\) their corresponding labels. By applying [PERSON] theorem and forcing independence between the model parameters and the input, the Bayesian posterior value is given by:  
\[p(\theta|D)=\frac{p(D_{y}|D_{x},\theta)p(\theta)}{\int_{\theta}p(D_{y}|D_{x}, \theta^{\prime})p(\theta^{\prime})d\theta^{\prime}}\propto p(D_{y}|D_{x}, \theta)p(\theta), \tag{17}\]  
with \(D_{x}\) being the training data inputs and \(D_{y}\) the training labels [17]. For further details on the underlying mathematical theory, please refer to [14; 16; 17; 65]. Uncertainties in the prediction of MP can arise due to measurement errors and noise, and insufficient data availability [17]. The former is referred to as aleatoric \(p(y|x,\theta)\) and the latter as epistemic \(p(\theta|D)\) uncertainty, whereas BNNs provide the further advantage of being able to distinguish between them. Aleatoric uncertainties lead to unpredictable differences even if a model prediction is repeated for a constant dataset. The uncertainties are inherent in the data, so increasing the size of the training dataset does not reduce these uncertainties. For BNN model generation and implementation in the direct inverse MPI framework, the libraries TensorFlow V.2.1.0, TensorFlow Probability V.0.9.0 (TFP), and Keras V.2.3.1 were used in this work, as for the MLPs and CNNs. To account for aleatoric uncertainties in the model, a TFP IndependentNormal layer was used. Consequently, instead of the deterministic MP, the model returns a distribution object with a learnable mean and standard deviation (STDV). Hence, instead of the MSE or the custom loss function, the negative log-likelihood (NLL) was used, in order to compute how likely it is to get the true data from the estimated distribution.