Bayesian neural network parameters provide insights into the earthquake rupture physics.
Authors: Sabber Ahamed
Venue:  (2021)
Keywords: Bayesian neural network
earthquake rupture
simulation
Explainable neural network

Abstract: I present a simple but informative approach to gain insight into the Bayesian neural network (BNN) trained parameters. I used 2000 dynamic rupture simulations to train a BNN model to predict if an earthquake can break through a simple 2D fault. In each simulation, fault geometry, stress conditions, and friction parameters vary. The trained BNN parameters show that the network learns the physics of earthquake rupture. Neurons with high positive weights contribute to the earthquake rupture and vice versa.  The results show that the stress condition of the fault plays a critical role in determining its strength. The stress is also the top source of uncertainty, followed by the dynamic friction coefficient. When stress and friction drop of a fault have higher value and are combined with higher weighted neurons, the prediction score increases, thus fault likely to be ruptured. Fault's width and height have the least amount of uncertainty, which may not be correct in a real scenario. The study shows that the potentiality of  BNN that provides data patterns about rupture physics to make an additional information source for scientists studying the earthquake rupture.

Under review as a conference paper at ICLR 2021
BAYESIAN NEURAL NETWORK PARAMETERS PROVIDE INSIGHTS INTO THE EARTHQUAKE RUPTURE PHYSICS
Anonymous authors Paper under double-blind review
ABSTRACT
present simple but informative approach to gain insight into the Bayesian neu- ral network (BNN) trained parameters. used 2000 dynamic rupture simulations to train a BNN model predict if an earthquake can break through simple 2D fault. In each simulation, fault geometry, stress conditions, and friction parame- ters vary. The trained BNN parameters show that the network learns the physics of earthquake rupture. Neurons with high positive weights contribute to the earth- quake rupture and vice versa. The results show that the stress condition of the fault plays critical role in determining its strength. The stress is also the top source of uncertainty, followed by the dynamic friction coefficient. When stress and friction drop of fault have higher value and combined with higher weighted neurons, the prediction score increases, thus fault likely to be ruptured. Fault's width and height have the least amount of uncertainty, which may not be correct in real scenario. The study shows that the potentiality of BNN that provides data pat- terns about rupture physics make an additional information source for scientists studying the earthquake rupture.
INTRODUCTION
Because of the limited observational data and computational cost, geoscientists often rely simple low-resolution simulations to study physical systems such as dynamic earthquake rupture, long-term tectonic process, etc. Such simplified models are indeed powerful tool beside the observational data but sometimes cannot capture the proper physics of the system. As a result, it becomes difficult to accurately identify and understand the underlying causes.
Machine learning (ML) approaches have been successfully used to solve many such geophysical problems with limited data and require computational overhead. For example, Ahamed & Daub (2019) used neural network and random forest algorithms to predict if an earthquake can break through a fault with geometric heterogeneity. The authors used 1600 simulated rupture data points to train the models. They identified several patterns responsible for earthquake rupture.
Machine learning approaches also used in seismic event detection (Rouet-Ledde et al.) 2017), carthquake detection (Perol et al. 20188), identifying faults from unprocessed raw seismic data (Last al. 2016) and to predict broadband earthquake ground motions from 3D physics-base -ased numerical simulations (Paolucci et al. 2018). All the examples show the potential application ML to solve many unsolved geophysical problems.
However, the machine learning model's performance usually depends on the quality and quantity of data. Bad quality or insufficient data increases the uncertainty of the predictions (Hoeting al.) 1999 Blei et al. 2017 Gal et al.) 2017). Therefore, estimating the source of uncertainty is vital to understanding the physics earthquake rupture and seismie risk. On top of that black-box nature of the ML algorithms inhibits mapping the input features with model output prediction. As result, it becomes challenging for scientists to make actionable decisions.
To overcome insufficient earthquake rupture data, I used the Bayesian neural network algorithm to develop a model reusing the simulations data of Ahamed & Daub 2019). present an exciting ap- proach learning the patterns of carthquake ruptures from the trained model parameters. Unlike regular neural networks, BNN works better with a small amount of data and provides prediction uncertainty. The approach gives more information on rupture physics than the traditional geophys-

---

Under review as a conference paper at ICLR 2021
ical methods. I also describe the workflow of (1) developing BNN and (2) estimating prediction uncertainty.
EARTHQUAKE RUPTURE SIMULATIONS
(a) Rupture domain
(b) Slip weakening law
1.0
Distance across fault (km)
T, p.d.
0.5
Nucleation
Starts here
Shear Stress
0.0
Fault barrier
Poo.
-0.5
d= Critical slip distance
-1.0
Slip
Distance along strike (km)
Figure I: (a) A zoomed view of the two-dimensional fault geometry. The domain is 32 km long along the strike of the fault and 24 kilometers wide across the fault. The rupture starts to nucleate 10 km to the left of the barrier and propagates from the hypocenter towards the barrier, (b) Linear slip-weakening friction law for an earthquake fault. The fault begins to slip when the shear stress reaches exceeds the peak strength of Tg. Ts decreases linearly with slip constant dynamic friction Ta over critical slip distance (d,). The shear strength is linearly proportional to the normal stress on and the friction coefficient varies with slip between μts and pd-
I used the simulated earthquake rupture dataset created by Ahamed & Daub 2019). The simulations are a wwo-dimensional rupture, illustrated in figure. | The domain is 32 km long and 24 km wide. Figure shows the zoomed view of the original domain for better visualization of the fault barrier. Rupture is nucleated 10 km to the left of the barrier and propagates towards the barrier. In each simulation, eight parameters were varied: and y components of normal stress (sxx and syy), shear stress (sxy), dynamic friction coefficient, friction drop (u, pa). critical slip distance (d,), and width and height of the fault. The fault starts to break when the shear stress (T) on the fault exceeds the peak strength T, My where (4 and on the static friction coefficient and normal stress, respectively. Over a critical slip distance de. the friction coefficient reduces linearly to constant dynamic friction pa-
1600 simulation data points were used to train, and 400 were used to test the model performance. The training dataset has imbalance class proportion of rupture arrest (65%) and rupture propaga- tion (35%). To avoid a bias toward rupture arrest, upsampled the rupture propagation examples. Before training, all the data were normalized by subtracting the mean and dividing by the standard deviation.
3 BAYESIAN NEURAL NETWORK
In a traditional neural network, weights are assigned as a single value or point estimate. In a BNN, weights are considered probability distribution. These probability distributions used to estimate the uncertainty in weights and predictions. Figure Siuress a schematic diagram of a BNN where weights are normally distributed. The posterior network parameters are calculated using the following equation:
P(WIX) P(XIW)P(W)
P(X)
Where X is the data, P(X is the likelihood of observing X, given weights (W). P(W) is the prior belief of the weights, and the denominator P(X) is the probability of data which is also known evidence. The equation requires integrating over all possible values of the weights as:
2

---

Under review as a conference paper at ICLR 2021
w -N(4 ①)
w ~NH4, O)
Imput
y
Figure 2: The schematic diagram shows the architecture the Bayesian neural network used in this work. The network has one input layer with eight parameters, one hidden layer with twelve nodes, and an output layer with a single node. Weights between input and hidden layers are defined by w which are normally distributed. i.j are the node input and hidden layer node index. Similarly, w the normal distribution of weights between hidden and the output layer. μ and a are the mean and standard deviation. At the output node, the network produces a distribution of prediction scores between and I.
P(X) f P(X)W)
(2)
Integrating over the indefinite weights in evidence makes it hard to find closed-form analytical solution. a result, simulation or numerical based alternative approaches such as Monte Carlo Markov chain (MCMC) and variational inference(VI] ence method in modern Bayesian statistics, perhaps widely studied and applied in many situations. However, the technique is slow for large datasets and complex models. Variational inference (VI), on the other hand, is faster. It has been applied to solve many large-scale computationally expensive neuroscience and computer vision problems (Blei al. 2017).
considered. MCMC sampling an infer-
In VI, ar new distribution Q(W10) is considered that approximates the true posterior P(W X). Q(W10) is parameterized by 0 over W and VI finds the right of 0 that minimizes the divergence of two distributions through optimization:
Q" (W) argmin KL (Q(W))])P((XX))
In equation-3] KL or Kullback-Leii Leibler divergence non-symmetric and information theoretic measure of similarity (relative entropy) between true and approximated distributions (Kullback 1997). The KL-ddivereence between Q(W and P(WIX) is defined as:
KL (L(WWO)] X)) - Q(Wl0))og Q(W10) dw P(W[X)
(4)
Replacing P(W[X) using equation
we get:

---

Under review as a conference paper at ICLR 2021
KL ([W = Q(W10) ((()) dW P(XIW)P(W)
Q(W10) Q(W log P(X) dw
Q(W16) Q(W(0) dW + Qgwl P(X) P(W)
/ Q(W () log P(X
(5)
Taking the expectation with respect to Q(W10), we get:
Q(W10) KL [Q(W1))))((XX)) E log P(W) log P(X) E log P(X]W))
The above equation shows the dependency of log P(X) that makes it difficult to compute. An alternative objective function therefore, derived by adding log P(X) with negative KL divergence. log P(X) is a constant with respect to Q(W)0). The new function is called as the evidence of lower bound (ELBO) and expressed as:
Q(W10) ELBO(Q) [log P(X[W)] log P(W)
E [log P(X[W)] KL [Q(W[))])P(M X)]
The first term called likelihood, and the second term the negative KL divergence between a vari- ational distribution and prior weight distribution. Therefore, ELBO balances between the likelihood and the prior. The ELBO objective function can be optimized to minimize the KL divergence using different optimizing algorithms like gradient descent.
4 TRAINING BAYESIAN NEURAL NETWORK
The BNN has the same NN architecture used in Ahamed & Daub 2019) to compare the performance between them. Like NN, BNN has one input layer with eight parameters, one hidden layer with twelve nodes, and output layer (Figure 2 nonlinear activation function ReLu (Hahnloser et al. 2000) was used at the hidden layer. ReLu passes all the values greater than zero and sets the negative output zero. The output layer uses i i activation function, which converts the outputs between zero and one.
Prior weights and biases are normally distributed with zero mean and one standard deviation. Fig- ure 3 shows the log density of prior and posterior weights (w and biases (6)). and are the index of the input and hidden layer nodes. ranges from to 7, and j ranges from 11. & is the index that maps two layers. For example, wis is the weight between the first input node and the fifth hidden node. The output node of the last layer produces distribution of prediction scores between 0 and 1. The prediction distributions are used to compute standard deviation, which is the uncertainty metric.
Adam optimization (extension of stochastic gradient descent) was used to minimize the KL diver- gence by finding suitable variational parameter 0. The initial learning rate is 0.5, which expo- nentially decays as the training progresses. To train the BNN, I use Edward (Tran et al. 2016 (Pedregosa et al. (011). Edward Python-bast based Bayesian deep learning library for probabilistic modeling. inference, and criti- cism. All the training data, codes, and the corresponding visualizations can be found on the Github https:: https://githat htt.h//....oommaaaee/e/eeeetatee
2017). ) TensoFl low (Abadi et al. 2013) and Scikit
repository:
baa__baassss ian_nn

---

Under review as a conference paper at ICLR 2021
(a] w
[b] we
0.6
Prue
V
Postector
Log densiis
0.4
Log Gensity
0.4
0.3
0.1
0.3
2
-10
-6 Weight
Weighn
(c)
(d)
0.8
Los: drasds
1xxt drusdyy
0.4
0.02
0.3
-44
-2
0..1
0002
0.00
0.02
0.01
Winght
Weight
Figure 3: The graph shows the distribution of prior and posterior weights (a) wD (b) w and biases (c) (d) 6.. Both location of the mean and magnitude of density of the posterior distributions (weights and biases) noticeably different from the priors which indicates that BNN has learned from the data and adjusted the posterior accordingly.
5 PRIOR AND POSTERIOR PARAMETER DISTRIBUTION
To evaluate the parameters (weights and biases) of the BNN, 1000 posterior samples of wy, w/, 8 and b' were used. Figure shows the prior and posterior distribution of mean weight and biases. The posterior location of the mean and density of the weights and biases are different from their priors. For example, the location of w shifts toward non-negative value, while the density remains similar. Whereas, the w-,, 10, and b1t have different posterior mean location and density than their prior. The differences between prior and posterior indicate that the BNN has learned from the data and adjusted the posterior distribution accordingly.
The performance of the BNN was evaluated using 400 test simulations. For given test example, 1000 posterior samples were used determine the class and associated uncertainty. Uncertainty is the standard deviation of the prediction scores. The test accuracy of the BNN is 83.34%, which is 2.34% higher than NN. In the following subsection, discuss how uncertainty can help us understand physics and find the parameter combinations responsible for an earthquake rupture.
5.1 LEARNING FROM NETWORK PARAMTERS
Estimating the uncertainty of neural network parameters (weights) helps us understand the black box behavior. The illustration Figure 4 shows the mean and standard deviation of W and W1 Wo maps the inputs to the hidden layer nodes, whereas W' maps the output node of the output layer the hidden layer's nodes. The colors in each cell indicate the magnitude of weight that connects two nodes. In the input and hidden layer, the Rel activation function was used. Relu passes all the positive output while setting a negative output to zero. Sigmoid activation used at the output layer, which pushes the larger weights toward one and smaller negative weights toward zero. Therefore, positive and high magnitude weights contribute to the earthquake rupture and vice versa.
In w nodes connected to friction drop, dynamic friction, shear, and normal stresses have variable positive and negative weights. The corresponding node in w! also has a strong positive or negative magnitude. For example, node- 12 has both positive and negative weights in wº. and the correspond- ing node in w has a high positive weight. Similarly, node-4 has a substantial negative weight in w and the corresponding nodes in ve have both positive and negative weights. On the other hand,
5

---

Under review as a conference paper at ICLR 2021
(w)
[e]
12-
12-
12-
2.5
11
23
1.4
1.2
1.0
1.5
-1.5
1,0
-3.0
0.5
Hrighh Weltts
Hrigha
Wibh
Droamic Plic
Fau Degp
Uugat Uni
Dunuis
Ougut
Figure 4: The illustration shows the posterior mean and standard deviations of wo and w (a) wo that map the inputs to the nodes of the hidden layer. The eight input parameters are on the horizontal axis, and the twelve nodes on the vertical axis. The colors in each cell are the magnitudes of mean weight. (b) w² maps the hidden layer to the output layer. (c) The standard deviation of w. Shear stress connected to node-4 of the hidden layer has the highest uncertainty. Similarly, the weights associated with the input parameters and node-5 have high uncertainty. Whereas, the weights associated with the input parameters and the nodes 7-11 have relatively low uncertainty. (d) Uncertainty of the weights associated with the hidden layer nodes and the output node. Weights in node and 8 have high uncertainty while the rest of the weights have relatively low uncertainty.
width, height, and de have a similar magnitude of weights, and the corresponding nodes w have moderate magnitude of weights. Thus the variable weights make friction drop, dynamic friction, shear, and normal stresses influential on the prediction score. Therefore, we can now detect the important features and their uncertainties from any input combinations.
For example, node- of w has positive weight (Figure 4p). In w", the corresponding connecting input features, friction drop, and shear stress have positive weight and low uncertainty, whereas the rest have a similar magnitude of weight. The combination of high friction drop and shear stress weight and low weights of other features increase the prediction score, thus likely to cause rupture to propagate. Friction drop and normal stresses also have high uncertainty (Figure (bb). For this combination of patterns, friction drop and normal stresses influence the prediction strongly and are also the sources of uncertainty. Thus, it gives the ability to investigate any rupture propagation example terms of uncertainty.
5.2 PREDICTION UNCERTAINITY
Proparary
Stovided dedattin
60-
0.4
100
0.5
L.0
Predictees Emosabiizie
Pradiction Probasiittie
Figure 5: The graph shows (a) frequency and (b) standard deviation of posterior prediction scores of the test data. Prediction scores are skewed toward the left side while slightly less on the right. The observation is consistent with the proportion of the rupture arrest (272) and rupture propagation (120) in the test data. Prediction scores close to zero are related to rupture arrest, and scores around one are the rupture propagation. Standard deviations are high with scores around 0.5.
Figure 5 and b show the frequency and standard deviation of test data prediction scores. The distribution is more skewed toward smaller scores than the higher ones. Scores close associated with rupture arrest, whereas scores close to one are rupture propagations. The observa- tion consistent with the class proportion of the test data. Rupture arrest has higher number of

---

Under review as a conference paper at ICLR 2021
examples (272) than rupture propagation (120). Figure 5h shows that scores roughly between 0.35 and 0.75 have a fewer number of examples and thus high uncertainty. The likely reason for the high uncertainty is that the examples have both rupture propagation and arrest properties. As result, the model gets confused and cannot classify the example correctly; thus, the misclassification rate is high in this region. The network's performance could be improved if sufficient similar example data are added to the training dataset.
5.3 ROLE OF INPUT FEATURES
Pemmitaiin Geter importare
(b) Unoettinily of festuee
Ruptore
Desjatia of sore
0.20
0.02
Uncortanity
0:15
0.10
-0:00
0.00
0.00
100
mod
baigl adrop sxy
wbith
437
3000
50X
way
hright
whith
Ratuus
Featurss
Figure 6: The illustration shows (a) permutation feature importance and (b) their uncertanitities.
I used the permutation importance method to determine the source of uncertainty in the test data. Permutation importance model agnostic method that measures feature's influencing capacity by shuffling it and measuring corresponding global performance deviation. If feature is good predictor, altering its values reduces the model's global performance significantly. The shuffled feature with the highest performance deviation is the most important and vice versa. In this work, the F-1 score is the performance measuring metric.
Figure ([a) shows the permutation importance of all the features. Normal stresses (sxx and syy) have the highest F-1 score deviation, which is approximate 7% less than the base performance, followed by the dynamic friction coefficient. Geometric feature width has the least contribution role to determine the earthquake rupture. These observations are consistent with the observation of Ahamed & Daub (2019), where the authors rank the features based on the random forest feature importance algorithm.
From the distribution of prediction scores, the standard deviation was calculated for each shuffled feature. Figure (b)) shows the uncertainty of each feature of earthquake rupture propagation and arrest. All features have higher uncertainty in the rupture propagation compare to the rupture ar- rest. In both classes, the major portion of uncertainty comes from normal stresses. Shear stress, height, width, and critical distance (da) have similar amount of uncertainty in both classes. The dynamic friction coefficient and friction drop are also the comparable uncertainty sources in rupture to propagate while slightly less earthquake arrest.
The above observations imply that normal stress and friction parameters have a more considerable influence in determining the earthquake rupture. Although the height and width of a fault are not a significant source of uncertainty, they play more influential role in influencing other features. For example, in a complex rough fault, the variation of the bending angle of barriers affects stress per- turbation, consequently increasing the uncertainty. If the angle near the bend is sharp, the traction variation at the releasing and restraining bend is more prominent. On the other hand, if the barrier broad, the stress perturbation at the restraining and releasing bend is less noticeable. Chester & Chester (2000) found a similar observation that fault geometry impacts the orientation and magni- tude of principal stress.
DISCUSSION AND CONCLUSION
In recent years, deep learning has been used to solve many real-life problems and achieve a state of the art performance. Some examples are facial recognition, language translation, self-driving cars,
7

---

Under review as a conference paper at ICLR 2021
disease identification, and more. Such successful applications require millions of data points train the model and achieve a state of the art performance. However, in many situations, data is limited data like earthquake rupture.
In this work, I used a Bayesian neural network to overcome the small data problem and estimate prediction uncertainty. Rupture simulation created by Ahamed & Daub (2019) used to develop a model to predict if an earthquake can break a fault. Each 2D simulation fault has a Gaussian geometric heterogeneity
the center. The eight fault parameters of normal shear stress, height, and width of the fault, stress drop, dynamic friction coefficient, critical distance were varied in each of the simulations. Sixteen hundred simulations were used to train the BNN, and 400 were used to test the model's performance.
The test Fl score is 0.8334, which is 2.34% higher than the NN, All the features have higher uncertainty in rupture propagation than the rupture arrest. The highest sources of uncertainty came from normal stresses, followed by a dynamic friction coefficient. Therefore, these features have higher influencing capacity in determining the prediction score. Shear stress has a moderate role, but the geometric features such as the fault's width and height are least significant rupture. Test examples with prediction scores around 0.5 have higher uncertainty than those with low (0-0.30) and high (0.7-1.0) prediction scores. Cases with prediction scores around 0.5 have mixed properties of rupture propagation and arrest.
determining
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jía, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing heterogeneous systems, 2015. URL hitps: www.tennorflo... org/ Software available from tensorflow.org.
Sabber Ahamed and Eric G Daub. Machine learning approach to earthquake rupture dynamics. 2019.
arXiv preprint
David Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: review for statisti- cians. Journal of the American Statistical Association, 112(518):859-888 2017.
Frederick M Chester and Judith S Chester. Stress and deformation along wavy frictional faults.
Journal of Geophysical Research: Solid Earth, 105(B (1)::11))334122350 -3440,, 2000.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning-Vol Volume 70, pp. 1183- 1192. JMLR. org, 2017.
Richard HR Hahnloser, Rahul Sarpeshkar, Misha Mahowald, Rodney J Douglas, and H Sebastian Seung. Digital selection and analogue amplification coexist in cortex-inspired silicon circuit. Nature, 405(6789):947, 2000.
Jennifer A Hoeting, David Madigan, Adrian E Raftery, and Chris T Volinsky. Bayesian model averaging: a tutorial. Statistical science, pp. 382-401, 1999.
Solomon Kullback. Information theory and statistics. Courier Corporation, 1997.
Mark Last, Nitzan Rabinowitz, and Gideon Leonard. Predicting the maximum earthquake magni- tude from seismic data in Israel and its neighboring countries. PloS one, 11(1):c0166101, 2016.
Roberto Paolucci, Filippo Gatti, Maria Infantino, Chiara Smerzini, Ali Güney Özcebe, and Marco Stupazzini. Broadband ground motions from 3D physics based numerical simulations using tificial neural networks. Bulletin of the Seismological Society of America, 108:34):122--1286 2018.
8

---

Under review as a conference paper at ICLR 2021
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Seikit-learn: Machine learning in python. Journal of machine learning research, 12(0ct):2825-258) 2830, 2011.
Thibaut Perol, Michaël Gharbi, and Marine Denolle. Convolutional neural network for earthquake detection and location. Science Advances, 4(2):e1700578, 2018.
Bertrand Rouet-Ledue, Claudia Hulbert, Nicholas Lubbers, Kipton Barros, Colin J. Humphreys, and Paul Johnson. Machine learning predicts laboratory earthquakes. Geophysical Research Letters, 2017. ISSN 1944-8007. doi: 00.10020011GGL776777 10. 1002/0022000 211077777 2017GL074677.
/RLhhttp://dxx http://xxxdoo..om org/
Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei. Edward: A library for probabilistic modeling, inference, and criticism. arXiv preprint arXiv:1610.09787 2016.
Dustin Tran, Matthew D. Hoffmmnn, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, and David M. Blei. Deep probabilistic programming. International Conference on Learning Representations, 2017.


Reviews:
[{"review_id": "yRllK5g6NV1", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The manuscript do not provide enough details about  the physics about which the BNN provides insight. \nThe manuscript employs the well developed machine learning algorithms in an application in Geoscience and do not provide a novel learning algorithm or contribute to machine learning topics. This paper does not meet the  ICLR standards and therefore I can not recommend for publication.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "brGJyqtZdLE", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a Bayesian neural network for predicting if an earthquake will break a fault or not, overcoming 'small data problem' and predicting model uncertainty. The data is composed of 8 features and a binary output, and the samples are all coming from simulations. An analysis on the means and standard deviations of the first and last layer of the neural network's weights has been carried out.\n\nThe problem is interesting, and the method is useful as the study on the weights means and stds is interesting. Yet, I think the paper is not well polished (many incoherence in the text and the figures) and I don't really understand why a synthetic dataset, coming from physical model equations, needs to go into this complex uncertainty quantification: we are making here a 'meta-model' of something that is fully described by equation, so why using machine learning? I also don't understand why the author talks about a 'small data problem', as here we could simply increase the number of simulated samples? Yet, I can see the interest of such a technique if the goal was to later try to apply it to real data, where some physics might be unknown or too complex. I also don't know if the findings of the paper are of interest for the ICLR community, but as I come from an interdisciplinary field too, I know how hard it can be to find an appropriate and yet good place to publish.\n \nOther questions/Remarks:\n - Equ. 8 : I don't see how we can go from Eq. 6 to Eq. 8? this part is not clear.\n - Please cite the reference papers for the Bayesian NN as well as for ELBO. There is clearly a part missing in the state-of-the-art regarding this, while the equations (such as the long and useless (5) one) are just explaining what is already known in this literature.\n - Figure 4: 'Shear stress connected to node-4 of the hidden layer has the highest uncertainty. Similarly,\nthe weights associated with the input parameters and node-5 have high uncertainty. Whereas, the\nweights associated with the input parameters and the nodes 7-11 have relatively low uncertainty. (d)\nUncertainty of the weights associated with the hidden layer nodes and the output node. Weights in\nnode 7 and 8 have high uncertainty while the rest of the weights have relatively low uncertainty.'\n\t--> please update the text or the figures so the numbers match. Same thing for the text. For now, I cannot understand anything.\n- Figure 1: Tau_s decreases linearly: not, tau_s is fixed, maybe you meant tau or 'the shear stress'\n\ntypos:\n- 'perhaps widely studied and applied in many situations' --> 'perhaps'? never seen this word in a paper...\n- ar new distribution\n- is therefore, derived by\n- which is approximate 7% less\n- data is limited data\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "acgIJLTnQSx", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\nIn the present paper, the author intends to get further insights into the physics behind earthquake ruptures using a BNN to model simulated data from the literature. By using a BNN, the parameters of the model are not deterministic scalar values, but complete probability distributions. Studying the change of the distributions in the parameters before and after training, the author tries to extract information about the relative importance of the input variables, and also comprehend the physical mechanisms behind earthquake ruptures. Results are shown in figure 3, on which the change of behavior of the distributions of the parameters can be observed, as well as in figure 4 where the mean and standard deviations for all the parameters are presented. The pattern in figure 6 seems to indicate that variables previously thought to be important in the task of predicting the presence of the rupture, such as normal stress and friction, are also pointed out as being important in this case.  Finally, the authors also claim an improvement in the F1 metric in comparison to previous NN methods. \n\n##### Pros\n* The idea of the paper seems well directed, i.e., gaining insight on complex physical procedures using an approach that results in the combination of NNs and a Bayesian approach. \n* Using a Bayesian approach is a good way of dealing with small datasets, and also allows to account for the uncertainty of all the latent parameters, while also providing more robust and sensible predictions when new data is presented\n* The approach seems to provide results consistent with the literature findings regarding the important variables in the prediction\n* The final performance of the algorithm seems to improve on the previous state-of-the-art methods by taking advantage of the properties that the Bayesian approach offers\n\n##### Cons\n* The key concern with this paper is that NNs, as well as BNNs, are notoriously black-box algorithms with no easy way of interpreting the inner parameters in most cases. Taking this into consideration, I would suggest the author to motivate in a stronger manner why the usage of BNNs is desirable for the proposed problem, and why not use other already established Bayesian approaches to assess the importance of the input variables. \n* Taking into account the previous point, I consider there is a general lack of rigorous experiments that could, in principle, suggest a clear advantage of using BNNs instead of any other approaches. No systematic comparisons with previous methods are present, such as for example with the Random Forest Feature Importance algorithm, which is mentioned a couple of times. If the main goal is to gain insights on the main variables involved in the presence of an earthquake rupture, I would expect a more detailed analysis comparing how good these insights provided by BNNs are and how do they stand in comparison with the established literature. \n* Other basic techniques for assessing the importance of the variables in the prediction tasks are not mentioned, although it would be nice to use them as a baseline to compare against. Examples such as PCA, LOO cross-validation and others could be used here. \n* The claim of an improvement of 2.34% w.r.t. NNs is not strongly addressed, since the NN experiments are not included here or, at least, there is no mention of the setup of these NNs. As before, there is no systematic comparison between the BNNs trained and the NNs that are used as baselines. \n* There is a lengthy discussion on how to obtain the ELBO for VI. However, in the end, there is no final expression for the loss function which is going to be employed. I would appreciate in section 4 an explicit description of the objective of the system since there's no mention of the final binary classification problem anywhere.\n* The prediction uncertainties lack a systematic evaluation as well since all that is provided is presented in figure 5. How well do the predictions provided stand against other methods for obtaining final predictive distributions? \n* VI is a method whose performance and final predictions are constrained due to its formulation. Is there any reason why using VI instead of any other approach to BNNs? In case that we wanted to study the final predictive distributions, why not use HMC or other, more flexible approaches than VI? \n* At the end of the section 5.1, first paragraph, it is claimed that \"positive and high magnitude weights contribute to the earthquake rupture and vice versa\". This sentence seems a bit confusing since it seems to imply a causal relation between the high magnitude of the weights and the appearing of ruptures. This, I think, is the other way around: very clear rupture conditions imply positive high magnitude weights, which, in turn, return a higher predicted probability of rupture. \n\n##### Minor comments:\n* Even though the paper tackles physical phenomena such as earthquake rupture, it does not provide any description of such process or the variables involved. Concepts such as \"nucleation\" and \"fault barrier\" should be at least briefly introduced, as well as the \"slip weakening law\" or the \"critical slip distance\". A short description of these terms and their relevance to the problem would help to interpret the final results obtained. Also, explicit expressions for the rupture physics would help a lot in section 2 to understand the different roles of the variables and their relations. \n* Throughout the whole text it is used the first person while writing. In case there is only one author this can be okay, I only point it out since it seems to be a uncommon choice.\n* There are a lot of typos all through the paper! Please perform a careful reading and correct them.\n* The description on figure 4 is confusing, does not seem to correspond to the presented images (either that or the text is unclear when selecting the important parts of the figures for the nodes mentioned). \n* 4th paragraph of introduction - not all ML algorithms are black boxes! NN are, but other such as linear regression, decision trees, etc., can be very interpretable!\n* 5th paragraph of introduction - \"exciting\" - avoid usage of these type of subjective adjectives all through the paper.\n* 5th paragraph of introduction - BNNs may work better with fewer data, but we have to pay close attention to the prior formulation to not introduce unreasonable biases. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "_GCxNlK9-b", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThis paper extends on previous works by Ahamed & Daub (2019), from a two-layers MLP, two a Bayesian NN version of it, for predictiong wether a piece of material will rupture, under some conditions.\nAlthough this increase in model complexity improves performance a little bit, it does not represent a major advance.\nThe main point of the paper is to show that Bayesian NN allow to get not only a prediciton, but also naturally provide uncertainties on these predictions.\nAlthough the paper clearly explains the basics of BNNs, it does not provide any new insight into them. The application to rupture physics is interesting, but does not seem groundbreaking.\n\nFor these reasons, I lean on **rejecting the paper.**\n\nAlso, given the github repo referred to, there is a breach of anonymity.\n\nOn the paper itself, I have a couple of remarks:\n- it is unclear as to where the data comes from. A simulation is mentionned, but not how it works. I see in Ahamed & Daub (2019) that it is a finite element simulation.\n- still about the simulation, it is uncelar whether the stress state is heterogeneous or not. It seems it ought to be, however in that case, the description of the stress state would consist in a full field of values, and not just a couple of numerical values.\n- the discussions of the uncertainty in various input variable is tedious and does not really highligh how BNNs help to get transparent interpretations.\n- Fig 5b: there is a clear sqrt(x)sqrt(1-x) shape of this curve. Do you have an epxlanation for that (and can you check the fit?)\n- the conclusion repeats some parts that were stated rearlier. It should instead focus on how BNNs help understand the physics. The physics of rupture itself is not very interesting to a ML audience.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers an interesting application of Bayesian neural nets to the geophysics domain;  however, the paper does not make a novel contribution from the machine learning perspective, and the improvements on top of the previously proposed approach by Ahamed & Daub (2019) seem to be quite modest. Overall, the paper does not seem to be ready for publication at ICLR.
 
Decision: Reject