Radial Spike and Slab Bayesian Neural Networks for Sparse Data in Ransomware Attacks
Authors: Jurijs Nazarovs
Jack W Stokes
Melissa Turcotte
Justin Carroll
Itai Grady
Venue: Submitted to ICLR 2023 (2023)

Abstract: Ransomware attacks are increasing at an alarming rate, leading to large financial losses, unrecov- erable encrypted data, data leakage, and privacy concerns. The prompt detection of ransomware attacks is required to minimize further damage, particularly during the encryption stage. However, the frequency and structure of the observed ransomware attack data makes this task difficult to accomplish in practice. The data corresponding to ransomware attacks represents temporal, high- dimensional sparse signals, with limited records and very imbalanced classes. While traditional deep learning models have been able to achieve state-of-the-art results in a wide variety of domains, Bayesian Neural Networks, which are a class of probabilistic models, are better suited to the issues of the ransomware data. These models combine ideas from Bayesian statistics with the rich expres- sive power of neural networks. In this paper, we propose the Radial Spike and Slab Bayesian Neural Network, which is a new type of Bayesian Neural network that includes a new form of the approx- imate posterior distribution. The model scales well to large architectures and recovers the sparse structure of target functions. We provide a theoretical justification for using this type of distribution, as well as a computationally efficient method to perform variational inference. We demonstrate the performance of our model on a real dataset of ransomware attacks and show improvement over a large number of baselines, including state-of-the-art models such as Neural ODEs (ordinary dif- ferential equations). In addition, we propose to represent low-level events as MITRE ATT&CK tactics, techniques, and procedures (TTPs) which allows the model to better generalize to unseen ransomware attacks.

Under review as a conference paper at ICLR 2023
RADIAL SPIKE AND SLAB BAYESIAN NEURAL NET- WORKS FOR SPARSE DATA IN RANSOMWARE ATTACKS
Anonymous authors Paper under double-blind review
Ransomware attacks are increasing erable encrypted data, data leakage, and privacy concerns. The prompt detection of ransomware attacks is required to minimize further damage, particularly during the eneryption stage. However, the frequency and structure of the observed ransomware attack data makes this task difficult to accomplish in practice. The data corresponding ransomware attacks represents temporal, high- dimensional sparse signals, with limited records and very imbalanced classes. While traditional deep learning models have been able achieve state-of-the of-the-art results in a wide variety of domains, Bayesian Neural Networks, which are a class of probabilistic models, are better suited to the issues of the ransomware data. These models combine ideas from Bayesian statistics with the rich expres- sive power of neural networks. In this paper, we propose the Radial Spike and Slab Bayesian Neural Network, which is a new type of Bayesian Neural network that includes a new form of the approx- imate posterior distribution. The model scales well large architectures and recovers the sparse structure of target functions. We provide a theoretical justification for using this type of distribution, well computationally efficient method to perform variational inference. We demonstrate the performance of our model on a real dataset of ransomware attacks and show improvement over a large number of baselines, including state-of-the-ard models such as Neural ODEs (ordinary dif- ferential equations). In addition, we propose to represent low-level events as MITRE ATT&CK tactics, techniques, and procedures (TTPs) which allows the model to better generalize to unseen ransomware attacks.
alarming rate, leading large financial losses, unrecov-
INTRODUCTION
Ransomware attacks are increasing rapidly and causing significant losses to governments, corpora- tions, non-governmental organizations, and individuals. The losses may include financial costs due to ransoms paid to decrypt assets, unrecoverable files when the ransom is not paid or the attacker fails to provide the decryption key, privacy and intellectual property theft when assets are exported, and even significant injury when ransomware impairs health care devices or patient records in hospitals. clear that the timely detection of ransomware incidents is necessary in order to minimize the number of assets that are enerypted or exfiltrated (Urooj al., 2021). To improve the ransomware response, this work proposes a new Bayesian Neural Network model that offers improved detection rates for organizations which employ analysts protect their assets and networks.
The problem is usually considered as a detection task, where the two classes are ransomware or not. The traditional methods of statistics and machine learning have been proposed to detect security threats in general and specifically ransomware in some cases. From the statistical perspective, a common approach is the application of Bayesian Networks (Perusquía et al., 2020; Oyen et al., 2016: Shin et al., 2015), whose main goal is to model the relationship between the observed signal and the class of the attack as graphical model. From the machine learning perspective, range of models were used to detect ransomware (Alhawi et al., 2018; Poudyal et al., 2018; Zhang et al., 2019; Larsen et al., 2021), such as Naive Bayes, Gradient Boosting, and Random Forests.
Bottleneck. To obtain the rich expressive power of traditional deep learning models, training usu- ally requires having access to a large number of records to successfully obtain robust generalized results. Unfortunately, the frequency and structure of commonly observed data corresponding to ran- somware attacks makes this task more difficult to accomplish. In particular, ransomware attack data can be represented as temporal high- nigh-dimensional sparse signals, with limited number of records and very imbalanced classes. In our data, the percentage of ransomware attacks to non non-ramsomware attacks is I % versus 99%, respectively.

---

Under review as a conference paper at ICLR 2023
Main ideas and contributions. To address these unique features of the ransomware data, first propose to represent ransomware signals according their MITRE ATT&CK tactics, techniques, and procedures (TTPs) which allows us to generalize ransomware and other attacks at a higher-level instead of the low-level detections associated with an individual attack. In addition, this allows for the detection of both human operated and automated ransomware attacks across multiple stages in the kill chain within an organization's 's network. Next, we propose a new probabilistic model which called the Radial Spike and Slab Bayesian Neural Network. It is a Bayesian Neural Network, where the approximate posterior is represented by mixture of distributions, resulting in Radial Spike and Slab distribution. Our model provides the following benefits including: (1) the Spike and Slab component handles missing or sparse data, (2) the Radial component scales well with the growth of the number of parameters in the deep neural network, and (3) the Bayesian component prevents overfitting in the limited data setup. From the theoretical perspective, we provide the justification for using this type of distribution, as well as a computationally efficient method to perform variational inference. In the results section, we demonstrate the performance of our model on a set of actual ransomware attacks and show improvement over a number of baselines, including the ofate-of-the-ar temporal models such RNNs (Cho al., 2014) and Neural ODEs (ordinary differential equations) (Chen et al., 2018 1)). Thus, the proposed model is an important tool for the critical problem of ransomware detection.
INCIDENT DATA DESCRIPTION
This work utilizes threat data provided by 'our industry partner' to detect ransomware and other types of cybersecurity attacks. Low-level event generators manually created by analysts (i.e., signatures) and are provided with a UUID (Universally Unique Identifier).
Features. Given each incident, features need to be extracted which capture the range of attack be- haviors observed across the kill chain and represent common behaviors across the different families of ransomware attacks. The low-level events cannot be used directly because there are too many to train our model, given the number of labeled examples, and they do not generalize well indi- vidually. To overcome these problems, we map a subset of the low-level events into higher-level representation using the MITRE ATT&CK framework (MITRE). We chose the MITRE ATT&CK framework for the mapping because it provides a knowledge base of adversary tactics, techniques, and procedures (TTPs) and is widely used across the industry for classifying attack behaviors and understanding the lifecycle of an attack. Using the MITRE ATT&CK TTPs is a natural choice for features as it is generalizable, interpretable, and easy to acquire for this data as each low-level event from 'the anonymized company' is tagged with the MITRE technique associated with the alerted behavior (MITRE). For example, one of the features can represent whether 'OS Credential Dump- ing' happened or not. Additional MITRE ATT&CK features are included in Table 2, and the entire set is provided by the MITRE corporation (MITRE, 2022a). The verbose definition of these features can be found in (MITRE). For example, feature T1059.001 "Command and Scripting Interpreter, Powershell" corresponds to "Adversaries may abuse PowerShell commands and scripts for execu- tion" (MITRE, 2022b). In total. our data is sparse binary, nigh-diimessioal vector of size 706, which contains 298 MITRE ATT&CK features and 408 additional signature- based features, each time point. One of the primary characteristics of the data is sparsity because only very few actions are completed at each time step during the attack.
Labels. Using manual investigation, an analyst provides a label for each incident indicating whether it is due to a ransomware attack or another type of attack. The ransomware incidents include both human operated ransomware (HumOR) and automated ransomware attacks described in Appendix in the Supplementary Material. However, our positive class label only indicates that an attack is ransomware and does not distinguish between the two classes of ransomware (i.c., HumOR, Auto- mated). Our goal is to build an naamm-ercomeennttioo system, which can not only detect a possible ransomware attack, but also provide an estimate of the uncertainty about the decision. We provide additional details about the training and testing data in Section 4.
Ethics. As part of the production data collection process, all data has been processed remove all personal identifiable information. The datasets we received for this analysis only included a randomly assigned UUID for the organization, and the incidents that included the MITRE events,
2

---

Under review as a conference paper at ICLR 2023
signature UUID identifiers and the labels. The data collected and adheres to the GDPR standard. There are no negative societal impacts for creating models to protect users from ransomware.
METHODOLOGY
Important features of probabilistic models, such providing notion of uncertainty, dealing with missing data, and preventing overfitting in a limited data regime, have generated a strong interest in deep Bayesian learning. In this section, provide more details regarding Bayesian Neural Networks, including different aspects of initializing and training the model. We then propose the Radial Spike and Slab Bayesian Neural Network model to address the problems of the ransomware data.
Bayesian Neural Networks. The main idea behind the Bayesian Neural Network is consider all weights as being samples from a random distribution. Formally, we denote the observed data as an imput to the network, and y is corresponding response. Let all weights of a BNN, W = (W1 ...... DDD), be a random vector, where D is the depth (i.e., number of layers) of the BNN and each W = (ueh. (u.........
v). where
:-... w.)d)) is random vector itself of all weights w per layer W of size 1j. To generate uncertainty of the prediction, we need to be able to compute p(v (z). However, since all weights of a BNN are considered to be random variables, we can rewrite the conditional probability as p(y[z) Smu p(y, W| W(r)dW
Sw p(y W x))(WWWW (x) Typically, the likelihood term xx) is defined by the problem setup, e.g., if we consider classification, as in ransomware incident detection, y ~ Bern(g(W (W, æ)) for some function g. Then, the main problem of training a BNN is to compute the posterior probability p(W[a)) [r), given the observed data x and a suitable prior for W.
In some simple cases of small neural networks, it may be possible to obtain a closed-form solution for the posterior if the prior and posterior are conjugate distributions. In other cases, a closed- form solution is unavailable, sampling- caspling-based strategies are required such as Markov Chain Monte Carlo schemes based on Gibbs or Metropolis Hasting samplers. While such an approach provides excellent statistical behavior with theoretical support, scalability ity of the problem is known to be a serious issue. The alternative for machine learning and vision problems is Variational Inference (VI) (Graves, 2011). The core concept of VI is based the fact that approximating the true posterior with another distribution may often be acceptable in practice. The computational advantages of VI permit estimation procedures in cases which would not other- wise be feasible. VI is now a mature technology, and its success has led to a number of follow-up developments focused on theoretical well as practical aspects (Blundell et al., 2015).
function of the dimensional-
When using VI in Bayesian Neural Networks, we approximate the true unknown posterior distri- bution P(W(x) (x)) with an approximate posterior distribution Qo of our choice, which depends on learned parameters O. Let W/ = (W).... ...... WWP denote a random vector with distribution Qo and probability distribution function (pdf) qa- VI seeks to find 0 such that Qo as close as pos- sible to the real (unknown) posterior ()) and this is accomplished by minimizing the Kull- back-Leibler (KL) divergence between Qo and P(W[x). Given a prior pdf of weights, p, with a likelihood term p(ylW,z). 2), and the common mean field assumption of independence for Wd and w& for d 1,...,D. D. i.e. p(W)= = IT (Wd) and qo - A (WW))
0" arg min KL (goll)) Eng [In [In((/W,,))]
D
KL (gellp)
= Σ d=
Eug In (w)) - -Egg [In)((w)]
(2)
By definition of the expected value Egg. it is necessary to compute the dulti-dimeensonall integral w.n.t ~ Qo solve equation 1. If such integrals are impossible to compute in closed-form, numerical approximation is used (Ranganath et al., 2014; Paisley et al., 2012: Miller et al., 2017). For example, Monte Carlo (MC) sampling yields asymptotically exact, unbiased estimator with
3

---

Under review as a conference paper at ICLR 2023
variance ( where M is the number of samples. For a function g()):
gg [g(w)] /
1 M Σ ((w)), where ws Qs.
x
The expected value terms in equation and equation 2 can be estimated by applying the method in equation 3, and in fact, even if closed-form expression can be computed, an MC approximation may perform similarly given enough samples (Blundell al., 2015).
Given a mechanism to solve equation 1, the main consideration in VI is the choice of prior p and the approximate posterior qo. A common choice for p and qo is Gaussian, which allows calculating equation 2 in closed-form. However, this type of distribution is mainly used for computational pur- poses and does not reflect the nature of the data. Choosing the correct distribution, especially the one which can incorporate the features of the analyzed data, is an open problem (Ghosh & Doshi Velez, 2017: Farquhar et al., 2020; McGregor et al., 2019; Krishnan et al., 2019). In the next section, we discuss our proposed distribution, which naturally fits the data encountered in ransomware incident detection. While we give the description of the analyzed data in Section 2, we next describe the features of the data, which are important encapsulate in the model design.
Spike and Slab distribution. The sparsity of the data is a common problem in many areas (Kang, 2013) and was previously approached from different perspectives. For example in the statistics com- munity, sparsity can be addressed with both Stochastic Regression Imputation and Likelihood Based Approaches (Lakshminarayann et al., 1999). In the machine learning community, methods based on k-nearest neighbor (Batista & Monard, 2003) and iterative techniques (Buuren & Groothuis- Oudshoorn, 2010) have been developed, including approaches with neural networks (Sharpe & Solly, 1995; Smieja et al., 2018). Another way tackle sparsity comes from regularization the- ory via L1 regularization, e.g., group LASSO (Meier et al., 2008), sparse group LASSO (Simon al., 2013) and graph LASSO (Jacob al., 2009).
However, we are interested in a probabilistic approach to address the sparsity in our data. From the probabilistic perspective, a common way account for sparsity of the data in the model is to consider an appropriate distribution. For example, the distribution can be the Horseshoe dis- tribution (Carvalho et al., 2009) or derivatives of the Laplace distribution (Babacan et al., 2009; Bhattacharya et al., 2015). Namely, in our case, we would like to model sparse data with sparse probabilistic Bayesian neural network. Since only a portion of the input variables are relevant to the response variable, want the weights be represented we should account for the input variables. Such a sparse Bayesian neural network can be repre- sented by 'sparse' distribution on its weights, e.g., the mixture of priors with Spike and Slab components which have been widely used for Bayesian variable selection (Mitchell & Beauchamp, 1988; George & McCulloch, 1997). In general, the form of the Spike and Slab distribution for ran- dom variable w can be written as: w ~ (1 - π )ōg + Tg. where π determines the probability for each mixture component, is spike component, which modeled with a Dirac delta function such that
on/off switches to understand whether
5(w) +00, = E and - 8(w)dw 1, and g is the slab component, which a general 0, w E distribution of the practioner's choice. The general idea to explicitly introduce the sparsity com- ponent in the distribution of the data, allowing the probability mass to fully concentrate on & = 0 with probability 1 - n, and with probability л spread the remaining mass over the domain of the slab component g. Notice, that can be considered as random variable itself, e.g., Bern(A), where À is either a learned parameter or a fixed value that is provided by a specialist.
The next questions to consider include: (1) how can the 'Spike and Slab' distribution be applied in a BNN, and (2) which slab component g should we consider?
Spike and Slab BNN. In the BNN, all of the neural network's 's weights W considered to be random variables, and to use VI to solve equation for each layer's set of weights W in W = (W', WP), it necessary to provide the prior p and the approximate posterior of- Without loss of generality, we consider a single weight w == witk dropping the indices j and k, and only work with the prior p and the approximate posterior q for the remainder of this section.
4

---

Under review as a conference paper at ICLR 2023
Incorporating Spike and Slab distribution both the prior p and the approximate posterior q, samples up from p and wa from have the following distribution:
Wplap (1 - (p-mp)6u + Ty9p and wylm ~ (1 my my)00 + mg9..
(4)
where Tp Bern(Ap), Ta ~ Bern(Xg), and 9p. 9y are distributions of our choice.
As we discussed previously, the main goal of is to learn parameters of an approximate posterior qo. by minimizing equation In the case of equation 4, (^g. fy), where Xy is the probability of the Bernouli distribution associated with Tg. and fg are the parameters of the Slab component 94- First, we state Theorem 3.1, which allows us to compute the KL term between two general Spike and Slab distributions.
Theorem 3.1. Given two general Spike and Slab distributions such that: p (1-xp))o (ulp,w) ((w)Ta)) = (1 - kq) (o()))(()))))))) + kggg(w); (w); Tp ~ p(()) = Bern(Xp), and TT q(T)) Bern( Ng), with dy being a dirac delta function at 0 and 9p: 9q are the pafs of the distributions of our choice, the LL (g(w, π T))lp(w, x) is equal to:
=
KL (Bern((a)))
Xg KL (ggllop)
(5)
The proof is shown in Appendix F.
Choice of Sy and gp: Radial distribution. So far, we have shown results for general Spike and Slab distribution. The important question is which slab components should we consider for our approach, and if gy and 9p should be from the same family? Authors in (Bai et al., 2020) consid- ered both 9n and Op to be the Gaussian distribution. However, there is emerging evidence (Farquhar al., 2020; Fortuin et al., 2020) that the Gaussian assumption results in poor performance of the medium to large-scale Bayesian Neural Networks. Authors regard this as being caused by the prob- ability mass in nigh-dimmnsional Gaussian distribution concentrating in a narrow "soap-bubble" far from the mean. For this reason, (Farquhar et al., 2020) proposed using Radial distribution with parameters (n,, σ), where samples can be generated as:
|r| Rodial(y, o), whereg MVN
(6)
|El
Following (Farquhar al., 2020), we set up our approximate posterior Gg to be the Radial distri- bution (H, o), while the prior 9p is Normal(O, 1). Given equation 5, it is necessary to define the KL (9g)l term. Unfortunately, a closed- form solution for our choice of gg and 9p not available, and we approximate the KL L term using Monte Carlo sampling from equation 3 with M samples. MM log(p(w:) (w,),, where w; is sampled from the Radial distribution (14, σ) as described in equation and p is the Likelihood of N 1). Note that running MC approximation for large M can lead to running out of memory in either a GPU or RAM, (Nazarovs et al., 2021). To tackle this issue, we follow (Nazarovs et al., 2021) and apply a graph parameterization for our Radial Spike and Slab distribution, allowing us to set M = 1000 without exhausting the memory.
This process leads to (up constant): KL (gyll9p) x - -log
Reparameterization trick: Gumbel-Softmax, Given Theorem 3.1, can rewrite equation 1 as:
0" arg min KL (Bern(>n)) ( + >, (gallsp) Eg [Inp(y] 0=)).00))
Recall, we can compute the K L (Bern(>x)]] Bern(>>)) in closed-form (inside the proof of The- orem 3.1) and approximate the KL (g,119p) term with MC sampling. Next, there are two main aspects left for our attention: (1) computing Eyg [In p(y W)). which is usually approximated with Monte-Carlo sampling (Kingma & Welling, 2013) because of the intractability issue, and (2) how to do back-propagation for optimization. The problem with back-propag in this setting that sampling directly from, e.g.. Nu,, σ) with learnable parameters μ and does not allow to back- bock-propegate through those parameters, and thus, they cannot be learned. This issue is addressed by applying recaaattpratntniitiioooonnnn trick (Kingma et al., 2015). For example, instead of sam- pling from w ~ N (p,o o) directly, we sample 2 ~ (O, 1) and compute: w = μ + az. This allows back pack-propagation to optimize the loss w.r.t. l and
5

---

Under review as a conference paper at ICLR 2023
While the recal-reparametteizzaioon trick is obvious for members of location family, like the Gaussian distribution, and even for the selected Radial distribution, it is not clear how to apply this trick to the Bernoulli distribution, Bern(A). One way to address this issue is to approximate samples from the Bernoulli distribution with the Gumbel-Softmax (Maddison et al., 2016; Jang et al., 2016; Bai et al., 2020). That is,
Bern (A) is approximated by म Gumbel-Softmax (A, r). where (1 exp (-n//)) -7 log r +og -- and u ({ 1). Here, 7 the parameter which referred as the temperature. When 7 approaches 0, i converges distribution x. However, in practice, 7 is usually chosen no smaller than 0.5 for numerical stability (Bai et al., 2020). Applying the Gumbel-Softman approximation instead of optimizing the loss for parameter Ag. we consider
a new parameter 0 = log -- Thus, Xy = S(O,) = Ite PF resulting in the final learned parameters: 0- a0,, 0g).
Final Loss and Method Summary. A step-by-step summary of the method in provided in Algo- rithm The final loss is given in Algorithm 2.
Algorithm 1: Learning the posterior distribution of BNN p(W (7) with a Radial Spike and Slab approximate
posterior, to account for sparsity of the data.
Input:
1: Neural Network of depth D with
which have 3: Spike and Slab Radial distribution Qo with pdf ga, s.t.
Weights W; = (W) (W)..... W.e).
) + (wr pt, o), (,: o) pdf of Rodial(u,
((w)(F/) = (1 = (
-Ty ~ Bern(S(0+)), where S the softmax, and 4: Prior Spike and Slab distribution P& with pdf p. s.t.
p(w|T,) = (1 Ty. ( Tpdp (w: (pp op); (p(m; Mp: op) is pdf of Gaussian distribution
~ Bern(Tp))
Output: Learned parameters 0 = (0)
Require:
Prior distribution's 's parameters Mp, (p)
5: while 0 has not converged do
6:
Minimize VI loss in equation 8, by using gradient descent algorithms (e.g.. SGD or Adam) and doing:
7: Forward pass: compute
with local reparameterizaation trick for both Radial and Bemoulli (using Gumbel-Softmax) K L terms and expected log likelihood term, using combination of closed-form and MC
8: Backward pass: compute gradients of
9: end while
Algorithm 2: Final loss used for optimization in Algorithm 1. KL (Bern(),) Bern (),)) 1)- KL (g,, 11p)) - Eq, W[,
Original:
Final:
L
Σ KL,a Eq, [lup(y) x)], where
(8)
KL,,A = (1-S(044)) log
1-5 -(( (
log
st0t*) +S(S5 { log k x1.
M logíp( w/
DM
1- x
Note that based on the mean field assumption of a BNN, the final loss L includes the sum over all K Ly, terms, which are computed for each k-th weight w of the j-th layer of the BNN with parameters 0i-k = (0 so'. x²**)) In this case, the final set of trainable parameters is = {05-a) for j = 1,..., D and A: - 1,..., 1,. In addition, Eq, can be computed either in a closed-form or approximated by MC. depending the complexity of the BNN.

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS
Data description. As described previously in Section 2, each incident is represented by a temporal sequence of events from a knowledge base of TTPs with an assigned label, which indicates whether it is ransomware or another type of attack. First, the company provided 201 incidents labeled as Ransomware and 24,913 with Non- Non-Ransomware labels for the initial dataset. All of the samples in this dataset were deduplicated and included 706 sparse binary features. This first dataset was randomly split with 80% of the examples assigned to the training set, while the remainder were used to create a validation set. Second, for the test set, we received a newer, deduplicated dataset making it independent of the training and validation sets. This dataset included 644 Ransomware incidents and 14,696 Non-Ransomware incidents.
Preprocessing of temporal information. Some of the models such as the Neural ODE benefit from knowledge of the actual time associated with the recorded event, while others, including the RNN with GRU cell, can be trained on the event sequence based solely on the event index (i.e., t=., Finally, other models such as the fully connected and Bayesian Neural Networks can be trained and tested using the aggregation of all of the events in the event sequence. To reduce the number of time steps for the time-based models for our study, we aggregated all TTP events observed within one minute window. We set the aggregation time to one minute after doing hyperparameter tuning on this value. This results in very few signals being recorded per aggregated time step, which is represented in Figure in Appendix D. We see that the majority of the data have a small number of features that set, namely less than 10 out of 706 possible. For the neural network models, we aggregated all of the TTP features into a single input vector. All of the sequences for the training and testing datasets were truncated after one hour from the time of the first event.
Models. In the experiments, we consider several baseline models from the traditional, temporal, and probabilistic deep learning settings, in addition to our proposed model. From the temporal perspec- rive, we consider two models including the Recurrent Neural Network with GRU cell (RNN) and the Neural ODE (NODE). As we mentioned earlier, the traditional recurrent neural network models (e.g., Simple RNN, GRU, LSTM) ignore the value of the time steps and only consider the order (i.e., index), in contrast to the Neural ODE which accounts for the time step value. Note, we originally considered several temporal models, which do not account for the time value, like the traditional (i.e., Simple) RNN, the RNN with a GRU cell, the LSTM, and the Bi-directional LSTM. However, among all of these models, the RNN with the GRU cell performed the best, and we only include this model in the analysis below.
In addition we consider the traditional fully connected neural network (FC), and four BNN models. The first two BNNs are the standard BNNs which have a Gaussian or Radial approximate posterior (BNN: Gaus, BNN: Radial), and the other two the corresponding Spike and Slab versions, BNN: Spike-Slab Gaussian and our proposed BNN: Spike-Slab Radial. For these networks, we ignore the temporal aspect of the data by aggregating all available features per entry with the 'logical or' operator. Since our features are binary, aggregation corresponds to summarizing the information into the set of events which occurred during the time period. In addition, we also considered an approach with a Bayesian Network (i.e., not BNN). However, the BN model failed to converge due to the sparsity and high dimensionality of the data. Furthermore, we also trained many variants of XGBoost (Chen & Guestrin, 2016), but all of the boosted decision tree models produced random results. Therefore, we did not include the results for XGBoost below.
Parameter settings/hardware All experiments were run on an NVIDIA P100. The code was implemented in PyTorch, using the Adam optimizer (Kingma & Ba, 2014) for all models, and trained for 400 epochs. The model with the lowest validation loss was selected for evaluation. The final hyperparameter settings are specified in Appendix A.
Ablation study. To understand the effect of the distribution on the BNN, we conduct ablation study between the Gaussian BNN, Radial BNN, and their Spike and Slab versions. We provide results in Table 1. Clearly our proposed method provides better results in number of metrics, including Specificity, Precision, F1, and FPR, which are important for Ransomware detection.
Model Evaluation. In Figure 1, provide the ROC curves for the proposed model and several baselines. For our Radial Spike and Slab BNN method and the Gaussian BNN method, we display the distribution of each model's ROC curves, shaded in green, together with its mean value (e.g..
7

---

Under review as a conference paper at ICLR 2023
IIN
BNN
BSSC
Stataiis
[1N0:
[Sondas
Ratid Sylio S.hhh
(Sasdize
Ratiy
RNN
Nexd onT
Stab
BNS-GRU
Nearal 061
PC
Sytke
0.83
0.90
0.53
0.82
0.80
Precusse
8.06
18:0
091
0.09
0.06
D066
18.0
018
016
16d5
18300
1133
0.12
8.15
ITK
11:00
01:31
0.11
0.10
11.7
0110
0:31
0.18
1131
1629
1130
8J8
0.27
FDR
0.56
0.97
0.3
0.8
0.94
0.4
0.80
0.91
0.6g
D94
0.90
0.55
0.55
0.57
Accasay
0.50
0.80
0.87
0.85
0.91
0000
0.85
0.80
0.70
[11
0.86
0.90
0.50
8.91
Acnrney
11.0
0.14
0.50
0.02
0.11
0.314
0.11
1112
11.3
11.00
11:0
0.21
G-Mon
0332
0173
0390
0.51
0,53
8,33
0883
0366
0.70
1689
11.22
6873
0.73
0.70
Table 1: Ablation study for both validation set and the test set, which contains data from the future.
green line). Figure la shows that, with respect to the distribution of the ROC curves, our model out- performs the baselines on average, particularly in the region of small false positive rates, Figure 1b, which is the most important for ransomware detection. In addition, the Radial Spike and Slab BNN able provide range of ROC curves which are significantly higher than the other baselines, if we consider the margins of the ROC distribution. Looking at the columns for the validation and testing set Table 1, we see that proposed BNN model outperforms the baseline methods, w.r.t. to AUC, accuracy, G-Mean and other statistics.
KNN GRU
Neural JUE
KC
BNN: Gaussian
BMC Radial Spike & Slab
True Postivv Rote
Thue Posetve Rabe
0.5
0.2
0.4
2.4
0.3
03
07
02
0.1
04
10.
0.5
0.3
0.5
04
0.
0.6%
51
A15
8
False Positive Rate
Felse Positive Rate
(a) Test Set
(b) Test Set Zoomed In
Figure 1: We present the ROC curves for the new data from the future time period in the Test Set. Because the BNN is a probabilistic model, show the distribution of the individual ROC curves (green shade) with the mean this distribution (green line).
Training and Test Times. Training the Radial Spike and Slab Bayesian Neural Network in single Azure-hosted Linux VM with an NVIDIA P100 for 400 epochs required hour, 32 minutes and 53 seconds. The time required to evaluate the 15,340 samples in the test set 19 seconds. How- ever, to create a confidence interval (CI), the evaluation is repeated 100 times. Thus, 19 seconds corresponds to 100 evaluations. We re-ran inference on an NVIDIA A100 for all of the models to compare to 1 run of deterministic models. The results include 4.07 sec (I iteration) and 10.37 see (100 iterations for CI) for the BNN: Radial Spike Slab (ours), 4.02 sec for the BNN: Gaussian, 7.72 sec for Fully Connected NN (FC), 3.12 see for the Neural ODE, and 2.35 see for the RNN-GPU.
Feature Importance and Interpretation.
We would like to understand which TTP features of the attack are considered to be important by our model when making a prediction whether an attack is ransomware or not. One way to do this is to investigate the posterior probabilities for the first layer weights of the BNN. However, while understanding the importance of the TTP features based on the BNN's trained weights conceptually makes sense, we instead follow more well-known and established way to interpret the features of a general neural network, called Integrated Gradients (Sundararajan et al., 2017). Both methods are discussed in Appendix E. In Table 2, we present the subset of features which are the most important for our model to identify whether an attack is ransomware or some other type of attack based on Integrated Gradients. Sorting the values of Integrated Gradients, we find that the MITRE ATT&CK
8

---

Under review as a conference paper at ICLR 2023
features significantly more important than the signature-based features. The "signature" in Ta- ble 2 is a low-level event generator from an analyst. As Table 2 shows, the MITRE events are much more important than the low-level signatures.
RELATED WORK
Recently, ransomware has become an active research area (Oz et al., 2022; McIn- tosh et al., 2021). Machine learning approaches have been proposed for the de- tection of ransomware at- tacks. A stacked, varia- tional autoencoder is used to detect ransomware in the industrial IoT (IIoT) set- ting (Al-Hawawreh & Sit- nikova, 2019). System API calls are used to de- tect ransomware using De- cision Trees, K-Nearest Neighbor classifier, and a Random Forest in (Sheen & Yadav, 2018). Takeuchi et al. (Takeuchi et al., 2018) also proposed using an SVM to detect ransomware using System API calls. Agrawal et al. (Agrawal et al., 2019) proposed input vector of an LSTM, an RNN and a GRU to improve the detection of ransomware attacks from API calls. An ensemble of network traffic classifiers the Locky family of ransomware in (Almashhadani et al., 2019). A Bayesian Network was the best performing flow-based classifier in this work while Random Tree the best for detecting packets in this work. HelDroid (Andronio et al., 2015) uses natural language processing techniques, along with static and dynamic analysis, to detect ransomware on mobile computing devices. Adamov and Carlsson (Adamov & Carlsson, 2020) use reinforcement learning to simulate ransomware attacks for testing ransomware detectors. Urooj et al. (Urooj et al., 2021) proposed an online classifier to predict early stage ransomware, but they do not provide any details for the classifier itself.
Id
Feature representation
T1059.001
Command and Scripting Interpreter, Powershell
T1105
Ingress Tool Transfer
T1087
Account Discovery
Signature
Suspicious activity was observed on this device
T1049
System Network Connections Discovery
T1027.002
Obfuscated Files Information: Software Packing
T1566.001
Phishing: Spearphishing Attachment
T1546.00)
Event Triggered Execution: Change Default Association
T1218.003
Signed Binary Proxy Execution: CMSTP
T1055.004
Process Injection: Asynchronous Procedure Call
Table 2: The Integrated Gradients method produces score for each of the TTP features which indicates the importance of the feature for predicting whether the attack is ransomware (top) another type (bottom). Features ranked from the highest to lowest Integrated Gradients scores.
attention mechanism on the
used to detect network packets and flows for
LIMITATIONS AND CONCLUSION
In this work, we propose the new Radial Spike and Slab Bayesian Neural Network and demonstrate that it outperforms the standard Bayesian Neural Network and other deep learning methods for the task of detecting ransomware attacks within the general class of all attacks, such as the dropping of commodity malware. The results can provide early indicator of potential ransomware attack for analysts to be able to confirm with additional investigation.
While the model is able to learn to distinguish between ransomware attacks and other attacks, the ROC curve indicates that it cannot be used by a fully automated system to completely disable com- puters or block network access due potential ransomware outbreak. However, since these attacks are being diagnosed by analysts, we believe that the model can alert these analysts about possible active ransomware attack on their network.
Given that ransomware attacks are relatively rare compared to the downloading of commodity mal- ware, the amount labeled data for these types of attacks is small. The size of our datasets from production security service reflects this limitation. Fortunately, Bayesian computational methods such as Bayesian Neural Networks can be used for training and inference without overfitting in scenarios where the amount of labeled data is limited.
In addition to security problems, two significant areas, which focus on the analysis of high- dimensional sparse data, Biostatistics and Geneties. While there notable developments in sparse methods in those areas, our proposed method is novel for them as well. Once the source code public, we hope that researchers from other fields will find the proposed model useful.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Alexander Adamov and Anders Carlsson. Reinforcement learning for anti-ransomware testing. In 2020 IEEE West Design Test Symposium (EWDTTS) pp- 1-5, 2020. doi: 10.1109/
EWDDTS0064.2020..055 221..225111.
Rakshit Agrawal, Jack W. Stokes, Karthik Selvaraj, and Mady Marinescu. Attention in recur- rent neural networks for ransomware detection. In ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 10. 10.1109/10 N0.100//CASSP.. 2019.8682899..
2019. doi:
Muna Al-Hawawreh and Elena Sitnikova. Industrial internet of things based ransomware detection using stacked variational neural network. Proceedings of the 3rd International Conference Big Data and Internet of Things, BDIOT 2019. pp. 126-130. New York. NY. USA. 2019. Association for Computing Machinery. ISBN 978 9781450372466. doi: 10.1145/3361558.33611600 URL httpt::tp//......... http//di..rr..o/// 0.1 / 33555/3611000
Omar MK Alhawi, James Baldwin, and Ali Dehghantanha. Leveraging machine learning techniques for windows ransomware network traffic detection. In Cyber threat intelligence, pp. 93-106.
Springer, 2018.
Ahmad O. Almashhadani, Mustafa Kaiiali, Sakir Sezer, and Philip O'Kane. A multi-classifier network- crypto ransomware detection system: case study of locky ransomware. IEEE Access, 7:47053 47067, 2019, doi: 10. 1000//CCCSSSS.. 2019.,9949999
Nicoló Andronio, Stefano 7anero, and Federico Maggi. HelDroid: Dissecting and Detecting Mobile Ransomware. In Proceedings of the 18th international conference on Research in Attacks, Intru-
sions, and Defenses, Lecture Notes in Computer Science, pp. 382-404. Springer International
Publishing. November 2015. ISBN 978-3-319-26361-88 97833 doi: 10. 1007/977 3-319-26362-5_18 URL http://tpe//.....oo/./../0//0030
S Derin Babacan, Rafael Molina, and Aggelos K Katsaggelos. Bayesian compressive sensing using laplace priors. IEEE Transactions on image processing, 19(1):53-63, 2009.
Jincheng Bai, Qifan Song, and Guang Cheng. Efficient variational inference for sparse deep learning with theoretical guarantee. arXiv preprint arXiv:2011.07439) 2020.
Gustavo Batista and Maria Carolina Monard. study of k-nearest neighbour as an imputation
method. In In HIS. Citeseer, 2003.
Anirban Bhattacharya, Debdeep Pati, Natesh S Pillai, and David B Dunson. Dirichlet-laplace priors for optimal shrinkage. Journal of the American Statistical Association, 110651)):149911490 2015.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. arXiv preprint arXiv:1505.05424 2015.
S van Buuren and Karin Stoothuis-Oudshooo mice: Multivariate imputation by chained equations in r. Journal of statistical software, pp. 1-68, 2010.
Carlos M Carvalho, Nicholas G Polson, and James Scott. Handling sparsity via the horseshoe. In Artificial Intelligence and Statistics, pp. 73-80. PMLR, 2009.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary 2018.
differential equations. Advances in neural information processing systems,
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 16, pp. 785-794, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 978 4500422222 doi: 00.1145/2939672.2997788 U 2939672.2939788
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre. Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using mn encoder-decoder for statistical machine translation. arXiv preprint arXiv: /406. 1078, 2014.
10

---

Under review as a conference paper at ICLR 2023
Sebastian Farquhar, Michael A Osbore,, and Yarin Gal. Radial bayesian neural networks: Beyond discrete support in large-scale bayesian deep learning. stat, 1050:7, 2020.
Vincent Fortuin, Adrià Garriga-Alonso, Florian Wenzel, Gunnar Ratsch, Richard E Turner, Mark van der Wilk, and Laurence Aitchison. Bayesian neural network priors revisited. In "I Can 'r Believe It's Not Retter!"NeurlP" 2020 workshop, 2020.
Edward I George and Robert E McCulloch. Approaches for bayesian variable selection. Statistica sinica, pp. 339-373, 1997.
Soumya Ghosh and Finale Doshi-Velez. Model selection in bayesian neural networks via horseshoe priors. arXiv preprint urXiv: 2017.
Alex Graves. Practical variational inference for neural networks. Advances in neural information processing systems, pp. 2348-23560 2011.
Laurent Jacob, Guillaume Obozinski, and Jean-Philippe Vert. Group lasso with overlap and graph lasso. In Proceedings of the 26th annual international conference on machine learning, pp. 433- 440, 2009.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizatioo with gumbel-softmax. arXiv preprint arXiv:1611.01144 2016.
Hyun Kang. The prevention and handling of the missing data, Korean journal of anesthesiology, 64 (5):402, 2013.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1rXiv:1312.6114, 2013.
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameteri- zation trick. In Advances in Neural Information Processing Systems, pp. 2575-2583, 2015.
Ranganath Krishnan, Mahesh Subedar, and Omesh Tickoo. Efficient priors for scalable variational inference bayesian deep neural networks. In Proceedings the IEEE International Conference on Computer Vision Workshops, pp. 0-0, 2019.
Kamakshi Lakshminarayan, Steven A Harp, and Tariq Samad. Imputation of missing data in indus- trial databases. Applied intelligence, 1(3))2599275, 1999.
Erik Larsen, David Noever, and Korey Vatti. A survey of machine learning algorithms for detecting ransomware encryption activity. arXiv preprint 1rXiv:2110.076::: 2021.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint wrXiv:1611.007/11 2016.
Felix McGregor, Amu Pretorius, Johan du Preez, and Steve Kroon. Stabilising priors for robust bayesian deep learning. arXiv preprint 1910: 2019.
Timothy McIntosh, S. M. Kayes, Yi-Ping Phoebe Chen, Alex Ng, and Paul Watters. Ransomware mitigation in the modern era: A comprehensive review, research challenges, and future directions. ACM Comput. Surv., 54(9), oct 2021. ISSN 0360-0300. doi: 10. 10.1145/3479393. URL https:
/ (ab...ogg/1..11/////1
Lukas Meier, Sara Van De Geer, and Peter Bühlmann. The group lasso for logistic regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 70(1):53-71, 2008.
Andrew Miller, Nick Foti, Alexander D' Amour, and Ryan P Adams. Reducing reparameterization gradient variance. In Advances Neural Information Processing Systems, pp. 3708-3718. 2017.
Toby Mitchell and John J Beauchamp. Bayesian variable selection in linear regression. Journal of the american statistical association, 83(404): 1023-1022, 1988.

---

Under review as a conference paper at ICLR 2023
MITRE. Enterprise Techniques.
httpos//attakknitreerr///hhhhaat enterprise/ [Online; accessed 20-January-2022].
MITRE. Mitre att&ck. https: http:://attackm/ttreorr/.//
MITRE. Command and scripting interpreter: Powershell. https://attackkmtrreorg
techni.qes/TT0559 /0000/0/01/,// 2022b.
Jurijs Nazarovs, Ronak R Mehta, Vishnu Suresh Lokhande, and Vikas Singh. Graph reparameteriza- tions for enabling 1000+ monte carlo iterations in bayesian deep neural networks. In Uncertainty in Artificial Intelligence, pp. 118-128. PMLR, 2021.
Diane Oyen, Blake Anderson, and Christine edge for malware phylogenetics. In Workshops at the Thirlieth AAAI Conference on Artificial Intelligence, 2016.
Bayesian networks with prior knowl-
Harun Oz, Ahmet Aris, Albert Levi, and A. Selcuk Uluagac. A survey on ransomware: Evolution, taxonomy, and defense solutions. ACM Comput. Surv., jan 2022. ISSN 0360-0300. doi: 10.1145/ 3514229. URL
12299. Just Accepted.
John Paisley, David Blei, and Michael Jordan. Variational bayesian inference with stochastic search. arXiv preprint arXiv: /206.640,, 2012.
José A Perusquía, Jim E Griffin, and Cristiano Villa. Bayesian models applied to cyber security anomaly detection problems. arXiv preprint arXiv:2003.10360 2020.
Subash Poudyal, Kul Prasad Subedi, and Dipankar Dasgupta. A framework for analyzing ran- somware using machine learning. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1692-1699. IEEE, 2018.
Rajesh Ranganath, Sean Gerrish, and David Blei. Black box variational inference. Intelligence and Statistics, pp. 814 822, 2014.
Artificial
Peter K. Sharpe and RJ Solly. Dealing with missing values in neural nework-based diagnostic systems. Neural Computing & Appliccations, 3(2):73-77, 1995.
Shina Sheen and Ashwitha Yadav. Ransomware detection by mining api call usage. In 2018 Inter- national Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 983-987, 2018. doi: 10. 1w..109/CACCC.2218855555555
Jinsoo Shin, Hanseong Son, Gyunyoung Heo, et al. Development of a cyber security risk model using bayesian networks. Reliability Engineering System Safery, 134:208-217, 2015.
Noah Simon, Jerome Friedman, Trevor Hastie, and Robert Tibshirani. A sparse-group lasso. Journal of computational and graphical statistics, 22(2):231 --45, 2013.
Marek Smieja, Lukasz Struski, Jacek Tabor, Bartosz Zieliński, and Przemyslaw Spurek, Processing of missing data by neural networks. In Advances in Neural Information Processing Systems, pp. 2719-2729, 2018
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pp. 3319-3328, PMLR, 2017.
Yuki Takeuchi, Kazuya Sakai, and Satoshi Fukumoto. Detecting ransomware using support vec- for machines. In Proceedings of the 47th International Conference Parallel Processing Companion. ICPP '18, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781453655399, doi: 10. 10.1145/3297710.322972 UtLhttps://aiiooose $229710.322999
Umara Urooj, Mohd Aizaini Bin Maarof, and Bander Ali Saleh Al-rimy. A proposed adaptive pre- encryption prypto-ransomyy Conference (CRC), pp. 1-6, 2021. doi: 10.1109/CRC50527
early detection model. In 2021 3rd International Cyber Resilience .9392488.
Hanqi Zhang. Xi Xiao, Francesco Mercaldo, Shiguang Ni, Fabio Martinelli, and Arun Kumar San- gaiah. Classification of ransomware families with machine learning based on n n-gram of opcodes. Future Generation Computer Systems, 00:211-2211 2019.
12

---

Under review as a conference paper at ICLR 2023
A HYPERPARAMETEE SETTINGS
For reproducibility, this appendix provides the hyperparameter settings used for the proposed model as well as those for the baseline models. The best general hyperparameter settings from tuning are provided in Table Other network hyperparameter are included in Table 4. The hyperparameters for the RNN, ODE, Fully Connected, and Bayesian models are provided in Tables, 5, 6, 7, and 8, respectively.
Parameter
Value
train batch size
100
maximum num train epochs
400
hidden size
706
learning rate
le-4
learning rate for probabilitics
le-3
Adam 81
0.5
Adam [32
0.999
Table 3: General hyperparameters used for training the proposed Bayesian neural networks and baseline mod- cls.
Parameter
Value
Include binary classification loss
True
Parameter for positive weight in the binary loss to represent imbalance of data
0.0068
std used in likelihood term (or MSE)
0.1
dropout retention rate for discriminator slope leaky relu function
0.9
0.2
Table 4: Network hyperparameters.
Parameter
Value
Number of layers in ODE fune in recognition ODE
100
Number of units per layer ODE func
0.0068
ODE solver
Euler
ODE fune units
300
ODE fune rec num layers
300
Table 5: ODE hyperparameters.
B RANSOMWARE ATTACKS
Ransomware attacks fall into two main categories, automated ransomware which include infamous cases such as WannaCry, and human operated ransomware (HumOR) conducted by actor groups such REvil and myriad of others. Although automated ransomware involves humans, the dis- tribution of the payload usually does not involve human interaction. HumOR attacks, however, involve wands-on-keyboarr activity, where an active human adversary has gained access net- work- whether through purchased access, malware, vulnerabilities, or other means - and progresses through the kill chain to escalate privileges, move laterally if possible, and distribute ransomware in the environment. Human operated attacks tend to be more severe, as the adversary is able to take steps to bypass protections and work ensure the ransomware payload executed successfully. Security solutions will actively monitor for these suspicious events across the different kill chain stages in a ransomware attack to detect and alert on the malicious behaviors.
Ransomware attackers will typically utilize multiple toolkits, custom malware, and scripts to con- duct their activity more effectively. Often this can also entail multiple operators for different stages in the kill chain, such with Ransomware-as-a--55 who work to create tools and provide access for vetted attackers - known as affiliates to conduct the majority of the ransomware attack. Complicating matters, many of the tools ransomware attack- ers frequently use are open-son open-source and have legitimate purposes, preventing outright detection and blocking unless the method of using the tools can specifically be classified malicious.
(RaaS) attacks. RaaS involves operators
13

---

Under review as a conference paper at ICLR 2023
Parameter
Value
RNN cell type
GRU
GRU units
300
Table 6: RNN hyperparameters.
Parameter
Value
Number of hidden layers in FC network Number hidden units FC network FC network
300
Table 7: Fully connected neural network hyperparameters.
There are several challenges for detecting and blocking ransomware attacks. First, there is a time criticality required for detection prior to the distribution and encryption of devices. Ideally, a good ransomware detection service can detect ransomware attack prior the encryption of any assets. This necessitates detecting the compromise as early in the kill chain as possible. However, the early stages of attack do not necessarily have clear and specific implications of ransomware and can often mirror attacks that are not ransomware in nature. Second, although ransomware attacks are increasing and regularly reported in the news, they are still rare, and the labeled data is limited. Therefore, ransomware detector must not overfit to sparse data. In addition, ransomware detection service must have access to signals from large number of computers or mobile devices in order to create datasets that can learn to detect important behaviors. Third, the system must generalize to handle polymorphism since the signals are polymorphic by their nature. Attackers may delay or reorder their activity, utilize open-source legitimate tools for malicious purposes, use polymorphic malware (c.g.. backdoors) or scripts, or fast flux networks for command and control to avoid detection. Finally, the input signals are often weak and often do not indicate a ransomware attack on their own. An effective ransomware detection service must be able to combine these low-level signals in order to produce successful high-level detection.
C THREAT MODEL
The BNN ransomware detector operates on data collected from the <the anonymized company's > currently operational backend security system. This system processes the low-level events which are generated by the device and stored a cloud service, or alternatively on-premise, and, like all security services, this creates several areas which must be protected from attack. The low-level events are generated by the device itself in kernel mode of the operating system. The system assumes that the events are successfully generated, transmitted, and received by the cloud on-premise backend service, and that the events have not been altered by the actor using attack. Next, the system assumes that the events have not been altered once they have received and stored in the backend service itself. Thus, the system assumes that there are no successful data poisoning or insider threat attacks. Finally, the system assumes that the ransomware alerts successfully transmitted to and correctly received by the customer's and the <the anonymized company's ' > analysis portals. It should be noted that of these system components are operational today, and the proposed model only affects the processing component in the backend service.
sin-the-miie middme
D TEMPORAL PREPROCESSING
In this appendix, we investigate the sparsity of the raw dataset. In Figure 2, we show the distribution of the number of features that are set during each single one minute time step (i.e., time interval) in the raw data. In general, ten or fewer features are set during a single one minute time interval. A few minute time steps have between 10 and 20 features that set, while small number of others have between 40 and 55 features set. Interestingly. we found that none of the time steps had between 20 and 40 features set.
14

---

Under review as a conference paper at ICLR 2023
Parameter
Value
KL coefficient
Graves
Method to compute KL
reparameterization
Number of samples evaluate the test
100
Table 8: Bayes hyperparameters.
Number of attaao comesporett aoonniiigg to the number TTP
Ransomware stfacks
50
30 Number TTP features por
20
Count of the number of features available for 1. one minute time step of the ransomware data. The main mass contains less than 10 features per time point.
Figure 2:
E FEATURE IMPORTANCE AND INTERPRETATION DETAILS
In this appendix, we discuss two methods to determine most important features for the Radial Spike and Slab Bayesian Neural Network ransomware detection model. The first method we con- sider is to rank the posterior probabilities which are found in the first layer weights of the BNN. Recall that the which models the probability S(0z) each node in the neural network to be included. Given that the first layer of the BNN is fully connected, can consider S(0,) of the first layer the impor- tance of each TTP for our network. Since we suspect that not all TTPs are equally important, we expect to observe spikes in the learned S(0z). The results in Figure confirm this hypothesis, where we start from the uninformative, uniform prior (left) and generate spikes in the learned S(0z) (right) after training. However, while understanding which TTP features are important based on the BNN's
idea behind BNN with Spike and Slab distributions is to learn parameter 0,,,
Prior probability to include the feature
Learned posterior probability to include the feature
C.4
0.4
6.-
6.2
600
0.0
Input features of the data (TTP)
features of the data (TTP)
Figure 3: One of the approaches to evaluate the importance of the input features for Spike and Slab BNN is evaluate the learned posterior probability S(0=) (right). Note how different it is from the non-informative prior probabilities (left).
trained weights conceptually makes sense, instead follow more well-known and established way to interpret the features of a general neural network, called Integrated Gradients (Sundararajan et al., 2017) as the second method to rank the features.
15

---

Under review as a conference paper at ICLR 2023
Based on this procedure, network, and these scores are represented in Figure 4 for our model. contrast to the Bayesian ap- proach, this method also includes the signs of the feature scores. A positive attribution score means that a partieular feature positively contributed to the final prediction of an attack being ransomware and negative score indicates the feature was important for predicting non-ransomware attacks. The magnitude of the attribution score signifies the strength of the contribution. A feature which does meaningfully contribute the final output has score of near zero.
generate importance score for each feature given trained
Importance score
-0.05
-0.10
-0.15
Input features data (TTP)
Figure 4: Applying the Integrated Gradients method generates scores which indicate importance input features. A higher, positive score means that the feature is relevant for predicting data belonging to the ransomware class, while negative score means the feature is more relevant to predicting the non-ransomware attack class.
F PROOF OF THEOREM 3.
In this appendix, provide the proof of Theorem 3.1.
16

---

Under review as a conference paper at ICLR 2023
Proof.
K (q(w, T) I)),,(,, r))
S log ww x) (ww x ddwdd
p(w, π))
given that g(w, m) ((wp))))()))))) and p(w, - p(w)+)p(T)
= -A log g(w, x) q(w n ddw q(p)dd
p(w,50 x)
given that ((x) Bern (2g) and p(x)) (p))(Xp)
= a(( = 0) {1/ log (ww(0)(( 0)
p(w(0)p(( 0) q( (0)00)
log g(w) 1 (g( 1)
+ ((( = 1)
p(w) 1)(( = q(w) [)d
w
Ag
/ 50(w)ydw }
= (1 Nn)
log
Xp
m
Ag
/ log (q(ww
Ag log
+
9y (W))d (o(w))
Ap
1
+ Xg log Ag Ap
(1
1
1 Ap
>g / log [u(w)
9p(w) gg
KL (Bern(Xy)) X, KL. (ggllop)
17


Reviews:
[{"review_id": "jSwzbVUIV-P", "reviewer": "Reviewer_zqwu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method called “Radial Spike and Slab Bayesian Neural Network (BNN)” to detect ransomware attacks when the data is sparse and imbalanced. This method is a type of BNN with the posterior distribution represented by a mixture of distributions resulting in a Radial Spike and Slab distribution. The usage of this distribution is justified theoretically. For evaluation, the authors compare the proposed method against several baselines and variants on an internal ransomware dataset.", "summary_of_the_review": "Good paper but evaluation is relatively limited to one internal dataset and it might not be best suited to ICLR.", "strength_and_weaknesses": "## pros\n-well-written and easy to read.\n\n-the proposed method seems to be effective on ransomware attacks detection.\n\n## cons\n-only one internal dataset is used to evaluate the proposed method.\nThe proposed method might overfit that internal dataset to some extent.\n\n-the paper might be better suited to a security conference.\nSince the evaluation is only conducted on one dataset in ransomware attack, it is difficult to tell how generalizable the method is for other applications.\n\n-appendix is not provided?\nThe paper refers to the appendix many times but the appendix is not provided? Or did I miss it?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "K6xT5w0eFWw", "reviewer": "Reviewer_F9pu", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors propose to detect ransomware-based attacks amongst a general class of malware attacks using a Bayesian Neural Network (BNN). In such settings, the feature set available to subsequently perform inference or detection of vulnerabilities is quite sparse, and imbalanced. In order to address this, the paper proposes to utilize a new architecture for the BNN, namely using a Radial Spike and Slab BNN. Here, the weights are assumed to arise from a mixture distribution, with the first component being a dirac mass (or “spike”) and the second having an atomless support (or “slab”), namely a Radial distribution (Farquhar et al., 2020).", "summary_of_the_review": "The paper studies the very narrow problem of the isolated detection of ransomware attacks amongst a larger class of malware attacks. Furthermore, the results obtained using the proposed Radial Spike and Slab BNN do not seem statistically significant in achieving improved detection over the baseline of Radial BNNs. ", "strength_and_weaknesses": "Strengths:\n1) The setting of malware detection through the use of variational inference with BNNs is indeed an important topic of study, with possible immediate implications towards real-world security.\n2) The proposed Radial Spike and Slab distribution for the BNN is indeed interesting, and helps simultaneously model a mixture distribution of discrete and atomless components to address the specific issues that are commonly seen to arise from the feature sets available in the setting of malware detection. The use of the Radial distribution (proposed by Farquhar et al., 2020) for the atomless component also helps circumvent the concentration of measure within an annulus as seen for multivariate gaussians in large dimensional settings.\n\n\n\nWeaknesses:\n1) The scope of the paper as presented is quite limited to the particular application of BNNs for detecting ransomware attacks amongst a set of more general malware attacks. This is quite disparate from that required in practical scenarios, wherein real-world security systems need to raise alerts amongst feature sets corresponding to (rare) occurrences of abnormal activity, amongst the general set of normal functioning parameters.\n2) Furthermore, it is unclear why ransomware attacks alone are studied, given that other rare modes of malware attacks exist with corresponding sparse feature sets and comparable severity in ramifications arising from the security breach, all of which could potentially be addressed using BNNs as well. Thus, it is unclear why the detection of ransomware attacks alone amongst a set of malware attacks with equivalent or greater severity is required.\n3)  The empirical evaluations presented are fairly limited, and could be significantly improved. For example, the paper states that variants of XGBoost produced random results. However, given that it is known that out of the sparse binary feature set of dimension 706, the 298 MITRE ATT&CK features are known to form the most crucial subset. Thus, for several models of lower complexity, this reduced feature set could be used to attain significantly improved baselines, by utilizing this domain-expert specific information to exclude near-irrelevant features.\n4) Furthermore, the improvement in detection performance achieved using the proposed method as compared to baseline approaches such as BNN-Radial are extremely marginal, if any, as observed from Table-1. It is unclear if any improvement in detection performance can be established in a statistically significant manner, without performance statistics reported over multiple training instances of these networks with different random seeds. \n\n\nMinor Typos:\nThe  abstract needs to be modified to follow the ICLR 2023 guidelines for typesetting and formatting.\nThe citation scheme used in the paper results in several instances where author names appear contiguously within sentences without demarcation. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Hqu49CRhhOW", "reviewer": "Reviewer_qQmb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a Bayesian Neural Network approach for ransomware attack detection. \n\nThe authors enumerate the challenges of ransomware detection: temporal, high-dimensional sparse signals with limited records and very imbalanced classes. They then propose using Bayesian Neural Networks is appropriate for these challenges and propose choices for the BNN setup suitable for ransomware attack detection. \n\nAn overview of their method is as follows:\n1. They first convert each incident's low-level events to high-dimensional MITRE ATT&CK features. \n2. To tackle signal sparsity, they make use of a commonly known mixture of priors with Spike and Slab components widely used in Bayesian Variable Selection.\n3. To employ (2) in BNNs, they use a spike and slab prior to the weights. They prove a simple form for the KL divergence of two random variables with Spike and Slab distributions, later used in optimizing the BNN.\n4. They reason Radial distribution is a better choice for the slab component.\n5. To enable backpropagation through the sampling mechanism used in computing the loss, they use the parametrization trick for the slab radial component and substitute the Bernoulli part with Gumble-Softmax.\n\nThen they compare their method with other approaches on one dataset. They also perform ablation studies and report runtime stats. They finally show how the BNN model can be used to give feature importance scores.", "summary_of_the_review": "Overall I think the paper proposes a novel method addressing the challenges of the ransomware detection problem, but the experiments are not convincing enough that the proposals are impactful. It can be outperforming a regular BNN without these modifications on another dataset. Specifically, since the data is imbalanced and the number of positives in this single dataset is low.", "strength_and_weaknesses": "The paper is well written, and although there is a lot of math the content is communicated well.\n\nThe proven theorem is interesting and simplifies calculating KL of weight priors to the approximate posterior distribution.\n\nThe relevant work is very properly cited making it easy to understand the paper.\n\nThe ROC curves show their method outperforms alternatives for ransomware attack detection. Their plot has confidence ranges that gives a better picture of their method's performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject