A DVERSARIAL D ETECTOR FOR D ECISION T REES E NSEMBLES U SING R EPRESENTATION L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Research on adversarial evasion attacks focuses mainly on neural network models.
Among other reasons, this is because of their popularity in certain fields (e.g., computer vision and NLP) and the models’ properties, making it easier to search for adversarial examples with minimal input change.
Decision trees and tree ensembles are still very popular due to their high performance in fields dominated by tabular data and their explainability.
In recent years, several works have defined new adversarial attacks targeting decision trees and tree ensembles.
As a result, several papers were published focusing on robust versions of tree ensembles.
This research aims to create an adversarial detector for attacks on an ensemble of decision trees.
While several previous works have demonstrated the generation of more robust tree ensembles, the process of considering evasion attacks during ensemble generation can affect model performance.
We demonstrate a method to detect adversarial samples without affecting either the target model structure or its original performance.
We showed that by using representation learning based on the structure of the trees, we achieved better detection rates than the state-ofthe-art technique and better than using the original representation of the dataset to train an adversarial detector.
I NTRODUCTION
In recent decades we have seen the introduction of machine learning algorithms in production environments into various fields such as medical imaging (Zhou et al., 2021), autonomous driving (Huang & Chen, 2020) and law enforcement (Vestby & Vestby, 2019).
With the leap in performance of those models and their integration into real-life systems, people began to investigate how to bypass classifiers and defend against those malicious attempts (Dalvi et al., 2004; Lowd & Meek, 2005).
Many papers have addressed examples of adversarial attacks that make small changes that are hard for a human to notice in the inputs of a machine learning model, usually a neural network, so their predictions are wrong.
These can be exploited by a malicious actor and used to bypass a model that might, for example, be responsible for a critical classification task affecting people’s lives.
As a result, various researchers published techniques to detect and defend against adversarial attempts.
Most of the research is focused on adversarial attacks targeting neural network models, among other things, because of the nature of their continuous learning space, which allows a gradient ascent process to maximize the model’s loss function given a specific input.
Thus defenses and detectors mainly target neural network models as well.
Tree-based models continue to be very popular, especially for tabular data tasks (Nielsen, 2016; Shwartz-Ziv & Armon, 2022; Grinsztajn et al., 2022), because they usually demand less data and are more interpretable.
There are fewer studies on adversarial attacks and defenses affecting decision tree models.
Gradient-descent-based methods commonly used in earlier attack models cannot be applied directly to evade decision trees due to the discrete nature of their non-differentiable decisionmaking paths and tree-splitting rules.
Unfortunately, this does not mean that decision trees are unaffected by evasion attacks.
In this work, we present a detection technique for adversarial evasion attacks against tree-based classifiers, focusing on boosting ensembles. Our main contributions are: ( i )
We defined a task that allows us to generate sample representations that rely on the distribution of the dataset in the different routes of a tree ensemble.
( ii ) We designed a pipeline to train and evaluate adversarial detection with reduced possibilities of overfitting or bias.
M OTIVATION
Chen et al.
(2019) proposed a robust decision trees technique against adversarial evasion attacks.
The model training algorithm was changed, as a result of which model itself was changed.
As part of the experiment, the new model’s accuracy was checked and compared to the non-robust model.
Of the eleven datasets tested, seven showed a decrease in accuracy.
Our primary motivation for this work is to create a defense layer for a decision tree ensemble against adversarial attacks. Our defense layer does not affect the model itself, allowing the model owner to decide if they want defense applied to their existing system.
Secondly, production tree-based models use well-known open-source libraries such as XGBoost (Chen & Guestrin, 2016b), CatBoost (Dorogush et al., 2018), and LightGBM (Ke et al., 2017).
These libraries are heavily used, tested, and improved, which is partly why they were chosen in the first place.
Currently, as of writing this paper, the above libraries do not contain an official version that is robust against adversarial attacks.
Therefore, to add adversarial robustness to a model, it is necessary to use a different third-party version of the model or develop a new one.
B ACKGROUND 3. R ELATED W ORK
We can split the field of adversarial learning into three primary sectors: attacking methods, defending methods, and detectors which aim to detect whether or not a sample is adversarial without changing the model itself.
Generating Adversarial Samples.
Early work around generating adversarial samples (Goodfellow et al., 2014; Kurakin et al., 2016) used backpropagation to try and discover which input features we should change to maximize the loss function of a model.
In the face of more recent attacks, a different loss functions was suggested to find an adversarial sample (Papernot et al., 2016b; Carlini & Wagner, 2017; Cheng et al., 2018).
Other works used concepts from geometry and the location of the boundaries between decision spaces to search for a minimal perturbation for creating an adversarial sample (Moosavi-Dezfooli et al., 2016; Yang et al., 2020).
Because decision tree classifiers are not a continuous space model, earlier backpropagation methods will not work in these cases.
Black-box methods, which ignore the internal inner structure of the model, try to approximate the gradients and can generate attacks for decision-trees-based classifiers using multiple queries to find the boundary between different classes in the unknown decision space (Cheng et al., 2018; Chen et al., 2020).
Some relevant white-box techniques focus specifically on the nature of decision trees (Papernot et al., 2016a; Kantchelian et al., 2016; Zhang et al., 2020).
Papernot et al.
(2016a) defined an algorithm to search for a given sample, the closest leaf in the neighborhood of the original leaf, and perturb the features between them.
Kantchelian et al.
(2016) formulated a set of equality and inequality constraints based on the tree structure to generate an optimal adversarial sample for tree ensembles using a mixed-integer linear program.
Model Defenses Against Adversarial Attacks.
Common approach for protecting models is to train a robust model for evasion attacks.
Adversarial training (Goodfellow et al., 2014) is one of these methods, with which one can generate adversarial samples and add them to the training data.
Other suggested solutions use known techniques with other purposes, such as knowledge distillation, as shown in Papernot et al.
(2016c), which used its traits to create a newer model version with smaller gradients to make it more difficult to generate adversarial samples.
Another work is Wang et al.
(2018), which used dropout in prediction time to reduce the dependency on specific neurons in a neural network.
Defending decision tree classifiers combine the structure of the trees with the methods mentioned above.
Adversarial boosting was suggested in Kurakin et al.
with an idea similar to adversarial training, and in each boosting round, adversarial samples are created and added to the next round’s training data.
Other works show how to generate more robust tree models with new optimization formulations while considering evasion attacks and perturbations (Chen et al., 2019;
Andriushchenko & Hein, 2019; Calzavara et al., 2020; Vos & Verwer, 2021; 2022).
Adversarial Detectors.
There are two main approaches when building an adversarial attack detector: using statistics and hypothesis testing to investigate if there is any difference between the regular samples distribution and the adversarial sample’s distribution (Feinman et al., 2017; Grosse et al., 2017; Katzir & Elovici, 2019), and training a machine learning classifier to act as a detector of adversarial attempts (Metzen et al., 2017; Fidel et al., 2020).
Recent work used the combinations of leaves in tree ensemble predictions, called output configurations, to detect abnormal leaves configuration of a sample compared to a reference dataset to detect an adversarial sample using a defined metric called OC-score (Devos et al., 2022).
We will compare our work to Devos et al.
(2022), which is considered the state-of-the-art in detecting adversarial samples on decision trees at the time of writing this paper.
3. P ROBLEM F ORMULATION
For a given classifier C , a given sample x , with set of features F and a label y in which C ( x ) =
y , an adversarial sample x ′ is defined using an adversarial perturbation δ : x ′ = x
+ δ .
An adversarial sample can be generated by a targeted or untargeted attack.
For an untargeted attack we want to find δ that meet with the condition:
[Equation]
Which means that C ( x ) ̸ = C ( x ′ ) where the p -norm of δ will be limited by a value ϵ .
For a targeted attack we need to define a target class t where t ̸ = y
and:
[Equation]
For p ∈ N 1 (All natural numbers without zero) the p -norm is defined:
|| δ || p :=
(Σ Fi =
( δ i ) p ) 1 p )
For p = ∞ - measures the largest absolute difference between two features and is defined: || δ || ∞ =
max x i | x i − x ′ i |
Given a decision tree model T and a sample x , our classification task is to detect whether the sample is normal or is an attempted adversarial attack.
M ETHOD 4. M ETHOD G ENERAL F LOW
Our method consists of 11 main steps:
Split the dataset into four different parts for different purposes: S T to train the tree model, S E to train our basic representation model, S D− train to train our adversarial detector and S D− test to evaluate our adversarial detector.
Train a tree model.
Generate a triplets dataset that will be used to initialize the new representations.
This is explained more fully in Subsection 4.2. 4.
Train our basic embedding model E .
This is explained more fully in Subsection 4.3. 5.
Split S D− train and S D− test into two parts.
Generate adversarial samples using an attack method A . 7.
Generate a new triplet dataset for each of the new sub-datasets.
This is explained more fully in Subsection 4.4. 8.
Optimize the representations of the new sub-datasets to our new embeddings using E , more details in Subsection 4.4.
Concatenate the new representation of every set to the original ones.
Train our adversarial detector.
This is explained more fully in Subsection 4.5.
Evaluate our adversarial detector.
This is explained more fully in Section 5.
A detailed visualization sketch shown in Figure 28 in Appendix H, together with a further explanation about the dataset splitting.
4. D ATASET R EPRESENTATION
At the heart of our method is the idea that we want to extract a new representation of a dataset based on the structure of a target tree ensemble model to understand the behavior of normal samples and detect adversarial samples.
To extract a meaningful representation, we took inspiration from an embedding process.
For each sample, we assign a vector of random numbers with d dimensions, called latent features, that will be optimized using a gradient descent process using a simple feedforward neural network based on the structure and traits of the tree.
For a given dataset S and a trees model T we define a new dataset:
[Equation]
Where N TS is the set of all internal nodes of T reached by at least one sample from S , n is a single node, and s is a single sample.
Each triplet in R TS contains two samples and a node.
Both samples reach the internal node n k in T .
We define the supervised task below:
[Equation]
the
[Equation] child [Equation] To collect this new dataset, we take our original dataset and traverse with each of the samples through the different routes in the trees in the ensemble and save the routes aside.
During the process, we document which sample passes in which node and create the mapping:
[Equation]
Which returns for a given node all the samples which reached it during the above traverse.
Then to generate R TS we can use Algorithm 1.
Algorithm 1 Generating triplets dataset to train the basic embedding model to fine-tune future representations
Algorithm 1 returns a list of triplets constructed from 2 samples and one node and a list of labels based on the above logic.
For a chosen N, which is the final size we choose for the dataset R TS , we generate the triplets described above.
After sampling for a node and relevant samples, in line 5, we check if both samples agree on the condition of the feature threshold in n k , in which case they move to the same child on n k , and the label of the new sample is set to 1.
Otherwise, it is set to 0.
4. E MBEDDING M ODEL
To generate our The basic embedding model is trained using R TS E .
We can see in Figure 1-(a) a general sketch of the architecture we used.
During the embedding model-training phase, we use two embedding matrices, one for the samples and one for the nodes, and we optimize the representation with a gradient descent process based on the task defined earlier.
Each sample vector is initiated with a random vector of dimension d s and each node with a random vector of dimension d n .
The vectors of each sample and the one representing the node are concatenated and passed through a single feed-forward layer with a ReLU activation function and finish in a feed-forward layer with a sigmoid activation function used to evaluate the loss value.
The model is trained with a binary cross-entropy loss function and an Adam optimizer. Then, the model weights will be saved to optimize the new sample’s representations.
4. N EW S AMPLES E MBEDDING
When we want to extract the representation of new samples set S new , we first create a new dataset:
[Equation]
This means the triplet in this new dataset is constructed from one sample from the samples used to train the embedding model ( S E ) and another sample from the new sample set whose representation we want to optimize.
The labels are set in the same manner as we described before, based on the fact that the two samples align with the feature threshold in the relevant node.
We use algorithmto construct this dataset.
Algorithm input train the embedding model M
E nodes samples the new dataset
N do 2:
7: label 8: else 9: label 10: end if 11: end for Similarly to Algorithm 1, in Algorithm 2 we choose N, which is the size we choose for the dataset
R TS E , S new .
In line 3, we take random samples from M TS ( n k ) , a sample that was used to train the basic representations that reached n k .
In line 4, we sample random samples from M TS new ( n k ) which is a sample from the new dataset whose representation we want to optimize.
Then, in line 6, we check if both samples agree on the condition of the feature threshold in n k , in which case they move to the same child of n k , and the label of the new sample is set to 1.
Otherwise, it is set to 0.
Algorithm 2 returns a list of triplets constructed from two samples and one node and a list of labels.
Then the weights of the trained embedding model are loaded to the same architecture with all weights frozen (both layers’ weights and biases, embedding matrix of the S E samples and embedding matrix of the nodes), and a new unfrozen embedding matrix for the new samples is initialized as described in Figure 1-(b).
The representations are then optimized with a gradient descent process while using R TS E , S new .
[Figure]
4. A DVERSARIAL C LASSIFIER
As suggested by Metzen et al.
(2017), we trained an adversarial samples detector based on a classifier to try and classify whether a specific sample is a normal sample that came from the original distribution of the input samples of the original dataset or an adversarial sample.
We extract four new sub-datasets from the datasets used for training and evaluating the detector.
We extract two from each one, a set that stayed as the normal samples, and using the other set, we generate adversarial samples and then throw the original samples away.
Afterward, we extracted each dataset’s new representations using the process described in Subsection 4.4.
We used an XGBoost as the classifier of the detector.
As an input to the classifier, we concatenated our new extracted representations to the original features and used them together as the final features set.
E VALUATION 5. E XPERIMENTAL S ETUP
In our experiments, we tested the performance of our method of creating new representations by training an adversarial-evasion-attack detector on 18 datasets, described in Subsection 5.2, with five different attacks, described in Subsection 5.3.
We compared our method to OC-score (Devos et al., 2022), considered the state-of-the-art for that task, and to detector classifiers that we trained on the original representations of the datasets.
We tested our method against two tree-based ensembles:
XGBoost and RandomForest - both implemented by Chen & Guestrin (2016a).
The full results tables for XGBoost are given in Appendix B and for RandomForest in Appendix C. To train our adversarial classifier, we used a second XGBoost model; our positive samples are the adversarial samples, and the negative samples are the normal ones.
To train and test our detector, we used no more than 100 samples for black-box attacks and 1000 for the white-box attack from S D− train and
S D− test .
While generating adversarial samples, we only attacked using samples originally classified correctly by the target model.
For the embedding dimensions we chose d s = 250 and d n = 100 .
We used ROC-AUC and PR-AUC as our metrics. For multi-class datasets, each metric is calculated for each of the different labels in a 1-vs-all manner, and eventually, an average is calculated.
Our code is online in github 1 .
5. E VALUATED D ATASETS
In our experiments, we tested with 18 classification datasets, some of them binary and others multiclass.
The datasets are described in Table 1.
The top section of Table 1 describes datasets that were used in Chen et al.
(2019) as benchmarks for their experiments, already preprocessed and split into training and test sets, and published publicly in a very convenient way on github 2 .
The bottom section of Table 1 describes more datasets that we added, mainly small binary-classification datasets, to have more variety.
The datasets vary by the number of samples, features
[Table]
and classes.
[Table]
References
[Table]
5. E VALUATED A DVERSARIAL A TTACKS
We evaluated our method with untargeted attacks, with 2 different norms - L 2 and L ∞ .
For the attack methods, we used four black-box attacks which are relevant to tree-based models: Sign-Opt attack (Cheng et al., 2019) , OPT attack (Cheng et al., 2018) , HopSkipJump attack (Chen et al., 2020) and Cube attack (Andriushchenko & Hein, 2019).
We used one white-box attack specifically for trees-based models: Leaf-Tuple attack (Zhang et al., 2020).
To execute our attacks we used implementation published by Zhang et al.
(2020) on github 3 .
5. E XPERIMENTS R ESULTS
We applied our method and calculated ROC-AUC and PR-AUC for each combination of the norm, attack method, dataset, and model algorithm.
We calculated the difference between our method’s performance, the OC-score method’s performance, and the performance of a detector trained on the original representation.
All of the distributions of the differences are shown in Appendix A as boxplots.
Due to space constraints, we only show here plots comparing our method to OC-score for L 2 norm, in Figure 2, for experiments targeting XGBoost tree ensembles and in Figure 3, for experiments that targeting RandomForest tree ensembles.
Each row is a different attack method, each white point is an experiment on a specific dataset, and the red vertical line is a total mean of all the experiments together.
The figures show the spread of the differences in ROC-AUC and PR-AUC for each norm and attack combination.
The boxplot shows us the different quartiles and the median.
As we can see in the figures, the total mean and the median of each section is positive, which means that our metrics yielded better results for the new representation in most of our experiments.
The full raw metrics for each one of the experiments are shown in tables in Appendix B for XGBoost and in appendix C for RandomForest.
[Figure]
[Figure]
Of our 338 experiments, our method had the best performance in 107 of them and was tied with one of the other methods as the best in 134 other experiments, which means our method was successful in 71.73% of the experiments.
We used the Friedman test on the ROC-AUC metric to validate the statistical significance of differences between the evaluated methods and datasets (Dem ˇ sar, 2006).
The above test is a non-parametric test that does not assume anything about the results distributions and is used when dealing with multiple methods and multiple datasets.
The null hypothesis that the 3 methods perform the same was rejected with F F (338 , 3)
= 288 with p < 0 .
As a second step, we used Nemneyi post-hoc test, which is often used as a second step if it is possible to reject the null hypothesis with a Friedman test and test for superiority between the different methods, and concluded that our new representation method outperforms the OC-score method and the usage of a machine learning detector based on the original representations with p < 0 .
D ISCUSSION
As we showed in our experiments, using representation learning to learn a better version of the dataset given a trained model helps to extract better performance when trying to protect our model against adversarial evasion attacks.
We showed this on different datasets, different tree-based ensembles, and with different attack methods.
An important area for discussion is adversarial attacks on tabular data, which are not as intuitive as adversarial attacks on more continuous inputs such as images or audio.
These usually clip the data to a valid range of values, which is not straightforward for tabular data.
This issue is not yet heavily researched but was already approached and discussed by Calzavara et al.
and Vos & Verwer (2021), where for each dataset, they defined a set of rewriting rules and possible budget with which to change the features.
We chose not to take up that challenge in this work, but for full context, we added in Appendix D statistics regarding our generated adversarial samples.
While analyzing our results, we noticed an interesting outcome of our new representationsgeneration process.
We used UMAP (McInnes et al., 2018) to extract 2-dimensional versions of the original representations and our new ones.
As we can see in Figure 4 on the left side, the reduction of the original representations, there the adversarial samples are scattered between the normal
C ONCLUSIONS AND F UTURE W ORK
In this work we presented a method to train an adversarial samples detector based on new representations of a dataset based on the target model trained on it.
We showed that using our new representation shows overall improvement compared to the current state-of-the-art detection of adversarial samples on tree models without changing the original model internals and compared to using the original samples representations to train a detector classifier
As for future work our new method should also be tested on robust versions of tree ensembles to see if it affects the results.
Another direction is to create a unified detector for all attacks together.
Our detector method was based on generating adversarial samples for each attack method and training a separate model.
However, the normal data behave the same regardless of the attack method and a new unified approach should be researched.
A A PPENDIX A - F ULL M ETRICS D IFFERENCES F IGURES
Here are all the different figures comparing our method to OC-Score and a detector based on the original representations of the dataset.
For the experiments compared to the OC-score (Figures 5, 6, 9, and 10) the total mean and the median of each section is positive in all of them, which means that our metrics yielded better results for the new representation in most of our experiments.
When looking at figures that compare our new method to the original representations (Figures 7, 8, 11, and 12) we can see the ROC-AUC and PR-AUC overall mean of the experiments is positive beside the experiments targeted XGBoost with L ∞ norm which the overall mean is negative.
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
B A PPENDIX B - XGB OOST F ULL E XPERIMENTS R ESULTS
List of tables Rows that fill C A PPENDIX C - R ANDOM F OREST F ULL E XPERIMENTS R ESULTS List of tables
Rows that
[Table]
[Table]
[Table] [Table] [Table] 710 Fashion-MNIST 0.
0.868 ijcnn 1. 0.893 MNIST 1.
[Table]
[Table] [Table] [Table] 945 MNIST2-
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table] [Table] sensorless [Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] 984 electricity [Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] 852 drybean
[Table]
[Table]
[Table]
[Table] [Table] [Table] 978 adult
[Table]
[Table] [Table] [Table]
MNIST
[Table] [Table]
MNIST2- [Table] [Table] Sensorless
[Table]
electricity
[Table] [Table]
0.809 Fashion-MNIST 1.
[Table]
[Table] [Table] 943 ijcnn 1. [Table]
[Table] [Table] [Table] 958 MNIST 1.
[Table]
[Table] [Table] [Table] 967 MNIST2-
[Table]
[Table] [Table] [Table] sensorless
[Table] [Table]
webspam [Table] [Table] [Table]
[Table] [Table] electricity [Table]
[Table]
[Table]
[Table] [Table] drybean [Table]
[Table] [Table] [Table] 991 adult
[Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
.870 MNIST 0.
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] MNIST2-
[Table] [Table] [Table] [Table]
[Table] [Table] [Table]
sensorless [Table] [Table] [Table]
984 webspam [Table] [Table] [Table]
electricity
[Table] [Table]
drybean
[Table]
[Table] [Table] [Table]
adult
[Table] [Table] [Table]
[Table] [Table] [Table] [Table]
472 banknote
[Table]
[Table] [Table] [Table] [Table]
D A PPENDIX D - A DVERSARIAL S AMPLES I NFORMATION
In the following tables, we calculated several statistics about the perturbations generated in our various experiments split by target model type and attack method.
Interesting behaviors we noticed:
• We can see that for SignOPT and OPT attacks in all of the experiments, a perturbation was applied to all of the features (max features changed is equal to the mean, which is also equal to the number of features for each dataset).
• The mean perturbation size for datasets adult and drybean seem very high for attack methods HopSkipJumpAttack, Cube, and LeafTuple.
D. XGB OOST PERTURBATION STATISTICS
[Figure]
[Table]
[Figure]
[Table]
electricity
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]
[Table] [Table]
D. R ANDOM F OREST P ERTURBATION S TATISTICS
[Figure]
[Table] [Table]
[Table] [Table]
[Table]
statistics
E A PPENDIX E - D ETECTOR A BLATION S TUDY
We wanted to investigate how three aspects of our detector affect our metrics results:
The dimensions of our new representations (for the samples and for the nodes).
The machine learning classifier that was chosen as the final adversarial detector.
Key hyperparameters of the final adversarial detector.
To do our tests, we took two different experiments and showed their results:
• XGBoost target with codrna dataset and OPT attack method.
• RandomForest target with sensorless dataset and HopSkipJumpAttack attack method.
Each one of the above with the two different norms.
E. D IFFERENT C LASSIFIER & H YPERPARAMETERS
We checked three different classifiers as our models for our adversarial evasion attack detector:
XGBoost, RandomForest, and K-nearest neighbors (KNN).
For the tree ensemble classifiers, we checked how much the number of estimators and their depth impacts the performance of the detector.
For the KNN classifier, we tested the results with different values of K.
E.1. XGB OOST D ETECTOR H YPERPARAMETERS
As we can see from the results in Figures 13, 14, 15, and 16 that in general the number of estimators has a large impact on both ROC-AUC and on PR-AUC until a certain point which is around 50 estimators and above usually there is improvements, but relatively smaller.
In most cases, the maximum depth of the trees has a very light impact on the results when the number of estimators is 50 or more.
[Figure] [Figure]
[Figure] [Figure]
E.1. R ANDOM F OREST
As we can see from the results in Figures 17, 18, and 20 that in general the number of estimators and the maximum depth of the trees has a large impact on both ROC-AUC and on PR-AUC.
In
[Figure] [Figure]
[Figure] [Figure]
E.1. KNN
We tested KNN performance with different Ks to investigate how much it changed the detector performance metrics and compared it to the results we got in our original experiments.
We used KNN based on Facebook AI fast similarity search (Johnson et al., 2019) implemented in DESlib (Cruz et al., 2020) due to time consideration for cases where the feature count is relatively high.
In Figures 21 and 22 we can see a comparison of ROC-AUC and PR-AUC of KNN classifier with different values of k neighbors.
Additionally, added the performance of KNN on the original representation of the dataset and the metric values of the original experiments done with our new method with
XGBoost classifier as the detector.
In all of the experiments, we can see the relative stability of the values with different K values.
Additionally, the metric values with the XGBoost classifier are higher.
[Figure] [Figure]
E. E MBEDDINGS D IMENSIONS
Here we tested how much the embedding dimension might affect the performance of our detector.
We have two different embeddings - for the samples and for the nodes.
In Figures 23, 24, and 26 we can see heatmaps for compare the performance with different embedding size.
The general pattern from the heatmaps in Figures 23 and 24 is that as our sample hidden size is larger and the node hidden size is lower the results are better.
In Figures 25 and 26 we don’t see that pattern.
[Figure] [Figure]
[Figure] [Figure]
F A PPENDIX F - T REE E NSEMBLE H YPERPARAMETERS
We used tree ensembles for two tasks: the target model to extract adversarial samples and the base for the adversarial detector.
For the target model, we used 40 estimators with a max depth offor both XGBoost and Random forest.
For the rest of the hyperparameters, we used the default values set in the XGBoost library package version 1.6.1.
To train the adversarial detector, we used the default parameters for XGBoost and RandomForest set in the XGBoost library package version 1.6.1.
G A PPENDIX G - T REE MODEL ’ S PERFORMANCE USING LESS DATA
A potential issue in our method is that we reduce the available data used to train the model; thus, the model performance might be affected.
As part of our method, we split each dataset we experimented on into several subsets for different roles described in Subsection 4.1 and Appendix H. We took the original representation of the datasets and compared the trained models from our experiments to a new version of the model trained on all data besides the test set.
Below, we can see in Figure 27 a comparison between the ROC-AUC of the model trained on our split version of the dataset (Y-axis) to the same model trained on more data (X-axis) on the test sets.
As we can see, there is a decrease in performance most of the time, but most of the degradation is minor.
For a complete description of the numbers in every experiment, refer to Tables 22, 23, 24, 25, 26, 27, 28, 29, 30, 31.
[Figure]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
H A PPENDIX H - M ETHOD F LOW - FURTHER EXPLANATIONS
In Figure 28, you can see a visualization of the points in Subsection 4.1, which we added here below as well, for readers’ convenience:
Split the dataset into four different parts for different purposes: S T to train the tree model, S E to train our basic representation model, S D− train to train our adversarial detector and S D− test to evaluate our adversarial detector.
Train a tree model.
Generate a triplets dataset that will be used to initialize the new representations.
This is explained more fully in Subsection 4.2.
Train our basic embedding model E .
This is explained more fully in Subsection 4.3. 5.
Split S D− train and S D− test into two parts.
Generate adversarial samples using an attack method A . 7.
Generate a new triplet dataset for each of the new sub-datasets.
This is explained more fully in Subsection 4.4. 8.
Optimize the representations of the new sub-datasets to our new embeddings using E , more details in Subsection 4.4. 9.
Concatenate the new representation of every set to the original ones.
Train our adversarial detector.
This is explained more fully in Subsection 4.5.
Evaluate our adversarial detector.
This is explained more fully in Section 5.
Our dataset is split into four main sub-datasets:
S T - a dataset that will be used to train the target tree model.
S E - a dataset that will be used to optimize a base set of embeddings that represent the general distribution of the original dataset.
This base set of embeddings used to optimize new samples representations.
S D− train - a dataset used to train our adversarial detector, it will be split into two sub-subdatasets to allow us to collect normal samples and adversarial samples.
S D− test - a dataset used to test our adversarial detector, it will be split into two sub-subdatasets to allow us to collect normal samples and adversarial samples.
We chose this split method to reduce bias and overfitting when optimizing the representations and training our adversarial detector.
Each of the above datasets is sampled randomly from the original dataset without replacements, meaning one sample will be only in one of the above sub-datasets.
As a result, the amount of data we use for each one of the steps is reduced, and we added information about it in Appendix G to show how much it affects the classification performance of the original tasks.