SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This optimization problem can be seen as the relaxation of the original adversarial training objective. This relaxation upper bounds the original objective because we robustly optimize only over  $Q$  and not jointly over  $Q$  and  $\theta$ . If we disregard the additional inner maximization, it is a classical bilevel optimization problem. Such problem can be solved using alternating gradient descent (Ghadimi & Wang, 2018). To optimize this objective for neural networks on large tabular datasets, we propose to use stochastic gradient descent. We alternate between  $Q$ -steps for the inner minimum and  $\theta$ -steps for the outer minimum. At each step, we run projected gradient descent for  $PGD$ -steps using the relaxation described in Eq. 5. We detail this approach in Algorithm 2.
Step 2: embedding merging algorithm. We observed that directly transferring the embeddings  $Q$  has little effect in terms of improving the robustness of decision trees (Appendix C.1). This is expected since even if two embeddings are very close to each other, a decision tree can still generate a split between them. To address this issue and provide decision trees with information about the relationships between embeddings, we propose a merging algorithm outlined in Algorithm 3. The main idea of the algorithm is to merge embeddings in  $Q$  that are very close to each other and therefore pass the information about distances between them to the decision tree.
Step 3: standard training of trees using universal robust embeddings. As the final step, we use standard tree training techniques with the merged embeddings  $Q'$ . This approach allows us to avoid the need for custom algorithms to solve the relaxed problem described in Eq. 5. Instead, we can
Algorithm 3 Embeddings merging algorithm
Input: Embeddings  $Q$ , percentile  $p$
Output: Embeddings  $Q^{\prime}$
1)  $D\coloneqq \mathbb{I}$
2) For each  $q_j, q_k \in Q_i$  compute  $d_{jk} = ||q_j - q_k||_2$  and put it to  $D$ .
3) Sort  $D$
4) For a given percentile, compute threshold  $t$ , s.t. all  $d > t$  belong to the percentile
5) Cluster  $Q$  using  $t$  as a maximum distance between two points in one cluster
6) In each cluster compute average embedding vector and put it into  $Q'$ .
leverage highly optimized libraries such as XGBoost or LightGBM (Chen & Guestrin, 2016; Ke et al., 2017b). The motivation behind this multi-step approach is to combine the benefits of gradient-based optimization for generating universal robust embeddings with the accuracy of tree-based models specifically designed for tabular data with categorical features.
In this section we evaluate the performance of our methods.
Models. We use three 'classic' classifiers widely used for tabular data tasks: RandomForest (Liaw & Wiener, 2002), LGBM (Ke et al., 2017a), and Gradient Boosted Stumps. Both for adversarial training and for robust embeddings used in this section, we use TabNetArik & Pfister (2019) which is a popular choice for tabular data. Additionally, we also show the same trend with FT-Transformer in Appendix C. The model hyperparameters are listed in Appendix B.
Attack. For evaluation, we use the graph-search based attack described by Kireev et al. (2022). This attack is model-agnostic and can generate adversarial examples for both discrete and continuous data, and is thus ideal for our tabular data task.
Comparison to previous work. In our evaluation, we compare our method with two previous proposals: the method of Wang et al. (2020), where the authors propose a verified decision trees robustness algorithm for  $l_{1}$  bounded perturbations; and the method of Kireev et al. (2022), which considers financial costs of the adversary, but using a different cost model than us and weaker relaxation.
5.1 Tabular data robustness evaluation benchmark
There is no consistent way to evaluate adversarial robustness for tabular data in the literature. Different works use different datasets, usually without providing a justification neither for the incentive of the adversary nor for the perturbation set. For example, Wang et al. (2020) and Chen et al. (2021) use the breast cancer dataset, where participants would not have incentives to game the classifier, as it would reflect poorly on their health. Even if they had an incentive, it is hard to find plausible transformation methods as all features relate to breast images taken by doctors. To solve this issue, we build our own benchmark. We select datasets according to the following criteria:
- Data include both numerical and categorical features, with a financial interpretation so that it is possible to model the adversarial manipulations and their cost.
- Tasks on the datasets have a financial meaning, and thus evaluating robustness requires a cost-aware adversary.
- Size is large enough to accurately represent the underlying distribution of the data and avoid overfitting. Besides that, it should enable the training of complex models such as neural networks.
It is worth mentioning that the financial requirement and the degree of sensitivity of data are highly correlated. This kind of data is usually not public and even hardly accessible due to privacy reasons. Moreover, the datasets are often imbalanced. Some of these issues can be solved during preprocessing stage (e.g., by balancing the dataset), but privacy requirements imply that benchmarks must mostly rely on anonymized or synthetic versions of a dataset.
Figure 1: Clean and robust model accuracy for cost-aware adversarial training. Each point represents a TabNet model trained with an assumption of an adversary's budget of  $\varepsilon$  in attacked by the adversary with that budget. Our method outperforms Kireev et al. (2022) in all setups.
Datasets. We select three publicly-available datasets that fit the criteria above. All three datasets are related to real-world financial problems where robustness can be crucially important. For each dataset, we select adversarial capabilities for which we can outline a plausible modification methodology, and we can assign a plausible cost for this transformation.
- IEEECIS. The IEEECIS fraud detection dataset (Kaggle, 2019) contains information about around 600K financial transactions. The task is to predict whether a transaction is fraudulent or benign.
- BAF. The Bank Account Fraud dataset was proposed in NeurIPS 2022 by Jesus et al. (2022) to evaluate different properties of ML algorithms for tabular data. The task is to predict if a given credit application is a fraud. It contains 1M entries for credit approval applications with different categorical and numerical features related to fraud detection.
- Credit. The credit card transaction dataset (Altman, 2021) contains around 20M simulated card transactions, mainly describing purchases of goods. The authors simulate a "virtual world" with a "virtual population" and claim that the resulting distribution is close to a real banking private dataset, which they cannot distribute.
All datasets were balanced for our experiments. The features for the adversarial perturbations along with the corresponding cost model are described in Appendix A.
Cost-aware Adversarial Training. We follow a standard evaluation procedure: we first perform adversarial training of the TabNet models using different cost bounds  $\varepsilon$ , representing different financial constraints for the adversary. Then, we evaluate these models' robustness using the attack by Kireev et al. (2022) configured for the same cost adversarial constraint as used during the training. The details about our training hyperparameters are listed in the Appendix B.
We show the resulting clean and robust performance in Figure 1. Our method outperforms the baseline in both metrics. The better performance can be attributed to a better representation of the threat model. For example, let us consider the email address feature in IEEECIS. The cost to change this feature varies from  $0.12$  (e.g., hotmail.com) to  $\infty$  (for emails that are no longer available). It
Table 1: Universal robust embedding evaluation. We report clean and robust accuracy (in percentage) for Light Gradient Boosting (LGBM), Random Forest (RF), Gradient Boosted Stumps (GBS), and CatBoost (CB). We indicate the robustness technique applied as a suffix: -R for robust embeddings, -W for the training proposed by Wang et al. (2020), and -C for method in Chen et al. (2019). Models fed with robust embeddings have higher robustness and outperform both clean and  $l_{1}$ -trained models (Wang et al., 2020).
Dataset	RF	RF-R	RF-C	LGBM	LGBM-R	GBS	GBS-R	GBS-W	CB	CB-R
IEEECIS
Clean	83.6	81.0	69.0	81.2	79.3	66.9	66.3	52.4	76.5	76.1
Robust	48.0	81.0	69.0	53.9	78.5	44.7	66.3	11.1	51.1	72.0
BAF
Clean	72.3	65.8	61.3	74.1	68.1	74.1	68.1	64.8	74.4	67.2
Robust	42.8	65.8	61.3	49.2	67.5	46.8	67.7	33.5	48.2	67.2
Credit
Clean	78.0	73.4	-	83.1	79.6	82.1	80.6	61.7	83.3	79.7
Robust	55.2	66.7	-	69.9	72.5	70.9	71.4	61.3	71.7	71.9
is unlikely that an adversary uses an expensive email (sometimes even impossible as in the case of unavailable domains), and therefore such domains are useful for classification as they indicate non-fraudulent transactions. On the other hand, an adversary can easily buy a gmail.com address even when their budget  $\varepsilon$  is a few dollars. Our method captures this difference. We see how there is a steep decrease in accuracy when the cost of the modification is low (under 1$) and thus the training needs to account for likely adversarial actions. Then, it enters a plateau when the cost of emails increases and no changes are possible given the adversary's budget. When the budget  $\varepsilon$  grows enough to buy more expensive emails, the accuracy decreases again. We also observe that the gain in clean accuracy is higher than for robust accuracy. This effect is due to a better representation of the underlying categorical data distribution.
Universal Robust Embeddings. In order to evaluate our Bilevel alternating minimization framework, we run Algorithm 2 on TabNet to produce robust embeddings, and we merge these embeddings using Algorithm 3. After that, we attack the model using budgets  $\varepsilon$  set to  $10$  for IEEECIS,  $1$  for BAF, and  $100$  for Credit. These budgets enable the adversary to perform most of the possible transformations. We compare our methods on gradient-boosted stumps because training gradient boosted decision-trees with the method of Wang et al. (2020) is computationally infeasible for an adequate number of estimators and high dimensional data.
We show the results in Table 1. In almost all setups, our method yields a significant increase in robustness. For example, for IEEECIS we increase the robust accuracy of LGBM by  $24\%$ , obtaining a model that combines both high clean and robust accuracy, and that outperforms TabNet. Our method outperforms Wang et al. (2020) both for clean and robust accuracy. These results confirm that training based on only  $l_{1}$  distance is not sufficient for tabular data.
We also compare the performance of the different methods with respect to the time spent on training and merging (see Table 2). We see that our method is considerably faster than previous work. The improvement is tremendous for IEEECIS and Credit where the dimensionality of the input is more than 200 and the number of estimators are 80 and 100 respectively. In the Appendix C, we also discuss how robust embeddings can be applied to a neural network of the same type.
Dataset	Training RE	GBS-RE	GBS-W
IEEECIS	9.5	0.06	233.3
BAF	3.17	0.02	4.97
Credit	3.24	0.07	174.7
Table 2: Computation time. Time is measured in minutes. The total time of training RE and training GBS-RE is less than Wang et al. (2020)'s training
In this work, we propose methods to improve the robustness of models trained on categorical data both for deep networks and tree-based models. We construct a benchmark with datasets that enable
robustness evaluation considering the financial constraints of a real-world adversary. Using this benchmark, we empirically show that our methods outperform previous work while providing a significant gain in efficiency.
Limitations and Future Work. In our work, we focused on development of a method to improve the robustness of a machine learning model, having as small degradation in performance as possible. However, there are applications where even a small accuracy drop would incur more financial losses than potential adversarial behaviour. These setups can be still not appropriate for our framework. Quantifying these trade-off can be a promising direction for future work, and one way of doing it would be to follow a utility-based approach introduced in Kireev et al. (2022). Besides that, we do not cover the extreme cases where the dataset is either too small and causes overfitting or too large and makes adversarial training more expensive. Addressing these extreme cases can be also considered as a research direction. Finally, we leave out of the scope potential problems in the estimation of adversarial capability, e.g., if the cost model which we assume is wrong and how it can affect both the robustness and utility of our system.
M.A. was supported by the Google Fellowship and Open Phil AI Fellowship.
Altman, E. Synthesizing credit card transactions. In Proceedings of the Second ACM International Conference on AI in Finance, pp. 1-9, 2021.
Andriushchenko, M. and Hein, M. Provably robust boosted decision stumps and trees against adversarial attacks. Advances in Neural Information Processing Systems, 32, 2019.
Apruzzese, G., Anderson, H. S., Dambra, S., Freeman, D., Pierazzi, F., and Roundy, K. A. "real attackers don't compute gradients": Bridging the gap between adversarial ml research and practice. arXiv preprint arXiv:2212.14315, 2022.
Arik, S. Ö. and Pfister, T. Tabnet: Attentive interpretable tabular learning. arxiv. arXiv preprint arXiv:2004.13912, 2019.
Bakiskan, C., Cekic, M., and Madhow, U. Early layers are more important for adversarial robustness. In ICLR 2022 Workshop on New Frontiers in Adversarial Machine Learning, 2022.
Boyle, J. P. and Dykstra, R. L. A method for finding projections onto the intersection of convex sets in hilbert spaces. In Dykstra, R., Robertson, T., and Wright, F. T. (eds.), Advances in Order Restricted Statistical Inference, pp. 28-47, New York, NY, 1986. Springer New York. ISBN 978-1-4613-9940-7.
Calzavara, S., Lucchese, C., Tolomei, G., Abebe, S. A., and Orlando, S. Treant: training evasion-aware decision trees. Data Mining and Knowledge Discovery, 34:1390-1420, 2020.
Chen, H., Zhang, H., Boning, D., and Hsieh, C.-J. Robust decision trees against adversarial examples. In International Conference on Machine Learning, pp. 1122-1131. PMLR, 2019.
Chen, T. and Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Chen, Y., Wang, S., Jiang, W., Cidon, A., and Jana, S. {Cost-Aware} robust tree ensembles for security applications. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2291-2308, 2021.
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., and Hein, M. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670, 2020.
Shi, S., Tse, R., Luo, W., D'Addona, S., and Pau, G. Machine learning-driven credit risk: a systemic review. Neural Computing and Applications, 34, 07 2022. doi: 10.1007/s00521-022-07472-2.
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
Wang, Y., Zhang, H., Chen, H., Boning, D., and Hsieh, C.-J. On lp-norm robustness of ensemble decision stumps and trees. In International Conference on Machine Learning, pp. 10104-10114. PMLR, 2020.
Yang, Y. and Zhai, P. Click-through rate prediction in online advertising: A literature review. Information Processing & Management, 59(2):102853, 2022. ISSN 0306-4573. doi: https://doi.org/10.1016/j.ipm.2021.102853. URL https://www.sciencedirect.com/science/article/pii/S0306457321003241.
Yang, Y., Wang, X., and He, K. Robust textual embedding against word-level adversarial attacks. arXiv preprint arXiv:2202.13817, 2022.
Zhang, D., Zhang, T., Lu, Y., Zhu, Z., and Dong, B. You only propagate once: Accelerating adversarial training via maximal principle. Advances in Neural Information Processing Systems, 32, 2019.
Zhang, X. and Evans, D. Cost-sensitive robustness against adversarial examples. *ICLR*, 2019.
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., and He, Q. A comprehensive survey on transfer learning. arxiv e-prints. arXiv preprint arXiv:1911.02685, 2019.
A Cost modelling and data preprocessing
For all datasets used in the evaluation, we perform threat modelling based on the information about possible perturbations. In this section, we describe how we assign cost matrices to modifications of the features in each dataset and the common methods which we use for data preprocessing. Note that in this section for some features we mostly provide the methods and examples, and all the exact values are listed in the source code (exp/loaders.py). It is also worth mentioning that the exact prices might fluctuate, and our goal is to provide an estimate to demonstrate the capabilities of our methods.
All the datasets were balanced for training using random undersampling. Numerical features were binned in 10 bins since a further increase in the number of bins gave no additional improvement. In order to avoid numerical issues, we set the minimal possible cost to  $0.1$ , and the maximal possible cost to  $\$10000$  (100 times greater than the largest  $\varepsilon$  used in the evaluation). For the attack, we assume that the adversary is attacking only for one target class (e.g. their aim is to get a transaction marked as "non-fraud").
A.2 IEEEECIS
In evaluation on IEEECIS we use the following features for our perturbation set:
- P_emaildomain - Email domain which can be used by an adversary. We assume that an adversary can either buy a new email or create it himself. In the first case we take the existing prices from dedicated online shops (e.g. adversary can buy *@yahoo.com email for 0.12$ at buyaccs.com). In the second case, we estimate minimal expenditures to register one. For example, some emails require buying services of an internet provider (*@earthlink.net), in this case, we estimate the minimal price of such service and count it as an adversarial cost (50$ for *@earthlink.net).
DeviceType - Device type represents the operating system installed on the adversary's device. This feature can be easily manipulated since the adversary controls the device and can change device information sent to the system or use an emulator. We ascribe minimal cost (0.1$) to this type of perturbation.
- Card_type - Card type is a brand of a credit or debit card used by the adversary. For this feature, we assume that the adversary needs to buy a stolen credit or debit card. We estimate the cost of such purchase to be  $20$  for Visa or MasterCard and  $25$  for less common Discovery or American Express. This estimation is based on stolen card prices in the darknet online marketplaces.
A.3 BAF
In the evaluation on the BAF dataset we use the following features for our perturbation set:
- income - Income of a credit applicant. This feature can be perturbed by buying a fake paystub, which costs around 10\$.
- zip_count_4w - Number of applicants with the same zip code. The adversary can change their address by paving just around 1$ (In US).
- Device-related features - Here we consider features which are related to the device/OS used by an adversary, and which can be manipulated the same way as DeviceType in IEEECIS foreign_request, source, keep_alive_session, device_os belongs to this feature type. The cost model for these features is the same as for DeviceType.
- email_is_free - This feature represents if the adversary uses a paid email or a free one. Since going from paid to free is cheap, we represent this transition with the smallest price of  $0.1$ . For going from a free email to the paid one we use the cheapest paid email from IEEECIS and set this cost to  $10$ .
- phone_home_valid, phone_MOBILE_valid - Validity of the provided phone number. The logic is the same as with the previous feature. It costs almost nothing (0.1$) to change from valid to non-valid, and 10$ vice versa.
- device_distinct Emails_8w - Number of distinct email addresses used by the adversary. This feature can only be increased since the adversary presumably cannot delete its records from the system. The cost for an increase is small (0.1$), decreasing is impossible.
A.4 Credit
Finally, for the Credit dataset we use:
- Merchant City - Location where the purchase was made. Without any prior knowledge, we assume that this feature can only be manipulated physically by moving to a different location. Therefore, in this case, the cost of change is the cost of a transport ticket to this location. As the lower bound, we use a transport price of  $0.1\mathrm{~s~}$  per km of distance between two cities. Therefore, the total cost of changing this feature from city A to city B, we estimate as the distance from A to B, multiplied by  $0.1\mathrm{~s~}$ .
- card_type, card_brand - For these 2 features we use the same cost model as for the Card_type feature in IEEECIS dataset.
B Additional details on experiments
All the final experiments were done on AMD Ryzen 4750G CPU, Nvidia RTX 3070 GPU, and Ubuntu 22.04 OS.
We list our evaluation parameters in Table 1. The TabNet parameters are denoted according to the original paper Arik & Pfister (2019). We set the virtual batch size to 512. Most of the hyperparameters were selected via a grid search.
During adversarial training with a large  $\varepsilon$ , we encountered catastrophic overfitting on IEEECIS and Credit. For example during adversarial training with  $\varepsilon = 100$  on IEEECIS we observe the overfitting pattern in test robust accuracy (Table 2). In these situations, we reduced the number of epochs.
In this section, we show some additional experiments clarifying our design decisions and describing some properties of universal robust embeddings.
C.1 Comparison of first and last layer embeddings
As we mentioned in our paper, we only consider the first-layer embeddings of the neural network. The main justification for this design choice is that for many tabular datasets, "classical" methods like
LGBM or Random Forest can outperform neural networks. A good example of this case is the IEEECIS dataset. Both in the original Kaggle competition and in our experiments, decision trees show better performance than neural networks. To demonstrate this effect we run a simple experiment where we pass the last-layer embeddings to the LGBM and RF and compare the performance with
Table 1: Hyperparameters used in the evaluation
Parameter	Value range
All Datasets
τ	0.9
Dykstra iterations	20
IEEECIS
Batch size	2048
Number of epochs	400
PGD iteration number	20
TabNet hyperparameters	ND=32, NA=32, Nsteps=4
ε,$	[0.1, 0.2, 0.3, 0.5, 1.0, 3.0, 5.0, 10.0, 30.0, 50.0, 100.0]
BAF
Batch size	1024
Number of epochs	30
PGD iteration number	20
TabNet hyperparameters	ND=16, NA=16, Nsteps=3
ε,$	[0.1, 0.2, 0.3, 0.5, 1.0, 3.0, 10.0, 30.0]
Credit
Batch size	2048
Number of epochs	100
PGD iteration number	20
TabNet hyperparameters	ND=64, NA=64, Nsteps=2
ε,x10$	[1.0, 10.0, 30.0, 100.0, 300.0]
Table 2: Overfitting behaviour
Epoch	1	5	10	15	20	30
Test Robust Accuracy	51.5	52.4	55.0	61.2	53.4	52.1
normal LGBM/RF training. We report the results in Table 3. The results are rather expected: due to the data processing inequality, we can expect information loss if the data is processed by an inferior classifier for this task (i.e., a neural network). Therefore, we can conclude that first-layer embeddings are a better choice if we want to preserve the standard accuracy of the decision-tree based classifier.
C.2 Influence of the embedding merging algorithm
Another topic which we want to cover in this section is the effect of the embedding merging algorithm. In this subsection, we would like to answer two questions: