SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Klim Kireev, Maksym Andriushchenko, Carmela Troncoso, Nicolas Flammarion EPFL
Research on adversarial robustness is primarily focused on image and text data. Yet, many scenarios in which lack of robustness can result in serious risks, such as fraud detection, medical diagnosis, or recommender systems often do not rely on images or text but instead on tabular data. Adversarial robustness in tabular data poses two serious challenges. First, tabular datasets often contain categorical features, and therefore cannot be tackled directly with existing optimization procedures. Second, in the tabular domain, algorithms that are not based on deep networks are widely used and offer great performance, but algorithms to enhance robustness are tailored to neural networks (e.g. adversarial training). The code for our method is publicly available at https://github.com/spring-epfl/Transferable-Cat-Robustness.
In this paper, we tackle both challenges. We present a method that allows us to train adversarially robust deep networks for tabular data and to transfer this robustness to other classifiers via universal robust embeddings tailored to categorical data. These embeddings, created using a bilevel alternating minimization framework, can be transferred to boosted trees or random forests making them robust without the need for adversarial training while preserving their high accuracy on tabular data. We show that our methods outperform existing techniques within a practical threat model suitable for tabular data. The code for our method is publicly available<sup>1</sup>.
Works on adversarial machine learning primarily focus on deep networks and are mostly evaluated on image data. Apruzzese et al. (2022) estimate that approximately  $70\% - 80\%$  of works in the literature fall in this category. Yet, a large number of high-stake tasks across fields like medical diagnosis (Shehab et al., 2022), fraud detection (Altman, 2021), click-through rate prediction (Yang & Zhai, 2022), or credit scoring (Shi et al., 2022) neither only rely on deep networks nor operate on images (Grinsztajn et al., 2022). These tasks often involve many discrete categorical features (e.g., country, email, day of the week), and the predominant models used are discrete tree-based (e.g., boosted tree ensembles, random forests). These two characteristics raise a number of challenges when trying to achieve robustness using previous works which focus on continuous features and continuous models (Goodfellow et al., 2014; Madry et al., 2018).
Accurate modelling of adversarial capability. The de-facto standard in adversarial robustness evaluation (Croce et al., 2020) is to model robustness to perturbations bounded in some  $\ell_p$  ball, mostly in the image domain. This approach, however, was shown to not accurately represent the capabilities of a real adversary in other domains (Kireev et al., 2022; Apruzzese et al., 2022). Instead, a realistic threat model would constrain the adversary with respect to their financial capabilities. This can be achieved by associating a financial cost with every input feature transformation, limiting the adversary to perform transformations within their total financial budget. Such a constraint is common
$^{1}$ https://github.com/spring-epfl/Transferable-Cat-Robustness
for computer security problems as it ties security, in this case, robustness, to real-world limitations. The inaccuracy of threat models in the literature translates on a lack of well-motivated benchmarks for robustness research on tabular data, unlike for image-oriented vision tasks (Croce et al., 2020; Koh et al., 2020).
Accounting for discrete categorical features. Tabular data is usually heterogeneous and often includes categorical features which can be manipulated by an adversary in a non-uniform way, depending on the characteristics of the real-world concept they represent. For example, buying an email address is not the same as buying a credit card or moving to a different city. Moreover, for a given feature, not all transformations have equal cost or are even possible: for example, one can easily change their email address to  $@$  gmail.com, while changing the domain name to  $@$  rr.com can be impossible, since this domain name is unavailable for new users. Hence, the definitions of perturbation sets describing the capabilities of potential adversaries should support complex and heterogeneous constraints.
Robustness for models other than neural networks. Gradient-based attacks based on projected gradient descent provide a simple and efficient method for crafting adversarial examples. They are effectively used for adversarial training, which became a de-facto standard defence against adversarial perturbations (Madry et al., 2018). Albeit defences and attacks are proposed for decision tree models, they often employ combinatorial methods and can be very inefficient time-wise (Kantchelian et al., 2016; Calzavara et al., 2020; Kireev et al., 2022). However, in tasks involving tabular data, these models must be prioritized as they are widely used because they can provide superior performance on some tabular datasets than neural networks (Grinsztajn et al., 2022).
Contributions. Our contributions address the challenges outlined above as follows:
- We propose a practical adversarial training algorithm supporting complex and heterogeneous constraints for categorical data that can accurately reflect financial costs for the adversary. Our training algorithm is based on the continuous relaxation of a discrete optimization problem and employs approaches from projections onto an intersection of convex sets.
- We propose a method to generate universal robust embeddings, which can be used for transferring robustness from neural networks to other types of machine learning models such as decision trees or random forests.
- We use existing datasets to build the first benchmark that allows us to evaluate robustness for tabular tasks in which the adversary is constrained by financial capabilities.
- Using the proposed benchmark, we empirically show that our proposed methods provide significantly better robustness than previous works.
Here we discuss the most relevant references related to the topics outlined in the introduction.
Gradient-based adversarial training. Adversarial training is the key algorithm for making neural networks robust to standard  $\ell_p$ -bounded adversarial examples. Szegedy et al. (2013) demonstrate that modern deep networks are susceptible to adversarial examples that can be easily found via gradient descent. Madry et al. (2018) perform successful adversarial training for deep networks where each iteration of training uses projected gradient descent to find approximately optimal adversarial examples. Related to our method of universal first-layer embeddings, Bakiskan et al. (2022) perform partial adversarial training (i.e., training the whole network adversarially/normally and then reinitializing some layer and training them in the opposite way) which is somewhat, showing that earlier layers are more important for robustness. On a related note, Zhang et al. (2019) do adversarial training on the whole network but do multiple forward-backward passes on the first layer with the goal of speeding up adversarial training. Yang et al. (2022) use metric learning with adversarial training to produce word embeddings robust to word-level adversarial attacks which can be used for downstream tasks. Dong et al. (2021) also produce robust word embeddings via a smart relaxation of the underlying discrete optimization problem. We take inspiration from this line of work when we adapt adversarial training for neural networks to categorical features and transfer the first-layer embeddings to tree-based models.
Robustness of tree-based models. Tree-based models such as XGBoost (Chen & Guestrin, 2016) are widely used in practice but not amenable to gradient-based adversarial training. Most of the works on adversarial robustness for trees focus on  $\ell_{\infty}$  adversarial examples since they are easier to work with due to the coordinate-aligned structure of decision trees. Kantchelian et al. (2016) is the first algorithm for training robust tree ensembles which are trained on a pool of adversarial examples updated on every iteration of boosting. This approach is refined by approximately solving the associated min-max problem for  $\ell_{\infty}$  robustness in Chen et al. (2019) and by minimizing an upper bound on the robust loss in Andriushchenko & Hein (2019) on each split of every decision tree in the ensemble. Chen et al. (2021) extend the  $\ell_{\infty}$  approach of Chen et al. (2019) to arbitrary box constraints and apply it to a cost-aware threat model on continuous features. Only few works tackle non- $\ell_{\infty}$  robust training of trees since their coordinate-aligned structure is not conducive to other norms. Most related work to ours is Wang et al. (2020) extend the upper bounding approach of Andriushchenko & Hein (2019) for arbitrary  $\ell_{p}$ -norms. However, they report that  $\ell_{\infty}$  robust training in most cases works similarly to  $\ell_{1}$  robust training, though in a few cases  $\ell_{1}$  adversarial training yields better performance. Moreover, they do not use categorical features which are typically present in tabular data which is the focus of our work.
Threat modeling for tabular data. Many prior works do not consider a realistic threat model and adversarial capabilities. First, popular benchmarks like Madry et al. (2018); Croce et al. (2020) assume that the adversary has an equal budget for perturbing each feature which is clearly not realistic for tabular data. To fix this, Chen et al. (2021) propose to consider different perturbation costs for different features. Kireev et al. (2022) argue that, as imperceptibility and semantic similarity are not necessarily meaningful considerations for tabular datasets, these costs must be based on financial constraints – i.e., how much money it costs for the adversary to execute an attack for a particular example. The importance of monetary costs was corroborated by studies involving practitioners dealing with adversarial machine learning in real applications (Apruzzese et al., 2022; Grosse et al., 2023). Second, popular benchmarks treat equally changes from the correct to any other class, but in practice the adversary is typically interested only in a specific class change, e.g., modifying a fraudulent transaction to be classified as non-fraudulent. To address this shortcoming, past works (Zhang & Evans, 2019; Shen et al., 2022) propose class-sensitive robustness formulations where an unequal cost can be assigned to every pair of classes that can be changed by the adversary. As a result, this type of adversarial training can achieve a better robustness-accuracy tradeoff against a cost-sensitive adversary. We take into account all these works and consider a cost-based threat model where we realistically model the costs and classes which are allowed to be changed for the particular datasets we use.
Input domain and Model. The input domain's feature space  $\mathbb{X}$  is composed of  $m$  features:  $\mathbb{X} \subseteq \mathbb{X}_1 \times \mathbb{X}_2 \times \dots \times \mathbb{X}_m$ . We denote as  $x_i$  the value of the  $i$ -th feature of  $x \in \mathbb{X}$ . Features  $x_i$  can be categorical, ordinal, or numeric, and we define tabular data as data consisting of these three types of features in any proportion. Categorical features are features  $x_i$  such that  $\mathbb{X}_i$  is a finite set of size  $|\mathbb{X}_i| = t_i$ , i.e.,  $t_i$  is the number of possible values that can take  $x_i$ . We denote by  $t = \sum_{i=1}^{m} t_i$ . We also denote as  $x_i^j$ , the  $j$ -th value of the feature  $x_i$ . We further assume that each example  $x \in \mathbb{X}$  is associated with a binary label  $y \in \{0,1\}$ . Finally, we consider a general learning model with parameters  $\theta$  and output  $\eta(\theta, x)$ . The model we consider in our framework can take on two forms: differentiable, such as a neural network, or non-differentiable, like a tree-based model.
Perturbation set. Following the principles described by Apruzzese et al. (2022); Kireev et al. (2022), we make the assumption that a realistic adversary is constrained by financial limitations. Specifically, we denote the cost of modifying the feature  $x_{i}$  to  $x_{i}^{\prime}$  as  $c_{i}(x_{i},x_{i}^{\prime})$ . For categorical features with a finite set  $\mathbb{X}_i$ , the cost function  $c_{i}(x_{i},x_{i}^{\prime})$  can be conveniently represented using a cost matrix  $C_i\in \mathbb{R}_{\geq 0}^{t_i\times t_i}$ . The cost of changing the feature  $x_{i}$  from the value  $j$  to the value  $k$  is given by
$$
c _ {i} (x _ {i} ^ {j}, x _ {i} ^ {k}) = C _ {i} ^ {j k}.
$$
If such transformation is impossible,  $C_i^{jk}$  is set to  $\infty$ . In a realistic threat model, any cost matrix  $C_i$  is possible, and costs need not be symmetric. The only reasonable assumption is that  $C_i^{jj} = 0$ , indicating no cost for staying at the same value. See realistic cost matrix examples in the Appendix A.
We assume that the cost of modifying features is additive: the total cost of crafting an adversarial example  $x'$  from an example  $x$  can be expressed as:
$$
c \left(x, x ^ {\prime}\right) = \sum_ {i = 1} ^ {m} c _ {i} \left(x _ {i}, x _ {i} ^ {\prime}\right). \tag {1}
$$
Encoding and Embeddings. Categorical features are typically preprocessed by encoding each value  $x_{i}$  using a one-hot encoding vector  $\overline{x}_i$ . For instance, if a categorical feature  $x_{i}$  can take four possible values  $\{1,2,3,4\}$  and  $x_{i} = 2$ , it is represented as  $\overline{x}_i = (0,1,0,0)^\top$ . We can then represent the feature-wise cost function as
$$
c _ {i} \left(x _ {i}, x _ {i} ^ {\prime}\right) = \left\| w _ {i} \odot \left(\bar {x} _ {i} - \bar {x} _ {i} ^ {\prime}\right) \right\| _ {1} = l _ {1, w _ {i}} \left(\bar {x} _ {i}, \bar {x} _ {i} ^ {\prime}\right), \tag {2}
$$
where  $l_{1,w}$  denotes the weighted  $l_{1}$  norm and  $w_{i} = C_{i}\overline{x}_{i}$  is the costs of transforming  $x_{i}$  to any other possible value in  $\mathbb{X}_i$ . Then according to Equation (1), the per sample cost function is:
$$
c \left(x, x ^ {\prime}\right) = \left\| w _ {i} \odot \left(\bar {x} - \bar {x} ^ {\prime}\right) \right\| _ {1} = l _ {1, w} \left(\bar {x}, \bar {x} ^ {\prime}\right), \tag {3}
$$
where the vectors  $w, \overline{x}, \overline{x}'$  are the contention of the vectors  $w_i, \overline{x}_i, \overline{x}_i'$  respectively for  $\forall i: 0 \leq i \leq m$ .
The one-hot-encoded representation may not be optimal for subsequent processing due to high dimensionality or data sparsity. We instead use embeddings and replace the categorical features with their corresponding embedding vector  $\phi(x) \in R^n$ . The model can then be represented by a function  $f$  of the features as
$$
\eta (\theta , x) = f (\theta , \phi (x)).
$$
The embedding function  $\phi$  is composed of embedding functions  $\phi_{i}$  for the subvector  $x_{i}$  as  $\phi (x) = (\phi_i(x))_{i = 1}^m$ . The embedding function  $\phi_{i}$  of the  $i$ -th feature can always be represented by a matrix  $Q_{i}$  since  $x_{i}$  takes discrete values. The columns of this matrix are embedding vectors which satisfy
$$
\phi_ {i} (x _ {i}) = Q _ {i} \bar {x} _ {i}.
$$
Therefore, we parametrize the embedding function  $\phi (x,Q)$  with a family of  $m$  matrices  $Q = (Q_{i})_{i = 1}^{m}$ .
Numerical features. For numerical features, we use a binning procedure which is a common practice in tabular machine learning. This technique facilitates the use of decision-tree-based classifiers as they naturally handle binned data (Chen & Guestrin, 2016; Ke et al., 2017b). Binning transforms numerical features into categorical ones, allowing a unique treatment of all the features. It is worth noting that for differentiable classifiers, the approach by Kireev et al. (2022) can enable us to directly use numerical features without binning.
4 Adversarial training
In this section, we outline our adversarial training procedure. When adapted to our specific setup, the adversarial training problem (Madry et al., 2018) can be formulated as:
$$
\min  _ {\theta , Q} \mathbb {E} _ {x, y \in D} \max  _ {c \left(x, x ^ {\prime}\right) \leq \varepsilon} \ell \left(\left(f \left(\phi \left(x ^ {\prime}, Q\right)\right), \theta\right), y\right). \tag {4}
$$
Such model would be robust to adversaries that can modify the value of any feature  $x_{i}$ , as long as the total cost of the modification is less than  $\varepsilon$ . While this formulation perfectly formalizes our final goal, direct optimization of this objective is infeasible in practice:
1. Optimizing directly with categorical features can be computationally demanding due to the need for discrete optimization algorithms. In our evaluation, we employ a graph-search-based procedure developed by Kireev et al. (2022). However, finding a single example using this algorithm can take between 1 to 10 seconds, making it impractical for multi-epoch training even on medium-sized datasets (approximately 100K samples).
2. There is currently no existing algorithm in the literature that enables us to operate within this cost-based objective for decision-tree based classifiers.
Algorithm 1 Cat-PGD. Relaxed projected gradient descent for categorical data.
Input: Data point  $\tilde{x},y$  Attack rate  $\alpha$  Cost bound  $\varepsilon$  ,Cost matrices  $C$  PGD_steps, D_steps Output: Adversarial sample  $\tilde{x}^{\prime}$ $\delta \coloneqq 0$    
for  $i = 1$  to PGD_steps do   
 $\nabla \coloneqq \nabla \delta \ell (f(Q(\tilde{x} +\delta)),y))$ $\delta \coloneqq \delta +\alpha \nabla$ $p\coloneqq 0$ $q\coloneqq 0$    
for  $i = 1$  to D_steps do   
 $z\coloneqq \Pi_{simplices}(\tilde{x},\delta +p)$ $p\coloneqq \delta +p - z$ $\delta \coloneqq \Pi_{cost}(z + q,C)$ $q\coloneqq z + q - \delta$    
end for   
end for   
 $\tilde{x}^{\prime}\coloneqq \tilde{x} +\delta$
4.1 Adversarial training for differentiable models
We begin by examining the scenario where our model  $f$  is a deep neural network. In this case, we are constrained solely by the first restriction. To address this constraint, we employ a relaxation technique: instead of working with the discrete set of feature vectors, we consider the convex hull of their one-hot encoding vectors. For each feature, this relaxation allows us to replace the optimization over  $x_{i}^{\prime} \in \mathbb{X}_{i}$  with an optimization over  $\tilde{x}_i^\prime \in \mathbb{R}_{\geq 0}^{t_i}$ . More precisely, we first replace the feature space  $\mathbb{X}_i$  by the set  $\{\overline{x}_i^\prime \in \{0,1\}^{t_i}, \sum_j \overline{x}_i^{\prime j} = 1\}$  using the one-hot encoding vectors. We then relax this discrete set into  $\{\tilde{x}_i^\prime \in \mathbb{R}_{\geq 0}^{t_i}, \sum_j \tilde{x}_i^{\prime j} = 1\}$ . The original constraint  $\{x^{\prime} \in \mathbb{X}, c(x,x^{\prime}) \leq \varepsilon\}$  is therefore relaxed as
$$
\{\tilde {x} _ {i} ^ {\prime} \in \mathbb {R} _ {\geq 0} ^ {t _ {i}}, \sum_ {j} \tilde {x} _ {i} ^ {\prime j} = 1, l _ {1, w} (\bar {x}, \tilde {x} ^ {\prime}) \leq \varepsilon \text {a n d} \tilde {x} ^ {\prime} = (\tilde {x} _ {i}) _ {i = 1} ^ {m} \}.
$$
The set obtained corresponds to the convex hull of the one-hot encoded vectors, providing the tightest convex relaxation of the initial discrete set of vectors  $\overline{x}$ . In contrast, the relaxation proposed by Kireev et al. (2022) relaxes  $\mathbb{X}_i$  to  $\mathbb{R}_{\geq 0}^{t_i}$ , leading to significantly worse performance as shown in Section 5.
The adversarial training objective for the neural networks then becomes:
$\min_{\theta ,Q}\underset {x,y\in D}{\mathbb{E}}\max_{\substack{l_{1,w}(\overline{x},\tilde{x}^{\prime})\leq \varepsilon \\ \| \tilde{x}_i^{\prime}\| _1 = 1,\tilde{x}_i^{\prime}\in \mathbb{R}_{\geq 0}^{t_i}\text{for} 1\leq i\leq m}}\ell (f(Q\tilde{x}^{\prime},\theta),y).$  (5)
In order to perform the inner maximization, we generate adversarial examples using projected gradient descent. The projection is computed using the Dykstra projection algorithm (Boyle & Dykstra, 1986) which enables us to project onto the intersection of multiple constraints. In each iteration of the projection algorithm, we first project onto the convex hull of each categorical feature  $(\Pi_{simplices})$  followed by a projection onto a weighted  $l_{1}$  ball  $(\Pi_{cost})$ . This dual projection approach allows us to generate perturbations that simultaneously respect the cost constraints of the adversary and stay inside the convex hull of original categorical features. We refer to this method as Cat-PGD, and its formal description can be found in Algorithm 1.
4.2 Bilevel alternating minimization for universal robust embeddings
While the technique described above is not directly applicable to non-differentiable models like decision trees, we can still leverage the strengths of both neural networks and tree-based models. By transferring the learnt robust embeddings from the first layer of the neural network to the decision tree classifier, we can potentially combine the robustness of the neural network with the accuracy of boosted decision trees.
Algorithm 2 Bilevel alternating minimization for universal robust embeddings
Input: Set of training samples  $X, y$ , cost bound  $\varepsilon$ ,  $\theta$ -steps,  $Q$ -steps,  $PGD$ -steps,  $N$ -iters.  
Output: Embeddings  $Q_{N\text{-iter}}$ .  
Randomly initialize  $Q$ ,  $\theta$ .  
for  $i = 1$  to  $N$ -iters do  
 $\theta_i^1 := \theta_i$ ,  $Q_i^1 := Q_i$   
for  $j = 1$  to  $\theta$ -steps do  
 $\theta_i^{j+1} := \theta_j - \alpha \nabla_\theta \ell(f(Q_i \overline{x}, \theta_i^j), y)$   
end for  
 $\theta_i := \theta_i^{\theta\text{-steps}}$   
for  $j = 1$  to  $Q$ -steps do  
 $\tilde{x}' := \mathbf{Attack}(\overline{x}, y, PGD\text{-steps}, Q_i^j, \theta_i)$ $Q_i^{j+1} := Q_i^j - \beta \nabla_Q \ell(f(Q_i^j \tilde{x}', \theta_i), y)$   
end for  
 $\theta_{i+1} := \theta_i^{\theta\text{-steps}}$ ,  $Q_{i+1} := Q_i^{Q\text{-steps}}$   
end for
Difference between input and output embeddings. Using embeddings obtained from the last layer of a neural network is a common approach in machine learning (Zhuang et al., 2019). However, in our study, we prefer to train and utilize embeddings from the first layer where the full information about the original features is still preserved. This choice is motivated by the superior performance of decision-tree-based classifiers over neural networks on tabular data in certain tasks. By using first layer embeddings, we avoid a potential loss of information (unless the embedding matrix  $Q$  has exactly identical rows which are unlikely in practice) that would occur if we were to use embeddings from the final layer.
To show this effect we ran a simple experiment where we compare first and last layer embeddings as an input for a random forest classifier. We report the results in Appendix C.1.
Step 1: bilevel alternating minimization framework. A natural approach is to use standard adversarial training to produce robust first layer embeddings. However, we noticed that this method is not effective in producing optimal first layer embeddings. Instead, we specifically produce robust first layer embeddings and ignore the robustness property for the rest of the layers. This idea leads to a new objective that we propose:
Bilevel minimization:  $\min_{Q}\underset {x,y\in D}{\mathbb{E}}\max_{\substack{l_{1,w}(\overline{x},\tilde{x}^{\prime})\leq \varepsilon \\ \| \tilde{x}_i^{\prime}\| _1 = 1,\tilde{x}_i^{\prime}\in \mathbb{R}_{\geq 0}^{t_i}}}\ell \Bigg(f\Big(Q\tilde{x},\arg \min_{\theta}\underset {x,y\in D}{\mathbb{E}}\ell \big(f(Q\tilde{x}^{\prime},\theta),y\big)\Big),y\Bigg).$  (6)