SlowGuess/NeurIPS_2023_OCR huggingface dataset:
i.e. try to minimize the distance between clean input  $x$  and perturbed input  $x'$  by transforming both via group actions. However, we evidently have  $\min_{g,h\in \mathbb{G}}d_{\mathrm{in}}(h\bullet x,g\bullet x')\leq \min_{g\in \mathbb{G}}d_{\mathrm{in}}(x,g\bullet x')$ . We can thus distinguish two cases
Case 1. Both action-induced distances are identical for all  $x, x' \in \mathbb{X}$  (which is always the case when  $\mathbb{G}$  acts isometrically). In this case, introducing the second group element is redundant.
Case 2. There is a pair  $x, x' \in \mathbb{X}$  such that  $\min_{g,h \in \mathbb{G}} d_{\mathrm{in}}(h \bullet x, g \bullet x') < \min_{g \in \mathbb{G}} d_{\mathrm{in}}(x, g \bullet x')$ . In this case, Desideratum 3 from Section 4 would be violated, i.e. the action-induced distance  $\hat{d}_{\mathrm{in}}$  would not optimally preserve the original distance  $d_{\mathrm{in}}$ .
Nevertheless, one could conceivably define robustness for equivariant tasks with non-isometric group actions as follows:
$$
\left(\max  _ {x ^ {\prime} \in \mathbb {X}} \max  _ {g, h \in \mathbb {G}} d _ {\mathrm {o u t}} (h ^ {- 1} \bullet f (h \bullet x), g ^ {- 1} \bullet f (g \bullet x ^ {\prime})\right) \text {s . t .} d _ {\mathrm {i n}} (x, x ^ {\prime}) \leq \epsilon) \leq \delta .
$$
All our discussions in Section 5 also apply to this notion of robustness.
However, this notion would be qualitatively different from Definition 1: Rather than requiring robustness robustness for a specific clean prediction  $f(x)$ , one would require robustness for the entire set of predictions  $\{f(h \bullet x) \mid h \in \mathbb{G}\}$ . As such, it is arguably not as good of a generalization of the classic notion of adversarial robustness. If one opts to use this notion of robustness for certification, one should at least state it explicitly. Afterall, being explicit about the underlying semantics of tasks and certificates is a main focus of this work.
Note that typically considered group actions (translation, rotation, reflection, permutation, etc.) act isometrically on the typically considered input spaces (e.g. Euclidean spaces), meaning these considerations should usually not be relevant in practice.
J Relation to transformation-specific robustness
As discussed in Section 2, works on transformation-specific robustness are concerned with robustness to unnoticeable parametric transformations, such as small rotations or translations. More formally, consider a parametric function  $\psi_{\theta}:\mathbb{X}\to \mathbb{X}$  with parameter  $\theta \in \Theta$ . A model is considered robust to transformation-specific attacks if  $\max_{\theta \in \Theta}d_{\mathrm{out}}(f(x),f(\psi_{\theta}(x))\leq \delta$  for some small  $\delta \in \mathbb{R}_{+}$ . Usually,  $d_{\mathrm{out}}$  is defined as the 0-1-loss  $\mathbb{1}[y\neq y^{\prime}]$  [56-65]. The underlying assumption is that the transformations  $\psi_{\theta}$  do not change the ground truth label of an input.
The key differences to our proposed notion of robustness are that (1) transformation-specific robustness does not consider that there are transformations for which predictions explicitly need to change and (2) works on transformation-specific usually do not consider unstructured perturbations like camera noise which are only constrained by a distance function  $d_{\mathrm{in}}$ .
In the following, we demonstrate that transformation-specific robustness can nevertheless be framed as a special case of our proposed notion of robustness — or relaxations thereof. We can distinguish three cases, depending on the structure of  $\{\psi_{\theta} \mid \theta \in \Theta\}$ .
Case 1. In the first case, the family of transformations  $\{\psi_{\theta} \mid \theta \in \Theta\}$  forms a group  $\mathbb{G}$  with composition as the group operator  $(\psi_{\theta} \cdot \psi_{\theta'} = \psi_{\theta} \circ \psi_{\theta'})$  and application of the transformation as the group action  $(\psi_{\theta} \bullet x = \psi_{\theta}(x))$ . In this case, the above definition of transformation-specific robustness can be reformulated as  $\max_{g \in \mathbb{G}} d_{\mathrm{out}}(f(x), f(g \bullet x)) \leq \delta$ . Assuming that our distance function fulfills  $d_{\mathrm{in}}(x, x') = 0 \iff x = x'$ , this is equivalent to
$$
\max  _ {x ^ {\prime} \in \mathbb {X}} \max  _ {g \in \mathbb {G}} d _ {\text {o u t}} (f (x), f (g \bullet x ^ {\prime})) \leq \delta \quad \text {s . t .} d _ {\text {i n}} (x, x ^ {\prime}) \leq 0. \tag {19}
$$
We observe that this is a special of our notion of robustness (see Definition 1), where the adversary has a budget of  $\epsilon = 0$  (i.e. can only apply group actions) and the task is group invariant.
Case 2. In the second case, the set of transformations is restricted to a subset  $\mathbb{H}$  of  $\mathbb{G}$  that is not a proper group. For example, one may restrict an adversary to small translations instead of arbitrary translations (see, e.g., [57]). In this case, transformation-specific robustness can be framed as
$$
\max  _ {x ^ {\prime} \in \mathbb {X}} \max  _ {g \in \mathbb {H}} d _ {\mathrm {o u t}} (f (x), f (g \bullet x ^ {\prime})) \leq \delta \quad \text {s . t .} d _ {\mathrm {i n}} (x, x ^ {\prime}) \leq 0,
$$
which is a relaxation of Eq. (19) since  $\mathbb{H}\subset \mathbb{G}$
Case 3. In the third case, the family  $\{\psi_{\theta} \mid \theta \in \Theta\}$  does not have a group structure or  $\psi_{\theta}(x)$  is not a proper group action (e.g. due to interpolation artifacts after rotation of an image). In this case, we can choose an arbitrary  $\epsilon \in \mathbb{R}$  and define an arbitrary  $d_{\mathrm{in}}(x,x')$  such that  $\{x' \mid d_{\mathrm{in}}(x,x') \leq \epsilon\} = \{\psi_{\theta}(x) \mid \theta \in \Theta\}$ . That is, the distance between  $x$  and  $x'$  is smaller than  $\epsilon$  if  $x'$  can be obtained via a transformation of  $x$ . With this choice of  $d_{\mathrm{in}}$ , transformation-specific robustness is equivalent to
$$
\max _ {x ^ {\prime} \in \mathbb {X}} d _ {\mathrm {o u t}} (f (x), f (x ^ {\prime})) \leq \delta \quad \mathrm {s . t .} d _ {\mathrm {i n}} (x, x ^ {\prime}) \leq \epsilon .
$$
This is an instance of classic adversarial robustness, which is a special case of our proposed notion of robustness (no equivariance).
Such a family of transformations is referred to as "resolvable" in transformation-specific robustness literature.