SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let  $k = |S_{\text{leftover}}| > 1$ . Then, for a fixed constant  $c = \frac{1}{(2e)^2}$ , with any scalar  $\epsilon < \sqrt{d}$ , with probability at least  $1 - 2\exp(-c\epsilon^2 / G^4)$  and  $\forall$  integers  $M \in \left[1, \left\lfloor \frac{n}{2k} \right\rfloor \right]$ ,
$$
\| \mathbf {w} _ {\text {s t a b l e}} \| ^ {2} \geq W _ {\text {s t a b l e}} = \frac {1}{4 + \frac {\left(\sqrt {d} + \epsilon\right) ^ {2}}{2 M k}}.
$$
Lemma. (8) Consider the following optimization problem from eq. (16) where  $n$  samples of  $\mathbf{x}_i, \mathbf{y}_i$  come from eq. (1) where  $\mathbf{x}_i \in \mathbf{R}^d$ :
$$
\mathbf {w} _ {s h o r t c u t} = \underset {\mathbf {w}} {\arg \min } \quad w _ {y} ^ {2} + w _ {z} ^ {2} + \| \mathbf {w} _ {e} \| ^ {2}
$$
$$
s. t. i \in S _ {\text {s h o r t c u t}} \quad w _ {y} + B w _ {z} + \mathbf {w} _ {e} ^ {\top} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} > 1 \tag {19}
$$
$$
i \in S _ {\text {l e f t o v e r}} \quad w _ {y} - B w _ {z} + \mathbf {w} _ {e} ^ {\top} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} > 1
$$
$$
\mathbf {w} _ {y} <   B \mathbf {w} _ {z}
$$
Let  $k = |S_{\text{leftover}}| \geq 1$ . Then, for a fixed constant  $c = \frac{1}{(2e)^2}$ , with any scalar  $\epsilon < \frac{1}{3}\sqrt{\frac{d}{k}} < \sqrt{d}$ , with probability at least  $1 - 2(2k + (n - k) + 1)\exp(-c\frac{\epsilon^2}{G^4})$ , for  $\gamma = \frac{2}{d - 4\epsilon\sqrt{k}d}$ ,
$$
\left\| \mathbf {w} _ {\text {s h o r t c u t}} \right\| ^ {2} \leq W _ {\text {s h o r t c u t}} = \gamma^ {2} k (\sqrt {d} + \epsilon) ^ {2} + \frac {\left(1 + \gamma \epsilon \sqrt {d k}\right) ^ {2}}{B ^ {2}}
$$
Together, the lemmas say that for any  $\forall$  integers  $M\in \left[1,\lfloor \frac{n}{2k}\rfloor \right]$  and  $\epsilon <  \frac{1}{3}\sqrt{\frac{d}{k}}$  , with probability  $\geq 1 - 2\exp (-c\epsilon^2 /G^4)$
$$
\left\| \mathbf {w} _ {\text {s t a b l e}} \right\| ^ {2} \geq W _ {\text {s t a b l e}} = \frac {1}{4 + \frac {\left(\sqrt {d} + \epsilon\right) ^ {2}}{2 M k}}.
$$
and with probability at least  $1 - 2(2k + (n - k) + 1)\exp (-c\frac{\epsilon^2}{G^4})$ , for  $\gamma = \frac{2}{d - 4\epsilon\sqrt{kd}} >0$
First, we choose  $\epsilon^2 = 2\frac{G^4}{c}\log (3n)$ . This gives us the probability with which these bounds hold: as  $k < 0.1n$  we have  $k + 2 < \frac{n}{2}$  and
$$
\begin{array}{l} 1 - 2 (2 k + (n - k) + 2) \exp (- c \frac {\epsilon^ {2}}{G ^ {4}}) = 1 - 2 (n + k + 2) \exp (- 2 \log (3 n)) \\ \geq 1 - 2 \left(\frac {3 n}{2}\right) \exp \left(- 2 \log (3 n)\right) \\ = 1 - \exp (- 2 \log (3 n) + \log (3 n)) \\ = 1 - \exp (- \log (3 n)) \\ = 1 - \frac {1}{3 n}. \\ \end{array}
$$
Next, we will instantiate the parameter  $M$  and set the constants  $C_1, C_2$  and the upper bound on  $k$  in theorem 1 to guarantee the following eq. (separation inequality):
$$
W _ {\text {s h o r t c u t}} = \gamma^ {2} k (\sqrt {d} + \epsilon) ^ {2} + \frac {\left(1 + \gamma \epsilon \sqrt {d k}\right) ^ {2}}{B ^ {2}} <   \frac {1}{4 + \frac {\left(\sqrt {d} + \epsilon\right) ^ {2}}{2 M k}} = W _ {\text {s t a b l e}},
$$
(separation inequality)
which then implies that  $\| \mathbf{w}_{\mathrm{shortcut}}\| ^2 <  \| \mathbf{w}_{\mathrm{stable}}\| ^2$  , concluding the proof.
Invoking the conditions in theorem 1 and setting the upper bound on  $k$ . We will keep the  $\epsilon$  as is for simplicity of reading but invoke the inequalities satisfied by  $\log(3n)$  from theorem 1:
$$
\exists \text {c o n s t a n t} C _ {1}, \quad d \geq C _ {1} k \log (3 n).
$$
Now we let  $C_1 = 2\frac{G^4}{cC^2}$  for a constant  $C \in \left(0, \frac{1}{3}\right)^3$ , such that
$$
\epsilon^ {2} = 2 \frac {G ^ {4}}{c} \log (3 n) <   C ^ {2} \frac {d}{k} \Rightarrow \epsilon <   C \sqrt {\frac {d}{k}} \text {a n d} \epsilon \sqrt {k d} <   C d. \tag {20}
$$
We next find a  $C \in \left(0, \frac{1}{3}\right)$  such that eq. (separation inequality) holds with  $M = 5$ , which upper bounds  $k$ :
$$
M <   \frac {n}{2 k} \Longrightarrow \frac {k}{n} <   \frac {1}{2 M} = \frac {1}{1 0} \Longrightarrow k <   \frac {n}{1 0}.
$$
Simplifying  $W_{\text{shortcut}}$  and  $W_{\text{stable}}$ . To actually show  $W_{\text{shortcut}} < W_{\text{stable}}$  in eq. (separation inequality), we compare a simplified strict upper bound on the LHS  $W_{\text{shortcut}}$  and a simplified strict lower bound on the RHS  $W_{\text{stable}}$
For the simplification of the RHS  $W_{\mathrm{stable}}$  of eq. (separation inequality), we will use the fact that  $d \geq 2\frac{G^4}{cC^2}\log(3n)k$ . Given the assumption  $n > N_0$ , choosing  $N_0$  to be an integer such that  $\log(3N_0) \geq \frac{40cC^2}{G^4}$  means that  $\log(3n) > \frac{40cC^2}{G^4}$  and we have
$$
\frac {d}{k} > 8 0 \Longrightarrow \frac {d}{1 0 k} > 8 \Longrightarrow \frac {1}{2} \frac {d}{1 0 k} > 4 \tag {21}
$$
which gives us, for  $M = 5$
$$
\begin{array}{l} W _ {\text {s t a b l e}} = \frac {1}{4 + \frac {(\sqrt {d} + \epsilon) ^ {2}}{2 M k}} (22) \\ = \frac {1}{4 + \frac {\left(\sqrt {d} + \epsilon\right) ^ {2}}{1 0 k}} (23) \\ \geq \frac {1}{\frac {3}{2} \left(\sqrt {d} + \epsilon\right) ^ {2}} \quad \{4 <   \frac {1}{2} \frac {d}{1 0 k} <   \frac {1}{2} \frac {(\sqrt {d} + \epsilon) ^ {2}}{1 0 k} \text {f r o m e q . (2 1)} \} (24) \\ = \frac {2 0 k}{3 (\sqrt {d} + \epsilon) ^ {2}} (25) \\ \geq \frac {2 0 k}{3 (\sqrt {d} + C \frac {\sqrt {d}}{\sqrt {k}}) ^ {2}} \quad \left\{\epsilon <   \frac {C \sqrt {d}}{\sqrt {k}} \text {f r o m e q . (2 0)} \right\} (26) \\ = \frac {2 0 k}{3 \left(1 + \frac {C}{\sqrt {k}}\right) ^ {2} d} (27) \\ > \frac {2 0 k}{3 (1 + C) ^ {2} d} \quad \{k \geq 1 \} (28) \\ \end{array}
$$
Now, we produce a simpler upper bound on the first part of the LHS of eq. (separation inequality): recalling that  $\gamma = \frac{2}{d - 4\epsilon\sqrt{kd}}$ , and substituting in the upper bounds on  $\epsilon$ ,
$$
\begin{array}{l} \gamma^ {2} k (\sqrt {d} + \epsilon) ^ {2} = \left(\frac {2 (\sqrt {d} + \epsilon)}{d - 4 \epsilon \sqrt {k d}}\right) ^ {2} k \\ <   4 \left(\frac {(\sqrt {d} + C \sqrt {\frac {d}{k}})}{d - 4 C d}\right) ^ {2} k \quad \{\epsilon <   \frac {C \sqrt {d}}{\sqrt {k}} \text {f r o m e q . (2 0)} \} \\ = \frac {4 k}{d} \left(\frac {(1 + \frac {C}{\sqrt {k}})}{1 - 4 C}\right) ^ {2} \\ \end{array}
$$
3The  $\frac{1}{3}$  comes from requiring that  $\epsilon < \frac{1}{3}\sqrt{\frac{d}{k}}$  from lemma 8.
$$
\leq \frac {4 k}{d} \left(\frac {1 + C}{1 - 4 C}\right) ^ {2}, \quad \{k \geq 1 \} \tag {29}
$$
Next is a simpler upper bound on the second part of the LHS of eq. (separation inequality). Again with  $\gamma = \frac{2}{d - 4\epsilon\sqrt{kd}}$ ,
$$
\begin{array}{l} \frac {\left(1 + \gamma \epsilon \sqrt {d k}\right) ^ {2}}{B ^ {2}} = \frac {\left(1 + \frac {2 \epsilon \sqrt {d k}}{d - 4 \epsilon \sqrt {k d}}\right) ^ {2}}{B ^ {2}} \\ \leq \frac {\left(1 + \frac {2 C d}{d - 4 C d}\right) ^ {2}}{B ^ {2}} \\ = \frac {\left(1 + \frac {2 C}{1 - 4 C}\right) ^ {2}}{B ^ {2}} \\ \end{array}
$$
Now setting
$$
B > \sqrt {2} \frac {\left(1 + \frac {2 C}{1 - 4 C}\right)}{\sqrt {\frac {4 k}{d}} \left(\frac {1 + C}{1 - 4 C}\right)}
$$
givesthe lower bound on  $B$  from theorem 1:
$$
B > C _ {2} \sqrt {\frac {d}{k}}, \quad \text {w h e r e} \quad C _ {2} = \frac {\left(1 + \frac {2 C}{1 - 4 C}\right)}{\sqrt {2} \left(\frac {1 + C}{1 - 4 C}\right)} = \frac {(1 - 2 C)}{\sqrt {2} (1 + C)}.
$$
Formally,
$$
B > C _ {2} \sqrt {\frac {d}{k}} \Rightarrow \frac {\left(1 + \frac {2 C}{1 - 4 C}\right) ^ {2}}{B ^ {2}} <   \frac {1}{2} \left(\sqrt {\frac {4 k}{d}} \left(\frac {1 + C}{1 - 4 C}\right)\right) ^ {2} = \frac {1}{2} \frac {4 k}{d} \left(\frac {1 + C}{1 - 4 C}\right) ^ {2}. \tag {30}
$$
By combining the upper bound from eq. (30) and the upper bound from eq. (29), we get an upper bound on the whole of the LHS of eq. (separation inequality), which in turn provides an upper bound on  $W_{\mathrm{shortcut}}$ :
$$
W _ {\mathrm {s h o r t c u t}} = \gamma^ {2} k (\sqrt {d} + \epsilon) ^ {2} + \frac {\left(1 + \gamma \epsilon \sqrt {d k}\right) ^ {2}}{B ^ {2}} <   \frac {3}{2} \frac {4 k}{d} \left(\frac {(1 + C)}{1 - 4 C}\right) ^ {2} \leq \frac {3}{2} \frac {4 k}{d} \left(\frac {(1 + C)}{1 - 4 C}\right) ^ {2},
$$
because  $k \geq 1$ . Note the upper bound is strict.
Concluding the proof. Now, we show that a  $C$  exists such that the following holds, which implies  $W_{\mathrm{shortcut}} < W_{\mathrm{stable}}$ , which in turn implies eq. (separation inequality) and the proof concludes:
$$
W _ {\text {s h o r t c u t}} <   \frac {3}{2} \frac {4 k}{d} \left(\frac {(1 + C)}{1 - 4 C}\right) ^ {2} \leq \frac {2 0 k}{3 (1 + C) ^ {2} d} <   W _ {\text {s t a b l e}}.
$$
The above inequality holds when
$$
6 \left(\frac {(1 + C)}{1 - 4 C}\right) ^ {2} \leq \frac {2 0}{3 (1 + C) ^ {2}} \quad \Longleftrightarrow \quad (1 + C) ^ {2} - \sqrt {\frac {1 0}{9}} (1 - 4 C) \leq 0.
$$
The right hand side holds when the quadratic equation  $(1 + C)^2 -\sqrt{\frac{10}{9}} (1 - 4C)$  is non-positive, which holds between the roots of the equation. The equation's positive solution is
$$
C = \frac {- 3 + \sqrt {1 0}}{3 + 2 \sqrt {1 0} + \sqrt {5 (8 + 3 \sqrt {1 0})}} \approx 0. 0 0 8.
$$
Setting  $C$  to this quantity satisfies the requirement that  $C \in (0, \frac{1}{3})$ .
Thus, a  $C$  exists such that eq. (separation inequality) holds which concludes the proof of theorem 1 for the following constants and constraints implied by  $C$  and  $M = 5$ :
$$
C _ {2} = \frac {(1 - 2 C)}{\sqrt {2} (1 + C)} \qquad C _ {1} = 2 \frac {G ^ {4}}{c C ^ {2}} \qquad k <   \frac {n}{1 0},
$$
where  $G$  is the  $\psi_{2}$ -norm of a standard normal r.v. and  $c$  is the absolute constant from the Bernstein inequality in lemma 2.
A.3 Lower bounding the norm of solutions that rely more on the stable feature
Lemma 7. Consider the following optimization problem from eq. (16) where  $n$  samples of  $\mathbf{x}_i, \mathbf{y}_i$  come from eq. (1) where  $\mathbf{x}_i \in \mathbf{R}^d$ :
$$
\mathbf {w} _ {s t a b l e} = \underset {\mathbf {w}} {\arg \min} \quad w _ {y} ^ {2} + w _ {z} ^ {2} + \left\| \mathbf {w} _ {e} \right\| ^ {2}
$$
$$
s. t. i \in S _ {\text {s h o r t c u t}} \quad w _ {y} + B w _ {z} + \mathbf {w} _ {e} ^ {\top} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} > 1 \tag {31}
$$
$$
\| \mathbf {w} _ {s t a b l e} \| ^ {2} \geq W _ {s t a b l e} = \frac {1}{4 + \frac {\left(\sqrt {d} + \epsilon\right) ^ {2}}{2 M k}}.
$$
Proof. By lemma 6, the dual of eq. (16) is the following for  $\zeta = [-B,1,\mathbf{0}^{d - 2}]$  and  $X$  is an  $n\times d$  matrix with rows  $\mathbf{y}_i\mathbf{x}_i$ :
$$
\max  _ {\lambda \geq 0, \nu \geq 0} - \frac {1}{4} \| \zeta \nu + X ^ {\top} \lambda \| ^ {2} + \mathbf {1} ^ {\top} \lambda \tag {32}
$$
Now by duality
$$
\| \mathbf {w} _ {\text {s t a b l e}} \| ^ {2} \geq \max  _ {\lambda \geq 0, \nu \geq 0} - \frac {1}{4} \| \zeta \nu + X ^ {\top} \lambda \| ^ {2} + \mathbf {1} ^ {\top} \lambda ,
$$
which means any feasible candidate to eq. (32) gives a lower bound on  $\| \mathbf{w}_{\mathrm{stable}}\|^2$
Feasible Candidates for  $\lambda, \nu$ . We now define a set  $U \subset [n]$ , and let  $\lambda_i = \frac{\alpha}{|U|} > 0$  for  $i \in U$  and  $0$  otherwise. For  $M \in (1, \lfloor \frac{n}{2k} \rfloor]$ , we take  $2Mk$  samples from the training data to be included in  $U$ . Formally,
$U = S_{\mathrm{leftover}} \cup (2M - 1)k$  a random sample from  $S_{\mathrm{shortcut}}$ ,
which gives the size  $|U| = 2Mk$ . Then, we let  $\nu = \alpha \frac{2(M - 1)}{2M} > 0$ .
Note that for the above choice of  $\lambda$ ,  $X^{\top} \lambda$  is a sum of the rows from  $U$  scaled by  $\frac{\alpha}{|U|}$ . Adding up  $k$  rows from  $S_{\mathrm{leftover}}$  and  $k$  rows from  $S_{\mathrm{shortcut}}$  cancels out the  $B$ s and, so in the  $B$  is accumulated  $|U| - 2k = 2(M - 1)k$  times, and so
$$
X ^ {\top} \lambda = \left[ \alpha * \frac {| U | - 2 k}{| U |} B, \alpha , \frac {\alpha}{| U |} \sum_ {i \in U} \pmb {\delta} _ {i} \right] = \left[ \alpha B \frac {2 (M - 1)}{2 M}, \alpha , \frac {\alpha}{| U |} \sum_ {i \in U} \pmb {\delta} _ {i} \right].
$$
As  $\lambda$  has  $\frac{\alpha}{|U|}$  on  $|U|$  elements and 0 otherwise,  $\lambda^{\top} \mathbf{1} = \alpha$
As we set  $\nu = \alpha \frac{2(M - 1)}{2M}$
$$
\nu \zeta + X ^ {\top} \lambda = \left[ - \alpha B \frac {2 (M - 1)}{2 M} + \alpha \frac {2 (M - 1)}{2 M} B, \alpha \frac {2 (M - 1)}{2 M} + \alpha , 0 + \frac {\alpha}{| U |} \sum_ {i} \delta_ {i} \right] \tag {33}
$$
$$
\begin{array}{l} = \left[ 0, \alpha \left(1 + \frac {2 (M - 1)}{2 M}\right), \quad \frac {\alpha}{| U |} \sum_ {i} \boldsymbol {\delta} _ {i} \right] (34) \\ \Longrightarrow \| \zeta \nu + X ^ {\top} \lambda \| ^ {2} = \left\| \left[ 0, \alpha \left(1 + \frac {2 (M - 1)}{2 M}\right), \frac {\alpha}{| U |} \sum_ {i \in U} \boldsymbol {\delta} _ {i} \right] \right\| ^ {2} (35) \\ = \alpha^ {2} \left\| \left[ 0, \left(1 + \frac {2 (M - 1)}{2 M}\right), \frac {1}{| U |} \sum_ {i \in U} \boldsymbol {\delta} _ {i} \right] \right\| (36) \\ \end{array}
$$
For the chosen values of  $\nu, \lambda$  the value of the objective in eq. (32) is
$$
\frac {- \alpha^ {2}}{4} \left\| \left[ 0, \left(1 + \frac {2 (M - 1)}{2 M}\right), \frac {1}{| U |} \sum_ {i \in U} \boldsymbol {\delta} _ {i} \right] \right\| ^ {2} + \alpha \tag {37}
$$
Letting
$$
\Gamma = \left\| \left[ 0, \left(1 + \frac {2 (M - 1)}{2 M}\right), \frac {1}{| U |} \sum_ {i \in U} \boldsymbol {\delta} _ {i} \right] \right\| ^ {2},
$$
the objective is of the form  $\alpha -\frac{\alpha^2\Gamma}{4}$ . To maximize with respect to  $\alpha$ , setting the derivative of the objective w.r.t  $\alpha$  to 0 gives:
$$
1 - \frac {2 \alpha \Gamma}{4} = 0 \Longrightarrow \alpha = \frac {2}{\Gamma} \Longrightarrow \alpha - \frac {\alpha^ {2} \Gamma}{4} = \frac {2}{\Gamma} - \frac {4}{\Gamma^ {2}} \frac {\Gamma}{4} = \frac {1}{\Gamma}.
$$
This immediately gives us
$$
\| \mathbf {w} _ {\mathrm {s t a b l e}} \| ^ {2} \geq \frac {1}{\Gamma},
$$
and we lower bound this quantity by upper bounding  $\Gamma$ .
By concentration of gaussian norm as in lemma 4, with probability at least  $1 - 2\exp \left(-c\frac{\epsilon^2}{G^4}\right)$
$$
\left\| \frac {1}{| U |} \sum_ {i \in U} \boldsymbol {\delta} _ {i} \right\| = \frac {1}{\sqrt {| U |}} \left\| \frac {1}{\sqrt {| U |}} \sum_ {i \in U} \boldsymbol {\delta} _ {i} \right\| \leq \frac {1}{\sqrt {| U |}} (\sqrt {d} + \epsilon).
$$
In turn, recalling that  $|U| = 2Mk$
$$
\Gamma \leq \left(\frac {(2 (M - 1) + 2 M)}{2 M}\right) ^ {2} + \left(\frac {\sqrt {d} + \epsilon}{\sqrt {| U |}}\right) ^ {2} <   4 + \left(\frac {\sqrt {d} + \epsilon}{\sqrt {| U |}}\right) ^ {2} \leq 4 + \frac {\left(\sqrt {d} + \epsilon\right) ^ {2}}{2 M k}
$$
The upper bound on  $\Gamma$  gives the following lower bound on  $\| \mathbf{w}_{\mathrm{stable}}\|^2$ :
$$
\| \mathbf {w} _ {\mathrm {s t a b l e}} \| ^ {2} \geq \frac {1}{\Gamma} \geq \frac {1}{4 + \frac {(\sqrt {d} + \epsilon) ^ {2}}{2 M k}}
$$
A.4 Upper bounding the norm of solutions that rely more on the shortcut.
Lemma 8. Consider the following optimization problem from eq. (16) where  $n$  samples of  $\mathbf{x}_i, \mathbf{y}_i$  come from eq. (1) where  $\mathbf{x}_i \in \mathbf{R}^d$ :
$$
\begin{array}{l} \mathbf {w} _ {s h o r t c u t} = \underset {\mathbf {w}} {\arg \min} \quad w _ {y} ^ {2} + w _ {z} ^ {2} + \left\| \mathbf {w} _ {e} \right\| ^ {2} \\ s. t. i \in S _ {\text {s h o r t c u t}} \quad w _ {y} + B w _ {z} + \mathbf {w} _ {e} ^ {\top} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} > 1 \tag {38} \\ i \in S _ {\text {l e f t o v e r}} \quad w _ {y} - B w _ {z} + \mathbf {w} _ {e} ^ {\top} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} > 1 \\ \mathbf {w} _ {y} <   B \mathbf {w} _ {z} \\ \end{array}
$$
Proof. Let  $k = |S_{\mathrm{leftover}}|$ . The candidate we will evaluate the objective for is
$$
\mathbf {w} = \left[ \frac {\beta}{B}, 0, \gamma \sum_ {j \in S _ {\text {l e f t o v e r}}} \mathbf {y} _ {j} \boldsymbol {\delta} _ {j} \right]. \tag {39}
$$
High-probability bounds on the margin achieved by the candidate and norm of  $\mathbf{w}$  The margins on the shortcut group and the leftover group along with the constraints are as follows:
$$
\forall j \in S _ {\text {s h o r t c u t}} \quad m _ {j} = 0 + B * \frac {\beta}{B} + \left\langle \mathbf {y} _ {j} \boldsymbol {\delta} _ {j}, \gamma \sum_ {i \in S _ {\text {l e f t o v e r}}} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} \right\rangle \geq 1 \tag {40}
$$
$$
\forall j \in S _ {\text {l e f t o v e r}} \quad m _ {j} = 0 - B * \frac {\beta}{B} + \left\langle \mathbf {y} _ {j} \boldsymbol {\delta} _ {j}, \gamma \sum_ {i \in S _ {\text {l e f t o v e r}}} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} \right\rangle \geq 1.
$$
Due to the standard normal distribution being isotropic, and  $\mathbf{y}_j\in \{-1,1\}$ ,  $\mathbf{y}_j\pmb {\delta}_j$  has the same distribution as  $\pmb{\delta}_{j}$ . Then, we apply lemma 5 with  $V = S_{\mathrm{leftover}},U = S_{\mathrm{shortcut}}$  — which means  $T_{v} = k$  and  $T_{u} = (n - k)$  — to bound the margin terms in eq. (40) and  $\| \mathbf{w}\| ^2$  with probability at least
$$
1 - 2 (2 k + (n - k) + 2) \exp (- c \frac {\epsilon^ {2}}{G ^ {4}}).
$$
Applying the bound in eq. (9) in lemma 5 between a sum of vectors and a different i.i.d vector,
$$
\forall j \in S _ {\text {s h o r t c u t}} \quad \left| \left\langle \mathbf {y} _ {j} \boldsymbol {\delta} _ {j}, \gamma \sum_ {i \in S _ {\text {l e f t o v e r}}} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} \right\rangle \right| \leq \gamma \epsilon \sqrt {k d} \tag {41}
$$
Applying the bound in eq. (8) from lemma 5
$$
\forall j \in S _ {\text {l e f t o v e r}} \quad \left\langle \mathbf {y} _ {j} \boldsymbol {\delta} _ {j}, \gamma \sum_ {i \in S _ {\text {l e f t o v e r}}} \mathbf {y} _ {i} \boldsymbol {\delta} _ {i} \right\rangle \geq \gamma (d - 3 \epsilon \sqrt {k d}) \tag {42}
$$
The margin constraints on the shortcut and leftover from eq. (40) respectively imply
$$
\beta - \gamma \epsilon \sqrt {d k} \geq 1 \quad - \beta + \gamma (d - 3 \epsilon \sqrt {k d}) \geq 1
$$
We choose  $\beta = 1 + \gamma \epsilon \sqrt{dk}$ , which implies an inequality that  $\gamma$  has to satisfy the following, which is due to  $d - 3\epsilon \sqrt{kd} > 0$ ,
$$
- (1 + \gamma \epsilon \sqrt {d k}) + \gamma (d - 3 \epsilon \sqrt {k d}) \geq 1 \Rightarrow \gamma \geq \frac {2}{d - 4 \epsilon \sqrt {k d}}
$$
Now, we choose
$$
\gamma = \frac {2}{d - 4 \epsilon \sqrt {k d}}.
$$
Computing the upper bound on the value of the objective in the primal problem in eq. (17)
The feasible candidate's norm  $\| \mathbf{w}\| ^2$  is an upper bound on the solution's norm  $\| \mathbf{w}_{\mathrm{shortcut}}\| ^2$  and so
$$
\| \mathbf {w} _ {\text {s h o r t c u t}} \| ^ {2} \leq \| \mathbf {w} \| ^ {2} = \frac {1}{B ^ {2}} \beta^ {2} + \left\| \gamma \sum_ {j \in S _ {\text {l e f t o v e r}}} \mathbf {y} _ {j} \boldsymbol {\delta} _ {j} \right\| ^ {2} = \gamma^ {2} k \left\| \frac {1}{\sqrt {k}} \sum_ {j \in S _ {\text {l e f t o v e r}}} \boldsymbol {\delta} _ {j} \right\| ^ {2} + \frac {\beta^ {2}}{B ^ {2}}
$$
By lemma 5 which we invoked,
$$
\left\| \frac {1}{\sqrt {k}} \sum_ {j \in S _ {\text {l e f t o v e r}}} \boldsymbol {\delta} _ {j} \right\| ^ {2} \leq (\sqrt {d} + \epsilon) ^ {2}.
$$
To conclude the proof, substitute  $\beta = 1 + \gamma \epsilon \sqrt{dk}$  and get the following upper bound with  $\gamma = \frac{2}{d - 3\epsilon\sqrt{kd}}$ :
$$
\left\| \mathbf {w} _ {\text {s h o r t c u t}} \right\| ^ {2} \leq \gamma^ {2} k (\sqrt {d} + \epsilon) ^ {2} + \frac {\beta^ {2}}{B ^ {2}} = \gamma^ {2} k (\sqrt {d} + \epsilon) ^ {2} + \frac {\left(1 + \gamma \epsilon \sqrt {d k}\right) ^ {2}}{B ^ {2}}.
$$
A.5 Concentration of  $k$  and intuition behind theorem 1
Concentration of  $k$  around  $(1 - \rho)n$ . Denote the event that the  $i$ th sample lies in the leftover group as  $I_i$ : then  $E[I_i] = 1 - \rho$  and the leftover group size is  $k = \sum_{i} I_i$ . Hoeffding's inequality (Theorem 2.2.6 in [Vershynin, 2018]) shows that for any  $t > 0$ ,  $k$  is at most  $(1 - \rho)n + t\sqrt{n}$  with probability at least  $1 - \exp(-2t^2)$ :
$$
p \left(k - (1 - \rho) n > t \sqrt {n}\right) = p \left(\sum_ {i} \left(I _ {i} - (1 - \rho)\right) > t \sqrt {n}\right) = p \left(\sum_ {i} \left(I _ {i} - E [ I _ {i} ]\right) > t \sqrt {n}\right) \leq \exp (- 2 t ^ {2}).
$$
Letting  $\rho = 0.9 + \sqrt{\frac{\log 3n}{n}}$  and  $t = \sqrt{\log 3n}$ , gives us
$$
\begin{array}{l} p \left(k - (1 - \rho) n > t \sqrt {n}\right) = p \left(k - 0. 1 n + \sqrt {n \log 3 n} > \sqrt {\log 3 n} \sqrt {n}\right) \\ = p (k - 0. 1 n > 0) \\ \leq \exp (- 2 t ^ {2}) \\ = \exp (- 2 \log 3 n). \\ = \left(\frac {1}{3 n}\right) ^ {2} \\ <   \frac {1}{3 n} \\ \end{array}
$$
To connect  $\rho$  to shortcut learning due to max-margin classification, we take a union bound of the event that  $k < 0.1n$ , which occurs with probability at least  $1 - \frac{1}{3n}$  and theorem 1 which occurs with probability at least  $1 - \frac{1}{3n}$ . This union bound guarantees that with probability at least  $1 - \frac{2}{3n}$  over sampling the training data, max-margin classification on  $n$  training samples from eq. (1) relies more on the shortcut feature if  $\rho$  is above a threshold; and this threshold converges to 0.9 at the rate of  $\sqrt{\log 3n / n}$ .
A.6 Bumpy losses improve ERM in the under-parameterized setting
Proof. Letting  $X$  be the matrix where each row is  $\mathbf{y}_i\mathbf{x}_i$ , the theorem statement says the solution  $\mathbf{w}^*$ .
$$
X \mathbf {w} ^ {*} = b \mathbf {1} \tag {43}
$$
First, split  $\mathbf{w}^{*} = [w_{z}^{*}, w_{y}^{*}, \mathbf{w}_{-y}^{*}]$ . Equation (43) says that the margin of the model on any sample satisfies
$$
\mathbf {y} \big (\mathbf {w} ^ {*} \big) ^ {\top} \mathbf {x} = w _ {y} ^ {*} \mathbf {y} ^ {2} + w _ {z} ^ {*} \mathbf {y} \mathbf {z} + \mathbf {y} \big (\mathbf {w} _ {- y} ^ {*} \big) ^ {\top} \boldsymbol {\delta} = b \quad \Longrightarrow \quad \mathbf {y} \big (\mathbf {w} _ {- y} ^ {*} \big) ^ {\top} \boldsymbol {\delta} = b - w _ {y} ^ {*} \mathbf {y} ^ {2} - w _ {z} ^ {*} \mathbf {y} \mathbf {z}
$$
We collect these equations for the whole training data by splitting  $X$  into columns: denoting  $Y, Z$  as vectors of  $\mathbf{y}_i$  and  $\mathbf{z}_i$  and using  $\cdot$  to denote element wise operation, split  $X$  into columns that correspond to  $\mathbf{y}, \mathbf{z}$  and  $\delta$  respectively as  $X = [Y \cdot Y \mid Y \cdot Z \mid X_{\delta}]$ . Rearranging terms gives us
$$
w _ {z} ^ {*} Y \cdot Z + w _ {y} ^ {*} \mathbf {1} + X _ {\delta} \mathbf {w} _ {\delta} ^ {*} = b \mathbf {1} \quad \Longrightarrow \quad X _ {\delta} \mathbf {w} _ {\delta} ^ {*} = (b - w _ {y} ^ {*}) \mathbf {1} - w _ {z} ^ {*} Y \cdot Z.
$$
The elements of  $Y \cdot Z$  lie in  $\{-1, 1\}$  and, as the shortcut feature does not always equal the label, the elements of  $Y \cdot Z$  are not all the same sign.
Solutions do not exist when one non-zero element exists in  $(b - w_{y}^{*})\mathbf{1} - w_{z}^{*}Y\cdot Z$  By definition of  $\mathbf{w}^*$
$$
X _ {\delta} \mathbf {w} _ {\delta} ^ {*} = (b - w _ {y} ^ {*}) \mathbf {1} - w _ {z} ^ {*} Y \cdot Z.
$$
Denote  $r = (b - w_y^*)\mathbf{1} - w_z^* Y \cdot Z$  and  $A = X_{\delta}$ . Now we show that w.p. 1 solutions do not exist for the following system of linear equations:
$$
A w = r.
$$
First, note that  $A = X_{\delta}$  has  $\mathbf{y}_i \delta_i$  for rows and as  $\mathbf{y}_i \perp \mathbf{\delta}_i$  and  $\mathbf{y}_i \in \{-1, 1\}$ , each vector  $\mathbf{y}_i \delta_i$  is distributed identically to a vector of independent standard Gaussian random variables. Thus,  $A$  is a matrix of IID standard Gaussian random variables.
Let  $U$  denote  $D - 2$  indices such that the corresponding rows of  $A$  form a matrix  $D - 2 \times D - 2$  matrix and  $r_U$  has at least one non-zero element; let  $A_U$  denote the resulting matrix. Now  $A_U$  is a  $D - 2 \times D - 2$  sized matrix where each element is a standard Gaussian random variable. Such matrices have rank  $D - 2$  with probability 1 because square singular matrices form a measure zero set under the Lebesgue measure over  $\mathbf{R}^{D - 2 \times D - 2}$  [Feng and Zhang, 2007].