SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Relative Information Gain for each problem size
Input variable	Output variable	3	4	5	6	7	8	9	10
an-3an-2an-1	on-1	0.95	0.95	0.95	0.95
an-3an-2an-1	on-2	0.87	0.87	0.87	0.87
an-3an-2an-1	on-3	0.64	0.64	0.64	0.64
a1a2a3	o1	1.00	0.96	0.97	0.97	0.97	0.97	0.97	0.97
a1a2a3	o2	1.00	0.91	0.92	0.91	0.92	0.91	0.92	0.91
a2a3a4	o2	0.56	0.55	0.55	0.55	0.55	0.55	0.56
a1a2a3	o3	1.00	0.66	0.73	0.71	0.72	0.72	0.72	0.72
a2a3a4	o3	0.86	0.77	0.78	0.78	0.78	0.78	0.78
a3a4a5	o3	0.67	0.66	0.66	0.66	0.66	0.66
a2a3a4	o4	0.94	0.64	0.7	0.68	0.69	0.69	0.69
a3a4a5	o4	0.88	0.79	0.81	0.8	0.8	0.8
a4a5a6	o4	0.63	0.62	0.62	0.62	0.62
a3a4a5	o5	0.95	0.65	0.71	0.69	0.7	0.7
a4a5a6	o5	0.87	0.78	0.79	0.79	0.79
a5a6a7	o5	0.64	0.63	0.63	0.64
a4a5a6	o6	0.94	0.64	0.71	0.69	0.7
a5a6a7	o6	0.87	0.78	0.8	0.8
a6a7a8	o6	0.64	0.62	0.63
a5a6a7	o7	0.95	0.64	0.71	0.69
a6a7a8	o7	0.87	0.78	0.8
a6a7a8	o8	0.95	0.64	0.71
a1a2a3	o4	0.12	0.1	0.11	0.11	0.11	0.11	0.11
a2a3a4	o5	0.1	0.09	0.1	0.09	0.1	0.1
a3a4a5	o6	0.11	0.1	0.1	0.1	0.11
a4a5a6	o7	0.11	0.09	0.1	0.11
a5a6a7	o8	0.11	0.09	0.11
Table 4: Highest Relative Information Gain Contiguous Triples, for DP problems of size  $3 \leq n \leq 10$ . We only show the (input, output) pairs where at least three problem sizes have RelativeIG>0, and at least one with RelativeIG>0.1.  $a_{n-1}$  refers to the last element of the sequence, regardless of its actual id in the sequence.
C.4 Empirical Surface Pattern Results for Dynamic Programming Task
We observe that all analyzed models match the Relative Information Gain prediction that  $o_1$  (whether the first element goes into the output sequence or not) should be the easiest value to predict (see Figures 30, 31, and 32). However, since GPT3 often predicts shorter output sequences than the required size, the analysis of the predictive power of  $o_{n-1}$  is only done for GPT4. In GPT4, we observe that  $o_{n-1}$  is among the easiest values to predict as expected by Relative Information Gain.
Figure 30: GPT4 five-shot with scratchpad accuracy in predicting output elements  $o_i$  in the DP task. All  $o_i$  are predicted with high accuracy with  $o_1$  and  $o_{n-1}$  being consistently among the highest. These observations go in line with the Relative Information Gain prediction.
Figure 31: GPT3 few-shot without scratchpad accuracy in predicting output elements  $o_i$  in the DP task. As predicted by Relative Information Gain, the model predicts  $o_1$  correctly with the highest probability. However, because GPT3 often does not produce the correct output size, it hinders us from analyzing  $o_{n-1}$ .
Figure 32: GPT3 fine-tuned without scratchpad accuracy in predicting output elements  $o_i$  in the DP task. As predicted by Relative Information Gain, the model predicts  $o_1$  correctly with the highest probability. However, because GPT3 often does not produce the correct output size, it hinders us from analyzing  $o_{n-1}$ .
D Theoretical Results: Derivations
D.1 Error accumulates with larger parallel applications of an estimated function (width)
Here we provide formal statements and derivations to Propositions 4.1 and 4.2 shown in the main paper. The mathematical framework used is a simplified representation of how multi-step reasoning works, showing two quintessential reasoning types: independent applications of the same step, or consecutive applications of the same step. We take an error estimation and accumulation perspective, since transformers are still being investigated from a theoretical standpoint.
Proposition D.1. Let  $f_{n}(\mathbf{x}) = h_{n}(g(\mathbf{x},1),g(\mathbf{x},2)),\ldots ,g(\mathbf{x},n))$ . Let  $\widehat{h}_n,\widehat{g},\widehat{f}_n$  be estimators of  $h_n,g,f_n$  respectively. Assume  $\mathbb{P}(h_n = \widehat{h}_n) = 1$  and  $\mathbb{P}(h_n(X) = h_n(Y)\mid X\neq Y) < c_n$ , where  $c_{n} < c$  for some constant  $c\ll 1$  (i.e.  $\widehat{h}_n$  perfectly estimates  $h_n$ , and  $h_n$  is almost injective). If  $\mathbb{P}(g\neq \widehat{g}) = \epsilon >0$  and errors in  $\widehat{g}$  are independent,  $\lim_{n\to +\infty}\inf_{n\to +\infty}\mathbb{P}(f_n\neq \widehat{f}_n)\geq 1 - c$ .
Moreover, if  $c_n \leq \beta \alpha^n$  for some some  $\alpha \in (0,1)$  and  $\beta > 0$ , then  $\lim_{n \to +\infty} \mathbb{P}(f_n \neq \widehat{f}_n) = 1$ .
Proof. For ease of writing, let  $X_{i} = g(X,i)$  and  $Y_{i} = \widehat{g} (X,i)$ , and let  $\mathbf{X} = (X_{1},\ldots ,X_{n})$  and  $\mathbf{Y} = (Y_{1},\dots ,Y_{n})$ . We will compute some auxiliary probabilities, and then upper bound  $\mathbb{P}(f = \widehat{f})$  to finally compute its limit.
$$
\begin{array}{l} \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y}) = \mathbb {P} (X _ {1} = Y _ {1}, X _ {2} = Y _ {2}, \dots , X _ {n} = Y _ {n}) \\ = \mathbb {P} \left(X _ {1} = Y _ {1}\right) \cdot \mathbb {P} \left(X _ {2} = Y _ {2}\right) \dots \cdot \mathbb {P} \left(X _ {n} = Y _ {n}\right) = \mathbb {P} (g = \widehat {g}) ^ {n} = (1 - \epsilon) ^ {n} \tag {2} \\ \end{array}
$$
Since by hypothesis we know  $\mathbb{P}(h_n(\mathbf{Y}) = \widehat{h}_n(\mathbf{Y})) = 1$ , we have that:
$$
\begin{array}{l} \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) = \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y}) \cap h _ {n} (\boldsymbol {Y}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) \\ = \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = h _ {n} (\boldsymbol {Y}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) \\ \leq \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = h _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) \\ <   c _ {n} \tag {3} \\ \end{array}
$$
We will now estimate  $\mathbb{P}(f_n = \widehat{f}_n)$  using the law of total probability w.r.t. the event  $X = Y$ .
$$
\begin{array}{l} \mathbb {P} \left(f _ {n} = \widehat {f _ {n}}\right) = \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y})\right) \\ = \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} = \boldsymbol {Y}\right) \cdot \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y}) + \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) \cdot \mathbb {P} (\boldsymbol {X} \neq \boldsymbol {Y}) \\ = \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {X})\right) \cdot \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y}) + \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) \cdot (1 - \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y})) \\ = 1 \cdot (1 - \epsilon) ^ {n} + \mathbb {P} \left(h _ {n} (\boldsymbol {X}) = \widehat {h} _ {n} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}\right) \cdot (1 - (1 - \epsilon) ^ {n}) \quad \text {(u s i n g 2 a n d h y p o t h e s i s)} \\ <   (1 - \epsilon) ^ {n} + c _ {n} \cdot (1 - (1 - \epsilon) ^ {n}) \quad (\text {u s i n g} 3) \\ <   c _ {n} + (1 - \epsilon) ^ {n} \cdot (1 - c _ {n}) \\ \end{array}
$$
To conclude our proof, we will compute a lower bound for  $\lim \inf_{n\to +\infty}\mathbb{P}(f_n\neq \widehat{f}_n)$ . Note that since  $c_{n} < c$  for all  $n$ , we know that  $\mathbb{P}(f_n = \widehat{f}_n) < c + (1 - \epsilon)^n\cdot (1 - c)$ . Then,  $\mathbb{P}(f_n\neq \widehat{f}_n) > 1 - c - (1 - \epsilon)^n\cdot (1 - c)$ . Since  $1 - \epsilon \in [0,1)$ ,  $\lim_{n\to +\infty}1 - c - (1 - \epsilon)^n\cdot (1 - c) = 1 - c$ . Thus,
$$
\lim  _ {n \rightarrow + \infty} \inf  _ {n \rightarrow + \infty} \mathbb {P} (f _ {n} \neq \widehat {f} _ {n}) \geq \lim  _ {n \rightarrow + \infty} \inf  _ {1 - c - (1 - \epsilon) ^ {n} \cdot (1 - c) = 1 - c
$$
which concludes our proof.
Note: In the case where  $c_{n} \leq \beta \alpha^{n}$ , we can derive an even stronger conclusion. In this case, we can prove that  $\lim_{n \to +\infty} \mathbb{P}(f_n = \widehat{f_n}) = 1$ . Recall that  $\mathbb{P}(f_n = \widehat{f_n}) < \alpha \beta^n + (1 - \epsilon)^n \cdot (1 - \alpha \beta^n)$ . Note that since  $1 - \epsilon \in [0,1)$  and  $\alpha \in (0,1)$ , trivially  $\lim_{n \to +\infty} \beta \alpha^n + (1 - \epsilon)^n \cdot (1 - \beta \alpha^n) = 0$ .
$$
0 \leq \lim  _ {n \rightarrow + \infty} \inf  _ {n \rightarrow + \infty} \mathbb {P} (f _ {n} = \widehat {f} _ {n}) \leq \lim  _ {n \rightarrow + \infty} \sup  _ {n \rightarrow + \infty} \mathbb {P} (f _ {n} = \widehat {f} _ {n}) \leq \lim  _ {n \rightarrow + \infty} \sup  _ {n \rightarrow + \infty} \beta \alpha^ {n} + (1 - \epsilon) ^ {n} \cdot (1 - \beta \alpha^ {n}) = 0
$$
Then,  $\lim_{n\to +\infty}\mathbb{P}(f_n = \widehat{f}_n) = 0$  and we conclude  $\lim_{n\to +\infty}\mathbb{P}(f_n\neq \widehat{f}_n) = 1$ , assuming that  $c_{n}\leq \beta \alpha^{n}$  for some adequate  $\alpha ,\beta$
Corollary D.1. Assume that a model  $\mathcal{M}$  solves shifted addition perfectly, but it incorrectly solves at least one  $m$  digit by 1 digit multiplication for some fixed  $m$ . Then, the probability that  $\mathcal{M}$  will solve any  $m$  digit by  $n$  digit multiplication using the long-form multiplication algorithm tends to 0 when  $n$  tends to infinity.
Proof. Let  $g = d \circ s$  define the base-10 multiplication between  $m$ -digit numbers  $(x_{1}x_{2}\ldots x_{m})$  and 1-digit numbers  $(x_{m + j})$ , where  $s: \mathbb{Z}_{10}^{m + n} \times \mathbb{N} \to \mathbb{N} \times \mathbb{N}$  denotes the selection of the numbers to multiply and  $d: \mathbb{N} \times \mathbb{Z}_{10} \to \mathbb{N}$  denotes the actual multiplication:
$$
g := d \circ s
$$
$$
d (x, y) := x \cdot y
$$
$$
s \left(\left[ x _ {1}, \dots , x _ {m}, x _ {m + 1}, \dots , x _ {m + n} \right], j\right) := \left(x _ {1} + x _ {2} + \dots + x _ {m}, x _ {m + j}\right)
$$
where  $x_{1} + x_{2} + \ldots +x_{m}$  denotes concatenating digits  $x_{i}$
Let  $h_n: \mathbb{N}^n \to \mathbb{N}$  describe the shifted addition used at the end of long-form multiplication to combine  $n$ $m$ -digit by 1-digit multiplications, and let  $f_n: \mathbb{Z}_{10}^{m+n} \to \mathbb{N}$  describe the long-form multiplication of  $m$ -digit by  $n$ -digit numbers:
$$
h _ {n} \left(x _ {1}, \dots , x _ {n}\right) := \sum_ {i = 1} ^ {n} x _ {i} 1 0 ^ {n - i}
$$
$$
f _ {n} (\mathbf {x}) := h _ {n} (g (\mathbf {x}, 1), g (\mathbf {x}, 2)), \dots , g (\mathbf {x}, n))
$$
By hypothesis,  $\mathbb{P}(g\neq \widehat{g}) = \epsilon >0$  and  $\mathbb{P}(h_n = \widehat{h}_n) = 1$  , where  $\widehat{g}$  and  $\widehat{h}_n$  denote estimators using model  $\mathcal{M}$  . It can be shown that  $\mathbb{P}(h_n(X) = h_n(Y)\mid X\neq Y) <   \beta \alpha^n$  for  $\alpha = 0.1$  and  $\beta = 10^{m}$  Using Lemma D.1,  $\lim_{n\to +\infty}\mathbb{P}(f_n\neq \widehat{f}_n) = 1$  , which concludes our proof.
Note that Lemma D.1's proofs gives us empirical bounds once  $\epsilon$  and  $\alpha$  are approximated. Also note that our definition of  $g$  in the proof of Corollary D.1 highlights two possible sources of exponentially-accumulating error: errors in the selection of the numbers to multiply  $s$ , and errors in the actual  $m$ -digit by 1-digit multiplication  $d$ .
D.2 Error accumulates with larger iterative applications of an estimated function (depth)
Proposition D.2. Let  $f_{n}(\mathbf{x}) = g^{n}(\mathbf{x})$ . Assume  $\mathbb{P}(g(X) = \widehat{g}(Y) \mid X \neq Y) \leq c$  (i.e. recovering from a mistake due to the randomness of applying the estimator on an incorrect input has probability at most  $c$ ). If  $\mathbb{P}(g \neq \widehat{g}) = \epsilon > 0$  with  $c + \epsilon < 1$ , then  $\lim_{n \to +\infty} \inf_{n \to +\infty} \mathbb{P}(f_n \neq \widehat{f}_n) \geq 1 - c / (c + \epsilon)$ .
Proof. We first derive a recursive upper bound using the law of total probability, and then prove a non-recursive upper bound by induction.
$$
\begin{array}{l} s _ {n} := \mathbb {P} \left(f _ {n} = \widehat {f} _ {n}\right) = \mathbb {P} \left(g \left(g ^ {n - 1} (Z)\right) = \widehat {g} \left(\widehat {g} ^ {n - 1} (Z)\right)\right) \\ = \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {Y})) \quad \text {w h e r e} \boldsymbol {X} := g ^ {n - 1} (Z) \text {a n d} \boldsymbol {Y} := \widehat {g} ^ {n - 1} (Z) \\ = \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {Y}) \mid \boldsymbol {X} = \boldsymbol {Y}) \cdot \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y}) + \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}) \cdot \mathbb {P} (\boldsymbol {X} \neq \boldsymbol {Y}) \\ = \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {X})) \cdot \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y}) + \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}) \cdot (1 - \mathbb {P} (\boldsymbol {X} = \boldsymbol {Y})) \\ = \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {X})) \cdot s _ {n - 1} + \mathbb {P} (g (\boldsymbol {X}) = \widehat {g} (\boldsymbol {Y}) \mid \boldsymbol {X} \neq \boldsymbol {Y}) \cdot (1 - s _ {n - 1}) \\ \leq (1 - \epsilon) \cdot s _ {n - 1} + c \cdot (1 - s _ {n - 1}) \\ \leq (1 - \epsilon - c) \cdot s _ {n - 1} + c \\ \end{array}
$$
We know  $s_1 = (1 - \epsilon)$  since  $s_1 = \mathbb{P}(f_1 = \widehat{f}_1) = \mathbb{P}(g = \widehat{g})$ . Let  $b := 1 - \epsilon - c$  for ease of writing. Then, we have
$$
s _ {n} \leq b \cdot s _ {n - 1} + c \tag {4}
$$
It can be easily shown by induction that  $s_n \leq b^{n-1}(1-\epsilon) + c\sum_{i=0}^{n-2}b^i$ :
- The base case  $n = 2$  is true since we know  $s_2 \leq b \cdot s_1 + c$ , and  $b \cdot s_1 + c = b(1 - \epsilon) + c = b^{2 - 1}(1 - \epsilon) + c\sum_{i = 0}^{2 - 2}b^i$ , thus showing  $s_2 \leq b^{2 - 1}(1 - \epsilon) + c\sum_{i = 0}^{2 - 2}b^i$ .
- The inductive step yields directly using Equation 4,
$$
\begin{array}{l} s _ {n} \leq b \cdot s _ {n - 1} + c \\ \leq b \cdot \left(b ^ {n - 2} (1 - \epsilon) + c \sum_ {i = 0} ^ {n - 3} b ^ {i}\right) + c \leq b ^ {n - 1} (1 - \epsilon) + c \sum_ {i = 1} ^ {n - 2} b ^ {i} + c \leq b ^ {n - 1} (1 - \epsilon) + c \sum_ {i = 0} ^ {n - 2} b ^ {i} \\ \end{array}
$$
We can rewrite the geometric series  $\sum_{i=0}^{n-2} b^i$  in its closed form  $\frac{1 - b^{n-1}}{1 - b}$ , and recalling  $b := 1 - \epsilon - c$ ,
$$
\begin{array}{l} s _ {n} \leq b ^ {n - 1} (1 - \epsilon) + c \frac {1 - b ^ {n - 1}}{1 - b} = b ^ {n - 1} (1 - \epsilon) + c \frac {1 - b ^ {n - 1}}{c + \epsilon} \\ = b ^ {n - 1} (1 - \epsilon) + \frac {c}{c + \epsilon} - b ^ {n - 1} \frac {c}{c + \epsilon} \\ = b ^ {n - 1} \left(1 - \epsilon - \frac {c}{c + \epsilon}\right) + \frac {c}{c + \epsilon} \\ \end{array}
$$
Recalling that  $s_n = \mathbb{P}(f_n = \widehat{f}_n)$ , we compute the limit inferior of  $\mathbb{P}(f_n \neq \widehat{f}_n) = 1 - s_n \geq 1 - b^{n-1} (1 - \epsilon - \frac{c}{c + \epsilon}) - \frac{c}{c + \epsilon}$ .
$$
\lim  _ {n \rightarrow + \infty} \inf  _ {n \rightarrow + \infty} \mathbb {P} (f _ {n} \neq \widehat {f} _ {n}) \geq \lim  _ {n \rightarrow + \infty} 1 - b ^ {n - 1} \left(1 - \epsilon - \frac {c}{c + \epsilon}\right) - \frac {c}{c + \epsilon} = 1 - \frac {c}{c + \epsilon}
$$
that concludes our proof.
We can generalize the proof in Lemma 4.2 to tasks where there are potentially many valid reasoning chains with the following alternative state-transition framing.
Lemma D.2. Let  $S$  denote the set of all possible states a language model can generate, and let  $z: S \to \{0,1\}$  defines if a state is valid ( $0 = \text{invalid}$ ). Let  $\widehat{g}: S \to \Pi(S)$  be a state-transition function representing a language model's probability distribution of generating each possible next state when attempting to perform a single reasoning step. Assume  $\mathbb{P}(z(\widehat{g}(X)) = 1 \mid z(X) = 0) \leq c$  and  $\mathbb{P}(z(\widehat{g}(X)) = 0 \mid z(X) = 1) = \epsilon > 0$  with  $c + \epsilon < 1$ . Then,  $\lim_{n \to +\infty} \inf_{n \to +\infty} \mathbb{P}(z(\widehat{g}^n) = 0) = 1 - c / (c + \epsilon)$ .
If for task  $T$  we know that all valid reasoning chains to arrive at a correct result have at least length  $n$  (i.e., the equivalent of defining  $f_{n} = g^{n}$  in Lemma D.1) then the probability of solving task  $T$  correctly tends to at most  $c / (c + \epsilon)$ .
Corollary D.3. The recursions for dynamic programming tasks, the  $m$ -by-1 digit multiplication, and the puzzle's elimination function are all tasks where there is a fixed reasoning step  $g$  being repeatedly applied. Therefore, we can directly apply Proposition 4.2 to these tasks.
Proof. Let's analyze the three tasks separately below.
$m$ -by-1 digit multiplication may be viewed as  $f^{m}(\mathbf{x})$  Let  $x = (x_{1}, \ldots, x_{m})$  be the  $m$ -digit number that we multiply by the 1-digit number  $y$  ( $0 \leq y < 10$ ). Let  $z = (z_{1}, \ldots, z_{m+1})$  denote  $z = x \cdot y$ , which is guaranteed to have exactly  $m + 1$  digits (with possibly leading zeros). We define  $f$  as:
$$
f \left(x _ {1}, \dots , x _ {m}, y, i, c\right) := \left(x _ {1}, \dots , x _ {i - 1}, x _ {i} ^ {\prime}, x _ {i + 1}, \dots x _ {m}, y, i - 1, c ^ {\prime}\right)
$$
where  $x_i' := (x_i \cdot y + c) \mod 10$  and  $c' := \lfloor (x_i \cdot y + c) / 10 \rfloor$ . Note that  $x_i' = z_{i+1}$  since  $f$  is performing one step of the long-form multiplication algorithm.
Let the initial input be  $\mathbf{x} \coloneqq (x_1, \ldots, x_m, y, m, 0)$ . Then, it can be easily shown that  $f^m(\mathbf{x}) = (z_2, \ldots, z_{m+1}, y, 0, c)$ . Since  $c$  is the left-most carry, it is the leading digit of  $z$ , i.e.  $c = z_1$  (possibly zero). Thus, the value of  $z$  can be directly extracted from  $f^m(\mathbf{x}) = (z_2, \ldots, z_{m+1}, y, 0, z_1)$ .
In the DP task,  $dp$ 's computation may be viewed as  $f^{m-2}(x)$  for a list of size  $m$ . See §A.3.1 for details on the solution to this problem. We will use identical notation. Let  $a_1, \ldots, a_m$  be an input list. Let  $\mathbf{x} = (a_1, \ldots, a_{m-2}, a_{m-1}', a_m', m-2)$ , where  $a_m' := \max(a_m, 0)$  and  $a_{m-1}' := \max(a_{m-1}, a_m, 0)$ . Intuitively, this means that we have applied the first two steps of the  $dp$  computation, and stored the results in  $a_{m-1}'$  and  $a_m'$ . Let  $f$  be a function representing the recursive computation of  $dp_i$ :
$$
f \left(a _ {1}, \dots , a _ {i}, a _ {i + 1} ^ {\prime}, \dots , a _ {m} ^ {\prime}, i\right) = \left(a _ {1}, \dots , a _ {i - 1}, a _ {i} ^ {\prime}, \dots , a _ {m} ^ {\prime}, i - 1\right)
$$
where  $a_i' \coloneqq \max(a_{i+1}', a_i + a_{i+2}', 0)$ .
Note that since  $a_{i+1}'$  stores the value of  $dp_{i+1}$  and  $a_{i+2}'$  stores the value of  $dp_{i+2}$ , it can be easily shown that  $f^{m-2}(\mathbf{x}) = (a_1', \ldots, a_m', 0) = (dp_1, \ldots, dp_m, 0)$ . Therefore,  $f^{m-2}$  computes all recursive values of  $dp_i$  when given the base cases.
In the DP task, the reconstruction of the desired subsequence given already computed  $dp$  values may be viewed as  $f^{m}(x)$  for an input list of size  $m$ . This case is similar to the previous one. Let  $r = (r_{1},\ldots ,r_{m})$  be the result, where  $r_i = 1$  if  $a_{i}$  was selected for the desired subsequence, and  $r_i = 2$  otherwise. Let  $\mathbf{x}\coloneqq (dp_1,\dots ,dp_m,0,0,a_1,\dots ,a_m,1,1)$ . Let  $f$  be defined as follows:
$$
f \left(d p _ {1}, \dots , d p _ {m}, 0, 0, a _ {1} ^ {\prime}, \dots , a _ {i - 1} ^ {\prime}, a _ {i}, \dots , a _ {m}, i, u\right) = \left(d p _ {1}, \dots , d p _ {m}, 0, 0, a _ {1} ^ {\prime}, \dots , a _ {i} ^ {\prime}, a _ {i + 1}, \dots , a _ {m}, i + 1, u ^ {\prime}\right)
$$
where  $a_i' := 2 - \mathbb{1}\{dp_i = a_i + dp_{i+2} \text{ and } u = 1\}$  and  $u := 1 - \mathbb{1}\{dp_i = a_i + dp_{i+2} \text{ and } u = 1\}$ . Intuitively,  $a_i'$  stores whether the  $i$ -th element of the list should be selected for the final subsequence, assigning 1 if the element should be taken, and 2 otherwise (i.e.,  $a_i' = r_i$ ). Moreover, if the  $i$ -th element has been selected, we mark that the next item will not be available using  $u'$ . Therefore,  $f$  performs one step of the final output reconstruction as defined in §A.3.1.
It can be easily shown that  $f^{m}(\mathbf{x}) \coloneqq (dp_{1},\ldots ,dp_{m},0,0,a_{1}^{\prime},\ldots ,a_{m}^{\prime},m + 1,u^{\prime}) = (dp_{1},\ldots ,dp_{m},0,0,r_{1},\ldots ,r_{m},m + 1,u^{\prime})$ . Note that the extra two elements in the input state allow lifting the special cases  $m - 1$  and  $m$  in the solution shown in §A.3.1 without falling out of bounds.
Solving the puzzle task may be seen as  $f^m$  for some  $m$ , where  $f$  is the elimination function. Let  $c_1, \ldots, c_n$  be the list of clues, let  $H$  be the number of houses, and let  $A$  be a partially filled solution of size  $K \times M$  as defined in §2.4. Each cell  $A_{ij}$  can take  $H + 1$  values: the  $H$  options for the cell and the value  $\varnothing$ , implying this cell has not been filled. An elimination step  $f$  may be defined as:
$$
f \left(c _ {1}, \dots , c _ {n}, A _ {1 1}, \dots A _ {1 M}, \dots , A _ {K 1}, \dots A _ {K M}\right) = \left(c _ {1}, \dots , c _ {n}, A _ {1 1} ^ {\prime}, \dots A _ {1 M} ^ {\prime}, \dots , A _ {K 1} ^ {\prime}, \dots A _ {K M} ^ {\prime}\right)
$$
where  $A'$  is also a partially filled matrix, with  $A_{ij} = A_{ij}'$  for every  $A_{ij} \neq \emptyset$  and where  $A'$  has at least one more filled cell.
Let  $\mathbf{x} = (c_{1},\dots ,c_{n},E)$  where  $E$  is an empty matrix of size  $K\times M$  (all cell values of  $E$  are  $\varnothing$ ).
Then, a full solution is computed as  $f^{m}(\mathbf{x})$  for some value of  $m$  that increases with the problem size. In contrast to other tasks, the value of  $m$  is not fixed, and depends on the task instance, but using solvers we know that  $m$  increases with problem size.
D.3 Discussing  $c \ll \epsilon$  in the context of Proposition 4.2
Note that in Proposition 4.2, if  $c \ll \epsilon$  then  $\lim_{n \to +\infty} \mathbb{P}(f_n \neq \widehat{f}_n) \approx 1$ . This is because assuming  $\epsilon = m \cdot c$  for some  $m > 0$ , we have  $1 - \frac{c}{c + \epsilon} = 1 - \frac{c}{c + m \cdot c} = 1 - \frac{1}{m + 1} = \frac{m}{m + 1}$ , and  $\frac{m}{m + 1}$  is a monotonically increasing function for all  $m > 0$  that tends to 1 when  $m$  goes to infinity. Therefore, large  $m$ 's (or alternatively,  $c \ll \epsilon$ ) imply  $\frac{m}{m + 1}$  will be close to 1.
It is reasonable to assume  $c \ll \epsilon$  when  $g$  has low collision, since  $c$  represents the probability of the estimator  $\widehat{g}(y)$  arriving at the correct output  $g(x)$  by chance when given the wrong input  $y \neq x$ .
If  $g$  is discrete, it can take  $|\operatorname{Im}(g)|$  values, where  $|\operatorname{Im}(g)|$  denotes the cardinal of the image space of  $g$ . Assuming approximately uniform errors,  $c \approx \epsilon / |\operatorname{Im}(g)|$ , which in turn implies  $c \ll \epsilon$  since  $g$  being low collision implies  $|\operatorname{Im}(g)|$  is large.
If  $g$  is continuous, under appropriate assumptions it seems plausible that we can prove that  $c \approx 0$  (e.g., if errors are approximately uniform).
Summarizing both cases, if errors are approximately evenly distributed we obtain that  $\lim_{n\to +\infty}\mathbb{P}(f_n\neq \widehat{f}_n)\approx 1$
E Additional Literature and Societal Impact
E.1 Additional Literature
Iterated Functions The process of repeatedly applying a noisy single operation or function  $f$  can be related to iterated random functions [24]. In this latter literature, the focus is usually on the contractive regime in which accrued errors can be kept under control, and the subsequent convergence guarantees (e.g., [23]). When  $f$  is an affine transformation, the process falls simultaneously between two perspectives: time series [31] and dynamic programming and control [6]. We leverage the former to discuss the often explosive errors of  $f^n$ .
E.2 Societal Impact Discussion