SlowGuess/ACL_2020_OCR huggingface dataset:
15 Intuitively,  $[\pi_t]_a$  corresponds to the operations push, noop, and pop, for the values  $a = 1,2,3$  respectively.
<sup>14</sup>Note that any periodic positional encoding will also have finite image.
Proof. Choose  $k = 1$ . Fix the controller to always push  $x_{t}$ . Then, the encoding at time  $t$  will be
$$
\mathbf {c} _ {t} = \sum_ {i = 1} ^ {t} \left(\frac {1}{2}\right) ^ {i - 1} x _ {i}. \tag {84}
$$
This is the value of the prefix  $x_{:t}$  in binary.
Rational Recurrence We provide another construction to show that the stack RNN can compute non-rational series. Thus, it is not RR.
Definition 7 (Geometric counting). Define  $f_2: \{a, b\}^* \to \mathbb{N}$  such that
$$
f _ {2} (x) = \exp_ {\frac {1}{2}} (\# _ {a - b} (x)) - 1.
$$
Like similar functions we analyzed in Section 3, the Hankel matrix  $H_{f_2}$  has infinite rank over the sub-block  $a^i b^j$ .
Lemma 6. The saturated stack RNN can compute  $f_{2}$ .
Proof. Choose  $k = 1$ . Fix the controller to push 1 for  $x_{t} = a$ , and pop otherwise.