SlowGuess/NAACL_2022_OCR huggingface dataset:
We implemented our models using TensorFlow v2.7 (Abadi et al., 2015) and the Hugging Face transformers library (Wolf et al., 2020). We also fine-tuned all model hyperparameters using Vizier (Golovin et al., 2017), a black-box optimization system, using streaming F1 score on the Switchboard validation set as our objective. The searched ranges for each hyperparameter were learning rate  $\in [1 \times 10^{-5}, 1 \times 10^{-1}]$ , number of training epochs  $\in [12, 20]$ ,  $\lambda \in [1 \times 10^{-8}, 1 \times 10^{-6}]$ ,  $\gamma \in [1, 10]$ . For most experiments we ran 30 trials total, with 10 evaluations in parallel. Each individual trial (one set of hyper-parameters) ran on a single NVIDIA P100 GPU. Experimental run time varied from about 13 to 24 hours, depending mostly on the number of epochs. For each model variant we present only the results from the configuration with the highest streaming  $F_{1}$  score on the Switchboard validation dataset. Our best performing streaming model used parameter values of  $\lambda = 1.5 \times 10^{-7}$ , learning rate  $1.2 \times 10^{-4}$ ,  $\gamma = 1.9$ , training batch size 8, and 12 epochs.
Time step	Model outputs
3	Input: “I think [the real,” Output: “I think the real”
4	Input: “I think [the real, + the] Output: “I think the <WAIT>”
5	Input: “I think [the real, + the princi-pal]” Output: “I think <DIS><DIS> the principal”
Table 4: Example of the model's outputs at each time step. (For brevity, we excerpt only a segment of the sentence that contains disfluencies.) A <WAIT> symbol indicates that the model decided to stop making predictions for the rest of the input sequence and to wait for further input instead. A <DIS> symbol indicates that the corresponding input token was given a classification of disfluent and therefore not included in the final edited output. For clarity we provide disfluency annotations in the form [Reparandum, + Repair], but these are not actually provided to the model as input.