SlowGuess/NAACL_2022_OCR huggingface dataset:
All experiments were performed on Linux operating system using Trax library version 1.3.9 along with all its dependencies from this particular release date. Additionally, to run shorten factor dropout experiments we modified the Transformer-XL codebase in PyTorch.
To ensure the reproducibility of this work and to support open science principles, we made our code publicly available at github.com/google/trax. In this repository, we also provide Google Colab notebooks where the evaluation of our main Enwik8 and ImageNet32/64 results can be reproduced.[23]
D.1 Randomness
Seeds in all experiments were chosen randomly, however each experiment contains history which allows retrieving all randomly set parameters for reproductions.
For each ablation described in the ablation study section, we rerun the baseline 3 times to calculate standard deviation. All other experiments are run only once due to costs and since the variance we noticed was minimal.
D.2 Experiment representation
Each experiment is represented by a configuration file that unambiguously determines the whole setup - all hyperparameters and training details like specific optimizers, data preprocessing functions, or batch size per device.
2https://github.com/google/trax/blob/master/trax/models/research/examples/hourglass_enwik8.ipynb
3https://github.com/google/trax/blob/master/trax/models/research/examples/hourglass_downsampled_imagenet.ipynb