SlowGuess/NeurIPS_2023_OCR huggingface dataset:
RxRx1-WILDS: We used the id_text group as the validation set. We followed [25] to fine-tune an ImageNet pretrained ResNet50. We used Adam with weight decay of  $10^{-5}$  and batch size of 75, for 90 epochs. We increased the learning rate from 0 to  $10^{-4}$  linearly for the first 10 epochs and decayed it following a cosine learning rate schedule.
Amazon-WILDS: We used the id_val group as the validation set. We followed [25] to fine-tune a DistilBERT-base-uncased model [35]. We used AdamW [28] with weight decay of  $10^{-2}$ , learning rate of  $10^{-5}$ , and batch size of 8, for 3 epochs. We set the maximum number of tokens to 512.
CivilComments-WILDS: We used the val group as the validation set. We followed [25] to fine-tune a DistilBERT-base-uncased model [35]. We used AdamW [28] with weight decay of  $10^{-2}$ , learning rate of  $10^{-5}$ , and batch size of 16, for 5 epochs. We set the maximum number of tokens to 300.