byt5 finetuned on MNLI dataset for 3 epochs, with lr=1e-4 

valid matched acc = 0.80