Question: what does the model learn in zero-shot setting?
Answer: we simply adopted the official training script of BERT, with default hyperparameters, to fine-tune each model until training loss converged