Task: Question Generation
Input (English): We call these techniques Recurrence over BERT (RoBERT) and Transformer over BERT (ToBERT). We observed that ToBERT outperforms RoBERT on pre-trained BERT features and fine-tuned BERT features for all our tasks. 
Output (English): On top of BERT does the RNN layer work better or the transformer layer?