Q: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
We call these techniques Recurrence over BERT (RoBERT) and Transformer over BERT (ToBERT). We observed that ToBERT outperforms RoBERT on pre-trained BERT features and fine-tuned BERT features for all our tasks.
Question: On top of BERT does the RNN layer work better or the transformer layer?
A:
Transformer over BERT (ToBERT)