Question: Is the improvement actually coming from using an RNN?
Answer: No