Question: Did they experiment with pre-training schemes?
Answer: No