Question: Does model uses pretrained Transformer encoders?
Answer: No