Before revision:  We compare transformer architectures trained on a generic corpus with models specialised in chemistry patents, and propose a new model based on the combination of existing architectures .
After revision:  We compare transformer architectures trained on a generic and specialised corpora to propose a new model based on the combination of existing architectures .