Chemical patent documents describe a broad range of applications holding key  information, such as chemical compounds, reactions, and specific properties. However, the key information should be enabled to be utilized in downstream tasks. Text mining provides means to extract relevant information from chemical patents through information extraction techniques. As part of the Information Extraction task of the Cheminformatics Elseiver Melbourne University challenge, in this work we study the effectiveness of contextualized language models to extract reaction information in chemical patents. We compare transformer architectures trained on a generic corpus with models specialised in chemistry patents, and propose a new model based on the combination of existing architectures . Our best model, based on the ensemble approach, achieves an exact F1-score of 92.30\% and a relaxed F1 -score of 96.24\%. We show that the ensemble of contextualized language models provides an effective method to extract information from chemical patents. As a next step, we will investigate the effect of transformer language models pre-trained in chemical patents.
Chemical patent documents describe a broad range of applications holding key reaction and compound information, such as chemical structure, reaction formulas, and molecular properties. These informational entities should be first identified in text passages to be utilized in downstream tasks. Text mining provides means to extract relevant information from chemical patents through information extraction techniques. As part of the Information Extraction task of the Cheminformatics Elsevier Melbourne University challenge, in this work we study the effectiveness of contextualized language models to extract reaction information in chemical patents. We assess transformer architectures trained on a generic and specialised corpora to propose a new ensemble model . Our best model, based on a majority ensemble approach, achieves an exact F1-score of 92.30\% and a relaxed F1-score of 96.24\%. The results show that ensemble of contextualized language models can provide an effective method to extract information from chemical patents.
A
reaction and compound
R
compounds, reactions, and specific properties. However, the key information should be enabled
structure, reaction formulas, and molecular properties. These informational entities should be first identified in text passages
R
Elseiver
Elsevier
R
compare
assess
R
corpus with models specialised in chemistry patents, and
and specialised corpora to
R
model based on the combination of existing architectures
ensemble model
R
the
a majority
R
F1 -score
F1-score
R
We show that the
The results show that
R
provides
can provide
D
chemical patents. As a next step, we will investigate the effect of transformer language models pre-trained in