言語処理学会 第26回年次大会 発表論文集 (2020年3月)

Extraction of Inorganic Material Synthesis Procedure from
Literature
Shanshan Liu1 Fusataka Kuniyoshi2,3 Jun Ozawa2,3
Masaki Kiyono2 Yuji Matsumoto1,4
1

Nara Institute of Science and Technology (NAIST)
2
Panasonic Corporation
3
National Institute of Advanced Industrial Science and Technology (AIST)4
RIKEN Center for Advanced Intelligence Project (AIP){kuniyoshi.fusataka, kiyono.masaki}@jp.panasonic.com{ozawa.jun}@aist.go.jp{liu.shanshan.lm2, matsu}@is.naist.jp

1

Introduction

Structured information extraction from the literature has always been an indispensable technique to support data-driven methods in various ﬁelds. However, due to the lack of machinereadable datasets and widely used structured
deﬁnitions of information, the extraction technology of inorganic material synthesis information is just beginning, related researchers havenot enjoyed the convenience of data-driven
methods yet.
Previous researches most focused on named entity recognition (NER) and action graphs extraction. Kim et al.[1] annotated some paragraphs with entity labels that were selectedfrom the literature including the synthetic procedure of inorganic substances. Mysore et al.[2]provided a more complete labeled dataset containing over 100 procedures. Making use ofexpert-annotated datasets, Mysore et al.[3] deﬁned a structured procedure as a set of linked
action events and extracted action graphs byheuristic model. Kim et al.[4] extracted the action series in the procedure and developed asystem to provide possible precursors and action graph given the target material. Tamari etal.[5] converted the procedure description into
instructions in a text-based interactive game,
construct the action graph of the procedurefrom the text.
On this basis, we try to challenge a more diﬃcult problem, extracting the complete material
synthesis procedure including condition factors
such as “pressure” using NER and Relation
Extraction (RE) techniques. After the manual analysis of the descriptions in the literature, we clearly deﬁned the procedure, provide
a pipeline extracting method. The result on
Kuniyoshi’s dataset[6] demonstrates that our
method is feasible and full of potential.

2

Methodology

2.1

Task Formulation

Multiple operations are performed sequentiallyin a synthesis procedure (see Figure 1 and 2).
An operation is carried out on certain precursor materials, generating some products under
speciﬁc environmental conditions. We can thus
represent a procedure by three types of entity
and six types of relation. A material m consistsof a mention d and property descriptions while
np is the number of property entities:m = {d, p1 , p2 , . . . , pnp }The set of precursor materials Min can be represented as a set of materials:
Min = {mi1 , mi2 , . . . , minin }The set of generated materials Mout is:
Mout = {mo1 , mo2 , . . . , monout }The set of conditions C of the operation is deﬁned as:
C = {c1 , c2 , . . . , cnc }Based on the above deﬁnitions, an operation O
with a mention o can be represented as:
O = {o, C, Min , Mout }A full material synthesis procedure S is a seriesof operations while no is the number of operations:
S = {O1 , O2 , . . . , Ono }2.2

Procedure Extraction Model

Our procedure extraction system (see Figure 3)
consists of one NER block and three RE blocks.
Considering the portion of cross-sentence relations in training data, we use Graph State― 1053 ―Copyright(C) 2020 The Association for Natural Language Processing.
All Rights Reserved.

Figure 1: A example of a paragraph describing synthesis procedure. “LiNO3”, “NaNO3”, and“KNO3” are precursors of the operation “melted”. After “melted”, “mixed”, “mechanical treatment”, “sintering”, the target material “(1 - x)MeNO3-xAl2O3” is obtained.

Figure 2: The structure of a synthesis procedure including 2 operations. The arrow represents therelationship between 2 entities. The dot line means two entities are the same material. The second
operation takes the output of the ﬁrst operation as its input to generate the target material.

Figure 3: The workﬂow of our procedure extraction system. The result of each block ispassed to others as arrows point.
Long Short-Term Memory (GS-LSTM) model
presented by (Song et al., 2018)[7] which
can extract both the intra-sentence and crosssentence relation. We implemented our model
except Same Material RE part so far.
NER We combine an ELMo embedding model
released by [4], bidirectional Long Short-Term
Memory (Bi-LSTM) model, and Conditional
Random Fields (CRF) model as our NER
framework that is widely used (Figure 4). A
rule-based method is also used for Operation
entity. We constructed a vocabulary for operation expressions common in material synthesis
after analyzed Kuniyoshi’s data, divided theminto 17 categories: add, cool, crush, deposit,
disperse, dry, ﬁnish, ground, heat, keep mill,mix, open, pelletize, press, wash, weigh. We
directly label the matching words in the text
with their category labels. For example, “dis-Figure 4: The architecture of the NER model.
solve” is classiﬁed as “mix” while “sinter” is to“heat”. In capturing the relation between “dissolve” and “sinter”, we also consider the possible connection between their categories “heat”
and “mix”. The construction of the action
graph may be more accurate additionally takethese category labels into consideration.
Intra-operation RE The extraction of intraoperation relations is to get “Material in”,
“Material out”, “Property of”, and “Condition of” relations. The positive instances predicted by the RE model are recorded to support
the Inter-operation RE.
Same material RE This part aims to makea compliment on other two RE tasks. Sometimes, diﬀerent mentions mean the same material while they may play roles in diﬀerent operations, providing information to capture therelation between operation.
Inter-operation RE This is for the relationships we deﬁned as “Next operation” that of― 1054 ―Copyright(C) 2020 The Association for Natural Language Processing.
All Rights Reserved.

Table 1: The statistics of Procedure dataset
Type#Train

#Dev

#Test

#All

EntityRelation
Operation
Paragraph

6559
6150
1309
193

938
888
202
25

926
846
169
25

8423
7884
1680
243

Dataset statistics about the number of named entities, relations and operations for training (#Train),development (#Dev), testing (#Test) and of allparagraphs (#All).

Table 2: Precision(P), Recall(R) and F1score(F1) of NER task
Task
Material (extended)
Material
Argument
Operation
Operation (rule)

P

R

F1

88.6
82.83
66.83
79.73
62.6

96.01
82.1
65.93
76.87
66.17

92.15
82.47
66.38
78.28
64.33

Operation (rule) identiﬁes the entity by exactmatching. Material (extended) takes both thedataset of [4] and [6] for training. Other tasks aretrained on Kuniyoshi’s dataset using the model like
Figure 4.

operation entity-pairs, while the subject is thecurrent operation, the object is the next one.
Considering the information we get in othertasks, we may take information of the key attributes in operation into account for classiﬁcation.

3

Experiment

3.1

Dataset

We utilized a procedure dataset constructedby Kuniyoshi et al.,[6] to evaluate our extraction approach (statistics as Table 1).This
dataset is a set of human-annotated paragraphsof material synthesis procedures selected fromthe literature. Each paragraph contains at least
one full synthesis procedure annotated with theentity label and relationship label by chemistry
experts. Besides, we use Kim et al., [4] to
improve the capability of identiﬁcation of “Material” with its 6,744 material entities labeledfrom a dataset contains 235 synthesis procedures.

3.2

Implementation

Pre-processing Given a plain-text, after theparsing by Stanford NLP Toolkit[8], each word
is annotated with its part-of-speech(POS) tag,and the general named entity tag (i.e., NUMBER), and align it to its human-annotatednamed entity label (i.e., Material). When we
combine Kim’s data with Kuniyoshi’s data, we

ﬁnd that the label standards of the two datasets
are not consistent. Kim subdivides the material mentions into “property misc, precursor,
material, target”, but Kuniyoshi marks the material uniformly as “Material”. They also diﬀer
in word segmentation. Inorganic substances inKim, whether they include brackets or not, aretreated as one token, but the words including
brackets in Kuniyoshi are cut into tokens by
the tokenizer. Therefore, all of the words in thetwo datasets that are in the form as [V2O5] areprocessed to [V(2)O(5)], and split into several
tokens by brackets. We built training instances
according to the form of (Peng et al., 2017)[9]taking the golden truth of NER.
Statistical analysis of training data shows mostof the relationships, two entities are within 5
adjacent sentences (when subject/object entity
is in the ﬁrst sentence of the paragraph, 97% of
the object/subject entity is in the second to theﬁfth sentence). When constructing the training
sample, we thus set the window size as 5 sweeping through each paragraph. If paragraph contains less than ﬁve sentences, window size willbe the length of paragraph. For each entity
appearing in the ﬁrst sentence, we combine it
to other entities in the window under the constraint by the entity types. Combined entitypairs are divided into 6 relation types we deﬁned and “No relation”.
NER This study aims to process the chemistry literature text, thus the ELMo pre-trainedmodels open-sourced by the [4] were selected.
This embedding model was trained on 2.5 million materials science journal articles. We
trained on only Kuniyoshi’s dataset for three
entity types.We trained a “Material” extraction model on extended dataset that including
all data of [4] and training set of Kuniyoshi’s.
For “Operation”, we divided text expressions of
the action that appeared in the training data
into 17 categories and identify entities by exactmatching.
Intra-operation RE We found the GS-LSTM
model can not learn well when imbalance skews
problem occurs. #Neg:#Pos reaches 13 : 1
in Intra-operation relation if window size = 5.
To cope with this, we used sampled data selected by the K-Means cluster while k = 8 fortraining. #Neg:#Pos in each sampled dataset
is 1 : 1 including all positive instances and sampled negative instances of original training set.
Inter-operation RE We tried four diﬀerent
strategies. 1) Given an operation entity, use therule-based method that sets the nearest operation that occurs after the given entity as thenext operation. 2) Use GS-LSTM as the encoder and directly use the encoded entity representation for the prediction of entity-pairs. We
do not sample the training data in this task
so far. 3) Encode the sentences by Bi-LSTM,― 1055 ―Copyright(C) 2020 The Association for Natural Language Processing.
All Rights Reserved.

Table 3: RE performance by GS-LSTM model
Task

Accu

P

R

F1

Intra-operation
Inter-operation

37.93
37.5

9.98
23.8

93.62
64.5

18.04
34.7

The accuracy (Accu) and other evaluation indicators on testing dataset.

Table 4: Performance of Inter-operation RE
Method

Accu

P

R

F1

Rule-based
GS-LSTM
Bi-LSTM
Ensemble

84.8
37.5
85.2
62.5

84.8
23.8
69.5
29.0

92.6
64.5
77.2
34.4

88.5
34.7
73.1
31.5

concatenate the encoded entity representation,
the embeddings of entity type and the distance
between the two entities to predict. 4) Ensemble the output of Bi-LSTM and the output ofrule-based method. Compared with the outputof the rule-based method, if they are the same,
it is directly used as the ﬁnal prediction result.
Otherwise, a linear layer with inputting the encoded entity representation and the embedded
rule-based result does the ﬁnal prediction.

3.3

Results and Analysis

The NER result (Table 2) shows our model can
get reliable entity labels for further RE tasks.
The operation vocabulary is not complete toachieve a higher score than NN-based method.
We have to enrich expressions to make our rulebased method more accurate.
The GS-LSTM model performs not promisingin Table 3. The recall is high in Intra-operation
RE, which means we can add one more classiﬁer
to remove irrelevant relations after we identify
relations. A better way to solve the imbalanced
skews problem is needed. Comparing the results of four strategies for Inter-operation RE,
the rule-based method is simplest but best (Table 4). Encoding sentences by Bi-LSTM but
considering entity types and distance between
entities reach an F1 score of 73.1. We can inferthat rule-based features can help to better performance of Inter-operation RE, however, we
still need to research on the way of ensembling.
We still not refer to the Inter-operation relation
information when connecting two operations,which means there is still an improvement zone
for our approach. The RE result on the groundtruth of entity labels is not good, the situationthat taking the result of NER as the input of
RE can be very challenging. To consider the
cascade errors in our pipeline model is particularly under low precision on Intra-operation
RE, changing pipeline learning to multi-task
learning may solve this problem.

4

Conclusion

In this paper, we provide a pipeline method
to extract the procedure of inorganic material
synthesis from literature. We deﬁne the procedure as various types of entities and connections between entities. The procedure extraction consists of two subtasks: Named Entity Recognition (NER) and Relation Extraction(RE). We evaluate the capability of our approach on a procedure dataset involving 243
paragraphs selected from the literature. The
result shows the feasibility that our approachcan extract named entity and relationships inprocedure well in such cases, get a more clear
vision of the diﬃculty to apply NLP techniquesin practice. We will keep researching on catching the relationships of attributes in the procedure, dealing with the imbalance skews problem and reducing the potential cascade errors
caused by the pipeline framework.

References[1] Edward Kim, Kevin Huang, Adam Saunders, Andrew
McCallum, Gerbrand Ceder, and Elsa Olivetti. Materials synthesis insights from scientiﬁc literature via
text extraction and machine learning. Chemistry ofMaterials, 29(21):9436–9444, 2017.
[2] Sheshera Mysore, Zach Jensen, Edward Kim, Kevin
Huang, Haw-Shiuan Chang, Emma Strubell, Jeﬀrey
Flanigan, Andrew McCallum, and Elsa Olivetti. The
materials science procedural text corpus: Annotating
materials synthesis procedures with shallow semantic
structures. arXiv preprint arXiv:1905.06939, 2019.
[3] Sheshera Mysore, Edward Kim, Emma Strubell,
Ao Liu, Haw-Shiuan Chang, Srikrishna Kompella,
Kevin Huang, Andrew McCallum, and Elsa Olivetti.
Automatically extracting action graphs from materials science synthesis procedures.
arXiv preprint
arXiv:1711.06872, 2017.
[4] Edward Kim, Zach Jensen, Alexander van Grootel,
Kevin Huang, Matthew Staib, Sheshera Mysore, HawShiuan Chang, Emma Strubell, Andrew McCallum,
Stefanie Jegelka, and Elsa Olivetti. Inorganic materials synthesis planning with literature-trained neuralnetworks, 2018.
[5] Ronen Tamari, Hiroyuki Shindo, Dafna Shahaf, andYuji Matsumoto. Playing by the book: An interactive
game approach for action graph extraction from text.
In Proceedings of the Workshop on Extracting Structured Knowledge from Scientiﬁc Publications, pages
62–71, 2019.
[6] Fusataka Kuniyoshi, Jun Ozawa, Makoto Miwa, et al.
Graph representation for synthesis process extraction
from inorganic material literature. In 15th Text Analytics Symposium in IEICE-NLC, 2019.
[7] Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel
Gildea. N-ary relation extraction using graph state
lstm. arXiv preprint arXiv:1808.09101, 2018.
[8] Richard Socher, John Bauer, Christopher D Manning,
and Andrew Y Ng. Parsing with compositional vector
grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers), pages 455–465, 2013.
[9] Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina
Toutanova, and Wen-tau Yih. Cross-sentence n-ary
relation extraction with graph lstms. Transactionsof the Association for Computational Linguistics,
5:101–115, 2017.

― 1056 ―Copyright(C) 2020 The Association for Natural Language Processing.
All Rights Reserved.