# Automatic Identification and Normalisation of Physical Measurements in Scientific Literature

Luca Foppiano, Laurent Romary, Masashi Ishii, Mikiko Tanifuji

► To cite this version:

Luca Foppiano, Laurent Romary, Masashi Ishii, Mikiko Tanifuji. Automatic Identification and Normalisation of Physical Measurements in Scientific Literature. DocEng '19 - ACM Symposium on Document Engineering 2019, Sep 2019, Berlin, Germany. pp.1-4, 10.1145/3342558.3345411. hal-02294424v2

HAL Id: hal-02294424

https://inria.hal.science/hal-02294424v2

Submitted on 28 Sep 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Automatic Identification and Normalisation of Physical Measurements in Scientific Literature

Luca Foppiano

FOPPIANO.Luca@nims.go.jp

National Institute for Materials Science (NIMS)

Tsukuba, Japan

Masashi Ishii

ISHII.Masashi@nims.go.jp

National Institute for Materials Science (NIMS)

Tsukuba, Japan

Laurent Romary

laurent.romary@inria.fr

Inria

Paris, France

Mikiko Tanifuji

TANIFUJI.Mikiko@nims.go.jp

National Institute for Materials Science (NIMS)

Tsukuba, Japan

## ABSTRACT

We present Grobid-quantities, an open-source application for extracting and normalising measurements from scientific and patent literature. Tools of this kind, aiming to understand and make unstructured information accessible, represent the building blocks for large-scale Text and Data Mining (TDM) systems. Grobid-quantities is a module built on top of Grobid [6] [13], a machine learning framework for parsing and structuring PDF documents. Designed to process large quantities of data, it provides a robust implementation accessible in batch mode or via a REST API. The machine learning engine architecture follows the cascade approach, where each model is specialised in the resolution of a specific task. The models are trained using CRF (Conditional Random Field) algorithm [12] for extracting quantities (atomic values, intervals and lists), units (such as length, weight) and different value representations (numeric, alphabetic or scientific notation). Identified measurements are normalised according to the International System of Units (SI). Thanks to its stable recall and reliable precision, Grobid-quantities has been integrated as the measurement-extraction engine in various TDM projects, such as Marve (Measurement Context Extraction from Text), for extracting semantic measurements and meaning in Earth Science [10]. At the National Institute for Materials Science in Japan (NIMS), it is used in an ongoing project to discover new superconducting materials. Normalised materials characteristics (such as critical temperature, pressure) extracted from scientific literature are a key resource for materials informatics (MI) [9].

## CCS CONCEPTS

* Applied computing → Document analysis; *Document metadata; Format and notation*.

## KEYWORDS

Machine Learning, TDM, Measurements, Physical quantities, Units of measurements

## ACM Reference Format:

Luca Foppiano, Laurent Romary, Masashi Ishii, and Mikiko Tanifuji. 2019. Automatic Identification and Normalisation of Physical Measurements in Scientific Literature. In *ACM Symposium on Document Engineering 2019 (DocEng '19), September 23-26, 2019, Berlin, Germany*. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3342558.3345411

## 1 INTRODUCTION

The data overflow in scientific publications makes rapid access to relevant information a challenging issue, for both researchers and readers. One of the essential element found in scientific literature is the physical quantity or measurement, which combine quantification of units (such as grams or micrometres) and quantified object or substances. The automatic extraction of measurements has been studied for many years. Nowadays, although the technology has been evolved, there are still several challenges to overcome: (1) natural language and writing style have varieties of expressions (for example length can be expressed as m, meter, metre). (2) Overlaps between the different units of measurement (*pico Henry* inductance and acidity share the same notation, *pH*). (3) The physical quantities or measurements are scalable by accompanying units (e.g., 1 pl. = 453.6 g), meaning that value and unit combination and its normalisation are necessary for semantic recognition. The need for a precise automatic generation of databases from physical measurements is common to a wide range of domains.

In this paper, we present Grobid-quantities, an open-source application for identifying, parsing and normalising measurements from scientific and patent literature. Using Conditional Random Field (CRF) [12], it provides a machine learning framework for extracting information in a robust manner, and then normalise them toward the International System of Units (SI). This article is organised as follow. In Section 2 we introduce related work. Then, we describe the system in Section 3 and report its evaluation results in Sections 4. Use cases and future scopes are described in Section 5. Section 6 concludes this paper.

## 2 RELATED WORK

Attempts to extract measurements from text have been made using rule-based (formal grammars engines, lookups in terminological

<footnote>Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.</footnote>

<footnote>*DocEng '19, September 23-26, 2019, Berlin, Germany*</footnote>

<footnote>© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.</footnote>

<footnote>ACM ISBN 978-1-4503-6887-2/19/09...$15.00</footnote>

<footnote>https://doi.org/10.1145/3342558.3345411</footnote>
databases) and ML approaches. A known commercial tool, Quant-
alyze<fnref n="1" />, was reported by [10] showing weak recall and supporting
only a limited subset of units [3]. Another approach [1], using
GATE (General Architecture for Text Engineering), addressed the
identification of numeric properties from patents. [2] investigated
issues applied to Russian-derived languages. These approaches lack
either the generalisation to an extensive corpus or deal mainly with
specific languages. [4] described an attempt to recognise units by
looking up terms from an ontology, using ML in combination with
pattern matching and string metrics. Other ML-based approaches
exist, although limited to specific domains: [11] and [8] describe
measurements extraction from experimental results in biology and
nanocrystal device development, respectively. Our work is not re-
stricted to a specific domain or subset of measurements and includes
a normalisation process.

3 SYSTEM DESCRIPTION

Grobid-quantities is a Java web application, based on Grobid (Gen-
eRatio Of Bibliographic Data) [6] [13], a machine learning frame-
work for parsing and structuring raw documents such as PDF or
plain text. Grobid-quantities is designed to process large quantity
of data, via web, through a REST (Representation State Transfer)
API or locally, via the file-system (batch mode). Output information
are standardised as stand-off annotations, and they can be stored
in databases or indexed in search engines. Each annotation can be
visualised on top of PDFs using the GROBID build-in positional
coordinates.

**3.1 Data model**

<figure><img src="image_2.png" /><figcaption>Figure 1: Schema of the data model.</figcaption></figure>

The data model (Figure 1) lay its foundation on the concept of
Measurement, which links an object or a substance with one or more
quantities. We defined four Measurements types: (a) atomic, in case
of a single measurement (e.g., 10 grams). (b) interval (from 3 to 5
km) and (c) range (100 ± 4 mm) for continuous values, and, (d) a list
of discrete values. A Quantity links the quantitative value and the
unit. Since data extracted from PDFs unavoidably present irregular
tokens from wrong UTF-8 encoding or missing fonts, we designed

this model to allow partial results. The *Value* and *Unit* entities allow three different representations (Figure 1): *Raw* as appear in input, *Parsed* unifies the value into the numerical expression, and the unit with its properties (system, type). Finally, *Normalised* contains the transformed unit and values to the SI system. *Value* object supports four types of representations: numeric (2, 1000), alphabetic (two, thousand), scientific notation (3 · 10⁵), and time, which is also an expression of measurement. Units objects are organised following the SI, which allows representing units as products of simpler compounds (e.g. m/s to m · s⁻<fnref n="1" />) further decomposed as triples (prefix, base and power).

3.2 Architecture

The system takes in input text or PDF (the content is extracted in a
structured way using the Grobid framework) and performs three
steps: (a) tokenisation, (b) measurement extraction and parsing and
(c) quantity normalisation. The details of each step are summarised
as follows.

3.2.1 Tokenisation. This process splits input data into tokens. Grobid-quantities uses a two-phase tokenisation: (1) first it splits by punctuation marks, then (2) each resulting token is re-tokenised to separate adjacent digits and alphanumeric characters. Given the example 25m^2, first returns a list [25m, ^, 2] and then recursively divides 25m as [25, m] resulting in [25, m, ^, 2].

3.2.2 *Extraction.* The tokens are passed through three ML models, in cascade: first the *Quantities* CRF model determines appropriate unit and value tags. Results are further processed by the respective *Units* and *Values* CRF models as illustrated in Figure 2.

<figure><img src="image_3.png" /><figcaption>Figure 2: The cascade approach in applied CRF models. The Quantities model recognises value and units which are passed, respectively, to Values and Units CRF sub-models for further extraction.</figcaption></figure>

Table 1 describes the labels predicted by the *Quantities* CRF model. Notice that, to reconstruct complex structured objects from the flat sequence generated by the engine, additional labels are necessary (such as `<unitLeft>`, `<unitRight>`, for units).

Previous work (Section 2) presented extensive use of databases
or ontologies. In our solution, we used a similar approach. We
created a list of units (in English, French and German) with their
characteristics: system (SI base, SI derived, imperial, ...) and type

<footnote n="1">¹https://www.quantanalyze.com/</footnote>
<table><caption>**Table 1:** Labels description for the Quantities CRF model. In bold are highlighted specific examples.</caption><thead><tr><th>Label</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td>&lt;valueAtomic&gt;</td><td>value of an atomic quantity</td><td>2 m</td></tr><tr><td>&lt;valueLeast&gt;</td><td>least value in an interval</td><td>from 2 m</td></tr><tr><td>&lt;valueMost&gt;</td><td>max value in an interval</td><td>up to 7 m</td></tr><tr><td>&lt;valueBase&gt;</td><td>base value in a range</td><td>20 ± 7 m</td></tr><tr><td>&lt;valueRange&gt;</td><td>range value in a range</td><td>20 ± 7 m</td></tr><tr><td>&lt;valueList&gt;</td><td>list of quantities</td><td>2, 3 and 10 m</td></tr><tr><td>&lt;unitLeft&gt;</td><td>left-attached unit</td><td>pH 2</td></tr><tr><td>&lt;unitRight&gt;</td><td>right-attached unit</td><td>2 m</td></tr><tr><td>&lt;other&gt;</td><td>everything else</td><td>-</td></tr></tbody></table>

(volume, length, ...), and their representations: notations (m³, m³), lemmas (cubic meter, cubic metre) and inflections (cubic meters, cubic metres). We made this list available through the *Unit Lexicon*, which offers unit lookups by properties (such as notation, lemma, inflexion). A second gazetteer was created to allow the transformation of alphabetic values in numeric ones (for example, twenty-one to 21).

Features in the *Quantities* CRF model are generated from preceding and following tokens, presence of capital, digits. Orthogonal features are obtained through the *Unit Lexicon*, like a Boolean indicating whether a token is a known unit or not. Typographical information (such as format, fonts, subscript and superscript) are ignored.

The *Units* CRF model works at character level and uses the *Unit Lexicon* to highlight known units or prefixes. The input tokens are parsed and transformed to a product of triples (prefix, base, power) as shown in Table 2. For example Kg/mm², corresponds to Kg·mm⁻² and becomes [(K, g, 1), (m, m, -2)] as product of triples.

<table><caption>**Table 2:** Labels description for the Units CRF model. In bold are highlighted specific examples.</caption><thead><tr><th>Label</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td>&lt;prefix&gt;</td><td>prefix of the unit</td><td>km²</td></tr><tr><td>&lt;base&gt;</td><td>unit base</td><td>km²</td></tr><tr><td>&lt;pow&gt;</td><td>unit power</td><td>km²</td></tr><tr><td>&lt;other&gt;</td><td>everything else</td><td>-</td></tr></tbody></table>

We then use the structured triples to fetch the corresponding information (system, type) from the *Unit Lexicon* and attach them to the resulting object. This implementation processes the unit characters using right-to-left order. Priority modifiers, such as parenthesis, are ignored. They are generally not frequent in units expressions, and require a more complex logic.

In parallel, the *CRF Values* model unifies the format of identified values into numerical formats. It supports four types: numeric, alphabetic, scientific notation, and time expression (see Table 3). Different techniques are applied for each type: alphabetic expressions are looked up in the word-to-number gazetteer, scientific notations are parsed and calculated mathematically. Time expressions are further segmented using the Grobid built-in Date CRF model.

<table><caption>**Table 3:** Labels description for the Values CRF model. In bold are highlighted specific examples.</caption><thead><tr><th>Label</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td>&lt;number&gt;</td><td>numeric value / coefficient</td><td>2.5 · 10⁵</td></tr><tr><td>&lt;alpha&gt;</td><td>alphabetic value</td><td>twenty</td></tr><tr><td>&lt;time&gt;</td><td>time expression</td><td>in 1970-01-02</td></tr><tr><td>&lt;base&gt;</td><td>base in scientific notation</td><td>2.5 · 10⁵</td></tr><tr><td>&lt;pow&gt;</td><td>exponent in scientific notation</td><td>2.5 · 10⁵</td></tr><tr><td>&lt;other&gt;</td><td>everything else</td><td>-</td></tr></tbody></table>

### 3.2.3 Normalisation.
The measurements extracted are transformed to the base SI unit (grams to kg, Celsius to Kelvin, etc.). We used an external Java library called Units of Measurement [5], which provides a set of standard interfaces and implementations for safely handling units and quantities. Manipulating measurements with transformations often lead to common mistakes due to wrong rounding and approximations. At the time this paper is being written, the final revised version of this library has been accepted under the Java Standardisation Process JSR-385.

## 4 EVALUATION AND RESULTS

We trained and evaluated our system’s models using a dataset based on 32 scientific publication (English, Open Access (OA)) and three patents (with translation in English, French and German) randomly selected from different domains such as medicine, robotics, astronomy, and physiology. The training data was generated automatically and then corrected and cross-checked by three annotators. We used 10-fold cross-validation to evaluate each CRF model, independently, and produce precision, recall and f1 scores, as summarised in Table 4.

The *Quantities* CRF model reported an f1 macro average of 80.14% with precision and recall of 84.93% and 76.86%, respectively. The paragraph accuracy was 68.97%, indicating that more than half of the evaluated paragraphs were correctly labelled. These scores are promising, considering the complexity of the task and the rather small size of the training corpus. In particular, <list> and <unitRight> require more example.

The *Units* CRF model f1 macro average was 98.86%, with precision and recall reaching 98.75% and 98.97%, respectively. Compared with our other models, performances were extremely high (more than 10% for f1 score). Such difference can be attributed to the data distribution and the lower variability of unit expressions. We analysed the training data, and we noticed that the distribution is biased toward simple units (composed by a single triple). Intuitively, this makes sense, because simple units are statistically more frequent; on the other hand, it highlights the necessity of having more complex examples in our dataset. Secondly, unit expressions appear, by nature, with lower variability, leading to the generation of more duplicates than in other models’ training datasets. For example, the expressions 1% and 2% have two different values (1, 2) and the same unit (%), which would appear twice. Since we cannot alter the statistical distribution of the dataset, we would obtain better and more precise measurements of the model generalisation capabilities by using a separate and independent evaluation corpus.
<table><caption>**Table 4:** Summary of the evaluation scores (precision, recall, F1-score) and label contribution (support) for Quantities, Units and Values CRF models, respectively.</caption><thead><tr><th>Label</th><th>Precision</th><th>Recall</th><th>F1</th><th>Support</th></tr></thead><tbody><tr><td colspan="5" style="text-align: center; font-weight: bold;">Quantities CRF model</td></tr><tr><td>&lt;unitLeft&gt;</td><td>96.76</td><td>94.71</td><td>95.71</td><td>2805</td></tr><tr><td>&lt;unitRight&gt;</td><td>93.06</td><td>72.1</td><td>80.02</td><td>120</td></tr><tr><td>&lt;valueAtomic&gt;</td><td>85</td><td>84.77</td><td>84.84</td><td>3599</td></tr><tr><td>&lt;valueBase&gt;</td><td>78.82</td><td>76.52</td><td>77.53</td><td>94</td></tr><tr><td>&lt;valueLeast&gt;</td><td>85.05</td><td>77.39</td><td>80.94</td><td>862</td></tr><tr><td>&lt;valueList&gt;</td><td>72.09</td><td>54.87</td><td>61.33</td><td>494</td></tr><tr><td>&lt;valueMost&gt;</td><td>84.09</td><td>73.03</td><td>78.07</td><td>878</td></tr><tr><td>&lt;valueRange&gt;</td><td>84.56</td><td>81.5</td><td>82.68</td><td>93</td></tr><tr><td>all (macro avg.)</td><td>84.93</td><td>76.86</td><td><strong>80.14</strong></td><td></td></tr><tr><td colspan="5" style="text-align: center; font-weight: bold;">Unit CRF model</td></tr><tr><td>&lt;base&gt;</td><td>99.02</td><td>99.22</td><td>99.12</td><td>3075</td></tr><tr><td>&lt;pow&gt;</td><td>98.04</td><td>98.9</td><td>98.46</td><td>322</td></tr><tr><td>&lt;prefix&gt;</td><td>99.19</td><td>98.8</td><td>98.99</td><td>821</td></tr><tr><td>all (macro avg.)</td><td>98.75</td><td>98.97</td><td><strong>98.86</strong></td><td></td></tr><tr><td colspan="5" style="text-align: center; font-weight: bold;">Values CRF model</td></tr><tr><td>&lt;alpha&gt;</td><td>96.64</td><td>98.65</td><td>97.62</td><td>826</td></tr><tr><td>&lt;base&gt;</td><td>83.06</td><td>69.23</td><td>72.77</td><td>58</td></tr><tr><td>&lt;number&gt;</td><td>98.01</td><td>99.02</td><td>98.52</td><td>3858</td></tr><tr><td>&lt;pow&gt;</td><td>76.45</td><td>74.67</td><td>74.58</td><td>56</td></tr><tr><td>&lt;time&gt;</td><td>72.54</td><td>87.83</td><td>79.34</td><td>322</td></tr><tr><td>all (macro avg.)</td><td>85.34</td><td>85.88</td><td><strong>84.57</strong></td><td>-</td></tr></tbody></table>

The Value CRF model scored average macro f1 at 85.64% with precision and recall at 81.82% and 93.29%, respectively. We noticed that both <base>, <pow> and <time> have lower f1-score. While <base> and <pow> require more training data, <time> expressions may overlap with <number> suggesting more contextual information should be introduced.

## 5 APPLICATIONS

Recently, the normalised data extraction is strongly required in materials research. The inverse problem in which high-performance materials are predicted from properties is expected to be solved with well-organised big data. At the National Institute for Materials Science (NIMS), a project to discover new superconducting materials from scientific literature is in progress. The system being developed relies on Grobid-quantities to extract and normalise superconducting properties, such as critical temperature (Tc) with units of mK and K and critical pressure expressed with units of Pa, MPa, and GPa [9].

Grobid-quantities was showcased in a Text and Data Mining (TDM) platform (within the scope of the French national-wide ISTEX [7] project) where it provided measurement annotations used to prototype a quantities-based semantic search<fnref n="2" />.

Finally, another use was made in a system for extracting semantic measurements and meaning in Earth Science, Marve [10].

## 6 CONCLUSION

In this paper, we presented Grobid-quantities, a system for extracting and normalising measurement from scientific and patent literature. The project, the training data and the documentation are accessible on Github<fnref n="3" />.

Results are promising, and the integration in real production platforms proved this application reached a certain level of maturity. Our dataset, although it requires more training examples, is released as open access and can be improved from external contributors. Moreover, as previously discussed, the introduction of an end to end evaluation corpus could provide more objective evaluation results.

In the future, we plan to introduce recurrent neural networks (RNN) and embeddings for sequence labelling. In particular, contextualised embeddings, trained with values and units could improve the model generalisation. Finally, we plan to integrate domain information and additional layout features (such as superscript/subscripts) to improve unit discrimination.

## ACKNOWLEDGMENTS

Our warmest thanks to Patrice Lopez (author of Grobid and many other open-source TDM tools), who initiated and supported Grobid-quantities. Thanks our colleagues at NIMS Thaer M. Dieb, and Akira Suzuki for the support received. Finally, thanks to Units of Measurements's contributors<fnref n="4" />.

## REFERENCES

[1] Milan Agatonovic, Niraj Aswani, Kalina Bontcheva, Hamish Cunningham, Thomas Heitz, Yaoyong Li, Ian Roberts, and Valentin Tablan. 2008. Large-scale, parallel automatic patent annotation. In *Proceedings of the 1st ACM workshop on Patent information retrieval*. ACM, 1–8.

[2] Skopinava AM and Lobanov BM. 2013. Processing of quantitative exPressions with units of measurement in scientific texts as aPlied to Belarusian and russian text-to-sPeech synthesis. (2013).

[3] Hidir Aras, René Hackl-Sommer, Michael Schwantner, and Mustafa Sofean. 2014. Applications and Challenges of Text Mining with Patents. In *IPaMin@KONVENS*.

[4] Soumia Lilia Berrahou, Patrice Buche, Juliette Dibie-Barthélemy, and Mathieu Roche. [n. d.]. How to Extract Unit of Measure in Scientific Documents?

[5] Contributors [n. d.]. Units of Measurement. https://github.com/unitsofmeasurement.

[6] Contributors 2008 – 2019. GROBID (GeneRation Of Bibliographic Data). https://github.com/kermitt2/grobid.swh:1:dir:6a298c1b2008913d62e01e5bc967510500f80710.

[7] André Dazy. 2014. ISTEX: a powerful project for scientific and technical electronic resources archives. *Insights* 27, 3 (2014).

[8] Thaer M Dieb, Masaharu Yoshioka, Shinjiro Hara, and Marcus C Newton. 2015. Framework for automatic information extraction from research papers on nanocrystal devices. *Beilstein journal of nanotechnology* 6, 1 (2015), 1872–1882.

[9] Luca Foppiano, M. Dieb Thaer, Akira Suzuki, and Masashi Ishii. 2019. Proposal for Automatic Extraction Framework of Superconductors Related Information from Scientific Literature. In *Letters and Technology News*, vol. 119, no. 66, SC2019-1 (no.66), Vol. 119. Tsukuba, 1–5. ISSN: 2432-6380.

[10] Kyle Hundman and Chris A Mattmann. 2017. Measurement Context Extraction from Text: Discovering Opportunities and Gaps in Earth Science. In *Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*. ACM.

[11] Yanna Shen Kang and Mehmet Kayaalp. 2013. Extracting laboratory test information from biomedical text. *Journal of pathology informatics* 4 (Aug. 2013), 23–23. https://doi.org/10.4103/2153-3539.117450

[12] John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. (2001).

[13] Patrice Lopez. 2009. GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In *International conference on theory and practice of digital libraries*. Springer, 473–474.

<footnote n="2">²The demo can be accessed at https://traces1.inria.fr/istex_sample/</footnote>

<footnote n="3">³http://github.com/kermitt2/grobid-quantities</footnote>

<footnote n="4">⁴https://github.com/orgs/unitsofmeasurement/people</footnote>