CQE : A Comprehensive Quantity Extractor (2023)

Abstract: Quantities are essential in documents to describe factual information. They are ubiquitous in application domains such as finance, business, medicine, and science in general. Compared to other information extraction approaches, interestingly only a few works exist that describe methods for a proper extraction and representation of quantities in text. In this paper, we present such a comprehensive quantity extraction framework from text data. It efficiently detects combinations of values and units, the behavior of a quantity (e.g., rising or falling), and the concept a quantity is associated with. Our framework makes use of dependency parsing and a dictionary of units, and it provides for a proper normalization and standardization of detected quantities. Using a novel dataset for evaluation, we show that our open source framework outperforms other systems and – to the best of our knowledge – is the first to detect concepts associated with identified quantities. The code and data underlying our framework are available at https://github.com/vivkaz/CQE.

## introduction quantities are the main tool for conveying factual and accurate information. news articles are filled with social and financial trends, and technical documents use measurable values to report their findings. despite their significance, a comprehensive system for quantity extraction and an evaluation framework to compare the performance of such systems are not yet at hand. in the literature, a few works directly study quantity extraction, but their focus is limited to physical and science domains @xcite . quantity extraction is often part of a larger system, where identification of quantities is required to improve numerical understanding in retrieval or textual entailment tasks @xcite @xcite @xcite . consequently, their performance is measured based on the downstream task, and the quality of the extractor, despite its contribution to the final result, is not separately evaluated. therefore, when in need of a quantity extractor, one has to resort to a number of open source packages, without a benchmark or a performance guarantee. since quantity extraction is rarely the main objective, the capabilities of the available systems and their definition of quantity vary based on the downstream task. as a result, the context information about a quantity is reduced to the essentials of each system. most systems consider a quantity as a number with a measurable and metric unit @xcite . however, outside of scientific domains any noun phrase describing a value is a potential unit, e.g., "5 bananas". moreover, a more meaningful representation of quantities should include their behaviour and associated concepts. for example, in the sentence "dax fell 2% and s&p gained more than 2%", the value/unit pair 2, percentage indicates two different quantities in association with different concepts, dax and s&p, with opposite behaviours, decreasing and increasing. these subtleties are not captured by simplified models. in this paper, we present a comprehensive quantity extraction (cqe) framework. our system is capable of extracting standardized values, physical and non-physical units, changes or trends in values, and concepts associated with detected values. furthermore, we introduce newsquant, a new benchmark dataset for quantity extraction, carefully selected from a diverse set of news articles in the categories of economics, sports, technology, cars, science, and companies. our system outperforms other libraries and extends on their capabilities to extract concepts associated with values. our software and data are publicly available. by introducing a strong baseline and novel dataset, we aim to motivate further research and development in this field. ## related work in literature, quantity extraction is mainly a component of a larger system for textual entailment or search. the only work that solely focuses on quantity extraction is grobid-quantities @xcite , which uses three conditional random field models in a cascade to find value/unit pairs and to determine their relation, where the units are limited to the scientific domain, a.k.a. si units. @xcite 's definition of a quantity is closer to ours and is based on forbus' theory @xcite . a quantity is a (value, unit, change) triplet, and noun-based units are also considered. extraction is performed as a step in their pipeline for quantity reasoning in terms of textual entailment. although they only evaluate on textual entailment, the extractor is released as part of the cogcomp natural language processing libraries, under the name illinois quantifier. 1 two prominent open source libraries for quantity extraction are (a) recognizers-text @xcite from microsoft and (b) quantulum3. 2 recognizers-text uses regular expressions for the resolution of numerical and temporal entities in ten languages. the system has separate models for the extraction of value/unit pairs for percentages, age, currencies, dimensions, and temperatures and is limited to only these quantity types. moreover, it cannot proactively distinguish the type of quantity for extraction and the user has to manually select the correct model. quantulum3 uses regular expression to extract quantities and a dictionary of units for normalization. for units with similar surface forms, a classifier based on glove embeddings @xcite is used for disambiguation, e.g., "pound" as weight or currency. recognizers-text is used in the work of @xcite to demonstrate quantity search, where the results are visualized in the form of tables or charts. they define quantity facts as triplets of @xcite . related is the quantitative related information, close to our definition of concept. however, it is not part of their quantity model but rather extracted separately using rules. they utilize the quantity facts for the visualization of results but do not evaluate their system or the quantity extrac-tion module. qfinder @xcite uses quantulum3 in a similar way to demonstrate quantity search on news articles, but does not comment on the extractor's performance. another system that indirectly considers concepts is xart @xcite , where instances of n-ary relations containing numerical values and unit are extracted and concepts are an argument in these relations. however, the concepts are limited to a domain ontology with specific concepts of a given application domain. a number of other works utilize quantity extraction as part of their system. mqsearch @xcite extracts quantities with a set of regular expressions for a search engine on numerical information. qsearch @xcite is another quantity search system, based on quantity facts extracted with the illinois quantifier. the works by @xcite focus on scoring quantity intervals in census data and tables. ## extraction of quantities in the following, we describe our quantity representation model and detail our extraction technique. ## evaluation cqe is compared against illinois quantifier (illq), quantulum3 (q3), recognizers-text (r-txt), gorbid-quantities (grbd) and gpt-3 with few-shot learning @xcite . from here on, the abbreviations are used to refer to the respective system. we first compare the functionality of the models, then describe our benchmark dataset and compare the models on precision, recall and f1-score for quantity extraction. finally, the unit disambiguation module is evaluated on a custom-made dataset against q3. our evaluation code and datasets are available at https: //github.com/satya77/cqe_evaluation . ## conclusion and ongoing work in this paper, we introduced cqe, a comprehensive quantity extractor for unstructured text. our system is not only significantly outperforming related methods as well as a gpt-3 neural model for the detection of values, units and changes but also introduces the novel task of concept detection. furthermore, we present the first benchmark dataset for the comprehensive evaluation of quantity extraction and make our code and data available to the community. we are currently extending the extractor by improving the quality of edge cases and looking at the compatibility of our rule set to other application domains, e.g., medical text.