POLYIE : A Dataset of Information Extraction from Polymer Material Scientific Literature (2024)

Abstract: Scientific information extraction (SciIE), which aims to automatically extract information from scientific literature, is becoming more important than ever. However, there are no existing SciIE datasets for polymer materials, which is an important class of materials used ubiquitously in our daily lives. To bridge this gap, we introduce POLYIE, a new SciIE dataset for polymer materials. POLYIE is curated from 146 full-length polymer scholarly articles, which are annotated with different named entities (i.e., materials, properties, values, conditions) as well as their N-ary relations by domain experts. POLYIE presents several unique challenges due to diverse lexical formats of entities, ambiguity between entities, and variable-length relations. We evaluate state-of-the-art named entity extraction and relation extraction models on POLYIE, analyze their strengths and weaknesses, and highlight some difficult cases for these models. To the best of our knowledge, POLYIE is the first SciIE benchmark for polymer materials, and we hope it will lead to more research efforts from the community on this challenging task. Our code and data are available on: https://github.com/jerry3027/PolyIE.

## introduction material science literature is growing at an unprecedented rate. for example, a simple search on google scholar with the term "polymers" returns more than 5 million articles on polymer materials. such literature reports valuable information on the latest advances in material science, ranging from experimental material properties to material synthesis recipes and procedures. as machine learning (ml) has achieved success in different applications of material science @xcite , scientific information extraction (sciie) from literature for supporting various figure 1 : an example of entity and relation annotations in polyie from a material science paper @xcite , including entity mentions as well as intrasentence and inter-sentence n-ary relations. while sciie has rapidly developed in domains such as biomedical science @xcite @xcite @xcite and chemistry @xcite @xcite , it has made limited progress in the material science domain. so far, there are only a handful of datasets for material information extraction. some earlier works use chemdataextractor @xcite to automatically generate datasets for battery materials @xcite and temperatures @xcite . more recent datasets are created manually for solid oxide fuel cells @xcite and material science synthesis procedures @xcite . however, none of these datasets cover polymer materials, which are an important class of organic materials that play critical and ubiquitous roles in our daily lives. due to their versatile properties, polymer materials are being widely used in applications such as packaging, coating, energy saving, and medical applications @xcite . as vast amounts of information on polymer development are being reported in literature data, there is a critical need for sciie benchmarks and tools to harvest such information from the polymer literature. to address this gap, we construct a dataset for extracting polymer property knowledge from unstructured literature data. our dataset polyie is curated from 146 full-length polymer scientific articles, which are annotated by domain experts with named entities (i.e., materials, properties, values, conditions) as well as the n -ary relations among them (see figure 1 ). polyie contains 41635 entity mentions and 4443 relations in total. it covers four different application domains of polymer materials: polymer solar cells (psc), ring-opening polymerization (rp), polymer membranes (pm), and polymers in lithium-ion batteries (lb). this diversity of content enables the training of models with enhanced generalization capabilities. to the best of our knowledge, polyie is the first benchmark for sciie from full-text polymer literature. from the natural language processing perspective, extracting information for polymers on polyie introduces unique challenges for both named entity recognition and relation extraction: diverse lexical formats of entities. polymerrelated entities often have different schemes of nomenclature, such as iupac names (e.g., 'poly(3hexylthiophene)'), abbreviations ('pdppnbr'), trade names ('styron'), common names ('abs plastic'), and sample labels ('pe-hdpe-01'). in addition, the identification of polymers can also be achieved through the concatenation of homopolymer names with hyphens or slashes, and the inclusion of numerical values for the component ratios and molecular weights ('pvc-ps-pc-20/30/50-800000'). this diversity of nomenclature in literature poses a challenge for named entity recognition. @xcite or n -ary relations with a fixed number n @xcite @xcite . in contrast, many relations described in polymer literature are variable-length n -ary relations. this is because 1) the reported properties may be describing one or several materials; and 2) different properties can be measured under specific conditions. furthermore, the elements in a relation tuple may span multiple sentences as shown in figure 1 . ## the polyie dataset in this section, we describe the details of the polyie dataset. we first formulate the two information extraction tasks for polymer material literature in § 3.1. we then describe the data preprocessing and annotation procedures in § 3.2 and § 3.3, and finally present the statistics and characteristics of the polyie dataset in § 3.4. ## modeling in this section, we describe how we model the named entity recognition and n -ary relation extraction tasks on polyie. ## conclusion we have curated a new dataset polyie for named entity recognition and n -ary relation extraction from polymer scientific literature. polyie covers thousands @xmath0, property, value, @xmath1. we have evaluated mainstay ner and re models on polyie and analyzed their perfor-mance and error cases. in addition, we have also tested the performance of the strongest llms, gpt-3.5 and gpt-4, on polyie. we found that even state-of-the-art models, either domain-specific pretrained language models or most advanced llms, can struggle with hard ner and re cases. through error analysis, we found that such difficulties arise from the diverse lexical formats and ambiguity of polymer named entities and also variable-length and cross-sentence n -ary relations. our work contributes the first polymer scientific information extraction dataset as well as insights into this dataset. we hope polyie will serve as a useful resource that will and attract more research efforts from the nlp community to push the boundary of this task.