Extracting Polymer Nanocomposite Samples from Full-Length Documents (2024)

Abstract: This paper investigates the use of large language models (LLMs) for extracting sample lists of polymer nanocomposites (PNCs) from full-length materials science research papers. The challenge lies in the complex nature of PNC samples, which have numerous attributes scattered throughout the text. The complexity of annotating detailed information on PNCs limits the availability of data, making conventional document-level relation extraction techniques impractical due to the challenge in creating comprehensive named entity span annotations.To address this, we introduce a new benchmark and an evaluation technique for this task and explore different prompting strategies in a zero-shot manner. We also incorporate self-consistency to improve the performance. Our findings show that even advanced LLMs struggle to extract all of the samples from an article. Finally, we analyze the errors encountered in this process, categorizing them into three main challenges, and discuss potential strategies for future research to overcome them.

## introduction research publications are the main source for the discovery of new materials in the field of materials science, providing a vast array of essential data. creating structured databases from these publications enhances discovery efficiency, as evidenced by ai tools like gnome @xcite . yet, the unstructured format of journal data complicates its extraction and use for future discoveries @xcite . furthermore, the manual extraction of material details is inefficient and prone to errors, underlining the necessity for automated systems to transform this data into a structured format for better retrieval and analysis @xcite . ## pncextract benchmark in this section, we first describe our dataset, including the problem definition, and the dataset preparation. then we describe our evaluation method for the described task. ## modeling sample list extractions from articles with llms as mentioned in section 1, our dataset is designed for a generative task, making encoder-only models unsuitable for two main reasons. first, these models demand extensive annotations, such as named entity spans, coreferences, and negative examples, a process that is both time-consuming and errorprone. second, encoder-only models struggle with processing long documents efficiently. while some studies have successfully used these models for long documents @xcite , they had access to significantly larger datasets. our dataset, however, contains detailed domain-specific information, making it challenging to obtain a similarly extensive dataset. consequently, within a zero-shot context foot_2 , we explore two prompting methods: named entity recognition plus relation extraction (ner+re) and an end-to-end (e2e) approach. llm instruction: identify the matrix, filler, and composition. ## related work early works have focused on training models specifically for the tasks of ner and re. building on this, recently @xcite and @xcite show that llms can effectively carry out these tasks through prompting. in the specific area of models trained on a materials science corpus, matscibert @xcite employs a bert @xcite model trained specifically on a materials science corpus. song et al. (2023b) 2023 ) our study also focuses on extracting n -ary relations from materials science papers. however, our approach diverges in two significant aspects: we analyze full-length papers, not just selected sentences, and we extend our evaluation to partial assessment of n -ary relations, rather than limiting it to binary assessments. ## discussion and future works we introduced pncextract, a benchmark focused on the extraction of pnc samples from full-length materials science articles. to the best of our knowledge, this is the first benchmark enabling detailed n -ary ie from full-length materials science articles. we hope that this effort encourages further research into generative end-to-end methods for scientific information extraction from full-length documents. future investigations should also consider more advanced techniques for condensing entire scientific papers. to overcome the challenges in pnc sample extraction discussed in section 4.3, future studies could investigate multimodal strategies that integrate text and visual data. additionally, experimenting fine-tuning methods could lead to more precise chemical name generation. ## ethics statement we do not believe there are significant ethical issues associated with this research.