ZSEE : A Dataset based on Zeolite Synthesis Event Extraction for Automated Synthesis Platform (2024)

Abstract: Automated synthesis of zeolite, one of the most important catalysts in chemical industries, holds great significance for attaining economic and environmental benefits. Structural synthesis data extracted through NLP technologies from zeolite experimental procedures can significantly expedite automated synthesis owing to its machine readability. However, the utilization of NLP technologies in information extraction of zeolite synthesis remains restricted due to the lack of annotated datasets. In this paper, we formulate an event extraction task to mine structural synthesis actions from experimental narratives for modular automated synthesis. Furthermore, we introduce ZSEE, a novel dataset containing fine-grained event annotations of zeolite synthesis actions. Our dataset features 16 event types and 13 argument roles which cover all the experimental operational steps of zeolite synthesis. We explore current state-of-the-art event extraction methods on ZSEE, perform error analysis based on the experimental results, and summarize the challenges and corresponding research directions to further facilitate the automated synthesis of zeolites. The code is publicly available at https://github.com/Hi-0317/ZSEE.

## introduction artificial intelligence is accelerating the autonomous unmannedness of the chemical industry @xcite . as one of the most widely used catalysts in chemical industries, the automated synthesis of zeolite can break the limitations of traditional synthesis process in terms of constrained experimental time of researchers, effectively improving the experimental efficiency @xcite . the first step in carrying out automated synthesis of zeolite is to enable the machine to understand the experimental procedures. recently, * corresponding author there has been a massive increase in the literature and patents of zeolite synthesis experiments, which have documented considerable chemical reaction steps. these synthesis steps can guide the synthesis of specific zeolites and enable the exploration of new zeolites. natural language processing (nlp) techniques allow automatic mining of these synthesis data from materials science literature on a large scale @xcite @xcite . the purpose of conducting such analyses falls into two categories: (1) to get deeper scientific understanding of materials synthesis @xcite ; (2) to implement further research on automated synthesis planning, e.g., enabling robots to perform certain experiments @xcite . but for intelligent machines, there is a huge gap between unstructured records of zeolite synthesis procedures and structured programming languages in terms of semantic understanding. therefore, it is still a challenge to analyze the unstructured experimental narratives and automatically extract machine-readable structured synthesis steps while implementing automated synthesis. previous works have leveraged nlp techniques to extract reaction steps from organic and inorganic chemical synthesis procedures, most of which mostly used named entity recognition (ner) or relationship extraction (re) to extract chemical entities and inter-entity relationships, ignoring the complete synthesis steps. aiming to extract structured synthesis information from experimental narratives for automated synthesis platform, @xcite summarized the chemical synthesis steps and designed a rule-based model to extract the details of these synthesis steps. @xcite constructed a sequence-to-sequence deep learning model to convert unstructured experimental narratives into predefined action sequences. however, the machine translation approach failed to yield fine-grained structured experimental actions. to promote the research on deep learning-based methods for chemical information extraction, @xcite introduced a dataset of 230 synthesis procedures, where the operations and entities (e.g., materials and conditions) were annotated. @xcite introduced event extraction task and released a dataset for chemical event extraction that only defined two event types, which failed to distinguish between the different experimental steps. thus, a dataset with comprehensive fine-grained synthesis information of experimental steps was urgently needed to perform automated and modular synthesis. in this paper, we formulate an event extraction (ee) task to extract fine-grained structured information of synthesis steps directly from zeolite experimental procedures. based on this task, we introduce a novel dataset, zsee, which contains nearly 5000 sentences extracted from the literature on the synthesis process of different types of zeolites. specifically, we summarize all experimental steps of zeolite synthesis and define 16 refined synthesis actions (e.g., add and stir) and the corresponding 13 synthesis properties (e.g., material, temperature and duration). the event triggers and arguments of each sentence in the zsee are annotated with text spans. both humans and intelligent machines can easily capture these synthesis details from our annotations (e.g., stir at 80 • c for 45 min). an annotated example is shown in figure 1 . to evaluate the performance of the state-of-theart (sota) ee methods for the zeolite synthesis event extraction task, we conduct extensive experiments in zsee. we implement two main classes of ee methods to evaluate event detection (ed) and event argument extraction (eae) tasks in zsee, namely classification-based and generation-based methods. the classification-based method performs best for ed with the exact match f1 score of 92.46%. the results of these methods are all actually competitive on the ed task. since triggers of zeolite synthesis events are with single expressions, deep learning-based models can detect these words well. the generation-based method achieves better results for eae with the exact match f1 score of 68.73%. observing the disparity between ed and eae results, we further explore the sota eae methods, which achieve 5.44% gains over the generation-based ee method. recalling the event arguments contain key information for accurate synthesis planning, we urge the development of a better model for event argument extraction of zeolite synthesis. our contributions can be summarized as follows: ## experiments inspired by phee @xcite , we explore the performance of two mainstream classes of event extraction methods (classification-based and generation-based methods) on zsee to reveal the challenges in zeolite synthesis event extraction. specifically, to achieve better performance, we design questions and templates with experimental logic of zeolite synthesis for these methods respectively. ## conclusion in this paper, we present the zeolite synthesis event extraction dataset, zsee, which contains nearly 5000 sentences collected from the literature. we design a comprehensive event schema including 16 event types and 13 event arguments, which cover all experimental steps of zeolite synthesis. fine-grained event annotations for each sentence are further provided. we have performed extensive experiments on zsee and analyzed the strengths and weaknesses of current state-of-the-art methods. we also explore the limitations of the large language model on zsee, which highlights the necessity of zsee. furthermore, we summarize the challenges and research directions in zsee, which can effectively drive the development of zeolite synthesis event extraction for automated synthesis.