XNLP : An Interactive Demonstration System for Universal Structured NLP (2024)

Abstract: Structured Natural Language Processing (XNLP) is an important subset of NLP that entails understanding the underlying semantic or syntactic structure of texts, which serves as a foundational component for many downstream applications. Despite certain recent efforts to explore universal solutions for specific categories of XNLP tasks, a comprehensive and effective approach for unifying all XNLP tasks long remains underdeveloped. Meanwhile, while XNLP demonstration systems are vital for researchers exploring various XNLP tasks, existing platforms can be limited to, e.g., supporting few XNLP tasks, lacking interactivity and universalness. To this end, we propose an advanced XNLP demonstration system, where we leverage LLM to achieve universal XNLP, with one model for all with high generalizability. Overall, our system advances in multiple aspects, including universal XNLP modeling, high performance, interpretability, scalability, and interactivity, offering a unified platform for exploring diverse XNLP tasks in the community.

## introduction xnlp has been referred to as a special form of nlp tasks that involves holistically analyzing and interpreting the underlying semantic or syntactic structure within a text, such as syntactic dependency parsing @xcite , information extraction @xcite , coreference resolution @xcite , and opinion extraction @xcite , etc. figure 1 (upper part) illustrates some representative xnlp tasks under different categories. xnlp has been infrastructural for a wide range of downstream nlp applications, such as knowledge graph construction @xcite , empathetic dialogue @xcite , and more newly-emerging applications and techniques @xcite ). 1 xnlp is online: https://xnlp.haofei.vip spans & span-attribute relations & relation-type dependency parsing consistency parsing syntactic parsing named entity recognition relation extraction information extraction event extraction semantic role labeling coreference resolution semantic analysis aspect-based sentiment analysis sentiment triplet extraction sentimen&opinion mining opinion role labeling det case nmod nsubj dt nns np s vb nn vp pronoun antecedent arg1 per loc org org per relation agent theme tr igger argument argument predicate arg0 arg-loc pronoun polarity polarity polarity aspect opinion polarity opinion tar get holder unification of xnlp more emerging xnlp tasks to define … one model for all as the key common characteristics, all the xnlp tasks have revolved around predicting two key elements from input: 1) textual spans and 2) relations between spans @xcite , as depicted in the figure 1 (lower part). traditional efforts for xnlp have treated each task independently, which has led to limited utilization of shared features among xnlp tasks @xcite , and sub-optimal model generalization across different datasets (chauhan et al., 2020), such as cross-language and cross-domain scenarios. in this paper, we emphasize the importance of model unification as a crucial topic in nlp. by unifying various nlp tasks under the xnlp framework, we can take advantage of the shared characteristics among tasks, leading to better model generalization and improved performance in realistic scenarios of product deployment. despite recent certain efforts in exploring universal solutions for some categories of xnlp tasks, such as unified sentiment analysis @xcite , and universal information extraction (uie; @xcite , a comprehensive and effective approach for unifying all xnlp tasks has not been fully established. fortunately, large language models (llms) @xcite present a potential solution for unification across all xnlp tasks. there is a recent development in the form of llms, e.g., chatgpt @xcite , llama @xcite and vicuna @xcite , that have shown promising advancements in nlp and other fields. llms, with sufficient sizes of model and data, have demonstrated impressive generalization capabilities, well supporting the idea of "one model for all" (ope-nai, 2023). in this work, we propose taking advantage of llms to achieve universal xnlp, addressing the lack of a well-defined and holistic approach. on the other hand, demonstration systems play a crucial role for researchers (especially beginners) exploring various xnlp tasks, providing a platform to analyze and understand the functionalities of different nlp components and their applications. while there are existing widely-used xnlp demo systems, such as corenlp foot_1 , allennlp foot_2 , we have observed several key issues with them: 1) limited to only a few specific tasks; 2) lacking interactive and extensible features, making it challenging to support dynamic growth in new xnlp tasks; 3) not universal systems, requiring separate models for each task, which can lead to increased overhead. to address these limitations, this work aims to build an advanced platform that provides superior xnlp demonstrations and benefits the broader nlp community. in summary, our system advances in the following aspects. 1) universalness ## system design architecture overview. we design our xnlp demo system into a web interface form. built based on the django foot_6 framework, xnlp divides the functions into the frontend module and the backend module. as shown in figure 2 , the frontend takes user inputs and displays the visualization of outputs, and the backend provides task prediction services with llm as its core engine, based on the in-context learning paradigm. also, it is possible for multi-turn interactions between frontend and backend. ## system walkthrough figure 3 gives a comprehensive walkthrough of how the system can be operated by users. the system to re-generate result; following we demonstrate xnlp system by walking readers through several important functions. ## performance evaluation to quantitatively verify the performance of the backbone llm on xnlp tasks, we now perform evaluations. we compare the vicuna (13b) with the chatgpt over 100 randomly selected test instances of 6 xnlp tasks. the experiments are based on one-shot in-context learning, i.e., with one demonstration as input. figure 6 shows the comparisons. we see vicuna has a slightly lower performance than chatgpt, while vicuna after broad-cover structure-aware instruction tuning (vi- ## conclusion we present xnlp, an advanced online demonstration system for interaction and visualization of xnlp tasks. xnlp, built upon llm, effectively models all the xnlp tasks universally, achieving one model for all in zero-shot or weak supervision. xnlp not only renders the output structures with delicate visualizations, but also provides rationales for interpretable predictions. also, xnlp allows the users to define newly emergent xnlp tasks; and enables users to dynamically revise the output with multi-turn interactions. our xnlp contributes to the community by paving the way for a unified, scalable, and interactive demonstration platform.