TKGT : Redefinition and A New Way of Text-to-Table Tasks Based on Real World Demands and Knowledge Graphs Augmented LLM s (2024)

Abstract: The task of text-to-table receives widespread attention, yet its importance and difficulty are underestimated. Existing works use simple datasets similar to table-to-text tasks and employ methods that ignore domain structures. As a bridge between raw text and statistical analysis, the text-to-table task often deals with complex semi-structured texts that refer to specific domain topics in the real world with entities and events, especially from those of social sciences. In this paper, we analyze the limitations of benchmark datasets and methods used in the text-to-table literature and redefine the text-to-table task to improve its compatibility with long text-processing tasks. Based on this redefinition, we propose a new dataset called CPL (Chinese Private Lending), which consists of judgments from China and is derived from a real-world legal academic project. We further propose TKGT (Text-KG-Table), a two stages domain-aware pipeline, which firstly generates domain knowledge graphs (KGs) classes semi-automatically from raw text with the mixed information extraction (Mixed-IE) method, then adopts the hybrid retrieval augmented generation (Hybird-RAG) method to transform it to tables for downstream needs under the guidance of KGs classes. Experiment results show that TKGT achieves state-of-the-art (SOTA) performance on both traditional datasets and the CPL. Our data and main code are available at https://github.com/jiangpw41/TKGT.

## introduction extracting structured information from unstructured or semi-structured text is crucial to natural language processing @xcite . it involves extracting valuable information through rule-based, statistical, or deep learning (dl) methods to compress texts and our cpl. the horizontal axis represents the percentile of the ordered word frequency lists, and the vertical axis represents the maximum similarity between each word and datasets' field sets. the intersection point is the maximum value point after 1% of each list, whose lengths and shapes are in the parentheses of each subgraph title. further explains are in or in section 2.2. and facilitate downstream applications @xcite @xcite . with the recent development of deep learning (dl), particularly the llms, several studies have explored the potential for transformer models to revolutionize traditional information extraction (ie) @xcite @xcite . some of these works focus on directly transforming raw text to structured forms such as kgs @xcite , mind maps @xcite , and tables @xcite @xcite , with tables being the most popular form. converting raw text to tabular data, or text-totable, is a widely recognized and essential task in ie due to its broad application potential. however, the complexity of text-to-table tasks is often artificially simplified to the point where it is divorced from real world demands. this issue manifests in two ways: (1) text used in current tasks is often structurally simple or fictional; (2) table is frequently simplified to single-digit dimensions, with the table fields often preset as known. this is primarily because the datasets traditionally used for text-to-table tasks are mainly derived from table-to-text tasks, which focus on generating brief descriptive content from a small set of database records @xcite @xcite . as shown in however, the real world is filled with a multitude of complex texts, such as cpl, which not only have longer lengths but also cover higher-dimensional information dimensions. due to the overly simplistic datasets used in existing text-to-table tasks, the methodologies developed from these are inadequate for addressing the need to structure lengthy and complex texts. textto-table is initially modeled as seq2seq tasks @xcite , embedding tokens to learn inner similarities and generate table rows end-to-end with the data-driven approach. later research introduced methods for inferring table fields @xcite before traversing texts with re and then merging the results @xcite . some works also utilize structures of text and hope to reduce difficulty through segmentation @xcite . after the emergence of llms, question and answer (q&a) is explored as an approach for ie @xcite . however, existing works often overlook the importance and difficulty of building table fields, treat them as known, or extract triples by simply crawling. such methods are only effective for simple formats, as identifying valuable information in complex texts and building appropriate fields require expert knowledge. ensuring completeness is also a challenge, especially for long texts whose valuable points may be scattered throughout the text or obscured by multiple perspectives. recognizing the importance of long, logically complex texts in capturing real-world information and the pressing need to structure them, especially in the social sciences, we propose a redefinition of the text-to-table task to enhance its compatibility with long text processing. specifically, the text should (1) concentrate on a specific domain topic; (2) possesses a certain logical flow and a relatively clear structure; (3) can be modeled as either a multi-attribute entity or a series of multientity events. as for the table, (1) it is typically crafted to fulfill practical needs; (2) its fields can be devised to be high-dimensional, limited, and well-defined to achieve comprehensive coverage of textual information. accordingly, we propose tkgt (text-kg-table ), a two-stage text-to-table method with kgs as middleware. in the first stage, the mixed-ie method based on regulations, statistics, and dl is used to obtain topic keywords and to construct domain kgs sketch, based on which users can better understand the datasets and easily form uninstantiated kgs adapting to downstream tabular needs using llms. in the second stage, based on dynamic prompts and hybrid-rag supported by descriptions of empty kgs classes, table content can be filled with llms q&a. through experiments, tkgt achieves sota performance on both traditional datasets and cpl. our contributions are summarized as follows: ## datasets and statistics considering the gaps in existing datasets from the real world, we construct the cpl dataset derived from a real world legal academic project initiated by the center for empirical legal studies of shanghai jiao tong university (cels) foot_0 . the dataset's raw texts are sourced from the china judgments online (cjo) foot_1 , compiled through the diligent efforts of legal experts, as outlined in appendix a. we also conduct experiments on two benchmark datasets from the traditional text-to-table task: rotowire @xcite and @xcite . ## experiments this section introduces the experimental setup and results of tkgt's two stages respectively.