PDF -to-Tree: Parsing PDF Text Blocks into a Tree (2024)

Abstract: In many PDF documents, the reading order of text blocks is missing, which can hinder machine understanding of the document’s content.Existing works try to extract one universal reading order for a PDF file.However, applications, like Retrieval Augmented Generation (RAG), require breaking long articles into sections and subsections for better indexing.For this reason, this paper introduces a new task and dataset, PDF-to-Tree, which organizes the text blocks of a PDF into a tree structure.Since a PDF may contain thousands of text blocks, far exceeding the number of words in a sentence, this paper proposes a transition-based parser that uses a greedy strategy to build the tree structure.Compared to parser for plain text, we also use multi-modal features to encode the parser state.Experiments show that our approach achieves an accuracy of 93.93%, surpassing the performance of baseline methods by an improvement of 6.72%.

## introduction document ai is a research field that has emerged in recent years. it focuses on automating the reading, comprehension, and analysis of data in pdf documents. these documents can be either scanned or digital-born(rendered) files. although many pdfs are digital-born, their formats were designed for layout purposes. as a result, the structural information retained within them is often incomplete, which can hinder machine understanding of the document's content. there's a lot of research on pdf layout analyzing, like categorizing text blocks and predicting the relationships between them. earlier studies relies on purely visual features, such as, deep-desrt @xcite , pdftabledetection @xcite , visualdetection @xcite and dhsegment(ares oliveira et al., title abstract section subsection paragraph token token paragraph token token subsection table table row cell footnote annotation [a] [b] [c] ), text blocks are independent of each other and don't have a specific order. the reading order prediction task @xcite ) can partially resolve this issue. however there's only one text sequence, footnotes, captions and other irrelevant text blocks are inserted into the main text sequence. this might lead to confusion. additionally, some applications, like rag, require breaking down long articles into sections for better content retrieval. to tackle this issue, we propose a new task and dataset, pdf-to-tree task([c]), which organizes text blocks into a tree structure for downstream task to retrieve. step stack buffer transition a ← ∅ 0 [root] [t1, t2, t3, t4, t5] arc(root, t1, title) a∪ (root, t1, title) 1 [root, t1] [t2, t3, t4, t5] arc(t1, t2, abstract) a∪ (t1, t2, abstract) 2 [root, t1, t2] [t3, t4, t5] shift a ... ... ... ... ... we leverage a transition-based parser to predict tree structure of a document from a sequence of input text blocks. the input document is processed by pdf miner or ocr tools to get text block sequence. then the sequence is processed by the parser to predict a serial of transition actions that build the tree. the parser archive this by using a buffer to hold input sequence and a stack to hold intermediate tree nodes. in each step, the parser predicts a shift or arc operation that pops elements from the buffer and reconstructs the tree in the stack. to address the issue of missing structure information in pdf documents, existing research develops the task of reading order prediction. this involves predicting a global rank for each text block and linking them into a text sequence. such efforts include layoutparser @xcite , layoutreader @xcite and ernielayout @xcite . however, we believe that placing all text blocks in the same sequence is insufficient. this is because there are independent document elements like footnotes and captions, which should have their own reading order. especially for rag applications, which require breaking down long documents into sections, subsections and table rows, a more detailed method is needed to represent the content of pdf documents. therefore, we introduce the pdf-to-tree task and dataset. by organizing text blocks into a tree structure, we aim to solve the issue of complex document structures. we manually annotate the tree structures of 9,310 pdf pages. as shown in figure 1 , through the pdf-to-tree task, text blocks in a pdf are organized into a tree structure. compared to reading unordered text blocks directly from a pdf file, downstream tasks can accurately access the content needed from the document through its tree structure. additionally, we propose a transition-based parser that effectively completes the pdf-to-tree task. there are multiple ways to construct a tree from a sequence. considering a pdf may contain thousands of text blocks, depending on the document's length, we opt for a transition-based parser to address this issue. compared to other algorithms (the minimum spanning tree or pairwise linking methods), the time and memory complexity of a transition-based parser scales linearly with the length of the document. unlike plain text dependency parsing, text blocks in pdf documents contain more information besides text, such as visual details and layout. we also use features from different modalities to encode the parser state. in general, our main contributions are in three folders: ## experiments in this section, we dive into the implementation details of pdf-to-tree and conduct experiments on the pdf-to-tree dataset. also, we create a baseline with bros @xcite , structext @xcite and layoutlmv2-re @xcite . those models are commonly used approaches in structured text understanding. all experiments were carried out using one to eight nvidia tesla a800 80gb gpus. ## results in this section, we compare our model with the baselines on both pdf-to-tree and funsd datasets. we also evaluate the effects of various modality encoders on structure parsing for the pdfto-tree dataset. additionally, we perform error analysis, ablation experiments, and inference speed analysis. ## related work document ai is a research area that has gained attention in recent years. there's a lot of valuable information stored in the form of digital documents. the goal of document ai is to extract and convert digital documents into structured data. jaume et al. divides document ai into two subtasks: one involves categorizing blocks within the document to obtain labels for these blocks, which is called entity labeling. the other involves establishing connections between blocks to identify the relationships between them, which is known as entity linking. in addition to these two tasks, there is also a lot of research that mainly focuses on how to obtain better multimodal feature representations of documents. ## discusion in this paper, we discuss the task of document structure parsing. this task is more intricate compared to the traditional reading order prediction. for complex layout documents, which may contain multiple reading orders.. to address these challenges, we introduce a transition-based parser as a solution. alongside this, we introduce a new dataset called pdf-to-tree to support this task. experimental results demonstrate the effectiveness of our approach. however, there is still room for improvement in identifying less common labels. moreover, there are areas where the efficiency of inference can be enhanced. ## limitation all the pdfs used in the pdf-to-tree dataset are born digital(rendered). in theory, our method could also be applied to scanned documents. however, due to resource constraints, it has not been used on the pdf-to-tree dataset yet. in future work, we plan to include scanned documents in our dataset as well. additionally, the high cost of manual labeling limits the amount of annotated data we can obtain. in future work, we believe it's worth discussing how to automatically label the tree structure of a document.