UWB @ F in TOC -2020 Shared Task: Financial Document Title Detection (2020)

Abstract: This paper describes our system created for the Financial Document Structure Extraction Shared Task (FinTOC-2020): Title Detection. We rely on the Apache PDFBox library to extract text and all additional information e.g. font type and font size from the financial prospectuses. Our constrained system uses only the provided training data without any additional external resources. Our system is based on the Maximum Entropy classifier and various features including font type and font size. Our system achieves F1 score 81% and #1 place in the French track and F1 score 77% and #2 place among 5 participating teams in the English track.

## introduction financial documents are used to report activities, financial situation, investment plans, and operational information to shareholders, investors, and financial markets. these reports are usually created on an annual basis in machine-readable formats often only with minimal structure information. the majority of these prospectuses are published without a table of content (toc), which is usually needed to help readers navigate within the document. the goal of the first financial document structure extraction shared task (fintoc-2019) @xcite was to analyse the financial prospectuses and automatically extract their structure similarly to @xcite . the second financial document structure extraction shared task (fintoc-2020) @xcite adds french documents and greatly simplifies the data formats at the cost of not providing any text representation of the pdf files. ## task the goal of fintoc-2020 shared task is to extract the table of content from the financial prospectuses. systems participating in this shared task were given a sample collection of financial prospectuses with different levels of structure and different lengths as training data. data statistics for the title detection subtask are shown in the shared task can be divided into two steps: 1) title detection classifies given text blocks as titles or non-titles. 2) toc generation organizes provided headers into a hierarchical table of content. we participated only in the title detection subtask for both languages. for additional information (e.g. about toc generation subtask) see the task description paper @xcite . ## dataset the provided training collection of documents contains the original documents in pdf format and annotations json file with gold labels. the json file consists of an array of toc items representing each title with the following properties: text -text of the title, id -order of occurrence the title, depth -depth level of the title, and page -page of title occurrence. in ## extraction we decided to use the apache pdfbox https://pdfbox.apache.org/ version 2.0.20 to extract text and other metadata from the pdf files and then we use our own algorithm to link the annotations to the extracted text representation. we consider each line of text a separate text segment and classify each segment as title or non-title. if there is a change in the font size or type we split the text into two lines. additional metadata are extracted from the first occurring word of the given line. the metadata include the following features: is bold, is italic, is all caps, begins with cap, begins with numbering, left position, font size, and font type. note that some of these features were difficult to extract as there are more ways to create e.g. bold text in pdf format and the library does not provide a convenient interface to access e.g. vector elements. ## issues the first shared task (fintoc-2019) had some issues with the mapping of the xml text representation to the annotated csv gold labels representation as reported by (hercig and král, 2019). the second shared task (fintoc-2020) removed the xml text representation and simplified the gold labels representation to json format, however, some of the problems still remained. we did not get any annotation guidelines or explanation of some labels. it seems that the annotation process was incoherent -leaving us with different levels of depth and various parts of the title included or left out depending probably on the annotator of the current file or title. we wrote an algorithm that tries to find the best mapping on a given page assuming the annotated text from the json training file appears in the same order of occurrence as the text extracted from the pdf file. unfortunately, that is not always true, thus we decided to modify the training json files and fix the issues, described in the following sections, which caused our algorithm to fail. we manually fixed only the necessary part of the dataset in order for our algorithm to work. ## conclusion in this paper, we described our uwb system participating in the fintoc 2020 shared task. our best results have been achieved by the maximum entropy classifier combining available metadata, such as font type and font size, by careful feature engineering. our system is ranked #1 in the french track and #2 among 5 participating teams in the english track.