AMEX - AI - LABS : Investigating Transfer Learning for Title Detection in Table of Contents Generation (2020)

Abstract: We present a transfer learning approach for Title Detection in FinToC 2020 challenge. Our proposed approach relies on the premise that the geometric layout and character features of the titles and non-titles can be learnt separately from a large corpus, and their learning can then be transferred to a domain-specific dataset. On a domain-specific dataset, we train a Deep Neural Net on the text of the document along with a pre-trained model for geometric and character features. We achieved an F-Score of 83.25 on the test set and secured top rank in the title detection task in FinToC 2020.

## introduction title detection and table of contents generation are important sub-parts of the bigger problem, known as, document structure analysis. understating the inherent document layout and structure benefits several downstream document ai tasks such as search, summarizing, entity extraction and table detection etc. from a document. humans glance at a document and comprehend the document structure including the titles vs non-titles as well as the overall hierarchy of the titles. many reasons can be attributed to it, like the sequential nature of the document, geometrical features or the semantic meaning of the sentences. we have tried to incorporate these basic human instincts into our model. humans have intuitive notions of how a document is structured and the assumption is confirmed after reading a text block. transfer learning can be used to model the structural properties of a general document. we use arxiv documents foot_0 available in the open-domain to learn the structural model of a general document. semantic properties are learned using lstm @xcite at title level for a domain specific document. the final model is trained on a domain-specific dataset with structural weights pre-trained from arxiv documents. we see considerable improvements by applying transfer learning to the title detection task. combining deep neural networks based on manual features and character cnn @xcite on the starting eight characters helps us model the structural signature of a general document. ## related work the literature on title detection can be classified broadly into three categories: works that deal with toc page of documents, works that use images of document pages and works which use the geometrical and textual features of the text blocks. in the approaches dealing with toc pages of documents, after the toc pages are detected, the title entries are extracted and mapped to the pages by finding links between title and corresponding pages. el @xcite used this approach in detecting titles in uk financial reports. as they rely on toc pages, they cannot be applied to documents that do not have toc pages. other approaches use computer vision to fragment the page image into entities such as text, title and table. @xcite used multimodal fully convolutional neural networks for this task. @xcite used convolutional neural networks combined with graphical models to identify the entities in a document page. few datasets for these tasks are publicly available. @xcite finally, some approaches use learning or rule-based methods to detect headers based on textual and geometrical features @xcite @xcite @xcite . these methods are usually used in digitally generated documents like webpages and native pdf documents. ## methodology we pre-process the pdf files by converting them to xml documents by poppler foot_1 library. these files are then parsed to merge elements similar in styling and located in close proximity. headers and footers are identified and removed by page association @xcite as they would hinder the process of title detection. our proposed title detection method has three components; pre-trained neural network to model general structural information, sequential network to learn domain-specific text and training of both the networks combined.