A Review on Document Information Extraction Approaches (2021)

Abstract: Information extraction from documents has become great use of novel natural language processing areas. Most of the entity extraction methodologies are variant in a context such as medical area, financial area, also come even limited to the given language. It is better to have one generic approach applicable for any document type to extract entity information regardless of language, context, and structure. Also, another issue in such research is structural analysis while keeping the hierarchical, semantic, and heuristic features. Another problem identified is that usually, it requires a massive training corpus. Therefore, this research focus on mitigating such barriers. Several approaches have been identifying towards building document information extractors focusing on different disciplines. This research area involves natural language processing, semantic analysis, information extraction, and conceptual modelling. This paper presents a review of the information extraction mechanism to construct a generic framework for document extraction with aim of providing a solid base for upcoming research.

## introduction digitization of documents and moving forward for the paperless community has been in crucial discussion in the past few decades, contributed to many research opportunities advancing from image processing, natural language processing, ontology, semantic analysis. such research has the focus on one particular narrowed problem. but none or fewer are supporting to provide a generalized approach for information extraction from documents. document processing is tremendous due to its diversity; even document generation could be in different ways, i.e., handwritten, scanned-handwritten, typewritten, machine-generated, machine-printed. in terms of content, again it is possible to divide among purpose (business, legal, technical, medical, academic, historical) @xcite , language (english, hindi, arabic, multi-lingual) @xcite . it is better to have a more generalized solution than limited to the scope of each application area. however, this would not be an easy task due to diversities in document nature, but worth exercising since it provides a novel way of addressing document processing issues. this paper focuses on querying documents tasks, such as in question answering, due to the expandability for several other applications. but, notably, the information extraction area itself is a vast research area employing different methodologies not only for targeting documents. such examples would be @xcite and @xcite discussing legacy open information extraction approaches. later, the works of @xcite , @xcite , @xcite and @xcite explains more advanced approaches in open information extraction, but this is not the scope in this paper as this emphasis on document information extraction aspects. in @xcite explain a reading strategy of the humanized approach of document extraction, where first skim through the test to get a general idea, read the question or query carefully with the equipped general understanding from the document and then search for the answer with the prior knowledge. this paper is structured to guide through a stepby-step approach of document information extraction while checking on crucial research at a legacy level and recent advancements. these ## ocr handwritten documents the paper @xcite discusses languageindependent rule-based classification for handwritten text and the printed text for data entry forms, mainly used in extracting information from form documents. it uses structural and statistical features derived into a rule-based system to determine the origin of the text. it considers baseline characteristics and stroke characteristics to differentiate. @xcite propose a methodology for full-page offline text recognition using deep neural networks incorporating object detection neural networks, multi-scale cnn and bidirectional lstm to perform text localization and then text recognition. in-text localization performs passage identification and line segmentation using iam datasets and resnet34 on imagenet. then during the text recognition phase, cnn-bilstm will output nxm in terms of probabilities, and with the language modelling, it has converted to a string of text using the beam search approach. figure 2 illustrates the overview of the system they proposed. ## derive concept relations a segmenting stream of documents using structural and factual descriptions discuss in @xcite . it explains 4 document levels: records, technical documents, cases, and fundamental documents. this research proposes a novel method for stream segmentation in stream data. in the paper, a logbook for the previous pages they have used to compare descriptors. the system diagram illustrates as follows in figure 1 . logical labelling has been discussed in the study @xcite , using 2d conditional random fields to derive the page structure in machineprinted documents. it constructs neighbourhood graphs and pairwise cliques templated while combining local and contextual features obtained from the pdf document. using an svm classifier, it derives the baseline without any contextual information considered. early knowledge-based context analysis considers primary context, the relation between text and text, non-text, and text. one of the ocr applications is digitizing historical books in libraries where the searchable text is derived. although the existing ocr applications extract simple structures such as figure 1: system diagram of descriptors (karpinski and bela¨ıd, 2016) ## derive semantic structure document semantic structure extraction is a separate research discipline where document image is segmented into the region of interest and provide a semantic explanation for each @xcite . this paper involves two phases, page segmentation and concept-relation analysis. @xcite present a multimodal fully convolutional network to extract semantic structure in terms of pixel-wise segmentation task. the paper @xcite proposes an approach to dimensionality reduction of feature vector size for web text document classification using wordnet ontology. this paper has proposed a vector space model for the mining task and used term frequency inverse document frequency for the task weighting. the suggested approach is ontology base, and for the evaluation, it uses principal component analysis. ## document entity recognition differential tools use for document entity recognition evaluate in @xcite . in the entity recognition process, it analyses the features of lexical, semantic, and heuristic. english corpus included around 579 words. based on entity types of results are evaluated. finally, this paper proposes a model that eliminates the limitations identified. also, the corresponding model. is based on the small corpus. the best tools are supersense -wnss and the lowest performance from afner. in @xcite uses an nlp pipeline that begins with part of speech tagging and then use chunking to obtain ner. it uses shallow syntax in tweets using annotated tweets and build-tools trained on data. also, have used features generated from t-pos & t-chunk in segmenting named entity. the paper @xcite discusses the meaning behind named entity recognition and analyses the field from theoretical and practical views considering drawbacks, challenges, and opportunities in the area. it brought the argument to show that this task mainly depends on the development and evaluation of the tools. ner can apply in many practical scenarios, such as in twitter post analysis as explained in @xcite . it identifies several named entities and classifies them among people, locations and organizations. also, it discusses identifying the language-specific features and methods effectively. further, this paper elaborates on how it can improve to the areas of ner performing on other text sources such as news articles and less structured microblogging texts. throughout the process of research, it identifies the difference between the microblogging text and the formal process. the paper @xcite proposes a framework for named entity detection from web content associated with semi-structured text data records by exploiting the inherent structure via a transformation process facilitating collective detection. this framework does not require training labels on the data records to learn the sequential classification model. instead, it makes use of existing named entity repositories such as @xcite dbpedia. it incorporates this external clue via distant supervision by making use of the generalized expectation constraint. finally, a collective detection model supporting the logical inference proposed to consider the consistency among potential named entities and header text. ## state-of-the-art language models several language models have been emerged in more recent research, word embeddings, transformer-based models being key highlights. these model architectures and pre-trained models are used for document-based language modelling successfully. the paper @xcite presents an attention-based novel architecture named. transformers. it mainly uses in natural language processing for tasks such as text summarization and translations. the main advantage in transformers is it does not require sequential data to insert in an orderly manner. because of that, it allows parallelization compared to recurrent neural networks and reduces the overall training time due to that. pretrained models such as bert (bidirectional encoder representations from transformers) @xcite are using this model. transformers follows encoder-decoder architecture where the encoder contains several encoding layers, and the decoder consists of several decoder layers. it was designed based on how long sentences memorize in neural machine translation. the input sentence is first embedded and then passed into a stack of encoders. then the output from the last encoder pass to each decoder layer. in terms of the internal architecture, there are two main parts in the encoder, namely, selfattention and feed-forward neural network. and the decoder follows similar architecture to the encoder. there is an encoder-decoder attention in between these two, which follows the same pattern as multiheaded self-attention. it takes sequential input for the decoder. the output will pass to another linear layer with softmax activations following the same concepts as forwarding propagation and loss functions such as crossentropy to update trainable parameters. elmo @xcite provides deep contextualized word representation using a bidirectional lstm model to get word representations. it analyses the words by considering the context used. as it uses characterbased representation, it creates dynamic vectors from the given input text. in elmo, it learns both word and linguistic context. the pre-trained model can utilize for any downstream required nlp task, such as question answering, coreference resolution, textual entailment, named entity extraction, semantic role labelling and sentiment analysis. multiple word embeddings produce for every single word considering different scenarios. higher-level layers use to capture contextdependent aspects. and the model aspects such as syntax are captured by lower-level layers. elmo @xcite performs language modelling by training to predict the next word for the given word sequence. this contextual embedding is by grouping hidden states by concatenation. the encoder part in bert @xcite ) is similar to the architecture in transformers as it consists of a feed-forward neural network and selfattention heads. these two segments act as the training parameters to differentiate within different bert @xcite models. the various applications of bert are question-answer modelling, classification tasks such as spam detection, answer generation from a given passage, sentence tagging such as entity recognition. ## question answering from documents since any generic document information extraction framework should be able to provide answers for the given queries, we must consider the design aspects of question answering systems, where it requires the composing of several components, named entity recognition, semantic analysis of the question, named entity disambiguation, query expansion and execution. the study @xcite has proposed, web and semantic knowledge driven (wad) approach to increase the accuracy of the document selection. this approach first evaluates the user query, entity linking method and the query expansion technique. finally, ontological information uses to rank the answers. the paper @xcite suggests a graphbased question answering system for reading comprehension tests, where it picks the sentence in the given passage which gives the best answer. it uses combined techniques of morphological analysis, coreference resolution and synonym checks to achieve higher accuracy. ## conclusion most of the research is firmly dependent upon the dataset/corpus and limited to the addressed language. the suggested model is usually trained and implemented within the given environment, whereas less performance for complex unstructured documents. for some research, a precompiled training dataset requires building the model so that when applying within a different environment, an additional effort need for preparing such datasets. also, the current state of art referring to this study is usually subjective for the problem area such as academic, business, biomedical. as per future trends regarding this research area, most recent studies are moving towards extracting information by considering the conceptual factors rather than digitizing the text and then grabbing the semantics. current approaches are limited to given scope, document type, structure, language. applying this research in real-time applications will not provide aid for day-to-day applications. in the existing approaches, a large corpus of documents uses for training and fitting the model. a requirement for a generalized framework capable of extracting information from any document exists due to that. the key idea is to mimic how a human understands an unknown text document through the content and the relation between the sections. such a generalized framework will employ the benefits from existing approaches, but not solely dependent, as the main framework components could be replaceable by any other future approaches by providing a baseline. this review will provide a solid background analysis before approaching this generic framework.