# EmmDocClassifier: Efficient Multimodal Document Image Classifier for Scarce Data
## 1 Introduction  
Documents play a pivotal role in all of the fields of business communication and record-keeping [1]. Automatic information extraction from documents is a challenging task [2]. In general, the physical documents are first scanned or photographed before the information extraction process can begin. Document classification is considered an essential task in various Document Image Processing Pipelines (DIPP). The classification of documents into different known classes helps to improve the overall performance of document processing systems [1]. Consequently, many approaches are proposed for document classification that uses either text content [3; 4; 5] or document structure [6; 7; 8; 9] to categorize documents into different classes or use both of the modalities [10; 11; 12; 13]. There has been much advancement in this area, especially using deep learning methods [6; 14; 15]. With the advent of AlexNet [16], there has been tremendous growth in the research and performance of Deep neural networks. There have been numerous experiments and much research conducted with different computer vision models such as VGNet [17], Resnet [18], InceptionNet [19], etc. The documents are treated as conventional images and thus have the models delivered satisfactory results with classification tasks. Many documents are structurally the same but differ in the textual content. Therefore, these document images convey high-level structural information with their features, but the low-level features that can disambiguate visually similar images have remained uninvestigated for a long time. Various papers investigate the possibility of involving additional features to improve the accuracy such as [10; 11; 13]. These papers obtained state-of-the-art results. They use a powerful OCR (Optical Character Recognition) engine such as [20] to extract the text from the given document images and learn textual features along with visual features. This helps in solving cases where the documents are visually similar. In this paper, we employ a similar approach, i.e., we use both the visual and textual features. We initially pass the document Images through a visual stream. Previous approaches used much deeper models such as Resnet [18] and InceptionNet [19]. However, these networks require an enormous amount of training time. To speed up this process, we train the visual stream with EfficientNet [21]. Here, the model is scaled up uniformly with careful balance on the network's depth, width, and resolution. EfficientNet offers various models with different compound coefficients. In particular, EfficientNet-B0 and EfficientNet-B7 have better results, being up to six times faster while reducing the network depth up to 12% of previous sizes. This paper has used EfficientNet-B0, a lighter version of the model, but the results are better than other deep networks such as Resnet-50. As reported in [21], its Top-1 accuracy is nearly 1.5% more than Resnet-50, while it uses nearly five times fewer parameters. Therefore, we used this network that does not only help in improving the accuracy of the visual stream but also reduces the training time considerably. As discussed above, in some documents, e.g., Letters, Memos, and Reports, the contents predominantly have text information. Visually all of them seem similar and also have high inter-class similarity. Therefore, incorporating text features and visual features can result in better performance. Thus, we incorporate Hierarchical Attention Network [5] to incorporate text features. The hierarchical Attention Network model performs conventionally better as it splits the text into words and sentences. It receives the information through the Attention Model on both levels and combines them hierarchically, thus learning rich low-level features. This helps immensely in classifying the overlapping type of classes. To enhance this, instead of static word embedding, which is unaware of the context, we replace it with dynamic word embedding, BERT [22].