Reviewerly/exHarmony huggingface dataset:
Context-Aware Chart Element Detection

As a prerequisite of chart data extraction, the accurate detection of chart basic elements is essential and mandatory. In contrast to object detection in the general image domain, chart element detection relies heavily on context information as charts are highly structured data visualization formats. To address this, we propose a novel method CACHED, which stands for Context-Aware Chart Element Detection, by integrating a local-global context fusion module consisting of visual context enhancement and positional context encoding with the Cascade R-CNN framework. To improve the generalization of our method for broader applicability, we refine the existing chart element categorization and standardized 18 classes for chart basic elements, excluding plot elements. Our CACHED method, with the updated category of chart elements, achieves state-of-the-art performance in our experiments, underscoring the importance of context in chart element detection. Extending our method to the bar plot detection task, we obtain the best result on the PMC test dataset. Our code and model are available at https://github.com/pengyu965/ChartDete .
The Bullinger Dataset: A Writer Adaptation Challenge

One of the main challenges of automatically transcribing large collections of handwritten letters is to cope with the high variability of writing styles present in the collection. In particular, the writing styles of non-frequent writers, who have contributed only few letters, are often missing in the annotated learning samples used for training handwriting recognition systems. In this paper, we introduce the Bullinger dataset for writer adaptation, which is based on the Heinrich Bullinger letter collection from the 16th century, using a subset of 3,622 annotated letters (about 1.2 million words) from 306 writers. We provide baseline results for handwriting recognition with modern recognizers, before and after the application of standard techniques for supervised adaptation of frequent writers and self-supervised adaptation of non-frequent writers.
TRACE: Table Reconstruction Aligned to Corner and Edges

A table is an object that captures structured and informative content within a document, and recognizing a table in an image is challenging due to the complexity and variety of table layouts. Many previous works typically adopt a two-stage approach; (1) Table detection(TD) localizes the table region in an image and (2) Table Structure Recognition(TSR) identifies row- and column-wise adjacency relations between the cells. The use of a two-stage approach often entails the consequences of error propagation between the modules and raises training and inference inefficiency. In this work, we analyze the natural characteristics of a table, where a table is composed of cells and each cell is made up of borders consisting of edges. We propose a novel method to reconstruct the table in a bottom-up manner. Through a simple process, the proposed method separates cell boundaries from low-level features, such as corners and edges, and localizes table positions by combining the cells. A simple design makes the model easier to train and requires less computation than previous two-stage methods. We achieve state-of-the-art performance on the ICDAR2013 table competition benchmark and Wired Table in the Wild(WTW) dataset.
Evaluation of Different Tagging Schemes for Named Entity Recognition in Handwritten Documents

Performing Named Entity Recognition on Handwritten Documents results in categorizing particular fragments of the automatic transcription which may be employed in information extraction processes. Different corpora employ different tagging notations to identify Named Entities, which may affect the performance of the trained model. In this work, we analyze three different tagging notations on three databases of handwritten line-level images. During the experimentation, we train the same Convolutional Recurrent Neural Network (CRNN) and n-gram character Language Model on the resulting data and observe how choosing the best tagging notation depending on the characteristics of each task leads to noticeable performance increments.
Language Independent Neuro-Symbolic Semantic Parsing for Form Understanding

Recent works on form understanding mostly employ multimodal transformers or large-scale pre-trained language models. These models need ample data for pre-training. In contrast, humans can usually identify key-value pairings from a form only by looking at layouts, even if they don’t comprehend the language used. No prior research has been conducted to investigate how helpful layout information alone is for form understanding. Hence, we propose a unique entity-relation graph parsing method for scanned forms called LaGNN, a language-independent Graph Neural Network model. Our model parses a form into a word-relation graph in order to identify entities and relations jointly and reduce the time complexity of inference. This graph is then transformed by deterministic rules into a fully connected entity-relation graph. Our model simply takes into account relative spacing between bounding boxes from layout information to facilitate easy transfer across languages. To further improve the performance of LaGNN, and achieve isomorphism between entity-relation graphs and word-relation graphs, we use integer linear programming (ILP) based inference. Code is publicly available at https://github.com/Bhanu068/LAGNN .
DQ-DETR: Dynamic Queries Enhanced Detection Transformer for Arbitrary Shape Text Detection

We propose a new Transformer-based text detection model, named Dynamic Queries enhanced DEtection TRansformer (DQ-DETR), to detect arbitrary shape text instances from images with high localization accuracy. Unlike previous Transformer-based methods which take all control points on the boundaries/center-lines of all text instances as the queries of each Transformer decoder layer, we extend the query set for each decoder layer gradually, allowing the DQ-DETR to achieve higher localization accuracy by detecting control points for each text instance progressively. Specifically, after refining the positions of existing control points from the preceding decoder layer, each decoder layer further appends a new point on each side of each center-line segment, which are input to the next decoder layer as additional queries for detecting new control points. As offsets from the new control points to the added reference points are small, their positions can be predicted more precisely, leading to higher center-line detection accuracy. Consequently, our DQ-DETR achieves state-of-the-art performance on five public text detection benchmarks, including MLT2017, Total-Text, CTW1500, ArT and DAST1500.
Information Extraction from Documents: Question Answering Vs Token Classification in Real-World Setups

Research in Document Intelligence and especially in Document Key Information Extraction (DocKIE) has been mainly solved as Token Classification problem. Recent breakthroughs in both natural language processing (NLP) and computer vision helped building document-focused pre-training methods, leveraging a multimodal understanding of the document text, layout and image modalities. However, these breakthroughs also led to the emergence of a new DocKIE subtask of extractive document Question Answering (DocQA), as part of the Machine Reading Comprehension (MRC) research field. In this work, we compare the Question Answering approach with the classical token classification approach for document key information extraction. We designed experiments to benchmark five different experimental setups : raw performances, robustness to noisy environment, capacity to extract long entities, fine-tuning speed on Few-Shot Learning and finally Zero-Shot Learning. Our research showed that when dealing with clean and relatively short entities, it is still best to use token classification-based approach, while the QA approach could be a good alternative for noisy environment or long entities use-cases.
Unraveling Confidence: Examining Confidence Scores as Proxy for OCR Quality

While performing Optical Character Recognition (OCR), most engines provide confidence scores. These scores give an indication on how certain an engine is that a word or character is correctly determined. The practical application of this score is not yet clear and various studies have discussed the (un)usability of these confidence score as an estimation of OCR quality. Using a dataset of 2000 historical Dutch newspapers we investigated different aspects of the confidence score as provided by ABBYY Finereader, while also looking for a way to use the confidence score as an indication of quality. Such an indication could be used by institutions to determine which part of their collection would benefit from re-OCRing or post-processing. We found that the reliability of the confidence score as a measure of quality is largely dependent on the way the engine has been configured. In addition we show that when there is a high enough correlation between the word confidence and the Word Character Error (order independent) the word confidence can be used to calculate a proxy measure for categorizing digitized texts. However, such a measure must be recalculated for individual OCR engine set ups and producers. For our dataset this proxy measure performs well for the separation of digitized texts into categories of those with a very good and those with a very bad quality with total accuracy of 83%.
Structure Diagram Recognition in Financial Announcements

Accurately extracting structured data from structure diagrams in financial announcements is of great practical importance for building financial knowledge graphs and further improving the efficiency of various financial applications. First, we proposed a new method for recognizing structure diagrams in financial announcements, which can better detect and extract different types of connecting lines, including straight lines, curves, and polylines of different orientations and angles. Second, we developed a semi-automated, two-stage method to efficiently generate the industry’s first benchmark of structure diagrams from Chinese financial announcements, where a large number of diagrams were synthesized and annotated using an automated tool to train a preliminary recognition model with fairly good performance, and then a high-quality benchmark can be obtained by automatically annotating the real-world structure diagrams using the preliminary model and then making few manual corrections. Finally, we experimentally verified the significant performance advantage of our structure diagram recognition method over previous methods.
PyramidTabNet: Transformer-Based Table Recognition in Image-Based Documents

Table detection and structure recognition is an important component of document analysis systems. Deep learning-based transformer models have recently demonstrated significant success in various computer vision and document analysis tasks. In this paper, we introduce PyramidTabNet (PTN), a method that builds upon Convolution-less Pyramid Vision Transformer to detect tables in document images. Furthermore, we present a tabular image generative augmentation technique to effectively train the architecture. The proposed augmentation process consists of three steps, namely, clustering, fusion, and patching, for the generation of new document images containing tables. Our proposed pipeline demonstrates significant performance improvements for table detection on several standard datasets. Additionally, it achieves performance comparable to the state-of-the-art methods for structure recognition tasks.
Line Graphics Digitization: A Step Towards Full Automation

The digitization of documents allows for wider accessibility and reproducibility. While automatic digitization of document layout and text content has been a long-standing focus of research, this problem in regard to graphical elements, such as statistical plots, has been under-explored. In this paper, we introduce the task of fine-grained visual understanding of mathematical graphics and present the Line Graphics (LG) dataset, which includes pixel-wise annotations of 5 coarse and 10 fine-grained categories. Our dataset covers 520 images of mathematical graphics collected from 450 documents from different disciplines. Our proposed dataset can support two different computer vision tasks, i.e., semantic segmentation and object detection. To benchmark our LG dataset, we explore 7 state-of-the-art models. To foster further research on the digitization of statistical graphs, we will make the dataset, code and models publicly available to the community.
Layout Analysis of Historical Document Images Using a Light Fully Convolutional Network

In the last few years, many deep neural network architectures, especially Fully Convolutional Networks (FCN), have been proposed in the literature to perform semantic segmentation. These architectures contain many parameters and layers to obtain good results. However, for Historical document images, we show in this paper that there is no need to use so many trainable parameters. An architecture with much fewer parameters can perform better while being lighter for training than the most popular variants of FCN. To have a fair and complete comparison, qualitative and quantitative evaluations are carried out on various datasets using standard pixel-level metrics.
Optimized Table Tokenization for Table Structure Recognition

Extracting tables from documents is a crucial task in any document conversion pipeline. Recently, transformer-based models have demonstrated that table-structure can be recognized with impressive accuracy using Image-to-Markup-Sequence (Im2Seq) approaches. Taking only the image of a table, such models predict a sequence of tokens (e.g. in HTML, LaTeX) which represent the structure of the table. Since the token representation of the table structure has a significant impact on the accuracy and run-time performance of any Im2Seq model, we investigate in this paper how table-structure representation can be optimised. We propose a new, optimised table-structure language (OTSL) with a minimized vocabulary and specific rules. The benefits of OTSL are that it reduces the number of tokens to 5 (HTML needs 28+) and shortens the sequence length to half of HTML on average. Consequently, model accuracy improves significantly, inference time is halved compared to HTML-based models, and the predicted table structures are always syntactically correct. This in turn eliminates most post-processing needs. Popular table structure data-sets will be published in OTSL format to the community.
Cyberbullying Detection and Classification in Social Media Texts Using Machine Learning Techniques

Cyberbullying (CB) is both a public health concern as well as an emotional problem. There have been efforts to mitigate this problem from different discipline dimensions. Artificial Intelligence (AI) has recently emerged as a solution to CB and outshined many earlier solutions. In this work, we have investigated the problem of CB on social media platforms using Machine/Deep Learning and Natural Language Processing (NLP) techniques. Using a dataset containing 47,692 tweets, we investigated the task of detecting CB from social media posts and classifying them as either age-based, religious, ethnic, and political CB or neutral (non-CB). We spot-checked 5 Machine Learning (ML) algorithms (Gradient Boosting, Logistic Regression, Naïve Bayes, Random Forest, and Support Vector Machine) and one Deep Learning algorithm (a sequence model). The algorithms were evaluated based on accuracy, precision, recall, and F1 score. Random Forest reported the best accuracy of 93% while Naive Bayes reported the worst accuracy of 84%, while the DL model had a classification accuracy of 91%. The developed models can help detect and classify CB sentiments in social media posts, thus reducing the harm caused by CB in the social media space.
Child Access Control Based on Age and Personality Traits

Exposure to harmful information on the Internet and constant use of digital devices (computer, phone, tablet, etc.) harm children’s psychology and health. The use of methods that control access to the Internet and filter web content considered harmful to children is an effective means of solving this problem. The article proposes a method to control access to the Internet, taking into account several personality traits of the user-child (age, eye diseases, heart diseases, neurological and psychological conditions, etc.) using Mamdani-based fuzzy logic inference system. The values of the input parameters of this system are described by five linguistic parameters, which are “very low”, “low”, “medium”, “high” and “very high” and with a triangular membership function. This approach is focused on the individual user and the main advantage is that parameters are used in vector form. This research is significant for the use of parents, guardians and those responsible for children.
Radial Mean LBP (RM-LBP) for Face Recognition

In literature various local descriptors has been invented for Face Recognition (FR) in unconstrained conditions. Most of these local descriptors are inspired from the LBP concept. With the result numerous LBP variants are introduced. Some achieved good results, and some are not. The reason behind unsatisfactory performance is the lack of building the discriminant concept in forming the descriptor. The major significance of any research is to develop the novel concept/method in form of descriptor, and it should be discriminant and robust, which should justify it by defeating the various benchmark methods. In proposed work such novel concept is introduced in the form of novel local descriptor called as Radial Mean LBP (RM-LBP), in different unconstrained conditions. In RM-LBP, initially mean value is obtained from 4 radial pixels located at 4 different radii, in eight directions of 9 × 9 patch. This forms the 3 × 3 patch, in which neighbors are filled with mean values and center place is filled with median of those. Ultimately for making the RM-LBP code, the neighbors are compared with the center pixel. After obtaining RM-LBP map image (by computing codes in each location) the size derived is 256. To compress and match reduced feature, PCA and SVMs are used. For experiments evaluation ORL face dataset is used. Results suggests that RM-LBP is better and efficient descriptor than many others. The 6 other evaluated descriptors and many literature approaches are easily outclassed by RM-LBP. RM-LBP achieves the ACC of [72.22% 84.37% 92.85%], which proves higher than many others. The significance of RM-LBP is justified when it defeats the numerous benchmark methods. The matlab version used for evaluation is MATLAB R2021a.
Target Performance of University Logistics Teachers Based on the Combination of AHP and Support Vector Machine in the Context of “Double First-Class” Project Construction

In the future for a long period of time, “double first-class” has become the direction and subject of higher education reform and development. Based on the historical background of “double first-class” construction, the construction of teacher performance evaluation system in colleges and universities will change accordingly. The construction of teacher performance evaluation system is particularly important. As a means and tool of evaluation, it can be applied to the management of teacher performance evaluation in colleges and universities to verify the performance of teachers’ responsibilities, improve their initiative, enhance their sense of competition, and improve their teaching level. Thus, a good competition and incentive mechanism can be formed, which has gradually become an important part of university management. Under the background of “double first-class” construction, this paper selects the main indicators that affect the target performance of logistics teachers in universities and colleges, and constructs an evaluation system based on them. At the same time, the principle of analytic hierarchy process is used to build the hierarchy index system, and the support vector machine is used for comprehensive evaluation, in order to further improve the teaching level of logistics teachers in colleges and universities, scientific research ability, team assistance and other aspects of exploration, in order to do their utmost.
Analyzing Ukrainian Media Texts by Means of Support Vector Machines: Aspects of Language and Copyright

The paper focuses on the possibilities of studying Ukrainian online media publications using the support vector machine. Through a database including about 95,000 texts of three Ukrainian online media («Censor.NET», «Ukrainska Pravda», «Zakarpattya Online») within the period from September 2021 to August 2022 there were traced possibilities for identifying the authorship of a media text (belonging to certain media). It was established that the accuracy of determining whether a text belongs to online media (regional, national) using SVM is significantly influenced by the presence in published texts keywords included in the names of the media: if these words remain in the dataset, the accuracy of identification of the media through the text fragment is 0.979, if these words are removed, the accuracy drops to 0.930. This means that the high frequency of use of certain words (primarily related to the name of the publication, which is typical for the texts of Ukrainian online media) significantly affects the results of the experiment. A more detailed analysis of language elements employing SVM makes it possible to conclude that the identification of the media occurs primarily on the basis of linguistic units of the lexical level. Therefore, for the analysis of texts written in the Ukrainian language, which has an inflectional system, the stemming procedure for variable words is essential. The study also considered the legal aspects of authorship determination using the proposed machine learning method and highlighted new legal challenges when authorship determination is associated with automated journalism, that is, the creation of texts using artificial intelligence.
Waste Classification Using Random Forest Classifier with DenseNet201 Deep Features

The successful management of solid waste, which has become a major issue in urban life, reduces environmental pollution. To address this issue, numerous studies using various deep learning models have been conducted. This study focuses on traditional convolutional neural networks and transformer-based deep learning architectures and provides a detailed examination summary. In this context, many experimental studies dealing with the PoolFormer and ResNet transformers and the DenseNet201 and ResNet50 pre-trained CNN models were run on the TrashNet dataset containing waste images. The Random Forest (RF) classifier was trained on different deep features extracted from 80% of this dataset, and the remaining 20% was reserved as the test dataset. Test dataset was used to validate the performance of the RF. According to the results, RF with DenseNet201 deep features presents 96.4% classification accuracy. Furthermore, the RF-based hybrid model fed with the trained DenseNet201 outperformed other studies using the same dataset in the literature in terms of classification accuracy. The hybrid method employed used in this study may serve as a model for future research.