Reviewerly/exHarmony huggingface dataset:
SeamFormer: High Precision Text Line Segmentation for Handwritten Documents

Historical manuscripts often contain dense unstructured text lines. The large diversity in sizes, scripts and appearance makes precise text line segmentation extremely challenging. Existing line segmentation approaches often associate diacritic elements incorrectly to text lines and also address above mentioned challenges inadequately. To tackle these issues, we introduce SeamFormer, a novel approach for high precision text line segmentation in handwritten manuscripts. In the first stage of our approach, a multi-task Transformer deep network outputs coarse line identifiers which we term ‘scribbles’ and the binarized manuscript image. In the second stage, a scribble-conditioned seam generation procedure utilizes outputs from first stage and feature maps derived from manuscript image to generate tight-fitting line segmentation polygons. In the process, we incorporate a novel diacritic feature map which enables improved diacritic and text line associations. Via experiments and evaluations on new and existing challenging palm leaf manuscript datasets, we show that SeamFormer outperforms competing approaches and generates precise text line segmentations.
OCR Language Models with Custom Vocabularies

Language models are useful adjuncts to optical models for producing accurate optical character recognition (OCR) results. One factor which limits the power of language models in this context is the existence of many specialized domains with language statistics very different from those implied by a general language model - think of checks, medical prescriptions, and many other specialized document classes. This paper introduces an algorithm for efficiently generating and attaching a domain specific word based language model at run time to a general language model in an OCR system. In order to best use this model the paper also introduces a modified CTC beam search decoder which effectively allows hypotheses to remain in contention based on possible future completion of vocabulary words. The result is a substantial reduction in word error rate in recognizing material from specialized domains.
A Unified Architecture for Urdu Printed and Handwritten Text Recognition

Urdu text recognition (handwritten or printed) remains a challenging task due to its diverse writing styles and fonts. State-of-the-art Transformer-based OCR systems are computationally expensive because they rely on computationally expensive pretraining over text images. To address this challenge, we propose a robust architecture that utilizes a custom CNN block with a Transformer encoder for image understanding and a pre-trained Transformer decoder on Urdu language modeling. The presented model generalized well even for scarce training data without the need for pre-training on synthetic text images. Experiments show that our proposed architecture outperforms the state-of-the-art methods for Urdu printed and handwritten text recognition on several publicly available datasets including UPTI, NUST-UHWR, and MMU-OCR-21. We also combined printed and handwriting datasets to train our architecture and propose a single unified model; capable of recognizing both printed and handwritten text for maximum variations of fonts and writing styles with state-of-the-art results.
Exploring Semantic Word Representations for Recognition-Free NLP on Handwritten Document Images

A semantic analysis of documents offers a wide range of practical application scenarios. Thereby, the combination of handwriting recognizer and textual NLP models constitutes an intuitive solution. However, due to the difficulty of recognizing handwriting and the error propagation problem, optimized architectures are required. Recognition-free approaches proved to be robust, but often produce poorer results compared to recognition-based methods. In our opinion, a major reason for this is that recognition-free approaches do not use largely pre-trained semantic word embeddings, which proves to be one of the most powerful method in the textual domain. To overcome this limitation, we explore and evaluate several semantic embeddings for word image representation. We are able to show that context-based embedding methods are well suited for static word representations and that they are more predictive at word image level compared to classical static embedding methods. Furthermore, our recognition-free approach with pre-trained semantic information outperforms recognition-free as well as recognition-based approaches from the literature on several Named Entity Recognition benchmark datasets.
A System for Processing and Recognition of Greek Byzantine and Post-Byzantine Documents

Processing and recognition of Greek Byzantine and Post-Byzantine (old Greek) Documents has been proven to be a tedious task in the domain of Historical Document Image Processing. Several unique characteristics of these documents (existence of character ligatures, abbreviations, lack of clear word division, existence of symbols or punctuations in an arbitrary position) impose significant difficulties for current processing and recognition tools. In this work, we introduce a system for processing and recognition of old Greek documents and give details about all the components that comprise it. These include an image pre-processing, a text line segmentation and a recognition module. In order to test the proposed system, we introduce and provide publicly a new dataset of old Greek Documents that includes text line images and the corresponding transcription. Using this dataset, we evaluate the embedded recognition engine of the proposed system which is the open-source Calamari-OCR engine employing a variety of configurations. The best result corresponded to a character error rate less than 1.5% which is acceptable and promising. Finally, we also achieved promising results when comparing the embedded OCR engine with other recognition methods already proposed for the recognition of old Greek Documents.
Incremental Teacher Model with Mixed Augmentations and Scheduled Pseudo-label Loss for Handwritten Text Recognition

We propose a training framework for deep neural network-based handwritten text recognizers using both labeled and unlabeled data. The proposed framework is a semi-supervised learning (SSL) framework based on Mixed Augmentations and Scheduled Pseudo-Label loss. Mixed Augmentations provide weakly and strongly transformed variants from each original sample so that the pseudo-label loss is computed between these two variants. The Scheduled Pseudo-Label loss is used to gradually include the pseudo-label loss into the optimizer to avoid the negative effect of incorrect pseudo labels. First, a student model is pre-trained by labeled samples and used to initiate a teacher model. Subsequently, the teacher model predicts a pseudo label from every weakly transformed variant. On the other hand, the student model is trained using the Scheduled Pseudo-Label loss. Next, the teacher model is incrementally updated using the student model. Finally, it is used to evaluate. We term the framework Incremental Teacher Model. The proposed framework was applied to four architectures of distinct handwriting recognizers. For almost every architecture, the recognizer trained by our method outperforms those trained by well-known SSL methods, namely Mean Teacher, Pseudo-Labeling, and FixMatch, evaluated using different ratios of labeled training samples on the IAM handwriting database.
SegCTC: Offline Handwritten Chinese Text Recognition via Better Fusion Between Explicit and Implicit Segmentation

Handwritten Chinese text recognition (HCTR) is still a challenging and unsolved problem. The existing recognition methods are mainly categorized into two: explicit vs implicit segmentation-based methods. Explicit segmentation recognition methods use explicit character location information to train the recognizers. However, the widely used weakly supervised training strategy based on pseudo-label makes it difficult to get effective supervised training for difficult character samples. In contrast, the implicit segmentation recognition method use all transcript annotations for supervised training, but it is prone to misalignment problem due to the lack of explicit supervised information of character positions. To take advantage of the complementary nature of explicit and implicit segmentation approaches, we propose a new method, SegCTC, which better integrates these two approaches into a unified to be a more powerful recognizer. Specifically, we designed a hybrid Segmentation-based and Segmentation-free Feature Fusion Module (S $$^2$$ FFM) to better fuse the features of both explicit and implicit segmentation-based recognition branches. Moreover, a co-transcription strategy is also proposed to better combine the predictions from different branches. Experiments on four widely used benchmarks including CASIA-HWDB, ICDAR2013, SCUT-HCCDoc and MTHv2 show that our method achieves state-of-the-art performance for the HCTR task under different scenarios.
Shared-Operation Hypercomplex Networks for Handwritten Text Recognition

Parameterized hypercomplex layers have recently emerged as very useful alternatives of standard neural network layers. They allow for the construction of extremely lightweight architectures, with little to no sacrifice of accuracy. We propose networks of Shared-Operation Parameterized Hypercomplex layers, where the operation parameterization is co-learned by all layers in tandem. In this manner, we mitigate the computational burden of operation parameterization, which grows cubically with respect to the hypercomplex dimension. We attain good word and character error rate at only a small fraction of the memory footprint of non-hypercomplex models as well as previous non-shared operation hypercomplex ones (up to $$-96.8\%$$ size reduction).
Linguistic Knowledge Within Handwritten Text Recognition Models: A Real-World Case Study

State-of-the-art handwritten text recognition models make frequent use of deep neural networks, with recurrent and connectionist temporal classification layers, which perform recognition over sequences of characters. This architecture may lead to the model learning statistical linguistic features of the training corpus, over and above graphic features. This in turn could lead to degraded performance if the evaluation dataset language differs from the training corpus language. We present a fundamental study aiming to understand the inner workings of OCR models and further our understanding of the use of RNNs as decoders. We examine a real-world example of two graphically similar medieval documents but in different languages: rabbinical Hebrew and Judeo-Arabic. We analyze, computationally and linguistically, the cross-language performance of the models over these documents, so as to gain some insight into the implicit language knowledge the models may have acquired. We find that the implicit language model impacts the final word error by around 10%. A combined qualitative and quantitative analysis allow us to isolate manifest linguistic hallucinations. However, we show that leveraging a pretrained (Hebrew, in our case) model allows one to boost the OCR accuracy for a resource-scarce language (such as Judeo-Arabic). All our data, code, and models are openly available at https://github.com/anutkk/ilmja .
Decoupled Learning for Long-Tailed Oracle Character Recognition

Oracle character recognition has recently made significant progress with the success of deep neural networks (DNNs), but it is far from being solved. Most works do not consider the long-tailed distribution issue in oracle character recognition, resulting in a biased DNN towards head classes. To overcome this issue, we propose a two-stage decoupled learning method to train an unbiased DNN model for long-tailed oracle character recognition. In the first stage, we optimize the DNN under instance-balanced sampling, obtaining a robust backbone but biased classifier. In the second stage, we propose two strategies to refine the classifier under class-balanced sampling. Specifically, we add a learnable weight scaling module which can adjust the classifier to respect tail classes; meanwhile, we integrate the KL-divergence loss to maintain attention to head classes through knowledge distillation from the first stage. Coupling these two designs enables us to train an unbiased DNN model in oracle character recognition. Our proposed method achieves new state-of-the-art performance on three benchmark datasets, including OBC306, Oracle-AYNU and Oracle-20K.
Sampling and Ranking for Digital Ink Generation on a Tight Computational Budget

Digital ink (online handwriting) generation has a number of potential applications for creating user-visible content, such as handwriting autocompletion, spelling correction, and beautification. Writing is personal and usually the processing is done on-device. Ink generative models thus need to produce high quality content quickly, in a resource constrained environment. In this work, we study ways to maximize the quality of the output of a trained digital ink generative model, while staying within an inference time budget. We use and compare the effect of multiple sampling and ranking techniques, in the first ablation study of its kind in the digital ink domain. We confirm our findings on multiple datasets - writing in English and Vietnamese, as well as mathematical formulas - using two model types and two common ink data representations. In all combinations, we report a meaningful improvement in the recognizability of the synthetic inks, in some cases more than halving the character error rate metric, and describe a way to select the optimal combination of sampling and ranking techniques for any given computational budget.
Public Data Auditing Scheme Using RSA and Blockchain for Cloud Storage

With the widespread cloud storage devices, most users store their data over cloud storage. A dishonest user might raise a false claim about the integrity of their data to get compensation. On the other hand, a corrupt Cloud Service Provider (CSP) might hide the fact from the user that their data has been compromised. So, ensuring the integrity of the data is of utmost importance in such cases. The integrity of user data is ensured through Third Party Auditor (TPA), who is supposed to be trusted and possess the computation resources to perform the auditing. Although this TPA reduces the burden of verifying the data, there’s still a risk of TPA, who may not be honest. The TPA might collude with the CSP or not perform its task correctly. This paper proposes a Public Data Auditing Scheme using RSA and Blockchain (PDASRSAB) for cloud storage to bring security and transparency into the auditing process, which increases confidence in cloud storage, along with the experimental analysis for the proposed scheme.
Analysis of Different Sampling Techniques for Software Fault Prediction

The process of predicting whether or not a software module is faulty based on specific metrics is known as software fault prediction. Software faults predicted in prior stages help in the management of resources and time required during software testing and maintenance. Over the years, various supervised machine learning-based techniques for fault prediction have been suggested. The models are trained using a labeled dataset that consists of multiple independent variables like lines of codes, the complexity of the software, the size of the software, etc., and a dependent binary variable that is either true or false. Recent research in software fault prediction focuses on data quality. In this paper, we have mainly focused on the class imbalance problem. In imbalanced data, one of the class labels has a higher number of observations, while the other class label has a lower number of observations. Different over-sampling and under-sampling techniques are used to tackle the class imbalance problem. In this paper, random under-sampling, random over-sampling, and SMOTE are applied to six PROMISE datasets. A decision tree classifier is used to create the model. The efficiency of different sampling techniques is compared using the ROC-AUC score and F1-score.
An Exploration of Digital Image Forensic Techniques: A Brief Survey

Digital image forensics deals with assessing whether an image is genuine or not. As an image may undergo several manipulations done to either improve its quality or to intentionally change its meaning it is very difficult to conclude if an image is forged or is genuine. In this paper, three important aspects of image forgery detection are explored. An in-depth discussion of image forgery detection, a technique which is based on assessing image features categorized under active and passive methods is present. Analysing both image features and device features to know the image capturing device while checking for forgery is explained in detail. Provenance analysis which is the entire derivation of image manipulation history is also expressed. Discussion regarding research directions in the domain of image forensics is mentioned in the paper.
A New BERT-Inspired Knowledge Distillation Approach Toward Compressed AI Models for Edge Devices

In the natural language processing (NLP) domain [7], the pre-trained models for tackling a variety of language understanding tasks (text-to-speech, speech-to-text, text summarization, etc.) are typically large in size. Because of their larger size, accommodating and running them in resource-constrained IoT edge devices are becoming a tough assignment. Large models generally consume a lot of computing and storage resources, waste precious power energy, and dissipate more heat into our fragile environment. Therefore, the concept of knowledge distillation has picked up fast as a viable and venerable method for model optimization. In this paper, we showcase a robust distillation approach for producing small-sized pre-trained models that can run on edge devices comfortably. For this unique distillation process, we have considered several pre-trained models and used BERT as a base model, and presented the results obtained through a few practical experiments. Distillation models are fine-tuned with good performance for a wide range of applications. Here, we have considered all 24 types of BERT models with different layers and included the hidden layers for distillation. This has given a performance improvement over the base model performance by employing fine-tuning for all types of models at various stages. We have reduced the model size by a margin of greater than 40%, while simultaneously retaining its original model language understanding capabilities and speed. The proposed model is smaller, faster, and cheaper to train and run.
A Conceptual Model for Click Fraud Detection and Prevention in Online Advertising Using Blockchain

Click frauds in online advertising are increasing intentionally or unintentionally which is unnecessarily raising the cost to the advertisers. As per our observation from the literature, the existing machine learning and statistical analysis-based solutions offer insufficient accuracy in click fraud detection. In addition, a user’s privacy, click data immutability and trust between the advertiser and publisher need to be achieved. To address these issues, we propose a conceptual model for click fraud detection and prevention using blockchain. It verifies ad clicks by mapping the user’s mail id with their phone number and corresponding national identity value (NIV) which is unique per user in order to distinguish genuine and fraud clicks. Based on this mapping, it filters the genuine and fraud clicks by considering one click per user, which are recorded as transactions over the blockchain. The privacy of the users’ information is preserved by exchanging hashed information using the SHA256 hash function. For the users, a reward mechanism is introduced to encourage the participation of genuine users. In addition, the genuine rate per advertisement is derived to represent the reputation of the publisher and website owner. The functionality of the proposed model is tested using the testbed at NIT Goa.
Vision-Based Weather Condition Recognition for Driver Assistance

Many camera and sensor-based technologies have been launched in recent years to aid drivers in assuring their safety in a variety of driving situations. The decreased scene visibility in unfavorable weather conditions (fog/rain) is one of the issues that drivers experience while driving. The contrast and color of acquired photographs are poor in hazy weather circumstances, making it difficult to discern the image. A technique to separate images or categorize them from a dataset classification into the following categories, cloudy, sunny, foggy, rainy, and snowy, is done. To fulfill the aim, the Gray-Level Co-occurrence Matrix technique was used to collect weather-related attributes, and the necessary data processing was done. The system achieves a weather forecast accuracy of roughly 89.86%.
A Novel Fake Job Posting Detection: An Empirical Study and Performance Evaluation Using ML and Ensemble Techniques

Recently, everything can be accomplished online, including education, shopping, banking, etc. This technological advancement makes it easy for fraudsters to scam people online and acquire easy money. Numerous cyber crimes worldwide exist, including identity theft and fake job postings. Nowadays, many companies post job openings online, making recruitment simple. Consequently, fraudsters also post job openings online to obtain money and personal information from job seekers. In the proposed work, we aimed to decrease the frequency of such scams by using ensemble techniques such as AdaBoost, Gradient Boost, Stacking classifier, XgBoost, Bagging, and Random Forest to identify fake job postings from genuine ones. This paper proposes various featurization techniques such as Response coding with Laplace smoothing, Average Word2vec, and term frequency-inverse document frequency weighted Word2vec. We compared the performance of ensemble techniques with machine learning (ML) algorithms on publicly available EMSCAD dataset using accuracy and F1-score. Bagging classifier outperformed all the models with an accuracy of 98.85% and an F1-score of 0.88 on imbalanced dataset. On balanced dataset, XgBoost achieved 97.89% accuracy and 0.98 F1-score. From the experimental results, it is observed that a combination of ensemble and featurization techniques using Laplace smoothed Response coding and BoW stood superior to most of the state-of-the-art works on fake job posting detection.
Motion Posture Recognition Method Based on RBF Neural Network Optimization

In order to improve the recognition rate of moving targets by radial basis function (RBF) neural network, a posture recognition algorithm based on optimized RBF is proposed. Firstly, the basic technology of human behavior recognition acquires in-depth analysis. Then, a dynamic adjustment of the important parameters of Levenberg-Marquardt with the number of iterative steps is put forward, to construct an adaptive RBF structure. Combined with the excellent global search ability of GA, it is also improved accordingly. Finally, the normalized motion history image (MHI) is used for image sequence representation for data processing. The simulation results show that our method reduces the over fitting degree of RBF, improves the generalization ability and calibration accuracy of LM-BP network, and can effectively recognize human posture categories.