Reviewerly/exHarmony huggingface dataset:
ViSA: Visual and Semantic Alignment for Robust Scene Text Recognition

Unsupervised domain adaptation is studied to meet the challenge of scene text recognition in diverse scenarios. Existing methods try to align source and target domain at the image or character level. However, these approaches are somewhat coarse-grained as they involve irrelevant information or ignore category attributes. To address the above issues, we propose a novel Visual and Semantic Alignment (ViSA) method to reduce the domain shifts in the high-frequency domain and category space. Specifically, the high-frequency domain alignment extracts the high-frequency components of global visual features, which allows the domain classifier to focus on the text-relevant features. Furthermore, the category space alignment is introduced to align character features at the category level. In the category space alignment, cross-domain contrastive learning and prototype-consistency matching are adopted to minimize the distance between domains. ViSA is flexible to be plugged into various existing recognizers. In addition, ViSA can be conducted in training and dropped in evaluation, which means no impact on inference speed. Adequate experiments verify the superiority of each module of ViSA, and our method achieves state-of-the-art results on several benchmarks, such as SVT, IC13 and IC15.
An Iterative Graph Learning Convolution Network for Key Information Extraction Based on the Document Inductive Bias

Recently, there has been growing interest in automating the extraction of key information from document images. Previous methods mainly focus on modelling the complex interactions between multimodal features(text, vision and layout) of documents to comprehend their content. However, only considering these interactions may not work well when dealing with unseen document templates. To address this issue, in this paper, we propose a novel approach that incorporates the concept of document inductive bias into the graph convolution framework. Our approach recognizes that the content of a text segment in a document is often determined by the context provided by its surrounding segments and utilizes an adjacency matrix hybrid strategy to integrate this bias into the model. As a result, the model is able to better understand the relationships between text segments even when faced with unseen templates. Besides, we employ an iterative method to perform graph convolution operation, making full use of the textual, visual, and spatial information contained within documents. Extensive experimental results on two publicly available datasets demonstrate the effectivness of our methods.
MUGS: A Multiple Granularity Semi-supervised Method for Text Recognition

Most text recognition methods are trained on large amounts of labeled data. Although text images are easily accessible, labeling them is costly. Thus how to utilize the unlabeled data is worth studying. In this paper, we propose a MUltiple Granularity Semi-supervised (MUGS) method using both labeled and unlabeled data for text recognition. Inspired by the hierarchical structure (sentence-word-character) of text, we apply semi-supervised learning at both word-level and character-level. Specifically, a Dynamic Aggregated Self-training (DAS) framework is introduced to generate pseudo-labels from unlabeled data at word-level. To ensure the quality and stability of the pseudo-labeling procedure, the pseudo-labels are aggregated from one dynamic model queue which keeps updating in the whole semi-supervised training process. At the character-level, a novel module named WTC (Word To Character) that can convert sequential features to character representations is invented. Next, contrastive learning is applied to these character representations for better fine-grained visual modeling. The characters from various images that share the same classes are pulled together and the ones in different classes are set far apart in the representation space. With the combination of supervisions in different granularities, more information can be exploited from the unlabeled data. The effectiveness and robustness of the model are enhanced by a large margin. Comprehensive experiments on several public benchmarks validate our method and competitive performance is achieved with much fewer labeled data.
DAMGCN: Entity Linking in Visually Rich Documents with Dependency-Aware Multimodal Graph Convolutional Network

Linking semantically related entities in visually rich documents (VRDs) is important for understanding the document structure and contents. Existing approaches usually predict the links using layout and textual semantic information simultaneously. They may generate false links or omit real links because of the insufficient utilization of the dependency among adjacent value entities. In this paper, we propose a dependence-aware multimodal graph convolutional network (DAMGCN) for improving the accuracy of entity linking. In DAMGCN, a graph construction approach based on layout-based k-NN is proposed to reduce invalid entity links. Then, the links are determined from a proposed local transformer block (LTB), which utilizes multiple layout information (angle, distance, intersection over union) to identify the adjacent entities with potential dependency with a given key entity. Experiments on the public FUNSD and XFUND datasets show that our DAMGCN achieves competitive results, i.e., F1 scores of 0.8063 and 0.7303 in entity linking tasks, respectively, while the model size is much smaller than the state-of-the-art models.
DTDT: Highly Accurate Dense Text Line Detection in Historical Documents via Dynamic Transformer

Text detection in historical documents is challenging owing to the dense distribution of texts with diverse scales and complex layouts, resulting in low detection accuracy under high Intersection over Union (IoU) conditions. Historical document digitization requires highly accurate detection results to preserve the contents completely. In this paper, we present an end-to-end text detection framework, namely Dynamic Text Detection Transformer (DTDT), for dense text detection in historical documents under high accuracy requirements. We introduce a deformable convolution-based dynamic encoder to strengthen the text representation ability at different scales. In addition, the parallel dynamic attention heads are designed to facilitate better interaction between the box and mask branches to obtain accurate text detection results. Experiments on the MTHv2 and ICDAR 2019 HDRC-CHINESE (short for “IC19 HDRC”) datasets show that the proposed DTDT method achieves state-of-the-art performance. Furthermore, our DTDT achieves competitive results in layout analysis on SCUT-CAB benchmark, demonstrating its excellent generalization capabilities.
EEBO-Verse: Sifting for Poetry in Large Early Modern Corpora Using Visual Features

One branch of important digital humanities research focuses on the study of poetry and verse, leveraging large corpora to reveal patterns and trends. However, this work is limited by currently available poetry corpora, which are restricted to few languages and consist mainly of works by well-known classic poets. In this paper, we develop a new large-scale poetry collection, EEBO-verse (Code and dataset is available on https://github.com/taineleau/ebbo-verse ), by automatically identifying the poems in a large Early Modern books collection — English Early-modern Printed Books Online (EEBO). Instead of training text-based classifiers to sub-select the 3.5% of EEBO that actually consists of poetry, we develop an image-based classifier that can operate directly on page scans, removing the need to perform OCR – which, in this domain, is often unreliable. We leverage large visual document encoders (DiT and BEiT), which are pretrained on general domain document images, by fine-tuning them on an in-domain annotated subset of EEBO. In experiments, we find that an appropriately trained image-only classifier performs as well or better than text-based poetry classifiers on human transcribed text, and far surpasses the performance of text-based classifiers on OCR output.
Re-Thinking Text Clustering for Images with Text

Text-VQA refers to the set of problems that reason about the text present in an image to answer specific questions regarding the image content. Previous works in text-VQA have largely followed the common strategy of feeding various input modalities (OCR, Objects, Question) to an attention-based learning framework. Such approaches treat the OCR tokens as independent entities and ignore the fact that these tokens often come correlated in an image representing a larger ‘meaningful’ entity. The ‘meaningful’ entity potentially represented by a group of OCR tokens could be primarily discerned by the layout of the text in the image along with the broader context it appears. In the proposed work, we aim to cluster the OCR tokens using a novel spatially-aware and knowledge-enabled clustering technique that uses an external knowledge graph to improve the answer prediction accuracy of the text-VQA problem. Our proposed algorithm is generic enough to be applied to any multi-modal transformer architecture used for text-VQA training. We showcase the objective and subjective effectiveness of the proposed approach by improving the performance of the M4C model on the Text-VQA datasets.
FCN-Boosted Historical Map Segmentation with Little Training Data

This paper deals with automatic image segmentation in poorly resourced areas. We concentrate on map content segmentation in historical maps as an example of such a domain. In such cases, conventional computer vision (CV) approaches fail in unexpected unique regions such as map content area exceeding the map frame, while deep learning methods lack boundary localization accuracy. Therefore, we propose an efficient approach that combines conventional CV techniques with deep learning and practically eliminates their drawbacks. To do so, we redefine the learning objective of a simple fully convolutional network to make the training easier and the model more robust even with few training samples. The presented method provides excellent results compared to more sophisticated but solely deep learning or traditional computer vision techniques as shown in “MapSeg” segmentation competition, where all other approaches were significantly outperformed. We further propose two additional approaches that improve the original method and set a new state-of-the-art result on the MapSeg dataset. The methods are further tested on an extended version of the Map Border dataset to show their robustness.
An Encoder-Decoder Method with Position-Aware for Printed Mathematical Expression Recognition

Printed mathematical expression recognition is to transform printed mathematical formula image into LaTeX sequence. Recently, many methods based on deep learning have been proposed to solve this task. However, the positional relationship between mathematical symbols is often ignored or represented insufficient, leading to the loss of structural features of mathematical formulas. To overcome this challenge, we propose a position-aware encoder-decoder model for printed mathematical expression recognition. We design a two-dimensional position encoding algorithm based on sin/cos function to capture positional relationship between mathematical symbols. Meanwhile, we adopt a more advanced image feature extraction network. In decoder component, we use Bi-GRU as the translator, and add attention mechanism to make decoder focus on the important local information. We conduct experiments on the public dataset IM2LaTeX-100K, and the results show that our proposed approach is more excellent than the majority of advanced methods.
Spatial-Temporal Graph Neural Network Based Anomaly Detection

Multivariate time series anomaly detection is an important task in the monitoring system. In practical applications, an efficient and accurate anomaly detection method is particularly important. Recently, the method of anomaly detection based on prediction has made significant progress, but there are still limitations. This paper proposes a paradigm for multivariate time series anomaly identification based on pre-training. The strategy of pre-training is to use Transformer’s encoder to learn the dense vector representation of multiple time series through autoregressive task, so as to enhance the predictability of time series. In the prediction module, we learn the feature dependence of time series through graph attention network, and design an interactive tree structure that takes full advantage of the unique characteristics of time series to capture its time dependence. In addition, our method is well interpretable and allows users to infer the root cause of exceptions. We have proved the effectiveness of our model through extensive experiments. It is significantly superior to the most advanced model in three real data sets.
Improving Handwritten OCR with Training Samples Generated by Glyph Conditional Denoising Diffusion Probabilistic Model

Constructing a highly accurate handwritten OCR system requires large amounts of representative training data, which is both time-consuming and expensive to collect. To mitigate the issue, we propose a denoising diffusion probabilistic model (DDPM) to generate training samples. This model conditions on a printed glyph image and creates mappings between printed characters and handwritten images, thus enabling the generation of photo-realistic handwritten samples with diverse styles and unseen text contents. However, the text contents in synthetic images are not always consistent with the glyph conditional images, leading to unreliable labels of synthetic samples. To address this issue, we further propose a progressive data filtering strategy to add those samples with a high confidence of correctness to the training set. Experimental results on IAM benchmark task show that OCR model trained with augmented DDPM-synthesized training samples can achieve about $$45\%$$ relative word error rate reduction compared with the one trained on real data only.
Adversarial Attacks on Convolutional Siamese Signature Verification Networks

A handwritten signature serves as an important biometric modality to identify individuals. The state-of-the-art methods for signature verification employ deep learning networks to perform the classification task. However, deep neural networks can be fooled by adversarial attacks that introduce small imperceptible perturbations to the input images. In this paper, we explore the vulnerability of signature verification systems against adversarial attacks. The state-of-the-art attacks developed by the machine learning community to fool image classifiers are unsuitable for attacking document classifiers as they are applied to the background of signature images making them quite perceptible. To overcome this challenge, we design an attack based on dictionary learning with the goal to perturb the foreground (strokes) of the signature image. The proposed method is evaluated in terms of attack success rate and imperceptibility. The experimental results on the benchmark CEDAR dataset using Siamese Deep Signet Model highlight the efficacy of the proposed approach as compared to other methods by achieving 95% and 98% attack success rates with our proposed approach.
Outlook and Future Research

Modern healthcare systems are investing heavily in advanced sensing and imaging technology to increase the ability of capturing extensive medical data from patients, which is conducive to improving information visibility and coping with disease complexity. On the other hand, simulation modeling of the heart provides a fundamental infrastructure for the visualization and quantitative understanding of spatiotemporal cardiac dynamic processes. Precision cardiology calls upon the development of advanced computation models that are able to effectively promote and leverage the synergy between physics-based cardiac simulation and data-driven machine learning models to exploit the medical data from different modalities, further constructing a digital twin of the heart. Such computational models will enable the integrative analysis of a patient’s medical data, low-cost and low-risk quantitative evaluation of alternative therapeutic designs, and generating diagnostic decisions that are free from subjective assessment.
A Shallow Graph Neural Network with Innovative Node Updating for Online Handwritten Stroke Classification

Stroke classification is important to the layout analysis of online handwritten documents. Due to the diversity of writing styles and the complexity of layout structure, stroke classification is challenging. Graph neural networks (GNNs) is one of the most effective frameworks for stroke classification. However, GNNs has the problem of node over-compression caused by the deep structure of GNNs, which will lead to loss of node information and hence may deteriorate the performance of stroke classification. In this paper, we propose a shallow graph neural network model that is capable of retaining long-term receptive field by constructing a more reasonable graph through edge classification before the node classification step. Moreover, a novel node learning method is used to integrate edge features into nodes, where edge features not only participate in the calculation of node attention weight as in previous GNN based methods, but also participate in the final node integration. Experiments on the IAMonDo dataset show that our proposed method achieves an accuracy of 97.71% that is superior to existing state-of-the-art methods, demonstrating the effectiveness of the proposed method.
Persistence and Domiciled Mutual Fund Managers Performance: Evidence for South Africa

This paper presents an overview of the South African domiciled mutual fund industry and study mutual fund performance using a survivorship bias-free sample of 2,843 share class of funds. Carhart (1997) 4-factor asset-pricing model is used to measure performance, and an additional two factors from the recent Fama and French (2015) framework have been considered. This study is able to provide further evidence on the performance of open-end mutual fund managers in a noticeable emerging market, South Africa. This paper investigates whether mutual fund managers' inability in developed markets to beat the market, which has been widely proved, also holds in less developed markets. Furthermore, I examine if South African fund managers exhibit persistence in performance. The negative net alpha for South African domiciled mutual fund managers shows that they under-perform their respective market benchmark and are not able to add value. At last, consistent with results in developed markets, for all portfolios, with different magnitude, the paper reveals that there exists a persistence in performance; however, the effect attributes to "icy-hands" rather than "hot-hands."
Documenting the API

The process of documenting an API should be given a well-deserved place in the API development process. Ultimately, we develop APIs to be consumed by different systems, whether inside our organization or not. Part of this process is related to interacting with an API in a simple manner.
Enhancing Security and Performance of Software Defect Prediction Models: A Literature Review

There have recently been many advances in software defect prediction (SDP). Just-in-time defect prediction (JIT), heterogeneous defect prediction (HDP), and cross-project defect prediction (CPDP) are now used for better performance. Some issues remain a hurdle in the proper implementation of the SDP models. Security, class imbalance, heterogeneity, and redundant and irrelevant features are some of the significant issues. This research study provides a systematic literature review (SLR) on various approaches to solving the above-mentioned issues. Research papers from various well-reputed online libraries such as IEEE, Springer Link, and ScienceDirect have been reviewed critically in a systematic manner. Three well-defined research questions on the issues of security, class imbalance, and feature selection have been addressed. Recent papers relevant to the topics were reviewed systematically to answer the questions. The effectiveness of privacy-focused SDP model techniques has been examined in this paper. In this study, various techniques that can help developers manage the issue of class imbalance have been mentioned that can improve the predictive performance of the SDP models. Effective feature selection techniques are reviewed, providing a solid basis for future extension and application of these approaches on commercial datasets. The developers introduced promising frameworks, such as JIT with LIME, FILTER with SVM, LTCA, CDAA, FRLGC, and FTLKD. The datasets used for the research and application were from public repositories like NASA, PROMISE, and Re-Link.
Routing

We talked briefly about routes in Chapter 4 . The routing mechanism determines which action to execute when a request is made to a specific URL, based on templates we define or on conventions already established in the platform.
A Critical Analysis of Learning Approaches for Image Annotation Based on Semantic Correlation

Automatic Image Annotation (AIA) is a critical research topic and plays an important role in image retrieval. Recently, AIA approaches are focused to label the relevant data to the images. The annotations give the labels to images. Most of the image annotation techniques initially extract the features from the testing and training images, and then the annotation framework provides the accurate annotation based on the training data. This research provides a comprehensive overview of the most current stage in the development of AIA methodologies by combining 32 literature published over the last three decades. AIA approaches are divided into five groups: (1) Kernel Logistic Regression (KLR), (2) Triple relational Graph (TG), (3) Semantically Regularized CNN-RNN (S-CNN-RNN), (4) Label Correlation guided Deep Multiple view (DMLC), and (5) Multiple Modal Semantic Hash Learning (MMSHL). Acquiring meaningful low-level visual characteristics and generating the task of high-level semantic correlation is difficult for (AIA). The key finding through analysis is that semantic correlation gap of various image representation features for large-scale datasets is still a challenging issue. Exhaustive large-scale image annotation can be done effectively by efficiently indentifying good semantic correlation among image-image, image-label and label-label features. Then the performance evaluation of mentioned learning methods is based on precision and recall for popular datasets like MIR flicker, DeviantArt images, NUS-WIDE, MSCOCO and Coral5k. The performance of the mentioned methodology is compared based on idea, framework, complexity and accuracy in terms of different performance metrics to show differentiation in performance efficiency.
Introducing Web API

This chapter introduces you to the world of APIs, including what kinds of APIs are out there, what their purposes are, and how you can categorize them. By the end of the chapter, you will have the correct vocabulary to use when dealing with different types of APIs.
Versioning the API

Versioning an API refers to the practice of assigning a unique identifier or version number to each iteration of an API. This is done to enable developers to make changes to the API without disrupting the functionality of applications that depend on it.
Web API: Building Blocks

Convention over configuration is a design philosophy that seeks to apply defaults implied by code structure rather than requiring explicit code.