nils-herrmann/clean_scirepeval huggingface dataset:
A Comprehensive Representation Model for Handwriting Dedicated to Word Spotting

In this paper, we propose an original representation model for handwriting document images. Most state-of-the-art handwriting representation models only use separately textural properties, selective dominant features (such as stroke orientation or gradient orientation) or structural properties. To avoid the drawbacks of using the properties from a single aspect, we design a comprehensive model that contains both morphological and topological information of handwriting. After interest points (the starting/ending points, branch points and high-curved points) are selected, an adapted version of Shape Context (SC) descriptor built on the interest points is employed to describe the contour of the text. In order to model the structural characteristics of the handwritten text, a graph is constructed based on the interest points and the skeleton of the text. With the graph, loops and specific strokes in the handwriting are detected and analyzed. Based on this model, a coarse-to-fine approach for word spotting application is introduced. Without segmenting texts into words, a group of regions of interest are selected by comparing textural features (orientation, projection profile, upper and lower border projection) using the DTW method. Afterwards, regions of interest and queries are represented by the proposed model. The final similarity measure is a weighted mixture of the SC cost, loop difference, stroke analysis and texture comparison with different weights. The validation of the model shows the significance of combining the various properties of the handwriting envisaged in its different aspects.
A Sparse Coding Based Approach for the Resolution Enhancement and Restoration of Printed and Handwritten Textual Images

Sparse coding has shown to be an effective technique in solving various reconstruction tasks such as denoising, in painting, and resolution enhancement of natural images. In this paper, we explore the use of this technique specifically to deal with low-resolution and degraded textual images. Firstly, we propose a sparse coding based resolution enhancement approach to recover a textual image with higher resolution than the input low-resolution one. It is based on the use of multiple coupled dictionaries which are learned from a clustered training low-resolution/high-resolution patch-pair database. A reconstruction scheme is then suggested in order to adaptively select the appropriate dictionaries that are useful for better recovering each local patch. This approach can be applied for the magnification of both printed and handwritten characters. Secondly, we propose to integrate the magnification in a restoration framework specifically to denoise and reconstruct at the same time degraded characters. The performances of these propositions are evaluated on various types of degraded printed and handwritten textual images where loss of details and background noise exist. Promising results are achieved when compared with results of other existing approaches.
Low-Complexity Approximate Convolutional Neural Networks

In this paper, we present an approach for minimizing the computational complexity of the trained convolutional neural networks (ConvNets). The idea is to approximate all elements of a given ConvNet and replace the original convolutional filters and parameters (pooling and bias coefficients; and activation function) with an efficient approximations capable of extreme reductions in computational complexity. Low-complexity convolution filters are obtained through a binary (zero and one) linear programming scheme based on the Frobenius norm over sets of dyadic rationals. The resulting matrices allow for multiplication-free computations requiring only addition and bit-shifting operations. Such low-complexity structures pave the way for low power, efficient hardware designs. We applied our approach on three use cases of different complexities: 1) a “light” but efficient ConvNet for face detection (with around 1000 parameters); 2) another one for hand-written digit classification (with more than 180 000 parameters); and 3) a significantly larger ConvNet: AlexNet with  $\approx 1.2$  million matrices. We evaluated the overall performance on the respective tasks for different levels of approximations. In all considered applications, very low-complexity approximations have been derived maintaining an almost equal classification performance.
A Novel Learning-Free Word Spotting Approach Based on Graph Representation

Effective information retrieval on handwritten document images has always been a challenging task. In this paper, we propose a novel handwritten word spotting approach based on graph representation. The presented model comprises both topological and morphological signatures of handwriting. Skeleton-based graphs with the Shape Context labelled vertexes are established for connected components. Each word image is represented as a sequence of graphs. In order to be robust to the handwriting variations, an exhaustive merging process based on DTW alignment result is introduced in the similarity measure between word images. With respect to the computation complexity, an approximate graph edit distance approach using bipartite matching is employed for graph matching. The experiments on the George Washington dataset and the marriage records from the Barcelona Cathedral dataset demonstrate that the proposed approach outperforms the state-of-the-art structural methods.
Multiple Learned Dictionaries Based Clustered Sparse Coding for the Super-Resolution of Single Text Image

This paper addresses the problem of generating a super-resolved version of a low-resolution textual image by using Sparse Coding (SC) which suggests that image patches can be sparsely represented from a suitable dictionary. In order to enhance the learning performance and improve the reconstruction ability, we propose in this paper a multiple learned dictionaries based clustered SC approach for single text image super resolution. For instance, a large High-Resolution/Low-Resolution (HR/LR) patch pair database is collected from a set of high quality character images and then partitioned into several clusters by performing an intelligent clustering algorithm. Two coupled HR/LR dictionaries are learned from each cluster. Based on SC principle, local patch of a LR image is represented from each LR dictionary generating multiple sparse representations of the same patch. The representation that minimizes the reconstruction error is retained and applied to generate a local HR patch from the corresponding HR dictionary. The performance of the proposed approach is evaluated and compared visually and quantitatively to other existing methods applied to text images. In addition, experimental results on character recognition illustrate that the proposed method outperforms the other methods, involved in this study, by providing better recognition rates.
Unsupervised online learning of visual focus of attention

In this paper, we propose a novel approach for estimating visual focus of attention in video streams. The method is based on an unsupervised algorithm that incrementally learns the different appearance clusters from low-level visual features extracted from face patches provided by a face tracker. The clusters learnt in that way can then be used to classify the different visual attention targets of a given person during a tracking run, without any prior knowledge on the environment and the configuration of the room or the visible persons. Experiments on public datasets containing almost two hours of annotated videos from meetings and video-conferencing show that the proposed algorithm produces state-of-the-art results and even outperforms a traditional supervised method that is based on head orientation estimation and that classifies visual focus of attention using Gaussian Mixture Models.
Digital architecture for real-time CNN-based face detection for video processing

In this paper, we propose a hardware computing architecture for face detection that classifies an image as a face or non-face. The computing architecture is first designed, modeled and tested in MATLAB Simulink using Xilinx block set and was later tested using a Virtex-6 FPGA ML605  The system uses learned filters which were previously extracted by training on a set of face and non-face patterns. The system is fully feature based and does not require any assumptions on specific image processing techniques. The proposed approach takes an input image as a whole and passes it through different modules that apply sub-algorithms based on image convolution and sub-sampling followed by a non-linear signal processor containing artificial neurons. The architecture takes the form of a deep convolutional neural network (CNN) which can classify if a search window inside a picture contains a human face or not.
A Coarse-to-Fine Word Spotting Approach for Historical Handwritten Documents Based on Graph Embedding and Graph Edit Distance

Effective information retrieval on handwritten document images has always been a challenging task, especially historical ones. In the paper, we propose a coarse-to-fine handwritten word spotting approach based on graph representation. The presented model comprises both the topological and morphological signatures of the handwriting. Skeleton-based graphs with the Shape Context labelled vertexes are established for connected components. Each word image is represented as a sequence of graphs. Aiming at developing a practical and efficient word spotting approach for large-scale historical handwritten documents, a fast and coarse comparison is first applied to prune the regions that are not similar to the query based on the graph embedding methodology. Afterwards, the query and regions of interest are compared by graph edit distance based on the Dynamic Time Warping alignment. The proposed approach is evaluated on a public dataset containing 50 pages of historical marriage license records. The results show that the proposed approach achieves a compromise between efficiency and accuracy.
Dysarthric speech recognition using a convolutive bottleneck network

In this paper, we investigate the recognition of speech produced by a person with an articulation disorder resulting from athetoid cerebral palsy. The articulation of the first spoken words tends to become unstable due to strain on speech muscles, and that causes a degradation of traditional speech recognition systems. Therefore, we propose a robust feature extraction method using a convolutive bottleneck network (CBN) instead of the well-known MFCC. The CBN stacks multiple various types of layers, such as a convolution layer, a subsampling layer, and a bottleneck layer, forming a deep network. Applying the CBN to feature extraction for dysarthric speech, we expect that the CBN will reduce the influence of the unstable speaking style caused by the athetoid symptoms. We confirmed its effectiveness through word-recognition experiments, where the CBN-based feature extraction method outperformed the conventional feature extraction method.
Learning a bag of features based nonlinear metric for facial similarity

This article presents a new method aiming at automatically learning a visual similarity between two images from a class model. This kind of problem is present in many research domains such as object tracking, image classification, signing identification, etc. We propose a new method for facial recognition with a system based on non-linear projection and metric learning. To achieve this objective, we feed a “Bag of Features” representation of the face images into a specific neural network that learns a mapping to a more compact and discriminant representation. This learning process aims at non-linearly projecting the facial features into a reduced space where two images belonging to the same category (i.e. a person) are “close” according to a given similarity metric and “distant” otherwise. The proposed method gives very promising results for face identification in adverse conditions like expression, illumination and facial pose variations. Experimental results give 97% correct recognition rate on the CMU PIE database containing 68 individuals, under vary variable pose and illumination conditions.
Resolution enhancement of textual images: a survey of single image-based methods

Super-resolution (SR) task has become an important research area due to the rapidly growing interest for high quality images in various computer vision and pattern recognition applications. This has led to the emergence of various SR approaches. According to the number of input images, two kinds of approaches could be distinguished: single or multi-input based approaches. Certainly, processing multiple inputs could lead to an interesting output, but this is not the case mainly for textual image processing. This study focuses on single image-based approaches. Most of the existing methods have been successfully applied on natural images. Nevertheless, their direct application on textual images is not enough efficient due to the specificities that distinguish these particular images from natural images. Therefore, SR approaches especially suited for textual images are proposed in the literature. Previous overviews of SR methods have been concentrated on natural images application with no real application on the textual ones. Thus, this study aims to tackle this lack by surveying methods that are mainly designed for enhancing low-resolution textual images. The authors further criticise these methods and discuss areas which promise improvements in such task. To the best of the authors’ knowledge, this survey is the first investigation in the literature.
Face detection in color images using wavelet packet analysis

We propose a novel scheme for automatic and fast detection of human faces in color images where the number, the location, the orientation and the size of the faces are unknown, under unconstrained scene conditions such as complex background and uncontrolled illumination. First, each frame is segmented using skin chrominance values, providing face area candidates. Then, shape analysis and wavelet packet decomposition are performed on the face area candidates in order to detect human faces. Each face area candidate is described by a subset of band filtered images containing wavelet coefficients. These coefficients characterize the face texture and a set of simple statistical data is extracted in order to form compact and meaningful feature vectors. Then, an efficient and reliable probabilistic metric derived from the Bhattacharrya distance is used in order to classify the face area candidate feature vectors into face or non-face areas.
Enhancing face recognition from video sequences using robust statistics

The aim of this work is to investigate a way of enhancing the performance of face recognition from video sequences by selecting only well-framed face images from those extracted from video sequences. It is known that noisy face images (e.g. not well-centered, non-frontal poses...) significantly reduce the performance of face recognition methods, and therefore, need to be filtered out during the training and the recognition. The proposed method is based on robust statistics, and more precisely, a recently proposed robust high-dimensional data analysis method, RobPCA. Experiments show that this filtering procedure improves the recognition rate by 10 to 20%.
Facial biometry by stimulating salient singularity masks

We present a novel approach for face recognition based on salient singularity descriptors. The automatic feature extraction is performed thanks to a salient point detector, and the singularity information selection is performed by a SOM region-based structuring. The spatial singularity distribution is preserved in order to activate specific neuron maps and the local salient signature stimuli reveals the individual identity. This proposed method appears to be particularly robust to facial expressions and facial poses, as demonstrated in various experiments on well-known databases.
Deep learning and recurrent connectionist-based approaches for Arabic text recognition in videos

This paper focuses on recognizing Arabic embedded text in videos. The proposed methods proceed without applying any prior pre-processing operations or character segmentation. Difficulties related to the video or text properties are faced using a learned robust representation of the input text image. This is performed using Convolutional Neural Networks and Deep Auto-Encoders. Features are computed using a multi-scale sliding window scheme. A connectionist recurrent approach is then used. It is trained to predict correct transcriptions of the input image from the associated sequence of features. Proposed methods are extensively evaluated on a large video database recorded from several Arabic TV channels.
Face recognition using non-linear image reconstruction

We present a face recognition technique based on a special type of convolutional neural network that is trained to extract characteristic features from face images and reconstruct the corresponding reference face images which are chosen beforehand for each individual to recognize. The reconstruction is realized by a so-called "bottle-neck" neural network that learns to project face images into a low-dimensional vector space and to reconstruct the respective reference images from the projected vectors. In contrast to methods based on the Principal Component Analysis (PCA), the Linear Discriminant Analysis (LDA) etc., the projection is non-linear and depends on the choice of the reference images. Moreover, local and global processing are closely interconnected and the respective parameters are conjointly learnt. Having trained the neural network, new face images can then be classified by comparing the respective projected vectors. We experimentally show that the choice of the reference images influences the final recognition performance and that this method outperforms linear projection methods in terms of precision and robustness.
Supervised Image Classification by SOM Activity Map Comparison

This article presents a method aiming at quantifying the visual similarity between two images. This kind of problem is recurrent in many applications such as object recognition, image classification, etc. In this paper, we propose to use self-organizing feature maps (SOM) to measure image similarity. To reach this goal, we feed local signatures associated to salient patches into the neural network. At the end of the learning step, each neural unit is tuned to a particular local signature prototype. During the recognition step, each image presented to the network generates a neural map that can be represented by an activity histogram. Image similarity is then computed by a quadratic distance between histograms. This scheme offers very promising results for image classification with a percentage of 84.47% of correct classification rates
An Efficient New PDE-based Characters Reconstruction after Graphics Removal

The separation between texts and graphics when they are overlapped is a challenging problem for digitization companies. In a previous work , we presented the first unsupervised fully automatic segmentation system adapted for colour business document with significant colour complexity and dithered background. The system achieves several operations to segment automatically colour images, separate text from noise and graphics and provides colour information about text colour. After split overlapped characters and separates characters from graphics, characters are broken. The OCR system becomes unable to recognize successfully broken characters and its efficiency is thus seriously affected. This paper presents the first Character Reconstruction System through a new PDE (Partial Differential Equation)-based approach. Our approach takes benefit of the combination of the anisotropic morphology proposed by Breuß and the Weickert Coherence enhancing shock filter diffusion. We introduce and present a continuous anisotropic morphology method driven by the main direction of the first order tensors applied in the neighborhood of the missing part left by the separation between text and graphics. It reconstructs the missing part even when the left area is larger than the strokes width. The coherency of the orientation of the tensors around missing parts overcomes the problem of image noises. The application of the ABBY FineReader OCR engine proves an important reduction in OCR errors. Our experiments show that our proposition compared to the existing state of the art requires no training steps and outperforms both of anisotropic morphology and the Weickert Coherence enhancing shock filter diffusion applied separately.
3D translational motion estimation from 2D displacements

Recovering 3D motion parameters from 2D displacements is a difficult task, given the influence of noise contained in these data, which correspond at best to a crude approximation of the real motion field. The need for stability of the system of equations to be solved is therefore essential. We present a novel method based on an unbiased estimator that aims at enhancing this stability and strongly reduces the influence of noise contamination. Experimental results using synthetic and real optical flows are presented to demonstrate the effectiveness of our method in comparison to a set of selected methods.
Visual Focus of Attention Estimation With Unsupervised Incremental Learning

In this paper, we propose a new method for estimating the visual focus of attention (VFOA) in a video stream captured by a single distant camera and showing several persons sitting around a table, like in formal meeting or video conferencing settings. The visual targets for a given person are automatically extracted online using an unsupervised algorithm that incrementally learns the different appearance clusters from low-level visual features computed from face patches provided by a face tracker without the need of an intermediate error-prone step of head pose estimation as in classical approaches. The clusters learned in that way can then be used to classify the different visual attention targets of the person during a tracking run, without any prior knowledge on the environment and the configuration of the room or the visible persons. The experiments on public datasets containing almost 2 h of annotated videos from meetings and video conferencing show that the proposed algorithm produces state-of-the-art results and even outperforms a traditional supervised method that is based on head orientation estimation and that classifies VFOA using Gaussian mixture models.
A neural architecture for fast and robust face detection

In this paper, we present a connectionist approach for detecting and precisely localizing semi-frontal human faces in complex images, making no assumption about the content or the lighting conditions of the scene, or about the size or the appearance of the faces. We propose a convolutional neural network architecture designed to recognize strongly variable face patterns directly from pixel images with no preprocessing, by automatically synthesizing its own set of feature extractors from a large training set of faces. We present in details the optimized design of our architecture, our learning strategy and the resulting process of face detection. We also provide experimental results to demonstrate the robustness of our approach and its capability to precisely detect extremely variable faces in uncontrolled environments.