nils-herrmann/clean_scirepeval huggingface dataset:
Reduced Reference Image Quality Assessment Based on Entropy of Classified Primitives

The human visual perception is a layered progressive process that brain assimilates visual information gradually, from primary information, structural information to detailed information. Recently, the visual primitives (atoms in the dictionary) extracted by sparse representation have been shown to be highly related to the layered progressive process of human visual perception. In this paper, the visual primitives are first classified into three categories: DCprimary, sketch and texture in terms of their inherent properties regarding tothe perceptual information. Then, we propose a novel reduced reference (RR) image quality assessment (IQA) metric using perceptual information represented by entropy of classified primitives (EoCP). Specifically, EoCP is a measurement of the distribution statistics of the visual primitives, which can represent the visual information. The differences of EoCPs between the reference image and its distorted version are calculated as features to characterize perceptual loss. The extracted features (only three scalars) are used to compute the quality score by a prediction function which is trained using support vector regression(SVR). Experimental results on LIVE, CSIQ and TID2013 image databases demonstrate that the proposed metric achieves high consistency with the human perception and show competitive performance with state-of-the-art IQA metrics.
Robust Stereo Matching Combining SIFT Descriptor with NCC under MRF Framework

There have been several stereo matching methods that perform well under the circumstance of color consistency. However, various factors including radiometric and device variations between images will drop color consistency and then seriously degrade the performance of those methods. In this paper, we propose a robust method to cover these situations. We use a measurement combining SIFT descriptor in intensity space after color histogram equalization with Normalized Cross Correlation (NCC) in color invariant log-chromaticity intensity space to compute the cost of point correspondence. The measurement ensures that illumination-independent gradient information and color invariant correlated information are integrated properly. We evaluate our method and find it outperform state-of-the-art algorithms, in particular on the datasets for radiometric variations.
Transition movement models for large vocabulary continuous sign language recognition

The major challenges that sign language recognition (SLR) now faces are developing methods that solve large vocabulary continuous sign problems. In this paper, large vocabulary continuous SLR based on transition movement models is proposed. The proposed method employs the temporal clustering algorithm to cluster a large amount of transition movements, and then the corresponding training algorithm is also presented for automatically segmenting and training these transition movement models. The clustered models can improve the generalization of transition movement models, and are very suitable for large vocabulary continuous SLR. At last, the estimated transition movement models, together with sign models, are viewed as candidate models of the Viterbi search algorithm for recognizing continuous sign language. Experiments show that continuous SLR based on transition movement models has good performance over a large vocabulary of 5113 signs.
An End-to-End Compression Framework Based on Convolutional Neural Networks

Traditional image coding standards (such as JPEG and JPEG2000) make the decoded image suffer from many blocking artifacts or noises since the use of big quantization steps. To overcome this problem, we proposed an end-to-end compression framework based on two CNNs, as shown in Figure 1, which produce a compact representation for encoding using a third party coding standard and reconstruct the decoded image, respectively. To make two CNNs effectively collaborate, we develop a unified end-to-end learning framework to simultaneously learn CrCNN and ReCNN such that the compact representation obtained by CrCNN preserves the structural information of the image, which facilitates to accurately reconstruct the decoded image using ReCNN and also makes the proposed compression framework compatible with existing image coding standards.
Prior-Based Quantization Bin Matching for Cloud Storage of JPEG Images

Millions of user-generated images are uploaded to social media sites like Facebook daily, which translate to a large storage cost. However, there exists an asymmetry in upload and download data: only a fraction of the uploaded images are subsequently retrieved for viewing. In this paper, we propose a cloud storage system that reduces the storage cost of all uploaded JPEG photos, at the expense of a controlled increase in computation mainly during download of requested image subset. Specifically, the system first selectively re-encodes code blocks of uploaded JPEG images using coarser quantization parameters for smaller storage sizes. Then during download, the system exploits known signal priors—sparsity prior and graph-signal smoothness prior—for reverse mapping to recover original fine quantization bin indices, with either deterministic guarantee (lossless mode) or statistical guarantee (near-lossless mode). For fast reverse mapping, we use small dictionaries and sparse graphs that are tailored for specific clusters of similar blocks, which are classified via tree-structured vector quantizer. During image upload, cluster indices identifying the appropriate dictionaries and graphs for the re-quantized blocks are encoded as side information using a differential distributed source coding scheme to facilitate reverse mapping during image download. Experimental results show that our system can reap significant storage savings (up to 12.05%) at roughly the same image PSNR (within 0.18 dB).
An End-to-End Compression Framework Based on Convolutional Neural Networks

Deep learning, e.g., convolutional neural networks (CNNs), has achieved great success in image processing and computer vision especially in high-level vision applications, such as recognition and understanding. However, it is rarely used to solve low-level vision problems such as image compression studied in this paper. Here, we move forward a step and propose a novel compression framework based on CNNs. To achieve high-quality image compression at low bit rates, two CNNs are seamlessly integrated into an end-to-end compression framework. The first CNN, named compact convolutional neural network (ComCNN), learns an optimal compact representation from an input image, which preserves the structural information and is then encoded using an image codec (e.g., JPEG, JPEG2000, or BPG). The second CNN, named reconstruction convolutional neural network (RecCNN), is used to reconstruct the decoded image with high quality in the decoding end. To make two CNNs effectively collaborate, we develop a unified end-to-end learning algorithm to simultaneously learn ComCNN and RecCNN, which facilitates the accurate reconstruction of the decoded image using RecCNN. Such a design also makes the proposed compression framework compatible with existing image coding standards. Experimental results validate that the proposed compression framework greatly outperforms several compression frameworks that use existing image coding standards with the state-of-the-art deblocking or denoising post-processing methods.
Shot Boundary Detection by a Hierarchical Supervised Approach

Video shot boundary detection plays an important role in video processing. It is the first step toward video-content analysis and content-based video retrieval. We develop a hierarchical approach for shot boundary detection based on the assumption that hierarchy helps to take decisions by reducing the amount of indeterminate transitions. Our method consists in first detecting abrupt transitions using a learning-based approach, then non-abrupt transitions are split into gradual transitions and normal frames. We describe in this paper, a machine learning system for shot boundary detection. The core of this system is a kernel-based SVM classifier. We present some results obtained for shot extraction TRECVID 2006
Active Boosting for Interactive Object Retrieval

This paper presents a new algorithm based on boosting for interactive object retrieval in images. Recent works propose ”online boosting” algorithms where weak classifier sets are iteratively trained from data. These algorithms are proposed for visual tracking in videos, and are not well adapted to ”online boosting” for interactive retrieval. We propose in this paper to iteratively build weak classifiers from images, labeled as positive by the user during a retrieval session. A novel active learning strategy for the selection of images for user annotation is also proposed. This strategy is used to enhance the strong classifier resulting from ”boosting” process, but also to build new weak classifiers. Experiments have been carried out on a generalist database in order to compare the proposed method to a SVM based reference approach.
B-spline active contours for fast video segmentation

Video segmentation is among the most important challenges of video processing and compression (MPEG4 and MPEG-7). A drawback of classical methods is the computational cost due to the model complexity. In this paper we propose to use a B-spline parametric contour to implement a region-based active contour segmentation Hence we get a fast variational method based on active contours with an intrinsic regularizing constraint. More precisely the evolution force follows from the minimization of a region-based criterion. The theory of B-splines allows analytical computation of the contour curvature at any point. The model complexity is fixed and depends on the desired level of detail. This complexity is highly compared to non-parametric methods. We compare our new approach to classical parametric polygon-based methods. We show experiments on real video sequences.
Fast approximate kernel-based similarity search for image retrieval task

In content based image retrieval, the success of any distance-based indexing scheme depends critically on the quality of the chosen distance metric. We propose in this paper a kernel-based similarity approach working on sets of vectors to represent images. We introduce a method for fast approximate similarity search in large image databases with our kernel-based similarity metric. We evaluate our algorithm on image retrieval task and show it to be accurate and faster than linear scanning.
3D Content-Based Retrieval in Artwork Databases

In this paper, we present first results obtained in the frame of the EROS-3D project, which aims at dealing with a collection of artwork 3D models, i.e. visualize them, classify them and compare them. Some 3D descriptors are used, in association with our active learning search engine RETIN. 3D features are described as well as our new system of classification and retrieval of objects, which we called RETIN-3D.
Video Segmentation by Supervised Learning

In most of video shot boundary detection algorithms, proposed in the literature, several parameters and thresholds have to be set in order to achieve good results. In this paper, to get rid of parameters and thresholds, we explore a supervised classification method for video shot segmentation. We transform the temporal segmentation into a class categorization issue. Our approach defines a uniform framework for combining different kinds of features extracted from the video. Our method does not require any pre-processing step to compensate motion or post-processing filtering to eliminate false detected transitions. The experiments, following strictly the TRECVID 2002 competition protocol, provide very good results dealing with a large amount of features thanks to our kernel-based SVM classification method
Segmentation of cellular structures in actin tagged fluorescence confocal microscopy images

The paper reports on a novel method for reconstruction of cellular features including cell nuclei and cellular boundaries from actin tagged fluorescence confocal microscopy images. Such reconstruction can provide spatial context for subsequent quantitative analysis of changes to actin organisation and cell morphology in both controlled and stressed cell cultures. The proposed method is fully automatic and is formulated within active contour multiphase level set framework. The derived level set evolution PDEs combine previously proposed curvature and advection flows with propagation flow defined by specially designed set of geodesic distance maps. Additionally the proposed PDEs include additional components to impose known inclusion/exclusion topological constraints between cellular structures. The paper gives an overview of the proposed methodology as well as reports on initial results obtained for monolayer of human prostate cells (PNT2) culture visualised using acting tagged fluorescence confocal microscopy.
Windows and facades retrieval using similarity on graph of contours

The development of street-level geoviewers become recently a very active and challenging research topic. In this context, the detection, representation and classification of windows can be beneficial for the identification of the respective facade. In this paper, a novel method for windows and facade retrieval is presented. This method, based on a similarity of graph of contours, introduces a new kernel on graph for inexact graph matching. We design a kernel similarity function for structured sets of contours which will take into account the variations of contour orientation inside a structure set, as well as spatial proximity. Then we are able to extract a window as a sub-graph of the graph of all contours of the facade image and to retrieve similar windows from a database of images of facades.
Comparative Study of Recent Multimodal Evolutionary Algorithms

Multimodal Optimization (MMO) aims at identifying several best solutions to a problem whereas classical optimization converge oftenly to only one good solution. MMO has been an active research area in the past years and several new evolutionary algorithms have been developed to tackle multimodal problems. In this work, we compare extensively three recent evolutionary algorithms (MoBiDE, Multimodal NSGAII and MOMMOP). Each algorithm uses multiobjectivization, together with niching techniques to address scalar (single objective) MMO problems. We have fully re-implemented MoBiDE and MM-NSGAII in order to better evaluate their sensitivity to parameter changes and their strengths and weaknesses. We have carefully evaluated all algorithms on the same benchmark functions and with the same parameters settings. The influence of the intrinsic parameters for each algorithm are stressed and the algorithms are also compared to a non-multimodal evolutionary algorithm to better highlight the impact of the multimodal adaptations. Moreover, full access to the detailed results and source code is granted on our website for the ease of reproducibility.
Locality-Sensitive Hashing for Chi2 Distance

In the past 10 years, new powerful algorithms based on efficient data structures have been proposed to solve the problem of Nearest Neighbors search (or Approximate Nearest Neighbors search). If the Euclidean Locality Sensitive Hashing algorithm, which provides approximate nearest neighbors in a euclidean space with sublinear complexity, is probably the most popular, the euclidean metric does not always provide as accurate and as relevant results when considering similarity measure as the Earth-Mover Distance and χ2 distances. In this paper, we present a new LSH scheme adapted to χ2 distance for approximate nearest neighbors search in high-dimensional spaces. We define the specific hashing functions, we prove their local-sensitivity, and compare, through experiments, our method with the Euclidean Locality Sensitive Hashing algorithm in the context of image retrieval on real image databases. The results prove the relevance of such a new LSH scheme either providing far better accuracy in the context of image retrieval than euclidean scheme for an equivalent speed, or providing an equivalent accuracy but with a high gain in terms of processing speed.
Optimization on active learning strategy for object category retrieval

Active learning is a machine learning technique which has attracted a lot of research interest in the content-based image retrieval (CBIR) in recent years. To be effective, an active learning system must be fast and efficient using as few (relevance) feedback iterations as possible. Scalability is the major problem for such an on-line learning method, since the complexity of such methods on a database of size n is in the best case O(n * log(n)). In this article we propose a strategy to overcome this limitation. Our technique exploits ultra fast retrieval methods like Locality Sensitive Hashing (LSH), recently applied for unsupervised image retrieval. Combined with active selection, our method is able to achieve very fast active learning task in very large database. Experiments on VOC2006 database are reported, results are obtained four times faster while preserving the accuracy.
Singlets: Multi-resolution Motion Singularities for Soccer Video Abstraction

The burst of video production appeals for new browsing frameworks. Chiefly in sports, TV companies have years of recorded match archives to exploit and sports fans are looking for replay, summary or collection of events. In this work, we design a new multi-resolution motion feature for video abstraction. This descriptor is based on optical flow singularities tracked along the video. We use these singlets in order to detect zooms, slow-motions and salient moments in soccer games and finally to produce an automatic summarization of a game. We produce a database for soccer video summarization composed of 4 soccer matches from HDTV games for the FIFA world cup 2014 annotated with goals, fouls, corners and salient moments to make a summary. We correctly detect 88.2% of saliant moments using this database. To highlight the generalization of our approach, we test our system on the final game of the handball world championship 2015 without any retraining, refining or adaptation.
A Tensor Based on Optical Flow for Global Description of Motion in Videos

Motion is one of the main characteristics that describe the semantic information of videos. In this work, a global video descriptor based on orientation tensors is proposed. This descriptor is obtained by combining polynomial coefficients calculated for each image in a video. The coefficients are found through the projection of the optical flow on Legendre polynomials, reducing the dimension of per frame motion estimations. The sequence of coefficients are then combined using orientation tensors. The global tensor descriptor created is evaluated by a classification of the KTH video database with a SVM classifier.
KPPF: Keypoint-Based Point-Pair-Feature for Scalable Automatic Global Registration of Large RGB-D Scans

One of the most important challenges in the field of 3D data processing is to be able to reconstruct a complete 3D scene with a high accuracy from several captures. Usually this process is achieved through two main phases: a coarse, or rough, alignment step then a fine alignment. In this article we propose an automatic scalable global registration method (i.e. without arbitrary pose of the sensor) under the following constraints: markerless, very large scale data (several, potentially many millions of points per scans), little overlap between scans, for more than two or three dozens of scans, without a priori knowledge on the 6 degrees of freedom. Here we only address the coarse alignment, and consider the fine alignment step tackled by dedicated existing approaches such as Iterative Closest Point (ICP) . We evaluate thoroughly our method on our own dataset of 33 real large scale scans of an indoor building. The data presents some pairs of scans with very little overlap, architectural challenges (a patio and a rotunda open through several levels of the buildings, etc), several millions of points per scan. We will make this dataset public as part of a benchmark available for the community. We have thus evaluated the accuracy of our method, the scalability to the initial amount of points and the robustness to occlusions, little scan overlap and architectural challenges.
Statistical region-based active contour using optimization of alpha-divergence family for image segmentation

This article deals with statistical region-based active contour segmentation using the alpha-divergence family as similarity measure between the density probability functions of the background and the object regions of interest. Following previous publications on that topic, main originality of this contribution is in the proposed joint optimization of the energy steering the evolution of the active curve and the parameter alpha related to the metric of the divergence and closely related to the statistical luminance distribution of the data. Experiments are shown on both synthetic noisy and textured data as well as on real images (natural and medical ones). We show that the joint optimization process leads to satisfying results for every targeted tasks: above all, it is shown that the proposed approach overcome classic statistical-based region active contour approach using Kullback-Leibler divergence as similarity measure, that can stuck in local extrema during the usual optimization process.
Efficient Bag-of-Feature kernel representation for image similarity search

Although “Bag-of-Features” image models have shown very good potential for object matching and image retrieval, such a complex data representation requires computationally expensive similarity measure evaluation. In this paper, we propose a framework unifying dictionary-based and kernel-based similarity functions that highlights the tradeoff between powerful data representation and eff cient similarity computation. On the basis of this formalism, we propose a new kernel-based similarity approach for Bag-of-Feature descriptions. We introduce a method for fast similarity search in large image databases. The conducted experiments prove that our approach is very competitive among State-of-the-art methods for similarity retrieval tasks.
Scalable active learning strategy for object category retrieval

Since the digital revolution, the volume of images to be processed has grown exponentially. Interactive search systems have to deal with these huge databases to remain effective. As the complexity of on-line learning methods is at least linear in the size of the database, scalability is the major problem for these methods. Fast retrieval systems, with index structures for fast navigation, have hence become like a Holy Grail. In this article, we propose a strategy to overcome this scalability limitation. Our technique exploits ultra fast retrieval methods as Locally Sensitive Hashing to speed up active learning system. Experiments on database of 180K images are reported. The results show that our method is 45 times faster than state of the art approaches for similar accuracy.