nils-herrmann/clean_scirepeval huggingface dataset:
Learning a temporally invariant representation for visual tracking

In this paper, we propose to learn temporally invariant features from a large number of image sequences to represent objects for visual tracking. These features are trained on a convolutional neural network with temporal invariance constraints and robust to diverse motion transformations. We employ linear correlation filters to encode the appearance templates of targets and perform the tracking task by searching for the maximum responses at each frame. The learned filters are updated online and adapt to significant appearance changes during tracking. Extensive experimental results on challenging sequences show that the proposed algorithm performs favorably against state-of-the-art methods in terms of efficiency, accuracy, and robustness.
Saliency Detection via Absorbing Markov Chain

In this paper, we formulate saliency detection via absorbing Markov chain on an image graph model. We jointly consider the appearance divergence and spatial distribution of salient objects and the background. The virtual boundary nodes are chosen as the absorbing nodes in a Markov chain and the absorbed time from each transient node to boundary absorbing nodes is computed. The absorbed time of transient node measures its global similarity with all absorbing nodes, and thus salient objects can be consistently separated from the background when the absorbed time is used as a metric. Since the time from transient node to absorbing nodes relies on the weights on the path and their spatial distance, the background region on the center of image may be salient. We further exploit the equilibrium distribution in an ergodic Markov chain to reduce the absorbed time in the long-range smooth background regions. Extensive experiments on four benchmark datasets demonstrate robustness and efficiency of the proposed method against the state-of-the-art methods.
Weakly Supervised Object Localization with Progressive Domain Adaptation

We address the problem of weakly supervised object localization where only image-level annotations are available for training. Many existing approaches tackle this problem through object proposal mining. However, a substantial amount of noise in object proposals causes ambiguities for learning discriminative object models. Such approaches are sensitive to model initialization and often converge to an undesirable local minimum. In this paper, we address this problem by progressive domain adaptation with two main steps: classification adaptation and detection adaptation. In classification adaptation, we transfer a pre-trained network to our multi-label classification task for recognizing the presence of a certain object in an image. In detection adaptation, we first use a mask-out strategy to collect class-specific object proposals and apply multiple instance learning to mine confident candidates. We then use these selected object proposals to fine-tune all the layers, resulting in a fully adapted detection network. We extensively evaluate the localization performance on the PASCAL VOC and ILSVRC datasets and demonstrate significant performance improvement over the state-of-the-art methods.
Higher-Dimensional Affine Registration and Vision Applications

Affine registration has a long and venerable history in computer vision literature, and in particular, extensive work has been done for affine registration in R2 and R3. This paper studies affine registration in Rm with m typically ranging from 4 to 12. To justify breaking of this dimension barrier, the first part of the paper describes three novel matching problems that can be formulated and solved as affine point-set registration problems in dimensions greater than three: stereo correspondence under motion, image set matching, and covariant point-set matching, problems that are not only interesting in their own right but also have potential for important vision applications. Unfortunately, most of the existing affine registration algorithms do not generalize easily to higher dimensions due to their inefficiency. Therefore, the second part of this paper develops a novel algorithm for estimating the affine transform between two point sets in Rm. Specifically, the algorithm follows the common approach of iteratively solving the correspondences and transform. The initial correspondences are determined using the novel notion of local spectral features, features constructed from local distance matrices. Unlike many correspondence-based methods, the proposed algorithm is capable of registering point sets of different size, and the use of local features provides some degree of robustness against noise and outliers. The proposed algorithm is validated on a variety of synthetic point sets in different dimensions with varying degrees of deformation and noise, and the paper also shows experimentally that several instances of the aforementioned three matching problems can indeed be solved satisfactorily using the proposed affine registration algorithm.
Robust Object Tracking via Locality Sensitive Histograms

This paper presents a novel locality sensitive histogram (LSH) algorithm for visual tracking. Unlike the conventional image histogram that counts the frequency of occurrence of each intensity value by adding ones to the corresponding bin, an LSH is computed at each pixel location, and a floating-point value is added to the corresponding bin for each occurrence of an intensity value. The floating-point value exponentially reduces with respect to the distance to the pixel location where the histogram is computed. An efficient algorithm is proposed that enables the LSHs to be computed in time linear in the image size and the number of bins. In addition, this efficient algorithm can be extended to exploit color images. A robust tracking framework based on the LSHs is proposed, which consists of two main components: a new feature for tracking that is robust to illumination change and a novel multiregion tracking algorithm that runs in real time even with hundreds of regions. Extensive experiments demonstrate that the proposed tracking framework outperforms the state-of-the-art methods in challenging scenarios, especially when the illumination changes dramatically. Evaluation using the latest benchmark shows that our algorithm is the top performer.
What Makes an Object Memorable?

Recent studies on image memorability have shed light on what distinguishes the memorability of different images and the intrinsic and extrinsic properties that make those images memorable. However, a clear understanding of the memorability of specific objects inside an image remains elusive. In this paper, we provide the first attempt to answer the question: what exactly is remembered about an image? We augment both the images and object segmentations from the PASCAL-S dataset with ground truth memorability scores and shed light on the various factors and properties that make an object memorable (or forgettable) to humans. We analyze various visual factors that may influence object memorability (e.g. color, visual saliency, and object categories). We also study the correlation between object and image memorability and find that image memorability is greatly affected by the memorability of its most memorable object. Lastly, we explore the effectiveness of deep learning and other computational approaches in predicting object memorability in images. Our efforts offer a deeper understanding of memorability in general thereby opening up avenues for a wide variety of applications.
Robust Kernel Estimation with Outliers Handling for Image Deblurring

Estimating blur kernels from real world images is a challenging problem as the linear image formation assumption does not hold when significant outliers, such as saturated pixels and non-Gaussian noise, are present. While some existing non-blind deblurring algorithms can deal with outliers to a certain extent, few blind deblurring methods are developed to well estimate the blur kernels from the blurred images with outliers. In this paper, we present an algorithm to address this problem by exploiting reliable edges and removing outliers in the intermediate latent images, thereby estimating blur kernels robustly. We analyze the effects of outliers on kernel estimation and show that most state-of-the-art blind deblurring methods may recover delta kernels when blurred images contain significant outliers. We propose a robust energy function which describes the properties of outliers for the final latent image restoration. Furthermore, we show that the proposed algorithm can be applied to improve existing methods to deblur images with outliers. Extensive experiments on different kinds of challenging blurry images with significant amount of outliers demonstrate the proposed algorithm performs favorably against the state-of-the-art methods.
Fast and Accurate Head Pose Estimation via Random Projection Forests

In this paper, we consider the problem of estimating the gaze direction of a person from a low-resolution image. Under this condition, reliably extracting facial features is very difficult. We propose a novel head pose estimation algorithm based on compressive sensing. Head image patches are mapped to a large feature space using the proposed extensive, yet efficient filter bank. The filter bank is designed to generate sparse responses of color and gradient information, which can be compressed using random projection, and classified by a random forest. Extensive experiments on challenging datasets show that the proposed algorithm performs favorably against the state-of-the-art methods on head pose estimation in low-resolution images degraded by noise, occlusion, and blurring.
Compressed face hallucination

In this paper, we propose an algorithm to hallucinate faces in the JPEG compressed domain, which has not been well addressed in the literature. The proposed approach hallucinates compressed face images through an exemplar-based framework and solves two main problems. First, image noise introduced by JPEG compression is exacerbated through the super-resolution process. We present a novel formulation for face hallucination that uses the JPEG quantization intervals as constraints to recover the feasible intensity values from each image patch of a low-resolution input. Second, existing face hallucination methods are sensitive to noise contained in the compressed images. We regularize the compression noise caused by block discrete cosine transform coding, and reconstruct high-resolution images with the proposed gradient-guided total variation. Numerous experimental results show that the proposed algorithm generates favorable results than the combination of state-of-the-art face hallucination and de-noising algorithms.
Object Tracking Benchmark

Object tracking has been one of the most important and active research areas in the field of computer vision. A large number of tracking algorithms have been proposed in recent years with demonstrated success. However, the set of sequences used for evaluation is often not sufficient or is sometimes biased for certain types of algorithms. Many datasets do not have common ground-truth object positions or extents, and this makes comparisons among the reported quantitative results difficult. In addition, the initial conditions or parameters of the evaluated tracking algorithms are not the same, and thus, the quantitative results reported in literature are incomparable or sometimes contradictory. To address these issues, we carry out an extensive evaluation of the state-of-the-art online object-tracking algorithms with various evaluation criteria to understand how these methods perform within the same framework. In this work, we first construct a large dataset with ground-truth object positions and extents for tracking and introduce the sequence attributes for the performance analysis. Second, we integrate most of the publicly available trackers into one code library with uniform input and output formats to facilitate large-scale performance evaluation. Third, we extensively evaluate the performance of 31 algorithms on 100 sequences with different initialization settings. By analyzing the quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.
Extracting Image Regions by Structured Edge Prediction

We present two approaches to extract regions from structured edge detection. While the state-of-the-art algorithm based on globalized probability of boundary (gPb) generates a hierarchical region tree, it entails significant computational load. In this work, we exploit an efficient algorithm for structured edge prediction to extract regions. To generate high quality regions, we develop a novel algorithm to link the structured edge and gPb hierarchical image segmentation framework with steerable filters. The extracted regions are grouped by the proposed hierarchical grouping method to generate object proposals for effective detection and recognition problems. We demonstrate the effectiveness of our region generation for image segmentation on the BSDS500 database, and region generation for object proposals on the PASCAL VOC 2007 benchmark database. Experimental results show that the proposed algorithm achieves the comparable or superior quality to the state-of-the-art methods.
Bayesian Saliency via Low and Mid Level Cues

Visual saliency detection is a challenging problem in computer vision, but one of great importance and numerous applications. In this paper, we propose a novel model for bottom-up saliency within the Bayesian framework by exploiting low and mid level cues. In contrast to most existing methods that operate directly on low level cues, we propose an algorithm in which a coarse saliency region is first obtained via a convex hull of interest points. We also analyze the saliency information with mid level visual cues via superpixels. We present a Laplacian sparse subspace clustering method to group superpixels with local features, and analyze the results with respect to the coarse saliency region to compute the prior saliency map. We use the low level visual cues based on the convex hull to compute the observation likelihood, thereby facilitating inference of Bayesian saliency at each pixel. Extensive experiments on a large data set show that our Bayesian saliency model performs favorably against the state-of-the-art algorithms.
$L_0$ -Regularized Intensity and Gradient Prior for Deblurring Text Images and Beyond

We propose a simple yet effective <inline-formula><tex-math notation="LaTeX">$L_0$</tex-math><alternatives> <inline-graphic xlink:href="yang-ieq2-2551244.gif"/></alternatives></inline-formula>-regularized prior based on intensity and gradient for text image deblurring. The proposed image prior is based on distinctive properties of text images, with which we develop an efficient optimization algorithm to generate reliable intermediate results for kernel estimation. The proposed algorithm does not require any heuristic edge selection methods, which are critical to the state-of-the-art edge-based deblurring methods. We discuss the relationship with other edge-based deblurring methods and present how to select salient edges more principally. For the final latent image restoration step, we present an effective method to remove artifacts for better deblurred results. We show the proposed algorithm can be extended to deblur natural images with complex scenes and low illumination, as well as non-uniform deblurring. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art image deblurring methods.
Head and Body Orientation Estimation Using Convolutional Random Projection Forests

In this paper, we consider the problem of estimating the head pose and body orientation of a person from a low-resolution image. Under this setting, it is difficult to reliably extract facial features or detect body parts. We propose a convolutional random projection forest (CRPforest) algorithm for these tasks. A convolutional random projection network (CRPnet) is used at each node of the forest. It maps an input image to a high-dimensional feature space using a rich filter bank. The filter bank is designed to generate sparse responses so that they can be efficiently computed by compressive sensing. A sparse random projection matrix can capture most essential information contained in the filter bank without using all the filters in it. Therefore, the CRPnet is fast, e.g., it requires <inline-formula><tex-math notation="LaTeX">$0.04\;\mathrm{ms}$</tex-math><alternatives> <inline-graphic xlink:href="yang-ieq1-2784424.gif"/></alternatives></inline-formula> to process an image of <inline-formula><tex-math notation="LaTeX">$50\times 50$</tex-math><alternatives> <inline-graphic xlink:href="yang-ieq2-2784424.gif"/></alternatives></inline-formula> pixels, due to the small number of convolutions (e.g., 0.01 percent of a layer of a neural network) at the expense of less than 2 percent accuracy. The overall forest estimates head and body pose well on benchmark datasets, e.g., over 98 percent on the HIIT dataset, while requiring <inline-formula><tex-math notation="LaTeX">$3.8\;\mathrm{ms}$</tex-math><alternatives> <inline-graphic xlink:href="yang-ieq3-2784424.gif"/></alternatives></inline-formula> without using a GPU. Extensive experiments on challenging datasets show that the proposed algorithm performs favorably against the state-of-the-art methods in low-resolution images with noise, occlusion, and motion blur.
Multiple Non-rigid Surface Detection and Registration

Detecting and registering nonrigid surfaces are two important research problems for computer vision. Much work has been done with the assumption that there exists only one instance in the image. In this work, we propose an algorithm that detects and registers multiple nonrigid instances of given objects in a cluttered image. Specifically, after we use low level feature points to obtain the initial matches between templates and the input image, a novel high-order affinity graph is constructed to model the consistency of local topology. A hierarchical clustering approach is then used to locate the nonrigid surfaces. To remove the outliers in the cluster, we propose a deterministic annealing approach based on the Thin Plate Spline (TPS) model. The proposed method achieves high accuracy even when the number of outliers is nineteen times larger than the inliers. As the matches may appear sparsely in each instance, we propose a TPS based match growing approach to propagate the matches. Finally, an approach that fuses feature and appearance information is proposed to register each nonrigid surface. Extensive experiments and evaluations demonstrate that the proposed algorithm achieves promising results in detecting and registering multiple non-rigid surfaces in a cluttered scene.
Salient object detection via bootstrap learning

We propose a bootstrap learning algorithm for salient object detection in which both weak and strong models are exploited. First, a weak saliency map is constructed based on image priors to generate training samples for a strong model. Second, a strong classifier based on samples directly from an input image is learned to detect salient pixels. Results from multiscale saliency maps are integrated to further improve the detection performance. Extensive experiments on six benchmark datasets demonstrate that the proposed bootstrap learning algorithm performs favorably against the state-of-the-art saliency detection methods. Furthermore, we show that the proposed bootstrap learning approach can be easily applied to other bottom-up saliency models for significant improvement.
Unsupervised Representation Learning by Sorting Sequences

We present an unsupervised representation learning approach using videos without semantic labels. We leverage the temporal coherence as a supervisory signal by formulating representation learning as a sequence sorting task. We take temporally shuffled frames (i.e., in non-chronological order) as inputs and train a convolutional neural network to sort the shuffled sequences. Similar to comparison-based sorting algorithms, we propose to extract features from all frame pairs and aggregate them to predict the correct order. As sorting shuffled image sequence requires an understanding of the statistical temporal structure of images, training with such a proxy task allows us to learn rich and generalizable visual representation. We validate the effectiveness of the learned representation using our method as pre-training on high-level recognition problems. The experimental results show that our method compares favorably against state-of-the-art methods on action recognition, image classification, and object detection tasks.
Robust Visual Tracking via Least Soft-Threshold Squares

In this paper, we propose an online tracking algorithm based on a novel robust linear regression estimator. In contrast to existing methods, the proposed least soft-threshold squares (LSS) algorithm models the error term with the Gaussian-Laplacian distribution, which can be efficiently solved. For visual tracking, the Gaussian-Laplacian noise assumption enables our LSS model to handle the normal appearance change and outlier simultaneously. Based on the maximum joint likelihood of parameters, we derive an LSS distance metric to measure the difference between an observation sample and a dictionary of positive templates. Compared with the distance derived from ordinary least squares methods, the proposed metric is more effective in dealing with the outliers. In addition, we provide insights on the relationships among the LSS problem, Huber loss function, and trivial templates, which facilitate better understandings of the existing tracking methods. Finally, we develop a robust tracking algorithm based on the LSS distance metric with an update scheme and negative templates, and speed it up with a particle selection mechanism. Experimental results on numerous challenging image sequences demonstrate that the proposed tracking algorithm performs favorably than the state-of-the-art methods.
Object Tracking via Partial Least Squares Analysis

We propose an object tracking algorithm that learns a set of appearance models for adaptive discriminative object representation. In this paper, object tracking is posed as a binary classification problem in which the correlation of object appearance and class labels from foreground and background is modeled by partial least squares (PLS) analysis, for generating a low-dimensional discriminative feature subspace. As object appearance is temporally correlated and likely to repeat over time, we learn and adapt multiple appearance models with PLS analysis for robust tracking. The proposed algorithm exploits both the ground truth appearance information of the target labeled in the first frame and the image observations obtained online, thereby alleviating the tracking drift problem caused by model update. Experiments on numerous challenging sequences and comparisons to state-of-the-art methods demonstrate favorable performance of the proposed tracking algorithm.