nils-herrmann/clean_scirepeval huggingface dataset:
Joint source-channel coding optimization with packet loss resilience for video transmission

We propose a novel joint source-channel coding (JSCC) framework that jointly optimizes encoding modes of macroblocks and unequal error protection (UEP) of packets for error-resilient video transmission, and address the problem of source packet loss due to bit errors incurred in error-prone channels. In the proposed framework, we consider the source packet loss probability as a function of the encoding configuration of a slice. The task of optimization is complicated by the inherent interdependency between macroblocks in the same slice, since the encoding mode optimization for each macroblock requires the encoding configuration of entire slice to be known in advance. We resolve such interdependency via an iterative method that alternates between the optimization of the slice size and the associated encoding modes, and we also utilize adaptive quantization for coding residual for source rate control in different channel conditions. Simulation results show that our framework outperforms existing approaches.
Exploiting image structural similarity for single image rain removal

Without any prior knowledge or user interaction, single image rain removal has been a challenging task. Typically, one needs to disregard image components associated with the rain patterns, so that rain removal can be achieved via image reconstruction. By observing the limitations of standard batch-mode learning-based methods, we propose to exploit the structural similarity of the image bases for solving this task. By formulating the basis selection as an optimization problem, we are able to disregard those associated with rain patterns while the detailed image information can be preserved. Experiments on both synthetic and real-world images will verify the effectiveness of our proposed method.
New Weighted Support Vector K-means Clustering for Hierarchical Multi-class Classification

We propose a binary hierarchical classification structure to address the multi-class classification problem with a new hierarchical design method, weighted support vector k-means clustering, which automatically separates a set of classes into two smaller groups at each node in the hierarchy. This method is able to visualize and cluster high-dimensional support vector data; therefore, it greatly improves upon prior hierarchical classifier design. At each node in the hierarchy, we apply an SVRDM (support vector representation and discrimination machine) classifier, which offers generalization and good rejection of unseen false objects, which is not achieved by the standard SVM classifier. We provide a new theoretical basis for the good SVRDM rejection obtained, due to its looser constrained optimization problem, compared to that of an SVM. New classification and rejection test results are presented on a real IR (infra-red) database.
Learning sparse image representation with support vector regression for single-image super-resolution

Learning-based approaches for super-resolution (SR) have been studied in the past few years. In this paper, a novel single-image SR framework based on the learning of sparse image representation with support vector regression (SVR) is presented. SVR is known to offer excellent generalization ability in predicting output class labels for input data. Given a low resolution image, we approach the SR problem as the estimation of pixel labels in its high resolution version. The feature considered in this work is the sparse representation of different types of image patches. Prior studies have shown that this feature is robust to noise and occlusions present in image data. Experimental results show that our method is quantitatively more effective than prior work using bicubic interpolation or SVR methods, and our computation time is significantly less than that of existing SVR-based methods due to the use of sparse image representations.
Least-squares LDA via rank-one updates with concept drift

Standard linear discriminant analysis (LDA) is known to be computationally expensive due to the need to perform eigen-analysis. Based on the recent success of least-squares LDA (LSLDA), we propose a novel rank-one update method for LSLDA, which not only alleviates the computation and memory requirements, and is also able to solve the adaptive learning task of concept drift. In other words, our proposed LSLDA can efficiently capture the information from recently received data with gradual or abrupt changes in distribution. Moreover, our LSLDA can be extended to recognize data with newly-added class labels during the learning process, and thus exhibits excellent scalability. Experimental results on both synthetic and real datasets confirm the effectiveness of our propose method.
Depth map super-resolution via Markov Random Fields without texture-copying artifacts

The use of time-of-flight sensors enables the record of full-frame depth maps at video frame rate, which benefits a variety of 3D image or video processing applications. However, such depth maps are typically corrupted by noise and with limited resolution. In this paper, we present a learning-based depth map super-resolution framework by solving a MRF labeling optimization problem. With the captured depth map and the associated high-resolution color image, our proposed method exhibits the capability of preserving the edges of range data while suppressing the artifacts of texture copying due to color discontinuities. Quantitative and qualitative experimental results demonstrate the effectiveness and robustness of our approach over prior depth map upsampling works.
Automatic object extraction in single-concept videos

We propose a motion-driven video object extraction (VOE) method, which is able to model and segment foreground objects in single-concept videos, i.e. videos which have only one object category of interest but may have multiple object instances with pose, scale, etc. variations. Given such a video, we construct a compact shape model induced by motion cues, and extract the foreground and background color information accordingly. We integrate these feature models into a unified framework via a conditional random field (CRF), and this CRF can be applied to video object segmentation and further video editing and retrieval applications. One of the advantages of our method is that we do not require the prior knowledge of the object of interest, and thus no training data or predetermined object detectors are needed; this makes our approach robust and practical to real-world problems. Very attractive empirical results on a variety of videos with highly articulated objects support the feasibility of our proposed method.
Style retrieval from natural images

It has been a challenging task to identify and distinguish between images of different styles. The challenges mainly come from the extraction of high-level image semantic information, and the presence of the associated ambiguity. In this work, we propose a ranking model for style identification. Given training images of different styles, we learn a pointwise ranking model for each style based on random forests. To handle the high dimensionality of visual features and to prevent against possible ambiguity, we further introduce dimension reduction and pruning techniques for our random forests. In our experiments, we provide quantitative evaluation for style categorization in terms of mean square error (MSE) and relative ranking accuracy. Moreover, our visualization and qualitative results support the use of the proposed method for style retrieval of natural images.
Superpixel-based large displacement optical flow

It has been a challenging task to estimate optical flow for videos in which either foreground or background exhibits remarkable motion information (i.e., large displacement), or those with insufficient resolution due to artifacts like motion blur or noise. We present a novel optical flow algorithm, which approaches the above problem as solving the task of energy minimization, which exploits image data and smoothness terms at the superpixel level. Our proposed method can be considered as an extended mean-shift algorithm, which advances color and gradient information of superpixels across consecutive frames with smoothness guarantees. Since we do not require assumptions of linearlization during optimization (as standard optical flow approaches do), we are able to alleviate local minimum problems and thus produce improved estimation results. Empirical results on the MPI-Sintel video dataset verify the effectiveness of our proposed method.
No More Discrimination: Cross City Adaptation of Road Scene Segmenters

Despite the recent success of deep-learning based semantic segmentation, deploying a pre-trained road scene segmenter to a city whose images are not presented in the training set would not achieve satisfactory performance due to dataset biases. Instead of collecting a large number of annotated images of each city of interest to train or refine the segmenter, we propose an unsupervised learning approach to adapt road scene segmenters across different cities. By utilizing Google Street View and its timemachine feature, we can collect unannotated images for each road scene at different times, so that the associated static-object priors can be extracted accordingly. By advancing a joint global and class-specific domain adversarial learning framework, adaptation of pre-trained segmenters to that city can be achieved without the need of any user annotation or interaction. We show that our method improves the performance of semantic segmentation in multiple cities across continents, while it performs favorably against state-of-the-art approaches requiring annotated training data.
A Novel Multiple Kernel Learning Framework for Heterogeneous Feature Fusion and Variable Selection

We propose a novel multiple kernel learning (MKL) algorithm with a group lasso regularizer, called group lasso regularized MKL (GL-MKL), for heterogeneous feature fusion and variable selection. For problems of feature fusion, assigning a group of base kernels for each feature type in an MKL framework provides a robust way in fitting data extracted from different feature domains. Adding a mixed norm constraint (i.e., group lasso) as the regularizer, we can enforce the sparsity at the group/feature level and automatically learn a compact feature set for recognition purposes. More precisely, our GL-MKL determines the optimal base kernels, including the associated weights and kernel parameters, and results in improved recognition performance. Besides, our GL-MKL can also be extended to address heterogeneous variable selection problems. For such problems, we aim to select a compact set of variables (i.e., feature attributes) for comparable or improved performance. Our proposed method does not need to exhaustively search for the entire variable space like prior sequential-based variable selection methods did, and we do not require any prior knowledge on the optimal size of the variable subset either. To verify the effectiveness and robustness of our GL-MKL, we conduct experiments on video and image datasets for heterogeneous feature fusion, and perform variable selection on various UCI datasets.
Active learning based clothing image recommendation with implicit user preferences

We address the problem of user-specific clothing image recommendation in this paper. Different from prior retrieval approaches, we advance an active learning scheme during retrieval for inferring user preferences. With a recently developed sparse-coding based algorithm for content-based image retrieval, we utilize support vector regression (SVR) with a user-interaction training stage to observe user preferences based on the feedback of retrieval results. Therefore, there is no need to explicitly ask his/her preferences such as desirable colors or patterns of clothing images. A subjective evaluation on a commercial clothing image dataset confirms the effectiveness of our method, which is shown to produce more satisfactory recommendation results when comparing to state-of-the-art content-based image retrieval approaches.
Propagated image filtering

We propose the propagation filter as a novel image filtering operator, with the goal of smoothing over neighboring image pixels while preserving image context like edges or textural regions. In particular, our filter does not to utilize explicit spatial kernel functions as bilateral and guided filters do. We will show that our propagation filter can be viewed as a robust estimator, which minimizes the expected difference between the filtered and desirable image outputs. We will also relate propagation filtering to belief propagation, and suggest techniques if further speedup of the filtering process is necessary. In our experiments, we apply our propagation filter to a variety of applications such as image denoising, smoothing, fusion, and high-dynamic-range (HDR) compression. We will show that improved performance over existing image filters can be achieved.
Heterogeneous Domain Adaptation and Classification by Exploiting the Correlation Subspace

We present a novel domain adaptation approach for solving cross-domain pattern recognition problems, i.e., the data or features to be processed and recognized are collected from different domains of interest. Inspired by canonical correlation analysis (CCA), we utilize the derived correlation subspace as a joint representation for associating data across different domains, and we advance reduced kernel techniques for kernel CCA (KCCA) if nonlinear correlation subspace are desirable. Such techniques not only makes KCCA computationally more efficient, potential over-fitting problems can be alleviated as well. Instead of directly performing recognition in the derived CCA subspace (as prior CCA-based domain adaptation methods did), we advocate the exploitation of domain transfer ability in this subspace, in which each dimension has a unique capability in associating cross-domain data. In particular, we propose a novel support vector machine (SVM) with a correlation regularizer, named correlation-transfer SVM, which incorporates the domain adaptation ability into classifier design for cross-domain recognition. We show that our proposed domain adaptation and classification approach can be successfully applied to a variety of cross-domain recognition tasks such as cross-view action recognition, handwritten digit recognition with different features, and image-to-text or text-to-image classification. From our empirical results, we verify that our proposed method outperforms state-of-the-art domain adaptation approaches in terms of recognition performance.
Low-rank matrix recovery with structural incoherence for robust face recognition

We address the problem of robust face recognition, in which both training and test image data might be corrupted due to occlusion and disguise. From standard face recognition algorithms such as Eigenfaces to recently proposed sparse representation-based classification (SRC) methods, most prior works did not consider possible contamination of data during training, and thus the associated performance might be degraded. Based on the recent success of low-rank matrix recovery, we propose a novel low-rank matrix approximation algorithm with structural incoherence for robust face recognition. Our method not only decomposes raw training data into a set of representative basis with corresponding sparse errors for better modeling the face images, we further advocate the structural incoherence between the basis learned from different classes. These basis are encouraged to be as independent as possible due to the regularization on structural incoherence. We show that this provides additional discriminating ability to the original low-rank models for improved performance. Experimental results on public face databases verify the effectiveness and robustness of our method, which is also shown to outperform state-of-the-art SRC based approaches.
Recognizing heterogeneous cross-domain data via generalized joint distribution adaptation

In this paper, we propose a novel algorithm of Generalized Joint Distribution Adaptation (G-JDA) for heterogeneous domain adaptation (HDA), which associates and recognizes cross-domain data observed in different feature spaces (and thus with different dimensionality). With the objective to derive a domain-invariant feature subspace for relating source and target-domain data, our G-JDA learns a pair of feature projection matrices (one for each domain), which allows us to eliminate the difference between projected cross-domain heterogeneous data by matching their marginal and class-conditional distributions. We conduct experiments on cross-domain classification tasks using data across different features, datasets, and modalities. We confirm that our G-JDA would perform favorably against state-of-the-art HDA approaches.
Learning patch-based anchors for face hallucination

With the goal of increasing the resolution of face images, recent face hallucination methods advance learning techniques which observe training low and high-resolution patches for recovering the output image of interest. Since most existing patch-based face hallucination approaches do not consider the location information of the patches to be hallucinated, the resulting performance might be limited. In this paper, we propose an anchored patch-based hallucination method, which is able to exploit and identify image patches exhibiting structurally and spatially similar information. With these representative anchors observed, improved performance and computation efficiency can be achieved. Experimental results demonstrate that our proposed method achieves satisfactory performance and performs favorably against recent face hallucination approaches.
A Self-Learning Approach to Single Image Super-Resolution

Learning-based approaches for image super-resolution (SR) have attracted the attention from researchers in the past few years. In this paper, we present a novel self-learning approach for SR. In our proposed framework, we advance support vector regression (SVR) with image sparse representation, which offers excellent generalization in modeling the relationship between images and their associated SR versions. Unlike most prior SR methods, our proposed framework does not require the collection of training low and high-resolution image data in advance, and we do not assume the reoccurrence (or self-similarity) of image patches within an image or across image scales. With theoretical supports of Bayes decision theory, we verify that our SR framework learns and selects the optimal SVR model when producing an SR image, which results in the minimum SR reconstruction error. We evaluate our method on a variety of images, and obtain very promising SR results. In most cases, our method quantitatively and qualitatively outperforms bicubic interpolation and state-of-the-art learning-based SR approaches.
Automatic annotation of Web videos

Most Web videos are captured in uncontrolled environments (e.g. videos captured by freely-moving cameras with low resolution); this makes automatic video annotation very difficult. To address this problem, we present a robust moving foreground object detection method followed by the integration of features collected from heterogeneous domains. We advance SIFT feature matching and present a probabilistic framework to construct consensus foreground object templates (CFOT). The CFOT can detect moving foreground objects of interest across video frames, and this allows us to extract visual features from foreground regions of interest. Together with the use of audio features, we are able to improve resulting annotation accuracy. We conduct experiments and achieve promising results on a Web video dataset collected from YouTube.
Undersampled Face Recognition via Robust Auxiliary Dictionary Learning

In this paper, we address the problem of robust face recognition with undersampled training data. Given only one or few training images available per subject, we present a novel recognition approach, which not only handles test images with large intraclass variations such as illumination and expression. The proposed method is also to handle the corrupted ones due to occlusion or disguise, which is not present during training. This is achieved by the learning of a robust auxiliary dictionary from the subjects not of interest. Together with the undersampled training data, both intra and interclass variations can thus be successfully handled, while the unseen occlusions can be automatically disregarded for improved recognition. Our experiments on four face image datasets confirm the effectiveness and robustness of our approach, which is shown to outperform state-of-the-art sparse representation-based methods.
Style-oriented landmark retrieval and summarization

While the task of visual summarization aims to select representative images from an image collection, we solve a unique problem of style-oriented landmark retrieval and summarization from photographic images of a city. Instead of performing summarization or clustering on landmark images from a city, we allow the user to provide a query input which is not from the city of interest, while the goal is to retrieve and summarize the landmark images with similar style-dependent landmark images, followed by a style-consistent image summarization across landmark categories. As a result, our summarized outputs from various landmarks would exhibit similar image style as that of the query. Our experiments will confirm that the use of our proposed method is able to perform favorably against existing or baseline approaches with improved query-dependent style consistency.
Domain Adaptive Self-Taught Learning for Heterogeneous Face Recognition

Recognizing image data across different domains has been a challenging task. For biometrics, heterogeneous face recognition (HFR) deals with recognition problems in which training/gallery images are collected in terms of one modality (e.g., photos), while test/probe images are observed in the other (e.g., sketches). In this paper, we present a domain adaptation approach for solving HFR problems. By utilizing external face images (i.e., those collected from the subjects not of interest) from both source and target domains, we propose a novel Domain-independent Component Analysis (DiCA) algorithm for deriving a common subspace for relating and representing cross-domain image data. In order to introduce improved representation ability, we further advance the self-taught learning strategy for learning a domain-independent dictionary in our DiCA subspace, which can be applied to both gallery and probe images of interest to improve representation and recognition. Different from some prior domain-adaptation approaches, we do not require the data correspondences (i.e., data pairs) when collecting external cross-domain image data, nor the label information is needed for learning the common feature space when associating different domains. Thus, our method is practical for real-world cross-domain classification problems. In our experiments, we consider sketch-to-photo and near-infrared (NIR) to visible spectrum (VIS) face recognition problems for evaluating the performance of our proposed approach.
Learning auxiliary dictionaries for undersampled face recognition

In this paper, we address the problem of robust face recognition using undersampled data. Given only one or few face images per class, our proposed method not only handles test images with large intra-class variations such as illumination and expression, it is also able to recognize the corrupted ones due to occlusion or disguise. In our work, we advocate the learning of auxiliary dictionaries from the subjects not of interest. With the proposed optimization algorithm which jointly solves the tasks of auxiliary dictionary learning and sparsere-presentation based face recognition, our approach is able to model the above intra-class variations and corruptions for improved recognition. Our experiments on two face image datasets confirm the effectiveness and robustness of our approach, which is shown to outperform state-of-the-art sparse representation based methods.