nils-herrmann/clean_scirepeval huggingface dataset:
Model-based clustering and analysis of video scenes

We make two contributions. First, we develop an unsupervised method to discover clusters of video scenes and summarize them with a concise Gaussian mixture model. To search for the best possible model, an effective procedure is devised to compare among models with different dimensions (i.e., numbers of mixture components) and, for a given dimension, among models with different parameters. Second, we propose a scene interference measure to characterize the interaction among different scenes of a video sequence. When applied to the clustered video scenes, the measure can reveal the dominant video segments of a class of videos without requiring much domain-specific knowledge. The proposed methods have been tested with a large number of sports videos and promising results are reported.
Set-to-set gait recognition across varying views and walking conditions

This paper examines the multiview gait recognition problem in which human gait sequences are collected from several different views simultaneously. Motivated by the fact that set-based feature representation can handle certain intra-subject variations, we propose a new Multiview Subspace Representation (MSR) method for gait recognition across varying views and walking conditions. It takes samples collected from different views of the same subject as a feature set and uses a subspace to represent such information. Then, the similarity of two subjects is measured by the distance between two subspaces and a simple yet effective Weighted Subspace Distance (WSD) algorithm is applied to calculate the similarity. There are two notable advantages of our proposed method: 1) we need not know the exact view of the test gait sequence in advance, and 2) some extent of intra-subject variations can be effectively handled. Experimental results on two benchmark multi-view gait databases are presented to demonstrate the effectiveness of the proposed method.
Discriminative Deep Metric Learning for Face Verification in the Wild

This paper presents a new discriminative deep metric learning (DDML) method for face verification in the wild. Different from existing metric learning-based face verification methods which aim to learn a Mahalanobis distance metric to maximize the inter-class variations and minimize the intra-class variations, simultaneously, the proposed DDML trains a deep neural network which learns a set of hierarchical nonlinear transformations to project face pairs into the same feature subspace, under which the distance of each positive face pair is less than a smaller threshold and that of each negative pair is higher than a larger threshold, respectively, so that discriminative information can be exploited in the deep network. Our method achieves very competitive face verification performance on the widely used LFW and YouTube Faces (YTF) datasets.
Fast block-based motion estimation using integral frames

Block-based motion estimation is widely used in video compression for exploitation of video temporal redundancy. Although effective, the process is arguably the most computationally intensive part of a typical video encoder. To speed up the process, a large number of fast block-matching algorithms (BMAs) have been proposed for motion estimation by limiting the number of search locations or simplifying the measure of match between two blocks under comparison. In this paper we propose a new BMA that measures the match by using the block sum of pixel values, a quantity that can be easily computed from an integral frame. Experimental results show that the proposed BMA can reduce notably the computational load and achieve compression performance very close to that of existing BMAs using conventional block-matching measures.
Collaborative multi-view metric learning for visual classification

Most of distance metric learning algorithms usually learn a single distance metric over the single-view data and cannot directly exploit multi-view data. In many visual classification applications, we have access to multi-view feature representations. To exploit more discriminative information for classification, it is desired to learn several distance metrics from multi-view data. To this aim, we propose a collaborative multi-view metric learning (CMML) method for visual classification. The proposed method jointly learns multiple distance metrics under which multiple feature representations are consistent across different views, i.e., the difference of the distance metrics learned in different views is enforced to be as small as possible. Experimental results on two visual classification tasks including face recognition and scene classification show the efficacy of the CMML method.
Adaptive binarization method for document image analysis

This paper proposes an adaptive binarization method, based on the criterion of maximizing local contrast, for document image analysis. The proposed method has overcome, to a large extent, the general problems of poor quality document images, such as non-uniform illumination, undesirable shadows and random noise. It was tested against a variety of challenging images, and the experimental results are presented to show the effectiveness and superiority of the proposed method.
Cost optimal video transcoding in media cloud: Insights from user viewing pattern

Video transcoding has been touted as an enabling technology to support growing media consumption over heterogenous devices. However, on-line transcoding could incur tremendous, if not prohibitive, cost in deploying or renting resources. In this research, we leverage an insight into the viewing pattern of video consumers to reduce the operating cost of video transcoding services. Specifically, it has been reported that viewers tend to terminate their session before the whole video is watched. As such, it is not cost-efficient for service providers to store or transcode all segments of the videos. Built upon this insight, we propose a partial transcoding scheme for content management in a media cloud to reduce the operating cost. Particularly, each content is split into multiple segments and stored in different files of varying playback rates. Some of the segments are stored in cache, resulting in storage cost; while some are transcoded in real-time in case of cache miss, resulting in computing cost. We aim to minimize the long-term operational cost by determining the number of segments for each playback rate to be cached or transcoded in real-time. We formulate this partial transcoding scheme as a constrained integer optimization problem. Leveraging Lagrangian relaxation and a subgradient method, we obtain the approximate solution to the integer program. Numerical results indicate that our proposed partial transcoding scheme can save more than 30% of operational cost, compared with a brute-force scheme of caching all the segments.
Efficient H.263 to H.264/AVC video transcoding using enhanced rate control

A new video coding standard H.264/AVC has been recently developed and standardized, which represents a number of advances in standard video coding technology and is expected to replace the existing standards such as H.263 and MPEG-1/2/4. In this paper, we propose an enhanced rate control method for H.263 to H.264/AVC transcoding. Specifically, we develop a model to approximate the relationship between the total number of coding bits and quantization step sizes between the preceded and transcoded videos for selecting quantization parameters at the sequence and frame levels. In addition, a new frame-layer bit allocation scheme based on the frame complexity obtained from the preceded video is employed in order to achieve more accurate bit rate and constant visual quality. The experimental results show the accuracy of the model and the effectiveness of the proposed method.
Hybrid Saliency Detection for Images

Saliency information interpreted from the visual stimuli can predict the attentional behaviour of human perception, thus playing a key role in visual signal processing. In this letter, we present a hybrid saliency detection method for images by which we automatically predict the saliency regions based on low-level and high-level cues. Unlike existing bottom-up and top-down attentional methods, we consider the high-level cue imposed by the photographer. Based on this assumption, we estimate the defocus map of the image and integrate it with other low-level features based on the Bayesian framework. We compare our algorithm to several state-of-the-art saliency detection methods based on the well-known 1000 image EPFL database, and demonstrate the superior performance of our proposed algorithm.
Image retrieval with SVM active learning embedding Euclidean search

Image retrieval with relevance feedback suffers from the small sample problem. Recently, SVM active learning has been proposed to tackle this problem, showing promising results. However, a small but sufficient number of initially labelled samples are still required to ensure subsequent efficient active learning and good retrieval performance. In the existing method, the user is asked to label more images before active learning starts. In this paper, a method of embedding Euclidean search into SVM active learning is proposed. With the help of Euclidean search, the adverse effect on retrieval performance due to lack of initially labelled samples can be reduced. Experimental results demonstrate the improvement by the proposed method, especially when the number of initially labelled samples is small.
Methods and needs for transcoding MPEG-4 fine granularity scalability video

We discuss the methods and needs for transcoding MPEG-4 fine granularity scalability (FGS) video. FGS is a scalable video-coding technique recently adopted by the MPEG-4 compression standard. The main feature of FGS coding is that a part of the video bit stream, namely the enhancement layer, can be truncated at any desirable number of bits to support continuous quality/bit-rate scalability. However, not every decoder would support FGS functions and the coding efficiency of FGS video is known to be inferior to that of non-scalable video. This paper proposes methods for transcoding FGS video and restoring, as much as possible, the coding efficiency that is lost in FGS coding. In particular, it is shown that there are cases where extra gain of video quality is possible by exploiting the FGS enhancement layer in the transcoding process. To some extend, the proposed transcoding methods can support spatial scalability that is lacking in FGS coding. Experiments have been conducted and the results are reported to demonstrate the advantages of the proposed transcoding schemes for FGS video.
Collaborative reconstruction-based manifold-manifold distance for face recognition with image sets

In this paper, we propose a new collaborative reconstruction-based manifold-manifold distance (CRMMD) method for face recognition with image sets, where each gallery and probe sample is a set of face images captured from varying poses, illuminations and expressions. Given each face image set, we first model it as a nonlinear manifold and then the recognition task is converted as a manifold-manifold matching problem. For each manifold, we divide it into several clusters and describe each cluster by using a local model. Then, we use the local models from each gallery manifold to collaboratively reconstruct each local model of the testing manifold and the minimal reconstruction error is used for classification. Experimental results on three widely used face datasets are presented to show the effectiveness of the proposed method.
Discriminative Multimanifold Analysis for Face Recognition from a Single Training Sample per Person

Conventional appearance-based face recognition methods usually assume that there are multiple samples per person (MSPP) available for discriminative feature extraction during the training phase. In many practical face recognition applications such as law enhancement, e-passport, and ID card identification, this assumption, however, may not hold as there is only a single sample per person (SSPP) enrolled or recorded in these systems. Many popular face recognition methods fail to work well in this scenario because there are not enough samples for discriminant learning. To address this problem, we propose in this paper a novel discriminative multimanifold analysis (DMMA) method by learning discriminative features from image patches. First, we partition each enrolled face image into several nonoverlapping patches to form an image set for each sample per person. Then, we formulate the SSPP face recognition as a manifold-manifold matching problem and learn multiple DMMA feature spaces to maximize the manifold margins of different persons. Finally, we present a reconstruction-based manifold-manifold distance to identify the unlabeled subjects. Experimental results on three widely used face databases are presented to demonstrate the efficacy of the proposed approach.
DelugeNets: Deep Networks with Efficient and Flexible Cross-Layer Information Inflows

Deluge Networks (DelugeNets) are deep neural networks which efficiently facilitate massive cross-layer information inflows from preceding layers to succeeding layers. The connections between layers in DelugeNets are established through cross-layer depthwise convolutional layers with learnable filters, acting as a flexible yet efficient selection mechanism. DelugeNets can propagate information across many layers with greater flexibility and utilize network parameters more effectively compared to ResNets, whilst being more efficient than DenseNets. Remarkably, a DelugeNet model with just model complexity of 4.31 GigaFLOPs and 20.2M network parameters, achieve classification errors of 3.76% and 19.02% on CIFAR-10 and CIFAR-100 dataset respectively. Moreover, DelugeNet-122 performs competitively to ResNet-200 on ImageNet dataset, despite costing merely half of the computations needed by the latter.
Joint Subspace Learning for View-Invariant Gait Recognition

We propose in this paper a novel joint subspace learning (JSL) method for view-invariant gait recognition. Inspired by the finding that if a 3-D object can be well represented by the weighted sum of a sufficiently small number of prototypes in the same view, then the representation coefficients are generally consistent across different views, we propose to use these coefficients as view-invariant features for gait recognition. Firstly, we conduct JSL to obtain the prototypes of different views. Then, we represent each sample in both the gallery set and probe set acquired from different views as a linear combination of these prototypes in the corresponding views, and extract the coefficients for feature representation. Lastly, we perform recognition by using a simple nearest neighbor rule. Experimental results on the widely used CASIA-B gait database demonstrate the effectiveness of the proposed method.
Color filter array demosaicking: new method and performance measures

Single-sensor digital cameras capture imagery by covering the sensor surface with a color filter array (CFA) such that each sensor pixel only samples one of three primary color values. To render a full-color image, an interpolation process, commonly referred to as CFA demosaicking, is required to estimate the other two missing color values at each pixel. In this paper, we present two contributions to the CFA demosaicking: a new and improved CFA demosaicking method for producing high quality color images and new image measures for quantifying the performance of demosaicking methods. The proposed demosaicking method consists of two successive steps: an interpolation step that estimates missing color values by exploiting spatial and spectral correlations among neighboring pixels, and a post-processing step that suppresses noticeable demosaicking artifacts by adaptive median filtering. Moreover, in recognition of the limitations of current image measures, we propose two types of image measures to quantify the performance of different demosaicking methods; the first type evaluates the fidelity of demosaicked images by computing the peak signal-to-noise ratio and CIELAB DeltaE(*)(ab) for edge and smooth regions separately, and the second type accounts for one major demosaicking artifact-zipper effect. We gauge the proposed demosaicking method and image measures using several existing methods as benchmarks, and demonstrate their efficacy using a variety of test images.
Video denoising using vector estimation of wavelet coefficients

Wavelet-based image denoising can be extended to a video by applying it to each video frame independently. The denoising performance can be improved by exploiting inter-frame correlations, for example, using appropriate temporal filtering. However, fixed temporal filters might not perform sufficiently well due to their inability to cope with the variability of inter-frame correlations across the video. While many adaptive temporal filtering approaches for denoising in spatial domain have been proposed, they do not straightforwardly extend to wavelet-based denoising. We propose a vector extension of popular hidden Markov tree modeling that flexibly exploits the color and frame dependency of wavelet coefficients. Experimental results confirm that the vector estimator of wavelet coefficients yields denoising performance superior to that of existing solutions, both in CPSNR and visual quality sense
Mobile media communication, processing, and analysis: A review of recent advances

In this paper, we review recent advances in mobile media communication, processing, and analysis. To identify the opportunities and challenges in fast growing mobile media computing, we discuss several emerging topics including mobile visual search, retargeting, mobile video streaming, and cloud based mobile media computing. According to the infrastructure of mobile devices vs. servers, we come up with essential concerns in mobile media computing such as wireless bandwidth consumption, mobile energy saving, media adaptation for better quality of services, the computational load shift from mobiles to servers, etc. With booming mobile Apps on diverse media consumption, it is envisioned that mobile media research and development is bringing about significant achievements in traditional topics of communication, processing, and analytics.
Cross-scene abnormal event detection

This paper presents an cross-scene abnormal event detection method by adopting Bag of Words (BoW) model with Spatial Pyramid Matching Kernel (SPM) cooperating with SIFT features and a SVM classifier. Different from existing abnormal event detection methods where abnormal events happened in a well-learned scene are considered and detected, we aim to detect concerned events in public where scenes can be unlearned before. Our method is motivated by the fact that the pattern of the notable events are similar and the learned models should be transferable to examine the events in other unlearned public scenes. To learn the patterns for an abnormal event, we divide the proposed method into two steps: feature coding and spatial pooling. For the feature coding step, the codebook is generated and the feature is quantized based on small patches. For the spatial pooling step, the patches are concatenating to exploit the spatial information of local regions. The intersection kernel is used to integrate with a SVM classifier. Experimental results on two benchmark databases demonstrate the efficacy of our proposed approach.
Efficient all-zero block detection algorithm for H.264 integer transform

A maximum magnitude probability (MMP) model is introduced in this paper to study contributions of individual DCT coefficient to the all-zero block (AZB) detection. Based on the MMP model, an efficient AZB detection algorithm is proposed to reduce the overall computation complexity. The new method is further combined with a recently published multi-threshold method  to achieve an adaptive detection of all-zero DCT sub-blocks. Experiment results show that the proposed algorithm can achieve a remarkable reduction of computational complexity from the original H.264 encoder with minor PSNR loss. Some comparison with similar lossy encoding schemes is also presented.
A robust sequential approach for the detection of defective pixels in an image sensor

Large image sensors usually contain same defects. Defects are pixels with abnormal photo-responsibility. As a result they often generate outputs different from their adjacent pixel outputs and seriously degrade the visual quality of the captured images. However, it is not economically feasible to produce sensors with no defects for rendering images. A limited number of defects are usually allowed in an image sensor as long as the defective outputs can be corrected with post signal processing techniques. In this paper we present a robust sequential approach for detecting sensor defects from a sequence of images captured by the sensor. With this approach no extra non-volatile memory is required in the sensor device to store the locations of sensor defects. In addition, the detection and correction of image defective outputs can be performed efficiently in a computer host. Experimental results of this approach are reported in the paper.
Binocular Just-Noticeable-Difference Model for Stereoscopic Images

Conventional 2-D Just-Noticeable-Difference (JND) models measure the perceptible distortion of visual signal based on monocular vision properties by presenting a single image for both eyes. However, they are not applicable for stereoscopic displays in which a pair of stereoscopic images is presented to a viewer's left and right eyes, respectively. Some unique binocular vision properties, e.g., binocular combination and rivalry, need to be considered in the development of a JND model for stereoscopic images. In this letter, we propose a binocular JND (BJND) model based on psychophysical experiments which are conducted to model the basic binocular vision properties in response to asymmetric noises in a pair of stereoscopic images. The first experiment exploits the joint visibility thresholds according to the luminance masking effect and the binocular combination of noises. The second experiment examines the reduction of visual sensitivity in binocular vision due to the contrast masking effect. Based on these experiments, the developed BJND model measures the perceptible distortion of binocular vision for stereoscopic images. Subjective evaluations on stereoscopic images validate of the proposed BJND model.
Generalized subspace distance for set-to-set image classification

Recent research in visual data classification often involves image sets and the measurement of dissimilarity between each pair of them. An effective solution is to model each image set using a subspace and compute the distance between these two subspaces as the dissimilarity between the sets. Several subspace similarity measures have been proposed in the literature. However, their relationships have not been well explored and most of them do not fully utilize the different importance of individual bases of each subspace. To consolidate this family of subspace-based measures, we propose a generalized subspace distance (GSD) framework and show that most existing subspace similarity measures can be considered as its special cases. To better utilize the different importance, we further propose a new fractional order weighted subspace distance (FOWSD) method within the GSD framework, by assigning different weights to the bases of each subspace and thus characterizing their different importance in similarity measurement. Experimental results on two image classification tasks including face recognition and object recognition are presented to show the effectiveness of the proposed method.