nils-herrmann/clean_scirepeval huggingface dataset:
Efficient packet scheduling for scalable video delivery to mobile clients

Scalable Video Coding (SVC) provides an efficient solution for video adaptation to satisfy different requirements from heterogeneous mobile clients due to their display sizes and channel conditions. Based on the encoding structure and dependency relationship, SVC packets are entitled with different priorities in presenting quality of video sequences. In this paper, a functional model is derived to calculate packet priority index for multiple scalable video streaming to heterogeneous mobile clients. SVC packet layer ID information is utilized for packet prioritization which provides a fast and efficient implementation in packet scheduling. The accuracy and efficacy of the model are validated by comparisons with the maximum performance achieved from the distortion based prioritization. The generality is considered under different coding structures and the utility is convinced through comparisons with conventional streaming schemes in which different resolution videos are transmitted through IEEE 802.11e wireless networks.
Efficient video transcoding between H.263 and H.264/AVC standards

A new video coding standard H.264/AVC has been recently developed and standardized, which represents a number of advances in standard video coding technology in terms of both coding efficiency enhancement and flexibility, and is expected to replace the existing standards such as H.263 and MPEG-1/2/4. In this paper we investigate and present efficient format transcoding and downsized transcoding methods between H.263 and H.264/AVC standards. Specifically, we use a vector median filter to estimate the new motion vectors required for transcoded videos. In addition, we also propose a fast intra-prediction mode selection based on the coarse edge information obtained from integer-transform coefficients. Experimental results show that the proposed method can gain a significant reduction in the complexity without degrading much the video quality.
Face clustering in videos using constraint propagation

In this paper, we propose a novel approach to automatic detection and clustering of human faces presented in videos. In each video shot, continuously appearing human faces are firstly associated to form face sequences. Instead of matching the face sequences directly, we partition them into subsequences consisting of similar poses for the ease of comparison. Face subsequences can then be clustered by graph partitioning with the computed affinity matrix. Prior to that, however, a set of constraints need to be formulated so as to incorporate domain knowledge into the graph. Moreover, we propose a constraint propagation algorithm to fully exploit the space-level implications of these constraints. Experimental results demonstrate the effectiveness of our approach in identifying the main cast in movie clips.
Discriminative Deep Metric Learning for Face and Kinship Verification

This paper presents a new discriminative deep metric learning (DDML) method for face and kinship verification in wild conditions. While metric learning has achieved reasonably good performance in face and kinship verification, most existing metric learning methods aim to learn a single Mahalanobis distance metric to maximize the inter-class variations and minimize the intra-class variations, which cannot capture the nonlinear manifold where face images usually lie on. To address this, we propose a DDML method to train a deep neural network to learn a set of hierarchical nonlinear transformations to project face pairs into the same latent feature space, under which the distance of each positive pair is reduced and that of each negative pair is enlarged. To better use the commonality of multiple feature descriptors to make all the features more robust for face and kinship verification, we develop a discriminative deep multi-metric learning method to jointly learn multiple neural networks, under which the correlation of different features of each sample is maximized, and the distance of each positive pair is reduced and that of each negative pair is enlarged. Extensive experimental results show that our proposed methods achieve the acceptable results in both face and kinship verification.
Shot boundary detection using unsupervised clustering and hypothesis testing

We propose a shot boundary detection approach based on unsupervised scenelet clustering and hypothesis testing. We define a video scenelet as a short consecutive samples of frames of a video sequence. The approach makes use of a typical k-means clustering algorithm to group the scenelets. Based on the clustering result, hypothesis testing can be performed to identify the shot boundaries at each level with a different cluster number. Combined with a cluster validity analysis to decide a suitable number of clusters, promising results can be obtained for shot boundary detection.
Rapid estimation of camera motion from compressed video with application to video annotation

As digital video becomes more pervasive, efficient ways of searching and annotating video according to content will be increasingly important. Such tasks arise, for example, in the management of digital video libraries for content-based retrieval and browsing. We develop tools based on camera motion for analyzing and annotating a class of structured video using the low-level information available directly from MPEG-compressed video. In particular, we show that in certain structured settings, it is possible to obtain reliable estimates of camera motion by directly processing data easily obtained from the MPEG format. Working directly with the compressed video greatly reduces the processing time and enhances storage efficiency. As an illustration of this idea, we have developed a simple basketball annotation system which combines the low-level information extracted from an MPEG stream with the prior knowledge of basketball structure to provide high-level content analysis, annotation, and browsing for events such as wide-angle and close-up views, fast breaks, probable shots at the basket, etc. The methods used in this example should also be useful in the analysis of high-level content of structured video in other domains.
An effective post-refinement method for shot boundary detection

In content-based video analysis, shot boundary detection is a common first step to segment video data into fundamental units of shots, each composing consecutive frames filmed with a single camera act. Many methods have been proposed in the literature for detection of shot boundaries. In this paper, we propose a new and effective post-refinement method on the detected shot boundaries by performing sequential detection of abrupt change in two underlying distributions. Experimental results show that the proposed method can eliminate most false detections and also recover many missed detections from the original detected shot boundaries, attaining better detection performance.
Face recognition using an enhanced age simulation method

We propose in this paper an enhanced age simulation method for face recognition across age differences. Since the within-class variations caused by age differences are generally much larger than the between-class variations caused by different identities at similar ages, we propose an age simulation method to reduce the facial appearance difference of the same person across age variations. Since it's very difficult to obtain a sequence of facial images of the person at continuous ages, we first implement a filling process to enlarge our limited training database and classify it into different age groups. Then, we represent each age group by a mean feature vector. With these mean feature vectors, typical vectors for different ages are generated. Lastly, we synthesize a new virtual facial image at a target age by combining the training samples and typical feature vectors. Experimental results are presented to demonstrate the effectiveness of the proposed method.
Graph-based sparse coding and embedding for activity-based human identification

In this paper, we propose a new graph-based sparse coding and embedding (GSCE) method for activity-based human identification. Different from human activity recognition which recognizes different types of human activities such as walking, running, eating, and drinking, in this study, we aim to identify persons from his/her activities. To our best knowledge, this problem has been seldom investigated in the literature. Given a training set of video clips, we first extract human body mask in each frame and learn a codebook to quantize these masks into a histogram feature by using a graph-based sparse coding technique to better preserve the similarity information of different frames within a same video clip. Moreover, we also learn a mapping to project each frame into a low-dimensional subspace to speed up the quantization procedure, such that more discriminative information can be further exploited for classification. Experimental results on three databases are presented to show the efficacy of the proposed method.
Deep Metric Learning for Visual Tracking

In this paper, we propose a deep metric learning (DML) approach for robust visual tracking under the particle filter framework. Unlike most existing appearance-based visual trackers, which use hand-crafted similarity metrics, our DML tracker learns a nonlinear distance metric to classify the target object and background regions using a feed-forward neural network architecture. Since there are usually large variations in visual objects caused by varying deformations, illuminations, occlusions, motions, rotations, scales, and cluttered backgrounds, conventional linear similarity metrics cannot work well in such scenarios. To address this, our proposed DML tracker first learns a set of hierarchical nonlinear transformations in the feed-forward neural network to project both the template and particles into the same feature space where the intra-class variations of positive training pairs are minimized and the interclass variations of negative training pairs are maximized simultaneously. Then, the candidate that is most similar to the template in the learned deep network is identified as the true target. Experiments on the benchmark data set including 51 challenging videos show that our DML tracker achieves a very competitive performance with the state-of-the-art trackers.
A Doubly Weighted Approach for Appearance-Based Subspace Learning Methods

We propose in this paper a doubly weighted subspace learning approach for face representation and recognition. Motivated by the fact that some face samples and parts are more effectual in characterizing and recognizing faces, we construct two weighting matrices based on pairwise similarity of face samples within a same class and discriminant score of each pixel within a face sample to duly emphasize both the between-sample and within-sample features. We then incorporate these two weighting matrices into three popular subspace learning methods, namely principal component analysis, linear discriminant analysis, and nonnegative matrix factorization, to obtain the discriminative features of faces for recognition. Moreover, the proposed doubly weighted technique can be readily extended to other newly proposed subspace learning algorithms to improve their performance. Experimental results show that the proposed approach can effectively enhance the discriminant power of the extracted face features and outperform existing, nonweighted subspace learning algorithms. The performance gain is even more apparent for cases with imbalanced training samples.
Query-Adaptive Small Object Search Using Object Proposals and Shape-Aware Descriptors

While there has been a significant amount of work on object search and image retrieval, the focus has primarily been on establishing effective models for the whole images, scenes, and objects occupying a large portion of an image. In this paper, we propose to leverage object proposals to identify small and smooth-structured objects in a large image database. Unlike popular methods exploring a coarse image-level pairwise similarity, the search is designed to exploit the similarity measures at the proposal level. An effective graph-based query expansion strategy is designed to assess each of these better matched proposals against all its neighbors within the same image for a precise localization. Combined with a shape-aware feature descriptor EdgeBoW, a set of more insightful edge-weights and node-utility measures, the proposed search strategy can handle varying view angles, illumination conditions, deformation, and occlusion efficiently. Experiments performed on a number of other benchmark datasets show the powerful and superior generalization ability of this single integrated framework in dealing with both clutter-intensive real-life images and poor-quality binary document images at equal dexterity.
Adaptive color filter array demosaicking using directional spatial and spectral correlations

Single-sensor digital cameras capture imagery by covering the sensor surface with a color filter array (CFA) such that each sensor pixel only samples one of three primary color values. To render a full-color image, an interpolation process commonly referred to as CFA demosaicking is required to estimate the other two missing color values at each pixel. In this paper, we present a new and improved CFA demosaicking method for producing high quality color images from CFA samples. The proposed method consists of two successive steps: an interpolation step estimating the missing color values by exploiting spatial and spectral correlations among neighboring pixels in different directions, and a post-processing step suppressing noticeable demosaicking artifacts by using adaptive median filtering. The superiority of the proposed method is demonstrated by comparing it against the other well-known or well-performed methods using a variety of test images.
Combined use of spatial and spectral correlations for enhanced color filter array demosaicking

Single-sensor digital still cameras capture imagery with a color filter array in a way that each sensor pixel samples only one of the primary color values. To render a full-color image from CFA samples, the missing color values need to be interpolated from the neighboring samples. This process is commonly referred to as demosaicking. In this paper, we present two main contributions to demosaicking. First, we stress the importance and necessity of well exploiting both image spatial and spectral correlations, and characterize the demosaicking artifacts due to inadequate use of either correlation. Second, based on the insights gained from our study, we propose several effective schemes to enhance two existing state-of-the-art demosaicking methods. Experimental results demonstrate the improvement of the enhanced methods, both visually and quantitatively.
Detection and Removal of Rainboweffect Artifacts

Due to the imperfect separation of luminance and chroma signals in receiver's demodulation, composite video signals suffer from rainbow effect artifacts, which present themselves as interlaced color stripes in regions of high luminance frequency and high luminance intensity. In this paper, we derive the formulas governing the rainbow effects. Based on these formulas, we propose a novel method to detect and remove these annoying artifacts. Experimental results on both captured video frames and simulated frames show that our method can remove the rainbow-effect artifacts effectively and improve the image quality notably.
Efficient video motion estimation using dual-cross search algorithms

We present in this paper two new and efficient block-matching algorithms (BMAs) for fast motion estimation which is commonly used in motion-compensated video compression. The proposed algorithms are an improvement on existing BMAs and consist of three effective steps: (1) initial search center prediction; (2) early search termination; and (3) dual-cross search. Extensive simulation results and comparative analysis have shown that the proposed algorithms outperform conventional algorithms, like the three-step search, orthogonal search and diamond search, as well as the newly proposed algorithms, like the hexagonal search and adaptive rood pattern search, in terms of both peak signal-to-noise ratio and the number of search points evaluated.
Recurrent Spatial Pyramid CNN for Optical Flow Estimation

Optical flow estimation plays an important role in many multimedia and computer vision tasks. Although great progress has been made in applying convolutional neural networks (CNNs) to estimate optical flow in recent works, it is still difficult for CNNs to generate optical flow with the desired effectiveness and efficiency. Compared to CNN-based methods, conventional variational methods normally perform to optimize an energy function and produce optical flow with more precise details. Inspired by the effectiveness of variational methods and deep CNNs, we propose a recurrent spatial pyramid (RecSPy) network for optical flow estimation. To deal with large displacements and to decrease the number of parameters, we formulate the spatial pyramid as a recurrent process, and adopt a CNN to refine optical flow at each spatial scale. Furthermore, to improve the results with more precise details, we propose an energy function that encodes structure and constancy constraints to help refine the optical flow at each spatial scale. The combination of the proposed RecSPy network and the proposed energy-based refinement enables our system to estimate optical flow effectively and efficiently. Experimental results on the benchmarks validate the effectiveness and efficiency of the proposed method.
Estimating relative objective quality among images compressed from the same original

This paper presents a new method for estimating the relative compression distortion among images encoded from the same original. Increasingly, many compressed versions of the same image are available over the Internet, but users may not have access to the original to measure their objective quality. We propose to cross-check each image against the quantization constraints of another version and obtain a conservative estimation of the difference in mean squared error (ΔMSE), indicating the relative objective quality between these images. Our proposed method is able to judge the relative quality correctly among compressed images with PSNR difference of as little as 0.5 dB, and it is applicable to images in mixed compression formats, such as JPEG and JPEG2000.
Efficient Video Clip Retrieval Using Index Structure

Retrieving similar video clips from large video database requires high query efficiency, precision and recall, which remains a challenging problem since the traditional query algorithms are inefficient and time-consuming. In this paper, we adopt the high-dimensional index structure vector-approximation file (VA-file) to organize the video database, and propose a new similarity measure which takes the temporal order among the video representations into account to improve the accuracy of query. Based on the VA-file and similarity measure, a new video clip retrieval algorithm is proposed in our method to achieve high query efficiency by using restricted sliding window to construct candidate video clips. Experimental results show that the proposed video retrieval method is efficient and effective
Frame Rate Up-Conversion Using Trilateral Filtering

Frame rate up-conversion (FRUC) can enhance the visual quality of low frame rate video presented on liquid crystal display. To minimize the difference between a reference block and an interpolated block, an effective FRUC algorithm partitions a large block into several sub-blocks of smaller size and estimates their motions. Motion estimation searches for the block which has the minimum difference (cost) with the processed block in terms of some block matching distortion and motion discontinuity. As convexity and convergence of the cost function are not guaranteed, the computational cost for such motion estimation is usually extensive or unpredictable. In our proposed FRUC method, the two predictions of a frame to be interpolated are generated through shifting its nearest neighbor frames in the previous and following directions with the motion vectors estimated between them. The initial interpolated frame and its pixel's reliability are subsequently estimated from these two predictions. We then apply a trilateral filter on the initial prediction to correct the unreliable pixels and to restore the missing pixels. Our proposed method not only reduces the computation for refining motion vectors, but also suppresses the interpolation noises and misregistration errors. We have conducted extensive experiments and the results show that the proposed algorithm outperforms the existing methods with better objective and subjective visual quality, and achieves about 3 dB on-average peak signal-to-noise ratio improvement.
Cost-sensitive subspace learning for human age estimation

This paper presents a novel cost-sensitive subspace learning approach for human age estimation using face and gait signatures. Motivated by the fact that mis-estimating the age information of a person from a facial image or gait sequence could lead to different errors, we propose in this paper two new cost-sensitive subspace learning methods for human age estimation. Our approach incorporates a cost matrix, which specifies the different error associated with mis-estimating each sample, into two popular subspace learning algorithms and devise the corresponding cost-sensitive methods, namely, cost-sensitive principal component analysis (CSPCA), and cost-sensitive locality preserving projections (CSLPP), to project high-dimensional face and gait samples into the low-dimensional subspaces derived. To uncover the relation of the projected features and the ground-truth age values, we learn a multiple linear regression function with a quadratic model for age estimation. Experimental results on the MORPH face database and the USF gait database are presented to demonstrate the efficacy of our proposed methods.
People monitoring using face recognition with observation constraints

We propose a people monitoring system to recognize people by probabilistic inference methods exploiting low-level facial feature and high-level domain knowledge. In particular, the faces of people in the view of a monitoring camera are first detected and modeled. Optimal recognition of people leaving and entering a closed-room is accomplished by exploiting temporal correlation and constraints among the observed face sequence. The optimality is achieved in the sense of maximizing a joint posterior probability of multiple observations. Experimental results of real and synthetic data suggest the efficacy of the proposed system.
Respiratory rate estimation via simultaneously tracking and segmentation

A novel strategy of simultaneously tracking and segmentation is proposed for human respiratory rate estimation from thermal infrared, which can be applicable to contact-free polygraphy, airport health screening and patient monitoring system. In this framework, by carefully selecting the adaptive observation model for the tracking template and taking the intensity variation pattern of breathing into account, we developed a tracker which is not only to track successfully the region of interest (ROI) which is subject to both physiological and abrupt motion change, but also to segment out the breathing flow region simultaneously, without increasing any large extra computation. The experimental result demonstrated both its stability in tracking and accuracy improvement in recovery of the temperature profile on breathing, compared with other trackers without involvement of segmentation. Besides estimating respiratory rate, the proposed algorithm can be also utilized for any applications regarding the automatic detection and tracking of pixels experiencing periodic intensity variation.