nils-herrmann/clean_scirepeval huggingface dataset:
Cost-sensitive subspace learning for face recognition

Conventional subspace learning-based face recognition aims to attain low recognition errors and assumes same loss from all misclassifications. In many real-world face recognition applications, however, this assumption may not hold as different misclassifications could lead to different losses. For example, it may cause inconvenience to a gallery person who is mis-recognized as an impostor and not allowed to enter the room by a face recognition-based door-locker, but it could result in a serious loss or damage if an impostor is mis-recognized as a gallery person and allowed to enter the room. Motivated by this concern, we propose in this paper a cost-sensitive subspace learning approach for face recognition. Our approach incorporates a cost matrix, which specifies the different costs associated with misclassifications of subjects, into three popular subspace learning algorithms and devise the corresponding cost-sensitive methods, namely, cost-sensitive principal component analysis (CSPCA), cost-sensitive linear discriminant analysis (CSLDA), and cost-sensitive locality preserving projections (CSLPP), to achieve a minimum overall recognition loss by performing recognition in the low-dimensional subspaces derived. Experimental results are presented to demonstrate the efficacy of the proposed approach.
Error Inhomogeneity in Wavelet-Based Compression

Despite the popularity of wavelet-based image compression, its shortcoming of having error inhomogeneity (EI), namely the error that is different for even and odd pixel location, has not been previously analyzed and formally addressed. The difference can be substantial, up to 3.4-dB peak signal-to-noise ratio (PSNR) for some images and compression ratios. In this paper, we show that the EI is caused by asymmetrical filtering of quantization errors after the upsampling step in wavelet synthesis process. Nonuniformity and correlation of quantization errors can also contribute to this EI, albeit to a smaller degree. In addition to explaining the source of EI, the model we developed in this paper also allows predicting its amount for a given wavelet. Furthermore, we show how to redesign wavelet filters to reduce this EI at a cost of a small reduction in the overall PSNR performance. For applications that are sensitive to PSNR degradation, we also show how to design wavelet filters that can gradually tradeoff PSNR performance for reduced EI.
Quickest change detection for health-care video surveillance

Detecting changes in video scenes is of fundamental importance for various video surveillance tasks. Of particular interest are abnormal changes of foreground human behaviors/activities that could pose damages or dangers to human properties and lives. In this paper, we propose a unified sequential approach to detecting, as soon as possible, human fall incidents for health-care purpose. Specifically, aspect ratio of human body is extracted as the representative feature, based on which an event-inference module parses observed feature sequences for possible falling behavioral signs. Experimental results are reported to show the efficacy of the proposed approach
Deep Video Hashing

In this work, we propose a deep video hashing (DVH) method for scalable video search. Unlike most existing video hashing methods that first extract features for each single frame and then use conventional image hashing techniques, our DVH learns binary codes for the entire video with a deep learning framework so that both the temporal and discriminative information can be well exploited. Specifically, we fuse the temporal information across different frames within each video to learn the feature representation under two criteria: the distance between a feature pair obtained at the top layer is small if they are from the same class, and large if they are from different classes; and the quantization loss between the real-valued features and the binary codes is minimized. We exploit different deep architectures to utilize spatial-temporal information in different manners and compare them with single-frame-based deep models and state-of-the-art image hashing methods. Experimental results demonstrate the effectiveness of our proposed method.
Enhanced H.263 to H.264/AVG Video Transcoding with Adaptive Intra-mode Decision

An enhancement of the H.264/AVC transcoding is proposed in this paper. The drift problem in the transcoder is addressed by including spatial intra prediction in image reconstruction. A two-step adaptive intra-mode decision algorithm based on angle information and SATD ranking is introduced to reduce the complexity. Experiment results show that the proposed enhancement significantly increases the speed of intra mode-decision while maintaining a similar PSNR and a slight increase of rate
Fusing shape and texture information for facial age estimation

This paper presents a new human age estimation method by using multiple feature fusion via facial image analysis. Motivated by the fact that both shape and texture information of facial images can provide complementary information in characterizing human age, we propose fusing these two sources of information at the feature level by using canonical correlation analysis (CCA), a powerful and well-known tool that is well suitable for relating two sets of measurements, for enhanced facial age estimation. Then, we learn a multiple linear regression function to uncover the relation of the fused features and the ground-truth age values for age prediction. Experimental results are presented to demonstrate the efficacy of the proposed method.
QoE analysis for scalable video adaptation

Quality of Experience (QoE) serves as a key service goal in video applications. In this paper, we study the QoE issue in scalable video adaptation by constructing a subjective video quality assessment database based on the full scalability of SVC. We derive the optimal scalability adaptation track for individual video and further summarize common scalability adaptation tracks for grouped videos. The common track provides useful guidelines on how to adapt scalable video based on their content characteristics. A rate-QoE model is proposed accordingly for the SVC adaptation. Experimental analyses show that the novel QoE-aware scalability adaptation scheme significantly outperforms the existing ones.
Activity-based human identification

We investigate in this paper the problem of activity-based human identification. Different from most existing gait recognition methods where only human walking activity is considered and utilized for person identification, we aim to identify people from various activities such as eating, jumping, and weaving. For each video clip, we first extract binary human body masks by using background substraction, followed by computing the average energy image (AEI) features to represent each video clip. Then, a mapping is learned by applying an adaptive discriminant analysis (ADA) method to project AEI features into a low-dimensional subspace, such that the intra-class (activities performed by the same person) variations are minimized and the interclass (activities performed by different persons) are maximized, simultaneously. Moreover, interclass samples with large similarity difference are deemphasized and those with small difference are emphasized, such that more discriminative information can be used for recognition. Experimental results on three publicly available databases show the efficacy of our proposed approach.
Throughput adaptation for scalable video multicast in wireless networks

We present a novel method of cross-layer adaptation for scalable video coding (SVC) extension of H.264/AVC multicast in wireless network. Transmission strategy for the single-hop multicast adapts according to wireless channel condition and SVC layer. Efficient transmit power and unequal FEC bit budget allocations are proposed while existing adaptive technique for throughput optimization is applied. We formulated a convex optimization problem to obtain the optimal time-division scheduler for all multicasts such that the wireless network delivers all SVC layers in noisy wireless channels and severe network capacity constraints. The cross-layer adaptation technique is verified by simulation results.
A unified transcoding approach to fast forward and reverse playback of compressed video

We propose a unified approach for realizing fast-forward and fast-reverse playback of a pre-coded video by generating a new and lower-bit rate video through video transcoding. To reduce the computational complexity, we develop fast algorithms to estimate the motion vectors required for transcoding the video. To accommodate changes due to frame-skipping and to sustain satisfactory coding efficiency, we adaptively alter the group-of-pictures structure of the transcoded video to suit for different playback speeds. Subjective tests are conducted to assess the perceived quality of video played back at various fast speeds. Experimental results and complexity analysis are presented in this paper to show the efficacy of the proposed approach.
Swimmer motion analysis with application to drowning detection

We present a novel swimmer motion analysis approach for detecting possible drowning incidents in a swimming pool. The drowning incidents are detected by examining the sequence of motion and shape features extracted from swimmers in the pools. Incorporated with the knowledge of professional practice in water crisis recognition, two event-inference modules have been developed: one evaluates the condition of a swimmer using a set of reasoning rules and another relies on hidden Markov models (HMMs) to recognize drowning behavioral signs. Both modules have been applied to a number of video clips of simulated drowning incidents and promising results have been obtained.
Regularized Locality Preserving Projections and Its Extensions for Face Recognition

We propose in this paper a parametric regularized locality preserving projections (LPP) method for face recognition. Our objective is to regulate the LPP space in a parametric manner and extract useful discriminant information from the whole feature space rather than a reduced projection subspace of principal component analysis. This results in better locality preserving power and higher recognition accuracy than the original LPP method. Moreover, the proposed regularization method can easily be extended to other manifold learning algorithms and to effectively address the small sample size problem. Experimental results on two widely used face databases demonstrate the efficacy of the proposed method.
Fast motion vector and bitrate re-estimation for arbitrary downsizing video transcoding

In video transcoding technique, many fast methods have been proposed to estimate the motion vectors required for downsizing a pre-coded video by an integer factor. In this paper we examine and compare several fast video transcoding methods for downsizing a pre-coded video by an arbitrary factor. Methods which out-perform others are identified and discussed. We also propose a new method to model the relation between bitrate and frame size under low bitrate condition. With this new model, we can re-estimate the bitrate according to a required frame size or select a suitable frame size for a given target bitrate, while maintaining good video quality. Experimental results are presented to show the performance of the proposed fast motion vector and bitrate re-estimation methods.
Improved color filter array demosaicking by accurate luminance estimation

Luminance information plays an important role in dictating the quality of a color image, and some existing color filter array (CFA) demosaicking methods achieve superiority by first reconstructing a satisfactory luminance plane. It is however difficult to accurately estimate the luminance from CFA samples since their color spectra are generally aliased. Extending a state-of-the-art luminance-based demosaicking scheme, in this paper we propose an improved demosaicking method using an efficient filter to estimate the luminance at green pixels and employing an effective edge-adaptive interpolation scheme to obtain the luminance at red and blue pixels. Experimental results demonstrate that not only is the proposed method less complex, it also performs noticeably better, both visually and in terms of peak signal-to-noise ratio, comparing with several recent methods.
Two-directional two-dimensional discriminant locality preserving projections for image recognition

We propose in this paper an improved manifold learning method called two-directional two-dimensional discriminant locality preserving projections, (2D)2-DLPP, for efficient image recognition. As the existing method of two-dimensional discriminant locality preserving projections (2D-DLPP) mainly relies upon the local structure information in the rows of images, we first derive an alternative 2D-DLPP algorithm that makes use of the information in the columns. Exploiting the local structure and discriminant information in both the rows and the columns, we develop the (2D)2-DLPP method for efficient image feature extraction and dimensionality reduction. Experimental results on two benchmark image datasets show the effectiveness of the proposed method.
Learning a cross-modal hashing network for multimedia search

In this paper, we propose a cross-modal hashing network (CMHN) method to learn compact binary codes for cross-modality multimedia search. Unlike most existing cross-modal hashing methods which learn a single pair of projections to map each example into a binary vector, we design a deep neural network to learn multiple pairs of hierarchical non-linear transformations, under which the nonlinear characteristics of samples can be well exploited and the modality gap is well reduced. Our model is trained under an iterative optimization procedure which learns a (1) unified binary code discretely and discriminatively through a classification-based hinge-loss criterion, and (2) cross-modal hashing network, one deep network for each modality, through minimizing the quantization loss between real-valued neural code and binary code, and maximizing the variance of the learned neural codes. Experimental results on two benchmark datasets show the efficacy of the proposed approach.
Co-Learned Multi-View Spectral Clustering for Face Recognition Based on Image Sets

Different from the existing approaches that usually utilize single view information of image sets to recognize persons, multi-view information of image sets is exploited in this paper, where a novel method called Co-Learned Multi-View Spectral Clustering (CMSC) is proposed to recognize faces based on image sets. In order to make sure that a data point under different views is assigned to the same cluster, we propose an objective function that optimizes the approximations of the cluster indicator vectors for each view and meanwhile maximizes the correlations among different views. Instead of using an iterative method, we relax the constraints such that the objective function can be solved immediately. Experiments are conducted to demonstrate the efficiency and accuracy of the proposed CMSC method.
Effective shot boundary classification using video spatial-temporal information

We propose in this paper a new video shot boundary classification (SBC) method. First we present a method to extract features, which exploits the spatial-temporal information of video frames in a sliding window centered at the detected shot boundary. Then feature projection and classification methods such as principal component analysis (PCA) and linear discriminant analysis (LDA) combined with k-nearest neighbor classification, and hierarchical discriminant regression (HDR) are proposed to classify different types of shot boundaries. Experimental results have shown the effectiveness of the proposed SBC method.
Sharable and Individual Multi-View Metric Learning

This paper presents a sharable and individual multi-view metric learning (MvML) approach for visual recognition. Unlike conventional metric leaning methods which learn a distance metric on either a single type of feature representation or a concatenated representation of multiple types of features, the proposed MvML jointly learns an optimal combination of multiple distance metrics on multi-view representations, where not only it learns an individual distance metric for each view to retain its specific property but also a shared representation for different views in a unified latent subspace to preserve the common properties. The objective function of the MvML is formulated in the large margin learning framework via pairwise constraints, under which the distance of each similar pair is smaller than that of each dissimilar pair by a margin. Moreover, to exploit the nonlinear structure of data points, we extend MvML to a sharable and individual multi-view deep metric learning (MvDML) method by utilizing the neural network architecture to seek multiple nonlinear transformations. Experimental results on face verification, kinship verification, and person re-identification show the effectiveness of the proposed sharable and individual multi-view metric learning methods.
Efficient block-matching motion estimation based on Integral frame attributes

Block-based motion estimation is widely used in video compression for exploiting video temporal redundancy. Although effective, the process is arguably the most computationally intensive part of a typical video encoder. To speed up the process, a large number of fast block-matching algorithms (BMAs) have been proposed for motion estimation by limiting the number of search locations or simplifying the measure of match between the two blocks under comparison. In this paper we propose a new BMA that measures the match by using such features as block sum and block variance, which can be easily computed using integral frame attributes. Experimental results show that the proposed BMA can reduce the computational complexity notably and achieve compression performance very close to that of existing BMAs using conventional block-matching measures.
Shot Boundary Classification and Refinement Using Inter-Frame Similarity Patterns

In content-based video analysis, shot boundary detection (SBD) is a common first step to segment a video into fundamental units of shots, each composing consecutive frames filmed with a single camera act. Many methods have been proposed in the literature for SBD, and many require the use of different models to determine the types of various shot boundaries. In this paper, we propose a new and effective shot boundary classification (SBC) and refinement method by analyzing the inter-frame similarity patterns around the detected shot boundaries. In particular, the proposed method can classify the detected shot boundaries into five types-shot cuts, wipes, dissolves, fades, and non-boundaries-using a more generic and accurate manner, and the non-boundaries obtained in the classification can be used to refine the SBD results by eliminating the false detections to attain a better detection performance, as evidenced by the experimental results presented
An Efficient and Effective Color Filter Array Demosaicking Method

To reduce the cost and size, most digital still cameras (DSCs) capture only one color value at each pixel, and the results - color filter array samples - are then interpolated by a demosaicking method to construct a full-color image. Many advanced demosaicking methods have been proposed recently. However, the high complexity of these methods could prevent them from being used in DSCs. In this paper we propose an efficient and effective demosaicking method, which substitutes high-frequency component of color values in the spatial rather than frequency domain. We also propose a simple ternary, anisotropic interpolation scheme to obtain an initial full-color image required in the spatial-domain high-frequency substitution. Experimental results show that the proposed method can outperform recent state-of-the-art methods in terms of both PSNR performance and perceptual results, at the same time reducing the computational cost substantially.
A Survey on Visual Analytics of Social Media Data

The unprecedented availability of social media data offers substantial opportunities for data owners, system operators, solution providers, and end users to explore and understand social dynamics. However, the exponential growth in the volume, velocity, and variability of social media data prevents people from fully utilizing such data. Visual analytics, which is an emerging research direction, has received considerable attention in recent years. Many visual analytics methods have been proposed across disciplines to understand large-scale structured and unstructured social media data. This objective, however, also poses significant challenges for researchers to obtain a comprehensive picture of the area, understand research challenges, and develop new techniques. In this paper, we present a comprehensive survey to characterize this fast-growing area and summarize the state-of-the-art techniques for analyzing social media data. In particular, we classify existing techniques into two categories: gathering information and understanding user behaviors. We aim to provide a clear overview of the research area through the established taxonomy. We then explore the design space and identify the research trends. Finally, we discuss challenges and open questions for future studies.
View recognition of human gait sequences in videos

We investigate in this paper the problem of view recognition of human gait sequences in videos. To our knowledge, the problem has not been formally addressed in the literature. Recognizing the views of human gait sequences has a number of potential applications, including visual surveillance and view-invariant human gait recognition. Motivated by the fact that human gait sequences collected from two views with small differences are more easily mis-recognized than those with large differences, we propose a new adaptive discriminant analysis (ADA) method by imposing large penalties on interclass samples with small differences and small penalties on those samples with large differences simultaneously, such that the discriminating power of the extracted features can be boosted for view recognition. Experimental results are presented to demonstrate the efficacy of the proposed approach.
Image-to-Set Face Recognition Using Locality Repulsion Projections and Sparse Reconstruction-Based Similarity Measure

For many practical face recognition systems such as law enforcement, e-passport, and ID card identification, there is usually only a single sample per person (SSPP) enrolled in these systems, and many existing face recognition methods may fail to work well because there are not enough samples for discriminative feature extraction in this scenario. However, the probe samples of these face recognition systems are usually captured on the spot, and it is possible to collect multiple face images per person for on-location probing, which is potentially useful to improve the recognition performance. In this paper, we propose a method based on locality repulsion projections (LRP) and a sparse reconstruction-based similarity measure (SRSM) to address the problem of SSPP face recognition using multiple probe images. The LRP method is motivated by our observation that similar face images from different people may lie in a locality in the feature space and cause misclassifications. We design the method with the aim of separating the samples of different classes within a neighborhood through subspace projections for easier classification. To better characterize the similarity between each gallery face and the probe image set, we propose a SRSM method for assigning a label to each probe image set. Experimental results on five widely used face datasets are presented to demonstrate the effectiveness of the proposed approach.