nils-herrmann/clean_scirepeval huggingface dataset:
A novel multi-scale spatial-color descriptor for content-based image retrieval

We propose in this paper a new approach to combine color and spatial features for content-based image retrieval applications. The color feature is derived using a fuzzy c-means clustering algorithm in a perceptually uniform color space. To further improve the image discrimination power, the spatial information of each image is measured by a multi-scale color histogram set derived from a set of hierarchical image partitions. The proposed method is capable of representing each image compactly and retrieving similar images effectively. Experimental results suggest that the proposed method can achieve consistently better performance compared to those using the popular conventional color histogram and generalized color histogram.
Frame-level quantization control for perceptual quality constrained H.264/AVC video coding

Achieving the desired visual quality is an important objective of video compression. In this paper, we present a frame-level quantization control approach for perceptual quality constrained video coding which aims to compress video at a certain perceptual quality level. We develop a general learning-based framework in which different video quality measures can be adopted. We model the rate-quality characteristic of the video using ν-support vector regression with a Gaussian radial basis function as the kernel. Such a rate-quality modeling is useful for practical video coding with perceptual quality constraint. Specifically, we show how to minimize the bit rate cost and satisfy the quality constraint by exploiting the relationship between the quantization and advanced video quality measure. The advantage of our proposed approach in practical H.264/AVC video coding is demonstrated through simulations and evidenced by favorable experimental results.
Ordinary preserving manifold analysis for human age estimation

We propose in this paper a novel ordinary preserving manifold analysis approach for human age estimation using face and gait features. Motivated by the fact that high-dimensional human facial images and gait sequences may reside in low-dimensional aging manifolds and two samples of face images or gait sequences with distinct age difference can provide different discriminative information for devising the low-dimensional aging manifold, we project the high-dimensional face or gait samples into a low-dimensional submanifold such that the samples with similar age values (i.e., smaller age difference) are projected to be as close as possible while those with dissimilar age values (i.e., larger age difference), as far as possible. To uncover the relation of the projected features and the ground-truth age values, we learn a multiple linear regression function with a quadratic model for age estimation. Experimental results on the MORPH face database and the USF gait database are presented to demonstrate the efficacy of our proposed approach.
A color histogram based people tracking system

A system using color histogram based recognition technique is presented for tracking of moving people. The proposed system aims to resolve the identity of each tracked person after an occlusion, which is a common problem encountered in tracking of multiple objects. The system recognizes and keeps track of moving people with the aid of their color histograms. To make the recognition more robust against change of background illumination, a shadow-removal scheme is designed to select only those pixels belonging to the tracked people for constructing their color histograms. Experimental results of real video data are reported to show the efficacy of the proposed system.
Combining Feature Context and Spatial Context for Image Pattern Discovery

Once an image is decomposed into a number of visual primitives, e.g., local interest points or salient image regions, it is of great interests to discover meaningful visual patterns from them. Conventional clustering (e.g., k-means) of visual primitives, however, usually ignores the spatial dependency among them, thus cannot discover the high-level visual patterns of complex spatial structure. To overcome this problem, we propose to consider both spatial and feature contexts among visual primitives for pattern discovery. By discovering both spatial co-occurrence patterns among visual primitives and feature co-occurrence patterns among different types of features, our method can better handle the ambiguities of visual primitives, by leveraging these co-occurrences. We formulate the problem as a regularized k-means clustering, and propose an iterative bottom-up/top-down self-learning procedure to gradually refine the result until it converges. The experiments of image text on discovery and image region clustering convince that combining spatial and feature contexts can significantly improve the pattern discovery results.
A novel study and analysis on segmental gait sequence recognition

This paper presents a novel study and analysis on two important problems in gait recognition: one is how to perform gait recognition with only segment of a complete gait cycle for test and the other is how static and dynamic information affect the recognition results in such situation. In conventional gait recognition research, the gait sequence of at least one cycle is usually needed to ensure the completeness of gait dynamics. And in this paper we will show that this is not a necessary condition and considerable recognition results could still be achieved with only part of the cycle based on appropriate similarity measure algorithms. Moreover, we will also show the different roles that static and dynamic information play in such situation through experimental study. To our knowledge, there is little work on these important problems, results from which can make gait recognition more applicable in frequently happened scenarios such as occlusion.
Enhanced shot boundary detection using video text information

Shot boundary detection is a pre-requisite process for content-based video indexing and retrieval, an application domain that has attracted much research and consumer attentions due to the explosive growth of video data available and accessible to users worldwide. Currently, a number of edge-based techniques have been proposed in the literature for detecting abrupt shot boundaries to avoid the influence of flashlights common in many video types, such as sports, news, entertainment, and interviews videos. However, these techniques are susceptible to miss and false detections of abrupt shot changes. Our study shows that one main reason for these errors is due to the presence of superimposed texts that are common in various video genres. To address this problem, we present in this paper an efficient method that utilizes the edge type - text-edge (edge in text region) or non-text-edge (edge in non-text region) - to reduce erroneous detection due to the sudden appearance and disappearance of superimposed text. Extensive experiments have been conducted and the results show that our proposed method, as compared with other existing methods, can obtain better performance in detecting abrupt shot boundaries and differentiating them from the effects of superimposed text.
Event detection using multimodal feature analysis

This paper presents an event detection framework using multimodal feature analysis. In this framework, multimodal features are extracted from video data and then analyzed to generate various mid-level concepts, such as video shot, face appearance and so on. Two schemes, the logistic regression and Bayesian belief network, are then employed to fuse the information obtained from multimodal feature analysis and detect the video events of interest. We aim to use this framework as a general template for event detection in different video domains. Experimental results on various test videos in different video domains suggest that the proposed event detection framework is promising.
Cross-Modal Deep Variational Hashing

In this paper, we propose a cross-modal deep variational hashing (CMDVH) method for cross-modality multimedia retrieval. Unlike existing cross-modal hashing methods which learn a single pair of projections to map each example as a binary vector, we design a couple of deep neural network to learn non-linear transformations from image-text input pairs, so that unified binary codes can be obtained. We then design the modality-specific neural networks in a probabilistic manner where we model a latent variable as close as possible from the inferred binary codes, which is approximated by a posterior distribution regularized by a known prior. Experimental results on three benchmark datasets show the efficacy of the proposed approach.
People Counting by Video Segmentation and Tracking

In this paper, we present a novel approach to counting number of people that pass the view of an overhead mounted camera. Moving people are first detected as blobs and represented by binary masks, based on which possible multi-person blobs are further segmented into isolated persons according to their areas and locations. Each single person is tracked through consecutive frames using a correlation-based algorithm and a state diagram is proposed to count people entering and leaving the scene. Experimental results show that our approach is able to achieve promising results
Reversing Demosaicking and Compression in Color Filter Array Image Processing: Performance Analysis and Modeling

In the conventional processing chain of single-sensor digital still cameras (DSCs), the images are captured with color filter arrays (CFAs) and the CFA samples are demosaicked into a full color image before compression. To avoid additional data redundancy created by the demosaicking process, an alternative processing chain has been proposed to move the compression process before the demosaicking. Recent empirical studies have shown that the alternative chain can outperform the conventional one in terms of image quality at low compression ratios. To provide a theoretically sound basis for such conclusion, we propose analytical models for the reconstruction errors of the two processing chains. The models developed confirm the results of existing empirical studies and provide better understanding of DSC processing chains. The modeling also allows performance predictions for more advanced compression and demosaicking methods, thus providing important cues for future development in this area
Cost-Sensitive Semi-Supervised Discriminant Analysis for Face Recognition

This paper presents a cost-sensitive semi-supervised discriminant analysis method for face recognition. While a number of semi-supervised dimensionality reduction algorithms have been proposed in the literature and successfully applied to face recognition in recent years, most of them aim to seek low-dimensional feature representations to achieve low classification errors and assume the same loss from all misclassifications in the feature representation/extraction phase. In many real-world face recognition applications, however, this assumption may not hold as different misclassifications could lead to different losses. For example, it may cause inconvenience to a gallery person who is misrecognized as an impostor and not allowed to enter the room by a face recognition-based door locker, but it could result in a serious loss or damage if an impostor is misrecognized as a gallery person and allowed to enter the room. Motivated by this concern, we propose in this paper a new method to learn a discriminative feature subspace by making use of both labeled and unlabeled samples and exploring different cost information of all the training samples simultaneously. Experimental results are presented to demonstrate the efficacy of the proposed method.
Video shot classification using human faces

People usually make up a lot of the information content in videos. The abilities to answer queries and facilitate browsing related to people in videos are crucial. In a single video sequence, a particular person may appear multiple number of times. We propose a scheme to automatically detect the repeated occurrences of the same people to enable fast people related searching. In particular, we propose a video shot classification scheme using human faces, regardless of scale and background. Video shots are classified by clustering facial features extracted from these shots. Potential applications include video indexing and browsing. Employing unsupervised clustering algorithms, this scheme requires no human intervention. Experimental results on a 4-minute news sequence show that it achieves encouraging results.
Fast object instance search in videos from one example

We present an efficient approach to search for and locate all occurrences of a specific object in large video volumes, given a single query example. Locations of object occurrences are returned as spatio-temporal trajectories in the 3D video volume. Despite much work on object instance search in image datasets, these methods locate the object independently in each image, therefore do not preserve the spatio-temporal consistency in consecutive video frames. This results in sub-optimal performance if directly applied to videos, as will be shown in our experiments. We propose to locate the object jointly across video frames using spatio-temporal search. The efficiency and effectiveness of the proposed approach is demonstrated on a consumer video dataset consisting of crawled YouTube videos and mobile captured consumer clips. Our method significantly improves the localized search accuracy over the baseline, which treats each frame independently. Moreover, it is able to find the top 100 object trajectories in the 5.5-hour dataset within 30 seconds.
Ordinary Preserving Manifold Analysis for Human Age and Head Pose Estimation

We propose in this paper an ordinary preserving manifold analysis approach for human age and head pose estimation. While a large number of manifold learning algorithms have been proposed in the literature and some of them have been successfully applied to age/pose estimation, the ordinary characteristics of the age/pose information of samples have not been fully exploited to learn the low-dimensional discriminative features for these estimation tasks. To address this, we propose an ordinary preserving manifold analysis approach to seek a low-dimensional subspace such that the samples with similar label values (i.e., small age/pose difference) are projected to be as close as possible and those with dissimilar label values (i.e., large age/pose difference) as far as possible, simultaneously. Subsequently, we learn a multiple linear regression model to uncover the relation of these low-dimensional features and the ground-truth values of samples for age/pose estimation. Experimental results on facial age estimation, gait-based human age estimation, and head pose estimation are presented to demonstrate the efficacy of our proposed approach.
Energy Minimization for Wireless Video Transmissions With Deadline and Reliability Constraints

In wireless video transmissions, encoded video frames are often large in data load and truncated into many transport packets (TPs) for reliable transmissions. These TPs are to be delivered before a deadline at certain reliability that depends on the importance of the video frame. High power and bitrate transmission schemes are often deployed to ensure low loss rate, but at the cost of substantial energy consumption. This paper addresses the energy-minimizing transmission policy for highly reliable transmissions of a group of TP with a common deadline. We jointly adapt the transmission rate, transmission power, and retransmission limit to minimize the transmission energy while ensuring that the video frame is reliably delivered before a deadline. In a slow fading channel, we formulate a deterministic transmission policy that allocates a retransmission limit to the TPs and jointly optimizes with the transmission bitrate and power. Contrary to the intuition that avoids packet loss and retransmission to preserve energy, we demonstrate that allowing some retransmissions in the joint optimization consumes less energy, without compromising the target reliability. In a Rayleigh fading channel, a conventional approach adopts a supportable transmission rate and declares off-channel at deep fading. We generalize the approach to include various transmission schemes at different fading states and propose the probabilistic combination of these schemes. Given the fading statistics and a channel state, our policy determines to pause or to deploy a proper transmission scheme such that the video frame transmission is energy minimized, timely, and highly reliable. Extensive simulations confirm that the proposed transmission policies consume less energy than existing methods. In particular, when the deadline or the target reliability are tightened, the proposed policies yield even higher energy efficiency.
View invariant gait recognition

In this paper, we attempt to enhance the overall recognition rate for view-invariant gait recognition. We propose a simple but efficient framework for this task with training gait sequences from multiple views. A most important problem in the framework is about the optimal choice for the training views, that is, how many views are enough to ensure a satisfying overall performance and how to combine these views to achieve the optimal performance. To solve this problem, we execute intensive experiments and give reasonable optimal choices based on the experimental results. Besides, the gait feature descriptor and the fusion method we develop for the framework also contribute to the promising results. We propose to use mean of Radon transforms of the silhouettes as the descriptor which is very competent for view-invariant application. Moreover, the combination of class correlation and view correlation is applied to score level fusion of results from different views. The CASIA database B which contains gait data from 11 views distributed uniformly in range of [0°, 180°] is chosen in our experiments.
An effective post-refinement method for shot boundary detection

In content-based video analysis, shot boundary detection (SBD) is a common first step which segments video data into elementary shots, each comprising a sequence of consecutive frames recording a video event or scene continuous in time and space. Many SBD methods have been proposed in the literature, and experimental results show that the existing methods work reasonably well for abrupt shot boundaries, but less effectively for gradual shot boundaries. In this paper, we propose an effective post-refinement method for identifying actual shot boundaries from the results obtained by existing SBD methods. The proposed method formulates the SBD problem as sequential detection of changes in the underlying feature distributions whose parameters are estimated from existing video shots. Specifically, the proposed post-refinement method enhances the performance of SBD by identifying as many false positives (false detections) and false negatives (miss detections) as possible. Experiments conducted on a large set of test videos, whose initial shot boundaries are obtained by four existing SBD methods, show that the proposed post-refinement method can improve markedly the detection recall and precision and is rather insensitive to the thresholds used by the existing methods in detecting the initial shot boundaries.
On the method of multicopy video enhancement in transform domain

Increasingly, we can obtain more than one compressed copy of the same video content with different levels of visual quality over the Internet. As the original source video is not always available, how to choose or derive a video of the best quality from these copies becomes a challenging and interesting problem. In this paper, we address this new research problem by blindly enhancing the quality of the video reconstructed from multiple compressed copies of the same video content. The aim is to reconstruct a video that achieves better quality than any of the available copies. Specifically, we propose to reconstruct each coefficient of the video in the transform domain by using a narrow quantization constraint set derived from the multiple compressed copies together with the Cauchy distribution model for each AC transform coefficient to minimize the distortion. Analytical and experimental results show the effectiveness of the proposed method.
An optimized resource allocation algorithm for scalable video delivery over wireless multicast links

In this paper, we investigate resource allocation of video multicast over resource-constrained wireless networks. The video multicast is a best-effort service which suffers from limited transmission energy and channel access time. To cater for multi-resolution video demands of heterogeneous mobile clients, we consider scalable video coding (SVC) which offers spatial, temporal and fidelity scalabilities. We formulate multicast strategy that maximizes video quality under transmission energy and channel access constraints. We show that the problem is a joint optimization of mode selection and transmission power allocation. It is a mixed-integer programming problem which is in general NP-hard. To solve the optimization problem, we first reduce the problem into a binary integer programming problem. By exploiting inter-layer dependency of SVC packets, we propose a novel solution based on the dynamic programming approach. Experiment results demonstrate that the proposed method archives notable improvement of average video quality over a conventional method.
A fast rate adaptation scheme for SVC based on the packet dependencies

The Scalable Video Coding (SVC) standard offers multiple scalabilities while maintaining high coding efficiency. However, the joint coding of multiple scalabilities complicates the rate adaptation as the SVC packets possess different priorities. To address this challenge, we propose in this paper a fast rate adaptation scheme for SVC in absence of the original video sequence. Specifically, we present an efficient algorithm to extract the SVC bitstream by a packet prioritization scheme, which is based on the analysis of packet dependencies (PD) in encoding of full scalability. Experimental results demonstrate that the proposed scheme achieves significant improvement in PSNR over the basic bit extraction scheme in SVC and attains comparable PSNR performance to the Quality Layer based rate adaptation scheme while significantly reducing the computational cost.
Layering-based color filter array interpolation

This paper presents a new method to interpolate the color filter array (CFA) pattern that is commonly used in a single-sensor digital camera. The proposed method involves two main steps: slicing the color planes into layers to identify smooth local image regions; estimating the missing color values based on the spectral correlation between color planes. The performance of the proposed algorithm is evaluated by comparing the results with those generated by other existing methods. It shows that the proposed method outperforms other methods in terms of both subjective and objective image quality.