nils-herrmann/clean_scirepeval huggingface dataset:
Human detection using partial least squares analysis

Significant research has been devoted to detecting people in images and videos. In this paper we describe a human detection method that augments widely used edge-based features with texture and color information, providing us with a much richer descriptor set. This augmentation results in an extremely high-dimensional feature space (more than 170,000 dimensions). In such high-dimensional spaces, classical machine learning algorithms such as SVMs are nearly intractable with respect to training. Furthermore, the number of training samples is much smaller than the dimensionality of the feature space, by at least an order of magnitude. Finally, the extraction of features from a densely sampled grid structure leads to a high degree of multicollinearity. To circumvent these data characteristics, we employ Partial Least Squares (PLS) analysis, an efficient dimensionality reduction technique, one which preserves significant discriminative information, to project the data onto a much lower dimensional subspace (20 dimensions, reduced from the original 170,000). Our human detection system, employing PLS analysis over the enriched descriptor set, is shown to outperform state-of-the-art techniques on three varied datasets including the popular INRIA pedestrian dataset, the low-resolution gray-scale DaimlerChrysler pedestrian dataset, and the ETHZ pedestrian dataset consisting of full-length videos of crowded scenes.
Change detection based on features invariant to monotonic transforms and spatial constrained matching

Discovering regions that have changed in a set of images acquired from a scene at different times and possibly from different view points and cameras is a crucial step for many image processing applications. Remote sensing, visual surveillance, medical diagnosis and treatment, civil infrastructure, and underwater sensing are some examples of such applications. This work proposes a novel approach to detect changes automatically without a learning step by using image analysis techniques and segmentation based on superpixels. Unlike most common approaches, which are pixel-based, we present an approach that combines super-pixel extraction, hierarchical clustering and segment matching. The experimental results show the effectiveness of the proposed approach comparing it a background subtraction technique, demonstrating the robustness of our algorithm to illumination variations, non-uniform attenuations, atmospheric absorption and swaying trees.
Kernel Partial Least Squares for person re-identification

Person re-identification (Re-ID) keeps the same identity for a person as he moves along an area with nonoverlapping surveillance cameras. Re-ID is a challenging task due to appearance changes caused by different camera viewpoints, occlusion and illumination conditions. While robust and discriminative descriptors are obtained combining texture, shape and color features in a high-dimensional representation, the achievement of accuracy and efficiency demands dimensionality reduction methods. At this paper, we propose variations of Kernel Partial Least Squares (KPLS) that simultaneously reduce the dimensionality and increase the discriminative power. The Cross-View KPLS (X-KPLS) and KPLS Mode A capture cross-view discriminative information and are successful for unsupervised and supervised Re-ID. Experimental results demonstrate that X-KPLS presents equal or higher matching results when compared to other methods in literature at PRID450S.
Face Identification Using Large Feature Sets

With the goal of matching unknown faces against a gallery of known people, the face identification task has been studied for several decades. There are very accurate techniques to perform face identification in controlled environments, particularly when large numbers of samples are available for each face. However, face identification under uncontrolled environments or with a lack of training data is still an unsolved problem. We employ a large and rich set of feature descriptors (with more than 70 000 descriptors) for face identification using partial least squares to perform multichannel feature weighting. Then, we extend the method to a tree-based discriminative structure to reduce the time required to evaluate probe samples. The method is evaluated on Facial Recognition Technology (FERET) and Face Recognition Grand Challenge (FRGC) data sets. Experiments show that our identification method outperforms current state-of-the-art results, particularly for identifying faces acquired across varying conditions.
Hyperspectral image interpretation based on partial least squares

Remote sensed hyperspectral images have been used for many purposes and have become one of the most important tools in remote sensing. Due to the large amount of available bands, e.g., a few hundreds, the feature extraction step plays an important role for hyperspectral images interpretation. In this paper, we extend a well-know feature extraction method called Extended Morphological Profile (EMP) which encodes spatial and spectral information by using Partial Least Squares (PLS) to emphasize the importance of the more discriminative features. PLS is employed twice in our proposal, i.e., to the EMP features and to the raw spatial information, which are then concatenated to be further interpreted by the SVM classifier. Our experiments in two well-known data sets, the Indian Pines and Pavia University, have shown that our proposal outperforms the accuracy of classification methods employing EMP and other baseline feature extraction methods with different classifiers.
Activity Recognition Based on a Magnitude-Orientation Stream Network

The temporal component of videos provides an important clue for activity recognition, as a number of activities can be reliably recognized based on the motion information. In view of that, this work proposes a novel temporal stream for two-stream convolutional networks based on images computed from the optical flow magnitude and orientation, named Magnitude-Orientation Stream (MOS), to learn the motion in a better and richer manner. Our method applies simple nonlinear transformations on the vertical and horizontal components of the optical flow to generate input images for the temporal stream. Experimental results, carried on two well-known datasets (HMDB51 and UCF101), demonstrate that using our proposed temporal stream as input to existing neural network architectures can improve their performance for activity recognition. Results demonstrate that our temporal stream provides complementary information able to improve the classical two-stream methods, indicating the suitability of our approach to be used as a temporal video representation.
Combination techniques for hyperspectral image interpretation

In this work, we propose two main contributions to hyperspectral image interpretation. Firstly, while the traditional Weighted Linear Combination optimized by Genetic Algorithms (WLC-GA)  intends to give more discriminant power to those classification approaches contributing the most, we extend it to make a fine tuning over the class probabilities within the combination process. Then, we compare both methods (WLC-GA and its extension) with a more complex non-linear meta learning strategy called Stacked Generalization in which Support Vector Machines with Radial Basis Function kernel was used as combiner . The experimental results, considering two widely used data sets, the Indian Pines and the Pavia University, are conducted in three different scenarios. Results show that both WLC-GA and its extended version achieve the best overall accuracy, and the proposed classification approach overcomes the accuracies of the other traditional ones used in this study.
Multimodal structure segmentation and analysis of music using audio and textual information

In this paper, we present a multimodal approach to structure segmentation of music with applications to audio content analysis and music information retrieval. In particular, since lyrics contain rich information about the semantic structure of a song, our approach incorporates lyrics to overcome the existing difficulties associated with large acoustic variation in music. We further design a constrained clustering algorithm for music segmentation and evaluate its performance on commercial recordings. Experimental results show that our method can effectively detect the boundaries and the types of semantic structure of music segments.
Integration of Digital Stabilizer With Video Codec for Digital Video Cameras

This paper presents three novel schemes for integrating digital stabilizer with video codec of a digital video camera, each designed for a different application scenario. Scheme 1 confines the global motion estimation (ME) to within a small background region determined by clustering the motion vectors (MVs) generated by the video encoder. Scheme 2 performs the main ME task at the digital stabilizer and sends the resulting motion vectors to the video encoder, where the motion vectors are further refined to subpixel accuracy. Scheme 3, which is applied on the decoder side, acquires the motion information directly from the video decoder to perform digital stabilization. These integration schemes achieve computational efficiency without affecting the performance of video coding and digital stabilization.
Ranking-Based Emotion Recognition for Music Organization and Retrieval

Determining the emotion of a song that best characterizes the affective content of the song is a challenging issue due to the difficulty of collecting reliable ground truth data and the semantic gap between human's perception and the music signal of the song. To address this issue, we represent an emotion as a point in the Cartesian space with valence and arousal as the dimensions and determine the coordinates of a song by the relative emotion of the song with respect to other songs. We also develop an RBF-ListNet algorithm to optimize the ranking-based objective function of our approach. The cognitive load of annotation, the accuracy of emotion recognition, and the subjective quality of the proposed approach are extensively evaluated. Experimental results show that this ranking-based approach simplifies emotion annotation and enhances the reliability of the ground truth. The performance of our algorithm for valence recognition reaches 0.326 in Gamma statistic.
A block transform coder for arbitrarily shaped image segments

This paper describes a method for coding arbitrarily shaped image segments. The method uses an iterative technique based on the theory of successive projection onto convex sets to determine the best transform coefficients. It uses block transforms with frequency domain region-zeroing and space domain region-enforcing operations for effective coding of image segments of arbitrary shape. A major strength of this method is that it can be implemented in real-time using existing codec hardware at an insignificant additional cost.<<ETX>>
A Scalable Peer-to-Peer IPTV System

HotStreaming is an overlay peer-to-peer (P2P) based IPTV system. It integrates the innovations in both overlay networking and video coding for optimal user experience. The HotStreaming system is composed of three key components: partnership formation, data request scheduling and multiple description coding (MDC). In the partnership formation component, we propose two policies to reduce the time of disconnection and the number of isolated peers. In the MDC component, we adopt MDC with spatial-temporal hybrid interpolation (MDC-STHI), which makes the peers in a P2P network to adjust the streaming traffic according to their bandwidth limitation and capability of devices. The experimental results show that the HotStreaming system improves the video quality over a lossy and dynamic networking environment.
Speech recognition for image animation and coding

We discuss some issues related to acoustic assisted image coding and animation. An approach of talker independent acoustic assisted image coding and animation scheme is studied. A perceptually based sliding window encoder is proposed. It utilizes the high rate (or oversampled) viseme sequence from the audio domain for image domain viseme interpolation and smoothing. The image domain visemes in our approach are dynamically constructed from a set of basic visemes. The look-ahead and look-back moving interpolations in the proposed approach provide an effective way to compensate the mismatch between auditory and visual perceptions.
Digital image stabilization and its integration with video encoder

Digital image stabilizer and video encoder are two important components of a digital video camera. The digital image stabilizer compensates the image movement caused by hand jiggles and thereby improves the perceptual quality of the captured image sequence, while the video encoder compresses the huge amount of video data down to a reasonable size. Both components require the motion information of the image sequence to perform their respective tasks. Since motion estimation is a computation intensive operation, we present an integration scheme for integrating the digital image stabilizer with the video encoder. The image stabilization algorithms and the technical issues involved in the integration are discussed. Simulation results are shown to illustrate the effectiveness of the proposed digital image stabilization system.
Light Field Analysis for Modeling Image Formation

Image formation is traditionally described by a number of individual models, one for each specific effect in the image formation process. However, it is difficult to aggregate the effects by concatenating such individual models. In this paper, we apply light transport analysis to derive a unified image formation model that represents the radiance along a light ray as a 4-D light field signal and physical phenomena such as lens refraction and blocking as linear transformations or modulations of the light field. This unified mathematical framework allows the entire image formation process to be elegantly described by a single equation. It also allows most geometric and photometric effects of imaging, including perspective transformation, defocus blur, and vignetting, to be represented in both 4-D primal and dual domains. The result matches that of traditional models. Generalizations and applications of this theoretic framework are discussed.
Perceptual Rate-Distortion Optimization Using Structural Similarity Index as Quality Metric

The rate-distortion optimization (RDO) framework for video coding achieves a tradeoff between bit-rate and quality. However, objective distortion metrics such as mean squared error traditionally used in this framework are poorly correlated with perceptual quality. We address this issue by proposing an approach that incorporates the structural similarity index as a quality metric into the framework. In particular, we develop a predictive Lagrange multiplier estimation method to resolve the chicken and egg dilemma of perceptual-based RDO and apply it to H.264 intra and inter mode decision. Given a perceptual quality level, the resulting video encoder achieves on the average 9% bit-rate reduction for intra-frame coding and 11% for inter-frame coding over the JM reference software. Subjective test further confirms that, at the same bit-rate, the proposed perceptual RDO indeed preserves image details and prevents block artifact better than traditional RDO.
Preserving image color appearance on non-white projection surfaces

The demand for quality is ever increasing for mobile and wearable devices. For a portable procam system to preserve the color appearance of an image projected on a color surface, the radiometric characteristics of the system and the color surface have to be determined at high accuracy. To achieve the goal, we propose in this paper an effective method to address the effect of inter-pixel coupling and inter-channel coupling on gamma function estimation. It brings the color appearance of an image on the color surface closer to that on an ideal white screen. Experimental results show that the proposed method generates more accurate gamut downscaling parameter in the color compensation process than previous techniques, particularly for dark projection surfaces.
Analysis and Compensation of Rolling Shutter Effect

Due to the sequential-readout structure of complementary metal-oxide semiconductor image sensor array, each scanline of the acquired image is exposed at a different time, resulting in the so-called electronic rolling shutter that induces geometric image distortion when the object or the video camera moves during image capture. In this paper, we propose an image processing technique using a planar motion model to address the problem. Unlike previous methods that involve complex 3-D feature correspondences, a simple approach to the analysis of inter- and intraframe distortions is presented. The high-resolution velocity estimates used for restoring the image are obtained by global motion estimation, Bezier curve fitting, and local motion estimation without resort to correspondence identification. Experimental results demonstrate the effectiveness of the algorithm.
Blood vessel extraction from OCT data by short-time RPCA

Optical coherence tomography (OCT) is a medical imaging technology that allows for non-invasive diagnosis of diseases in the early stage. Because blood flow anomalies provide useful information for many diseases, we develop an automatic blood vessel detection algorithm based on the robust principle component analysis (RPCA) technique. Specifically, we propose a short-time RPCA method that divides an OCT volume into segments and decomposes each segment into a low-rank structure representing relatively static tissues and a sparse matrix representing the blood vessels. It efficiently extracts blood vessel structure from OCT data by distinguishing static and dynamic components. This work serves as the foundation for further blood flow analysis.
A Collaborative Transcoding Strategy for Live Broadcasting Over Peer-to-Peer IPTV Networks

Real-time video transcoding that is often needed for robust video broadcasting over heterogeneous networks is not supported in most existing devices. To address this problem, we propose a collaborative strategy that leverages the peering architecture of peer-to-peer Internet protocol television networks and makes the computational resources of peers sharable. The video transcoding task is distributed among the peers and completed collaboratively. A prototype of the live video broadcasting system is evaluated over a 100-node testbed on the PlanetLab. The experimental results show that the proposed strategy works effectively even when the majority of the peers have limited computational resource and bandwidth.
Quality Improvement of Video Codec by Rate-Distortion Optimized Quantization

Conventional quantization methods consider only the distortion between original and reconstructed video as the cost of compression. Considering the time-varying nature of network bandwidth for multimedia services, we believe a video coding system can provide a better quality of experience if it takes the bit rate of the compressed bit stream into consideration as well when optimizing the quantization. In this paper we present a rate-distortion optimization approach to the quantization of video coding. This approach is able to balance between rate and distortion for quantization and enhance the overall quality of the entire coding system, with only a slight increase in computational overhead. We implement this method in H.264/AVC, and the extensive experimental data obtained under various test conditions show that the performance of the R-D optimized quantization is indeed better than the H.264 reference software.
SVC or MDC? That's the question

Multi-rate video scalable codecs, SVC and MDC, provide as plausible solutions to deal with heterogeneous environment of Internet. They, however, also give rise to a wide debate over which one is more efficient supporting P2P IPTV systems. Our goal in this work is to resolve the debate by providing a quantitative comparison of P2P IPTV systems given different choices of coding schemes and P2P network formations. The answer is rather subtle. MDC-based systems, though outperform SVC-based ones under certain network formation with bottleneck in terms of network throughput, suffer from a lower level of perceptual quality in terms of PSNR due to the coding inefficiency. The results drawn from this paper can be provided not only a lesson to the design of large-scale heterogeneous P2P IPTV systems but also as a strong evidence that a poor choice of codec at the higher level might over shadow the network-level designs and the codec and network formation components ought to be co-designed for optimal user experience.
Fast H.264 selective intra mode decision for inter-frame coding

Despite its remarkable coding efficiency, the computational complexity of the spatial-domain intra prediction scheme of H.264 remains a challenging issue especially for applications of the High Profile, as it incorporates the additional intra 8×8 prediction mode. To address this issue, we propose in this paper a fast selective intra mode decision algorithm that exploits a newly discovered property of inter-frame coding that the difference in rate-distortion cost between the best inter mode and the intra 16×16 mode is highly related to the type of the best coding mode. The proposed algorithm has high prediction accuracy, low computational overhead, and negligible effect on PSNR and bit-rate. Performance comparison is provided to show the superiority of the algorithm.
Analysis of Disparity Error for Stereo Autofocus

As more and more stereo cameras are installed on electronic devices, we are motivated to investigate how to leverage disparity information for autofocus. The main challenge is that stereo images captured for disparity estimation are subject to defocus blur unless the lenses of the stereo cameras are at the in-focus position. Therefore, it is important to investigate how the presence of defocus blur would affect stereo matching and, in turn, the performance of disparity estimation. In this paper, we give an analytical treatment of this fundamental issue of disparity-based autofocus by examining the relation between image sharpness and disparity error. A statistical approach that treats the disparity estimate as a random variable is developed. Our analysis provides a theoretical backbone for the empirical observation that, regardless of the initial lens position, disparity-based autofocus can bring the lens to the hill zone of the focus profile in one movement. The insight gained from the analysis is useful for the implementation of an autofocus system.
Matching 3-D Line Segments with Applications to Multiple-Object Motion Estimation

A two-stage algorithm for matching line segments using three-dimensional data is presented. In the first stage, a tree-search based on the orientation of the line segments is applied to establish potential matches. the sign ambiguity of line segments is fixed by a simple congruency constraint. In the second stage, a Hough clustering technique based on the position of line segments is applied to verify potential matches. Any paired line segments of a match that cannot be brought to overlap by the translation determined by the clustering are removed from the match. Unlike previous methods, this algorithm combats noise more effectively, and ensures the global consistency of a match. While the original motivation for the algorithm is multiple-object motion estimation from stereo image sequences, the algorithm can also be applied to other domains, such as object recognition and object model construction from multiple views.