nils-herrmann/clean_scirepeval huggingface dataset:
Optimum bit allocation for FGS video coding

This paper proposes a new bit allocation scheme for fine-granular-scalability (FGS) video coding, through which we can achieve better video quality. Different from traditional rate-distortion (R-D) optimization schemes, we consider the characteristics of the bit-plane (BF) coding method. To be specific, we first find the approximate linear relationship between the bit rate of the FGS-layer and the percentage of nonzero binary-scaled coefficients (NZBC) in each BF; second, with mathematical justification, we derive an optimal strategy by analyzing the overall distortion with respect to NZBC. Finally, we perform our optimum bit allocation (OBA) on a FGS coder. Experimental results prove that our scheme can achieve smooth video quality with a higher average PSNR gain compared with uniform bit allocation (UBA). And for certain frames with lower PSNR, it has a gain of up to 3 dB. It is highly source-independent and more robust compared with previous bit allocation schemes.
Local features are not lonely – Laplacian sparse coding for image classification

Sparse coding which encodes the original signal in a sparse signal space, has shown its state-of-the-art performance in the visual codebook generation and feature quantization process of BoW based image representation. However, in the feature quantization process of sparse coding, some similar local features may be quantized into different visual words of the codebook due to the sensitiveness of quantization. In this paper, to alleviate the impact of this problem, we propose a Laplacian sparse coding method, which will exploit the dependence among the local features. Specifically, we propose to use histogram intersection based kNN method to construct a Laplacian matrix, which can well characterize the similarity of local features. In addition, we incorporate this Laplacian matrix into the objective function of sparse coding to preserve the consistence in sparse representation of similar local features. Comprehensive experimental results show that our method achieves or outperforms existing state-of-the-art results, and exhibits excellent performance on Scene 15 data set.
Semantic Knowledge Building for Image Database by Analyzing Web Page Contents

In this paper, we present a method of semantic knowledge building for image database by extracting semantic meanings from Web page contents. The novelty of our method is that it is able to effectively extract media with a high degree of relevancy to a specific topic by incorporating word similarity and ontologies. The method is implemented in our Web image crawler and analysis system (WICAS). The system downloads Web pages and media automatically and further analyzes the semantic meanings of page contents to build up semantic knowledge for media entities. Subsequently, our system accepts high-level query terms and returns relevant media efficiently. Our experiment results show that with this new method of high-level content abstraction, media retrieval accuracy can be improved tremendously over traditional methods
Nonintrusive Quality Assessment of Noise Suppressed Speech With Mel-Filtered Energies and Support Vector Regression

Objective speech quality assessment is a challenging task which aims to emulate human judgment in the complex and time consuming task of subjective assessment. It is difficult to perform in line with the human perception due the complex and nonlinear nature of the human auditory system. The challenge lies in representing speech signals using appropriate features and subsequently mapping these features into a quality score. This paper proposes a nonintrusive metric for the quality assessment of noise-suppressed speech. The originality of the proposed approach lies primarily in the use of Mel filter bank energies (FBEs) as features and the use of support vector regression (SVR) for feature mapping. We utilize the sensitivity of FBEs to noise in order to obtain an effective representation of speech towards quality assessment. In addition, the use of SVR exploits the advantages of kernels which allow the regression algorithm to learn complex data patterns via nonlinear transformation for an effective and generalized mapping of features into the quality score. Extensive experiments conducted using two third party databases with different noise-suppressed speech signals show the effectiveness of the proposed approach.
Discriminative Signatures for Image Classification

Bag-of-words representation has shown to be a powerful technique for image classification. In this paper, we propose a new approach to discover the discriminability of each visual word (image feature) in the codebook for each image category. A general linear model (GLM) is employed to construct new histograms of the images which are the basis for image classification. We also discuss the relations between our approach and boosting approaches and non-negative matrix factorization (NMF).
Fourier Transform-Based Scalable Image Quality Measure

We present a new image quality assessment algorithm based on the phase and magnitude of the 2-D discrete Fourier transform. The basic idea is to compare the phase and magnitude of the reference and distorted images to compute the quality score. However, it is well known that the human visual system's sensitivity to different frequency components is not the same. We accommodate this fact via a simple yet effective strategy of non-uniform binning of the frequency components. This process also leads to reduced space representation of the image thereby enabling the reduced-reference (RR) prospects of the proposed scheme. We employ linear regression to integrate the effects of the changes in phase and magnitude. In this way, the required weights are determined via proper training and hence more convincing and effective. Last, using the fact that phase usually conveys more information than magnitude, we use only the phase for RR quality assessment. This provides the crucial advantage of further reduction in the required amount of reference image information. The proposed method is, therefore, further scalable for RR scenarios. We report extensive experimental results using a total of nine publicly available databases: seven image (with a total of 3832 distorted images with diverse distortions) and two video databases (totally 228 distorted videos). These show that the proposed method is overall better than several of the existing full-reference algorithms and two RR algorithms. Additionally, there is a graceful degradation in prediction performance as the amount of reference image information is reduced thereby confirming its scalability prospects. To enable comparisons and future study, a Matlab implementation of the proposed algorithm is available at http://www.ntu.edu.sg/home/wslin/reduced_phase.rar.
MobileMaps@SG - Mappedia Version 1.1

Technology has always been moving. Throughout the decades, improvements in various technological areas have led to a greater sense of convenience to ordinary people, whether it is cutting down time in accessing normal-to-day activities or getting privileged services. One of the technological areas that had been moving very rapidly is that of mobile computing. The common mobile device now has the mobility, provides entertainment via multimedia, connects to the Internet and is powered by intelligent and powerful chips. This paper will touch on an idea that is currently in the works, an integration of a recent technology that has netizens talking all over the world; Google Maps, that provide street and satellite images via the internet to the PC and Wikipedia 's user content support idea, the biggest free-content encyclopedia on the Internet. We will hit on how it is able to integrate such a technology with the idea of free form editing into one application in a small mobile device. The new features provided by this application will work toward supporting the development of multimedia application and computing.
Spatiotemporal Saliency Detection via Sparse Representation

Multimedia applications like retrieval, copy detection etc. can gain from saliency detection, which is essentially a method to identify areas in images and videos that capture the attention of the human visual system. In this paper, we propose a new spatiotemporal saliency framework for videos based on sparse representation. For temporal saliency, we model the movement of the target patch as a reconstruction process, and the overlapping patches in neighboring frames are used to reconstruct the target patch. The learned coefficients encode the positions of the matched patches, which are able to represent the motion trajectory of the target patch. We also introduce a smoothing term into our sparse coding framework to learn coherent motion trajectories. Based on the psychological findings that abrupt stimulus could cause a rapid and involuntary deployment of attention, our temporal model combines the reconstruction error, sparsity regularizer, and local trajectory contrast to measure the motion saliency. For spatial saliency, a similar sparse reconstruction process is adopted to capture the regions with high center-surround contrast. Finally, the temporal saliency and spatial saliency are combined by agreement to favor the salient regions with high confidence. Experimental results on a human fixation video dataset show our method achieved the best performance over five state-of-the-art approaches.
Analyzing trajectories on Grassmann manifold for early emotion detection from depth videos

This paper proposes a new framework for online detection of spontaneous emotions from low-resolution depth sequences of the upper part of the body. To face the challenges of this scenario, depth videos are decomposed into subsequences, each modeled as a linear subspace, which in turn is represented as a point on a Grassmann manifold. Modeling the temporal evolution of distances between subsequences of the underlying manifold as a one-dimensional signature, termed Geometric Motion History, permits us to encompass the temporal signature into an early detection framework using Structured Output SVM, thus enabling online emotion detection. Results obtained on the publicly available Cam3D Kinect database validate the proposed solution, also demonstrating that the upper body, instead of the face alone, can improve the performance of emotion detection.
Extraction of resource descriptors for distributed content based image retrieval

Content based retrieval from distributed libraries raises new and challenging issues with respect to retrieval from a single repository. In particular, an effective management of distributed libraries develops upon three main processes: resource description (extraction of descriptors that qualify the content of a given archive), resource selection (given a user query, select the resources that contain relevant documents) and results merging (organize and present items returned by individual libraries). So far, these issues have been mainly addressed for text archives. We propose an approach to the extraction and verification of resource descriptions for distributed collections of images.
The computational aspect of retrieval by spatial arrangement

Image retrieval by spatial arrangement underlies a matching problem for the interpretation of entities specified in the user query on the entities appearing in the image of the database, and for the joint comparison of their features and spatial relationships. In this paper, we provide a graph-theoretical formulation of the problem and discuss its implication on indexing and matching. We first identify an indexing scheme which may fit the characteristics of the problem, and then expound and evaluate an efficient graph matching technique which makes this indexing approach viable.
3D Face Recognition Using Isogeodesic Stripes

In this paper, we present a novel approach to 3D face matching that shows high effectiveness in distinguishing facial differences between distinct individuals from differences induced by nonneutral expressions within the same individual. The approach takes into account geometrical information of the 3D face and encodes the relevant information into a compact representation in the form of a graph. Nodes of the graph represent equal width isogeodesic facial stripes. Arcs between pairs of nodes are labeled with descriptors, referred to as 3D Weighted Walkthroughs (3DWWs), that capture the mutual relative spatial displacement between all the pairs of points of the corresponding stripes. Face partitioning into isogeodesic stripes and 3DWWs together provide an approximate representation of local morphology of faces that exhibits smooth variations for changes induced by facial expressions. The graph-based representation permits very efficient matching for face recognition and is also suited to being employed for face identification in very large data sets with the support of appropriate index structures. The method obtained the best ranking at the SHREC 2008 contest for 3D face recognition. We present an extensive comparative evaluation of the performance with the FRGC v2.0 data set and the SHREC08 data set.
Sparse Matching of Salient Facial Curves for Recognition of 3-D Faces With Missing Parts

In this work, we propose and experiment a 3-D face recognition approach capable of performing accurate face matching also in the case where just parts of probe scans are available. This is obtained through an original face representation and matching solution that first extracts keypoints of the 3-D depth image of the face and then measures how the face depth changes along facial curves connecting pairs of keypoints. Face similarity is evaluated by sparse comparison of facial curves defined across inlier pairs of matching keypoints between probe and gallery scans. In doing so, a statistical model is also proposed to associate facial curves of the gallery scans with a saliency measure so that curves that model characterizing traits of some subjects are distinguished from curves that are frequently observed in the face of many different subjects. Following recent related work, the recognition accuracy of the approach is experimented using two datasets, both comprising scans with missing parts: the Face Recognition Grand Challenge v2.0 dataset combined with the University of Notre Dame probes; the Gavab dataset.
Facial curves between keypoints for recognition of 3D faces with missing parts

In this work, we propose and experiment an original solution to 3D face recognition that supports accurate face matching also in cases where just some parts of probe scans are available. In the proposed approach, distinguishing traits of the face are captured by first extracting keypoints of the 3D depth image and then measuring how the face depth changes along facial curves connecting pairs of key-points. Face similarity is evaluated by comparing facial curves across inlier pairs of keypoints that match between probe and gallery scans. In doing so, facial curves of the gallery scans are associated with a saliency measure in order to distinguish curves that model characterizing traits of some subjects from curves that are frequently observed in the face of many different subjects. The recognition accuracy of the approach is experimented using the Face Recognition Grand Challenge v2.0 dataset.
Geodesic Distances for 3D-3D and 2D-3D Face Recognition

In this paper, we propose an original framework for representing 2D and 3D face information using geodesic distances. This aims to define a representation enabling the direct comparison between 2D face images of an individual against its 3D face model. This representation is extracted by measuring geodesic distances in 2D and 3D. In 3D, the geodesic distance between two points on a surface is computed as the length of the shortest path connecting the two points. In 2D, the geodesic distance between two pixels is computed based on the differences of gray level intensities along the segment connecting the two pixels. Experimental results are shown to demonstrate the viability of the proposed solution.
Using Geodesic Distances for 2D-3D and 3D-3D Face Recognition

In this paper, we propose an original framework for rep resenting 2D and 3D face information using geodesic distances. This aims to define a representation enabling 3D- 3D face recognition as well as the direct comparison between 2D face images of a subject against its 3D face model. This representation is extracted by measuring geodesic distances in 3D and 2D. In 3D, the geodesic distance between two points on a surface is computed as the length of the shortest path connecting the two points. In 2D, the geodesic distance between two pixels is computed based on the differences of gray level intensities along the segment connecting the two pixels. Experimental results are reported for 3D-3D and 2D-3D face recognition, in order to demonstrate the potential of the proposed solution.
Content based retrieval of 3D cellular structures

Recent advances in management of multimedia digital libraries enable effective retrieval of information in the form of audio, image and video. However, retrieval of information in the form of 3D objects has received limited attention so far. Yet many archives of 3D objects already exist and are expected to grow both in relevance and size. In this paper, we address the problem of effective description and retrieval of 3D data representing intracellular structures. These structures are represented in the form of image stacks, being an image stack a set of 2D images representing planar sections of a cellular body at different heights. In the proposed method, 2D visual feature descriptors and Hidden Markov Models are combined to obtain a representation model which is able to distinguish such intracellular structures as Golgi, nucleus, endoplasmic reticulum and lysosomes. Preliminary results are presented to show the effectiveness of the proposed representation model.
Learning shape variations of motion trajectories for gait analysis

The analysis of human gait is more and more investigated due to its large panel of potential applications in various domains, like rehabilitation, deficiency diagnosis, surveillance and movement optimization. In addition, the release of depth sensors offers new opportunities to achieve gait analysis in a non-intrusive context. In this paper, we propose a gait analysis method from depth sequences by analyzing separately each step so as to be robust to gait duration and incomplete cycles. We analyze the shape of the motion trajectory as signature of the gait and consider shape variations within a Riemannian manifold to learn step models. During classification, the derivation of each performed step is evaluated in an online manner to qualitatively analyze the gait. Experiments are carried out in the context of abnormal gait detection and person re-identification trough gait recognition. Results demonstrated the potential of the method in both scenarios.
Using indexing structures for resource descriptors extraction from distributed image repositories

Content based retrieval from distributed libraries raises new and challenging issues with respect to retrieval from a single repository. In particular, an effective management of distributed libraries develops upon three main processes: resource description (extraction of descriptors that qualify the content of a given archive), resource selection (given a user query, analyze resource descriptions and select the resources that contain relevant documents) and results merging (organize and present items returned by individual libraries). So far, these issues have been mainly addressed for text archives. We present a solution to resource descriptors extraction, developing on the use of techniques for multidimensional data indexing. In particular, we implement and compare the extraction of resource descriptors computed through two different indexing approaches; namely m-tree indexing and fuzzy clustering. Comparative results are presented for a test database of about 1000 images.
Efficient Matching and Indexing of Graph Models in Content-Based Retrieval

In retrieval from image databases, evaluation of similarity, based both on the appearance of spatial entities and on their mutual relationships, depends on content representation based on attributed relational graphs. This kind of modeling entails complex matching and indexing, which presently prevents its usage within comprehensive applications. In this paper, we provide a graph-theoretical formulation for the problem of retrieval based on the joint similarity of individual entities and of their mutual relationships and we expound its implications on indexing and matching. In particular, we propose the usage of metric indexing to organize large archives of graph models, and we propose an original look-ahead method which represents an efficient solution for the (sub)graph error correcting isomorphism problem needed to compute object distances. Analytic comparison and experimental results show that the proposed lookahead improves the state-of-the-art in state-space search methods and that the combined use of the proposed matching and indexing scheme permits for the management of the complexity of a typical application of retrieval by spatial arrangement.
3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold

Recognizing human actions in 3-D video sequences is an important open problem that is currently at the heart of many research domains including surveillance, natural interfaces and rehabilitation. However, the design and development of models for action recognition that are both accurate and efficient is a challenging task due to the variability of the human pose, clothing and appearance. In this paper, we propose a new framework to extract a compact representation of a human action captured through a depth sensor, and enable accurate action recognition. The proposed solution develops on fitting a human skeleton model to acquired data so as to represent the 3-D coordinates of the joints and their change over time as a trajectory in a suitable action space. Thanks to such a 3-D joint-based framework, the proposed solution is capable to capture both the shape and the dynamics of the human body, simultaneously. The action recognition problem is then formulated as the problem of computing the similarity between the shape of trajectories in a Riemannian manifold. Classification using k-nearest neighbors is finally performed on this manifold taking advantage of Riemannian geometry in the open curve shape space. Experiments are carried out on four representative benchmarks to demonstrate the potential of the proposed solution in terms of accuracy/latency for a low-latency action recognition. Comparative results with state-of-the-art methods are reported.
Reconstructing high-resolution face models from Kinect depth sequences acquired in uncooperative contexts

Performing face recognition across 3D scans of different resolution is now attracting an increasing interest thanks to the introduction of a new generation of depth cameras, capable of acquiring color/depth images over time. However, these devices have still a much lower resolution than the 3D high-resolution scanners typically used for face recognition applications. If data are acquired without user cooperation, the problem is even more challenging and the gap of resolution between probe and gallery scans can yield to a severe loss in terms of recognition accuracy. Based on these premises, we propose a method to build a higher-resolution 3D face model from 3D data acquired by a low-resolution scanner. This face model is built using data acquired when a person passes in front of the scanner, following an uncooperative protocol. To perform non-rigid registration of point sets and account for deformation of the face during the acquisition process, the Coherent Point Drift (CPD) method is used. Registered 3D data are filtered through a variant of the lowess method to remove outliers and build the final face model. The proposed approach is evaluated in terms of accuracy of face reconstruction and face recognition.