nils-herrmann/clean_scirepeval huggingface dataset:
Image Retargeting for Preserving Robust Local Feature: Application to Mobile Visual Search

With the sharp increasing of mobile devices, conducting search on mobile devices becomes pervasive, and one of the most popular applications is mobile visual search. To achieve low bit-rate visual search, most of the existing works focus on addressing local descriptor coding and BoW histogram compression . In this paper, we extend the concept of image retargeting and propose a new image resizing approach that is devoted to preserving the robust local features in the query image while resizing it. Based on the extended concept, we introduce a novel mobile-visual-search scheme that conducts the proposed approach to reduce the size of the query image for achieving low bit-rate visual search. Extensive experiments on Oxford 5 K and Flickr 100k datasets show that our approach obtains superior retrieval performance than state-of-the-art image resizing approaches at the similar query size; meanwhile, it is cost effective in terms of processing time.
Robust speech recognition based on multi-band spectral subtraction

In order to reduce the degradation of the speech recognition accuracy while the testing condition are mismatched with the training condition around noisy environment, a kind of multi-band spectral subtraction has been proposed. The estimated noise signals were extracted from the first few frames of the noisy speech. The noisy speech and estimation of noise signals by the frequency were divided into non-overlapping M frequency bands. According to the SNR (signal-to-noise ratio) of noise speech in each frequency band, the band noise spectral subtraction parameters can be determined. The front-end speech enhancement module and the speech recognizer constitute a robust speech recognition system. The results of simulation experiments indicate that the recognition accuracy of multi-band spectral subtraction robust speech recognition system is obviously superior to the basic spectral subtraction in different signal-to-noise ratios and different noise's types.
The Period Estimation of PN Sequence Using Reprocessing Spectrum of DS-UWB signal

A kind of method which is based on the reprocessing power spectrum density of signals is proposed. In the situation of deficient prior knowledge of received UWB signals, the difficult problem of detecting its PN period can be effectively solved. The power spectrum density of direct sequence ultra wideband (DS-UWB) is regarded as observed signals in this method. Computing its power spectrum density again, we find that the pulses interval of DS-UWB’s reprocessing power spectrum is equal to the period of PN sequence. The method can be used in non-cooperate communication situations. When the distance of pulse peak of power spectrum reprocessing is scanned, the period of PN sequence can be estimated correctly. Finally ,the method can be validated by the computer simulation program, which  shows that the algorithm is effective.
Sequential Video VLAD: Training the Aggregation Locally and Temporally

As characterizing videos simultaneously from spatial and temporal cues has been shown crucial for the video analysis, the combination of convolutional neural networks and recurrent neural networks, i.e., recurrent convolution networks (RCNs), should be a native framework for learning the spatio-temporal video features. In this paper, we develop a novel sequential vector of locally aggregated descriptor (VLAD) layer, named SeqVLAD, to combine a trainable VLAD encoding process and the RCNs architecture into a whole framework. In particular, sequential convolutional feature maps extracted from successive video frames are fed into the RCNs to learn soft spatio-temporal assignment parameters, so as to aggregate not only detailed spatial information in separate video frames but also fine motion information in successive video frames. Moreover, we improve the gated recurrent unit (GRU) of RCNs by sharing the input-to-hidden parameters and propose an improved GRU-RCN architecture named shared GRU-RCN (SGRU-RCN). Thus, our SGRU-RCN has a fewer parameters and a less possibility of overfitting. In experiments, we evaluate SeqVLAD with the tasks of video captioning and video action recognition. Experimental results on Microsoft Research Video Description Corpus, Montreal Video Annotation Dataset, UCF101, and HMDB51 demonstrate the effectiveness and good performance of our method.
Incorporate support vector machines to content-based image retrieval with relevance feedback

By using relevance feedback, content-based image retrieval (CBIR) allows the user to retrieve images interactively. Beginning with a coarse query, the user can select the most relevant images and provide a weight of preference for each relevant image to refine the query. The high level concept borne by the user and perception subjectivity of the user can be automatically captured by the system to some degree. This paper proposes an approach to utilize both positive and negative feedbacks for image retrieval. Support vector machines (SVM) is applied to classifying the positive and negative images. The SVM learning results are used to update the preference weights for the relevant images. This approach releases the user from manually providing preference weight for each positive example. Experimental results show that the proposed approach has improvement over the previous approach (Rui et al. 1997) that uses positive examples only.
A quantization-based image authentication system

In this paper, we introduce a robust and secure digital signature scheme for image content authentication, by integrating content feature extraction, watermarking, cryptographic hashing, and digital signature into a unified framework. Given an image, a cryptographic signature is robustly generated based on extracted invariant features and then embedded into the DCT coefficients as watermarks. Quantization is applied to both robust feature extraction and watermarking. Simulation results further demonstrate the practicability of our method in terms of system robustness and system security.
Spatial-DiscLDA for visual recognition

Topic models such as pLSA, LDA and their variants have been widely adopted for visual recognition. However, most of the adopted models, if not all, are unsupervised, which neglected the valuable supervised labels during model training. In this paper, we exploit recent advancement in supervised topic modeling, more particularly, the DiscLDA model for object recognition. We extend it to a part based visual representation to automatically identify and model different object parts. We call the proposed model as Spatial-DiscLDA (S-DiscLDA). It models the appearances and locations of the object parts simultaneously, which also takes the supervised labels into consideration. It can be directly used as a classifier to recognize the object. This is performed by an approximate inference algorithm based on Gibbs sampling and bridge sampling methods. We examine the performance of our model by comparing its performance with another supervised topic model on two scene category datasets, i.e., LabelMe and UIUC-sport dataset. We also compare our approach with other approaches which model spatial structures of visual features on the popular Caltech-4 dataset. The experimental results illustrate that it provides competitive performance.
Toward Intelligent Use of Semantic Information on Subspace Discovery for Image Retrieval

Image retrieval has been widely used in many fields of science and engineering. The semantic concept of user interest is obtained by a learning process. Traditional techniques often assume the images are from certain distribution and all images from the same class are visually similar. Our study shows that those assumptions are inappropriate in many cases. To solve this problem we model the images as lying on non-linear subspaces embedded in the high dimensional space. We also find that a set of low-level feature subspaces may correspond to one high-level semantic concept. Unlike most unsupervised subspace learning techniques, we propose to intelligently use the semantic similarity and dissimilarity information provided by user in discovering the discriminant structure of image subspaces in respect to classification. Theoretical study shows that our methods converge to linear discriminant analysis if certain criteria are met. Extensive experiments are designed to evaluate the performance of our method and compare it to other state-of-the-art techniques. The results show the superior performance of our proposed method
Democratic Diffusion Aggregation for Image Retrieval

Content-based image retrieval is an important research topic in the multimedia field. In large-scale image search using local features, image features are encoded and aggregated into a compact vector to avoid indexing each feature individually. In the aggregation step, sum-aggregation is wildly used in many existing works and demonstrates promising performance. However, it is based on a strong and implicit assumption that the local descriptors of an image are identically and independently distributed in descriptor space and image plane. To address this problem, we propose a new aggregation method named democratic diffusion aggregation (DDA) with weak spatial context embedded. The main idea of our aggregation method is to re-weight the embedded vectors before sum-aggregation by considering the relevance among local descriptors. Different from previous work, by conducting a diffusion process on the improved kernel matrix, we calculate the weighting coefficients more efficiently without any iterative optimization. Besides considering the relevance of local descriptors from different images, we also discuss an efficient query fusion strategy which uses the initial top-ranked image vectors to enhance the retrieval performance. Experimental results show that our aggregation method exhibits much higher efficiency (about × 14 faster) and better retrieval accuracy compared with previous methods, and the query fusion strategy consistently improves the retrieval quality.
Toward an improved error metric

In many computer vision algorithms, the well known Euclidean or SSD (sum of the squared differences) metric is prevalent and justified from a maximum likelihood perspective when the additive noise is Gaussian. However, Gaussian noise distribution assumption is often invalid. Previous research has found that other metrics such as double exponential metric or Cauchy metric provide better results, in accordance with the maximum likelihood approach. In this paper, we examine different error metrics and provide a theoretical approach to derive a rich set of nonlinear estimations. Our results on image databases show more robust results are obtained for noise estimation based on the proposed error metric analysis.
Learning Microarray Gene Expression Data by Hybrid Discriminant Analysis

Microarray technology offers a high-throughput means to study expression networks and gene regulatory networks in cells. The intrinsic nature of high dimensionality and small sample size in microarray data calls for effective computational methods. In this article, we propose a novel hybrid dimension reduction technique for classification that combines principal component analysis (PCA) and linear discriminant analysis (LDA)-hybrid PCA and LDA analysis. This technique effectively solves the singular scatter matrix problem caused by small training samples and increases the effective dimension of the projected subspace. It offers more flexibility and a richer set of alternatives to LDA and PCA in the parametric space. In addition, we propose a boosted hybrid discriminant analysis (HDA), using the AdaBoost algorithm which provides a unified and stable solution to find close to the optimal PCA-LDA prediction result and also reduces computational complexity. Extensive experiments on the yeast cell cycle regulation data set show the superior performance of the hybrid analysis, as we explain.
Pooling the Convolutional Layers in Deep ConvNets for Video Action Recognition

Deep ConvNets have shown their good performance in image classification tasks. However, there still remains problems in deep video representations for action recognition. On one hand, current video ConvNets are relatively shallow compared with image ConvNets, which limits their capability of capturing the complex video action information; on the other hand, temporal information of videos is not properly utilized to pool and encode the video sequences. Toward these issues, in this paper we utilize two state-of-the-art ConvNets, i.e., the very deep spatial net (VGGNet ) and the temporal net from Two-Stream ConvNets , for action representation. The convolutional layers and the proposed new layer, called frame-diff layer, are extracted and pooled with two temporal pooling strategies: Trajectory pooling and Line pooling. The pooled local descriptors are then encoded with vector of locally aggregated descriptors (VLAD)  to form the video representations. In order to verify the effectiveness of the proposed framework, we conduct experiments on UCF101 and HMDB51 data sets. It achieves accuracy of 92.08% on UCF101, which is the state-of-the-art, and the accuracy of 65.62% on HMDB51, which is comparable to the state-of-the-art. In addition, we propose the new Line pooling strategy, which can speed up the extraction of feature and achieve the comparable performance of the Trajectory pooling.
An On-Line Unsupervised Neural Network to Adaptive Feature Extraction of Lower SNR DS-SS signals

This paper introduces an on-line unsupervised learning neural network (NN) for adaptive feature extraction via principal component analysis (LEAP) of lower signal to noise ratios (SNR) direct sequence spread spectrum (DS-SS) signals. The proposed method is based on eigen-analysis of DS-SS signals. The PN sequence and the strength of the signal can be extracted by the first and second principal eigenvectors and their associated eigenvalues of autocorrelation matrix of DS-SS signals blindly. However the eigen-analysis method belongs to a batch method, it is difficult to real-time implementation. In this case, we can use the LEAP NN method to realize on-line adaptive principal feature extraction of the lower SNR received DS-SS signals effectively. Unlike other feature extraction methods, the estimate of the PN sequence and signal strength improves steadily with the number of code repeats. The method is applicable to arbitrary PN spreading sequence and message sequences and can theoretically operate in environments containing arbitrary levels of white background noise, and for signals with arbitrary unknown timing phase. The method requires the DS-SS signal to have a constant-modulus spreading sequence and unrelated message and code-repeat rates. This paper introduces the basic technique, and demonstrates the algorithm via computer simulation for a single DS-SS signal received in the presence of white Gaussian noise
Saliency-Aware Nonparametric Foreground Annotation Based on Weakly Labeled Data

In this paper, we focus on annotating the foreground of an image. More precisely, we predict both image-level labels (category labels) and object-level labels (locations) for objects within a target image in a unified framework. Traditional learning-based image annotation approaches are cumbersome, because they need to establish complex mathematical models and be frequently updated as the scale of training data varies considerably. Thus, we advocate the nonparametric method, which has shown potential in numerous applications and turned out to be attractive thanks to its advantages, i.e., lightweight training load and scalability. In particular, we exploit the salient object windows to describe images, which is beneficial to image retrieval and, thus, the subsequent image-level annotation and localization tasks. Our method, namely, saliency-aware nonparametric foreground annotation, is practical to alleviate the full label requirement of training data, and effectively addresses the problem of foreground annotation. The proposed method only relies on retrieval results from the image database, while pretrained object detectors are no longer necessary. Experimental results on the challenging PASCAL VOC 2007 and PASCAL VOC 2008 demonstrate the advance of our method.
Group Activity Recognition by Gaussian Processes Estimation

Human action recognition has been well studied recently, but recognizing the activities of more than three persons remains a challenging task. In this paper, we propose a motion trajectory based method to classify human group activities. Gaussian Processes are introduced to represent human motion trajectories from a probabilistic perspective to handle the variability of people’s activities in group. With respect to the relationships of persons in group activities, three discriminative descriptors are designed, which are Individual, Dual and Unitized Group Activity Pattern. We adopt the Bag of Words approach to solve the problem of unbalanced number of persons in different activities. Experiments are conducted on the human group-activity video database, and the results show that our approach outperforms the state-of-the-art.
Video-Based Person Re-identification by Deep Feature Guided Pooling

Person re-identification (re-id) aims to match a specific person across non-overlapping views of different cameras, which is currently one of the hot topics in computer vision. Compared with image-based person re-id, video-based techniques could achieve better performance by fully utilizing the space-time information. This paper presents a novel video-based person re-id method named Deep Feature Guided Pooling (DFGP), which can take full advantage of the space-time information. The contributions of the method are in the following aspects: (1) PCA-based convolutional network (PCN), a lightweight deep learning network, is trained to generate deep features of video frames. Deep features are aggregated by average pooling to obtain person deep feature vectors. The vectors are utilized to guide the generation of human appearance features, which makes the appearance features robust to the severe noise in videos. (2) Hand-crafted local features of videos are aggregated by max pooling to reinforce the motion variations of different persons. In this way, the human descriptors are more discriminative. (3) The final human descriptors are composed of deep features and hand-crafted local features to take their own advantages and the performance of identification is promoted. Experimental results show that our approach outperforms six other state-of-the-art video-based methods on the challenging PRID 2011 and iLIDS-VID video-based person re-id datasets.
Image Annotation by Input–Output Structural Grouping Sparsity

Automatic image annotation (AIA) is very important to image retrieval and image understanding. Two key issues in AIA are explored in detail in this paper, i.e., structured visual feature selection and the implementation of hierarchical correlated structures among multiple tags to boost the performance of image annotation. This paper simultaneously introduces an input and output structural grouping sparsity into a regularized regression model for image annotation. For input high-dimensional heterogeneous features such as color, texture, and shape, different kinds (groups) of features have different intrinsic discriminative power for the recognition of certain concepts. The proposed structured feature selection by structural grouping sparsity can be used not only to select group-of-features but also to conduct within-group selection. Hierarchical correlations among output labels are well represented by a tree structure, and therefore, the proposed tree-structured grouping sparsity can be used to boost the performance of multitag image annotation. In order to efficiently solve the proposed regression model, we relax the solving process as a framework of the bilayer regression model for multilabel boosting by the selection of heterogeneous features with structural grouping sparsity (Bi-MtBGS). The first-layer regression is to select the discriminative features for each label. The aim of the second-layer regression is to refine the feature selection model learned from the first layer, which can be taken as a multilabel boosting process. Extensive experiments on public benchmark image data sets and real-world image data sets demonstrate that the proposed approach has better performance of multitag image annotation and leads to a quite interpretable model for image understanding.
Fusion of Multichannel Local and Global Structural Cues for Photo Aesthetics Evaluation

Photo aesthetic quality evaluation is a fundamental yet under addressed task in computer vision and image processing fields. Conventional approaches are frustrated by the following two drawbacks. First, both the local and global spatial arrangements of image regions play an important role in photo aesthetics. However, existing rules, e.g., visual balance, heuristically define which spatial distribution among the salient regions of a photo is aesthetically pleasing. Second, it is difficult to adjust visual cues from multiple channels automatically in photo aesthetics assessment. To solve these problems, we propose a new photo aesthetics evaluation framework, focusing on learning the image descriptors that characterize local and global structural aesthetics from multiple visual channels. In particular, to describe the spatial structure of the image local regions, we construct graphlets small-sized connected graphs by connecting spatially adjacent atomic regions. Since spatially adjacent graphlets distribute closely in their feature space, we project them onto a manifold and subsequently propose an embedding algorithm. The embedding algorithm encodes the photo global spatial layout into graphlets. Simultaneously, the importance of graphlets from multiple visual channels are dynamically adjusted. Finally, these post-embedding graphlets are integrated for photo aesthetics evaluation using a probabilistic model. Experimental results show that: 1) the visualized graphlets explicitly capture the aesthetically arranged atomic regions; 2) the proposed approach generalizes and improves four prominent aesthetic rules; and 3) our approach significantly outperforms state-of-the-art algorithms in photo aesthetics prediction.
Affective Visualization and Retrieval for Music Video

In modern times, music video (MV) has become an important favorite pastime to people because of its conciseness, convenience, and the ability to bring both audio and visual experiences to audiences. As the amount of MVs is explosively increasing, it has become an important task to develop new techniques for effective MV analysis, retrieval, and management. By stimulating the human affective response mechanism, affective video content analysis extracts the affective information contained in videos, and, with the affective information, natural, user-friendly, and effective MV access strategies could be developed. In this paper, a novel integrated system (i.MV) is proposed for personalized MV affective analysis, visualization, and retrieval. In i.MV, we not only perform the personalized MV affective analysis, which is a challenging and insufficiently covered problem in current affective content analysis field, but also propose novel affective visualization to convert the abstract affective states intuitive and friendly to users. Based on the affective analysis and visualization, affective information based MV retrieval is achieved. Both comprehensive experiments and subjective user studies on a large MV dataset demonstrate that our personalized affective analysis is more effective than the previous algorithms. In addition, affective visualization is proved to be more suitable for affective information-based MV retrieval than the commonly used affective state representation strategies.