nils-herrmann/clean_scirepeval huggingface dataset:
Real-Time Visual Tracking via Incremental Covariance Model Update on Log-Euclidean Riemannian Manifold

Visual tracking is a challenging problem, as an object may change its appearance due to pose variations, illumination changes, and occlusions. Many algorithms have been proposed to update the target model using the large volume of available information during tracking, but at the cost of high computational complexity. To address this problem, we present a tracking approach that incrementally learns a low-dimensional covariance tensor representation on Log-Euclidean Riemannian manifold, efficiently adapting online to appearance changes of the target. Moreover, a weighting scheme is adopted to ensure less modeling power is expended fitting older observations. Tracking is then led by the Bayesian inference framework in which a particle filter is used to propagate sample distributions over time. With the help of integral images, our tracker achieves real-time performance. Extensive experiments demonstrate the effectiveness of the proposed tracking algorithm for the targets undergoing appearance variations.
Online video advertising based on user’s attention relavancy computing

Information overload has become an important problem in the Internet, and that all kinds of existing ads flood into peoplepsilas eyes causes scarcity of userpsilas attention. To provide relevant information under userpsilas control, we propose an online video advertising framework based on userpsilas attention relevancy computing. Users receive relevant video ads in exchange of their attention consumption. Multimodal concept detectors are trained to annotate the video databases, and a multimodal video ads categorization and related concept-to-ad relevancy and ad-to-concept relevancy ranking algorithm are proposed to compute userpsilas attention relevancy. Experiments and a subjective evaluation show the feasibility and effectiveness of the proposed approach.
Learning Adaptive Receptive Fields for Deep Image Parsing Network

In this paper, we introduce a novel approach to regulate receptive field in deep image parsing network automatically. Unlike previous works which have stressed much importance on obtaining better receptive fields using manually selected dilated convolutional kernels, our approach uses two affine transformation layers in the networks backbone and operates on feature maps. Feature maps will be inflated/shrinked by the new layer and therefore receptive fields in following layers are changed accordingly. By end-to-end training, the whole framework is data-driven without laborious manual intervention. The proposed method is generic across dataset and different tasks. We conduct extensive experiments on both general parsing task and face parsing task as concrete examples to demonstrate the methods superior regulation ability over manual designs.
Relaxing from Vocabulary: Robust Weakly-Supervised Deep Learning for Vocabulary-Free Image Tagging

The development of deep learning has empowered machines with comparable capability of recognizing limited image categories to human beings. However, most existing approaches heavily rely on human-curated training data, which hinders the scalability to large and unlabeled vocabularies in image tagging. In this paper, we propose a weakly-supervised deep learning model which can be trained from the readily available Web images to relax the dependence on human labors and scale up to arbitrary tags (categories). Specifically, based on the assumption that features of true samples in a category tend to be similar and noises tend to be variant, we embed the feature map of the last deep layer into a new affinity representation, and further minimize the discrepancy between the affinity representation and its low-rank approximation. The discrepancy is finally transformed into the objective function to give relevance feedback to back propagation. Experiments show that we can achieve a performance gain of 14.0% in terms of a semantic-based relevance metric in image tagging with 63,043 tags from the WordNet, against the typical deep model trained on the ImageNet 1,000 vocabulary set.
Feature Distilled Tracking

Feature extraction and representation is one of the most important components for fast, accurate, and robust visual tracking. Very deep convolutional neural networks (CNNs) provide effective tools for feature extraction with good generalization ability. However, extracting features using very deep CNN models needs high performance hardware due to its large computation complexity, which prohibits its extensions in real-time applications. To alleviate this problem, we aim at obtaining small and fast-to-execute shallow models based on model compression for visual tracking. Specifically, we propose a small feature distilled network (FDN) for tracking by imitating the intermediate representations of a much deeper network. The FDN extracts rich visual features with higher speed than the original deeper network. To further speed-up, we introduce a shift-and-stitch method to reduce the arithmetic operations, while preserving the spatial resolution of the distilled feature maps unchanged. Finally, a scale adaptive discriminative correlation filter is learned on the distilled feature for visual tracking to handle scale variation of the target. Comprehensive experimental results on object tracking benchmark datasets show that the proposed approach achieves  ${5\times }$  speed-up with competitive performance to the state-of-the-art deep trackers.
Recommendation on Flickr by combining community user ratings and item importance

Photo recommendation in photo-sharing social networks like Flickr is an important problem. Collaborative filtering is very popular, which assumes each item has the same weight for recommendation. In practice some items are representatives for a class of items and therefore are more important for recommendation. In this paper, we model the importance for items by examining sentiment from the general public towards items. Specifically we propose a model using the temporal dynamic user `favor' information to infer photo importance on Flickr. It is further combined with local community user ratings to improve the Probabilistic Matrix Factorization (PMF) framework for photo recommendation. Experiment results show the effectiveness of the proposed approach.
Context saliency based image summarization

Image summarization is to determine a smaller but faithful representation of the original visual content. In this paper, we propose a context saliency based image summarization approach, incorporating statistical saliency and geometric information as the importance measurement instead of visual saliency. To ensure image summaries to be adaptive to target device under perception constraint, we present a gridbased piecewise linear image warping scaleplate, and adopt the sweet spot evaluation to generate a flexible model combining the cropping and warping methods. Additionally, we explore potential extensions on image retargeting, thumbnail generation, digitalmatting and photo browsing. Experimental results show comparable performance compared to the-stateofart on common data sets.
Robust Commercial Retrieval in Video Streams

TV commercial video is a kind of informative medium. To fast and robustly index and retrieve commercial videos is of interest to commercial monitor, copyright protection, and commercial management, we propose a coarse-to-fine scheme to robustly retrieve commercial videos. Different from previous work using clip or key frames-based matching, our scheme has incorporated the commercial production knowledge to search the candidate commercial positions. Color and ordinal features are extracted for locating the exact commercial positions with dynamic time warping distance. Comparison experiments were carried out over TRECVID 2006 news videos and some videos from Chinese channels. Our scheme has achieved promising simulation results.
Semantic Event Extraction from Basketball Games using Multi-Modal Analysis

In this paper, we present a novel multi-modal framework for semantic event extraction from basketball games based on Webcasting text and broadcast video. We propose novel approaches to text analysis for event detection and semantics extraction, video analysis for event structure modeling and event moment detection, and text/video alignment for event boundary detection in the video. Compared with existing approaches to event detection in sports video which rely heavily on low-level features directly extracted from video itself, our approach aims to bridge the semantic gap between low-level features and high-level events and facilitates personalization of the sports video. Promising results are reported on real-world video clips by using text analysis, video analysis and text/video alignment.
Brand Image Detection in Broadcast Video Streams

Ads exist everywhere and all the time of our daily lives. Semantic analysis of ad videos is a challenge task for its creative design and dynamic content. In this paper, we put forward a novel semantic concept - brand image shared by most ad videos, which highlights the product or service and capture the viewers' attention. Some global and local features are extracted from the shot key frames to detect brand image. And supervised SVM classifiers are applied to recognized brand images. Experiments in east broadcast videos and TRECVID 2006 news videos show the promise of the proposed algorithm. Experiments also show that the brand image is a robust feature for ad block detection.
Video Reshuffling with Narratives toward Effective Video Browsing

With the rapid increasing of video cameras, large amount of video data everyday brings the problem of video storage and browsing. In this paper, we propose a novel approach to video reshuffling with a group of static images to effectively summarize the video content. Each static image called narrative is generated to depict the behavior of a specific object or a special event. Firstly background subtraction and object tracking are employed to extract the segmentations of moving objects and corresponding trajectories. After that, we apply three sampling rules to optimized select representative object samples from the spatial-temporal object tube and stitch them to the background image by Poisson blending. Experimental results show the promise of the proposed approach.
A novel contextual descriptors for category recognition

In this paper, we propose a novel contextual descriptor which combines the contextual information and local appearance. Based on Gibbs distribution, a local descriptor is designed. By assembling the contextual information and local descriptors, a new partial contextual descriptor (PCD) is finally presented. Combining pyramid match kernel (PMK) and SVM, we test our new descriptor and obtain higher average precision of classification than using local appearance descriptor.
Hand posture recognition with co-training

As an emerging human-computer interaction approach vision based hand interaction is more natural and efficient. However in order to achieve high accuracy, most of the existing hand posture recognition methods need a large number of labeled samples which is expensive or unavailable in practice. In this paper, a co-training based method is proposed to recognize different hand postures with a small quantity of labeled data. Hand postures examples are represented with different features and disparate classifiers are trained simultaneously with labeled data. Then the semi-supervised learning treats each new posture as unlabeled data and updates the classifiers in a co-training framework. Experiments show that the proposed method outperforms the traditional methods with much less labeled examples.
Bilayer Sparse Topic Model for Scene Analysis in Imbalanced Surveillance Videos

Dynamic scene analysis has become a popular research area especially in video surveillance. The goal of this paper is to mine semantic motion patterns and detect abnormalities deviating from normal ones occurring in complex dynamic scenarios. To address this problem, we propose a data-driven and scene-independent approach, namely, Bilayer sparse topic model (BiSTM), where a given surveillance video is represented by a word-document hierarchical generative process. In this BiSTM, motion patterns are treated as latent topics sparsely distributed over low-level motion vectors, whereas a video clip can be sparsely reconstructed by a mixture of topics (motion pattern). In addition to capture the characteristic of extreme imbalance between numerous typical normal activities and few rare abnormalities in surveillance video data, a one-class constraint is directly imposed on the distribution of documents as a discriminant priori. By jointly learning topics and one-class document representation within a discriminative framework, the topic (pattern) space is more specific and explicit. An effective alternative iteration algorithm is presented for the model learning. Experimental results and comparisons on various public data sets demonstrate the promise of the proposed approach.
Deep embedding network for robust age estimation

Estimating age through a single facial image is a classic and challenging topic in computer vision. Since facial images of the same age vary considerably, while those from different ages may look very similar. To address these problems, we propose an end-to-end deep embedding neural network for robust age estimation. Specifically, we jointly use classification loss and triplet-based ranking loss to train a deep embedding network, which maps the input facial images into an embedding metric space where features of the same age are compact and those from different ages are pushed away. Thus the deep embedding network can learn more discriminative features and improves the performance for age estimation. Additionally, to accelerate the convergence of the network, we adopt an online hard negative mining strategy during the triplet loss computation. Experimental results on public datasets MORPH II and FG-NET show the superiority of our approach compared to the state-of-the-art.
A robust lane detection method for autonomous car-like robot

Due to illumination variation, view changes, and complex road conditions, automatic lane detection is crucial for path finding and planning for autonomous car-like robots. In this paper, a robust lane detection method is proposed. Firstly, in order to extract the edges of lanes in traffic scenarios, we adopt an adaptive thresholding strategy to binarize a gradient image and trace edges by using their local gradient information. Moreover, by integrating gradient constraints and introducing a reverse voting strategy to the standard Hough transform, we greatly improved speed and stability of line extraction. Further, through inverse perspective mapping the endpoints of extracted lines to the world coordination, we can combine the extracted lines from different cameras. Finally, the lane could be detected by matching two points instead of two parallel lines in parameter space. Extensive experiments and comparisons show the efficiency of the proposed method.
Object tracking with part-based discriminative context models

Object tracking is a classic problem in computer vision. Part-based appearance model has been applied to object tracking and shown good performance. However, how to initialize the parts is still an open question. In this paper, we believe that the selection of discriminative parts and effectively modeling the structural context information could improve the tracking performance. Therefore, we tackle the tracking problem by discovering discriminative parts through exemplar-SVM in the initialization, and then exploit the structural relationship between discriminative context parts and the object in the process of tracking, which is consensual in the spatio-temporal domain. Experimental results demonstrate that our approach outperforms state-of-the-art trackers on benchmark videos.
Multiple deep features learning for object retrieval in surveillance videos

Efficient indexing and retrieving objects of interest from large-scale surveillance videos are a significant and challenging topic. In this study, the authors present an effective multiple deep features learning approach for object retrieval in surveillance videos. Based on the discriminative convolutional neural network (CNN), they can learn multiple deep features to comprehensively describe the visual object. To be specific, they utilise the CNN model pre-trained on ImageNet ILSVRC12 and fine-tuned on our dataset to abstract structure information. In addition, they train another CNN model supervised by 11 colour names to deliver the colour information. To improve the retrieval performance, the deep features are encoded into short binary codes by locality-sensitive hash and fused to fast retrieve the object of interest. Retrieval experiments are performed on a dataset of 100k objects extracted from multi-camera surveillance videos. Comparison results with other common visual features show the effectiveness of the proposed approach.
Image Tag Refinement With View-Dependent Concept Representations

Image tag refinement is the task of refining initial tags of an image such that the refined tags can better reflect the content of the image and, therefore, can help users better access that image. The quality of tag refinement depends on the quality of concept representations that build a mapping from concepts to visual images. While good progress was made in the past decade on tag refinement, the previous approaches only achieved a limited success due to their limited concept representations. In this paper, we show that the visual appearances of a concept consist of both a generic view and a specific view, and therefore we can comprehensively represent a concept by two components. To ensure a clean concept representation, this representation is learned on clean click-through data, where noises are greatly reduced. In the framework, a coarse-to-fine image tag refinement is proposed, which: (1) first generates an efficient star graph to find candidate tags but missing in the initial tag list of an input image and (2) guided by this view-dependent concept representation, formulates a probabilistic objective function to eliminate irrelevant tags. Extensive experiments on two widely used standard data sets (MIRFlickr-25K and NUS-WIDE-270K) demonstrate the effectiveness of our approach.
Real-time visual tracking via Incremental Covariance Tensor Learning

Visual tracking is a challenging problem, as an object may change its appearance due to pose variations, illumination changes, and occlusions. Many algorithms have been proposed to update the target model using the large volume of available information during tracking, but at the cost of high computational complexity. To address this problem, we present a tracking approach that incrementally learns a low-dimensional covariance tensor representation, efficiently adapting online to appearance changes for each mode of the target with only ̃(1) computational complexity. Moreover, a weighting scheme is adopted to ensure less modeling power is expended fitting older observations. Both of these features contribute measurably to improving overall tracking performance. Tracking is then led by the Bayesian inference framework in which a particle filter is used to propagate sample distributions over time. With the help of integral images, our tracker achieves real-time performance. Extensive experiments demonstrate the effectiveness of the proposed tracking algorithm for the targets undergoing appearance variations.
Multiple features based shared models for background subtraction

Pixel based approaches usually ignore the spatial coherence, while region based approaches are sensitive to region size and scene complexity. In this paper, we propose a robust background subtraction approach via multiple features based shared models. Each shared model is represented by a sequence of samples based on sample consensus. Each pixel dynamically searches a matched model around the neighborhood. This shared mechanism not only enhances the robustness for background noise and jitter but also significantly reduces the number of models and samples for each model. Besides, we concatenate color and texture features as multiple features according to the discriminability and complementarity, so that each pixel can find a proper model more easily. Finally, the shared models are updated by random selecting a pixel matched the model with an adaptive update rate. Experiments on ChangeDetection benchmark 2014 show that the proposed approach outperforms the state-of-the-art methods.
Consistent DNN uncertainty training and decoding for robust ASR

We consider the problem of robust automatic speech recognition (ASR) in noisy conditions. The performance improvement brought by speech enhancement is often limited by residual distortions of the enhanced features, which can be seen as a form of statistical uncertainty. Uncertainty estimation and propagation methods have recently been proposed to improve the ASR performance with deep neural network (DNN) acoustic models. However, the performance is still limited due to the use of uncertainty only during decoding. In this paper, we propose a consistent approach to account for uncertainty in the enhanced features during both training and decoding. We estimate the variance of the distortions using a DNN uncertainty estimator that operates directly in the feature maximum likelihood linear regression (fMLLR) domain and we then sample the uncertain features using the unscented transform (UT). We report the resulting ASR performance on the CHiME-2 and CHiME-3 datasets for different uncertainty estimation/propagation techniques. The proposed DNN uncertainty training method brings 4% and 8% relative improvement on these two datasets, respectively, compared to a competitive fMLLR-domain DNN acoustic modeling baseline.
Extraction of slow oxide trap profiles by low-frequency noise analysis: Application to hot-electron-induced degradation

We have developed a two-dimensional noise model based on a Green's function approach. This model allows slow trap density profiles to be determined. The model was applied in the investigation of low- frequency degradation of MOSFETs stressed by hot- carriers, and the generated slow oxide trap density profiles were deduced. For short stress times, the generated traps were localized in the LDD regions, whereas in the case of long stress times, traps were created in both the LDD and the channel regions.
Localizing an intermittent and moving sound source using a mobile robot

This paper addresses the problem of localizing and tracking one intermittent, moving sound source using a microphone array on a mobile robot. Robot motion provides a solution for estimating the distance to the source and avoiding front-back ambiguity. We propose a mixture Kalman filter (MKF) framework in order to fuse the robot motion information and the measurements taken at different poses of the robot. Experiments and statistical results demonstrate the ability of the proposed method to track one intermittent sound source in a reverberant environment where false measurements of the source angle of arrival (AoA) and the source activity often occur compared to a method that does not consider tracking source activity into account.