nils-herrmann/clean_scirepeval huggingface dataset:
Distance Map of Various Weights: A new feature for adaptive object tracking

In this paper, we propose a new feature, Distance Map of Various Weights (DMVW) based on distances between rows' textures, to perform tracking. The proposed new feature provides an effective object appearance model which is both illumination-invariant and robust to occlusion. We also develop a 2D PCA based method to effectively evaluate the new feature. We demonstrate the validity of the rows' or column's weights in computing 2D PCA subspaces. To balance the importance of local and global information, we define a coefficient to revise the locality extent of the proposed feature. A new method based on entropy of candidate state evaluation is proposed to select the most discriminative coefficient. Experimental results on challenging video sequences demonstrated the effectiveness of our method.
FatRegion: A Fast Adaptive Tree-Structured Region Extraction Approach

Coherent image regions can be used as good features for many computer vision tasks, such as object tracking, segmentation, and recognition. Most of previous region extraction methods, however, are not suitable for online applications because of their either heavy computations or unsatisfactory results. We propose a seed-based region growing and merging approach to generate simultaneously coherent and discriminative image regions. We present a quadtree-based seed initialization algorithm to adaptively place seeds into different image areas and then grow them into regions by a color- and edge-guided growing procedure. To merge these regions in different levels, we propose to use the generalized boundary strength to measure the quality of region merging result. In addition, we present a region merging algorithm of linear time complexity to perform efficient and effective region merging. Overall, our new approach simultaneously holds these advantages: 1) it is extremely fast with linear complexity in both time and space, which takes less than 50 ms to process an HVGA image; 2) it can give a direct control of the region number and well adapt to image regions with various sizes and shapes; and 3) it provides a tree-structured representation of the regions and thus can model the image from multiple scales. We evaluate the proposed approach on the standard benchmarks with extensive comparisons with the state-of-the-art methods. The experimental results demonstrate its good comprehensive performances. Example applications using the extracted regions as features for online object tracking and multiclass object segmentation also exhibit its potential for many computer vision tasks.
Multi-feature max-margin hierarchical Bayesian model for action recognition

In this paper, a multi-feature max-margin hierarchical Bayesian model (M3HBM) is proposed for action recognition. Different from existing methods which separate representation and classification into two steps, M3HBM jointly learns a high-level representation by combining a hierarchical generative model (HGM) and discriminative max-margin classifiers in a unified Bayesian framework. Specifically, HGM is proposed to represent actions by distributions over latent spatial temporal patterns (STPs) which are learned from multiple feature modalities and shared among different classes. For recognition, we employ Gibbs classifiers to minimize the expected loss function based on the max-margin principle and use the classifiers as regularization terms of M3HBM to perform Bayeisan estimation for classifier parameters together with the learning of STPs. In addition, multi-task learning is applied to learn the model from multiple feature modalities for different classes. For test videos, we obtain the representations by the inference process and perform action recognition by the learned Gibbs classifiers. For the learning and inference process, we derive an efficient Gibbs sampling algorithm to solve the proposed M3HBM. Extensive experiments on several datasets demonstrate both the representation power and the classification capability of our approach for action recognition.
Active Contour-Based Visual Tracking by Integrating Colors, Shapes, and Motions

In this paper, we present a framework for active contour-based visual tracking using level sets. The main components of our framework include contour-based tracking initialization, color-based contour evolution, adaptive shape-based contour evolution for non-periodic motions, dynamic shape-based contour evolution for periodic motions, and the handling of abrupt motions. For the initialization of contour-based tracking, we develop an optical flow-based algorithm for automatically initializing contours at the first frame. For the color-based contour evolution, Markov random field theory is used to measure correlations between values of neighboring pixels for posterior probability estimation. For adaptive shape-based contour evolution, the global shape information and the local color information are combined to hierarchically evolve the contour, and a flexible shape updating model is constructed. For the dynamic shape-based contour evolution, a shape mode transition matrix is learnt to characterize the temporal correlations of object shapes. For the handling of abrupt motions, particle swarm optimization is adopted to capture the global motion which is applied to the contour in the current frame to produce an initial contour in the next frame.
Bootstrapping deep feature hierarchy for pornographic image recognition

Automatically recognizing pornographic images from the Web is a vital step to purify Internet environment. Inspired by the rapid developments of deep learning models, we present a deep architecture of convolutional neural network (CNN) for high accuracy pornographic image recognition. The proposed architecture is built upon existing CNNs which accepts input images of different sizes and incorporates features from different hierarchy to perform prediction. To effectively train the model, we propose a two-stage training strategy to learn the model parameters from scratch and end-to-end. During the training procedure, we also employ a hard negative sampling strategy to further reduce the false positive rate of the model. Experimental results on a large dataset demonstrate good performance of the proposed model and the effectiveness of our training strategies, with a considerable improvement over some traditional methods using hand-crafted features and deep learning method using mainstream CNN architecture.
Discriminative Level Set for Contour Tracking

Conventional contour tracking algorithms with level set often use generative models to construct the energy function. For tracking through cluttered and noisy background, however, a generative model may not be discriminative enough. In this paper we integrate the discriminative methods into a level set framework when constructing the level set energy function. We train a set of weak classifiers to distinguish the object from the background. Each weak classifier is designed to select the most discriminative feature space and integrated via AdaBoost according to their training errors. We also introduce a novel interaction term to explore the correlation between pixels near the object edge. This term together with the discriminative model both enhance the discriminative power of the level set. The experimental results show that the contour tracked by our approach is more accurate than the conventional algorithms with the generative model. Our algorithm successfully tracks the object contour even in a cluttered environment.
Multi-target Tracking by Rank-1 Tensor Approximation

In this paper we formulate multi-target tracking (MTT) as a rank-1 tensor approximation problem and propose an ℓ1 norm tensor power iteration solution. In particular, a high order tensor is constructed based on trajectories in the time window, with each tensor element as the affinity of the corresponding trajectory candidate. The local assignment variables are the ℓ1 normalized vectors, which are used to approximate the rank-1 tensor. Our approach provides a flexible and effective formulation where both pairwise and high-order association energies can be used expediently. We also show the close relation between our formulation and the multi-dimensional assignment (MDA) model. To solve the optimization in the rank-1 tensor approximation, we propose an algorithm that iteratively powers the intermediate solution followed by an ℓ1 normalization. Aside from effectively capturing high-order motion information, the proposed solver runs efficiently with proved convergence. The experimental validations are conducted on two challenging datasets and our method demonstrates promising performances on both.
SVD based Kalman particle filter for robust visual tracking

Object tracking is one of the most important tasks in computer vision. The unscented particle filter algorithm has been extensively used to tackle this problem and achieved a great success, because it uses the UKF (unscented Kalman filter) to generate a sophisticated proposal distributions which incorporates the newest observations into the state transition distribution and thus overcomes the sample impoverishment problem suffered by the particle filter. However, UKF often encounters the ill-conditioned problem when solving the square root of the covariance matrix in practice. In this paper, we propose a novel Kalman particle filter based on SVD (singular value decomposition), and apply it for visual tracking. Experimental results demonstrate that, compared with the particle filter and the unscented particle filter, the proposed algorithm is more robust in tracking performance.
Visual tracking via incremental Log-Euclidean Riemannian subspace learning

Recently, a novel Log-Euclidean Riemannian metric is proposed for statistics on symmetric positive definite (SPD) matrices. Under this metric, distances and Riemannian means take a much simpler form than the widely used affine-invariant Riemannian metric. Based on the Log-Euclidean Riemannian metric, we develop a tracking framework in this paper. In the framework, the covariance matrices of image features in the five modes are used to represent object appearance. Since a nonsingular covariance matrix is a SPD matrix lying on a connected Riemannian manifold, the Log-Euclidean Riemannian metric is used for statistics on the covariance matrices of image features. Further, we present an effective online Log-Euclidean Riemannian subspace learning algorithm which models the appearance changes of an object by incrementally learning a low-order Log-Euclidean eigenspace representation through adaptively updating the sample mean and eigenbasis. Tracking is then led by the Bayesian state inference framework in which a particle filter is used for propagating sample distributions over the time. Theoretic analysis and experimental evaluations demonstrate the promise and effectiveness of the proposed framework.
Bin Ratio-Based Histogram Distances and Their Application to Image Classification

Large variations in image background may cause partial matching and normalization problems for histogram-based representations, i.e., the histograms of the same category may have bins which are significantly different, and normalization may produce large changes in the differences between corresponding bins. In this paper, we deal with this problem by using the ratios between bin values of histograms, rather than bin values' differences which are used in the traditional histogram distances. We propose a bin ratio-based histogram distance (BRD), which is an intra-cross-bin distance, in contrast with previous bin-to-bin distances and cross-bin distances. The BRD is robust to partial matching and histogram normalization, and captures correlations between bins with only a linear computational complexity. We combine the BRD with the ℓ1 histogram distance and the χ2 histogram distance to generate the ℓ1 BRD and the χ2 BRD, respectively. These combinations exploit and benefit from the robustness of the BRD under partial matching and the robustness of the ℓ1 and χ2 distances to small noise. We propose a method for assessing the robustness of histogram distances to partial matching. The BRDs and logistic regression-based histogram fusion are applied to image classification. The experimental results on synthetic data sets show the robustness of the BRDs to partial matching, and the experiments on seven benchmark data sets demonstrate promising results of the BRDs for image classification.
A Survey on Visual Content-Based Video Indexing and Retrieval

Video indexing and retrieval have a wide spectrum of promising applications, motivating the interest of researchers worldwide. This paper offers a tutorial and an overview of the landscape of general strategies in visual content-based video indexing and retrieval, focusing on methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, video retrieval including query interfaces, similarity measure and relevance feedback, and video browsing. Finally, we analyze future research directions.
Learning silhouette dynamics for human action recognition

In this paper, we address the problem of recognizing human actions with motion dynamics alone. For this purpose, we propose to use silhouette sequences to represent the human actions by discarding the appearance information, and then model the sequences with linear dynamical systems (LDSs). Recognition is achieved by directly comparing the distance between LDSs, rather than resorting to complex Bayesian learning and inference. In particular, we introduce an efficient optimization method to learn robust LDSs, and develop a shift invariant distance metric to measure the similarity on the LDSs space. We evaluate our approach on the human action data set and achieve comparable results.
Multi-Perspective Cost-Sensitive Context-Aware Multi-Instance Sparse Coding and Its Application to Sensitive Video Recognition

With the development of video-sharing websites, P2P, micro-blog, mobile WAP websites, and so on, sensitive videos can be more easily accessed. Effective sensitive video recognition is necessary for web content security. Among web sensitive videos, this paper focuses on violent and horror videos. Based on color emotion and color harmony theories, we extract visual emotional features from videos. A video is viewed as a bag and each shot in the video is represented by a key frame which is treated as an instance in the bag. Then, we combine multi-instance learning (MIL) with sparse coding to recognize violent and horror videos. The resulting MIL-based model can be updated online to adapt to changing web environments. We propose a cost-sensitive context-aware multi- instance sparse coding (MI-SC) method, in which the contextual structure of the key frames is modeled using a graph, and fusion between audio and visual features is carried out by extending the classic sparse coding into cost-sensitive sparse coding. We then propose a multi-perspective multi- instance joint sparse coding (MI-J-SC) method that handles each bag of instances from an independent perspective, a contextual perspective, and a holistic perspective. The experiments demonstrate that the features with an emotional meaning are effective for violent and horror video recognition, and our cost-sensitive context-aware MI-SC and multi-perspective MI-J-SC methods outperform the traditional MIL methods and the traditional SVM and KNN-based methods.
Indexing and Matching of Video Shots Based on Motion and Color Analysis

This paper concerns two fundamental issues in video shots retrieval: key frame identification and similarity measurement between the key frames. We propose a simple key frame extraction algorithm based on optical flow. The algorithm emphasizes the motion extremum in the shot. Color histograms in HSV color space are adopted to describe the content of the extracted key frames and a new model is proposed to measure the similarity between the key frames from different shots. Preliminary experiments have shown that the proposed method outperforms existing ones in retrieving sport video shots
Image Set Classification Using Multi-layer Multiple Instance Learning with Application to Cannabis Website Classification

We propose using multi-layer multiple instance learning (MMIL) for image set classification and applying it to the task of cannabis website classification. We treat each image as an instance in an image set, then each image is further viewed as containing instances of local image patches. This representation naturally extends traditional multiple instance learning (MIL) to multi-layers. We then show that, when using the set kernels for all layers, an MMIL problem can be flattened to a simple one-layer MIL. This flattening, when combined with quantized local image patch representation, drastically improves the computational efficiency by two orders. The flattened set kernel is further improved by weighted codewords and an exponential kernel. The proposed approach is applied to a cannabis website classification task, in which we collected a dataset containing more than 220,000 images from 600 websites. In the experiments our approach compares favorably with several state-of-the-art methods.
Range Flow in Varying Illumination: Algorithms and Comparisons

We extend estimation of range flow to handle brightness changes in image data caused by inhomogeneous illumination. Standard range flow computes 3D velocity fields using both range and intensity image sequences. Toward this end, range flow estimation combines a depth change model with a brightness constancy model. However, local brightness is generally not preserved when object surfaces rotate relative to the camera or the light sources, or when surfaces move in inhomogeneous illumination. We describe and investigate different approaches to handle such brightness changes. A straightforward approach is to prefilter the intensity data such that brightness changes are suppressed, for instance, by a highpass or a homomorphic filter. Such prefiltering may, though, reduce the signal-to-noise ratio. An alternative novel approach is to replace the brightness constancy model by 1) a gradient constancy model, or 2) by a combination of gradient and brightness constancy constraints used earlier successfully for optical flow, or 3) by a physics-based brightness change model. In performance tests, the standard version and the novel versions of range flow estimation are investigated using prefiltered or nonprefiltered synthetic data with available ground truth. Furthermore, the influences of additive Gaussian noise and simulated shot noise are investigated. Finally, we compare all range flow estimators on real data.
Segmentation, tracking, and analysis of focal adhesion dynamics in cellular microscopy imaging

Cell adhesion, a process mediated by the formation of discrete structures known as focal adhesions (FAs), is pivotal to many biological events including cell motility. Much is known about the molecular composition of FAs, although our knowledge of the spatio-temporal recruitment and the amount of the individual components present in the FAs is still incomplete. To fill this gap, an essential prerequisite is a reliable procedure for the recognition, segmentation, and tracking of FAs. While manual segmentation and tracking may provide some advantages when done by an expert, its performance is usually hindered by subjective judgement and long time required to analyse large data sets. Here, we develop a model-based segmentation and tracking algorithm that overcomes these problems. We found that our algorithm matches FA segmentation done by an expert. In addition, it contains a dedicated computational approach to correct single frame segmentation errors that may arise from the analysis of poorly defined FAs. We believe that our work establishes the basis for a comprehensive qualitative and quantitative analysis of FA dynamics and the later development of mathematical models that simulate FAs behaviour.
MATLAB Meets LEGO Mindstorms—A Freshman Introduction Course Into Practical Engineering

In today's teaching and learning approaches for first-semester students, practical courses more and more often complement traditional theoretical lectures. This practical element allows an early insight into the real world of engineering, augments student motivation, and enables students to acquire soft skills early. This paper describes a new freshman introduction course into practical engineering, which has been established within the Bachelor of Science curriculum of Electrical Engineering and Information Technology of RWTH Aachen University, Germany. The course is organized as an eight-day, full-time block laboratory for over 300 freshman students, who were supervised by more than 60 tutors from 23 institutes of the Electrical Engineering Department. Based on a threefold learning concept comprising mathematical methods, MATLAB programming, and practical engineering, the students were required to transfer mathematical basics to algorithms in MATLAB in order to control LEGO Mindstorms robots. Toward this end, a new toolbox, called the ¿RWTH-Mindstorms NXT Toolbox,¿ was developed, which enables the robots to be controlled remotely via MATLAB from a host computer. This paper describes how the laboratory course is organized and how it induces students to think as actual engineers would in solving real-world tasks with limited resources. Evaluation results show that the project improves the students' MATLAB programming skills, enhances motivation, and enables a peer learning process.
Shift variance, cyclostationarity and expected shift variance in multirate LPSV systems

The shift variant properties of linear periodically shift variant (LPSV) systems are generally analysed using deterministic signals. Shift variance is, however, closely related to cyclostationarities introduced by the LPSV system into originally wide sense stationary (WSS) random signals passing through the system. We first develop a Hilbert space distance measure for shift variance of LPSV operators with deterministic input signals. For an LPSV system with WSS random input signals, we examine the covariance operator associated to its wide-sense cyclostationary (WSCS) output signal. The output signal is WSCS if its covariance operator is LPSV, and WSS if its covariance operator is shift invariant. Applying the shift variance measure to the covariance operator therefore provides a direct measure of the cyclostationarity generated by the LPSV system. To comprehensively capture the shift variant effects of the LPSV system on WSS random signals, we analyze the changes in its output signal when shift operator and system are interchanged. These changes are WSCS, and can be quantified by the expected shift variance over one cycle. We apply the framework to various two-channel filter banks.
A novel object-oriented approach to image analysis and retrieval

Common image processing tasks such as quantitative analysis, classification, or image retrieval require content-based techniques to firstly detect visually perceivable structures that have a semantic interpretation for a specific observer in a certain context and secondly to describe their properties in a comprehensive way. To achieve these aims, we propose an object-oriented approach to image interpretation utilizing a morphological multiscale decomposition to transform an image into a hierarchical data structure that represents image objects by their topological relations and descriptive attributes. The object hierarchy can be stored in a relational image archive and serves as interface to a rule-based expert system that either evaluates image objects directly or compares them with those of the stored images. Thus, both image analysis and retrieval can be realized by appropriate queries to the expert system. The system has already been used successfully for quantitative analysis and classification of biomedical and aerial images.