nils-herrmann/clean_scirepeval huggingface dataset:
Trust-region methods for real-time tracking

Optimization methods based on iterative schemes can be divided into two classes: linesearch methods and trust-region methods. While linesearch techniques are commonly found in various vision applications, not much attention is paid to trust-region methods. Motivated by the fact that linesearch methods can be considered as special cases of trust-region methods, we propose to apply trust-region methods to visual tracking problems. Our approach integrates trust-region methods with the Kullback Leibler distance to track a rigid or non-rigid object in real-time. If not limited by the speed of a camera, the algorithm can achieve frame rate above 60 fps. To justify our method, a variety of experiments/comparisons are carried out for the trust-region tracker and a linesearch-based mean-shift tracker with same initial conditions. The experimental results support our conjecture that a trust-region tracker should perform superiorly to a linesearch one.
Learning a Scene Background Model via Classification

Learning to efficiently construct a scene background model is crucial for tracking techniques relying on background subtraction. Our proposed method is motivated by criteria leading to what a general and reasonable background model should be, and realized by a practical classification technique. Specifically, we consider a two-level approximation scheme that elegantly combines the bottom-up and top-down information for deriving a background model in real time. The key idea of our approach is simple but effective: If a classifier can be used to determine which image blocks are part of the background, its outcomes can help to carry out appropriate blockwise updates in learning such a model. The quality of the solution is further improved by global validations of the local updates to maintain the interblock consistency. And a complete background model can then be obtained based on a measurement of model completion. To demonstrate the effectiveness of our method, various experimental results and comparisons are included.
Fusing generic objectness and visual saliency for salient object detection

We present a novel computational model to explore the relatedness of objectness and saliency, each of which plays an important role in the study of visual attention. The proposed framework conceptually integrates these two concepts via constructing a graphical model to account for their relationships, and concurrently improves their estimation by iteratively optimizing a novel energy function realizing the model. Specifically, the energy function comprises the objectness, the saliency, and the interaction energy, respectively corresponding to explain their individual regularities and the mutual effects. Minimizing the energy by fixing one or the other would elegantly transform the model into solving the problem of objectness or saliency estimation, while the useful information from the other concept can be utilized through the interaction term. Experimental results on two benchmark datasets demonstrate that the proposed model can simultaneously yield a saliency map of better quality and a more meaningful objectness output for salient object detection.
Probabilistic tracking with adaptive feature selection

We propose a color-based tracking framework that infers alternately an object's configuration and good color features via particle filtering. The tracker adaptively selects discriminative color features that well distinguish foregrounds from backgrounds. The effectiveness of a feature is weighted by the Kullback-Leibler observation model, which measures dissimilarities between the color histograms of foregrounds and backgrounds. Experimental results show that the probabilistic tracker with adaptive feature selection is resilient to lighting changes and background distractions.
Multi-object tracking using dynamical graph matching

We describe a tracking algorithm to address the interactions among objects, and to track them individually and confidently via a static camera. It is achieved by constructing an invariant bipartite graph to model the dynamics of the tracking process, of which the nodes are classified into objects and profiles. The best match of the graph corresponds to an optimal assignment for resolving the identities of the detected objects. Since objects may enter/exit the scene indefinitely, or when interactions occur/conclude they could form/leave a group, the number of nodes in the graph changes dynamically. Therefore it is critical to maintain an invariant property to assure that the numbers of nodes of both types are kept the same so that the matching problem is manageable. In addition, several important issues are also discussed, including reducing the effect of shadows, extracting objects' shapes, and adapting large abrupt changes in the scene background. Finally, experimental results are provided to illustrate the efficiency of our approach.
Probabilistic tracking with optimal scale and orientation selection

We describe a probabilistic framework based on a trust-region method to track rigid or non-rigid objects with automatic optimal scale and orientation selection. The approach uses a flexible probability model to represent an object by its salient features such as color or intensity gradient. Depending on the weighting scheme, features will contribute to the distribution differently according to their positions. We adopt a bivariate normal as the weighting function that only features within the induced covariance ellipse are considered. Notice that characterizing an object by a covariance ellipse makes it easier to define its orientation and scale. To perform tracking, a trust-region scheme is carried out for each image frame to detect a distribution similar to the target's accounting for the translation, scale, and orientation factors simultaneously. Unlike other work, the optimization process is executed over a continuous space. Consequently, our method is more robust and accurate as demonstrated in the experimental results.
From co-saliency to co-segmentation: An efficient and fully unsupervised energy minimization model

We address two key issues of co-segmentation over multiple images. The first is whether a pure unsupervised algorithm can satisfactorily solve this problem. Without the user's guidance, segmenting the foregrounds implied by the common object is quite a challenging task, especially when substantial variations in the object's appearance, shape, and scale are allowed. The second issue concerns the efficiency if the technique can lead to practical uses. With these in mind, we establish an MRF optimization model that has an energy function with nice properties and can be shown to effectively resolve the two difficulties. Specifically, instead of relying on the user inputs, our approach introduces a co-saliency prior as the hint about possible foreground locations, and uses it to construct the MRF data terms. To complete the optimization framework, we include a novel global term that is more appropriate to co-segmentation, and results in a submodular energy function. The proposed model can thus be optimally solved by graph cuts. We demonstrate these advantages by testing our method on several benchmark datasets.
Deep-net fusion to classify shots in concert videos

Varying types of shots is a fundamental element in the language of film, commonly used by a visual storytelling director to convey the emotion, ideas, and art. To classify such types of shots from images, we present a new framework that facilitates the intriguing task by addressing two key issues. We first focus on learning more effective features by fusing the layer-wise outputs extracted from a deep convolutional neural network (CNN), pre-trained on a large-scale dataset for object recognition. We then introduce a probabilistic fusion model, termed as error weighted deep cross-correlation model (EW-Deep-CCM), to boost the classification accuracy. Specifically, the deep neural network-based cross-correlation model (Deep-CCM) is constructed to not only model the extracted feature hierarchies of CNN independently but also relate the statistical dependencies of paired features from different layers. Then, a Bayesian error weighting scheme for classifier combination is adopted to explore the contributions from individual Deep-CCM classifiers to enhance the accuracy of shot classification. We provide extensive experimental results on a dataset of live concert videos to demonstrate the advantage of the proposed EW-Deep-CCM over existing popular fusion approaches. The video demos can be found at https://sites.google.com/site/ewdeepccm2/demo.
Local discriminant embedding and its variants

We present a new approach, called local discriminant embedding (LDE), to manifold learning and pattern classification. In our framework, the neighbor and class relations of data are used to construct the embedding for classification problems. The proposed algorithm learns the embedding for the submanifold of each class by solving an optimization problem. After being embedded into a low-dimensional subspace, data points of the same class maintain their intrinsic neighbor relations, whereas neighboring points of different classes no longer stick to one another. Via embedding, new test data are thus more reliably classified by the nearest neighbor rule, owing to the locally discriminating nature. We also describe two useful variants: two-dimensional LDE and kernel LDE. Comprehensive comparisons and extensive experiments on face recognition are included to demonstrate the effectiveness of our method.
Learning partially-observed hidden conditional random fields for facial expression recognition

This paper describes a novel graphical model approach to seamlessly coupling and simultaneously analyzing facial emotions and the action units. Our method is based on the hidden conditional random fields (HCRFs) where we link the output class label to the underlying emotion of a facial expression sequence, and connect the hidden variables to the image frame-wise action units. As HCRFs are formulated with only the clique constraints, their labeling for hidden variables often lacks a coherent and meaningful configuration. We resolve this matter by introducing a partially-observed HCRF model, and establish an efficient scheme via Bethe energy approximation to overcome the resulting difficulties in training. For real-time applications, we also propose an online implementation to perform incremental inference with satisfactory accuracy.
Exploring Depth Information for Object Segmentation and Detection

We propose a new framework for performing object segmentation and detection simultaneously. Our method leverages with an MRF graphical model that comprises two kinds of nodes and two types of labels for inference. Specifically, we decompose an image into super pixels and generate segment proposals from each super pixel. The super pixels are then duplicated to form the two types of nodes. For each segmentation node, the model is to predict the object class label, while it is to decide the label corresponding to the best segment proposal selection at each detection node. The former is clearly a segmentation problem and the latter a detection problem. We link the two tasks by establishing a unified energy function that has a joint energy term accounting for the compatibility of the segmentation and detection labelings. Marginalizing by fixing either type of variables, the energy function can be switched into the one specifically for detection or segmentation. This property enables an alternating procedure to conveniently obtain the optimal labelings. To better explain the geometry about the objects and the scene, we use the depth information so that 3-D distances between super pixels are available in computing each energy term. Experimental results on a dataset with depth information are provided to support the effectiveness of our method.
Recognizing articulated objects with information theoretic methods

This paper addresses the problem of recognizing articulated and deformable objects. In particular we are interested in human arm and leg articulations. Our approach is a Bayesian-Information integration of shape similarity and snakes, and naturally combines top-down and bottom-up algorithms. The bottom-up method extracts edges, then constructs snakes (or contours) by grouping edge elements and feeds the shape analysis. The top-down one uses shape analysis, by comparing the object model with the extracted snakes, to guide/prune the search for other snakes. The optimizations are based on Dijkstra algorithm and further pruning of this algorithm is obtained by "integration by parts". Our approach is general enough to handle three dimensional objects, but our focus here is on two dimensional contours.
On the distribution-based tracking systems

We investigate the issues of object representation and search techniques for distribution-based tracking systems. While representing objects by color distributions has the advantage to capture the essential portion of a tracked object, it generally does not handle scale changes appropriately. We thus adopt a new object representation by integrating color and edge information via two coupled weighting schemes derived from a covariance ellipse model. The representation allows a system to perform optimization over a continuous space and to yield better tracking performances. On the aspect of search techniques, we discuss two popular iterative optimization approaches: line-search and trust-region methods. We demonstrate the differences of the two by analyzing the quality of their respective solutions through numerical and real tracking examples.
Learning effective image metrics from few pairwise examples

We present a new approach to learning image metrics. The main advantage of our method lies in a formulation that requires only a few pairwise examples. Apparently, based on the little amount of side-information, it would take a very effective learning scheme to yield a useful image metric. Our algorithm achieves this goal by addressing two key issues. First, we establish a global-local (glocal) image representation that induces two structure-meaningful vector spaces to respectively describe the global and the local image properties. Second, we develop a metric optimization framework that finds an optimal bilinear transform to best explain the given side-information. We emphasize it is the glocal image representation that makes the use of bilinear transform more powerful. Experimental results on classifications of face images and visual tracking are included to demonstrate the contributions of the proposed method
Recursive reduction net for large-scale high-dimensional data

Performing dimensionality reduction on features is essential in tackling a majority of large-scale computer vision and pattern recognition problems. The popularity of adopting high-dimensional descriptors has caused conventional techniques such as PCA inefficient or even unfeasible. We introduce an unsupervised deep-net approach, termed as recursive reduction net (RRN), to carrying out dimensionality reduction for large-scale high-dimensional data. The proposed iterative algorithm is designed to learn how to merge piecewise reduction results effectively. To this end, we use PCA as the teacher model to establish a reduction net and a fusion net, respectively. To demonstrate the usefulness of RRN, we evaluate the property of variance explaining and carry out extensive experiments on similarity search via binary coding, which would benefit from a proper dimensionality-reduction scheme.
A generalized shape-axis model for planar shapes

We describe a generalized shape-axis (SA) model for representing both open and closed planar curves. The SA model is an effective way to represent shapes by comparing their self-similarities. Given a 2D shape, whether it is closed or open, we use two different parametrizations for the curve. To study the self-similarity, the two parametrizations are matched to each other via a variational framework, where the self-similarity criterion is to be defined depending on the class of shapes and human perception factors. Useful self-similarity criteria include symmetry, parallelism and convexity, etc. A match is allowed to have discontinuities, and the optimal match can be computed by a dynamic programming algorithm in O(N/sup 4/) time, where N is the size of the shape. We use a grouping process for the shape axis to construct a unique SA-tree, however, when a planar shape is open, it is possible to derive an SA-forest. The generalized SA model provides a compact and informative way for 2D shape representation.
Approximate tree matching and shape similarity

We present a framework for 2D shape contour (silhouette) comparison that can account for stretchings, occlusions and region information. Topological changes due to the original 3D scenarios and articulations are also addressed. To compare the degree of similarity between any two shapes, our approach is to represent each shape contour with a free tree structure derived from a shape axis (SA) model, which we have recently proposed. We then use a tree matching scheme to find the best approximate match and the matching cost. To deal with articulations, stretchings and occlusions, three local tree matching operations, merge, cut, and merge-and-cut, are introduced to yield optimally approximate matches, which can accommodate not only one-to-one but many-to-many mappings. The optimization process gives guaranteed globally optimal match efficiently. Experimental results on a variety of shape contours are provided.
Coherent Deep-Net Fusion To Classify Shots In Concert Videos

Varying types of shots is a fundamental element in the language of film, commonly used by a visual storytelling director. The technique is often used in creating professional recordings of a live concert, but meanwhile may not be appropriately applied in audience recordings of the same event. Such variations could cause the task of classifying shots in concert videos, professional or amateur, very challenging. To achieve more reliable shot classification, we propose a novel probabilistic-based approach, named as coherent classification net (CC-Net), by addressing three crucial issues. First, we focus on learning more effective features by fusing the layer-wise outputs extracted from a deep convolutional neural network (CNN), pretrained on a large-scale data set for object recognition. Second, we introduce a frame-wise classification scheme, the error weighted deep cross-correlation model (EW-Deep-CCM), to boost the classification accuracy. Specifically, the deep neural network-based cross-correlation model (deep-CCM) is constructed to not only model the extracted feature hierarchies of CNN independently, but also relate the statistical dependencies of paired features from different layers. Then, a Bayesian error weighting scheme for a classifier combination is adopted to explore the contributions from individual Deep-CCM classifiers to enhance the accuracy of shot classification in each image frame. Third, we feed the frame-wise classification results to a linear-chain conditional random field module to refine the shot predictions by taking into account the global and temporal regularities. We provide extensive experimental results on a data set of live concert videos to demonstrate the advantage of the proposed CC-Net over existing popular fusion approaches for shot classification.
Representation and Self-Similarity of Shapes

Representing shapes in a compact and informative form is a significant problem for vision systems that must recognize or classify objects. We describe a compact representation model for two-dimensional (2D) shapes by investigating their self-similarities and constructing their shape axis trees (SA-trees). Our approach can be formulated as a variational one (or, equivalently, as MAP estimation of a Markov random field). We start with a 2D shape, its boundary contour, and two different parameterizations for the contour (one parameterization is oriented counterclockwise and the other clockwise). To measure its self-similarity, the two parameterizations are matched to derive the best set of one-to-one point-to-point correspondences along the contour. The cost functional used in the matching may vary and is determined by the adopted self-similarity criteria, e.g., cocircularity, distance variation, parallelism, and region homogeneity. The loci of middle points of the pairing contour points yield the shape axis and they can be grouped into a unique free tree structure, the SA-tree. By implicitly encoding the (local and global) shape information into an SA-tree, a variety of vision tasks, e.g., shape recognition, comparison, and retrieval, can be performed in a more robust and efficient way via various tree-based algorithms. A dynamic programming algorithm gives the optimal solution in O(N/sup 1/), where N is the size of the contour.
Finding Familiar Objects and their Depth from a Single Image

We present a classification-based method to identify objects of interest, and judge their depth in a single image. Our approach is motivated by a postulate of human depth perception that people can give a credible depth estimation for an object whose familiar size is known, even without using stereo vision. To emulate the mechanism, we categorize objects into the same class if they have similar sizes and shapes, and model the sense of discovering a familiar object by applying multiple kernel logistic regression to the conditional probability of feature types. The depth of a detected target can then be obtained by referencing its corresponding object category. Overall, the proposed algorithm is efficient in both the training and testing phases, and does not require a large amount of training images for good performances.
A probabilistic SVM approach for background scene initialization

Visual tracking systems using background subtraction have been very popular largely due to their efficiency in extracting moving objects. However, such systems often compute the reference background by assuming no moving objects are present during the initialization stage, though the assumption may not be realistic. We propose an automatic way to perform background initialization using a probabilistic SVM (support vector machine). By formulating the problem as an on-line classification one, our approach has the potential to be real-time. SVM classification is carried out for all elements of each image frame by computing the output probabilities. Newly found background elements are evaluated and determined if they should be added to the solution. The process of background initialization continues until there are no more new background elements to be considered. As the features used in an SVM dictate the outcome of classification, we find that optical flow value and inter-frame difference are the two most important ones. Experimental results are included to demonstrate the efficiency of our method.