nils-herrmann/clean_scirepeval huggingface dataset:
A geometric framework for dynamic vision

This thesis explores the problem of inferring information about the three-dimensional world from its projections onto a camera (images). Among all visual cues, we do not address "pictorial" ones, such as texture or shading. Instead, we concentrate on "dynamic" cues, which are associated with variations of the image over time. 
In order to eliminate pictorial cues, one may represent the world as a collection of geometric primitives, such as points, curves or surfaces in three-dimensional space. Then, from the two-dimensional motion of the projection of such primitives onto the image, one can infer the three-dimensional structure of the world and its motion relative to the viewer. 
"Three-dimensional structure from two-dimensional images" has now been a central theme in Computer Vision for over two decades, and tools from Linear Algebra and Projective Geometry have been widely employed to attack the problem as a "static" task. It is only in recent years that the role of time has started to be recognized, after the influential work of Dickmanns and his coworkers on vehicle guidance on freeways. 
We do not impose restrictions on the structure of the environment, and we cast the problem of general three-dimensional structure and motion estimation within the framework of Dynamical Systems. We show how different algebraic constraints on the image projections can be interpreted as nonlinear and implicit dynamical models whose (unknown) parameters live in peculiar differentiable manifolds that encode three-dimensional information. Recovering such three-dimensional information then amounts to identifying dynamical models while taking into account the geometry of the parameter manifolds.
Efficient visual event detection using volumetric features

This paper studies the use of volumetric features as an alternative to popular local descriptor approaches for event detection in video sequences. Motivated by the recent success of similar ideas in object detection on static images, we generalize the notion of 2D box features to 3D spatio-temporal volumetric features. This general framework enables us to do real-time video analysis. We construct a realtime event detector for each action of interest by learning a cascade of filters based on volumetric features that efficiently scans video sequences in space and time. This event detector recognizes actions that are traditionally problematic for interest point methods - such as smooth motions where insufficient space-time interest points are available. Our experiments demonstrate that the technique accurately detects actions on real-world sequences and is robust to changes in viewpoint, scale and action speed. We also adapt our technique to the related task of human action classification and confirm that it achieves performance comparable to a current interest point based human activity recognizer on a standard database of human activities.
Scale selection for classification of point-sampled 3D surfaces

Three-dimensional ladar data are commonly used to perform scene understanding for outdoor mobile robots, specifically in natural terrain. One effective method is to classify points using features based on local point cloud distribution into surfaces, linear structures or clutter volumes. But the local features are computed using 3D points within a support-volume. Local and global point density variations and the presence of multiple manifolds make the problem of selecting the size of this support volume, or scale, challenging. In this paper, we adopt an approach inspired by recent developments in computational geometry (Mitra et al., 2005) and investigate the problem of automatic data-driven scale selection to improve point cloud classification. The approach is validated with results using data from different sensors in various environments classified into different terrain types (vegetation, solid surface and linear structure).
Incorporating Background Invariance into Feature-Based Object Recognition

Current feature-based object recognition methods use information derived from local image patches. For robustness, features are engineered for invariance to various transformations, such as rotation, scaling, or affine warping. When patches overlap object boundaries, however, errors in both detection and matching will almost certainly occur due to inclusion of unwanted background pixels. This is common in real images, which often contain significant background clutter, objects which are not heavily textured, or objects which occupy a relatively small portion of the image. We suggest improvements to the popular scale invariant feature transform (SIFT) which incorporate local object boundary information. The resulting feature detection and descriptor creation processes are invariant to changes in background. We call this method the background and scale invariant feature transform (BSIFT). We demonstrate BSIFT's superior performance in feature detection and matching on synthetic and natural images.
Effective Multi-Model Tracking Using Team Actuation Models

Extended Abstract Robots need to track objects. We identify two kinds of tracking problems in which: (i) the tracker is static or does not actuate on the tracked object and (ii) the tracker actu-ates on the object. This thesis focuses on the latter problem. Tracking is performed by a robot executing specific tasks acting over the object being tracked, such as a Segway RMP soccer robot grabbing and kicking a ball. Object tracking efficiency completely depends on the accuracy of the motion model and of the sensory information. The motion model of the target becomes particulary complex in our case and highly dependent on the robot's actions. In addition, this thesis also considers the challenging environment of multiple team members actuating the object being tracked. In this case, the motion can become highly discontinuous and nonlinear. We assume that robots know their own actions, and robots in a team are collaborating according to the coordination plans. We claim that the knowledge in terms of the single robot control strategy and the multi-robot coordination plan can be a valuable source of information for tracking. Communication between robots is another interesting information source. This thesis contributes an approach to incorporate prior and dynamic knowledge into tracking. The expected outcome is a considerable improvement in tracking performance. There are several approaches incorporating some kind of prior knowledge related to the general problem of tracking under no actuation. For example, hard constraints on target position, speed or acceleration have been considered in tracking problems to improve tracking performance. This kind of information is in general simple and easy to represent as a truncated density. Another example is the situation where a number of targets are moving in formation, and there is a strong dependency between the individual sensor measurements, which provides valuable information on target behavior. In terrain-aided tracking, using the ground moving target indicator (GMTI), one may have some prior information of the terrain, road maps, and visibility conditions. A related tracking approach concerned with our problem of actuation over tracked objects, considers that a Sony AIBO robot actuates on a ball and models the motion using a fixed transition model between different single models. In my current research, I have successfully introduced a preliminary tracking filter that dynamically switches among target motion models as a function of one robot's actions. This tracker, implemented using a particle filter and a dynamic Bayes network …
A Flow-Based Approach to Vehicle Detection and Background Mosaicking in Airborne Video

In this work, we address the detection of vehicles in a video stream obtained from a moving airborne platform. We propose a Bayesian framework for estimating dense optical flow over time that explicitly estimates a persistent model of background appearance. The approach assumes that the scene can be described by background and occlusion layers, estimated within an expectation-maximization framework. The mathematical formulation of the paper is an extension of the work in (H. Yalcin et al., 2005) where motion and appearance models for foreground and background layers are estimated simultaneously in a Bayesian framework.
Measures of Similarity

Quantitative evaluation and comparison of image segmentation algorithms is now feasible owing to the recent availability of collections of hand-labeled images. However, little attention has been paid to the design of measures to compare one segmentation result to one or more manual segmentations of the same image. Existing measures in statistics and computer vision literature suffer either from intolerance to labeling refinement, making them unsuitable for image segmentation, or from the existence of degenerate cases, making the process of training algorithms using the measures to be prone to failure. This paper surveys previous work on measures of similarity and illustrates scenarios where they are applicable for performance evaluation in computer vision. For the image segmentation problem, we propose a measure that addresses the above concerns and has desirable properties such as accommodation of labeling errors at segment boundaries, region sensitive refinement, and compensation for differences in segment ambiguity between images
Automatic Class Selection and Prototyping for 3-D Object Classification

Most research on 3-D object classification and recognition focuses on recognition of objects in 3-D scenes from a small database of known 3-D models. Such an approach does not scale well to large databases of objects and does not generalize well to unknown (but similar) object classification. This paper presents two ideas to address these problems (i) class selection, i.e., grouping similar objects into classes (ii) class prototyping, i.e., exploiting common structure within classes to represent the classes. At run time matching a query against the prototypes is sufficient for classification. This approach will not only reduce the retrieval time but also will help increase the generalizing power of the classification algorithm. Objects are segmented into classes automatically using an agglomerative clustering algorithm. Prototypes from these classes are extracted using one of three class prototyping algorithms. Experimental results demonstrate the effectiveness of the two steps in speeding up the classification process without sacrificing accuracy.
A spectral technique for correspondence problems using pairwise constraints

We present an efficient spectral method for finding consistent correspondences between two sets of features. We build the adjacency matrix M of a graph whose nodes represent the potential correspondences and the weights on the links represent pairwise agreements between potential correspondences. Correct assignments are likely to establish links among each other and thus form a strongly connected cluster. Incorrect correspondences establish links with the other correspondences only accidentally, so they are unlikely to belong to strongly connected clusters. We recover the correct assignments based on how strongly they belong to the main cluster of M, by using the principal eigenvector of M and imposing the mapping constraints required by the overall correspondence mapping (one-to-one or one-to-many). The experimental evaluation shows that our method is robust to outliers, accurate in terms of matching rate, while being much faster than existing methods
Semi-Supervised Self-Training of Object Detection Models

The construction of appearance-based object detection systems is time-consuming and difficult because a large number of training examples must be collected and manually labeled in order to capture variations in object appearance. Semi-supervised training is a means for reducing the effort needed to prepare the training set by training the model with a small number of fully labeled examples and an additional set of unlabeled or weakly labeled examples. In this work we present a semi-supervised approach to training object detection systems based on self-training. We implement our approach as a wrapper around the training process of an existing object detector and present empirical results. The key contributions of this empirical study is to demonstrate that a model trained in this manner can achieve results comparable to a model trained in the traditional manner using a much larger set of fully labeled data, and that a training data selection metric that is defined independently of the detector greatly outperforms a selection metric based on the detection confidence generated by the detector.
Bearings-only localization and mapping

In many applications, mobile robots must be able to localize themselves with respect to environments which are not known a priori in order to navigate and accomplish tasks. This means that the robot must be able to build a map of an unknown environment while simultaneously localizing itself within that map. The so called Simultaneous Localization and Mapping or SLAM problem is a formulation of this requirement, and has been the subject of a considerable amount of robotics research in the last decade. 
This thesis looks at the problem of localization and mapping when the only information available to the robot is measurements of relative motion and bearings to features. The relative motion sensor measures displacement from one time to the next through some means such as inertial measurement or odometry, as opposed to externally referenced position measurements like compass or GPS. The bearing sensor measures the direction toward features from the robot through a sensor such as an omnidirectional camera, as opposed to bearing and range sensors such as laser rangefinders, sonar, or millimeter wave radar. 
A full solution to the bearing-only SLAM problem must take into consideration detecting and identifying features and estimating the location of the features as well as the motion of the robot using the measurements. This thesis focuses on the estimation problem given that feature detection and data association are available. Estimation requires a solution that is fast, accurate, consistent, and robust. 
In an applied sense, this dissertation puts forth a methodology for building maps and localizing a mobile robot using odometry and monocular vision. This sensor suite is chosen for its simplicity and generality, and in some sense represents a minimal configuration for localization and mapping. 
In a broader sense, the dissertation describes a novel method for state estimation applicable to problems which exhibit particular nonlinearity and sparseness properties. The method relies on deterministic sampling in order to compute sufficient statistics at each time step in a recursive filter. The relationship of the new algorithm to bundle adjustment and Kalman filtering (including some of its variants) is discussed.
Efficient Temporal Mean Shift for Activity Recognition in Video

We propose a temporal mean shift algorithm that clusters spatio-temporal regions in video by exploiting the temporal nature of video. Extracting spatio-temporal regions is often one of the first pre-processing steps in an activity recognition system. Our key contribution is the insight that mean shift clustering can exploit the fact that there is typically very little change between successive video frames. Most of the pixels, and therefore the clusters, shift only slightly from frame to frame. Since mean shift is an iterative procedure, fewer iterations are required for convergence if the initial search is already close to the local optimum. Our temporal mean shift algorithm exploits the temporal similarity between successive frames by initializing the search using the modes found in the previous frame. Standard Mean Shift Clustering •Finds local modes in the clusters of points •Iterative gradient ascent procedure •Let fi be a d-dimensional point in a set of n feature points f1...fn. We calculate where j = 1, 2, ..., with kernel g, typically a Gaussian kernel, and bandwidth h. •The mode is the limit of the series, where y(i,j) converges to a fixed point. •The inner loop of mean shift requires a range-search of neighbors fk near y(i,j) , which is particularly time-consuming in high dimensions. •Previous methods in optimizing standard mean shift focused on reducing the time needed for range-search of neighbors. Temporal Mean Shift Clustering •We observe that in most videos, there is very little change between successive frames.
Accelerations of a 3D Tracking Method for Non-Rigid Objects

This study aims to track a non-rigidly deformable object in real-time using successive range images. Our approach assumes that the object is globally rigid and partially non-rigid during a short interval between 3D snapshots. The tracking process consists of 3 steps; estimating the rigid transformation by ICP algorithm, extracting the non-rigidly deformed areas, and estimating their deformation vectors by Thin Plate Spline Robust Point Matching (TPS-RPM) algorithm. In our past research, it took mach time for estimating the deformation vectors. Then, in this paper, techniques for accelerating the tracking process are described.
Analysis and Removal of Artifacts in 3-D LADAR Data

Errors in laser based range measurements can be divided into two categories: intrinsic sensor errors (range drift with temperature, systematic and random errors), and errors due to the interaction of the laser beam with the environment. The former have traditionally received attention and can be modeled. The latter in contrast have long been observed but not well characterized. We propose to do so in this paper. In addition, we present a sensor independent method to remove such artifacts. The objective is to improve the overall quality of 3-D scene reconstruction to perform terrain classification of scenes with vegetation.
Automatic photo pop-up

This paper presents a fully automatic method for creating a 3D model from a single photograph. The model is made up of several texture-mapped planar billboards and has the complexity of a typical children's pop-up book illustration. Our main insight is that instead of attempting to recover precise geometry, we statistically model geometric classes defined by their orientations in the scene. Our algorithm labels regions of the input image into coarse categories: "ground", "sky", and "vertical". These labels are then used to "cut and fold" the image into a pop-up model using a set of simple assumptions. Because of the inherent ambiguity of the problem and the statistical nature of the approach, the algorithm is not expected to work on every image. However. it performs surprisingly well for a wide range of scenes taken from a typical person's photo album.
Data Structure for Efficient Processing in 3-D

Autonomous navigation in natural environment requires three-dimensional (3-D) scene representation and interpretation. High density laser-based sensing is commonly used to capture the geometry of the scene, producing large amount of 3-D points with variable spatial density. We proposed a terrain classification method using such data. The approach relies on the computation of local features in 3-D using a support volume and belongs, as such, to a larger class of computational problems where range searches are necessary. This operation on traditional data structure is very expensive and, in this paper, we present an approach to address this issue. The method relies on reusing already computed data as the terrain classification process progresses over the environment representation. We present results that show significant speed improvement using ladar data collected in various environments with a ground mobile robot.
A hierarchical field framework for unified context-based classification

We present a two-layer hierarchical formulation to exploit different levels of contextual information in images for robust classification. Each layer is modeled as a conditional field that allows one to capture arbitrary observation-dependent label interactions. The proposed framework has two main advantages. First, it encodes both the short-range interactions (e.g., pixelwise label smoothing) as well as the long-range interactions (e.g., relative configurations of objects or regions) in a tractable manner. Second, the formulation is general enough to be applied to different domains ranging from pixelwise image labeling to contextual object detection. The parameters of the model are learned using a sequential maximum-likelihood approximation. The benefits of the proposed framework are demonstrated on four different datasets and comparison results are presented
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera

External calibration of a camera to a laser rangefinder is a common pre-requisite on today’s multi-sensor mobile robot platforms. However, the process of doing so is relatively poorly documented and almost always time-consuming. This document outlines an easy and portable technique for external calibration of a camera to a laser rangefinder. It describes the usage of the Laser-Camera Calibration Toolbox (LCCT), a Matlab R -based graphical user interface that is meant to accompany this document and facilitates the calibration procedure. We also summarize the math behind its development. The software is accessible online at www.cs.cmu.edu/ ranjith/lcct.html, as well as at the VMR Lab Software page at www.cs.cmu.edu/ vmr/software/ software.html .
Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study

Estimation of parameters of random field models from labeled training data is crucial for their good performance in many image analysis applications. In this paper, we present an approach for approximate maximum likelihood parameter learning in discriminative field models, which is based on approximating true expectations with simple piecewise constant functions constructed using inference techniques. Gradient ascent with these updates exhibits compelling limit cycle behavior which is tied closely to the number of errors made during inference. The performance of various approximations was evaluated with different inference techniques showing that the learned parameters lead to good classification performance so long as the method used for approximating the gradient is consistent with the inference mechanism. The proposed approach is general enough to be used for the training of, e.g., smoothing parameters of conventional Markov Random Fields (MRFs).
A Measure for Objective Evaluation of Image Segmentation Algorithms

Despite significant advances in image segmentation techniques, evaluation of these techniques thus far has been largely subjective. Typically, the effectiveness of a new algorithm is demonstrated only by the presentation of a few segmented images and is otherwise left to subjective evaluation by the reader. Little effort has been spent on the design of perceptually correct measures to compare an automatic segmentation of an image to a set of hand-segmented examples of the same image. This paper demonstrates how a modification of the Rand index, the Normalized Probabilistic Rand (NPR) index, meets the requirements of largescale performance evaluation of image segmentation. We show that the measure has a clear probabilistic interpretation as the maximum likelihood estimator of an underlying Gibbs model, can be correctly normalized to account for the inherent similarity in a set of ground truth images, and can be computed efficiently for large datasets. Results are presented on images from the publicly available Berkeley Segmentation dataset.