nils-herrmann/clean_scirepeval huggingface dataset:
Moving obstacle detection in highly dynamic scenes

We address the problem of vision-based multi-person tracking in busy pedestrian zones using a stereo rig mounted on a mobile platform. Specifically, we are interested in the application of such a system for supporting path planning algorithms in the avoidance of dynamic obstacles. The complexity of the problem calls for an integrated solution, which extracts as much visual information as possible and combines it through cognitive feedback. We propose such an approach, which jointly estimates camera position, stereo depth, object detections, and trajectories based only on visual information. The interplay between these components is represented in a graphical model. For each frame, we first estimate the ground surface together with a set of object detections. Based on these results, we then address object interactions and estimate trajectories. Finally, we employ the tracking results to predict future motion for dynamic objects and fuse this information with a static occupancy map estimated from dense stereo. The approach is experimentally evaluated on several long and challenging video sequences from busy inner-city locations recorded with different mobile setups. The results show that the proposed integration makes stable tracking and motion prediction possible, and thereby enables path planning in complex and highly dynamic scenes.
Efficient regression priors for reducing image compression artifacts

Lossy image compression allows for large storage savings but at the cost of reduced fidelity of the compressed images. There is a fair amount of literature aiming at restoration by suppressing the compression artifacts. Very recently a learned semi-local Gaussian Processes-based solution (SLGP) has been proposed with impressive results. However, when applied to top compression schemes such as JPEG 2000, the improvement is less significant. In our paper we propose an efficient novel artifact reduction algorithm based on the adjusted anchored neighborhood regression (A+), a method from image super-resolution literature. We double the relative gains in PSNR when compared with the state-of-the-art methods such as SLGP, while being order(s) of magnitude faster.
Wrong turn - No dead end: A stochastic pedestrian motion model

This paper addresses the use of social behavior models for the prediction of a pedestrian's future motion. Recently, such models have been shown to outperform simple constant velocity models in cases where data association becomes ambiguous, e.g. in case of occlusion, bad image quality, or low frame rates. However, to account for the multiple alternatives a pedestrian can choose from, one has to go beyond the currently available deterministic models. To this end, we propose a stochastic extension of a recently proposed simulation-based motion model. This new instantiation can cater for the possible behaviors in an entire scene in a multi-hypothesis approach, using a principled modeling of uncertainties. In a set of experiments for prediction and template-based tracking, we compare it to a deterministic instantiation and investigate the general value of using an advanced motion prior in tracking.
Object Referring in Visual Scene with Spoken Language

Object referring has important applications, especially for human-machine interaction. While having received great attention, the task is mainly attacked with written language (text) as input rather than spoken language (speech), which is more natural. This paper investigates Object Referring with Spoken Language (ORSpoken) by presenting two datasets and one novel approach. Objects are annotated with their locations in images, text descriptions and speech descriptions. This makes the datasets ideal for multi-modality learning. The approach is developed by carefully taking down ORSpoken problem into three sub-problems and introducing taskspecific vision-language interactions at the corresponding levels. Experiments show that our method outperforms competing methods consistently and significantly. The approach is also evaluated in the presence of audio noise, showing the efficacy of the proposed vision-language interaction methods in counteracting background noise.
Beyond semi-supervised tracking: Tracking should be as simple as detection, but not simpler than recognition

We present a multiple classifier system for model-free tracking. The tasks of detection (finding the object of interest), recognition (distinguishing similar objects in a scene), and tracking (retrieving the object to be tracked) are split into separate classifiers in the spirit of simplifying each classification task. The supervised and semi-supervised classifiers are carefully trained on-line in order to increase adaptivity while limiting accumulation of errors, i.e. drifting. In the experiments, we demonstrate real-time tracking on several challenging sequences, including multi-object tracking of faces, humans, and other objects. We outperform other on-line tracking methods especially in case of occlusions and presence of similar objects.
An Integer Linear Programming Model for View Selection on Overlapping Camera Clusters

Multi-View Stereo (MVS) algorithms scale poorly on large image sets, and quickly become unfeasible to run on a single machine with limited memory. Typical solutions to lower the complexity include reducing the redundancy of the image set (view selection), and dividing the image set in groups to be processed independently (view clustering). A novel formulation for view selection is proposed here. We express the problem with an Integer Linear Programming (ILP) model, where cameras are modeled with binary variables, while the linear constraints enforce the completeness of the 3D reconstruction. The solution of the ILP leads to an optimal subset of selected cameras. As a second contribution, we integrate ILP camera selection with a view clustering approach which exploits Leveraged Affinity Propagation (LAP). LAP clustering can efficiently deal with large camera sets. We adapt the original algorithm so that it provides a set of overlapping clusters where the minimum and maximum sizes and the number of overlapping cameras can be specified. Evaluations on four different dataset show our solution provides significant complexity reductions and guarantees near-perfect coverage, making large reconstructions feasible even on a single machine.
Derivative-Based Scale Invariant Image Feature Detector With Error Resilience

We present a novel scale-invariant image feature detection algorithm (D-SIFER) using a newly proposed scale-space optimal 10th-order Gaussian derivative (GDO-10) filter, which reaches the jointly optimal Heisenberg's uncertainty of its impulse response in scale and space simultaneously (i.e., we minimize the maximum of the two moments). The D-SIFER algorithm using this filter leads to an outstanding quality of image feature detection, with a factor of three quality improvement over state-of-the-art scale-invariant feature transform (SIFT) and speeded up robust features (SURF) methods that use the second-order Gaussian derivative filters. To reach low computational complexity, we also present a technique approximating the GDO-10 filters with a fixed-length implementation, which is independent of the scale. The final approximation error remains far below the noise margin, providing constant time, low cost, but nevertheless high-quality feature detection and registration capabilities. D-SIFER is validated on a real-life hyperspectral image registration application, precisely aligning up to hundreds of successive narrowband color images, despite their strong artifacts (blurring, low-light noise) typically occurring in such delicate optical system setups.
Deep Domain Adaptation by Geodesic Distance Minimization

In this paper, we propose a new approach called Deep LogCORAL for unsupervised visual domain adaptation. Our work builds on the recently proposed Deep CORAL method, which aims to train a convolutional neural network and simultaneously minimize the Euclidean distance of convariance matrices between the source and target domains. By observing that the second order statistical information (i.e. the covariance matrix) lies on a Riemannian manifold, we propose to use the Riemannian distance, approximated by Log-Euclidean distance, to replace the naive Euclidean distance in Deep CORAL. We also consider first-order information, and minimize the distance of mean vectors between two domains. We build an end-to-end model, in which we minimize both the classification loss, and the domain difference based on the first-order and second-order information between two domains. Our experiments on the benchmark Office dataset demonstrates the improvements of our newly proposed Deep LogCORAL approach over the Deep CORAL method, as well as the further improvement when optimizing both orders of information.
Face recognition based on regularized nearest points between image sets

In this paper, a novel regularized nearest points (RNP) method is proposed for image sets based face recognition. By modeling an image set as a regularized affine hull (RAH), two regularized nearest points (RNP), one on each image set's RAH, are automatically determined by an efficient iterative solver. The between-set distance of RNP is then defined by considering both the distance between the RNPs and the structure of image sets. Compared with the recently developed sparse approximated nearest points (SANP) method, RNP has a more concise formulation, less variables and lower time complexity. Extensive experiments on benchmark databases (e.g., Honda/UCSD, CMU Mobo and YouTube databases) clearly show that our proposed RNP consistently outperforms state-of-the-art methods in both accuracy and efficiency.
Feature article: Robust Aerial Object Tracking from an Airborne platform

While the average remaining time to collision of about 20 s is quite promising, there are also a couple of scenarios where it drops below 10 s. The main issue there is generally the low contrast between the terrain as background and the approaching traffic aircraft-a result of the exceptional dynamic range if the sun is close to the camera FOV and shady terrain is visible at the same time. These situations could certainly be improved by using a higher bit depth instead of the 8-bit used for the scenarios presented here.
Model estimation for photometric changes of outdoor planar color surfaces caused by changes in illumination and viewpoint

We compare different ways of representing the global photometric changes in image intensities caused by changes in illumination and viewpoint, aiming at a balance between goodness-of-fit and low complexity. A series of model selection tests are performed for the case of outdoor imagery consisting of several views of several instances of billboards taken under different viewing angles and different illumination (natural light). Possible candidates for a transformation model on (R,G,B) color space are investigated and different approaches for the model selection problem are considered. The results are used within ongoing research into computation of new invariant features for planar color patterns, as the model choice is an important issue to decide on when extracting invariants. These results can be of benefit to other areas of research into color pattern or object recognition.
A sensor that extracts both 3D shape and surface texture

A sensor is proposed with interesting characteristics for applications such as autonomous navigation and virtual reality. The system extracts both 3D shape and surface texture. It is based on the projection of a regular pattern, observed by a single camera. It is cheap, requiring only a slide projector and a camera. Shape and intensity are extracted from one and the same image, thereby avoiding shape/texture alignment problems. Furthermore, the system is easy to use and calibrate. Because both the shape and the intensity information is obtained from a single image, the system can be used in dynamic environments, allowing robots to navigate during data acquisition or graphics designers to model moving objects for animation and virtual reality purposes. The system combines advantages of passive and active range acquisition.
Face animation based on observed 3D speech dynamics

Realistic face animation is especially hard as we are all experts in the perception and interpretation of face dynamics. One approach is to simulate facial anatomy. Alternatively, animation can be based on first observing the visible 3D dynamics, extracting the basic modes, and then putting these together according to the required performance. This is the strategy followed in this paper, which focuses on speech. The approach follows a kind of bootstrap procedure. First, 3D shape statistics are learned from a talking face with a relatively small number of markers. A 3D reconstruction is produced at temporal intervals of 1/25 s. A topological mask of the lower half of the face is fitted to the motion. Principal component analysis (PCA) of the mask shapes reduces the dimension of the mask shape space. The result is two-fold. On the one hand, the face can be animated (in our case, it can be made to speak new sentences). On the other hand, face dynamics can be tracked in 3D without markers for performance capture.
Beyond Novelty Detection: Incongruent Events, When General and Specific Classifiers Disagree

Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of-Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.
Automatic Interactive Calibration of Multi-Projector-Camera Systems

This paper presents a novel interactive Projector Calibration for arbitrary Multi-Projector-Camera environments. The method does not require any calibration rig and is not restricted to any special arrangement of the display devices. The cameras in the system need to be precalibrated so that a common world coordinate system can be defined. Each projector is sequentially activated and a series of self-identifying tags is generated. These tags are accurately and robustly detected and must be seen by a minimum subset of the cameras. This is achieved by freely moving a target object, i.e. a sheet of paper, onto which the tags are projected. Point correspondences for each tag in threespace are computed by minimizing the reprojection errors in the relevant images and eliminating potential outliers. If a tag has been seen a sufficient number of times, it is masked out in the display so that a visual feedback of the current detection state is given to the user. The resulting 3D-point cloud coupled with the known 2D tag-position in the projector frame serves as input for a nonlinear optimization of the projector’s intrinsic and extrinsic parameters as well as distortion factors. We show that the overall procedure takes less than a minute and results in low reprojection errors.
Geometry-Aware Similarity Learning on SPD Manifolds for Visual Recognition

Symmetric positive definite (SPD) matrices have been employed for data representation in many visual recognition tasks. The success is mainly attributed to learning discriminative SPD matrices encoding the Riemannian geometry of the underlying SPD manifolds. In this paper, we propose a geometry-aware SPD similarity learning (SPDSL) framework to learn discriminative SPD features by directly pursuing a manifold-manifold transformation matrix of full column rank. Specifically, by exploiting the Riemannian geometry of the manifolds of fixed-rank positive semidefinite (PSD) matrices, we present a new solution to reduce optimization over the space of column full-rank transformation matrices to optimization on the PSD manifold, which has a well-established Riemannian structure. Under this solution, we exploit a new supervised SPDSL technique to learn the manifold–manifold transformation by regressing the similarities of selected SPD data pairs to their ground-truth similarities on the target SPD manifold. To optimize the proposed objective function, we further derive an optimization algorithm on the PSD manifold. Evaluations on three visual classification tasks show the advantages of the proposed approach over the existing SPD-based discriminant learning methods.
Stochastic predictive control of robot tracking systems with dynamic visual feedback

A vision-guided robot workstation is presented which picks up workpieces from a fast-moving conveyor belt. The role of computer vision as the feedback transducer strongly affects the closed-loop dynamics of the overall system, and a tracking controller with dynamic visual feedback is designed for achieving fast response and high control accuracy. In view of the long time delay and the heavy noise corruption embedded in visual data, the problem of visual controller design is posed in the framework of stochastic optimal control theory. The Kalman filter is chosen to estimate the state of the target motion and formulated as a joint detection and adaptive estimation method. The generalized predictive control strategy is utilized to compute the optimal path control data and implemented in a weighted version. Experimental results are given to show the effectiveness of the approach.<<ETX>>
Joint vanishing point extraction and tracking

We present a novel vanishing point (VP) detection and tracking algorithm for calibrated monocular image sequences. Previous VP detection and tracking methods usually assume known camera poses for all frames or detect and track separately. We advance the state-of-the-art by combining VP extraction on a Gaussian sphere with recent advances in multi-target tracking on probabilistic occupancy fields. The solution is obtained by solving a Linear Program (LP). This enables the joint detection and tracking of multiple VPs over sequences. Unlike existing works we do not need known camera poses, and at the same time avoid detecting and tracking in separate steps. We also propose an extension to enforce VP orthogonality. We augment an existing video dataset consisting of 48 monocular videos with multiple annotated VPs in 14448 frames for evaluation. Although the method is designed for unknown camera poses, it is also helpful in scenarios with known poses, since a multi-frame approach in VP detection helps to regularize in frames with weak VP line support.
Light Source Calibration for IBR and BTF Acquisition Setups

Working with IBR/BTF data requires a complete calibration. Modern setups speed up recordings by using multiple lamps and cameras. Therefore, the calibration task gets more time consuming and challenging, especially for an accurate calibration of multiple light sources. Most works are dedicated to the calibration of cameras, whereas the light field calibration problem remains as a rule overlooked. Our experiments have shown that the spatial variance of light strength can be vigorous, inducing serious damage to the IBR/BTF data. We propose a novel method based on Helmholtz reciprocity, which derives light field information directly from the target IBR/BTF data rather than from specially dedicated calibration objects. Instead of repeating the recording of a huge number of images, only one additional image is needed.
Efficient Model-Free Anthropometry from Depth Data

Existing depth-based approaches to predicting anthropometric measurements, such as body height, arm span and hip circumference, either directly compute the measurements on 3D point clouds, and thus are sensitive to noise, or fit a model to the observed depth values, which typically is time-consuming. In this paper, we rely on the intuition that, to predict a specific anthropometric measurement, one does not need to have detailed information about the entire body shape. We therefore introduce an approach to anthropometry based on a random regression forest trained from local depth cues. The local predictions are then accumulated into one global, image-level anthropometric measurement prediction. We introduce a forest refinement scheme, whose objective function directly relies on both the image-level prediction, as well as on the local predictions' reliability. The resulting approach has the advantage of being both computationally highly efficient and accurate.
Low-cost scene modeling using a density function improves segmentation performance

We propose a low cost and effective way to combine a free simulation software and free CAD models for modeling human-object interaction in order to improve human & object segmentation. It is intended for research scenarios related to safe human-robot collaboration (SHRC) and interaction (SHRI) in the industrial domain. The task of human and object modeling has been used for detecting activity, and for inferring and predicting actions, different from those works, we do human and object modeling in order to learn interactions in RGB-D data for improving segmentation. For this purpose, we define a novel density function to model a three dimensional (3D) scene in a virtual environment (VREP). This density function takes into account various possible configurations of human-object and object-object relationships and interactions governed by their affordances. Using this function, we synthesize a large, realistic and highly varied synthetic RGB-D dataset that we use for training. We train a random forest classifier, and the pixelwise predictions obtained is integrated as a unary term in a pairwise conditional random fields (CRF). Our evaluation shows that modeling these interactions improves segmentation performance by ~7% in mean average precision and recall over state-of-the-art methods that ignore these interactions in real-world data. Our approach is computationally efficient, robust and can run real-time on consumer hardware.
Joint integral histograms and its application in stereo matching

In this paper, we first propose a technique, referred as joint integral histograms, for weighted filtering with O(1) computational complexity. The technique is built on the classic integral images and the recent integral histograms. In a joint integral histogram, instead of remembering bin occurrences, the value at each bin indicates an integral defined by two signals. Beyond the integral histograms, our method supports weighted filtering with a more general form, where the weight could be a function of a signal different from the signal to be filtered. Then, we present a local stereo matching approach as an instantiation of the technique. Using the joint integral histograms, we achieve a speedup factor of about two orders of magnitude. Thanks to the huge speedup, the stereo method is among the best local approaches in terms of the trade-off between matching accuracy and execution speed. Experimental results demonstrate the advantages of both the joint integral histograms technique and the stereo matching approach.