nils-herrmann/clean_scirepeval huggingface dataset:
Using stereo geometry towards accurate 3D reconstruction

This paper addresses the problem of threedimensional reconstruction and how the geometry of the stereo vision system affects the quality of such a reconstruction. We have considered the general case of non calibrated cameras with approximately known intrinsic parameters. The latter could be known either from previous experiments or from the manufacturer's specifications. In particular, the paper aims at finding the best geometric configuration of the stereo vision system(rotation and translation between the two cameras) that is the least sensitive to errors on the intrinsic parameters. Given that in most cases, in robotics for instance, we have the freedom to set-up the geometry of the stereo vision system, finding how cameras' geometry interacts with errors from different sources is a central issue. Furthermore, when the intrinsic parameters are completely unknown, we have used a single constraint from the scene that has allowed us to calculate the focal length and therefore, the Euclidean reconstruction. We have used extensive simulations where the stereo vision geometry has ranged from pure translation to general 3D motion. The obtained results have clearly shown that a pure translation is always better, as it is not significantly affected by errors on the intrinsic parameters. When assuming that the intrinsic parameters are not known, the use of the perpendicularity constraint from the scene has made it possible to avoid the classical and tedious calibration process.
Three-dimensional projective reconstruction from three views

In the computation of the 3D structure, it has been demonstrated that by introducing a third camera provides improvement over the use of two views only. In this paper, we propose a new algebraic derivation for the projection matrix of the third camera. This derivation is obtained by using the relationships between the projection matrices, the epipolar geometry and the trilinear tensors. Compared to previous methods, our method has the advantage to work in the special case where the camera centers are collinear. In addition, it has the advantage of simplicity and it is very straightforward to implement since it is a linear method. In our experiments, we used the view synthesis to high-light the quality of the computed projective structure. Tests have been performed on both synthetic and real data.
Epipole and fundamental matrix estimation using virtual parallax

The paper addresses the problem of computing the fundamental matrix which describes a geometric relationship between a pair of stereo images: the epipolar geometry. We propose a novel method based on virtual parallax. Instead of computing directly the 3/spl times/3 fundamental matrix, we compute a homography with one epipole position, and show that this is equivalent to computing the fundamental matrix. Simple equations are derived by reducing the number of parameters to estimate. As a consequence, we obtain an accurate fundamental matrix of rank two with a stable linear computation. Experiments with simulated and real images validate our method and clearly show the improvement over existing methods.<<ETX>>
A new method for linear affine self-calibration of stationary zooming stereo cameras

This paper presents a simple, yet effective, method to recover the affine structure of a scene from a (stereo) pair of stationary zooming cameras. The proposed method solely relies on point correspondences across images and no knowledge about the scene whatsoever is required. Our method exploits implicit properties of the projective camera matrices of zooming cameras and allows to estimate the affine structure of a scene by solving a linear system of equations. The 3D reconstruction results obtained by using our method, on both real and simulated data, have remarkably validated its feasibility.
GPU-Accelerated Human Detection Using Fast Directional Chamfer Matching

We present a GPU-accelerated, real-time and practical, pedestrian detection system, which efficiently computes pedestrian-specific shape and motion cues and combines them in a probabilistic manner to infer the location and occlusion status of pedestrians viewed by a stationary camera. The articulated pedestrian shape is approximated by a mean contour template, where template matching against an incoming image is carried out using line integral based, Fast Directional Chamfer Matching, employing variable scale templates (hybrid CPU-GPU). The motion cue is obtained by employing a compressed non-parametric background model (GPU). Given the probabilistic output from the two cues, the spatial configuration of hypothesized human body locations is obtained by an iterative optimization scheme taking into account the depth ordering and occlusion status of individual hypotheses. The method achieves fast computation times (32 fps) even in complex scenarios with a high pedestrian density. Employed computational schemes are described in detail and the validity of the approach is demonstrated on three PETS2009 datasets depicting increasing pedestrian density.
Memory efficient fingerprint verification

Fingerprint recognition and verification are often based on local fingerprint features, usually ridge endings or terminations, also called minutiae. By exploiting the structural uniqueness of the image region around a minutia, the fingerprint recognition performance can be significantly enhanced. However, for most fingerprint images the number of minutia image regions (MIRs) becomes dramatically large, which imposes - especially for embedded systems - an enormous memory requirement. Therefore, we are investigating different algorithms for compression of minutia regions. The requirement for these algorithms is to achieve a high compression rate (about 20) with minimum loss in the matching performance of minutia image region matching. We investigate the matching performance for compression algorithms based on the principal component and the wavelet transformation. The matching results are presented in form of normalized ROC curves and interpreted in terms of compression rates and the MIR dimension.
Reliable intruder detection using combined modalities of intensity, thermal infrared and stereo depth

The task of large-area visual monitoring for the protection of critical and public infrastructures calls for reliable automated visual surveillance systems. Reliability in this context implies that a high detection accuracy of critical events shall be maintained independent from observation conditions, appearance and pose variations of observed objects (persons, cars), while accomplishing a low-rate of false alarms. In this paper we propose a real-time intruder detection system based on a combination of multiple complementary sensor modalities. The proposed system employs a trinocular stereo setup consisting of intensity and thermal infrared cameras capable to cover an observation area of 60m deep × 20m wide where modalities of intensity, stereo depth and thermal infrared are measured and combined to detect, track and classify objects entering the observed area. The individual modalities are combined using an estimated 3d ground plane as a common reference, yielding a probabilistic occupancy map for object candidates. A fast non-parametric clustering technique well coping with noise and multiple nearby objects is used to delineate objects, taking scale information, given the ground plane, into account. The proposed system is validated on two challenging video sequences. Promising intruder detection results are presented in terms detection and false alarm rates.
Pedestrian detection using GPU-accelerated multiple cue computation

Achieving accurate pedestrian detection for practically relevant scenarios in real-time is an important problem for many applications, while representing a major scientific challenge at the same time. In this paper we present an algorithmic framework which efficiently computes pedestrian-specific shape and motion cues and combines them in a probabilistic manner to infer the location and occlusion status of pedestrians viewed by a stationary camera. The articulated pedestrian shape is represented by a set of sparse contour templates, where fast template matching against image features is carried out using integral images built along oriented scan-lines. The motion cue is obtained by employing a non-parametric background model using the YCbCr color space. Both cues are computed and evaluated on the GPU. Given the probabilistic output from the two cues the spatial configuration of hypothesized human body locations is obtained by an iterative optimization scheme taking into account the depth ordering and occlusion status of individual hypotheses. The method achieves fast computation times even in complex scenarios with a high pedestrian density. Employed computational schemes are described in detail and the validity of the approach is demonstrated on three PETS2009 datasets depicting increasing pedestrian density. Evaluation results and comparison with state of the art are presented.
Road Sign Detection from Edge Orientation Histograms

This paper presents a system for road sign detection based on edge orientation histograms. Edge orientation histograms are reliable, scale and contrast invariant features that can be extracted efficiently using integral images. A learning method is introduced that selects features based on the implicit transmission function of the designer's template to the object's appearance in the image. The system is able to detect 85% of the objects on from 12 pixels width and 95% for objects on from 24 pixels width at a low false alarm rate.
Multi-resolution binary shape tree for efficient 2D clustering

The analysis of discrete two-dimensional distributions is a relevant task in computer vision, since many intermediate representations are generated inform of a two-dimensional map. Probabilistic inference or the response of discriminative classification often yield multi-modal distributions in form of 2D digital images, where the accurate and computationally efficient delineation of structures with varying attributes such as scale, orientation and shape represents a challenge. The simplest example is non-maximum suppression, where typically the response of a center-surround structural element applied as a filter is used to suppress spurious detection responses. In this paper we propose a simple scheme which is capable to delineate the shape of arbitrary distributions around a local density maximum driven by a local binary shape model, resulting in consistent object hypotheses. We employ a coarse-to-fine analysis scheme where learned binary shapes of increasing resolution guide a shape matching process. We demonstrate applicability for delineating compact clusters in a noisy probabilistic occupancy map, and the capability for detecting structurally consistent line structures in a text detector response map. Results are compared to other spatial grouping schemes and obtained results demonstrate a fast and accurate delineation performance.
Multiple object tracking by hierarchical association of spatio-temporal data

This paper presents a data-oriented tracking framework which aims to recover the spatio-temporal trajectories for an unknown number of interacting objects appearing and disappearing at arbitrary times. Data association is performed at three-levels of a hierarchy: (i) first, trajectory segments and an associated quality measure are generated by a local analysis of the space-time distribution of observations; (ii) a conservatively constrained association step links nearby consistent segments into intermediate trajectory fragments; and (iii) a last association step taking into account all available data (observations, trajectory fragments) generates the final trajectory estimates. The association step relies on the Hungarian algorithm and it also considers detection responses below the detection threshold as evidence associated with high ambiguity. We demonstrate the feasibility of the proposed approach applied to the pedestrian tracking task on two challenging datasets.
Dense appearance modeling and efficient learning of camera transitions for person re-identification

One central task in many visual surveillance scenarios is person re-identification, i.e., recognizing an individual person across a network of spatially disjoint cameras. Most successful recognition approaches are either based on direct modeling of the human appearance or on machine learning. In this work, we aim at taking advantage of both directions of research. On the one hand side, we compute a descriptive appearance representation encoding the vertical color structure of pedestrians. To improve the classification results, we additionally estimate the transition between two cameras using a pair-wisely estimated metric. In particular, we introduce 4D spatial color histograms and adopt Large Margin Nearest Neighbor (LMNN) metric learning. The approach is demonstrated for two publicly available datasets, showing competitive results, however, on lower computational costs.
Fast human detection in crowded scenes by contour integration and local shape estimation

The complexity of human detection increases significantly with a growing density of humans populating a scene. This paper presents a Bayesian detection framework using shape and motion cues to obtain a maximum a posteriori (MAP) solution for human configurations consisting of many, possibly occluded pedestrians viewed by a stationary camera. The paper contains two novel contributions for the human detection task: 1. computationally efficient detection based on shape templates using contour integration by means of integral images which are built by oriented string scans; (2) a non-parametric approach using an approximated version of the shape context descriptor which generates informative object parts and infers the presence of humans despite occlusions. The outputs of the two detectors are used to generate a spatial configuration of hypothesized human body locations. The configuration is iteratively optimized while taking into account the depth ordering and occlusion status of the hypotheses. The method achieves fast computation times even in complex scenarios with a high density of people. Its validity is demonstrated on a substantial amount of image data using the CAVIAR and our own datasets. Evaluation results and comparison with state of the art are presented.
A pedestrian detection system combining motion detection, spatial grouping and classification

Motion is a strong cue for the pedestrian detection task. Several motion detection approaches exist which segment moving foreground regions quite reliably, nevertheless, correct estimation of a class label for the segmented objects still represents a challenge. Certain object classes such as pedestrian groups and vehicles are difficult to discriminate from each other based on the geometric properties of foreground segments only. While appearance-based detection approaches enable class-specific detectors, pose and view-point variations, small-sized objects adversely affect the detection performance. In this paper we combine a (i) motion-based detector - having the generic ability of detecting and outlining arbitrary moving objects-, and (ii) an appearance-based detector exhibiting class specificity. Scale-adaptive mean shift clustering is used to delineate regions of moving foreground. Within the delineated clusters the appearance-based pedestrian detector is used to estimate the label of the object class. For the pedestrian class, such as a group of pedestrians, a simple model-based verification is used to estimate the location of humans. The proposed use of spatial context shows to improve the performance of the overall pedestrian detection system in terms of significantly lower false alarm rates while maintaining about the same detection rate, as evaluated in the paper quantitatively. Real-time performance is achieved.
Multiple Object Tracking Using Local PCA

Tracking multiple interacting objects represents a challenging area in computer vision. The tracking problem in general can be formulated as the task of recovering the spatio-temporal trajectories for an unknown number of objects appearing and disappearing at arbitrary times. Observations are noisy, their origin is unknown, generated by true detections or false alarms. Data association and the estimation of object states are two crucial tasks to be solved in this context. This work describes a novel, computationally efficient tracking approach to generate consistent trajectories. First, trajectory segments are created by analyzing the spatio-temporal data distribution using local principal component analysis. Subsequently, linking between trajectory segments is carried out relying on spatial proximity and kinematic smoothness constraints. Tracking results are demonstrated in the context of human tracking and compared to results of a frame-to-frame-based tracking approach
Shape-based detection of humans for video surveillance applications

In this paper we describe a surveillance system that is not only able to detect blobs and track them but also determines if a blob is a person. The given blob is segmented into sub-regions. A person model is fit to these regions such that a likelihood measure is maximized. The likelihood measure depends on the number of identified body parts, their length, location, and aspect ratio. The method is translation, rotation, and scale invariant and computationally efficient. The results obtained for test video sequences are very encouraging.
Abnormal behaviour detection on queue analysis from stereo cameras

In this paper we perform an analysis of human behaviour for people standing in a queue with the aim to discover, in an unsupervised way, ongoing unusual or suspicious activities. The main activity types we focus on are detecting people loitering around the queue and people going against the flow of the queue or undertaking a suspicious path. The proposed approach works by first detecting and tracking moving individuals from a stereo depth map in real time. Activity zones (including queue zones) are then automatically learnt employing a soft computing-based algorithm which takes as input the trajectory of detected mobile objects. Statistical properties on zone occupancy and transition between zones makes it possible to discover abnormalities without the need to learn abnormal models beforehand. The approach has been tested on a dataset realistically representing a border crossing and its environment. The current results suggest that the proposed approach constitutes a robust knowledge discovery tool able to extract queue abnormalities.
AVSS2011 demo session: Real-time human detection using fast contour template matching for visual surveillance

Anthropomatics addresses the symbiosis between humans and machines, focusing on a deeper understanding of the cooperation, interaction and coexistence between humans and machines stimulating and strengthen advanced and deep research in response to the challenges of increasingly smart environments and multimodal access to various complex technical systems. At KIT the Focus Anthropomatics and Robotics - APR has been set up by a number of research groups focusing on the research field of Anthropomatics and Robotics with more than 250 researchers. Modelling humans and their capabilities requires a deep understanding of the principle of biomechanics and kinematics, as well as the underlaying neural control principles and the perceptive and actuatoric system. Modelling and understanding of the sensomotoric mechanisms, learning and developement of skills and cognititve capabilities to enable humans to interact with the world is of high importance to design technical systems operating closely and interactively with humans via various modalities like speech, haptics, vision, grasping and locomotion. Typical research fields are related to active vision, interpretation of scenes and human activities, recognition and tracking technologies multimodal & perceptual user interfaces, understanding and translation of speech. Complementary research needed is related to the retrieval & access and summarization of multimedia data sources, translation of spoken text, context aware learning computers, implicit services and many more. The robotics application field ranges from interactive industrial robotics, service robotic companions, humanoids and medical robotics. In all domains the integrating aspects are focusing on algorithms processing real word data as well as open self-organizing architectures which allow autonomy, skill an
Human tracking by mode seeking

Change detection by background subtraction is a common approach to detect moving foreground. The resulting difference image is usually thresholded to obtain objects based on pixel connectedness and resulting blob objects are subsequently tracked. This paper proposes a detection approach not requiring the binarization of the difference image. Local density maxima in the difference image - usually representing moving objects - are outlined by a fast non-parametric mean shift clustering procedure. Object tracking is carried out by updating and propagating cluster parameters over time using the mode seeking property of the mean shift procedure. For occluding targets, a fast procedure determining the object configuration maximizing image likelihood is presented. Detection and tracking results are demonstrated for a crowded scene and evaluation of the proposed tracking framework is presented.
Reliable Left Luggage Detection Using Stereo Depth and Intensity Cues

Reliable and timely detection of abandoned items in public places still represents an unsolved problem for automated visual surveillance. Typical surveilled scenarios are associated with high visual ambiguity such as shadows, occlusions, illumination changes and substantial clutter consisting of a mixture of dynamic and stationary objects. Motivated by these challenges we propose a reliable left item detection approach based on the combination of intensity and depth data from a passive stereo setup. The employed in-house developed stereo system consists of low-cost sensors and it is capable to perform detection in environments of up to 10m x 10m in size. The proposed algorithm is tested on a set of indoor sequences and compared to manually annotated ground truth data. Obtained results show that many failure modes of intensity-based approaches are absent and even small-sized objects such as a handbag can be reliably detected when left behind in a scene. The presented results display a very promising approach, which can robustly detect left luggage in dynamic environments at a close to real-time computational speed.
GPGPU-accelerated visual search in large surveillance archives

Surveillance archives encompass vast amount of data. Given the amount of data the need for search and data exploration arises naturally. Various authorities such as infrastructure operators and law enforcement agencies are confronted with search needs based on a visual description (size, color, clothing, number plates, facial biometry, etc.) and/or behavioral patterns (limping, loitering, etc.) in order to find a ”needle in a haystack” of digital data. In this paper we present a framework which allows for an efficient video archive forensic search and data exploration in an interactive manner, and exploiting hardware accelerated video analytics at the same time. Furthermore we present a query concept to facilitate and improve the search for a specific person in large video surveillance archives using a synthetic human model in a query-by-example manner. The presented overall framework combines know-how on user interfaces, computer vision algorithms and video archive management. The system is designed with an open archive interface in mind enabling it to operate with CCTV (Closed Circuit Tele Vision) video archives from a wide variety of manufacturers. (6 pages)