nils-herrmann/clean_scirepeval huggingface dataset:
Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles

We present a novel approach for multi-object tracking which considers object detection and spacetime trajectory estimation as a coupled optimization problem. Our approach is formulated in a minimum description length hypothesis selection framework, which allows our system to recover from mismatches and temporarily lost tracks. Building upon a state-of-the-art object detector, it performs multiview/multicategory object recognition to detect cars and pedestrians in the input images. The 2D object detections are checked for their consistency with (automatically estimated) scene geometry and are converted to 3D observations which are accumulated in a world coordinate frame. A subsequent trajectory estimation module analyzes the resulting 3D observations to find physically plausible spacetime trajectories. Tracking is achieved by performing model selection after every frame. At each time instant, our approach searches for the globally optimal set of spacetime trajectories which provides the best explanation for the current image and for all evidence collected so far while satisfying the constraints that no two objects may occupy the same physical space nor explain the same image pixels at any point in time. Successful trajectory hypotheses are then fed back to guide object detection in future frames. The optimization procedure is kept efficient through incremental computation and conservative hypothesis pruning. We evaluate our approach on several challenging video sequences and demonstrate its performance on both a surveillance-type scenario and a scenario where the input videos are taken from inside a moving vehicle passing through crowded city areas.
Pedestrian detection at 100 frames per second

We present a new pedestrian detector that improves both in speed and quality over state-of-the-art. By efficiently handling different scales and transferring computation from test time to training time, detection speed is improved. When processing monocular images, our system provides high quality detections at 50 fps. We also propose a new method for exploiting geometric context extracted from stereo images. On a single CPU+GPU desktop machine, we reach 135 fps, when processing street scenes, from rectified input to detections output.
Tracker trees for unusual event detection

We present an approach for unusual event detection, based on a tree of trackers. At lower levels, the trackers are trained on broad classes of targets. At higher levels, they aim at more specific targets. For instance, at the root, a general blob tracker could operate which may track any object. The next level could already use information about human appearance to better track people. A further level could go after specific types of actions like walking, running, or sitting. Yet another level up, several walking trackers can be tuned to the gait of a particular person each. Thus, at each layer, one or more families of more specific trackers are available. As long as the target behaves according to expectations, a member of a higher up such family will be better tuned to the data than its parent tracker at a lower level. Typically, a better informed tracker performs more robustly. But in cases where unusual events occur and the normal assumptions about the world no longer hold, they loose their reliability. In such cases, a less informed tracker, not relying on what has now become false information, has a good chance of performing better. Such performance inversion signals an unusual event. Inversions between levels higher up represent deviations that are semantically more subtle than inversions lower down: for instance an unknown intruder entering a house rather than seeing a non-human target.
Robust aerial object tracking in images with lens flare

The goal of integrating drones into the civil airspace requires a technical system which robustly detects, tracks and finally avoids aerial objects. Electro-optical cameras have proven to be an adequate sensor to detect traffic, especially for smaller aircraft, gliders or paragliders. However the very challenging environmental conditions and image artifacts such as lens flares often result in a high number of false detections. Depending on the solar radiation lens flares are very common in aerial images and hard to distinguish from aerial objects on a collision course due to their similar size, shape, brightness and trajectories. In this paper we present an efficient method to detect lens flares within aerial images based on the position of the sun with respect to the observer. Using the date, time, position and attitude of the observer we predict the lens flare direction within the image. Once the direction is known the position, size and shape of the lens flares are extracted. Experiments show that our approach is able to compensate for errors in the parameters influencing the calculation of the lens flare direction. We further integrate the lens flare detection into an aerial object tracking framework. A detailed evaluation of the framework with and without lens flare filter shows that false tracks due to lens flares are successfully suppressed without degrading the overall tracking system performance.
Accurate and robust registration for in-hand modeling

We present fast 3D surface registration methods for in-hand modeling. This allows users to scan complete objects swiftly by simply turning them around in front of the scanner. The paper makes two main contributions. First, we propose an efficient method for detecting registration failures, which is a vital property of any automatic modeling system. Our method is based on two different consistency tests, one based on geometry and one based on texture. Second, we extend ICP by three additional fast registration methods for both coarse and fine alignment based on both texture and geometry. Each of those methods brings in additional information that can compensate for ambiguities in the other cues. Together, they allow for the robust reconstruction of a large variety of objects with different geometric and photometric properties. Finally, we show how both failure detection and fast registration can be combined in a practical and robust in-hand modeling system that operates at interactive frame rates.
Improved person detection in industrial environments using multiple self-calibrated cameras

Person detection is a challenging task in industrial environments which typically feature rapidly changing conditions of illumination and the presence of occluding objects and cluttered background. This paper proposes a series of algorithms for improving the robustness of person detection in such harsh industrial environments. Based on a state-of-the-art person detector, significant robustness and automation is achieved by introducing automatic ground plane estimation, confidence filtering, cross-camera correspondence estimation and multi-camera fusion. Detailed experiments made on an industrial dataset that captures an automotive assembly process show the stepwise improvement when combining the above mentioned techniques in a fully unsupervised manner.
Sparse Quantization for Patch Description

The representation of local image patches is crucial for the good performance and efficiency of many vision tasks. Patch descriptors have been designed to generalize towards diverse variations, depending on the application, as well as the desired compromise between accuracy and efficiency. We present a novel formulation of patch description, that serves such issues well. Sparse quantization lies at its heart. This allows for efficient encodings, leading to powerful, novel binary descriptors, yet also to the generalization of existing descriptors like SIFT or BRIEF. We demonstrate the capabilities of our formulation for both key point matching and image classification. Our binary descriptors achieve state-of-the-art results for two key point matching benchmarks, namely those by Brown and Mikolajczyk. For image classification, we propose new descriptors, that perform similar to SIFT on Caltech101 and PASCAL VOC07.
Wide-baseline stereo matching with line segments

We present a new method for matching line segments between two uncalibrated wide-baseline images. Most current techniques for wide-baseline matching are based on viewpoint invariant regions. Those methods work well with highly textured scenes, but fail with poorly textured ones. We show that such scenes can be successfully matched using line segments. Moreover, since line segments and regions provide complementary information, their combined matching allows to deal with a broader range of scenes. We generate an initial set of line segment correspondences, and then iteratively increase their number by adding matches consistent with the topological structure of the current ones. Finally, a coplanar grouping stage allows to estimate the fundamental matrix even from line segments only.
Robust and accurate partial surface registration based on variational implicit surfaces for automatic 3D model building

Three-dimensional models are often assembled from several partial reconstructions from unknown viewpoints. In order to provide a fully automatic, robust and accurate method for aligning and integrating partial reconstructions without any prior knowledge of the relative viewpoints of the sensor or the geometry of the imaging process, we propose a 4-step registration and integration algorithm based on a common Variational Implicit Surface (VIS) representation of the partial surface reconstructions. First, a global crude registration without a priori knowledge is performed followed by a pose refinement of partial reconstruction pairs. Pair-wise registrations are converted into a multi-view registration, before a final integration of the reconstructions into one entity or model occurs. Furthermore, making use of the smoothing properties of the VIS representations, the algorithm proves to be robust against noise in the reconstruction data. Experimental results on real-live, as well as noiseless and noisy simulated data are presented to show the feasibility, the accuracy and robustness of our registration scheme.
Real time 3D head pose estimation: Recent achievements and future challenges

Most automatic face recognition algorithms try to normalize facial images in order to remove variations caused by anything but the identity of the person. Lighting conditions being less problematic since the introduction of reliable and affordable depth sensors, head pose is the other great source of un-desired variations in facial images. In this paper, we describe recent state-of-the-art methods for real time head pose estimation from depth data, present available databases, and discuss open problems to be addressed by future research.
Repeated Pattern Detection Using CNN Activations

We propose a new approach for detecting repeated patterns on a grid in a single image. To do so, we detect repetitions in the space of pre-trained deep CNN filter responses at all layer levels. These encode features at several conceptual levels (from low-level patches to high-level semantics) as well as scales (from local to global). As a result, our repeated pattern detector is robust to challenging cases where repeated tiles show strong variation in visual appearance due to occlusions, lighting or background clutter. Our method contrasts with previous approaches that rely on keypoint extraction, description and clustering or on patch correlation. These generally only detect low-level feature clusters that do not handle variations in visual appearance of the patterns very well. Our method is simpler, yet incorporates high level features implicitly. As such, we can demonstrate detections of repetitions with strong appearance variations, organized on a nearly-regular axis-aligned grid Results show robustness and consistency throughout a varied database of more than 150 images.
Video Object Segmentation without Temporal Information

Video Object Segmentation, and video processing in general, has been historically dominated by methods that rely on the temporal consistency and redundancy in consecutive video frames. When the temporal smoothness is suddenly broken, such as when an object is occluded, or some frames are missing in a sequence, the result of these methods can deteriorate significantly. This paper explores the orthogonal approach of processing each frame independently, i.e., disregarding the temporal information. In particular, it tackles the task of semi-supervised video object segmentation: the separation of an object from the background in a video, given its mask in the first frame. We present Semantic One-Shot Video Object Segmentation (OSVOS$^\mathrm {S}$S), based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one shot). We show that instance-level semantic information, when combined effectively, can dramatically improve the results of our previous method,  We perform experiments on two recent single-object video segmentation databases, which show that OSVOS$^\mathrm {S}$S is both the fastest and most accurate method in the state of the art. Experiments on multi-object video segmentation show that OSVOS$^\mathrm {S}$S obtains competitive results.
Fast scale invariant feature detection and matching on programmable graphics hardware

Ever since the introduction of freely programmable hardware components into modern graphics hardware, graphics processing units (GPUs) have become increasingly popular for general purpose computations. Especially when applied to computer vision algorithms where a Single set of Instructions has to be executed on Multiple Data (SIMD), GPU-based algorithms can provide a major increase in processing speed compared to their CPU counterparts. This paper presents methods that take full advantage of modern graphics card hardware for real-time scale invariant feature detection and matching. The focus lies on the extraction of feature locations and the generation of feature descriptors from natural images. The generation of these feature-vectors is based on the Speeded Up Robust Features (SURF) method  due to its high stability against rotation, scale and changes in lighting condition of the processed images. With the presented methods feature detection and matching can be performed at framerates exceeding 100 frames per second for 640 times 480 images. The remaining time can then be spent on fast matching against large feature databases on the GPU while the CPU can be used for other tasks.
High speed view interpolation for tele-teaching and tele-conferencing

This paper presents an algorithm to generate an interpolated view between two camera viewpoints in a fast and automatic way (6-7 fps on a PentIV @ 2.6 GHz, Geforce FX AGP 4). Nothing more than a desktop PC and a set of low end consumer grade cameras are needed to simulate the video stream of any intermediate camera. Parallel use of the GPU ('plane sweep' algorithm) and the CPU ('min-cut/max-flow' regularisation algorithm) is made to calculate the depth values. The final interpolations for any intermediate camera position are obtained by a projectively correct blended warp of the input images on a 3D mesh. Limited extrapolation is also feasible. The goal is to develop more advanced tele-teaching and videoconferencing environments, and this without the need of many cameras. Camera movements can be simulated and the best view can be selected whether this is recorded by a real camera or not. Compared to putting a human editor in control, the cost decreases dramatically, without losing all the added value of video stream editing.
Dilemma First Search for effortless optimization of NP-hard problems

To tackle the exponentiality associated with NP-hard problems, two paradigms have been proposed. First, Branch & Bound, like Dynamic Programming, achieve efficient exact inference but requires extensive information and analysis about the problem at hand. Second, meta-heuristics are easier to implement but comparatively inefficient. As a result, a number of problems have been left unoptimized and plain greedy solutions are used. We introduce a theoretical framework and propose a powerful yet simple search method called Dilemma First Search (DFS). DFS exploits the decision heuristic needed for the greedy solution for further optimization. DFS is useful when it is hard to design efficient exact inference. We evaluate DFS on two problems: First, the Knapsack problem, for which efficient algorithms exist, serves as a toy example. Second, Decision Tree inference, where state-of-the-art algorithms rely on the greedy or randomness-based solutions. We further show that decision trees benefit from optimizations that are performed in a fraction of the iterations required by a random-based search.
Real-time connectivity constrained depth map computation using programmable graphics hardware

This paper presents a passive depth map computation algorithm which runs entirely on programmable 3D graphics hardware at real-time speed. Previous implementations relied on the use of either mipmapping or correlation windows to increase the robustness of the calculated depth map. These methods, however, are inherently unable to compute the depths for fine 3D structures. To overcome this obstacle we introduce a pixel-based method which incorporates connectivity information between neighboring pixels, while maintaining the performance boost gained by implementing the algorithm on a highly parallelized architecture such as graphics hardware.
Tackling Shapes and BRDFs Head-On

In this work, we investigate the use of simple flash-based photography to capture an object's 3D shape and reflectance characteristics at the same time. The presented method is based on the principles of Structure from Motion (SfM) and Photometric Stereo (PS), yet, we make sure not to use more than readily-available consumer equipment, like a camera with flash. Starting from a SfM-generated mesh, we apply PS to refine both geometry and reflectance, where the latter is expressed in terms of data-driven Bidirectional Reflectance Distribution Function (BRDF) representations. We also introduce a novel approach to infer complete BRDFs starting from the sparsely sampled data-driven reflectance information captured with this setup. Our approach is experimentally validated by modeling several challenging objects, both synthetic and real.
Scalable multi-class object detection

Scalability of object detectors with respect to the number of classes is a very important issue for applications where many object classes need to be detected. While combining single-class detectors yields a linear complexity for testing, multi-class detectors that localize all objects at once come often at the cost of a reduced detection accuracy. In this work, we present a scalable multi-class detection algorithm which scales sublinearly with the number of classes without compromising accuracy. To this end, a shared discriminative codebook of feature appearances is jointly trained for all classes and detection is also performed for all classes jointly. Based on the learned sharing distributions of features among classes, we build a taxonomy of object classes. The taxonomy is then exploited to further reduce the cost of multi-class object detection. Our method has linear training and sublinear detection complexity in the number of classes. We have evaluated our method on the challenging PASCAL VOC'06 and PASCAL VOC'07 datasets and show that scaling the system does not lead to a loss in accuracy.
Scale-Aware Alignment of Hierarchical Image Segmentation

Image segmentation is a key component in many computer vision systems, and it is recovering a prominent spot in the literature as methods improve and overcome their limitations. The outputs of most recent algorithms are in the form of a hierarchical segmentation, which provides segmentation at different scales in a single tree-like structure. Commonly, these hierarchical methods start from some low-level features, and are not aware of the scale information of the different regions in them. As such, one might need to work on many different levels of the hierarchy to find the objects in the scene. This work tries to modify the existing hierarchical algorithm by improving their alignment, that is, by trying to modify the depth of the regions in the tree to better couple depth and scale. To do so, we first train a regressor to predict the scale of regions using mid-level features. We then define the anchor slice as the set of regions that better balance between over-segmentation and under-segmentation. The output of our method is an improved hierarchy, re-aligned by the anchor slice. To demonstrate the power of our method, we perform comprehensive experiments, which show that our method, as a post-processing step, can significantly improve the quality of the hierarchical segmentation representations, and ease the usage of hierarchical image segmentation to high-level vision tasks such as object segmentation. We also prove that the improvement generalizes well across different algorithms and datasets, with a low computational cost.
DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks

Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations – small sensor size, compact lenses and the lack of specific hardware, – impede them to achieve the quality results of DSLR cameras. In this work we present an end-to-end deep learning approach that bridges this gap by translating ordinary photos into DSLR-quality images. We propose learning the translation function using a residual convolutional neural network that improves both color rendition and image sharpness. Since the standard mean squared loss is not well suited for measuring perceptual image quality, we introduce a composite perceptual error function that combines content, color and texture losses. The first two losses are defined analytically, while the texture loss is learned in an adversarial fashion. We also present DPED, a large-scale dataset that consists of real photos captured from three different phones and one high-end reflex camera. Our quantitative and qualitative assessments reveal that the enhanced image quality is comparable to that of DSLR-taken photos, while the methodology is generalized to any type of digital camera.
Invariant-based registration of surface patches

3D shape models are often put together from several partial reconstructions. There are good algorithms available now to perform the necessary, precise registration automatically, but only after the partial reconstructions have been brought into approximate positions. This paper proposes a technique to do precisely that. Automatic 'crude registration' is illustrated for Euclidean and affine transformations between parts. The technique is based on the extraction and invariant characterisation of bitangent curve pairs. These can be found and matched efficiently. Registration of these curves yields the crude initialisation that more precise registration algorithms like ICP or mutual information maximisation can start from.