nils-herrmann/clean_scirepeval huggingface dataset:
DARN: A Deep Adversarial Residual Network for Intrinsic Image Decomposition

We present a new deep supervised learning method for intrinsic decomposition of a single image into its albedo and shading components. Our contributions are based on a new fully convolutional neural network that estimates absolute albedo and shading jointly. Our solution relies on a single end-to-end deep sequence of residual blocks and a perceptually-motivated metric formed by two adversarially trained discriminators. As opposed to classical intrinsic image decomposition work, it is fully data-driven, hence does not require any physical priors like shading smoothness or albedo sparsity, nor does it rely on geometric information such as depth. Compared to recent deep learning techniques, we simplify the architecture, making it easier to build and train, and constrain it to generate a valid and reversible decomposition. We rediscuss and augment the set of quantitative metrics so as to account for the more challenging recovery of non scale-invariant quantities. We train and demonstrate our architecture on the publicly available MPI Sintel dataset and its intrinsic image decomposition, show attenuated overfitting issues and discuss generalizability to other data. Results show that our work outperforms the state of the art deep algorithms both on the qualitative and quantitative aspect.
Constant Time Joint Bilateral Filtering Using Joint Integral Histograms

In this brief, we present a constant time method for the joint bilateral filtering. First, we propose an image data structure, coined as joint integral histograms (JIHs). Extending the classic integral images and the integral histograms, it represents the global information of two correlated images. In a JIH, the value at each bin indicates an integral determined by the two images. Then, the joint bilateral filtering is transformed to computation and manipulation of histograms. Utilizing the JIHs, we are capable of joint bilateral filtering in constant time. Its performance is validated in a digital photography approach using Flash–noFlash image pairs. Compared with the brute-force method, the proposed method achieves a speedup factor of 2–3 orders of magnitude while producing similar filtering results.
Interactive object detection

In recent years, the rise of digital image and video data available has led to an increasing demand for image annotation. In this paper, we propose an interactive object annotation method that incrementally trains an object detector while the user provides annotations. In the design of the system, we have focused on minimizing human annotation time rather than pure algorithm learning performance. To this end, we optimize the detector based on a realistic annotation cost model based on a user study. Since our system gives live feedback to the user by detecting objects on the fly and predicts the potential annotation costs of unseen images, data can be efficiently annotated by a single user without excessive waiting time. In contrast to popular tracking-based methods for video annotation, our method is suitable for both still images and video. We have evaluated our interactive annotation approach on three datasets, ranging from surveillance, television, to cell microscopy.
Visual recognition of CAD objects with aspect graphs

The authors analyze how the geometrical model of objects from a CAD system can be used to build exactly a new representation of these objects that is better suited for visual recognition. A 3-D object has in general an infinite number of different 2-D images, depending on the viewpoint. Its image contours have, however, only a finite number of different topological aspects. An aspect graph is a graph in which the nodes are the stable aspects, and the arcs are the visual events that make the image contours switch from one stable aspect to another. It is shown how to compute exact aspect graphs of general CAD objects under perspective projection using results obtained here and algorithms from real algebraic geometry.<<ETX>>
Fast Compact City Modeling for Navigation Pre-Visualization

Nowadays, GPS-based car navigation systems mainly use speech and aerial views of simplified road maps to guide drivers to their destination. However, drivers often experience difficulties in linking the simple 2D aerial map with the visual impression that they get from the real environment, which is inherently ground-level based. Therefore, supplying realistically textured 3D city models at ground-level proves very useful for pre-visualizing an upcoming traffic situation. Because this pre-visualization can be rendered from the expected future viewpoints of the driver, the latter will more easily understand the required maneuver. 3D city models can be reconstructed from the imagery recorded by surveying vehicles. The vastness of image material gathered by these vehicles, however, puts extreme demands on vision algorithms to ensure their practical usability. Algorithms need to be as fast as possible and should result in compact, memory efficient 3D city models for future ease of distribution and visualization. For the considered application, these are not contradictory demands. Simplified geometry assumptions can speed up vision algorithms while automatically guaranteeing compact geometry models. We present a novel city modeling framework which builds upon this philosophy to create 3D content at high speed which could allow for pre-visualization of any conceivable traffic situation by car navigation modules.
Traffic sign recognition — How far are we from the solution?

Traffic sign recognition has been a recurring application domain for visual objects detection. The public datasets have only recently reached large enough size and variety to enable proper empirical studies. We revisit the topic by showing how modern methods perform on two large detection and classification datasets (thousand of images, tens of categories) captured in Belgium and Germany. We show that, without any application specific modification, existing methods for pedestrian detection, and for digit and face classification; can reach performances in the range of 95% ~ 99% of the perfect solution. We show detailed experiments and discuss the trade-off of different options. Our top performing methods use modern variants of HOG features for detection, and sparse representations for classification.
Nonuniform image patch exemplars for low level vision

In this paper we propose the use of nonuniformly resized image patch exemplars for solving low level vision problems like denoising and super-resolution. While patch-based methods have been shown to be successful for several such applications, these methods have so far assumed uniform sizes for image patches. In this paper we address this restriction. We use an integral image representation for efficient computation of the matching cost for variable-sized patches. We show that nonuniform image patch exemplars are useful in improving classic techniques for nonlocal means-based denoising and example-based superresolution. We provide refinement cues to further improve the patch size estimation. This method can be adopted for a large number of related methods and applications due to its simplicity and generality.
Real time head pose estimation with random regression forests

Fast and reliable algorithms for estimating the head pose are essential for many applications and higher-level face analysis tasks. We address the problem of head pose estimation from depth data, which can be captured using the ever more affordable 3D sensing technologies available today. To achieve robustness, we formulate pose estimation as a regression problem. While detecting specific face parts like the nose is sensitive to occlusions, learning the regression on rather generic surface patches requires enormous amount of training data in order to achieve accurate estimates. We propose to use random regression forests for the task at hand, given their capability to handle large training datasets. Moreover, we synthesize a great amount of annotated training data using a statistical model of the human face. In our experiments, we show that our approach can handle real data presenting large pose changes, partial occlusions, and facial expressions, even though it is trained only on synthetic neutral face data. We have thoroughly evaluated our system on a publicly available database on which we achieve state-of-the-art performance without having to resort to the graphics card.
Robust Low Complexity Corner Detector

Corner feature point detection with both the high-speed and high-quality is still very demanding for many real-time computer vision applications. The Harris and Kanade-Lucas-Tomasi (KLT) are widely adopted good quality corner feature point detection algorithms due to their invariance to rotation, noise, illumination, and limited view point change. Although they are widely adopted corner feature point detectors, their applications are rather limited because of their inability to achieve real-time performance due to their high complexity. In this paper, we redesigned Harris and KLT algorithms to reduce their complexity in each stage of the algorithm: Gaussian derivative, cornerness response, and non-maximum suppression (NMS). The complexity of the Gaussian derivative and cornerness stage is reduced by using an integral image. In NMS stage, we replaced a highly complex sorting and NMS by the efficient NMS followed by sorting the result. The detected feature points are further interpolated for sub-pixel accuracy of the feature point location. Our experimental results on publicly available evaluation data-sets for the feature point detectors show that our low complexity corner detector is both very fast and similar in feature point detection quality compared to the original algorithm. We achieve a complexity reduction by a factor of 9.8 and attain 50 f/s processing speed for images of size 640×480 on a commodity central processing unit with 2.53 GHz and 3 GB random access memory.
Hierarchical Co-Segmentation of Building Facades

We introduce a new system for automatic discovery of high-level structural representations of building facades. Under the assumption that each facade can be represented as a hierarchy of rectilinear subdivisions, our goal is to find the optimal direction of splitting, along with the number and positions of the split lines at each level of the tree. Unlike previous approaches, where each facade is analysed in isolation, we propose a joint analysis of a set of facade images. Initially, a co-segmentation approach is used to produce consistent decompositions across all facade images. Afterwards, a clustering step identifies semantically similar segments. Each cluster of similar segments is then used as the input for the joint segmentation in the next level of the hierarchy. We show that our approach produces consistent hierarchical segmentations on two different facade datasets. Furthermore, we argue that the discovered hierarchies capture essential structural information, which is demonstrated on the tasks of facade retrieval and virtual facade synthesis.
Multiple Bernoulli relevance models for image and video annotation

Retrieving images in response to textual queries requires some knowledge of the semantics of the picture. Here, we show how we can do both automatic image annotation and retrieval (using one word queries) from images and videos using a multiple Bernoulli relevance model. The model assumes that a training set of images or videos along with keyword annotations is provided. Multiple keywords are provided for an image and the specific correspondence between a keyword and an image is not provided. Each image is partitioned into a set of rectangular regions and a real-valued feature vector is computed over these regions. The relevance model is a joint probability distribution of the word annotations and the image feature vectors and is computed using the training set. The word probabilities are estimated using a multiple Bernoulli model and the image feature probabilities using a non-parametric kernel density estimate. The model is then used to annotate images in a test set. We show experiments on both images from a standard Corel data set and a set of video key frames from NIST's video tree. Comparative experiments show that the model performs better than a model based on estimating word probabilities using the popular multinomial distribution. The results also show that our model significantly outperforms previously reported results on the task of image and video annotation.
An object-dependent hand pose prior from sparse training data

In this paper, we propose a prior for hand pose estimation that integrates the direct relation between a manipulating hand and a 3d object. This is of particular interest for a variety of applications since many tasks performed by humans require hand-object interaction. Inspired by the ability of humans to learn the handling of an object from a single example, our focus lies on very sparse training data. We express estimated hand poses in local object coordinates and extract for each individual hand segment, the relative position and orientation as well as contact points on the object. The prior is then modeled as a spatial distribution conditioned to the object. Given a new object of the same object class and new hand dimensions, we can transfer the prior by a procedure involving a geometric warp. In our experiments, we demonstrate that the prior may be used to improve the robustness of a 3d hand tracker and to synthesize a new hand grasping a new object. For this, we integrate the prior into a unified belief propagation framework for tracking and synthesis.
Non-parametric motion-priors for flow understanding

We present a novel method for extracting the dominant dynamic properties of crowded scenes from a single, static, uncalibrated camera using a codebook of tracklets. Our approach relies only on tracklets of fixed length which are generated based on sparse optical flow. A grid of points is placed on the image plane and local meanshift clustering is employed to extract dominant directions of tracklets in the neighborhood. A Gaussian Process (GP) is fitted to each tracklet resulting in a codebook, with each codeword representing a local motion model. At test time, a mixture of weighted local GP experts is applied, providing multimodal density estimates for next object location and simulation of full object trajectories. Our scenarios come from challenging crowded scenes, from which we extract dominant local motion-patterns and use the model to simulate full object trajectories. In addition, we apply the learnt model to multiple object tracking. Random trajectories are sampled from the model that match the learnt scene dynamics. Minimum Description Length (MDL) is employed to pick the best trajectories in order to associate sparse detections over short time windows. Also, we modify a state-of-the-art multiple object tracking algorithm leading to significant improvement. Our results compare favorably to a state-of-the-art algorithm and we introduce a new challenging dataset for multiple object tracking.
Towards On-Line Digital Doubles

We present a modular system for real-time 3D-scanning of human bodies under motion. The high-resolution shape and colour appearance is captured by several scanning units positioned around the object of interest. Each of these units performs a foreground-background segmentation and computes a valid depth-range for the spatially neighbouring units. Multiple depth-ranges are combined in a visual hull representation, which limits the search-range for the 3D-reconstruction. Depth-estimation is based on a hierarchical multi-view-stereo plane sweep approach. Robustness and accuracy is increased by incorporating imperceptible infrared illumination as well as adding local pixel gradient information. All parts of the processing pipeline, involving camera color conversions, segmentation, depth-range computation, visual-hull generation, lossless image compression, network transfer of the infrared and colour images, and the plane sweep algorithm, are implemented on the GPU and highly optimized for speed, allowing scanning times of less than 40 ms per frame. Experimental results demonstrate the applicability of our system to the creation of high-density on-line digital doubles.
PDE-based multi-view depth estimation

The paper describes a method for depth extraction from multiple, calibrated images. Emphasis lies on the integration of multiple views during the matching process. This process is guided by the relative confidence that the system has in the data coming from the different views. This weighing is fine-grained in that it is determined for every pixel at every iteration. Reliable information spreads fast at the expense of less reliable data, both in terms of spatial communications and in terms of exchange between views. The resulting system can handle large disparities, depth discontinuities and occlusions. Moreover provisions are made to deal with intensity changes between corresponding pixels. Experimental results corroborate the viability of the approach and the improved results that can be expected from the system's ability to deal with variable intensities.
[POSTER] Augmented Reality for User-Friendly Intra-Oral Scanning

Digital impressions of teeth, obtained through intra-oral scanning, allow for more efficient and cost effective treatments of many dental indications. Current state-of-the-art intra-oral impression acquisition systems make use of a separate monitor to show the scanning progress, forcing the dentist to divert attention away from the scanner and the patient. In this paper, we present an augmented reality based solution to this problem. During the scanning process, an optical see-through head-mounted display is used to show an online overlay of the dynamic dental model onto the patient's teeth. The dentist can then fully focus on the patient and the scanner, while still being able to keep track of the current state of the model. This type of novel application, which fundamentally changes the humancomputer interaction of intra-oral scanning systems, requires a fast and accurate registration of a dynamically growing model onto a glossy, partially occluded surface at a very small scale. To meet this demand, we propose application tailored algorithms for indirect high accuracy online 3D teeth tracking and optical see-through head-mounted display calibration. Experimental results indicate that our system does have a potential to noticeably facilitate intraoral scanning in the future.
The Synthesizability of Texture Examples

Example-based texture synthesis (ETS) has been widely used to generate high quality textures of desired sizes from a small example. However, not all textures are equally well reproducible that way. We predict how synthesizable a particular texture is by ETS. We introduce a dataset (21, 302 textures) of which all images have been annotated in terms of their synthesizability. We design a set of texture features, such as 'textureness', homogeneity, repetitiveness, and irregularity, and train a predictor using these features on the data collection. This work is the first attempt to quantify this image property, and we find that texture synthesizability can be learned and predicted. We use this insight to trim images to parts that are more synthesizable. Also we suggest which texture synthesis method is best suited to synthesise a given texture. Our approach can be seen as 'winner-uses-all': picking one method among several alternatives, ending up with an overall superior ETS method. Such strategy could also be considered for other vision tasks: rather than building an even stronger method, choose from existing methods based on some simple preprocessing.
Procedural 3D Building Reconstruction Using Shape Grammars and Detectors

We propose a novel grammar-driven approach for reconstruction of buildings and landmarks. Our approach complements Structure-from-Motion and image-based analysis with a 'inverse' procedural modeling strategy. So far, procedural modeling has mostly been used for creation of virtual buildings, while the inverse approaches typically focus on reconstruction of single facades. In our work, we reconstruct complete buildings as procedural models using template shape grammars. In the reconstruction process, we let the grammar interpreter automatically decide on which step to take next. The process can be seen as instantiating the template by determining the correct grammar parameters. As an example, we have chosen the reconstruction of Greek Doric temples. This process significantly differs from single facade segmentation due to the immediate need for 3D reconstruction.
Multiview texture models

Mapping textured images on smoothly approximated surfaces is often used to conceal the loss of their real, fine-grained relief. A limitation of mapping a fixed texture in such cases is that it will only be correct for one viewing and one illumination direction. The presence of geometric surface details causes changes that simple foreshortening and global color scaling cannot model well. Hence, one would like to synthesize different textures for different viewing conditions. A texture model is presented that takes account of viewpoint dependent changes in texture appearance. It is highly compact and avoids copy-and-paste like repetitions. The model is learned from example images taken from different viewpoints. It supports texture synthesis for previously unseen conditions.
Affine Reconstruction from Perspective Image Pairs with a Relative Object-Camera Translation in Between

A method is described to recover the three-dimensional affine structure of a scene consisting of at least five points identified in two perspective views with a relative object-camera translation in between. When compared to the results for arbitrary stereo views, a more detailed reconstruction is possible using less information. The method presented only assumes that the two images are obtained by identical cameras, but no knowledge about the intrinsic parameters of the camera(s) or about the performed translation is assumed. By the same method, affine 3D reconstruction from a single view can be achieved for parallel structures. In that case, four points suffice for affine reconstruction.
Online Multiperson Tracking-by-Detection from a Single, Uncalibrated Camera

In this paper, we address the problem of automatically detecting and tracking a variable number of persons in complex scenes using a monocular, potentially moving, uncalibrated camera. We propose a novel approach for multiperson tracking-by-detection in a particle filtering framework. In addition to final high-confidence detections, our algorithm uses the continuous confidence of pedestrian detectors and online-trained, instance-specific classifiers as a graded observation model. Thus, generic object category knowledge is complemented by instance-specific information. The main contribution of this paper is to explore how these unreliable information sources can be used for robust multiperson tracking. The algorithm detects and tracks a large number of dynamically moving people in complex scenes with occlusions, does not rely on background modeling, requires no camera or ground plane calibration, and only makes use of information from the past. Hence, it imposes very few restrictions and is suitable for online applications. Our experiments show that the method yields good tracking performance in a large variety of highly dynamic scenarios, such as typical surveillance videos, webcam footage, or sports sequences. We demonstrate that our algorithm outperforms other methods that rely on additional information. Furthermore, we analyze the influence of different algorithm components on the robustness.