OpenRaiser/Intern-Atlas huggingface dataset:
Detecting and matching repeated patterns for automatic geo-tagging in urban environments
Authors: Grant Schindler, Panchapagesan Krishnamurthy, Roberto Lublinerman, Yanxi Liu 0001, Frank Dellaert
Venue: Conference CVPR 2008
Year: 2008

We present a novel method for automatically geo-tagging photographs of man-made environments via detection and matching of repeated patterns. Highly repetitive environments introduce numerous correspondence ambiguities and are problematic for traditional wide-baseline matching methods. Our method exploits the highly repetitive nature of urban environments, detecting multiple perspectively distorted periodic 2D patterns in an image and matching them to a 3D database of textured facades by reasoning about the underlying canonical forms of each pattern. Multiple 2D-to-3D pattern correspondences enable robust recovery of camera orientation and location. We demonstrate the success of this method in a large urban environment.
Repetition-based dense single-view reconstruction
Authors: Changchang Wu, Jan-Michael Frahm, Marc Pollefeys
Venue: Conference CVPR 2011
Year: 2011

This paper presents a novel approach for dense reconstruction from a single-view of a repetitive scene structure. Given an image and its detected repetition regions, we model the shape recovery as the dense pixel correspondences within a single image. The correspondences are represented by an interval map that tells the distance of each pixel to its matched pixels within the single image. In order to obtain dense repetitive structures, we develop a new repetition constraint that penalizes the inconsistency between the repetition intervals of the dynamically corresponding pixel pairs. We deploy a graph-cut to balance between the high-level constraint of geometric repetition and the low-level constraints of photometric consistency and spatial smoothness. We demonstrate the accurate reconstruction of dense 3D repetitive structures through a variety of experiments, which prove the robustness of our approach to outliers such as structure variations, illumination changes, and occlusions.
Salient Object Detection by Composition
Authors: Jie Feng 0012, Yichen Wei, Litian Tao, Chao Zhang 0001, Jian Sun 0001
Venue: Conference ICCV 2011
Year: 2011

Conventional saliency analysis methods measure the saliency of individual pixels. The resulting saliency map inevitably loses information in the original image and finding salient objects in it is difficult. We propose to detect salient objects by directly measuring the saliency of an image window in the original image and adopt the well established sliding window based object detection paradigm. We present a simple definition for window saliency, i.e., the cost of composing the window using the remaining parts of the image. The definition uses the entire image as the context and agrees with human intuition. It no longer relies on idealistic assumptions usually used before (e.g., "back- ground is homogenous") and generalizes well to complex objects and backgrounds in real world images. To realize the definition, we illustrate how to incorporate different cues such as appearance, position, and size. Based on a segment-based representation, the window composition cost function can be efficiently evaluated by a greedy optimization algorithm. Extensive evaluation on challenging object detection datasets verifies better efficacy and efficiency of the proposed method comparing to the state-of-the-art, making it a good pre-processing tool for subsequent applications. Moreover, we hope to stimulate further work towards the challenging yet important problem of generic salient object detection.
Online Food Recipe Title Semantics: Combining Nutrient Facts and Topics
Authors: Tomasz Kusmierczyk, Kjetil Nørvåg
Venue: Conference CIKM 2016
Year: 2016

Dietary pattern analysis is an important research area, and recently the availability of rich resources in food-focused social networks has enabled new opportunities in that field. However, there is a little understanding of how online textual content is related to actual health factors, e.g., nutritional values. To contribute to this lack of knowledge, we present a novel approach to mine and model online food content by combining text topics with related nutrient facts. Our empirical analysis reveals a strong correlation between them and our experiments show the extent to which it is possible to predict nutrient facts from meal name.
Image hallucination with primal sketch priors
Authors: Jian Sun 0001, Nanning Zheng 0001, Hai Tao, Heung-Yeung Shum
Venue: Conference CVPR 2003
Year: 2003

We propose a Bayesian approach to image hallucination. Given a generic low resolution image, we hallucinate a high resolution image using a set of training images. Our work is inspired by recent progress on natural image statistics that the priors of image primitives can be well represented by examples. Specifically, primal sketch priors (e.g., edges, ridges and corners) are constructed and used to enhance the quality of the hallucinated high resolution image. Moreover, a contour smoothness constraint enforces consistency of primitives in the hallucinated image by a Markov-chain based inference algorithm. A reconstruction constraint is also applied to further improve the quality of the hallucinated image. Experiments demonstrate that our approach can hallucinate high quality super-resolution images.
Blindly separating mixtures of multiple layers with spatial shifts
Authors: Kun Gai, Zhenwei Shi 0001, Changshui Zhang
Venue: Conference CVPR 2008
Year: 2008

We address the problem of blindly separating mixtures of multiple layer images with unknown spatial shifts and mixing coefficients. Our proposed method can handle the over-determined, determined and under-determined cases where mixtures are more than, as many as and fewer than layers, respectively. The method is fast in over-determined and determined cases, with the same complexity as the fast Fourier transform (FFT), and can separate more layers from fewer mixtures in the under-determined case. It consists of two main steps. First, a novel sparse blind separation algorithm is applied, to estimate the spatial shifts, the mixing coefficients and the edge image of each layer. Second, all layers are reconstructed, by large scale linear programming in the under-determined case, or by least-squares solutions in other cases. The effectiveness of this technology is shown in the experiments on two simulated mixtures of four layers with spatial shifts, real mixture photos containing transparency and reflections, and real mixture images in a dissolve from a video.
Blind separation of superimposed images with unknown motions
Authors: Kun Gai, Zhenwei Shi 0001, Changshui Zhang
Venue: Conference CVPR 2009
Year: 2009

We consider the blind separation of source layers from superimposed mixtures thereof, involving unknown motions and unknown mixing coefficients of layers in each mixture. Previous blind separation approaches for such problems assume motions to be uniform translations, and hence are limited for real world applications. In this paper, we develop a sparse blind separation algorithm to estimate both parameterized motions and mixing coefficients. Then, a novel reconstruction approach is presented to recover all layers, by utilizing not only the mixing model but also the statistical properties of natural images. The whole method can handle more general motions than translations, including scalings, rotations and other transformations. In addition, the number of layers is automatically identified, and all layers can be recovered even in the under-determined case where mixtures are fewer than layers. The effectiveness of this technology is shown in the experiments on two simulated mixtures of four layers, real photos containing transparency and reflections, and real crossfade images from videos.
Determining the radiometric response function from a single grayscale image
Authors: Stephen Lin, Lei Zhang
Venue: Conference CVPR 2005
Year: 2005

A method is presented for computing the radiometric response function of a camera from a single grayscale image. While most previous techniques require a set of registered images with different exposures to obtain response data, our approach capitalizes on a statistical feature of graylevel histograms at edge regions to gain information for radio-metric calibration. Appropriate edge regions are automatically determined by our technique, and a prior model of radiometric response functions is employed to deal with incomplete data. With this single-image method, radiometric calibration becomes possible to perform in many instances where the camera is unknown.
Radiometric calibration from noise distributions
Authors: Yasuyuki Matsushita, Stephen Lin 0001
Venue: Conference CVPR 2007
Year: 2007

A method is proposed for estimating radiometric response functions from noise observations. From the statistical properties of noise sources, the noise distribution for each scene radiance value is shown to be symmetric for a radiometrically calibrated camera. However, due to the non-linearity of camera response functions, the observed noise distributions become skewed in an uncalibrated camera. In this paper, we capitalize on these asymmetric profiles of measured noise distributions to estimate radiometric response functions. Unlike prior approaches, the proposed method is not sensitive to noise level, and is therefore particularly useful when the noise level is high. Also, the proposed method does not require registered input images taken with different exposures; only statistical noise distributions at multiple intensity levels are used. Real-world experiments demonstrate the effectiveness of the proposed approach in comparison to standard calibration techniques.
Using geometry invariants for camera response function estimation
Authors: Tian-Tsong Ng, Shih-Fu Chang, Mao-Pei Tsui
Venue: Conference CVPR 2007
Year: 2007

In this paper, we present a new single-image camera response function (CRF) estimation method using geometry invariants (GI). We derive mathematical properties and geometric interpretation for GI, which lend insight to addressing various algorithm implementation issues in a principled way. In contrast to the previous single-image CRF estimation methods, our method provides a constraint equation for selecting the potential target data points. Comparing to the prior work, our experiment is conducted over more extensive data and our method is flexible in that its estimation accuracy and stability can be improved whenever more than one image is available. The geometry invariance theory is novel and may be of wide interest.
Shape and albedo from multiple images using integrability
Authors: Alan L. Yuille, Daniel Snow
Venue: Conference CVPR 1997
Year: 1997

Previous work has developed an approach for estimating shape and albedo from multiple images assuming Lambertian reflectance with single light sources. The main contributions of this paper are: (i) to show how the approach can be generalized to include ambient background illumination, (ii) to demonstrate the use of the integrability constraint for solving this problem, and (iii) an iterative algorithm which is able to improve the analysis by finding shadows and rejecting them.
The limitations of adversarial training and the blind-spot attack
Authors: Huan Zhang 0001, Hongge Chen, Zhao Song 0002, Duane S. Boning, Inderjit S. Dhillon, Cho-Jui Hsieh
Venue: Conference ICLR 2019
Year: 2019

The adversarial training procedure proposed by Madry et al. (2018) is one of the most effective methods to defend against adversarial examples in deep neural networks (DNNs). In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on test set) of adversarial training has a strong correlation with the distance between a test point and the manifold of training data embedded by the network. Test examples that are relatively far away from this manifold are more likely to be vulnerable to adversarial attacks. Consequentially, an adversarial training based defense is susceptible to a new class of attacks, the "blind-spot attack", where the input images reside in "blind-spots" (low density regions) of the empirical distribution of training data but is still on the ground-truth data manifold. For MNIST, we found that these blind-spots can be easily found by simply scaling and shifting image pixel values. Most importantly, for large datasets with high dimensional and complex data manifold (CIFAR, ImageNet, etc), the existence of blind-spots in adversarial training makes defending on any valid test examples difficult due to the curse of dimensionality and the scarcity of training data. Additionally, we find that blind-spots also exist on provable defenses including (Wong & Kolter, 2018) and (Sinha et al., 2018) because these trainable robustness certificates can only be practically optimized on a limited set of training data.
Robust real-time face pose and facial expression recovery
Authors: Zhiwei Zhu, Qiang Ji
Venue: Conference CVPR 2006
Year: 2006

Face motion is the sum of rigid motion related with face pose and non-rigid motion related with facial expression. Both motions are coupled in the captured image so that they can not be easily recovered from the image directly. In this paper, a novel technique is proposed to recover 3D face pose and facial expression simultaneously from a monocular video sequence in real time. First, twenty-eight salient facial features are detected and tracked robustly under various face orientations and facial expressions. Second, after modelling the coupling between face pose and facial expression in the 2D image as a nonlinear function, a normalized SVD (N-SVD) decomposition technique is proposed to recover the pose and expression parameters analytically. A nonlinear technique is subsequently utilized to refine the solution obtained from the N-SVD technique by imposing the orthonormality constraint on the pose parameters. Compared to the original SVD technique proposed in [1], which is very sensitive to the image noise and numerically unstable in practice, the proposed method can recover the face pose and facial expression robustly and accurately. Finally, the performance of the proposed technique is evaluated in the experiments using both synthetic and real image sequences.
Curvature regularity for region-based image segmentation and inpainting: A linear programming relaxation
Authors: Thomas Schoenemann, Fredrik Kahl, Daniel Cremers
Venue: Conference ICCV 2009
Year: 2009

We consider a class of region-based energies for image segmentation and inpainting which combine region integrals with curvature regularity of the region boundary. To minimize such energies, we formulate an integer linear program which jointly estimates regions and their boundaries. Curvature regularity is imposed by respective costs on pairs of adjacent boundary segments. By solving the associated linear programming relaxation and thresholding the solution one obtains an approximate solution to the original integer problem. To our knowledge this is the first approach to impose curvature regularity in region-based formulations in a manner that is independent of initialization and allows to compute a bound on the optimal energy. In a variety of experiments on segmentation and inpainting, we demonstrate the advantages of higher-order regularity. Moreover, we demonstrate that for most experiments the optimality gap is smaller than 2% of the global optimum. For many instances we are even able to compute the global optimum.
Learning based digital matting
Authors: Yuanjie Zheng, Chandra Kambhamettu
Venue: Conference ICCV 2009
Year: 2009

We cast some new insights into solving the digital matting problem by treating it as a semi-supervised learning task in machine learning. A local learning based approach and a global learning based approach are then produced, to fit better the scribble based matting and the trimap based matting, respectively. Our approaches are easy to implement because only some simple matrix operations are needed. They are also extremely accurate because they can efficiently handle the nonlinear local color distributions by incorporating the kernel trick, that are beyond the ability of many previous works. Our approaches can outperform many recent matting methods, as shown by the theoretical analysis and comprehensive experiments. The new insights may also inspire several more works.
Assessing the aesthetic quality of photographs using generic image descriptors
Authors: Luca Marchesotti, Florent Perronnin, Diane Larlus, Gabriela Csurka
Venue: Conference ICCV 2011
Year: 2011

In this paper, we automatically assess the aesthetic properties of images. In the past, this problem has been addressed by hand-crafting features which would correlate with best photographic practices (e.g. “Does this image respect the rule of thirds?”) or with photographic techniques (e.g. “Is this image a macro?”). We depart from this line of research and propose to use generic image descriptors to assess aesthetic quality. We experimentally show that the descriptors we use, which aggregate statistics computed from low-level local features, implicitly encode the aesthetic properties explicitly used by state-of-the-art methods and outperform them by a significant margin.
Recovering free space of indoor scenes from a single image
Authors: Varsha Hedau, Derek Hoiem, David A. Forsyth
Venue: Conference CVPR 2012
Year: 2012

In this paper we consider the problem of recovering the free space of an indoor scene from its single image. We show that exploiting the box like geometric structure of furniture and constraints provided by the scene, allows us to recover the extent of major furniture objects in 3D. Our “boxy” detector localizes box shaped objects oriented parallel to the scene across different scales and object types, and thus blocks out the occupied space in the scene. To localize the objects more accurately in 3D we introduce a set of specially designed features that capture the floor contact points of the objects. Image based metrics are not very indicative of performance in 3D. We make the first attempt to evaluate single view based occupancy estimates for 3D errors and propose several task driven performance measures towards it. On our dataset of 592 indoor images marked with full 3D geometry of the scene, we show that: (a) our detector works well using image based metrics; (b) our refinement method produces significant improvements in localization in 3D; and (c) if one evaluates using 3D metrics, our method offers major improvements over other single view based scene geometry estimation methods.
Combining two-view constraints for motion estimation
Authors: Venu Madhav Govindu
Venue: Conference CVPR 2001
Year: 2001

In this paper we describe two methods for estimating the motion parameters of an image sequence. For a sequence of N images, the global motion can be described by N-1 independent motion models. On the other hand, in a sequence there exist as many as /sub 2///sup N(N-1)/ pairwise relative motion constraints that can be solve for efficiently. In this paper we show how to linearly solve for consistent global motion models using this highly redundant set of constraints. In the first case, our method involves estimating all available pairwise relative motions and linearly fining a global motion model to these estimates. In the second instance, we exploit the fact that algebraic (i.e. epipolar) constraints between various image pairs are all related to each other by the global motion model. This results in an estimation method that directly computes the motion of the sequence by using all possible algebraic constraints. Unlike using reprojection error, our optimisation method does not solve for the structure of points resulting in a reduction of the dimensionality of the search space. Our algorithms are used for both 3D camera motion estimation and camera calibration. We provide real examples of both applications.
Seeing double without confusion: Structure-from-motion in highly ambiguous scenes
Authors: Nianjuan Jiang, Ping Tan, Loong Fah Cheong
Venue: Conference CVPR 2012
Year: 2012

3D reconstruction from an unordered set of images may fail due to incorrect epipolar geometries (EG) between image pairs arising from ambiguous feature correspondences. Previous methods often analyze the consistency between different EGs, and regard the largest subset of self-consistent EGs as correct. However, as demonstrated in [14], such a largest self-consistent set often corresponds to incorrect result, especially when there are duplicate structures in the scene. We propose a novel optimization criteria based on the idea of `missing correspondences'. The global minimum of our optimization objective function is associated with the correct solution. We then design an efficient algorithm for minimization, whose convergence to a local minimum is guaranteed. Experimental results show our method outperforms the state-of-the-art.
Structure from motion for scenes with large duplicate structures
Authors: Richard Roberts 0001, Sudipta N. Sinha, Richard Szeliski, Drew Steedly
Venue: Conference CVPR 2011
Year: 2011

Most existing structure from motion (SFM) approaches for unordered images cannot handle multiple instances of the same structure in the scene. When image pairs containing different instances are matched based on visual similarity, the pairwise geometric relations as well as the correspondences inferred from such pairs are erroneous, which can lead to catastrophic failures in the reconstruction. In this paper, we investigate the geometric ambiguities caused by the presence of repeated or duplicate structures and show that to disambiguate between multiple hypotheses requires more than pure geometric reasoning. We couple an expectation maximization (EM)-based algorithm that estimates camera poses and identifies the false match-pairs with an efficient sampling method to discover plausible data association hypotheses. The sampling method is informed by geometric and image-based cues. Our algorithm usually recovers the correct data association, even in the presence of large numbers of false pairwise matches.
Regional relation modeling for visual place recognition
Authors: Yingying Zhu 0001, Biao Li, Jiong Wang, Zhou Zhao 0001
Venue: Conference SIGIR 2020
Year: 2020

In the process of visual perception, humans perceive not only the appearance of objects existing in a place but also their relationships (e.g. spatial layout). However, the dominant works on visual place recognition are always based on the assumption that two images depict the same place if they contain enough similar objects, while the relation information is neglected. In this paper, we propose a regional relation module which models the regional relationships and converts the convolutional feature maps to the relational feature maps. We further design a cascaded pooling method to get discriminative relation descriptors by preventing the influence of confusing relations and preserving as much useful information as possible. Extensive experiments on two place recognition benchmarks demonstrate that training with the proposed regional relation module improves the appearance descriptors and the relation descriptors are complementary to appearance descriptors. When these two kinds of descriptors are concatenated together, the resulting combined descriptors outperform the state-of-the-art methods.