OpenRaiser/Intern-Atlas huggingface dataset:
Hyperplane margin classifiers on the multinomial manifold
Authors: Guy Lebanon, John D. Lafferty
Venue: Conference ICML 2004
Year: 2004

The assumptions behind linear classifiers for categorical data are examined and reformulated in the context of the multinomial manifold, the simplex of multinomial models furnished with the Riemannian structure induced by the Fisher information. This leads to a new view of hyperplane classifiers which, together with a generalized margin concept, shows how to adapt existing margin-based hyperplane models to multinomial geometry. Experiments show the new classification framework to be effective for text classification, where the categorical structure of the data is modeled naturally within the multinomial family.
Better optical triangulation through spacetime analysis
Authors: Brian Curless, Marc Levoy
Venue: Conference ICCV 1995
Year: 1995

The standard methods for extracting range data from optical triangulation scanners are accurate only for planar objects of uniform reflectance illuminated by an incoherent source. Using these methods, curved surfaces, discontinuous surfaces, and surfaces of varying reflectance cause systematic distortions of the range data. Coherent light sources such as lasers introduce speckle artifacts that further degrade the data. We present a new ranging method based on analyzing the time evolution of the structured light reflections. Using our spacetime analysis, we can correct for each of these artifacts, thereby attaining significantly higher accuracy using existing technology. We present results that demonstrate the validity of our method using a commercial laser stripe triangulation scanner.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Multiplexed illumination for scene recovery in the presence of global illumination
Authors: Jinwei Gu, Toshihiro Kobayashi, Mohit Gupta 0001, Shree K. Nayar
Venue: Conference ICCV 2011
Year: 2011

Global illumination effects such as inter-reflections and subsurface scattering result in systematic, and often significant errors in scene recovery using active illumination. Recently, it was shown that the direct and global components could be separated efficiently for a scene illuminated with a single light source. In this paper, we study the problem of direct-global separation for multiple light sources. We derive a theoretical lower bound for the number of required images, and propose a multiplexed illumination scheme which achieves this lower bound. We analyze the signal-to-noise ratio (SNR) characteristics of the proposed illumination multiplexing method in the context of direct-global separation. We apply our method to several scene recovery techniques requiring multiple light sources, including shape from shading, structured light 3D scanning, photometric stereo, and reflectance estimation. Both simulation and experimental results show that the proposed method can accurately recover scene information with fewer images compared to sequentially separating direct-global components for each light source.
Learning to group: A bottom-up framework for 3d part discovery in unseen categories
Authors: Tiange Luo, Kaichun Mo, Zhiao Huang, Jiarui Xu, Siyu Hu, Liwei Wang 0001, Hao Su 0001
Venue: Conference ICLR 2020
Year: 2020

We address the problem of discovering 3D parts for objects in unseen categories. Being able to learn the geometry prior of parts and transfer this prior to unseen categories pose fundamental challenges on data-driven shape segmentation approaches. Formulated as a contextual bandit problem, we propose a learning-based agglomerative clustering framework which learns a grouping policy to progressively group small part proposals into bigger ones in a bottom-up fashion. At the core of our approach is to restrict the local context for extracting part-level features, which encourages the generalizability to unseen categories. On the large-scale fine-grained 3D part dataset, PartNet, we demonstrate that our method can transfer knowledge of parts learned from 3 training categories to 21 unseen testing categories without seeing any annotated samples. Quantitative comparisons against four shape segmentation baselines shows that our approach achieve the state-of-the-art performance.
Multispectral imaging using multiplexed illumination
Authors: Jong-Il Park, Moon-Hyun Lee, Michael D. Grossberg, Shree K. Nayar
Venue: Conference ICCV 2007
Year: 2007

Many vision tasks such as scene segmentation, or the recognition of materials within a scene, become considerably easier when it is possible to measure the spectral reflectance of scene surfaces. In this paper, we present an efficient and robust approach for recovering spectral reflectance in a scene that combines the advantages of using multiple spectral sources and a multispectral camera. We have implemented a system based on this approach using a cluster of light sources with different spectra to illuminate the scene and a conventional RGB camera to acquire images. Rather than sequentially activating the sources, we have developed a novel technique to determine the optimal multiplexing sequence of spectral sources so as to minimize the number of acquired images. We use our recovered spectral measurements to recover the continuous spectral reflectance for each scene point by using a linear model for spectral reflectance. Our imaging system can produce multispectral videos of scenes at 30fps. We demonstrate the effectiveness of our system through extensive evaluation. As a demonstration, we present the results of applying data recovered by our system to material segmentation and spectral relighting.
Generating labels from clicks
Authors: Rakesh Agrawal, Alan Halverson, Krishnaram Kenthapadi, Nina Mishra, Panayiotis Tsaparas
Venue: Conference WSDM 2009
Year: 2009

The ranking function used by search engines to order results is learned from labeled training data. Each training point is a (query, URL) pair that is labeled by a human judge who assigns a score of Perfect, Excellent, etc., depending on how well the URL matches the query. In this paper, we study whether clicks can be used to automatically generate good labels. Intuitively, documents that are clicked (resp., skipped) in aggregate can indicate relevance (resp., lack of relevance). We give a novel way of transforming clicks into weighted, directed graphs inspired by eye-tracking studies and then devise an objective function for finding cuts in these graphs that induce a good labeling. In its full generality, the problem is NP-hard, but we show that, in the case of two labels, an optimum labeling can be found in linear time. For the more general case, we propose heuristic solutions. Experiments on real click logs show that click-based labels align with the opinion of a panel of judges, especially as the consensus of the panel grows stronger.
City-scale landmark identification on mobile devices
Authors: David M. Chen, Georges Baatz, Kevin Köser, Sam S. Tsai, Ramakrishna Vedantham, Timo Pylvänäinen, Kimmo Roimela, Xin Chen, Jeff Bach, Marc Pollefeys, Bernd Girod, Radek Grzeszczuk
Venue: Conference CVPR 2011
Year: 2011

With recent advances in mobile computing, the demand for visual localization or landmark identification on mobile devices is gaining interest. We advance the state of the art in this area by fusing two popular representations of street-level image data - facade-aligned and viewpoint-aligned - and show that they contain complementary information that can be exploited to significantly improve the recall rates on the city scale. We also improve feature detection in low contrast parts of the street-level data, and discuss how to incorporate priors on a user's position (e.g. given by noisy GPS readings or network cells), which previous approaches often ignore. Finally, and maybe most importantly, we present our results according to a carefully designed, repeatable evaluation scheme and make publicly available a set of 1.7 million images with ground truth labels, geotags, and calibration data, as well as a difficult set of cell phone query images. We provide these resources as a benchmark to facilitate further research in the area.
Partial order embedding with multiple kernels
Authors: Brian McFee, Gert R. G. Lanckriet
Venue: Conference ICML 2009
Year: 2009

We consider the problem of embedding arbitrary objects (e.g., images, audio, documents) into Euclidean space subject to a partial order over pair-wise distances. Partial order constraints arise naturally when modeling human perception of similarity. Our partial order framework enables the use of graph-theoretic tools to more efficiently produce the embedding, and exploit global structure within the constraint set.
Visual saliency detection by spatially weighted dissimilarity
Authors: Lijuan Duan, Chunpeng Wu, Jun Miao, Laiyun Qing, Yu Fu
Venue: Conference CVPR 2011
Year: 2011

In this paper, a new visual saliency detection method is proposed based on the spatially weighted dissimilarity. We measured the saliency by integrating three elements as follows: the dissimilarities between image patches, which were evaluated in the reduced dimensional space, the spatial distance between image patches and the central bias. The dissimilarities were inversely weighted based on the corresponding spatial distance. A weighting mechanism, indicating a bias for human fixations to the center of the image, was employed. The principal component analysis (PCA) was the dimension reducing method used in our system. We extracted the principal components (PCs) by sampling the patches from the current image. Our method was compared with four saliency detection approaches using three image datasets. Experimental results show that our method outperforms current state-of-the-art methods on predicting human fixations.
Geometrically consistent elastic matching of 3d shapes: A linear programming solution
Authors: Thomas Windheuser, Ulrich Schlickewei, Frank R. Schmidt, Daniel Cremers
Venue: Conference ICCV 2011
Year: 2011

We propose a novel method for computing a geometrically consistent and spatially dense matching between two 3D shapes. Rather than mapping points to points we match infinitesimal surface patches while preserving the geometric structures. In this spirit we consider matchings as diffeomorphisms between the objects' surfaces which are by definition geometrically consistent. Based on the observation that such diffeomorphisms can be represented as closed and continuous surfaces in the product space of the two shapes we are led to a minimal surface problem in this product space. The proposed discrete formulation describes the search space with linear constraints. Computationally, our approach leads to a binary linear program whose relaxed version can be solved efficiently in a globally optimal manner. As cost function for matching, we consider a thin shell energy, measuring the physical energy necessary to deform one shape into the other. Experimental results demonstrate that the proposed LP relaxation allows to compute highquality matchings which reliably put into correspondence articulated 3D shapes. Moreover a quantitative evaluation shows improvements over existing works.
Reliable feature matching across widely separated views
Authors: Adam Baumberg
Venue: Conference CVPR 2000
Year: 2000

We present a robust method for automatically matching features in images corresponding to the same physical point on an object seen from two arbitrary viewpoints. Unlike conventional stereo matching approaches we assume no prior knowledge about the relative camera positions and orientations. In fact in our application this is the information we wish to determine from the image feature matches. Features are detected in two or more images and characterised using affine texture invariants. The problem of window effects is explicitly addressed by our method-our feature characterisation is invariant to linear transformations of the image data including rotation, stretch and skew. The feature matching process is optimised for a structure-from-motion application where we wish to ignore unreliable matches at the expense of reducing the number of feature matches.
Indexing based on scale invariant interest points
Authors: Krystian Mikolajczyk, Cordelia Schmid
Venue: Conference ICCV 2001
Year: 2001

This paper presents a new method for detecting scale invariant interest points. The method is based on two recent results on scale space: (1) Interest points can be adapted to scale and give repeatable results (geometrically stable). (2) Local extrema over scale of normalized derivatives indicate the presence of characteristic local structures. Our method first computes a multi-scale representation for the Harris interest point detector. We then select points at which a local measure (the Laplacian) is maximal over scales. This allows a selection of distinctive points for which the characteristic scale is known. These points are invariant to scale, rotation and translation as well as robust to illumination changes and limited changes of viewpoint. For indexing, the image is characterized by a set of scale invariant points; the scale associated with each point allows the computation of a scale invariant descriptor. Our descriptors are, in addition, invariant to image rotation, of affine illumination changes and robust to small perspective deformations. Experimental results for indexing show an excellent performance up to a scale factor of 4 for a database with more than 5000 images.
Multicore bundle adjustment
Authors: Changchang Wu, Sameer Agarwal 0001, Brian Curless, Steven M. Seitz
Venue: Conference CVPR 2011
Year: 2011

We present the design and implementation of new inexact Newton type Bundle Adjustment algorithms that exploit hardware parallelism for efficiently solving large scale 3D scene reconstruction problems. We explore the use of multicore CPU as well as multicore GPUs for this purpose. We show that overcoming the severe memory and bandwidth limitations of current generation GPUs not only leads to more space efficient algorithms, but also to surprising savings in runtime. Our CPU based system is up to ten times and our GPU based system is up to thirty times faster than the current state of the art methods, while maintaining comparable convergence behavior. The code and additional results are available at http://grail.cs.washington.edu/projects/mcba.
Analysis of light transport in scattering media
Authors: Yasuhiro Mukaigawa, Yasushi Yagi, Ramesh Raskar
Venue: Conference CVPR 2010
Year: 2010

We propose a new method to analyze light transport in homogeneous scattering media. The incident light undergoes multiple bounces in translucent objects, and produces a complex light field. Our method analyzes the light transport in two steps. First, single and multiple scattering are separated by projecting high-frequency stripe patterns. Then, multiple scattering is decomposed into each bounce component based on the light transport equation. The light field for each bounce is recursively estimated. Experimental results show that light transport in scattering media can be decomposed and visualized for each bounce.
Directional statistics BRDF model
Authors: Ko Nishino
Venue: Conference ICCV 2009
Year: 2009

We introduce a novel parametric BRDF model that can accurately encode a wide variety of real-world isotropic BRDFs with a small number of parameters. The key observation we make is that a BRDF may be viewed as a statistical distribution on a unit hemisphere. We derive a novel directional statistics distribution, which we refer to as the hemispherical exponential power distribution, and model an isotropic BRDF with a mixture of it. The novel directional statistics BRDF model allows us to derive a canonical probabilistic method for estimating its parameters including the number of components. We show that the model captures the full spectrum of real-world isotropic BRDFs with accuracy comparable to non-parametric models but with a much more compact representation. We also experimentally show that the model achieves better accuracy with less measurements compared with such non-parametric models. We further demonstrate the advantages of the novel BRDF model by showing its use for reflection component separation and for exploring the space of isotropic BRDFs.
High quality image reconstruction from raw and jpeg image pair
Authors: Lu Yuan, Jian Sun 0001
Venue: Conference ICCV 2011
Year: 2011

A camera RAW file contains minimally processed data from the image sensor. The contents of the RAW file include more information, and potentially higher quality, than the commonly used JPEG file. But the RAW file is typically several times larger than the JPEG file (taking fewer images, slower quick shooting) and lacks the standard file format (not ready-to-use, prolonging the image workflow). These drawbacks limit its applications. In this paper, we suggest a new “hybrid” image capture mode: a high-res JPEG file and a low-res RAW file as alternative of the original RAW file. Most RAW users can be benefited from such a combination. To address this problem, we provide an effective approach to reconstruct a high quality image by combining the advantages of two kinds of files. We formulate this reconstruction process as a global optimization problem by enforcing two constraints: reconstruction constraint and detail consistency constraint. The final recovered image is smaller than the full-res RAW file, enables faster quick shooting, and has both richer information (e.g., color space, dynamic range, lossless 14 bits data) and higher resolution. In practice, the functionality of capturing such a “hybrid” image pair in one-shot has been supported in some existing digital cameras.
Overcoming visual reverberations
Authors: Yaron Diamant, Yoav Y. Schechner
Venue: Conference CVPR 2008
Year: 2008

An image acquired through a glass window is a super-position of two sources: a scene behind the window, and a reflection of a scene in front of the window. Light rays incident on the window are reflected back and forth inside the glass. Such internal reflections affect the radiance of both sources: a spatial effect is created of dimmed and shifted replications. Our work generalizes the treatment of transparent scenes to deal with this effect. First, we present a physical model of the image formation. It turns out that each of the transmitted and reflected scenes undergoes a convolution with a particular point spread function (PSF), composed of distinct delta functions. Therefore, scene recovery involves inversion of these PSFs. We analyze the fundamental limitations faced by any attempt to solve this inverse problem. We then present a solution approach. The approach is based on deconvolution by linear filtering and simple optimization. The input to the algorithm is a pair of frames, taken through a polarizing filter. The method is demonstrated experimentally.
Increasing the Cost of Model Extraction with Calibrated Proof of Work
Authors: Adam Dziedzic, Muhammad Ahmad Kaleem, Yu Shen Lu, Nicolas Papernot
Venue: Conference ICLR 2022
Year: 2022

In model extraction attacks, adversaries can steal a machine learning model exposed via a public API by repeatedly querying it and adjusting their own model based on obtained predictions. To prevent model stealing, existing defenses focus on detecting malicious queries, truncating, or distorting outputs, thus necessarily introducing a tradeoff between robustness and model utility for legitimate users. Instead, we propose to impede model extraction by requiring users to complete a proof-of-work before they can read the model's predictions. This deters attackers by greatly increasing (even up to 100x) the computational effort needed to leverage query access for model extraction. Since we calibrate the effort required to complete the proof-of-work to each query, this only introduces a slight overhead for regular users (up to 2x). To achieve this, our calibration applies tools from differential privacy to measure the information revealed by a query. Our method requires no modification of the victim model and can be applied by machine learning practitioners to guard their publicly exposed models against being easily stolen.
Optimal correspondences from pairwise constraints
Authors: Olof Enqvist, Klas Josephson, Fredrik Kahl
Venue: Conference ICCV 2009
Year: 2009

Correspondence problems are of great importance in computer vision. They appear as subtasks in many applications such as object recognition, merging partial 3D reconstructions and image alignment. Automatically matching features from appearance only is difficult and errors are frequent. Thus, it is necessary to use geometric consistency to remove incorrect correspondences. Typically heuristic methods like RANSAC or EM-like algorithms are used, but they risk getting trapped in local optima and are in no way guaranteed to find the best solution. This paper illustrates how pairwise constraints in combination with graph methods can be used to efficiently find optimal correspondences. These ideas are implemented on two basic geometric problems, 3D-3D registration and 2D-3D registration. The developed scheme can handle large rates of outliers and cope with multiple hypotheses. Despite the combinatorial explosion, the resulting algorithm which has been extensively evaluated on real data, yields competitive running times compared to state of the art.
Shape from shading under various imaging conditions
Authors: Abdelrehim H. Ahmed, Aly A. Farag
Venue: Conference CVPR 2007
Year: 2007

Most of the shape from shading (SFS) algorithms have been developed under the simplifying assumptions of a Lambertian surface, an orthographic projection, and a distant light source. Due to the difficulty of the SFS problem, only a small number of algorithms have been proposed for surfaces with non-Lambertian reflectance, and among those, only very few algorithms are applicable for surfaces with specular and diffuse reflectance. In this paper we propose a unified framework that is capable of solving the SFS problem under various settings of imaging conditions i.e., Lambertian or non-Lambertian, orthographic or perspective projection, and distant or nearby light source. The proposed algorithm represents the image irradiance equation of each setting as an explicit Partial Differential Equation (PDF). In our implementation we use the Lax-Friedrichs sweeping method to solve this PDF. To demonstrate the efficiency of the proposed algorithm, several comparisons with the state of the art of the SFS literature are given.
Attached shadow coding: Estimating surface normals from shadows under unknown reflectance and lighting conditions
Authors: Takahiro Okabe, Imari Sato, Yoichi Sato 0001
Venue: Conference ICCV 2009
Year: 2009

We present a novel technique, termed attached shadow coding, for estimating surface normals from shadows when the reflectance and lighting conditions are unknown. Our key idea is encoding surface points via attached shadows observed under different light source directions and then estimating surface normals on the basis of the similarity of the attached shadow codes. Because shadows do not rely on reflectance properties, our method is applicable to surfaces with various complex reflectances such as anisotropic and composite materials. Moreover, our method is robust against noise because it takes advantage of the combination of weak constraints imposed by a number of light sources. We theoretically show that the distance between the codes at two surface points is equal to the angle between the corresponding surface normals under the assumption of uniform lighting and a convex object. Our method embeds high-dimensional codes into a 3D surface normal space so that the inter-code distances are preserved. Furthermore, we extend the method in order to alleviate the effects of nonuniform lighting and cast shadows. Experimental results demonstrate the effectiveness of our method.