OpenRaiser/Intern-Atlas huggingface dataset:
Dense shape reconstruction of a moving object under arbitrary, unknown lighting
Authors: Denis Simakov, Darya Frolova, Ronen Basri
Venue: Conference ICCV 2003
Year: 2003

We present a method for shape reconstruction from several images of a moving object. The reconstruction is dense (up to image resolution). The method assumes that the motion is known, e.g., by tracking a small number of feature points on the object. The object is assumed Lambertian (completely matte), light sources should not be very close to the object but otherwise arbitrary, and no knowledge of lighting conditions is required. An object changes its appearance significantly when it changes its orientation relative to light sources, causing violation of the common brightness constancy assumption. While a lot of effort is devoted to deal with this violation, we demonstrate how to exploit it to recover 3D structure from 2D images. We propose a new correspondence measure that enables point matching across views of a moving object. The method has been tested both on computer simulated examples and on a real object.
Leveraging temporal, contextual and ordering constraints for recognizing complex activities in video
Authors: Benjamin Laxton, Jongwoo Lim, David J. Kriegman
Venue: Conference CVPR 2007
Year: 2007

We present a scalable approach to recognizing and describing complex activities in video sequences. We are interested in long-term, sequential activities that may have several parallel streams of action. Our approach integrates temporal, contextual and ordering constraints with output from low-level visual detectors to recognize complex, long-term activities. We argue that a hierarchical, object-oriented design lends our solution to be scalable in that higher-level reasoning components are independent from the particular low-level detector implementation and that recognition of additional activities and actions can easily be added. Three major components to realize this design are: a dynamic Bayesian network structure for representing activities comprised of partially ordered sub-actions, an object-oriented action hierarchy for building arbitrarily complex action detectors and an approximate Viterbi-like algorithm for inferring the most likely observed sequence of actions. Additionally, this study proposes the Erlang distribution as a comprehensive model of idle time between actions and frequency of observing new actions. We show results for our approach on real video sequences containing complex activities.
Shading primitives: Finding folds and shallow grooves
Authors: John A. Haddon, David A. Forsyth
Venue: Conference ICCV 1998
Year: 1998

Diffuse interreflections cause effects that make current theories of shape from shading unsatisfactory. We show that distant radiating surfaces produce radiosity effects at low spatial frequencies. This means that, if a shading pattern has a small region of support, unseen surfaces in the environment can only produce effects that vary slowly over the support region. It is therefore relatively easy to construct matching processes for such patterns that are robust to interreflections. We call regions with these patterns "shading primitives". Folds and grooves on surfaces provide two examples of shading primitives; the shading pattern is relatively independent of surface shape at a fold or a groove, and the pattern is localised. We show that the pattern of shading can be predicted accurately by a simple model, and derive a matching process from this model. Both groove and fold matchers are shown to work well on images of real scenes.
Image restoration using online photo collections
Authors: Kevin Dale, Micah K. Johnson, Kalyan Sunkavalli, Wojciech Matusik, Hanspeter Pfister
Venue: Conference ICCV 2009
Year: 2009

We present an image restoration method that leverages a large database of images gathered from the web. Given an input image, we execute an efficient visual search to find the closest images in the database; these images define the input's visual context. We use the visual context as an image-specific prior and show its value in a variety of image restoration operations, including white balance correction, exposure correction, and contrast enhancement. We evaluate our approach using a database of 1 million images downloaded from Flickr and demonstrate the effect of database size on performance. Our results show that priors based on the visual context consistently out-perform generic or even domain-specific priors for these operations.
Optimizing polynomial solvers for minimal geometry problems
Authors: Oleg Naroditsky, Kostas Daniilidis
Venue: Conference ICCV 2011
Year: 2011

In recent years polynomial solvers based on algebraic geometry techniques, and specifically the action matrix method, have become popular for solving minimal problems in computer vision. In this paper we develop a new method for reducing the computational time and improving numerical stability of algorithms using this method. To achieve this, we propose and prove a set of algebraic conditions which allow us to reduce the size of the elimination template (polynomial coefficient matrix), which leads to faster LU or QR decomposition. Our technique is generic and has potential to improve performance of many solvers that use the action matrix method. We demonstrate the approach on specific examples, including an image stitching algorithm where computation time is halved and single precision arithmetic can be used.
Face re-lighting from a single image under harsh lighting conditions
Authors: Yang Wang 0001, Zicheng Liu 0001, Gang Hua 0001, Zhen Wen, Zhengyou Zhang, Dimitris Samaras
Venue: Conference CVPR 2007
Year: 2007

In this paper, we present a new method to change the illumination condition of a face image, with unknown face geometry and albedo information. This problem is particularly difficult when there is only one single image of the subject available and it was taken under a harsh lighting condition. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using spherical harmonic representation. However, the approximation error can be large under harsh lighting conditions thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion based framework that uses a Markov Random Field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to harsh lighting conditions, but insensitive to partial occlusions as well. The performance of our framework is demonstrated through various experimental results, including the improvement to the face recognition rate under harsh lighting conditions.
Optimal segmentation of dynamic scenes from two perspective views
Authors: René Vidal, Shankar Sastry 0001
Venue: Conference CVPR 2003
Year: 2003

We present a novel algorithm for optimally segmenting dynamic scenes containing multiple rigidly moving objects. We cast the motion segmentation problem as a constrained nonlinear least squares problem, which minimizes the reprojection error subject to all multibody epipolar constraints. By converting this constrained problem into an unconstrained one, we obtain an objective function that depends on the motion parameters only (fundamental matrices), but is independent on the segmentation of the image features. Therefore, our algorithm does not iterate between feature segmentation and single body motion estimation. Instead, it uses standard nonlinear optimization techniques to simultaneously recover all the fundamental matrices, without prior segmentation. We test our approach on a real sequence.
Multi-label learning with highly incomplete data via collaborative embedding
Authors: Yufei Han, Guolei Sun, Yun Shen, Xiangliang Zhang
Venue: Conference KDD 2018
Year: 2018

Tremendous efforts have been dedicated to improving the effectiveness of multi-label learning with incomplete label assignments. Most of the current techniques assume that the input features of data instances are complete. Nevertheless, the co-occurrence of highly incomplete features and weak label assignments is a challenging and widely perceived issue in real-world multi-label learning applications due to a number of practical reasons including incomplete data collection, moderate labels from annotators, etc. Existing multi-label learning algorithms are not directly applicable when the observed features are highly incomplete. In this work, we attack this problem by proposing a weakly supervised multi-label learning approach, based on the idea of collaborative embedding. This approach provides a flexible framework to conduct efficient multi-label classification at both transductive and inductive mode by coupling the process of reconstructing missing features and weak label assignments in a joint optimisation framework. It is designed to collaboratively recover feature and label information, and extract the predictive association between the feature profile and the multi-label tag of the same data instance. Substantial experiments on public benchmark datasets and real security event data validate that our proposed method can provide distinctively more accurate transductive and inductive classification than other state-of-the-art algorithms.
Cast shadow removal combining local and global features
Authors: Zhou Liu, Kaiqi Huang, Tieniu Tan, Liangsheng Wang
Venue: Conference CVPR 2007
Year: 2007

In this paper, we present a method using pixel-level information, local region-level information and global-level information to remove shadow. At the pixel-level, we employ GMM to model the behavior of cast shadow for every pixel in the HSV color space, as it can deal with complex illumination conditions. However, unlike the GMM for background which can obtain sample every frame, this model for shadow needs more frames to get the same number of sample, because shadow may not appear at the same pixel for each frame. Therefore, it will take a long time to converge. To overcome this drawback, we use the local region-level information to get more samples and global-level information to improve a preclassifier and then, by using it, we get samples which are more likely to be shadow. Also, at the local region-level, we use Markov random fields to represent dependencies between the label of single pixel and labels of its neighborhood. Moreover, to make global level information more robust, tracking information is used. Experimental results show that the proposed method is efficient and robust.
Discriminative human action segmentation and recognition using semi-markov model
Authors: Qinfeng Shi, Li Wang 0033, Li Cheng 0001, Alexander J. Smola
Venue: Conference CVPR 2008
Year: 2008

Given an input video sequence of one person conducting a sequence of continuous actions, we consider the problem of jointly segmenting and recognizing actions. We propose a discriminative approach to this problem under a semi-Markov model framework, where we are able to define a set of features over input-output space that captures the characteristics on boundary frames, action segments and neighboring action segments, respectively. In addition, we show that this method can also be used to recognize the person who performs in this video sequence. A Viterbi-like algorithm is devised to help efficiently solve the induced optimization problem. Experiments on a variety of datasets demonstrate the effectiveness of the proposed method.
Automatic face recognition for film character retrieval in feature-length films
Authors: Ognjen Arandjelovic, Andrew Zisserman
Venue: Conference CVPR 2005
Year: 2005

The objective of this work is to recognize all the frontal faces of a character in the closed world of a movie or situation comedy, given a small number of query faces. This is challenging because faces in a feature-length film are relatively uncontrolled with a wide variability of scale, pose, illumination, and expressions, and also may be partially occluded. We develop a recognition method based on a cascade of processing steps that normalize for the effects of the changing imaging environment. In particular there are three areas of novelty: (i) we suppress the background surrounding the face, enabling the maximum area of the face to be retained for recognition rather than a subset; (ii) we include a pose refinement step to optimize the registration between the test image and face exemplar; and (iii) we use robust distance to a sub-space to allow for partial occlusion and expression change. The method is applied and evaluated on several feature length films. It is demonstrated that high recall rates (over 92%) can be achieved whilst maintaining good precision (over 93%).
Flow mosaicking: Real-time pedestrian counting without scene-specific learning
Authors: Yang Cong, Haifeng Gong, Song Chun Zhu, Yandong Tang
Venue: Conference CVPR 2009
Year: 2009

In this paper, we present a novel algorithm based on flow velocity field estimation to count the number of pedestrians across a detection line or inside a specified region. We regard pedestrians across the line as fluid flow, and design a novel model to estimate the flow velocity field. By integrating over time, the dynamic mosaics are constructed to count the number of pixels and edges passed through the line. Consequentially, the number of pedestrians can be estimated by quadratic regression, with the number of weighted pixels and edges as input. The regressors are learned off line from several camera tilt angles, and have taken the calibration information into account.We use tilt-angle-specific learning to ensure direct deployment and avoid overfitting while the commonly used scene-specific learning scheme needs on-site annotation and always trends to overfitting. Experiments on a variety of videos verified that the proposed method can give accurate estimation under different camera setup in real-time.
A maximum-flow formulation of the n-camera stereo correspondence problem
Authors: Sébastien Roy 0001, Ingemar J. Cox
Venue: Conference ICCV 1998
Year: 1998

This paper describes a new algorithm for solving the N-camera stereo correspondence problem by transforming it into a maximum-flow problem. Once solved, the minimum-cut associated to the maximum-flow yields a disparity surface for the whole image at once. This global approach to stereo analysis provides a more accurate and coherent depth map than the traditional line-by-line stereo. Moreover, the optimality of the depth surface is guaranteed and can be shown to be a generalization of the dynamic programming approach that is widely used in standard stereo. Results show improved depth estimation as well as better handling of depth discontinuities. While the worst case running time is O(n/sup 2/d/sup 2/log(nd)), the observed average running time is O(n/sup 1.2/ d/sup 1.3/) for an image size of n pixels and depth resolution d.
Storyboard sketches for content based video retrieval
Authors: John P. Collomosse, Graham McNeill, Yu Qian
Venue: Conference ICCV 2009
Year: 2009

We present a novel Content Based Video Retrieval (CBVR) system, driven by free-hand sketch queries depicting both objects and their movement (via dynamic cues; streak-lines and arrows). Our main contribution is a probabilistic model of video clips (based on Linear Dynamical Systems), leading to an algorithm for matching descriptions of sketched objects to video. We demonstrate our model fitting to clips under static and moving camera conditions, exhibiting linear and oscillatory motion. We evaluate retrieval on two real video data sets, and on a video data set exhibiting controlled variation in shape, color, motion and clutter.
Accurate and Robust Registration for In-Hand Modeling
Authors: Thibaut Weise, Bastian Leibe, Luc Van Gool
Venue: Conference CVPR 2008
Year: 2008

We present fast 3D surface registration methods for in-hand modeling. This allows users to scan complete objects swiftly by simply turning them around in front of the scanner. The paper makes two main contributions. First, we propose an efficient method for detecting registration failures, which is a vital property of any automatic modeling system. Our method is based on two different consistency tests, one based on geometry and one based on texture. Second, we extend ICP by three additional fast registration methods for both coarse and fine alignment based on both texture and geometry. Each of those methods brings in additional information that can compensate for ambiguities in the other cues. Together, they allow for the robust reconstruction of a large variety of objects with different geometric and photometric properties. Finally, we show how both failure detection and fast registration can be combined in a practical and robust in-hand modeling system that operates at interactive frame rates.
Adversarial attacks and defenses: Frontiers, advances and practice
Authors: Han Xu 0002, Yaxin Li 0001, Wei Jin 0009, Jiliang Tang
Venue: Conference KDD 2020
Year: 2020

Deep neural networks (DNN) have achieved unprecedented success in numerous machine learning tasks in various domains. However, the existence of adversarial examples leaves us a big hesitation when applying DNN models on safety-critical tasks such as autonomous vehicles and malware detection. These adversarial examples are intentionally crafted instances, either appearing in the train or test phase, which can fool the DNN models to make severe mistakes. Therefore, people are dedicated to devising more robust models to resist adversarial examples, but usually they are broken by new stronger attacks. This arms-race between adversarial attacks and defenses has been drawn increasing attention in recent years. In this tutorial, we provide a comprehensive overview on the frontiers and advances of adversarial attacks and their countermeasures. In particular, we give a detailed introduction of different types of attacks under different scenarios, including evasion and poisoning attacks, white-box and black box attacks. We will also discuss how the defending strategies develop to compete against these attacks, and how new attacks come out to break these defenses. Moreover, we will discuss the story of adversarial attacks and defenses in other data domains, especially in graph structured data. Then, we introduce DeepRobust, a Pytorch adversarial learning library which aims to build a comprehensive and easy-to-use platform to foster this research field. Finally, we summarize the tutorial with discussions on open issues and challenges about adversarial attacks and defenses. Via our tutorial, our audience can grip the main idea and key approaches of the game between adversarial attacks and defenses.
An iterative optimization approach for unified image segmentation and matting
Authors: Jue Wang 0001, Michael F. Cohen
Venue: Conference ICCV 2005
Year: 2005

Separating a foreground object from the background in a static image involves determining both full and partial pixel coverages, also known as extracting a matte. Previous approaches require the input image to be presegmented into three regions: foreground, background and unknown, which are called a trimap. Partial opacity values are then computed only for pixels inside the unknown region. This presegmentation based approach fails for images with large portions of semitransparent foreground where the trimap is difficult to create even manually. In this paper, we combine the segmentation and matting problem together and propose a unified optimization approach based on belief propagation. We iteratively estimate the opacity value for every pixel in the image, based on a small sample of foreground and background pixels marked by the user. Experimental results show that compared with previous approaches, our method is more efficient to extract high quality mattes for foregrounds with significant semitransparent regions
Towards hybrid human-ai workflows for unknown unknown detection
Authors: Anthony Z. Liu, Santiago Guerra, Isaac Fung, Gabriel Matute, Ece Kamar, Walter S. Lasecki
Venue: Conference WWW 2020
Year: 2020

Predictive models are susceptible to errors called unknown unknowns, in which the model assigns incorrect labels to instances with high confidence. These commonly arise when training data does not represent variations of a class encountered at model deployment. Prior work showed that crowd workers can identify instances of unknown unknowns, but asking the crowd to identify a sufficient number of individual instances can be costly to acquire [2]. Instead, this paper presents an approach that leverages people’s ability to find patterns to retrain classifiers more effectively with fewer examples. We ask crowd workers to suggest and verify patterns in unknown unknowns. We then use these patterns to train an expansion classifier to identify additional examples from existing data that the primary classifier has encountered (and potentially misclassified) in the past. Our experiments show that our approach outperforms existing unknown unknown detection methods at improving classifier performance. This work is the first to leverage crowds to identify error patterns in large datasets to improve ML training.
Inverse polarization raytracing: estimating surface shapes of transparent objects
Authors: Daisuke Miyazaki, Katsushi Ikeuchi
Venue: Conference CVPR 2005
Year: 2005

We propose a novel method for estimating the surface shapes of transparent objects by analyzing the polarization state of the light. Existing methods do not fully consider the reflection, refraction, and transmission of the light occurring inside a transparent object. We employ a polarization raytracing method to compute both the path of the light and its polarization state. Our proposed iterative computation method estimates the surface shape of the transparent object by minimizing the difference between the polarization data rendered by the polarization raytracing method and the polarization data obtained from a real object.
Enhancing by saliency-guided decolorization
Authors: Codruta O. Ancuti, Cosmin Ancuti, Philippe Bekaert
Venue: Conference CVPR 2011
Year: 2011

This paper introduces an effective decolorization algorithm that preserves the appearance of the original color image. Guided by the original saliency, the method blends the luminance and the chrominance information in order to conserve the initial color disparity while enhancing the chromatic contrast. As a result, our straightforward fusing strategy generates a new spatial distribution that discriminates better the illuminated areas and color features. Since we do not employ quantization or a per-pixel optimization (computationally expensive), the algorithm has a linear runtime, and depending on the image resolution it could be used in real-time applications. Extensive experiments and a comprehensive evaluation against existing state-of-the-art methods demonstrate the potential of our grayscale operator. Furthermore, since the method accurately preserves the finest details while enhancing the chromatic contrast, the utility and versatility of our operator have been proved for several other challenging applications such as video decolorization, detail enhancement, single image dehazing and segmentation under different illuminants.