OpenRaiser/Intern-Atlas huggingface dataset:
A perceptually motivated online benchmark for image matting
Authors: Christoph Rhemann, Carsten Rother, Jue Wang 0001, Margrit Gelautz, Pushmeet Kohli, Pamela Rott
Venue: Conference CVPR 2009
Year: 2009

The availability of quantitative online benchmarks for low-level vision tasks such as stereo and optical flow has led to significant progress in the respective fields. This paper introduces such a benchmark for image matting. There are three key factors for a successful benchmarking system: (a) a challenging, high-quality ground truth test set; (b) an online evaluation repository that is dynamically updated with new results; (c) perceptually motivated error functions. Our new benchmark strives to meet all three criteria. We evaluated several matting methods with our benchmark and show that their performance varies depending on the error function. Also, our challenging test set reveals problems of existing algorithms, not reflected in previously reported results. We hope that our effort will lead to considerable progress in the field of image matting, and welcome the reader to visit our benchmark at www.aIphamatting.com.
Looking around the corner using transient imaging
Authors: Ahmed Kirmani, Tyler Hutchison, James Davis 0001, Ramesh Raskar
Venue: Conference ICCV 2009
Year: 2009

We show that multi-path analysis using images from a timeof-flight (ToF) camera provides a tantalizing opportunity to infer about 3D geometry of not only visible but hidden parts of a scene. We provide a novel framework for reconstructing scene geometry from a single viewpoint using a camera that captures a 3D time-image I(x, y, t) for each pixel. We propose a framework that uses the time-image and transient reasoning to expose scene properties that may be beyond the reach of traditional computer vision. We corroborate our theory with free space hardware experiments using a femtosecond laser and an ultrafast photo detector array. The ability to compute the geometry of hidden elements, unobservable by both the camera and illumination source, will create a range of new computer vision opportunities.
i-RevNet: Deep invertible networks
Authors: Jörn-Henrik Jacobsen, Arnold W. M. Smeulders, Edouard Oyallon
Venue: Conference ICLR 2018
Year: 2018

It is widely believed that the success of deep convolutional networks is based on progressively discarding uninformative variability about the input with respect to the problem at hand. This is supported empirically by the difficulty of recovering images from their hidden representations, in most commonly used network architectures. In this paper we show via a one-to-one mapping that this loss of information is not a necessary condition to learn representations that generalize well on complicated problems, such as ImageNet. Via a cascade of homeomorphic layers, we build the i-RevNet, a network that can be fully inverted up to the final projection onto the classes, i.e. no information is discarded. Building an invertible architecture is difficult, for one, because the local inversion is ill-conditioned, we overcome this by providing an explicit inverse. An analysis of i-RevNets learned representations suggests an alternative explanation for the success of deep networks by a progressive contraction and linear separation with depth. To shed light on the nature of the model learned by the i-RevNet we reconstruct linear interpolations between natural image representations.
Video stabilization using robust feature trajectories
Authors: Ken-Yi Lee, Yung-Yu Chuang, Bing-Yu Chen 0004, Ming Ouhyoung
Venue: Conference ICCV 2009
Year: 2009

This paper proposes a new approach for video stabilization. Most existing video stabilization methods adopt a framework of three steps, motion estimation, motion compensation and image composition. Camera motion is often estimated based on pairwise registration between frames. Thus, these methods often assume static scenes or distant backgrounds. Furthermore, for scenes with moving objects, robust methods are required for finding the dominant motion. Such assumptions and judgements could lead to errors in motion parameters. Errors are compounded by motion compensation which smoothes motion parameters. This paper proposes a method to directly stabilize a video without explicitly estimating camera motion, thus assuming neither motion models nor dominant motion. The method first extracts robust feature trajectories from the input video. Optimization is then performed to find a set of transformations to smooth out these trajectories and stabilize the video. In addition, the optimization also considers quality of the stabilized video and selects a video with not only smooth camera motion but also less unfilled area after stabilization. Experiments show that our method can deal with complicated videos containing near, large and multiple moving objects.
Modeling age progression in young faces
Authors: Narayanan Ramanathan, Rama Chellappa
Venue: Conference CVPR 2006
Year: 2006

We propose a craniofacial growth model that characterizes growth related shape variations observed in human faces during formative years. The model draws inspiration from the ‘revised’ cardioidal strain transformation model proposed in psychophysical studies related to craniofacial growth. The model takes into account anthropometric evidences collected on facial growth and hence is in accordance with the observed growth patterns in human faces across years. We characterize facial growth by means of growth parameters defined over facial landmarks often used in anthropometric studies. We illustrate how the age-based anthropometric constraints on facial proportions translate into linear and non-linear constraints on facial growth parameters and propose methods to compute the optimal growth parameters. The proposed craniofacial growth model can be used to predict one’s appearance across years and to perform face recognition across age progression. This is demonstrated on a database of age separated face images of individuals under 18 years of age.
Binary action search for learning continuous-action control policies
Authors: Jason Pazis, Michail G. Lagoudakis
Venue: Conference ICML 2009
Year: 2009

Reinforcement Learning methods for controlling stochastic processes typically assume a small and discrete action space. While continuous action spaces are quite common in real-world problems, the most common approach still employed in practice is coarse discretization of the action space. This paper presents a novel method, called Binary Action Search, for realizing continuousaction policies by searching efficiently the entire action range through increment and decrement modifications to the values of the action variables according to an internal binary policy defined over an augmented state space. The proposed approach essentially approximates any continuous action space to arbitrary resolution and can be combined with any discrete-action reinforcement learning algorithm for learning continuous-action policies. Binary Action Search eliminates the restrictive modification steps of Adaptive Action Modification and requires no temporal action locality in the domain. Our approach is coupled with two well-known reinforcement learning algorithms (Least-Squares Policy Iteration and Fitted Q-Iteration) and its use and properties are thoroughly investigated and demonstrated on the continuous state-action Inverted Pendulum, Double Integrator, and Car on the Hill domains.
Factorization for non-rigid and articulated structure using metric projections
Authors: Marco Paladini, Alessio Del Bue, Marko Stosic, Marija Dodig, João M. F. Xavier, Lourdes Agapito
Venue: Conference CVPR 2009
Year: 2009

This paper describes a new algorithm for recovering the 3D shape and motion of deformable and articulated objects purely from uncalibrated 2D image measurements using an iterative factorization approach. Most solutions to non-rigid and articulated structure from motion require metric constraints to be enforced on the motion matrix to solve for the transformation that upgrades the solution to metric space. While in the case of rigid structure the metric upgrade step is simple since the motion constraints are linear, deformability in the shape introduces non-linearities. In this paper we propose an alternating least-squares approach associated with a globally optimal projection step onto the manifold of metric constraints. An important advantage of this new algorithm is its ability to handle missing data which becomes crucial when dealing with real video sequences with self-occlusions. We show successful results of our algorithms on synthetic and real sequences of both deformable and articulated data.
Multiscale score matching for out-of-distribution detection
Authors: Ahsan Mahmood, Junier Oliva, Martin Andreas Styner
Venue: Conference ICLR 2021
Year: 2021

We present a new methodology for detecting out-of-distribution (OOD) images by utilizing norms of the score estimates at multiple noise scales. A score is defined to be the gradient of the log density with respect to the input data. Our methodology is completely unsupervised and follows a straight forward training scheme. First, we train a deep network to estimate scores for levels of noise. Once trained, we calculate the noisy score estimates for N in-distribution samples and take the L2-norms across the input dimensions (resulting in an NxL matrix). Then we train an auxiliary model (such as a Gaussian Mixture Model) to learn the in-distribution spatial regions in this L-dimensional space. This auxiliary model can now be used to identify points that reside outside the learned space. Despite its simplicity, our experiments show that this methodology significantly outperforms the state-of-the-art in detecting out-of-distribution images. For example, our method can effectively separate CIFAR-10 (inlier) and SVHN (OOD) images, a setting which has been previously shown to be difficult for deep likelihood models.
Spatial-depth super resolution for range images
Authors: Qingxiong Yang, Ruigang Yang, James Davis 0001, David Nistér
Venue: Conference CVPR 2007
Year: 2007

We present a new post-processing step to enhance the resolution of range images. Using one or two registered and potentially high-resolution color images as reference, we iteratively refine the input low-resolution range image, in terms of both its spatial resolution and depth precision. Evaluation using the Middlebury benchmark shows across-the-board improvement for sub-pixel accuracy. We also demonstrated its effectiveness for spatial resolution enhancement up to 100 times with a single reference image.
ORB: An Efficient Alternative to SIFT or SURF
Authors: Ethan Rublee, Vincent Rabaud, Kurt Konolige, Gary R. Bradski
Venue: Conference ICCV 2011
Year: 2011

Feature matching is at the base of many computer vision problems, such as object recognition or structure from motion. Current methods rely on costly descriptors for detection and matching. In this paper, we propose a very fast binary descriptor based on BRIEF, called ORB, which is rotation invariant and resistant to noise. We demonstrate through experiments how ORB is at two orders of magnitude faster than SIFT, while performing as well in many situations. The efficiency is tested on several real-world applications, including object detection and patch-tracking on a smart phone.
Reconstruction of specular surfaces using polarization imaging
Authors: Stefan Rahmann, Nikos Canterakis
Venue: Conference CVPR 2001
Year: 2001

Traditional intensity imaging does not offer a general approach for the perception of textureless and specular reflecting surfaces. Intensity based methods for shape reconstruction of specular surfaces rely on virtual (i.e. mirrored) features moving over the surface under viewer motion. We present a novel method based on polarization imaging for shape recovery of specular surfaces. This method overcomes the limitations of the intensity based approach because no virtual features are required. It recovers whole surface patches and not only single curves on the surface. The presented solution is general as it is independent of the illumination. The polarization image encodes the projection of the surface normals onto the image and therefore provides constraints on the surface geometry. Taking polarization images from multiple views produces enough constraints to infer the complete surface shape. The reconstruction problem is solved by an optimization scheme where the surface geometry is modelled by a set of hierarchical basis functions. The optimization algorithm proves to be well converging, accurate and noise resistant. The work is substantiated by experiments on synthetic and real data.
Pruning-aware merging for efficient multitask inference
Authors: Xiaoxi He, Dawei Gao, Zimu Zhou, Yongxin Tong, Lothar Thiele
Venue: Conference KDD 2021
Year: 2021

Many mobile applications demand selective execution of multiple correlated deep learning inference tasks on resource-constrained platforms. Given a set of deep neural networks, each pre-trained for a single task, it is desired that executing arbitrary combinations of tasks yields minimal computation cost. Pruning each network separately yields suboptimal computation cost due to task relatedness. A promising remedy is to merge the networks into a multitask network to eliminate redundancy across tasks before network pruning. However, pruning a multitask network combined by existing network merging schemes cannot minimise the computation cost of every task combination because they do not consider such a future pruning. To this end, we theoretically identify the conditions such that pruning a multitask network minimises the computation of all task combinations. On this basis, we propose Pruning-Aware Merging (PAM), a heuristic network merging scheme to construct a multitask network that approximates these conditions. The merged network is then ready to be further pruned by existing network pruning methods. Evaluations with different pruning schemes, datasets, and network architectures show that PAM achieves up to 4.87x less computation against the baseline without network merging, and up to 2.01x less computation against the baseline with a state-of-the-art network merging scheme.
A hierarchical model of shape and appearance for human action classification
Authors: Juan Carlos Niebles, Li Fei-Fei 0001
Venue: Conference CVPR 2007
Year: 2007

We present a novel model for human action categorization. A video sequence is represented as a collection of spatial and spatial-temporal features by extracting static and dynamic interest points. We propose a hierarchical model that can be characterized as a constellation of bags-of-features and that is able to combine both spatial and spatial-temporal features. Given a novel video sequence, the model is able to categorize human actions in a frame-by-frame basis. We test the model on a publicly available human action dataset [2] and show that our new method performs well on the classification task. We also conducted control experiments to show that the use of the proposed mixture of hierarchical models improves the classification performance over bag of feature models. An additional experiment shows that using both dynamic and static features provides a richer representation of human actions when compared to the use of a single feature type, as demonstrated by our evaluation in the classification task.
Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes
Authors: Stefan Hinterstoisser, Stefan Holzer, Cedric Cagniart, Slobodan Ilic, Kurt Konolige, Nassir Navab, Vincent Lepetit
Venue: Conference ICCV 2011
Year: 2011

We present a method for detecting 3D objects using multi-modalities. While it is generic, we demonstrate it on the combination of an image and a dense depth map which give complementary object information. It works in real-time, under heavy clutter, does not require a time consuming training stage, and can handle untextured objects. It is based on an efficient representation of templates that capture the different modalities, and we show in many experiments on commodity hardware that our approach significantly outperforms state-of-the-art methods on single modalities.
Inferring human gaze from appearance via adaptive linear regression
Authors: Feng Lu 0005, Yusuke Sugano, Takahiro Okabe, Yoichi Sato 0001
Venue: Conference ICCV 2011
Year: 2011

The problem of estimating human gaze from eye appearance is regarded as mapping high-dimensional features to low-dimensional target space. Conventional methods require densely obtained training samples on the eye appearance manifold, which results in a tedious calibration stage. In this paper, we introduce an adaptive linear regression (ALR) method for accurate mapping via sparsely collected training samples. The key idea is to adaptively find the subset of training samples where the test sample is most linearly representable. We solve the problem via l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> -optimization and thoroughly study the key issues to seek for the best solution for regression. The proposed gaze estimation approach based on ALR is naturally sparse and low-dimensional, giving the ability to infer human gaze from variant resolution eye images using much fewer training samples than existing methods. Especially, the optimization procedure in ALR is extended to solve the subpixel alignment problem simultaneously for low resolution test eye images. Performance of the proposed method is evaluated by extensive experiments against various factors such as number of training samples, feature dimensionality and eye image resolution to verify its effectiveness.
Robust higher order potentials for enforcing label consistency
Authors: Pushmeet Kohli, Lubor Ladicky, Philip H. S. Torr
Venue: Conference CVPR 2008
Year: 2008

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner. Our method is based on higher order conditional random fields and uses potentials defined on sets of pixels (image segments) generated using unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and can be seen as a strict generalization of the commonly used pairwise contrast sensitive smoothness potentials. The higher order potential functions used in our framework take the form of the Robust P <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> model. This enables the use of powerful graph cut based move making algorithms for performing inference in the framework [14]. We test our method on the problem of multi-class object segmentation by augmenting the conventional CRF used for object segmentation with higher order potentials defined on image regions. Experiments on challenging data sets show that integration of higher order potentials quantitatively and qualitatively improves results leading to much better definition of object boundaries. We believe that this method can be used to yield similar improvements for many other labelling problems.
Joint optimization of segmentation and appearance models
Authors: Sara Vicente, Vladimir Kolmogorov, Carsten Rother
Venue: Conference ICCV 2009
Year: 2009

Many interactive image segmentation approaches use an objective function which includes appearance models as an unknown variable. Since the resulting optimization problem is NP-hard the segmentation and appearance are typically optimized separately, in an EM-style fashion. One contribution of this paper is to express the objective function purely in terms of the unknown segmentation, using higher-order cliques. This formulation reveals an interesting bias of the model towards balanced segmentations. Furthermore, it enables us to develop a new dual decomposition optimization procedure, which provides additionally a lower bound. Hence, we are able to improve on existing optimizers, and verify that for a considerable number of real world examples we even achieve global optimality. This is important since we are able, for the first time, to analyze the deficiencies of the model. Another contribution is to establish a property of a particular dual decomposition approach which involves convex functions depending on foreground area. As a consequence, we show that the optimal decomposition for our problem can be computed efficiently via a parametric maxflow algorithm.
Segmentation with pairwise attraction and repulsion
Authors: Stella X. Yu, Jianbo Shi
Venue: Conference ICCV 2001
Year: 2001

We propose a method of image segmentation by integrating pairwise attraction and directional repulsion derived from local grouping and figure-ground cues. These two kinds of pairwise relationships are encoded in the real and imaginary parts of an Hermitian graph weight matrix, through which we can directly generalize the normalized cuts criterion. With bi-graph constructions, this method can be readily extended to handle nondirectional repulsion that captures dissimilarity. We demonstrate the use of repulsion in image segmentation with relative depth cues, which allows segmentation and figure-ground segregation to be computed simultaneously. As a general mechanism to represent the dual measures of attraction and repulsion, this method can also be employed to solve other constraint satisfaction and optimization problems.
Action mach a spatio-temporal maximum average correlation height filter for action recognition
Authors: Mikel D. Rodriguez, Javed Ahmed, Mubarak Shah
Venue: Conference CVPR 2008
Year: 2008

In this paper we introduce a template-based method for recognizing human actions called action MACH. Our approach is based on a maximum average correlation height (MACH) filter. A common limitation of template-based methods is their inability to generate a single template using a collection of examples. MACH is capable of capturing intra-class variability by synthesizing a single Action MACH filter for a given action class. We generalize the traditional MACH filter to video (3D spatiotemporal volume), and vector valued data. By analyzing the response of the filter in the frequency domain, we avoid the high computational cost commonly incurred in template-based approaches. Vector valued data is analyzed using the Clifford Fourier transform, a generalization of the Fourier transform intended for both scalar and vector-valued data. Finally, we perform an extensive set of experiments and compare our method with some of the most recent approaches in the field by using publicly available datasets, and two new annotated human action datasets which include actions performed in classic feature films and sports broadcast television.
Visual object tracking using adaptive correlation filters
Authors: David S. Bolme, J. Ross Beveridge, Bruce A. Draper, Yui Man Lui
Venue: Conference CVPR 2010
Year: 2010

Although not commonly used, correlation filters can track complex objects through rotations, occlusions and other distractions at over 20 times the rate of current state-of-the-art techniques. The oldest and simplest correlation filters use simple templates and generally fail when applied to tracking. More modern approaches such as ASEF and UMACE perform better, but their training needs are poorly suited to tracking. Visual tracking requires robust filters to be trained from a single frame and dynamically adapted as the appearance of the target object changes. This paper presents a new type of correlation filter, a Minimum Output Sum of Squared Error (MOSSE) filter, which produces stable correlation filters when initialized using a single frame. A tracker based upon MOSSE filters is robust to variations in lighting, scale, pose, and nonrigid deformations while operating at 669 frames per second. Occlusion is detected based upon the peak-to-sidelobe ratio, which enables the tracker to pause and resume where it left off when the object reappears.