mtybilly/Instruct2DS huggingface dataset:
Shell PCA: Statistical Shape Modelling in Shell Space
Authors: Behrend Heeren, Martin Rumpf, William A. P. Smith Chao Zhang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper we describe how to perform Principal Components Analysis in "shell space". Thin shells are a physical model for surfaces with non-zero thickness whose deformation dissipates elastic energy. Thin shells, or their discrete counterparts, can be considered to reside in a shell space in which the notion of distance is given by the elastic energy required to deform one shape into another. It is in this setting that we show how to perform statistical analysis of a set of shapes (meshes in dense correspondence), providing a hybrid between physical and statistical shape modelling. The resulting models are better able to capture non-linear deformations, for example resulting from articulated motion, even when training data is very sparse compared to the dimensionality of the observation space.
Learning to Combine Mid-Level Cues for Object Proposal Generation
Authors: Sanja Fidler, Sven Dickinson Tom Lee
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In recent years, region proposals have replaced sliding windows in support of object recognition, offering more discriminating shape and appearance information through improved localization.  One powerful approach for generating region proposals is based on minimizing parametric energy functions with parametric maxflow.  In this paper, we introduce Parametric Min-Loss (PML), a novel structured learning framework for parametric energy functions.  While PML is generally applicable to different domains, we use it in the context of region proposals to learn to combine a set of mid-level grouping cues to yield a small set of object region proposals with high recall.  Our learning framework accounts for multiple diverse outputs, and is complemented by diversification seeds based on image location and color.  This approach casts perceptual grouping and cue combination in a novel structured learning framework which yields baseline improvements on VOC 2012 and COCO 2014.
Enhancing Road Maps by Parsing Aerial Images Around the World
Authors: Shenlong Wang, Sanja Fidler, Raquel Urtasun Gellert Mattyus
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In recent years, contextual models that exploit maps have been shown to be very effective for many recognition and localization tasks.  In this paper we propose to exploit aerial images in order to enhance  freely available world maps. Towards this goal, we make use of OpenStreetMap and formulate the problem as the one of inference in a Markov random field parameterized in terms of  the location of the road-segment centerlines as well as  their width.  This parameterization enables very efficient inference and  returns only topologically correct roads. In particular, we can segment all OSM roads in the  whole world in a single day using a small cluster of 10 computers. Importantly, our approach generalizes very well; it can be trained using only 1.5 km2 aerial imagery and produce very accurate results in any location across the globe.   We demonstrate the effectiveness of our approach outperforming the state-of-the-art in two new benchmarks that we collect. We then show how our enhanced maps are beneficial for semantic segmentation of ground images.
Probabilistic Appearance Models for Segmentation and Classification
Authors: Jan Ehrhardt, Heinz Handels Julia Kruger
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Statistical shape and appearance models are often based on the accurate identification of one-to-one correspondences in a training data set. At the same time, the determination of these corresponding landmarks is the most challenging part of such methods. Hufnagel etal developed an alternative method using correspondence probabilities for a statistical shape model.  We propose the use of probabilistic correspondences for statistical appearance models by incorporating appearance information into the framework. A point-based representation is employed representing the image by a set of vectors assembling position and appearances. Using probabilistic correspondences between these multi-dimensional feature vectors eliminates the need for extensive preprocessing to find corresponding landmarks and reduces the dependence of the generated model on the landmark positions. Then, a maximum a-posteriori approach is used to derive a single global optimization criterion with respect to model parameters and observation dependent parameters, that directly affects shape and appearance information of the considered structures. Model generation and fitting can be expressed by optimizing the same criterion.  The developed framework describes the modeling process in a concise and flexible mathematical way and allows for additional constraints as topological regularity in the modeling process. Furthermore, it eliminates the demand for costly correspondence determination.  We apply the model for segmentation and landmark identification in hand X-ray images, where segmentation information is modeled as further features in the vectorial image representation. The results demonstrate the feasibility of the model to reconstruct contours and landmarks for unseen test images. Furthermore, we apply the model for tissue classification, where a model is generated for healthy brain tissue using 2D MRI slices. Applying the model to images of stroke patients the probabilistic correspondences are used to classify between healthy and pathological structures. The results demonstrate the ability of the probabilistic model to recognize healthy and pathological tissue automatically.
A Randomized Ensemble Approach to Industrial CT Segmentation
Authors: Jayaraman Jayaraman J. Thiagarajan, Peer-Timo Bremer Hyojin Kim
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Tuning the models and parameters of common segmentation approaches is challenging especially in the presence of noise and artifacts. Ensemble-based techniques attempt to compensate by randomly varying models and/or parameters to create a diverse set of hypotheses, which are subsequently ranked to arrive at the best solution. However, these methods have been restricted to cases where the underlying models are well-established, e.g. natural images. In practice, it is difficult to determine a suitable base-model and the amount of randomization required. Furthermore, for multi-object scenes no single hypothesis may perform well for all objects, reducing the overall quality of the results. This paper presents a new ensemble-based segmentation framework for industrial CT images demonstrating that comparatively simple models and randomization strategies can significantly improve the result over existing techniques. Furthermore, we introduce a per-object based ranking, followed by a consensus inference that can outperform even the best case scenario of existing hypothesis ranking approaches. We demonstrate the effectiveness of our approach using a set of noise and artifact rich CT images from baggage security and show that it significantly outperforms existing solutions in this area.
Semi-Supervised Normalized Cuts for Image Segmentation
Authors: Nathan D. Cahill Selene E. Chew
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Since its introduction as a  powerful graph-based method for image segmentation, the Normalized Cuts (NCuts) algorithm has been generalized to incorporate expert knowledge about how certain pixels or regions should be grouped, or how the resulting segmentation should be biased to be correlated with priors. Previous approaches incorporate hard must-link constraints on how certain pixels should be grouped as well as hard cannot-link constraints on how other pixels should be separated into different groups. In this paper, we reformulate NCuts to allow both sets of constraints to be handled in a soft manner, enabling the user to tune the degree to which the constraints are satisfied. An approximate spectral solution to the reformulated problem exists without requiring explicit construction of a large, dense matrix; hence, computation time is comparable to that of unconstrained NCuts. Using synthetic data and real imagery, we show that soft handling of constraints yields better results than unconstrained NCuts and enables more robust clustering and segmentation than is possible when the constraints are strictly enforced.
StereoSnakes: Contour Based Consistent Object Extraction For Stereo Images
Authors: Tongwei Ren, Gangshan Wu Ran Ju
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Consistent object extraction plays an essential role for stereo image editing with the population of stereoscopic 3D media. Most previous methods perform segmentation on entire images for both views using dense stereo correspondence constraints. We find that for such kind of methods the computation is highly redundant since the two views are near-duplicate. Besides, the consistency may be violated due to the imperfectness of current stereo matching algorithms. In this paper, we propose a contour based method which searches for consistent object contours instead of regions. It integrates both stereo correspondence and object boundary constraints into an energy minimization framework. The proposed method has several advantages compared to previous works. First, the searching space is restricted in object boundaries thus the efficiency significantly improved. Second, the discriminative power of object contours results in a more consistent segmentation. Furthermore, the proposed method can effortlessly extend existing single-image segmentation methods to work in stereo scenarios. The experiment on the Adobe benchmark shows superior extraction accuracy and significant improvement of efficiency of our method to state-of-the-art. We also demonstrate in a few applications how our method can be used as a basic tool for stereo image editing.
Semantic Segmentation of RGBD Images With Mutex Constraints
Authors: Sinisa Todorovic, Longin Jan Latecki Zhuo Deng
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we address the problem of semantic scene segmentation of RGB-D images of indoor scenes. We propose a novel image region labeling method which augments CRF formulation with hard mutual exclusion (mutex) constraints. This way our approach can make use of rich and accurate 3D geometric structure coming from Kinect in a principled manner. The final labeling result must satisfy all mutex constraints, which allows us to eliminate configurations that violate common sense physics laws like placing a floor above a night stand. Three classes of mutex constraints are proposed: global object co-occurrence constraint, relative height relationship constraint, and local support relationship constraint. We evaluate our approach on the NYU-Depth V2 dataset, which consists of 1449 cluttered indoor scenes, and also test generalization of our model trained on NYU-Depth V2 dataset directly on a recent SUN3D dataset without any new training. The experimental results show that we significantly outperform the state-of-the-art methods in scene labeling on both datasets.
Weakly- and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation
Authors: Liang-Chieh Chen, Kevin P. Murphy, Alan L. Yuille George Papandreou
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Deep convolutional neural networks (DCNNs) trained on a large number of images with strong pixel-level annotations have recently significantly pushed the state-of-art in semantic image segmentation. We study the more challenging problem of learning DCNNs for semantic image segmentation from either (1) weakly annotated training data such as bounding boxes or image-level labels or (2) a combination of few strongly labeled and many weakly labeled images, sourced from one or multiple datasets. We develop Expectation-Maximization (EM) methods for semantic image segmentation model training under these weakly supervised and semi-supervised settings. Extensive experimental evaluation shows that the proposed techniques can learn models delivering competitive results on the challenging PASCAL VOC 2012 image segmentation benchmark, while requiring significantly less annotation effort. We share source code implementing the proposed system at https://bitbucket.org/deeplab/deeplab-public.
Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts
Authors: Evgeny Levinkov, Nicolas Bonneel, Guillaume Lavoue, Thomas Brox, Bjorn Andres Margret Keuper
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Formulations of the Image Decomposition Problem as a Multicut Problem (MP) w.r.t. a superpixel graph have received considerable attention. In contrast, instances of the MP w.r.t. a pixel grid graph have received little attention, firstly, because the MP is NP-hard and instances w.r.t. a pixel grid graph are hard to solve in practice, and, secondly, due to the lack of long-range terms in the objective function of the MP. We propose a generalization of the MP with long-range terms (LMP). We design and implement two efficient algorithms (primal feasible heuristics) for the MP and LMP which allow us to study instances of both problems w.r.t. the pixel grid graphs of the images in the BSDS-500 benchmark. The decompositions we obtain do not differ significantly from the state of the art, suggesting that the LMP is a competitive formulation of the Image Decomposition Problem. To demonstrate the generality of the LMP, we apply it also to the Mesh Decomposition Problem posed by the Princeton benchmark, obtaining state-of-the-art decompositions.
Parsimonious Labeling
Authors: M. Pawan Kumar Puneet K. Dokania
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose a new family of discrete energy minimization problems, which we call parsimonious labeling. Our energy function consists of unary potentials and high-order clique potentials. While the unary potentials are arbitrary, the clique potentials are proportional to the diversity of the set of unique labels assigned to the clique. Intuitively, our energy function encourages the labeling to be parsimonious, that is, use as few labels as possible. This in turn allows us to capture useful cues for important computer vision applications such as stereo correspondence and image denoising. Furthermore, we propose an efficient graph-cuts based algorithm for the parsimonious labeling problem that provides strong theoretical guarantees on the quality of the solution. Our algorithm consists of three steps. First, we approximate a given diversity using a mixture of a novel hierarchical Pn Potts model. Second, we use a divide-and-conquer approach for each mixture component, where each subproblem is solved using an efficient alpha-expansion algorithm. This provides us with a small number of putative labelings, one for each mixture component. Third, we choose the best putative labeling in terms of the energy value. Using both synthetic and standard real datasets, we show that our algorithm significantly outperforms other graph-cuts based approaches.
Volumetric Bias in Segmentation and Reconstruction: Secrets and Solutions
Authors: Hossam Isack, Carl Olsson, Ismail Ben Ayed Yuri Boykov
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Many standard optimization methods for segmentation and reconstruction compute ML model estimates for appearance or geometry of segments, e.g. Zhu-Yuille 1996, Torr 1998,  Chan-Vese 2001, GrabCut 2004, Delong et al. 2012.  We observe that the standard likelihood term in these formulations corresponds to a generalized probabilistic K-means energy. In learning it is well known that this energy has a strong bias to clusters of equal size, which we express as a penalty for KL divergence from a uniform distribution of cardinalities. However, this volumetric bias has been mostly ignored in computer vision. We demonstrate significant artifacts in standard segmentation and reconstruction methods due to this bias. Moreover, we propose binary and multi-label optimization techniques that either (a) remove this bias or (b) replace it by a KL divergence term for any given target volume distribution. Our general ideas apply to continuous or discrete energy formulations in segmentation, stereo, and other reconstruction problems.
Entropy Minimization for Convex Relaxation Approaches
Authors: Martin R. Oswald, Youngwook Kee, Junmo Kim, Marc Pollefeys, Daniel Cremers Mohamed Souiai
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Despite their enormous success in solving hard combinatorial problems, convex relaxation approaches often suffer from the fact that the computed solutions are far from binary and that subsequent heuristic binarization may substantially degrade the quality of computed solutions.  In this paper, we propose a novel relaxation technique which incorporates the entropy of the objective variable as a measure of relaxation tightness. We show both theoretically and experimentally that augmenting the objective function with an entropy term gives rise to more binary solutions and consequently solutions with a substantially tighter optimality gap.  We use difference of convex function (DC) programming as an efficient and provably convergent solver for the arising convex-concave minimization problem.  We evaluate this approach on three prominent non-convex computer vision challenges: multi-label inpainting, image segmentation and spatio-temporal multi-view reconstruction.  These experiments show that our approach consistently yields better solutions with respect to the original integral optimization problem
Adaptively Unified Semi-Supervised Dictionary Learning With Active Points
Authors: Xiaojie Guo, Stan Z. Li Xiaobo Wang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Semi-supervised dictionary learning aims to construct a dictionary by utilizing both labeled and unlabeled data. To enhance the discriminative capability of the learned dictionary, numerous discriminative terms have been proposed by evaluating either the prediction loss or the class separation criterion on the coding vectors of labeled data, but with rare consideration of the power of the coding vectors corresponding to unlabeled data. In this paper, we present a novel semi-supervised dictionary learning method, which uses the informative coding vectors of both labeled and unlabeled data, and adaptively emphasizes the high confidence coding vectors of unlabeled data to enhance the dictionary discriminative capability simultaneously. By doing so, we integrate the discrimination of dictionary, the induction of classifier to new testing data and the transduction of labels to unlabeled data into a unified framework. To solve the proposed problem, an effective iterative algorithm is designed. Experimental results on a series of benchmark databases show that our method outperforms other state-of-the-art dictionary learning methods in most cases.
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
Authors: Philipp Krahenbuhl, Trevor Darrell Deepak Pathak
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We present an approach to learn a dense pixel-wise labeling from image-level tags. Each image-level tag imposes constraints on the output labeling of a Convolutional Neural Network (CNN) classifier. We propose Constrained CNN (CCNN), a method which uses a novel loss function to optimize for any set of linear constraints on the output space (i.e. predicted label distribution) of a CNN. Our loss formulation is easy to optimize and can be incorporated directly into standard stochastic gradient descent optimization. The key idea is to phrase the training objective as a biconvex optimization for linear models, which we then relax to nonlinear deep networks. Extensive experiments demonstrate the generality of our new learning framework. The constrained loss yields state-of-the-art results on weakly supervised semantic image segmentation. We further demonstrate that adding slightly more supervision can greatly improve the performance of the learning algorithm.
A Multiscale Variable-Grouping Framework for MRF Energy Minimization
Authors: Meirav Galun, Stav Yagev, Ronen Basri, Irad Yavneh Omer Meir
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We present a multiscale approach for minimizing the energy associated with Markov Random Fields (MRFs) with energy functions that include arbitrary pairwise potentials. The MRF is represented on a hierarchy of successively coarser scales, where the problem on each scale is itself an MRF with suitably defined potentials. These representations are used to construct an efficient multiscale algorithm that seeks a minimal-energy solution to the original problem. The algorithm is iterative and features a bidirectional crosstalk between fine and coarse representations. We use consistency criteria to guarantee that the energy is nonincreasing throughout the iterative process. The algorithm is evaluated on real-world datasets, achieving competitive performance in relatively short run-times.
Inferring M-Best Diverse Labelings in a Single One
Authors: Bogdan Savchynskyy, Dmitrij Schlesinger, Dmitry Vetrov, Carsten Rother Alexander Kirillov
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We consider the task of finding M-best diverse solutions in a graphical model. In a previous work by Batra et al. an algorithmic approach for finding such solutions was proposed, and its usefulness was shown in numerous applications. Contrary to previous work we propose a novel formulation of the problem in form of a single energy minimization problem in a specially constructed graphical model. We show that the method of Batra et al. can be considered as a greedy approximate algorithm for our model, whereas we introduce an efficient specialized optimization technique for it, based on alpha-expansion. We evaluate our method on two application scenarios, interactive and semantic image segmentation, with binary and multiple labels. In both cases we achieve considerably better error rates than state-of-the art diversity methods. Furthermore, we empirically discover that in the binary label case we were able to reach global optimality for all test instances.
Convolutional Sparse Coding for Image Super-Resolution
Authors: Wangmeng Zuo, Qi Xie, Deyu Meng, Xiangchu Feng, Lei Zhang Shuhang Gu
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Sparse coding (SC) plays an important role in versatile computer vision applications such as image super-resolution (SR). Most of the previous SC based SR methods partition the image into overlapped patches, and process each patch separately. These methods, however, ignore the consistency of pixels in overlapped patches, which is a strong constraint for image reconstruction. In this paper, we propose a convolutional sparse coding (CSC) based SR (CSC-SR) method to address the consistency issue. Our CSC-SR involves three groups of parameters to be learned: (i) a set of filters to decompose the low resolution (LR) image into LR sparse feature maps; (ii) a  mapping function to predict the high resolution (HR) feature maps from the LR ones; and (iii) a set of filters to reconstruct the HR images from the predicted HR feature maps via simple convolution operations. By working directly on the whole image, the proposed CSC-SR algorithm does not need to divide the image into overlapped patches, and can exploit the image global correlation to produce more robust reconstruction of image local structures. Experimental results clearly validate the advantages of CSC over patch based SC in SR application. Compared with state-of-the-art SR methods, the proposed CSC-SR method achieves highly competitive PSNR results, while demonstrating better edge and texture preservation performance.
A Wavefront Marching Method for Solving the Eikonal Equation on Cartesian Grids
Authors: Marcos Ortega, Manuel G. Penedo Brais Cancela
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This paper presents a new wavefront propagation method for dealing with the classic Eikonal equation. While classic Dijkstra-like graph-based techniques achieve the solution in O(M log M), they do not approximate the unique physically relevant solution very well. Fast Marching Methods (FMM) were created to efficiently solve the continuous problem. The proposed approximation tries to maintain the complexity, in order to make the algorithm useful in a wide range of contexts. The key idea behind our method is the creation of 'mini wave-fronts', which are combined to propagate the solution. Experimental results show the improvement in the accuracy with respect to the state of the art, while the average computational speed is maintained in O(M log M), similar to the FMM techniques.
A Projection Free Method for Generalized Eigenvalue Problem With a Nonsmooth Regularizer
Authors: Maxwell D. Collins, Sathya N. Ravi, Vamsi K. Ithapu, Nagesh Adluru, Sterling C. Johnson, Vikas Singh Seong Jae Hwang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Eigenvalue problems are ubiquitous in computer vision, covering a very broad spectrum of applications ranging from estimation problems in multi-view geometry to image segmentation. Few other linear algebra problems have a more mature set of numerical routines available and many computer vision libraries leverage such tools extensively. However, the ability to call the underlying solver only as a ``black box'' can often become restrictive. Many `human in the loop' settings in vision frequently exploit supervision from an expert, to the extent that the user can be considered a subroutine in the overall system. In other cases, there is additional domain knowledge, side or even partial information that one may want to incorporate within the formulation. In general, regularizing a (generalized) eigenvalue problem with such side information remains difficult. Motivated by these needs, this paper presents an optimization scheme to solve generalized eigenvalue problems (GEP) involving a (nonsmooth) regularizer. We start from an alternative formulation of GEP where the feasibility set of the model involves the Stiefel manifold. The core of this paper presents an end to end stochastic optimization scheme for the resultant problem. We show how this general algorithm enables improved statistical analysis of brain imaging data where the regularizer is derived from other `views' of the disease pathology, involving clinical measurements and other image-derived representations.
Optimizing Expected Intersection-Over-Union With Candidate-Constrained CRFs
Authors: Dany Tarlow, Dhruv Batra Faruk Ahmed
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We study the question of how to make loss-aware predictions in image segmentation settings where the evaluation function is the Intersection-over-Union (IoU) measure that is used widely in evaluating image segmentation systems. Currently, there are two dominant approaches: the first approximates the Expected-IoU (EIoU) score as Expected-Intersection-over-Expected-Union (EIoEU); and the second approach is to compute exact EIoU but only over a small set of high-quality candidate solutions. We begin by asking which approach we should favor for two typical image segmentation tasks. Studying this question leads to two new methods that draw ideas from both existing approaches. Our new methods use the EIoEU approximation paired with high quality candidate solutions. Experimentally we show that our new approaches lead to improved performance on both image segmentation tasks.
Higher-Order Inference for Multi-Class Log-Supermodular Models
Authors: Josip Djolonga, Andreas Krause Jian Zhang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Higher-order models have been shown to be very useful for a plethora of computer vision tasks. However, existing techniques have focused mainly on MAP inference. In this paper, we present the first efficient approach towards approximate Bayesian marginal inference in a general class of high-order, multi-label attractive models, where previous techniques slow down exponentially with the order (clique size). We formalize this task as performing inference in log-supermodular models under partition constraints, and present an efficient variational inference technique. The resulting optimization problems are convex and yield bounds on the partition function. We also obtain a fully factorized approximation to the posterior, which can be used in lieu of the true complicated distribution. We empirically demonstrate the performance of our approach by comparing it to traditional inference methods on a challenging high-fidelity multi-label image segmentation dataset. We obtain state-of-the-art classification accuracy for MAP inference, and substantially improved ROC curves using the approximate marginals.
Depth-Based Hand Pose Estimation: Data, Methods, and Challenges
Authors: Gregory Rogez, Yi Yang, Jamie Shotton, Deva Ramanan James S. Supancic III
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Hand pose estimation has matured rapidly in recent years. The introduction of commodity depth sensors and a multitude of practical applications have spurred new advances. We provide an extensive analysis of the state-of-the-art, focusing on hand pose estimation from a single depth frame. To do so, we have implemented a considerable number of systems, and will release all software and evaluation code. We summarize important conclusions here: (1) Pose estimation appears roughly solved for scenes with isolated hands. However, methods still struggle to analyze cluttered scenes where hands may be interacting with nearby objects and surfaces. To spur further progress we introduce a challenging new dataset with diverse, cluttered scenes. (2) Many methods evaluate themselves with disparate criteria, making comparisons difficult. We define a consistent evaluation criteria, rigorously motivated by human experiments. (3) We introduce a simple nearest-neighbor baseline that outperforms most existing systems. This implies that most systems do not generalize beyond their training sets. This also reinforces the under-appreciated point that training data is as important as the model itself. We conclude with directions for future progress.
Adaptive Dither Voting for Robust Spatial Verification
Authors: Kunio Kashino Xiaomeng Wu
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Hough voting in a geometric transformation space allows us to realize spatial verification, but remains sensitive to feature detection errors because of the inflexible quantization of single feature correspondences. To handle this problem, we propose a new method, called adaptive dither voting, for robust spatial verification. For each correspondence, instead of hard-mapping it to a single transformation, the method augments its description by using multiple dithered transformations that are deterministically generated by the other correspondences. The method reduces the probability of losing correspondences during transformation quantization, and provides high robustness as regards mismatches by imposing three geometric constraints on the dithering process. We also propose exploiting the non-uniformity of a Hough histogram as the spatial similarity to handle multiple matching surfaces. Extensive experiments conducted on four datasets show the superiority of our method. The method outperforms its state-of-the-art counterparts in both accuracy and scalability, especially when it comes to the retrieval of small, rotated objects.
Alternating Co-Quantization for Cross-Modal Hashing
Authors: Hiroyuki Arai, Yukinobu Taniguchi Go Irie
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: This paper addresses the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. Many unimodal hashing studies have proven that both similarity preservation of data and maintenance of quantization quality are essential for improving retrieval performance with binary hash codes. However, most existing cross-modal hashing methods mainly have focused on the former, and the latter still remains almost untouched. We propose a method to minimize the binary quantization errors, which is tailored to cross-modal hashing. Our approach, named Alternating Co-Quantization (ACQ), alternately seeks binary quantizers for each modality space with the help of connections to other modality data so that they give minimal quantization errors while preserving data similarities. ACQ can be coupled with various existing cross-modal dimension reduction methods such as Canonical Correlation Analysis (CCA) and substantially boosts their retrieval performance in the Hamming space. Extensive experiments demonstrate that ACQ can outperform several state-of-the-art methods, even when it is combined with simple CCA.
Learning Deep Representation With Large-Scale Attributes
Authors: Hongyang Li, Xingyu Zeng, Xiaogang Wang Wanli Ouyang
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Learning strong feature representations from large scale supervision has achieved remarkable success in computer vision as the emergence of deep learning techniques. It is driven by big visual data with rich annotations. This paper contributes a large-scale object attribute database (The dataset is available on  www.ee.cuhk.edu.hk/ xgwang/ImageNetAttribute.html) that contains  rich attribute annotations (over 300 attributes) for ~180k samples and 494 object classes. Based on the ImageNet object detection dataset, it annotates the rotation, viewpoint, object part location, part occlusion, part existence, common attributes, and class-specific attributes. Then we use this dataset to train deep representations and extensively evaluate how these attributes are useful on the general object detection task. In order to make better use of the attribute annotations, a deep learning scheme is proposed by modeling the relationship of attributes and hierarchically clustering them into semantically meaningful mixture types. Experimental results show that the attributes are helpful in learning better features and improving the object detection accuracy by 2.6% in mAP on the ILSVRC 2014 object detection dataset and 2.4% in mAP on PASCAL VOC 2007 object detection dataset. Such improvement is well generalized across datasets.
Deep Learning Strong Parts for Pedestrian Detection
Authors: Ping Luo, Xiaogang Wang, Xiaoou Tang Yonglong Tian
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Recent advances in pedestrian detection are attained by transferring the learned features of Convolutional Neural Network (ConvNet) to pedestrians. This ConvNet is typically pre-trained with massive general object categories (e.g. ImageNet). Although these features are able to handle variations such as poses, viewpoints, and lightings, they may fail when pedestrian images with complex occlusions are present. Occlusion handling is one of the most important problem in pedestrian detection. Unlike previous deep models that directly learned a single detector for pedestrian detection, we propose DeepParts, which consists of extensive part detectors. DeepParts has several appealing properties. First, DeepParts can be trained on weakly labeled data, i.e. only pedestrian bounding boxes without part annotations are provided. Second, DeepParts is able to handle low IoU positive proposals that shift away from ground truth. Third, each part detector in DeepParts is a strong detector that can detect pedestrian by observing only a part of a proposal. Extensive experiments in Caltech dataset demonstrate the effectiveness of DeepParts, which yields a new state-of-the-art miss rate of 11:89%, outperforming the second best method by 10%.
Flowing ConvNets for Human Pose Estimation in Videos
Authors: James Charles, Andrew Zisserman Tomas Pfister
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: The objective of this work is human pose estimation in videos, where multiple frames are available.  We investigate a ConvNet architecture that is able to benefit from temporal context by combining information across the multiple frames using optical flow.  To this end we propose a network architecture with the following novelties: (i) a deeper network than previously investigated for regressing heatmaps; (ii) spatial fusion layers that learn an implicit spatial model; (iii) optical flow is used to align heatmap predictions from neighbouring frames; and (iv) a final parametric pooling layer which learns to combine the aligned heatmaps into a pooled confidence map.  We show that this architecture outperforms a number of others, including one that uses optical flow solely at the input layers, one that regresses joint coordinates directly, and one that predicts heatmaps without spatial fusion.  The new architecture outperforms the state of the art by a large margin on three video pose estimation datasets, including the very challenging Poses in the Wild dataset, and outperforms other deep methods that don't use a graphical model on the single-image FLIC benchmark (and also Chen & Yuille and Tompson et al. in the high precision region).
Top Rank Supervised Binary Coding for Visual Search
Authors: Wei Liu, Rongrong Ji, David A. Meyer, John R. Smith Dongjin Song
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In recent years, binary coding techniques are becoming increasingly popular because of their high efficiency in handling large-scale computer vision applications. It has been demonstrated that supervised binary coding techniques that leverage supervised information can significantly enhance the coding quality, and hence greatly benefit visual search tasks. Typically, a modern binary coding method seeks to learn a group of coding functions which compress data samples into binary codes. However, few methods pursued the coding functions such that the precision at the top of a ranking list according to Hamming distances of the generated binary codes is optimized. In this paper, we propose a novel supervised binary coding approach, namely Top Rank Supervised Binary Coding (Top-RSBC), which explicitly focuses on optimizing the precision of top positions in a Hamming-distance ranking list towards preserving the supervision information. The core idea is to train the disciplined coding functions, by which the mistakes at the top of a Hamming-distance ranking list are penalized more than those at the bottom. To solve such coding functions, we relax the original discrete optimization objective with a continuous surrogate, and derive a stochastic gradient descent to optimize the surrogate objective. To further reduce the training time cost, we also design an online learning algorithm to optimize the surrogate objective more efficiently. Empirical studies based upon three benchmark image datasets demonstrate that the proposed binary coding approach achieves superior image search accuracy over the state-of-the-arts.
BubbLeNet: Foveated Imaging for Visual Discovery
Authors: Noah Snavely Kevin Matzen
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose a new method for turning an Internet-scale corpus of categorized images into a small set of human-interpretable discriminative visual elements using powerful tools based on deep learning. A key challenge with deep learning methods is generating human-interpretable models. To address this, we propose a new technique that uses bubble images -- images where most of the content has been obscured -- to identify spatially localized, discriminative content in each image. By modifying the model training procedure to use both the source imagery and these bubble images, we can arrive at final models which retain much of the original classification performance, but are much more amenable to identifying interpretable visual elements. We apply our algorithm to a wide variety of datasets, including two new Internet-scale datasets of people and places, and show applications to visual mining and discovery. Our method is simple, scalable, and produces visual elements that are highly representative compared to prior work.
PQTable: Fast Exact Asymmetric Distance Neighbor Search for Product Quantization Using Hash Tables
Authors: Toshihiko Yamasaki, Kiyoharu Aizawa Yusuke Matsui
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: We propose the product quantization table (PQTable), a product quantization-based hash table that is fast and requires neither parameter tuning nor training steps. The PQTable produces exactly the same results as a linear PQ search, and is 10^2 to 10^5 times faster when tested on the SIFT1B data. In addition, although state-of-the-art performance can be achieved by previous inverted-indexing-based approaches, such methods do require manually designed parameter setting and much training, whereas our method is free from them. Therefore, PQTable offers a practical and useful solution for real-world problems.
Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions
Authors: Stefan Lee, David J. Crandall, Chen Yu Sven Bambach
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Hands appear very often in egocentric video, and their appearance and pose give important cues about what people are doing and what they are paying attention to. But existing work in hand detection has made strong assumptions that work well in only simple scenarios, such as with limited interaction with other people or in lab settings. We develop methods to locate and distinguish between hands in egocentric video using strong appearance models with Convolutional Neural Networks, and introduce a simple candidate region generation approach that outperforms existing techniques at a fraction of the computational cost. We show how these high-quality bounding boxes can be used to create accurate pixelwise hand regions, and as an application, we investigate the extent to which hand segmentation alone can distinguish between different activities.  We evaluate these techniques on a new dataset of 48 first-person videos (along with pixel-level ground truth for over 15,000 hand instances) of people interacting in realistic environments.
Fast and Accurate Head Pose Estimation via Random Projection Forests
Authors: Ming-Hsuan Yang, Songhwai Oh Donghoon Lee
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: In this paper, we consider the problem of estimating the gaze direction of a person from a low-resolution image.  Under this condition, reliably extracting facial features is very difficult.  We propose a novel head pose estimation algorithm based on compressive sensing.  Head image patches are mapped to a large feature space using the proposed extensive, yet efficient filter bank.  The filter bank is designed to generate sparse responses of color and gradient information, which can be compressed using random projection, and classified by a random forest.  Extensive experiments on challenging datasets show that the proposed algorithm performs favorably against the state-of-the-art methods on head pose estimation in low-resolution images degraded by noise, occlusion, and blurring.
An MRF-Poselets Model for Detecting Highly Articulated Humans
Authors: Minh-Khoi Tran, Sai-Kit Yeung Duc Thanh Nguyen
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Detecting highly articulated objects such as humans is a challenging problem. This paper proposes a novel part-based model built upon poselets, a notion of parts, and Markov Random Field (MRF) for modelling the human body structure under the variation of human poses and viewpoints. The problem of human detection is then formulated as maximum a posteriori (MAP) estimation in the MRF model. Variational mean field method, a robust statistical inference, is adopted to approximate the MAP estimation. The proposed method was evaluated and compared with existing methods on different test sets including H3D and PASCAL VOC 2007-2009. Experimental results have favourbly shown the robustness of the proposed method in comparison to the state-of-the-art.
Beyond Tree Structure Models: A New Occlusion Aware Graphical Model for Human Pose Estimation
Authors: Junge Zhang, Kaiqi Huang Lianrui Fu
Published in: IEEE International Conference on Computer Vision (ICCV), 2015 (Main Conference Track), 2015

Abstract: Occlusion is a main challenge for human pose estimation, which is largely ignored in popular tree structure models. The tree structure model is simple and convenient for exact inference, but short in modeling the occlusion coherence especially in the case of self-occlusion. We propose an occlusion aware graphical model which is able to model both self-occlusion and occlusion by the other objects simultaneously. The proposed model structure can encode the interactions between human body parts and objects, and hence enable it to learn occlusion coherence from data discriminatively. We evaluate our model on several public benchmarks for human pose estimation including challenging subsets featuring significant occlusion. The experimental results show that our method obtains comparable accuracy with the state-of-the-arts, and achieves promising performance in 2D human pose estimation with occlusion.