pwc-archive/papers-with-abstracts huggingface dataset:
Detailed Full-Body Reconstructions of Moving People From Monocular RGB-D Sequences
Authors: Federica Bogo, Michael J. Black, Matthew Loper, Javier Romero
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: 3D geometry

We accurately estimate the 3D geometry and appearance of the human body from a monocular RGB-D sequence of a user moving freely in front of the sensor. Range data in each frame is first brought into alignment with a multi-resolution 3D body model in a coarse-to-fine process. The method then uses geometry and image texture over time to obtain accurate shape, pose, and appearance information despite unconstrained motion, partial views, varying resolution, occlusion, and soft tissue deformation. Our novel body model has variable shape detail, allowing it to capture faces with a high-resolution deformable head model and body shape with lower-resolution. Finally we combine range data from an entire sequence to estimate a high-resolution displacement map that captures fine shape details. We compare our recovered models with high-resolution scans from a professional system and with avatars created by a commercial product. We extract accurate 3D avatars from challenging motion sequences and even capture soft tissue dynamics.
Efficient Solution to the Epipolar Geometry for Radially Distorted Cameras
Authors: Zuzana Kukelova, Jan Heller, Martin Bujnak, Andrew Fitzgibbon, Tomas Pajdla
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12

The estimation of the epipolar geometry of two cameras from image matches is a fundamental problem of computer vision with many applications. While the closely related problem of estimating relative pose of two different uncalibrated cameras with radial distortion is of particular importance, none of the previously published methods is suitable for practical applications. These solutions are either numerically unstable, sensitive to noise, based on a large number of point correspondences, or simply too slow for real-time applications. In this paper, we present a new efficient solution to this problem that uses 10 image correspondences. By manipulating ten input polynomial equations, we derive a degree 10 polynomial equation in one variable. The solutions to this equation are efficiently found using the Sturm sequences method. In the experiments, we show that the proposed solution is stable, noise resistant, and fast, and as such efficiently usable in a practical Structure-from-Motion pipeline.
Learning a Descriptor-Specific 3D Keypoint Detector
Authors: Samuele Salti, Federico Tombari, Riccardo Spezialetti, Luigi Di Stefano
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Binary Classification, Keypoint Detection

Keypoint detection represents the first stage in the majority of modern computer vision pipelines based on automatically established correspondences between local descriptors. However, no standard solution has emerged yet in the case of 3D data such as point clouds or meshes, which exhibit high variability in level of detail and noise. More importantly, existing proposals for 3D keypoint detection rely on geometric saliency functions that attempt to maximize repeatability rather than distinctiveness of the selected regions, which may lead to sub-optimal performance of the overall pipeline. To overcome these shortcomings, we cast 3D keypoint detection as a binary classification between points whose support can be correctly matched by a predefined 3D descriptor or not, thereby learning a descriptor-specific detector that adapts seamlessly to different scenarios. Through  experiments on several public datasets, we show that this novel approach to the design of a keypoint detector represents a flexible solution that, nonetheless, can provide state-of-the-art descriptor matching performance.
Component-Wise Modeling of Articulated Objects
Authors: Valsamis Ntouskos, Marta Sanzari, Bruno Cafaro, Federico Nardi, Fabrizio Natola, Fiora Pirri, Manuel Ruiz
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Object

We introduce a novel framework for modeling articulated objects based on the aspects of their components. By decomposing the object into components, we divide the problem in smaller modeling tasks. After obtaining 3D models for each component aspect by employing a shape deformation paradigm, we merge them together, forming the object components. The final model is obtained by assembling the components using an optimization scheme which fits the respective 3D models to the corresponding apparent contours in a reference pose.  The results suggest that our approach can produce realistic 3D models of articulated objects in reasonable time.
A Collaborative Filtering Approach to Real-Time Hand Pose Estimation
Authors: Chiho Choi, Ayan Sinha, Joon Hee Choi, Sujin Jang, Karthik Ramani
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Collaborative Filtering, Hand Pose Estimation, Pose Estimation, Recommendation Systems

Collaborative filtering aims to predict unknown user ratings in a recommender system by collectively assessing known user preferences. In this paper, we first draw analogies between collaborative filtering and the pose estimation problem. Specifically, we recast the hand pose estimation problem as the cold-start problem for a new user with unknown item ratings in a recommender system. Inspired by fast and accurate matrix factorization techniques for collaborative filtering, we develop a real-time algorithm for estimating the hand pose from RGB-D data of a commercial depth camera. First, we efficiently identify nearest neighbors using local shape descriptors in the RGB-D domain from a library of hand poses with known pose parameter values. We then use this information to evaluate the unknown pose parameters using a joint matrix factorization and completion (JMFC) approach. Our quantitative and qualitative results suggest that our approach is robust to variation in hand configurations while achieving real time performance (29 FPS) on a standard computer.
On the Equivalence of Moving Entrance Pupil and Radial Distortion for Camera Calibration
Authors: Avinash Kumar, Narendra Ahuja
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Camera Calibration

Radial distortion for ordinary (non-fisheye) camera lenses has traditionally been modeled as an infinite series function of radial location of an image pixel from the image center. While there has been enough empirical evidence  to show that such a model is accurate and sufficient for radial distortion calibration, there has not been much analysis on the geometric/physical understanding of radial distortion from a camera calibration perspective. In this paper, we show using a thick-lens imaging model, that the variation of entrance pupil location as a function of incident image ray angle is directly responsible for radial distortion in captured images. Thus, unlike as proposed in the current state-of-the-art in camera calibration, radial distortion  and entrance pupil movement are equivalent and need not be modeled together.  By modeling only entrance pupil motion instead of radial distortion,  we achieve two main benefits;  first, we obtain comparable if not better pixel re-projection error than traditional methods; second, and more importantly, we directly back-project a radially distorted image pixel along the  true image ray which formed it. Using a thick-lens setting, we show that such a back-projection is more  accurate than the two-step method of undistorting an image pixel and  then back-projecting it. We have applied this calibration method to the problem of generative depth-from-focus using focal stack to get accurate depth estimates.
A Linear Generalized Camera Calibration From Three Intersecting Reference Planes
Authors: Mai Nishimura, Shohei Nobuhara, Takashi Matsuyama, Shinya Shimizu, Kensaku Fujii
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Camera Calibration

This paper presents a new generalized (or ray-pixel, raxel) camera calibration algorithm for camera systems involving distortions by unknown refraction and reflection processes. The key idea is use of intersections of calibration planes, while conventional methods utilized collinearity constraints of points on the planes. We show that intersections of calibration planes can realize a simple linear algorithm, and that our method can be applied to any ray-distributions while conventional methods require knowing the ray-distribution class in advance. Evaluations using synthesized and real datasets demonstrate the performance of our method quantitatively and qualitatively.
Towards Pointless Structure From Motion: 3D Reconstruction and Camera Parameters From General 3D Curves
Authors: Irina Nurutdinova, Andrew Fitzgibbon
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: 3D Reconstruction, Position

Modern structure from motion (SfM) remains dependent on point features to recover camera positions, meaning that reconstruction is severely hampered in low-texture environments, for example scanning a plain coffee cup on an uncluttered table.  We show how 3D curves can be used to refine camera position estimation in challenging low-texture scenes.  In contrast to previous work, we allow the curves to be partially observed in all images, meaning that for the first time, curve-based SfM can be demonstrated in realistic scenes.  The algorithm is based on bundle adjustment, so needs an initial estimate, but even a poor estimate from a few point correspondences can be substantially improved by including curves, suggesting that this method would benefit many existing systems.
Attributed Grammars for Joint Estimation of Human Attributes, Part and Pose
Authors: Se-Young Park, Song-Chun Zhu
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Attribute, Human Parsing, Pose Estimation

In this paper, we are interested in developing compositional models to explicit representing  pose, parts and attributes and tackling the tasks of attribute recognition, pose estimation and part localization jointly. This is different from the recent trend of using CNN-based approaches for training and testing on these tasks separately with a large amount of data. Conventional attribute models typically use a large number of region-based attribute classifiers on parts of pre-trained pose estimator without explicitly detecting the object or its parts, or considering the correlations between attributes. In contrast, our approach jointly represents both the object parts and their semantic attributes within a unified compositional hierarchy. We apply our attributed grammar model to the task of human parsing by simultaneously performing part localization and attribute recognition. We show our modeling helps performance improvements on pose-estimation task and also outperforms on other existing methods on attribute prediction task.
Real-Time Pose Estimation Piggybacked on Object Detection
Authors: Roman Juranek, Adam Herout, Marketa Dubska, Pavel Zemcik
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: C++ code, Object, object-detection, Object Detection, Pose Estimation, vehicle detection

We present an object detector coupled with pose estimation directly in a single compact and simple model, where the detector shares extracted image features with the pose estimator. The output of the classification of each candidate window consists of both object score and likelihood map of poses. This extension introduces negligible overhead during detection so that the detector is still capable of real time operation. We evaluated the proposed approach on the problem of vehicle detection. We used existing datasets with viewpoint/pose annotation (WCVP, 3D objects, KITTI).  Besides that, we collected a new traffic surveillance dataset COD20k which fills certain gaps of the existing datasets and we make it public. The experimental results show that the proposed approach is comparable with state-of-the-art approaches in terms of accuracy, but it is considerably faster - easily operating in real time (Matlab with C++ code). The source codes and the collected COD20k dataset are made public along with the paper.
Understanding and Predicting Image Memorability at a Large Scale
Authors: Aditya Khosla, Akhil S. Raju, Antonio Torralba, Aude Oliva
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12

Progress in estimating visual memorability has been limited by the small scale and lack of variety of benchmark data. Here, we introduce a novel experimental procedure to objectively measure human memory, building the largest annotated image memorability dataset to date (with 60,000 labeled images from a diverse array of sources). Using Convolutional Neural Networks (CNNs), we show that fine-tuned deep features outperform all other features by a large margin, reaching a rank correlation of 0.64, near human consistency (0.68). Analysis of the responses of the high-level CNN layers shows which objects and regions are positively, and negatively, correlated with memorability, allowing us to create memorability maps for each image and provide a concrete method to perform image memorability manipulation. This work demonstrates that one can now robustly estimate the memorability of images from many different classes, positioning memorability and deep memorability features as prime candidates to estimate the utility of information for cognitive systems.
Multiple Granularity Descriptors for Fine-Grained Categorization
Authors: Dequan Wang, Zhiqiang Shen, Jie Shao, Wei zhang, xiangyang xue, Zheng Zhang
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12

Fine-grained categorization, which aims to distinguish subordinate-level categories such as bird species or dog breeds, is an extremely challenging task. This is due to two main issues: how to localize discriminative regions for recognition and how to learn sophisticated features for representation. Neither of them is easy to handle if there is insufficient labeled data. We leverage the fact that a subordinate-level object already has other labels in its ontology tree. These "free" labels can be used to train a series of CNN-based classifiers, each specialized at one grain level. The internal representations of these networks have different region of interests, allowing the construction of multi-grained descriptors that encode informative and discriminative features covering all the grain levels. Our multiple granularity framework can be learned with the weakest supervision, requiring only image-level label and avoiding the use of labor-intensive bounding box or part annotations. Experimental results on three challenging fine-grained image datasets demonstrate that our approach outperforms state-of-the-art algorithms, including those requiring strong labels.
Guiding the Long-Short Term Memory Model for Image Caption Generation
Authors: Xu Jia, Efstratios Gavves, Basura Fernando, Tinne Tuytelaars
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Caption Generation

In this work we focus on the problem of image caption generation. We propose an extension of the long short term memory (LSTM)  model, which we coin gLSTM for short. In particular, we add semantic information extracted from the image as extra input to each unit of the LSTM block, with the aim of guiding the model towards solutions that are more tightly coupled to the image content. Additionally, we explore different length normalization strategies for beam search to avoid bias towards short sentences. On various benchmark datasets such as Flickr8K, Flickr30K and MS COCO, we obtain results that are on par with or better than the current state-of-the-art.
Just Noticeable Differences in Visual Attributes
Authors: Aron Yu, Kristen Grauman
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Attribute

We explore the problem of predicting "just noticeable differences" in a visual attribute.  While some pairs of images have a clear ordering for an attribute (e.g., A is more sporty than B), for others the difference may be indistinguishable to human observers.  However, existing relative attribute models are unequipped to infer partial orders on novel data.   Attempting to map relative attribute ranks to equality predictions is non-trivial, particularly since the span of indistinguishable pairs in attribute space may vary in different parts of the feature space.  We develop a Bayesian local learning strategy to infer when images are indistinguishable for a given attribute.  On the UT-Zap50K shoes and LFW-10 faces datasets, we outperform a variety of alternative methods.  In addition, we show the practical impact on fine-grained visual search.
Localize Me Anywhere, Anytime: A Multi-Task Point-Retrieval Approach
Authors: Guoyu Lu, Yan Yan, Li Ren, Jingkuan Song, Nicu Sebe, Chandra Kambhamettu
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Image-Based Localization, Multi-Task Learning, Retrieval

Image-based localization is an essential complement to GPS localization. Current image-based localization methods are based on either 2D-to-3D or 3D-to-2D to find the correspondences, which ignore the real scene geometric attributes. The main contribution of our paper is that we use a 3D model reconstructed by a short video as the query to realize 3D-to-3D localization under a multi-task point retrieval framework. Firstly, the use of a 3D model as the query enables us to efficiently select location candidates. Furthermore, the reconstruction of 3D model exploits the correlations among different images, based on the fact that images captured from different views for SfM share information through matching features. By exploring shared information (matching features) across multiple related tasks (images of the same scene captured from different views), the visual feature's view-invariance property can be improved in order to get to a higher point retrieval accuracy. More specifically, we use multi-task point retrieval framework to explore the relationship between descriptors and the 3D points, which extracts the discriminant points for more accurate 3D-to-3D correspondences retrieval. We further apply multi-task learning (MTL) retrieval approach on thermal images to prove that our MTL retrieval framework also provides superior performance for the thermal domain. This application is exceptionally helpful to cope with the localization problem in an environment with limited light sources.
Unsupervised Domain Adaptation for Zero-Shot Learning
Authors: Elyor Kodirov, Tao Xiang, Zhen-Yong Fu, Shaogang Gong
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Action Recognition, Domain Adaptation, Temporal Action Localization, Transfer Learning, Unsupervised Domain Adaptation, Zero-Shot Learning

Zero-shot learning (ZSL) can be considered as a special case of transfer learning where the source and target domains have different tasks/label spaces and the target domain is unlabelled, providing little guidance for the knowledge transfer. A ZSL method typically assumes that the two domains share a common semantic representation space, where a visual feature vector extracted from an image/video can be  projected/embedded using a projection function. Existing approaches learn the projection function from the source domain and apply it without adaptation to the target domain. They are thus based on naive knowledge transfer and the learned projections are prone to the domain shift problem. In this paper a novel ZSL method is proposed based on unsupervised domain adaptation. Specifically, we formulate a novel regularised sparse coding framework which uses the target domain class labels' projections in the semantic space  to regularise the learned target domain projection thus effectively overcoming the projection domain shift problem. Extensive experiments on four object and action recognition benchmark datasets show that the proposed ZSL method significantly outperforms the state-of-the-arts.
Visual Madlibs: Fill in the Blank Description Generation and Question Answering
Authors: Licheng Yu, Eunbyung Park, Alexander C. Berg, Tamara L. Berg
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Multiple-choice, Question Answering

In this paper, we introduce a new dataset consisting of 360,001 focused natural language descriptions for 10,738 images.  This dataset, the Visual Madlibs dataset, is collected using automatically produced fill-in-the-blank templates designed to gather targeted descriptions about: people and objects, their appearances, activities, and interactions, as well as inferences about the general scene or its broader context. We provide several analyses of the Visual Madlibs dataset and demonstrate its applicability to two new description generation tasks: focused description generation, and multiple-choice question-answering for images. Experiments using joint-embedding and deep learning methods show promising results on these tasks.
Scene-Domain Active Part Models for Object Representation
Authors: Zhou Ren, Chaohui Wang, Alan L. Yuille
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Object, Viewpoint Estimation

In this paper, we are interested in enhancing the expressivity and robustness of part-based models for object representation, in the common scenario where the training data are based on 2D images. To this end, we propose scene-domain active part models (SDAPM), which reconstruct and characterize the 3D geometric statistics between object's parts in 3D scene-domain by using 2D training data in the image-domain alone. And on top of this, we explicitly model and handle occlusions in SDAPM. Together with the developed learning and inference algorithms, such a model provides rich object descriptions, including 2D object and parts localization, 3D landmark shape and camera viewpoint, which offers an effective representation to various image understanding tasks, such as object and parts detection, 3D landmark shape and viewpoint estimation from images. Experiments on the above tasks show that SDAPM outperforms previous part-based models, and thus demonstrates the potential of the proposed technique.
A Unified Multiplicative Framework for Attribute Learning
Authors: Kongming Liang, Hong Chang, Shiguang Shan, Xilin Chen
Date: 2015-12-01 00:00:00
Proceeding: ICCV 2015 12
Tasks: Attribute, Zero-Shot Learning

Attributes are mid-level semantic properties of objects. Recent research has shown that visual attributes can benefit many traditional learning problems in computer vision community. However, attribute learning is still a challenging problem as the attributes may not always be predictable directly from input images and the variation of visual attributes is sometimes large across categories. In this paper, we propose a unified multiplicative framework for attribute learning, which tackles the key problems. Specifically, images and category information are jointly projected into a shared feature space, where the latent factors are disentangled and multiplied for attribute prediction. The resulting attribute classifier is category-specific instead of being shared by all  categories. Moreover, our method can leverage auxiliary data to enhance the predictive ability of attribute classifiers, reducing the effort of instance-level attribute annotation to some extent. Experimental results show that our method achieves superior performance on both instance-level and category-level attribute prediction. For zero-shot learning based on attributes, our method significantly improves the state-of-the-art performance on AwA dataset and achieves comparable performance on CUB dataset.