mtybilly/Instruct2DS huggingface dataset:
Embedded Phase Shifting: Robust Phase Shifting With Embedded Signals
Authors: Daniel Moreno, Kilho Son, Gabriel Taubin
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We introduce Embedded PS, a new robust and accurate phase shifting algorithm for 3D scanning. The method projects only high frequency sinusoidal patterns in order to reduce errors due to global illumination effects, such as subsurface scattering and interreflections. The frequency set for the projected patterns is specially designed so that our algorithm can extract a set of embedded low frequency sinusoidals with simple math. All the signals, patterns high and embedded low frequencies, are used with temporal phase unwrapping to compute absolute phase values in closed-form, without quantization or approximation via LUT, resulting in fast computation. The absolute phases provide correspondences from projector to camera pixels which enable to recover 3D points using optical triangulation. The algorithm estimates multiple absolute phase values per pixel which are combined to reduce measurement noise while preserving fine details. We prove that embedded periodic signals can be recovered from any periodic signal, not just sinusoidal signals, which may result in further improvements for other 3D imaging methods. Several experiments are presented showing that our algorithm produces more robust and accurate 3D scanning results than state-of-the-art methods for challenging surface materials, with an equal or smaller number of projected patterns and at lower computational cost.
Shape and Light Directions From Shading and Polarization
Authors: Trung Ngo Thanh, Hajime Nagahara, Rin-ichiro Taniguchi
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We introduce a method to recover the shape of a smooth dielectric object from polarization images taken with a light source from different directions. We present two constraints on shading and polarization and use both in a single optimization scheme. This integration is motivated by the fact that photometric stereo and polarization-based methods have complementary abilities. The polarization-based method can give strong cues for the surface orientation and refractive index, which are independent of the light direction. However, it has ambiguities in selecting between two ambiguous choices of the surface orientation, in the relationship between refractive index and zenith angle (observing angle), and limited performance for surface points with small zenith angles, where the polarization effect is weak. In contrast, photometric stereo method with multiple light sources can disambiguate the surface orientation and give a strong relationship between the surface normals and light directions. However, it has limited performance for large zenith angles, refractive index estimation, and faces the ambiguity in case the light direction is unknown. Taking their advantages, our proposed method can recover the surface normals for both small and large zenith angles, the light directions, and the refractive indexes of the object. The proposed method is successfully evaluated by simulation and real-world experiments.
3D Deep Shape Descriptor
Authors: Yi Fang, Jin Xie, Guoxian Dai, Meng Wang, Fan Zhu, Tiantian Xu, Edward Wong
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Shape descriptor is a concise yet informative representation that provides a 3D object with an identification as a member of some category. This paper developed a concise deep shape descriptor for the first time to address challenging issues from ever-growing 3D datasets in areas as diverse as engineering, medicine, and biology. Specifically, the proposed approach developed novel techniques to extract concise but geometrically informative shape descriptor, new definitions of Eigen-shape descriptor and Fisher-shape descriptor to guide the training strategy for deep neural network, and deep shape descriptor with discriminative capacity of maximizing the inter-class margin while minimizing the intra-class variance. Our approach addressed the challenges for shape analysis techniques posed by the complexity of 3D model and data representation and geometric structural variations and noise present in 3D models. The experimental results on 3D shape retrieval demonstrate that our proposed deep shape descriptor is superior to other state-of-the-art approaches on handling noise, incompleteness and 3D shape structural variations.
Cross-Age Face Verification by Coordinating With Cross-Face Age Verification
Authors: Liang Du, Haibin Ling
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper we present a novel framework for cross-age face verification (FV) by seeking help from its ``competitor" named cross-face age verification (AV), i.e., deciding whether two face photos are taken at similar ages. While FV and AV share some common features, FV pursues age insensitivity and AV seeks age sensitivity.  Such correlation suggests that AV may be used to guide feature selection in FV, i.e., by reducing the chance of choosing age sensitive features.    Driven by this intuition, we propose to learn a solution for cross-age face verification by coordinating with a solution for age verification. Specifically, a joint additive model is devised to simultaneously handling both tasks, while encoding feature coordination by a competition regularization term.    Then, an alternating greedy coordinate descent (AGCD) algorithm is developed to solve this joint model.  As shown in our experiments, the algorithm effectively balances feature sharing and feature exclusion between the two tasks; and, for face verification, the algorithm effectively removes distracting features used in age verification.    To evaluate the proposed algorithm, we conduct cross-age face verification experiments using two benchmark cross-age face datasets, FG-Net and MORPH. In all experiments, our algorithm achieves very promising results and outperforms all previously tested solutions.
Beyond Mahalanobis Metric: Cayley-Klein Metric Learning
Authors: Yanhong Bi, Bin Fan, Fuchao Wu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Cayley-Klein metric is a kind of non-Euclidean metric suitable for projective space. In this paper, we introduce it into the computer vision community as a powerful metric and an alternative to the widely studied Mahalanobis metric. We show that besides its good characteristic in non-Euclidean space, it is a generalization of Mahalanobis metric in some specific cases. Furthermore, as many Mahalanobis metric learning, we give two kinds of Cayley-Klein metric learning methods: MMC Cayley-Klein metric learning and LMNN Cayley-Klein metric learning. Experiments have shown the superiority of Cayley-Klein metric over Mahalanobis ones and the effectiveness of our Cayley-Klein metric learning methods.
From Dictionary of Visual Words to Subspaces: Locality-Constrained Affine Subspace Coding
Authors: Peihua Li, Xiaoxiao Lu, Qilong Wang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The locality-constrained linear coding (LLC) is a very successful feature coding method in image classification. It makes known the importance of locality constraint which brings high efficiency and local smoothness of the codes. However, in the LLC method the geometry of feature space is described by an ensemble of representative points (visual words) while discarding the geometric structure immediately surrounding them. Such a dictionary only provides a crude, piecewise constant approximation of the data manifold. To approach this problem, we propose a novel feature coding method called locality-constrained affine subspace coding (LASC). The data manifold in LASC is characterized by an ensemble of subspaces attached to the representative points (or affine subspaces), which can provide a piecewise linear approximation of the manifold. Given an input descriptor, we find its top-k neighboring subspaces, in which the descriptor is linearly decomposed and weighted to form the first-order LASC vector. Inspired by the success of usage of higher-order information in image classification, we propose the second-order LASC vector based on the Fisher information metric for further performance improvement. We make experiments on challenging benchmarks and experiments have shown the LASC method is very competitive.
FPA-CS: Focal Plane Array-Based Compressive Imaging in Short-Wave Infrared
Authors: Huaijin Chen, M. Salman Asif, Aswin C. Sankaranarayanan, Ashok Veeraraghavan
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Cameras for imaging in short and mid-wave infrared spectra are significantly more expensive than their counterparts in visible imaging. As a result, high-resolution imaging in those spectrum remains beyond the reach of most consumers. Over the last decade, compressive sensing (CS) has emerged as a potential means to realize inexpensive short-wave infrared cameras. One approach for doing this is the single-pixel camera (SPC) where a single detector acquires coded measurements of a high-resolution image. A computational reconstruction algorithm is then used to recover the image from these coded measurements. Unfortunately, the measurement rate of a SPC is insufficient to enable imaging at high spatial and temporal resolutions. We present a focal plane array-based compressive sensing (FPA-CS) architecture that achieves high spatial and temporal resolutions. The idea is to use an array of SPCs that sense in parallel to increase the measurement rate, and consequently, the achievable spatio-temporal resolution of the camera. We develop a proof-of-concept prototype in the short-wave infrared using a sensor with 64 x 64 pixels; the prototype provides a 4096x increase in the measurement rate compared to the SPC and achieves a megapixel resolution at video rate using CS techniques.
BOLD - Binary Online Learned Descriptor For Efficient Image Matching
Authors: Vassileios Balntas, Lilian Tang, Krystian Mikolajczyk
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper we propose a novel approach to generate a binary descriptor optimized for each image patch independently.    The approach is inspired by the linear discriminant embedding that simultaneously increases inter and decreases intra class distances. A set of discriminative and uncorrelated binary tests is established from all possible tests in an offline training process. The patch adapted descriptors are then efficiently built online from a subset of tests which lead to lower intra class distances thus a more robust descriptor. A patch descriptor consists of two binary strings where one represents the results of the tests and the other indicates the subset of the patch-related robust tests that are used for calculating a masked Hamming distance.  Our experiments on three different benchmarks demonstrate improvements in  matching performance, and illustrate that per-patch optimization outperforms global optimization.
Defocus Deblurring and Superresolution for Time-of-Flight Depth Cameras
Authors: Lei Xiao, Felix Heide, Matthew O'Toole, Andreas Kolb, Matthias B. Hullin, Kyros Kutulakos, Wolfgang Heidrich
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Continuous-wave time-of-flight (ToF) cameras show great promise as low-cost depth image sensors in mobile applications. However, they also suffer from several challenges, including limited illumination intensity, which mandates the use of large numerical aperture lenses, and thus results in a shallow depth of field, making it difficult to capture scenes with large variations in depth. Another shortcoming is the limited spatial resolution of currently available ToF sensors.  In this paper we analyze the image formation model for blurred ToF images. By directly working with raw sensor measurements but regularizing the recovered depth and amplitude images, we are able to simultaneously deblur and super-resolve the output of ToF cameras. Our method outperforms existing methods on both synthetic and real datasets. In the future our algorithm should extend easily to cameras that do not follow the cosine model of continuous-wave sensors, as well as to multi-frequency or multi-phase imaging employed in more recent ToF cameras.
Burst Deblurring: Removing Camera Shake Through Fourier Burst Accumulation
Authors: Mauricio Delbracio, Guillermo Sapiro
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Numerous recent approaches attempt to remove image blur due to camera shake,  either with one or multiple input images, by explicitly solving an inverse and inherently ill-posed deconvolution problem. If the photographer takes a burst of images, a modality available in virtually  all modern digital cameras, we show that it is possible to combine them to get a clean sharp version. This is done without explicitly solving any blur estimation and subsequent  inverse problem.  The proposed algorithm is strikingly simple: it performs a weighted average in the Fourier domain, with weights depending on the Fourier spectrum magnitude. The method's rationale is that camera shake has a random nature and therefore  each image in the burst is generally blurred differently.  Experiments with real camera data show that the proposed Fourier Burst Accumulation algorithm achieves state-of-the-art results an order of magnitude faster, with simplicity for on-board implementation on camera phones.
SOM: Semantic Obviousness Metric for Image Quality Assessment
Authors: Peng Zhang, Wengang Zhou, Lei Wu, Houqiang Li
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Image quality assessment (IQA) tries to estimate human perception based image visual quality in an objective manner.  Existing approaches target this problem with or without reference images.  For no-reference image quality assessment, there is no given reference image or any knowledge of the distortion type of the image.  Previous approaches measure the image quality from signal level rather than semantic analysis.  They typically depend on various features to represent local characteristic of an image.  In this paper we propose a new no-reference (NR) image quality assessment (IQA) framework based on semantic obviousness.  We discover that semantic-level factors affect human perception of image quality. With such observation, we explore semantic obviousness as a metric to perceive objects of an image. We propose to extract two types of features, one to measure the semantic obviousness of the image and the other to discover local characteristic.  Then the two kinds of features are combined for image quality estimation. The principles proposed in our approach can also be incorporated with many existing IQA algorithms to boost their performance.  We evaluate our approach on the LIVE dataset. Our approach is demonstrated to be superior to the existing NR-IQA algorithms and comparable to the state-of-the-art full-reference IQA (FR-IQA) methods. Cross-dataset experiments show the generalization ability of our approach.
DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
Authors: Wanli Ouyang, Xiaogang Wang, Xingyu Zeng, Shi Qiu, Ping Luo, Yonglong Tian, Hongsheng Li, Shuo Yang, Zhe Wang, Chen-Change Loy, Xiaoou Tang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper, we propose deformable deep convolutional neural networks for generic object detection. This new deep learning object detection diagram has innovations in multiple aspects. In the proposed new deep architecture, a new deformation constrained pooling (def-pooling) layer models the deformation of object parts with geometric constraint and penalty. A new pre-training strategy is proposed to learn feature representations more suitable for the object detection task and with good generalization capability. By changing the net structures, training strategies, adding and removing some key components in the detection pipeline, a set of models with large diversity are obtained, which significantly improves the effectiveness of model averaging. The proposed approach improves the mean averaged precision obtained by RCNN, which is the state-of-the-art, from $31\%$ to $50.3\%$ on the ILSVRC2014 detection dataset. Detailed component-wise analysis is also provided through extensive experimental evaluation, which provide a global view for people to understand the deep learning object detection pipeline.
Efficient Globally Optimal Consensus Maximisation With Tree Search
Authors: Tat-Jun Chin, Pulak Purkait, Anders Eriksson, David Suter
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Maximum consensus is one of the most popular criteria for robust estimation in computer vision. Despite its widespread use, optimising the criterion is still customarily done by randomised sample-and-test techniques, which do not guarantee optimality of the result. Several globally optimal algorithms exist, but they are too slow to challenge the dominance of randomised methods. We aim to change this state of affairs by proposing a very efficient algorithm for global maximisation of consensus. Under the framework of LP-type methods, we show how consensus maximisation for a wide variety of vision tasks can be posed as a tree search problem. This insight leads to a novel algorithm based on A* search. We propose efficient heuristic and support set updating routines that enable A* search to rapidly find globally optimal results. On common estimation problems, our algorithm is several orders of magnitude faster than previous exact methods. Our work identifies a promising solution for globally optimal consensus maximisation.
Mind's Eye: A Recurrent Visual Representation for Image Caption Generation
Authors: Xinlei Chen, C. Lawrence Zitnick
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper we explore the bi-directional mapping between images and their sentence-based descriptions. Critical to our approach is a recurrent neural network that attempts to dynamically build a visual representation of the scene as a caption is being generated or read. The representation automatically learns to remember long-term visual concepts. Our model is capable of both generating novel captions given an image, and reconstructing visual features given an image description.  We evaluate our approach on several tasks. These include sentence generation, sentence retrieval and image retrieval. State-of-the-art results are shown for the task of generating novel image descriptions. When compared to human generated captions, our automatically generated captions are equal to or preferred by humans $21.0\%$ of the time. Results are better than or comparable to state-of-the-art results on the image and sentence retrieval tasks for methods using similar visual features.
Joint Vanishing Point Extraction and Tracking
Authors: Till Kroeger, Dengxin Dai, Luc Van Gool
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We present a novel vanishing point (VP) detection and tracking algorithm for calibrated monocular image sequences.   Previous VP detection and tracking methods usually assume known camera poses for all frames or detect and track separately. We advance the state-of-the-art by combining VP extraction on a Gaussian sphere with recent advances in multi-target tracking on probabilistic occupancy fields.  The solution is obtained by solving a Linear Program (LP). This enables the joint detection and tracking of multiple VPs over sequences. Unlike existing works we do not need known camera poses, and at the same time avoid detecting and tracking in separate steps. We also propose an extension to enforce VP orthogonality. We augment an existing video dataset consisting of 48 monocular videos with multiple annotated VPs in 14448 frames for evaluation. Although the method is designed for unknown camera poses, it is also helpful in scenarios with known poses, since a multi-frame approach in VP detection helps to regularize in frames with weak VP line support.
Learning a Non-Linear Knowledge Transfer Model for Cross-View Action Recognition
Authors: Hossein Rahmani, Ajmal Mian
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper concerns action recognition from unseen and unknown views. We propose unsupervised learning of a non-linear model that  transfers knowledge from multiple views to a canonical view. The proposed Non-linear Knowledge Transfer Model (NKTM) is a deep network, with weight decay and sparsity constraints, which finds a shared high-level virtual path from videos captured from different unknown viewpoints to the same canonical view. The strength of our technique is that we learn a single NKTM for all actions and all camera viewing directions. Thus, NKTM does not require action labels during learning and knowledge of the camera viewpoints during training or testing. NKTM is learned once only from dense trajectories of synthetic points fitted to mocap data and then applied to real video data. Trajectories are coded with a general codebook learned from the same mocap data. NKTM is scalable to new action classes and training data as it does not require re-learning. Experiments on the IXMAS and N-UCLA datasets show that NKTM outperforms existing state-of-the-art methods for cross-view action recognition.
Random Tree Walk Toward Instantaneous 3D Human Pose Estimation
Authors: Ho Yub Jung, Soochahn Lee, Yong Seok Heo, Il Dong Yun
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The availability of accurate depth cameras have made real-time human pose estimation possible; however, there are still demands for faster algorithms on low power processors. This paper introduces 1000 frames per second pose estimation method on a single core CPU. A large computation gain is achieved by random walk sub-sampling. Instead of training trees for pixel-wise classification, a regression tree is trained to estimate the probability distribution to the direction toward the particular joint, relative to the current position. At test time, the direction for the random walk is randomly chosen from a set of representative directions. The new position is found by a constant step toward the direction, and the distribution for next direction is found at the new position. The continual random walk through 3D space will eventually produce an expectation of step positions, which we estimate as the joint position. A regression tree is built separately for each joint. The number of random walk steps can be assigned for each joint so that the computation time is consistent regardless of the size of body segmentation. The experiments show that even with large computation gain, the accuracy is higher or comparable to the state-of-the-art pose estimation methods.
Deep Hashing for Compact Binary Codes Learning
Authors: Venice Erin Liong, Jiwen Lu, Gang Wang, Pierre Moulin, Jie Zhou
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper, we propose a new deep hashing (DH) approach to learn compact binary codes for large scale visual search. Unlike most existing binary codes learning methods which seek a single linear projection to map each sample into a binary vector, we develop a deep neural network to seek multiple hierarchical non-linear transformations to learn these binary codes, so that the nonlinear relationship of samples can be well exploited. Our model is learned under three constraints at the top layer of the deep network: 1) the loss between the original real-valued feature descriptor and the learned binary vector is minimized, 2) the binary codes distribute evenly on each bit, and 3) different bits are as independent as possible. To further improve the discriminative power of the learned binary codes, we extend DH into supervised DH (SDH) by including one discriminative term into the objective function of DH which simultaneously maximizes the inter-class variations and minimizes the intra-class variations of the learned binary codes. Experimental results show the superiority of the proposed approach over the state-of-the-arts.
Completing 3D Object Shape From One Depth Image
Authors: Jason Rock, Tanmay Gupta, Justin Thorsen, JunYoung Gwak, Daeyun Shin, Derek Hoiem
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Our goal is to recover a complete 3D model from a depth image of an object. Existing approaches rely on user interaction or apply to a limited class of objects, such as chairs. We aim to fully automatically reconstruct a 3D model from any category. We take an exemplar-based approach: retrieve similar objects in a database of 3D models using view-based matching and transfer the symmetries and surfaces from retrieved models. We investigate completion of 3D models in three cases: novel view (model in database); novel model (models for other objects of the same category in database); and novel category (no models from the category in database).
Encoding Based Saliency Detection for Videos and Images
Authors: Thomas Mauthner, Horst Possegger, Georg Waltner, Horst Bischof
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We present a novel video saliency detection method to support human activity recognition and weakly supervised training of activity detection algorithms. Recent research has emphasized the need for analyzing salient information in videos to minimize dataset bias or to supervise weakly labeled training of activity detectors. In contrast to previous methods we do not rely on training information given by either eye-gaze or annotation data, but propose a fully unsupervised algorithm to find salient regions within videos. In general, we enforce the Gestalt principle of figure-ground segregation for both appearance and motion cues. We introduce an encoding approach that allows for efficient computation of saliency by approximating joint feature distributions. We evaluate our approach on several datasets, including challenging scenarios with cluttered background and camera motion, as well as salient object detection in images. Overall, we demonstrate favorable performance compared to state-of-the-art methods in estimating both ground-truth eye-gaze and activity annotations.
Online Sketching Hashing
Authors: Cong Leng, Jiaxiang Wu, Jian Cheng, Xiao Bai, Hanqing Lu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Recently, hashing based approximate nearest neighbor (ANN) search has attracted much attention. Extensive new algorithms have been developed and successfully applied to different applications. However, two critical problems are rarely mentioned. First, in real-world applications, the data often comes in a streaming fashion but most of existing hashing methods are batch based models. Second, when the dataset becomes huge, it is almost impossible to load all the data into memory to train hashing models. In this paper, we propose a novel approach to handle these two problems simultaneously based on the idea of data sketching. A sketch of one dataset preserves its major characters but with significantly smaller size. With a small size sketch, our method can learn hash functions in an online fashion, while needs rather low computational complexity and storage space. Extensive experiments on two large scale benchmarks and one synthetic dataset demonstrate the efficacy of the proposed method.
Enriching Object Detection With 2D-3D Registration and Continuous Viewpoint Estimation
Authors: Christopher Bongsoo Choy, Michael Stark, Sam Corbett-Davies, Silvio Savarese
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: A large body of recent work on object detection has focused on exploiting 3D CAD model databases to improve detection performance. Many of these approaches work by aligning exact 3D models to images using templates generated from renderings of the 3D models at a set of discrete viewpoints. However, the training procedures for these approaches are computationally expensive and require gigabytes of memory and storage, while the viewpoint discretization hampers pose estimation performance.  We propose an efficient method for synthesizing templates from 3D models that runs on the fly -- that is, it quickly produces detectors for an arbitrary viewpoint of a 3D model without expensive dataset-dependent training or template storage. Given a 3D model and an arbitrary continuous detection viewpoint, our method synthesizes a discriminative template by extracting features from a rendered view of the object and decorrelating spatial dependences among the features. Our decorrelation procedure relies on a gradient-based algorithm that is more numerically stable than standard decomposition-based procedures, and we efficiently search for candidate detections by computing FFT-based template convolutions. Due to the speed of our template synthesis procedure, we are able to perform joint optimization of scale, translation, continuous rotation, and focal length using Metropolis-Hastings algorithm. We provide an efficient GPU implementation of our algorithm, and we validate its performance on 3D Object Classes and PASCAL3D+ datasets.
Representing 3D Texture on Mesh Manifolds for Retrieval and Recognition Applications
Authors: Naoufel Werghi, Claudio Tortorici, Stefano Berretti, Alberto Del Bimbo
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper, we present and experiment a novel approach for representing texture of 3D mesh manifolds using local binary patterns (LBP). Using a recently proposed framework [37], we compute LBP directly on the mesh surface, either using geometric or photometric appearance. Compared to its depth-image counterpart, our approach is distinguished by the following features: a) inherits the intrinsic advantages of mesh surface (e.g., preservation of the full geometry); b) does not require normalization; c) can accommodate partial matching. In addition, it allows early-level fusion of the geometry and photometric texture modalities. Through experiments conducted on two application scenarios, namely, 3D texture retrieval and 3D face recognition, we assess the effectiveness of the proposed solution with respect to state of the art approaches.
Saliency Propagation From Simple to Difficult
Authors: Chen Gong, Dacheng Tao, Wei Liu, Stephen J. Maybank, Meng Fang, Keren Fu, Jie Yang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Saliency propagation has been widely adopted for identifying the most attractive object in an image. The propagation sequence generated by existing saliency detection methods is governed by the spatial relationships of image regions, i.e., the saliency value is transmitted between two adjacent regions. However, for the inhomogeneous difficult adjacent regions, such a sequence may incur wrong propagations. In this paper, we attempt to manipulate the propagation sequence for optimizing the propagation quality. Intuitively, we postpone the propagations to difficult regions and meanwhile advance the propagations to less ambiguous simple regions. Inspired by the theoretical results in educational psychology, a novel propagation algorithm employing the teaching-to-learn and learning-to-teach strategies is proposed to explicitly improve the propagation quality. In the teaching-to-learn step, a teacher is designed to arrange the regions from simple to difficult and then assign the simplest regions to the learner. In the learning-to-teach step, the learner delivers its learning confidence to the teacher to assist the teacher to choose the subsequent simple regions. Due to the interactions between the teacher and learner, the uncertainty of original difficult regions is gradually reduced, yielding manifest salient objects with optimized background suppression. Extensive experimental results on benchmark saliency datasets demonstrate the superiority of the proposed algorithm over twelve representative saliency detectors.
Learning an Efficient Model of Hand Shape Variation From Depth Images
Authors: Sameh Khamis, Jonathan Taylor, Jamie Shotton, Cem Keskin, Shahram Izadi, Andrew Fitzgibbon
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We describe how to learn a compact and efficient model of the surface deformation of human hands. The model is built from a set of noisy depth images of a diverse set of subjects performing different poses with their hands. We represent the observed surface using Loop subdivision of a control mesh that is deformed by our learned parametric shape and pose model. The model simultaneously accounts for variation in subject-specific shape and subject-agnostic pose. Specifically, hand shape is parameterized as a linear combination of a mean mesh in a neutral pose with a small number of offset vectors. This mesh is then articulated using standard linear blend skinning (LBS) to generate the control mesh of a subdivision surface. We define an energy that encourages each depth pixel to be explained by our model, and the use of a smooth subdivision surface allows us to optimize for all parameters jointly from a rough initialization. The efficacy of our method is demonstrated using both synthetic and real data, where it is shown that hand shape variation can be represented using only a small number of basis directions. We compare with other approaches including PCA and show a substantial improvement in the representation power of our model, while maintaining the efficiency of a linear shape basis.
On the Minimal Problems of Low-Rank Matrix Factorization
Authors: Fangyuan Jiang, Magnus Oskarsson, Kalle Astrom
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Low-rank matrix factorization is an essential problem in many areas including computer vision, with applications in e.g.  affine structure-from-motion, photometric stereo, and non-rigid structure from motion. However, very little attention  has been drawn to minimal cases for this problem or to using the minimal configuration of observations to find the solution. Minimal problems are useful when either outliers are present or  the observation matrix is sparse. In this paper, we first give some theoretical insights on how to generate all the minimal problems of a given size using Laman graph theory. We then propose a new parametrization and a building-block scheme to solve these minimal problems by extending the solution from a small sized minimal problem. We test our solvers on synthetic data as well as real data with outliers or a large portion of missing data and show that our method can handle the cases when other iterative methods, based on convex relaxation, fail.
Symmetry-Based Text Line Detection in Natural Scenes
Authors: Zheng Zhang, Wei Shen, Cong Yao, Xiang Bai
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Recently, a variety of real-world applications have triggered huge demand for techniques that can extract textual information from natural scenes. Therefore, scene text detection and recognition have become active research topics in computer vision. In this work, we investigate the problem of scene text detection from an alternative perspective and propose a novel algorithm for it. Different from traditional methods, which mainly make use of the properties of single characters or strokes, the proposed algorithm exploits the symmetry property of character groups and allows for direct extraction of text lines from natural images. The experiments on the latest ICDAR benchmarks demonstrate that the proposed algorithm achieves state-of-the-art performance. Moreover, compared to conventional approaches, the proposed algorithm shows stronger adaptability to texts in challenging scenarios.
DevNet: A Deep Event Network for Multimedia Event Detection and Evidence Recounting
Authors: Chuang Gan, Naiyan Wang, Yi Yang, Dit-Yan Yeung, Alex G. Hauptmann
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper, we focus on complex event detection in internet videos while also providing the key evidences of the detection results. Convolutional Neural Networks (CNNs) have achieved promising performance in image classification and action recognition tasks. However, it remains an open problem how to use CNNs for video event detection and recounting, mainly due to the complexity and diversity of video events. In this work, we propose a flexible deep CNN infrastructure, namely Deep Event Network (DevNet), that simultaneously detects pre-defined events and provides key spatial temporal evidences. Taking key frames of videos as input, we first detect the event of interest at the video level by aggregating the CNN features of the key frames. The pieces of evidences which recount the detection results, are also automatically localized, both temporally and spatially. The challenge is that we only have video level labels, while the key evidences usually take place at the frame levels. Based on the intrinsic property of CNNs, we first generate a spatial-temporal saliency map by back passing through DevNet, which then can be used to find the key frames which are most indicative to the event, as well as to localize the specific spatial position, usually an object, in the frame of the highly indicative area. Experiments on the large scale TRECVID 2014 MEDTest dataset demonstrate the promising performance of our method, both for event detection and evidence recounting.
Learning to Detect Motion Boundaries
Authors: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We propose a learning-based approach for motion boundary detection. Precise localization of motion boundaries is essential for the success of optical flow estimation, as motion boundaries correspond to discontinuities of the optical flow field. The proposed approach allows to predict motion boundaries, using a structured random forest trained on the ground-truth of the MPI-Sintel dataset. The random forest leverages several cues at the patch level, namely appearance (RGB color) and motion cues (optical flow estimated by state-of-the-art algorithms). Experimental results show that the proposed approach is both robust and computationally efficient. It significantly outperforms state-of-the-art motion-difference approaches on the MPI-Sintel and Middlebury datasets. We compare the results obtained with several state-of-the-art optical flow approaches and study the impact of the different cues used in the random forest.Furthermore, we introduce a new dataset, the YouTube Motion Boundaries dataset (YMB), that comprises 60 sequences taken from real-world videos with manually annotated motion boundaries. On this dataset, our approach, although trained on MPI-Sintel, also outperforms by a large margin state-of-the-art optical flow algorithms.
Improving Object Proposals With Multi-Thresholding Straddling Expansion
Authors: Xiaozhi Chen, Huimin Ma, Xiang Wang, Zhichen Zhao
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Recent advances in object detection have exploited object proposals to speed up object searching. However, many of existing object proposal generators have strong localization bias or require computationally expensive diversification strategies. In this paper, we present an effective approach to address these issues. We first propose a simple and useful localization bias measure, called superpixel tightness. Based on the characteristics of superpixel tightness distribution, we propose an effective method, namely multi-thresholding straddling expansion (MTSE) to reduce localization bias via fast diversification. Our method is essentially a box refinement process, which is intuitive and beneficial, but seldom exploited before. The greatest benefit of our method is that it can be integrated into any existing model to achieve consistently high recall across various intersection over union thresholds. Experiments on PASCAL VOC dataset demonstrates that our approach improves numerous existing models significantly with little computational overhead.
Visual Recognition by Counting Instances: A Multi-Instance Cardinality Potential Kernel
Authors: Hossein Hajimirsadeghi, Wang Yan, Arash Vahdat, Greg Mori
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Many visual recognition problems can be approached by counting instances. To determine whether an event is present in a long internet video, one could count how many frames seem to contain the activity.  Classifying the activity of a group of people can be done by counting the actions of individual people.  Encoding these cardinality relationships can reduce sensitivity to clutter, in the form of irrelevant frames or individuals not involved in a group activity. Learned parameters can encode how many instances tend to occur in a class of interest.  To this end, this paper develops a powerful and flexible framework to infer any cardinality relation between latent labels in a multi-instance model. Hard or soft cardinality relations can be encoded to tackle diverse levels of ambiguity. Experiments on tasks such as human activity recognition, video event detection, and video summarization demonstrate the effectiveness of using cardinality relations for improving recognition results.
Unconstrained 3D Face Reconstruction
Authors: Joseph Roth, Yiying Tong, Xiaoming Liu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper presents an algorithm for unconstrained 3D face reconstruction. The input to our algorithm is an "unconstrained" collection of face images captured under a diverse variation of poses, expressions, and illuminations, without meta data about cameras or timing. The output of our algorithm is a true 3D face surface model represented as a watertight triangulated surface with albedo data or texture information. 3D face reconstruction from a collection of unconstrained 2D images is a long-standing computer vision problem. Motivated by the success of the state-of-the-art method, we developed a novel photometric stereo-based method with two distinct novelties. First, working with a true 3D model allows us to enjoy the benefits of using images from all possible poses, including profiles.  Second, by leveraging emerging face alignment techniques and our novel normal field-based Laplace editing,  a combination of landmark constraints and photometric stereo-based normals drives our surface reconstruction. Given large photo collections and a ground truth 3D surface, we demonstrate the effectiveness and strength of our algorithm both qualitatively and quantitatively.
Becoming the Expert - Interactive Multi-Class Machine Teaching
Authors: Edward Johns, Oisin Mac Aodha, Gabriel J. Brostow
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Compared to machines, humans are extremely good at classifying images into categories, especially when they possess prior knowledge of the categories at hand. If this prior information is not available, supervision in the form of teaching images is required. To learn categories more quickly, people should see important and representative images first, followed by less important images later - or not at all. However, image-importance is individual-specific, i.e. a teaching image is important to a student if it changes their overall ability to discriminate between classes. Further, students keep learning, so while image-importance depends on their current knowledge, it also varies with time.  In this work we propose an Interactive Machine Teaching algorithm that enables a computer to teach challenging visual concepts to a human. Our adaptive algorithm chooses, online, which labeled images from a teaching set should be shown to the student as they learn. We show that a teaching strategy that probabilistically models the student's ability and progress, based on their correct and incorrect answers, produces better `experts'. We present results using real human participants across several varied and challenging real-world datasets.
Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
Authors: Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, Trevor Darrell
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Models comprised of deep convolutional network layers have dominated recent image interpretation tasks; we investigate whether models which are also compositional, or "deep", temporally are effective on tasks involving visual sequences or label sequences. We develop a novel recurrent convolutional architecture suitable for large-scale visual learning which is end-to-end trainable, and demonstrate the value of these models on benchmark video recognition tasks, image to sentence generation problems, and video narration challenges. In contrast to current models which assume a fixed spatio-temporal receptive field or simple temporal averaging for sequential processing, recurrent convolutional models are "doubly deep" in that they can be compositional in spatial and temporal "layers". Such models may have advantages when target concepts are complex and/or training data are limited. Learning long-term dependencies is possible when nonlinearities are incorporated into the network state updates. Long-term RNN models are appealing in that they directly can map variable length inputs (i.e. video frames) to variable length outputs (i.e. natural language text) and can model complex temporal dynamics; yet they can be optimized with backpropagation. Our recurrent long-term models are directly connected to state-of-the-art visual convnet models and can jointly trained, updating temporal dynamics and convolutional perceptual representations simultaneously. Our results show such models have distinct advantages over state-of-the-art models for recognition or generation which are separately defined and/or optimized.