mtybilly/Instruct2DS huggingface dataset:
Feature-Independent Context Estimation for Automatic Image Annotation
Authors: Amara Tariq, Hassan Foroosh
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Automatic image annotation is a highly valuable tool for image search, retrieval and archival systems. In the absence of an annotation tool, such systems have to rely on either users' input or large amount of text on the webpage of the image, to acquire its textual description. Users may provide insufficient/noisy tags and all the text on the webpage may not be a description or an explanation of the accompanying image. Therefore, it is of extreme importance to develop efficient tools for automatic annotation of images with correct and sufficient tags. The context of the image plays a significant role in this process, along with the content of the image. A suitable quantification of the context of the image may reduce the semantic gap between visual features and appropriate textual description of the image. In this paper, we present an unsupervised feature-independent quantification of the context of the image through tensor decomposition. We incorporate the estimated context as prior knowledge in the process of automatic image annotation. Evaluation of the predicted annotations provides evidence of the effectiveness of our feature-independent context estimation method.
Category-Specific Object Reconstruction From a Single Image
Authors: Abhishek Kar, Shubham Tulsiani, Joao Carreira, Jitendra Malik
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Object reconstruction from a single image -- in the wild -- is a problem where we can make progress and get meaningful results today. This is the main message of this paper, which introduces an automated pipeline with pixels as inputs and 3D surfaces of various rigid categories as outputs in images of realistic scenes. At the core of our approach are deformable 3D models that can be learned from 2D annotations available in existing object detection datasets, that can be driven by noisy automatic object segmentations and which we complement with a bottom-up module for recovering high-frequency shape details. We perform a comprehensive quantitative analysis and ablation study of our approach using the recently introduced PASCAL 3D+ dataset and show very encouraging automatic reconstructions on PASCAL VOC.
Active Sample Selection and Correction Propagation on a Gradually-Augmented Graph
Authors: Hang Su, Zhaozheng Yin, Takeo Kanade, Seungil Huh
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: When data have a complex manifold structure or the characteristics of data evolve over time, it is unrealistic to expect a graph-based semi-supervised learning method to achieve flawless classification given a small number of initial annotations. To address this issue with minimal human interventions, we propose (i) a sample selection criterion used for \textit{active} query of informative samples by minimizing the expected prediction error, and (ii) an efficient {\it correction propagation} method that propagates human correction on selected samples over a {\it gradually-augmented graph} to unlabeled samples without rebuilding the affinity graph. Experimental results conducted on three real world datasets validate that our active sample selection and correction propagation algorithm quickly reaches high quality classification results with minimal human interventions.
Efficient and Accurate Approximations of Nonlinear Convolutional Networks
Authors: Xiangyu Zhang, Jianhua Zou, Xiang Ming, Kaiming He, Jian Sun
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper aims to accelerate the test-time computation of deep convolutional neural networks (CNNs). Unlike existing methods that are designed for approximating linear filters or linear responses, our method takes the nonlinear units into account. We minimize the reconstruction error of the nonlinear responses, subject to a low-rank constraint which helps to reduce the complexity of filters. We develop an effective solution to this constrained nonlinear optimization problem. An algorithm is also presented for reducing the accumulated error when multiple layers are approximated. A whole-model speedup ratio of 4x is demonstrated   on a large network trained for ImageNet, while the top-5 error rate is only increased by 0.9%. Our accelerated model has a comparably fast speed as the "AlexNet", but is 4.7% more accurate.
Ranking and Retrieval of Image Sequences From Multiple Paragraph Queries
Authors: Gunhee Kim, Seungwhan Moon, Leonid Sigal
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We propose a method to rank and retrieve image sequences from a natural language text query, consisting of multiple sentences or paragraphs. One of the method's key applications is to visualize visitors' text-only reviews on TRIPADVISOR or YELP, by automatically retrieving the most illustrative image sequences. While most previous work has dealt with the relations between a natural language sentence and an image or a video, our work extends to the relations between paragraphs and image sequences. Our approach leverages the vast user-generated resource of blog posts and photo streams on the Web. We use blog posts as text-image parallel training data that co-locate informative text with representative images that are carefully selected by users. We exploit large-scale photo streams to augment the image samples for retrieval. We design a latent structural SVM framework to learn the semantic relevance relations between text and image sequences. We present both quantitative and qualitative results on the newly created DISNEYLAND dataset.
Casual Stereoscopic Panorama Stitching
Authors: Fan Zhang, Feng Liu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper presents a method for stitching stereoscopic panoramas from stereo images casually taken using a stereo camera. This method addresses three challenges of stereoscopic image stitching: how to handle parallax, how to stitch the left- and right-view panorama consistently, and how to take care of disparity during stitching. This method addresses these challenges using a three-step approach. First, we employ a state-of-the-art stitching algorithm that handles parallax well to stitch the left views of input stereo images and create the left view of the final stereoscopic panorama. Second, we stitch the input disparity maps to obtain the target disparity map for the stereoscopic panorama by solving a Poisson's equation. This target disparity map is optimized such that there are no vertical disparities and the original perceived depth distribution is preserved. Finally, we warp the right views of the input stereo images and stitch them into the right view of the final stereoscopic panorama according to the target disparity map. The stitching of the right views is formulated as a labeling problem that is constrained by the stitching of the left views to make the left- and right-view panorama consistent to avoid retinal rivalry. Our experiments show that our method can effectively stitch casually taken stereo images and produce high-quality stereoscopic panoramas that deliver a pleasant stereoscopic 3D viewing experience.
Superpixel Meshes for Fast Edge-Preserving Surface Reconstruction
Authors: Andras Bodis-Szomoru, Hayko Riemenschneider, Luc Van Gool
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Multi-View-Stereo (MVS) methods aim for the highest detail possible, however, such detail is often not required. In this work, we propose a novel surface reconstruction method based on image edges, superpixels and second-order smoothness constraints, producing meshes comparable to classic MVS surfaces in quality but orders of magnitudes faster. Our method performs per-view dense depth optimization directly over sparse 3D Ground Control Points (GCPs), hence, removing the need for view pairing, image rectification, and stereo depth estimation, and allowing for full per-image parallelization. We use Structure-from-Motion (SfM) points as GCPs, but the method is not specific to these, e.g.~LiDAR or RGB-D can also be used. The resulting meshes are compact and inherently edge-aligned with image gradients, enabling good-quality lightweight per-face flat renderings. Our experiments demonstrate on a variety of 3D datasets the superiority in speed and competitive surface quality.
Best-Buddies Similarity for Robust Template Matching
Authors: Tali Dekel, Shaul Oron, Michael Rubinstein, Shai Avidan, William T. Freeman
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We propose a novel method for template matching in unconstrained environments. Its essence is the Best Buddies Similarity (BBS), a useful, robust, and parameter-free similarity measure between two sets of points. BBS is based on a count of Best Buddies Pairs (BBPs)--pairs of points in which each one is the nearest neighbor of the other. BBS has several key features that make it robust against complex geometric deformations and high levels of outliers, such as those arising from background clutter and occlusions. We study these properties, provide a statistical analysis that justifies them, and demonstrate the consistent success of BBS on a challenging real-world dataset.
Superdifferential Cuts for Binary Energies
Authors: Tatsunori Taniai, Yasuyuki Matsushita, Takeshi Naemura
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We propose an efficient and general purpose energy optimization method for binary variable energies used in various low-level vision tasks. The proposed method can be used for broad classes of higher-order and pairwise non-submodular functions. We first revisit a submodular-supermodular procedure (SSP) [Narasimhan05], which is previously studied for higher-order energy optimization. We then present our method as generalization of SSP, which is further shown to generalize several state-of-the-art techniques for higher-order and pairwise non-submodular functions [Ayed13, Gorelick14, Tang14]. In the experiments, we apply our method to image segmentation, deconvolution, and binarization, and show improvements over state-of-the-art methods.
The S-Hock Dataset: Analyzing Crowds at the Stadium
Authors: Davide Conigliaro, Paolo Rota, Francesco Setti, Chiara Bassetti, Nicola Conci, Nicu Sebe, Marco Cristani
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The topic of crowd modeling in computer vision usually assumes a single generic typology of crowd, which is very simplistic. In this paper we adopt a taxonomy that is widely accepted in sociology, focusing on a particular category, the spectator crowd, which is formed by people "interested in watching something specific that they came to see". This can be found at the stadiums, amphitheaters, cinema, etc. In particular, we propose a novel dataset, the Spectators Hockey (S-Hock), which deals with 4 hockey matches during an international tournament. In the dataset, a massive annotation has been carried out, focusing on the spectators at different levels of details: at a higher level, people have been labeled depending on the team they are supporting and the fact that they know the people close to them; going to the lower levels, standard pose information has been considered (regarding the head, the body) but also fine grained actions such as hands on hips, clapping hands etc. The labeling focused on the game field also, permitting to relate what is going on in the match with the crowd behavior. This brought to more than 100 millions of annotations, useful for standard applications as people counting and head pose estimation but also for novel tasks as spectator categorization. For all of these we provide protocols and baseline results, encouraging further research.
Discriminant Analysis on Riemannian Manifold of Gaussian Distributions for Face Recognition With Image Sets
Authors: Wen Wang, Ruiping Wang, Zhiwu Huang, Shiguang Shan, Xilin Chen
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper presents a method named Discriminant Analysis on Riemannian manifold of Gaussian distributions (DARG) to solve the problem of face recognition with image sets. Our goal is to capture the underlying data distribution in each set and thus facilitate more robust classification. To this end, we represent image set as Gaussian Mixture Model (GMM) comprising a number of Gaussian components with prior probabilities and seek to discriminate Gaussian components from different classes. In the light of information geometry, the Gaussians lie on a specific Riemannian manifold. To encode such Riemannian geometry properly, we investigate several distances between Gaussians and further derive a series of provably positive definite probabilistic kernels. Through these kernels, a weighted Kernel Discriminant Analysis is finally devised which treats the Gaussians in GMMs as samples and their prior probabilities as sample weights. The proposed method is evaluated by face identification and verification tasks on four most challenging and largest databases, YouTube Celebrities, COX, YouTube Face DB and Point-and-Shoot Challenge, to demonstrate its superiority over the state-of-the-art.
Texture Representations for Image and Video Synthesis
Authors: Georgios Georgiadis, Alessandro Chiuso, Stefano Soatto
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In texture synthesis and classification, algorithms require a small texture to be provided as an input, which is assumed to be representative of a larger region to be re-synthesized or categorized. We focus on how to characterize such textures and automatically retrieve them. Most works generate these small input textures manually by cropping, which does not ensure maximal compression, nor that the selection is the best representative of the original. We construct a new representation that compactly summarizes a texture, while using less storage, that can be used for texture compression and synthesis. We also demonstrate how the representation can be integrated in our proposed video texture synthesis algorithm to generate novel instances of textures and video hole-filling. Finally, we propose a novel criterion that measures structural and statistical dissimilarity between textures.
Shadow Optimization From Structured Deep Edge Detection
Authors: Li Shen, Teck Wee Chua, Karianto Leman
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We present a novel learning-based framework for shadow detection from a single image. The local structure of shadow boundaries as well as the global interactions of the shadow and non-shadow regions remain largely unexploited by previous learning-based approaches. In this paper, we propose an efficient structured labelling framework for shadow detection from a single image. A convolutional Neural Networks framework is designed to capture the local structure information of shadow edge and to learn the most relevant features. We further propose and formulate a global shadow optimization framework which can model the complex global interactions over the shadow and light regions. Using the shadow edges detected by our proposed method, the shadow map can be solved by efficient least-square optimization. Our proposed framework is efficient and achieves state-of-the-art results on the major shadow benchmark databases collected under a variety of conditions.
Total Variation Regularization of Shape Signals
Authors: Maximilian Baust, Laurent Demaret, Martin Storath, Nassir Navab, Andreas Weinmann
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper introduces the concept of shape signals, i.e., series of shapes which have a natural temporal or spatial ordering, as well as a variational formulation for the regularization of these signals. The proposed formulation can be seen as the shape-valued generalization of the Rudin-Osher-Fatemi (ROF) functional for intensity images. We derive a variant of the classical finite-dimensional representation of Kendall, but our framework is generic in the sense that it can be combined with any shape space. This representation allows for the explicit computation of geodesics and thus facilitates the efficient numerical treatment of the variational formulation by means of the cyclic proximal point algorithm. Similar to the ROF-functional, we demonstrate experimentally that l_1-type penalties both for data fidelity term and regularizer perform best in regularizing shape signals. Finally, we show applications of our method to shape signals obtained from synthetic, photometric, and medical data sets.
Learning Similarity Metrics for Dynamic Scene Segmentation
Authors: Damien Teney, Matthew Brown, Dmitry Kit, Peter Hall
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper addresses the segmentation of videos with arbitrary motion, including dynamic textures, using novel motion features and a supervised learning approach. Dynamic textures are commonplace in natural scenes, and exhibit complex patterns of appearance and motion (e.g. water, smoke, swaying foliage). These are difficult for existing segmentation algorithms, often violate the brightness constancy assumption needed for optical flow, and have complex segment characteristics beyond uniform appearance or motion. Our solution uses custom spatiotemporal filters that capture texture and motion cues, along with a novel metric-learning framework that optimizes this representation for specific objects and scenes. This is used within a hierarchical, graph-based segmentation setting, yielding state-of-the-art results for dynamic texture segmentation. We also demonstrate the applicability of our approach to general object and motion segmentation, showing significant improvements over unsupervised segmentation and results comparable to the best task specific approaches.
Subspace Clustering by Mixture of Gaussian Regression
Authors: Baohua Li, Ying Zhang, Zhouchen Lin, Huchuan Lu
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Subspace clustering is a problem of finding a multisubspace representation that best fits sample points drawn from a high-dimensional space. The existing clustering models generally adopt different norms to describe noise, which is equivalent to assuming that the data are corrupted by specific types of noise. In practice, however, noise is much more complex. So it is inappropriate to simply use a certain norm to model noise. Therefore, we propose Mixture of Gaussian Regression (MoG Regression) for subspace clustering by modeling noise as a Mixture of Gaussians (MoG). The MoG Regression provides an effective way to model a much broader range of noise distributions. As a result, the obtained affinity matrix is better at characterizing the structure of data in real applications. Experimental results on multiple datasets demonstrate that MoG Regression significantly outperforms state-of-the-art subspace clustering methods.
DASC: Dense Adaptive Self-Correlation Descriptor for Multi-Modal and Multi-Spectral Correspondence
Authors: Seungryong Kim, Dongbo Min, Bumsub Ham, Seungchul Ryu, Minh N. Do, Kwanghoon Sohn
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Establishing dense visual correspondence between multiple images is a fundamental task in many applications of computer vision and computational photography. Classical approaches, which aim to estimate dense stereo and optical flow fields for images adjacent in viewpoint or in time, have been dramatically advanced in recent studies. However, finding reliable visual correspondence in multi-modal or multi-spectral images still remains unsolved. In this paper, we propose a new dense matching descriptor, called dense adaptive self-correlation (DASC), to effectively address this kind of matching scenarios. Based on the observation that a self-similarity existing within images is less sensitive to modality variations, we define a novel descriptor with a series of an adaptive self-correlation similarity for patches within a local support window. To further improve the matching quality and runtime efficiency, we propose a patch-wise receptive field pooling, in which a sampling pattern is optimized with a discriminative learning. Moreover, the computational redundancy that arises when computing densely sampled descriptor over an entire image is dramatically reduced by applying fast edge-aware filtering. Experiments demonstrate the outstanding performance of the DASC descriptor in many cases of multi-modal and multi-spectral correspondence.
In Defense of Color-Based Model-Free Tracking
Authors: Horst Possegger, Thomas Mauthner, Horst Bischof
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper, we address the problem of model-free online object tracking based on color representations. According to the findings of recent benchmark evaluations, such trackers often tend to drift towards regions which exhibit a similar appearance compared to the object of interest. To overcome this limitation, we propose an efficient discriminative object model which allows us to identify potentially distracting regions in advance. Furthermore, we exploit this knowledge to adapt the object representation beforehand so that distractors are suppressed and the risk of drifting is significantly reduced. We evaluate our approach on recent online tracking benchmark datasets demonstrating state-of-the-art results. In particular, our approach performs favorably both in terms of accuracy and robustness compared to recent tracking algorithms. Moreover, the proposed approach allows for an efficient implementation to enable online object tracking in real-time.
Best of Both Worlds: Human-Machine Collaboration for Object Annotation
Authors: Olga Russakovsky, Li-Jia Li, Li Fei-Fei
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The long-standing goal of localizing every object in an image remains elusive. Manually annotating objects is quite expensive despite crowd engineering innovations. Current state-of-the-art automatic object detectors can accurately detect at most a few objects per image. This paper brings together the latest advancements in object detection and in crowd engineering into a principled framework for accurately and efficiently localizing objects in images. The input to the system is an image to annotate and a set of annotation constraints: desired precision, utility and/or human cost of the labeling. The output is a set of object annotations, informed by human feedback and computer vision. Our model seamlessly integrates multiple computer vision models with multiple sources of human input in a Markov Decision Process. We empirically validate the effectiveness of our human-in-the-loop labeling approach on the ILSVRC2014 object detection dataset.
Robust Multiple Homography Estimation: An Ill-Solved Problem
Authors: Zygmunt L. Szpak, Wojciech Chojnacki, Anton van den Hengel
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The estimation of multiple homographies between two piecewise planar views of a rigid scene is often assumed to be a solved problem. We show that contrary to popular opinion various crucial aspects of the task have not been adequately emphasised. We are motivated by a growing body of literature in robust multi-structure estimation that purports to solve the multi-homography estimation problem but in fact does not. We demonstrate that the estimation of multiple homographies is an ill-solved problem by deriving new constraints that a set of mutually compatible homographies must satisfy, and by showing that homographies estimated with prevailing methods fail to satisfy the requisite constraints on real-world data. We also explain why incompatible homographies imply inconsistent epipolar geometries. The arguments and experiments presented in this paper signal the need for a new generation of robust multi-structure estimation methods that have the capacity to enforce constraints on projective entities such as homography matrices.
Semi-Supervised Domain Adaptation With Subspace Learning for Visual Recognition
Authors: Ting Yao, Yingwei Pan, Chong-Wah Ngo, Houqiang Li, Tao Mei
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In many real-world applications, we are often facing the problem of cross domain learning, i.e., to borrow the labeled data or transfer the already learnt knowledge from a source domain to a target domain. However, simply applying existing source data or knowledge may even hurt the performance, especially when the data distribution in the source and target domain is quite different, or there are very few labeled data available in the target domain. This paper proposes a novel domain adaptation framework, named Semi-supervised Domain Adaptation with Subspace Learning (SDASL), which jointly explores invariant low-dimensional structures across domains to correct data distribution mismatch and leverages available unlabeled target examples to exploit the underlying intrinsic information in the target domain. Specifically, SDASL conducts the learning by simultaneously minimizing the classification error, preserving the structure within and across domains, and restricting similarity defined on unlabeled target examples. Encouraging results are reported for two challenging domain transfer tasks (including image-to-image and image-to-video transfers) on several standard datasets in the context of both image object recognition and video concept detection.
Articulated Motion Discovery Using Pairs of Trajectories
Authors: Luca Del Pero, Susanna Ricco, Rahul Sukthankar, Vittorio Ferrari
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We propose an unsupervised approach for discovering characteristic motion patterns in videos of highly articulated objects performing natural, unscripted behaviors, such as tigers in the wild. We discover consistent patterns in a bottom-up manner by analyzing the relative displacements of large numbers of ordered trajectory pairs through time, such that each trajectory is attached to a different moving part on the object. The pairs of trajectories descriptor relies entirely on motion and is more discriminative than state-of-the-art features that employ single trajectories. Our method generates temporal video intervals, each automatically trimmed to one instance of the discovered behavior, and clusters them by type (e.g., running, turning head, drinking water). We present experiments on two datasets: dogs from YouTube-Objects and a new dataset of National Geographic tiger videos. Results confirm that our proposed descriptor outperforms existing appearance- and trajectory-based descriptors (e.g., HOG and DTFs) on both datasets and enables us to segment unconstrained animal video into intervals containing single behaviors.
A Solution for Multi-Alignment by Transformation Synchronisation
Authors: Florian Bernard, Johan Thunberg, Peter Gemmar, Frank Hertel, Andreas Husch, Jorge Goncalves
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The alignment of a set of objects by means of transformations plays an important role in computer vision. Whilst the case for only two objects can be solved globally, when multiple objects are considered usually iterative methods are used. In practice the iterative methods perform well if the relative transformations between any pair of objects are free of noise. However, if only noisy relative transformations are available (e.g. due to missing data or wrong correspondences) the iterative methods may fail.  Based on the observation that the underlying noise-free transformations can be retrieved from the null space of a matrix that can directly be obtained from pairwise alignments, this paper presents a novel method for the synchronisation of pairwise transformations such that they are transitively consistent.   Simulations demonstrate that for noisy transformations, a large proportion of missing data and even for wrong correspondence assignments the method delivers encouraging results.
A Convex Optimization Approach to Robust Fundamental Matrix Estimation
Authors: Yongfang Cheng, Jose A. Lopez, Octavia Camps, Mario Sznaier
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: This paper considers the problem of recovering a subspace arrangement from noisy samples, potentially corrupted with outliers. Our main result shows that this problem  can be formulated as a  constrained polynomial optimization, for which a  monotonically convergent sequence of tractable convex relaxations can be obtained by exploiting  recent developments in sparse polynomial optimization. Further, these results allow for deriving conditions certifying that a finite order relaxation has converged to a solution.  A salient feature of the proposed approach is its ability to incorporate existing a-priori information about the noise,  co-ocurrences, and percentage of outliers. These results are illustrated  with several examples where the proposed algorithm is shown to outperform existing approaches.
Simultaneous Pose and Non-Rigid Shape With Particle Dynamics
Authors: Antonio Agudo, Francesc Moreno-Noguer
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In this paper, we propose a sequential solution to simultaneously estimate camera pose and non-rigid 3D shape from a monocular video. In contrast to most existing approaches that rely on global representations of the shape, we model the object at a local level, as an ensemble of particles, each ruled by the linear equation of the Newton's second law of motion. This dynamic model is incorporated into a bundle adjustment framework, in combination with simple regularization components that ensure temporal and spatial consistency of the estimated shape and camera poses. The resulting approach is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions, while it does not require any training data at all. Validation is done in a variety of real video sequences, including articulated and non-rigid motion, both for continuous and discontinuous shapes. Our system is shown to perform comparable to competing batch, computationally expensive, methods and shows remarkable improvement with respect to the sequential ones.
Semi-Supervised Learning With Explicit Relationship Regularization
Authors: Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: In many learning tasks, the structure of the target space of a function holds rich information about the relationships between evaluations of functions on different data points. Existing approaches attempt to exploit this relationship information implicitly by enforcing smoothness on function evaluations only. However, what happens if we explicitly regularize the relationships between function evaluations? Inspired by homophily, we regularize based on a smooth relationship function, either defined from the data or with labels. In experiments, we demonstrate that this significantly improves the performance of state-of-the-art algorithms in semi-supervised classification and in spectral data embedding for constrained clustering and dimensionality reduction.
Person Re-Identification by Local Maximal Occurrence Representation and Metric Learning
Authors: Shengcai Liao, Yang Hu, Xiangyu Zhu, Stan Z. Li
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Person re-identification is an important technique towards automatic search of a person's presence in a surveillance video. Two fundamental problems are critical for person re-identification, feature representation and metric learning. An effective feature representation should be robust to illumination and viewpoint changes, and a discriminant metric should be learned to match various person images. In this paper, we propose an effective feature representation called Local Maximal Occurrence (LOMO), and a subspace and metric learning method called Cross-view Quadratic Discriminant Analysis (XQDA). The LOMO feature analyzes the horizontal occurrence of local features, and maximizes the occurrence to make a stable representation against viewpoint changes. Besides, to handle illumination variations, we apply the Retinex transform and a scale invariant texture operator. To learn a discriminant metric, we propose to learn a discriminant low dimensional subspace by cross-view quadratic discriminant analysis, and simultaneously, a QDA metric is learned on the derived subspace. We also present a practical computation method for XQDA, as well as its regularization. Experiments on four challenging person re-identification databases, VIPeR, QMUL GRID, CUHK Campus, and CUHK03, show that the proposed method improves the state-of-the-art rank-1 identification rates by 2.2%, 4.88%, 28.91%, and 31.55% on the four databases, respectively.
Joint Patch and Multi-Label Learning for Facial Action Unit Detection
Authors: Kaili Zhao, Wen-Sheng Chu, Fernando De la Torre, Jeffrey F. Cohn, Honggang Zhang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: The face is one of the most powerful channel of non-verbal communication. The most commonly used taxonomy to describe facial behaviour is the Facial Action Coding System (FACS).  FACS segments the visible effects of facial muscle activation into 30+ action units (AUs).  AUs, which may occur alone and in thousands of combinations, can describe nearly all-possible facial expressions.  Most existing methods for automatic AU detection treat the problem using one-vs-all classifiers and fail to exploit dependencies among AU and facial features.  We introduce joint-patch and multi-label learning (JPML) to address these issues. JPML leverages group sparsity by selecting a sparse subset of facial patches while learning a multi-label classifier. In four of five comparisons on three diverse datasets, CK+, GFT, and BP4D, JPML produced the highest average F1 scores in comparison with state-of-the art.
Real-Time Visual Analysis of Microvascular Blood Flow for Critical Care
Authors: Chao Liu, Hernando Gomez, Srinivasa Narasimhan, Artur Dubrawski, Michael R. Pinsky, Brian Zuckerbraun
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Microcirculatory monitoring plays an important role in diagnosis and treatment of critical care patients. Sidestream Dark Field (SDF) imaging devices have been used to visualize and support interpretation of the micro-vascular blood flow.  However, due to subsurface scattering within the tissue that embeds the capillaries, transparency of plasma, imaging noise and lack of features, it is difficult to obtain reliable physiological data from SDF videos. Therefore, thus far microcirculatory videos have been analyzed manually with significant input from expert clinicians. In this paper, we present a framework that automates the analysis process. It includes stages of video stabilization, enhancement, and micro-vessel extraction, in order to automatically estimate statistics of the micro blood flows from SDF videos. Our method has been validated in critical care experiments conducted carefully to record the microcirculatory blood flow in test animal subjects before, during and after induced bleeding episodes, as well as to study the effect of fluid resuscitation. Our method is able to extract microcirculatory measurements that are consistent with clinical intuition and it has a potential to become a useful tool in critical care medicine.
JOTS: Joint Online Tracking and Segmentation
Authors: Longyin Wen, Dawei Du, Zhen Lei, Stan Z. Li, Ming-Hsuan Yang
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We present a novel Joint Online Tracking and Segmentation (JOTS) algorithm which integrates the multi-part tracking and segmentation into a unified energy optimization framework to handle the video segmentation task. The multi-part segmentation is posed as a pixel-level label assignment task with regularization according to the estimated part models, and tracking is formulated as estimating the part models based on the pixel labels, which in turn is used to refine the model. The multi-part tracking and segmentation are carried out iteratively to minimize the proposed objective function by a RANSAC-style approach. Extensive experiments on the SegTrack and SegTrack v2 databases demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.
Gaze-Enabled Egocentric Video Summarization via Constrained Submodular Maximization
Authors: Jia Xu, Lopamudra Mukherjee, Yin Li, Jamieson Warner, James M. Rehg, Vikas Singh
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: With the proliferation of wearable cameras, the number of videos of users documenting their personal lives using   such devices is rapidly increasing. Since such videos may span hours,   there is an important need for mechanisms that represent the information content in a compact form (i.e., shorter videos which are more easily browsable/sharable). Motivated by these applications, this paper focuses on the problem of egocentric video summarization. Such videos are usually continuous with significant camera shake and other quality issues. Because of these reasons, there is growing consensus that direct application of standard video summarization tools to such data yields unsatisfactory performance.   In this paper, we demonstrate that using gaze tracking information (such as fixation and saccade) significantly helps the summarization task.  It allows meaningful comparison of different image frames  and enables deriving personalized   summaries (gaze provides a sense of the camera wearer's intent).   We formulate a summarization model which captures common-sense properties of a good summary, and show that   it can be solved as a submodular function maximization with partition matroid constraints, opening the door to a   rich body of work from combinatorial optimization.   We evaluate our approach on a new gaze-enabled egocentric video dataset (over 15 hours),     which will be a valuable standalone resource.
Sparse Depth Super Resolution
Authors: Jiajun Lu, David Forsyth
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We describe a method to produce detailed high resolution depth maps from aggressively subsampled depth measurements. Our method fully uses the relationship between image segmentation boundaries and depth boundaries. It uses an image combined with a low resolution depth map. 1) The image is segmented with the guidance of sparse depth samples. 2) Each segment has its depth field reconstructed independently using a novel smoothing method. 3) For videos, time-stamped samples from near frames are incorporated. The paper shows reconstruction results of super resolution from x4 to x100, while previous methods mainly work on x2 to x16. The method is tested on four different datasets and six video sequences, covering quite different regimes, and it outperforms recent state of the art methods quantitatively and qualitatively. We also demonstrate that depth maps produced by our method can be used by applications such as hand trackers, while depth maps from other methods have problems.
Efficient Illuminant Estimation for Color Constancy Using Grey Pixels
Authors: Kai-Fu Yang, Shao-Bing Gao, Yong-Jie Li
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Illuminant estimation is a key step for computational color constancy. Instead of using the grey world or grey edge assumptions, we propose in this paper a novel method for illuminant estimation by using the information of grey pixels detected in a given color-biased image. The underlying hypothesis is that most of the natural images include some detectable pixels that are at least approximately grey, which can be reliably utilized for illuminant estimation. We first validate our assumption through comprehensive statistical evaluation on diverse collection of datasets and then put forward a novel grey pixel detection method based on the illuminant-invariant measure (IIM) in three logarithmic color channels. Then the light source color of a scene can be easily estimated from the detected grey pixels. Experimental results on four benchmark datasets (three recorded under single illuminant and one under multiple illuminants) show that the proposed method outperforms most of the state-of-the-art color constancy approaches with the inherent merit of low computational cost.
Can Humans Fly? Action Understanding With Multiple Classes of Actors
Authors: Chenliang Xu, Shao-Hang Hsieh, Caiming Xiong, Jason J. Corso
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Can humans fly? Emphatically no. Can cars eat? Again, absolutely not. Yet, these absurd inferences result from the current disregard for particular types of actors in action understanding. There is no work we know of on simultaneously inferring actors and actions in the video, not to mention a dataset to experiment with. Our paper hence marks the first effort in the computer vision community to jointly consider various types of actors undergoing various actions. To start with the problem, we collect a dataset of 3782 videos from YouTube and label both pixel-level actors and actions in each video. We formulate the general actor-action understanding problem and instantiate it at various granularities: both video-level single- and multiple-label actor-action recognition and pixel-level actor-action semantic segmentation. Our experiments demonstrate that inference jointly over actors and actions outperforms inference independently over them, and hence concludes our argument of the value of explicit consideration of various actors in comprehensive action understanding.
Reweighted Laplace Prior Based Hyperspectral Compressive Sensing for Unknown Sparsity
Authors: Lei Zhang, Wei Wei, Yanning Zhang, Chunna Tian, Fei Li
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Compressive sensing(CS) has been exploited for hypespectral image(HSI) compression in recent years. Though it can greatly reduce the costs of computation and storage, the reconstruction of HSI from a few linear measurements is challenging. The underlying sparsity of HSI is crucial to improve the reconstruction accuracy. However, the sparsity of HSI is unknown in reality and varied with different noise, which makes the sparsity estimation difficult. To address this problem, a novel reweighted Laplace prior based hyperspectral compressive sensing method is proposed in this study. First, the reweighted Laplace prior is proposed to model the distribution of sparsity in HSI. Second, the latent variable Bayes model is employed to learn the optimal configuration of the reweighted Laplace prior from the measurements. The model unifies signal recovery, prior learning and noise estimation into a variational framework to infer the parameters automatically. The learned sparsity prior can represent the underlying structure of the sparse signal very well and is adaptive to the unknown noise, which improves the reconstruction accuracy of HSI. The experimental results on three hyperspectral datasets demonstrate the proposed method outperforms serveral state-of-the-art hyperspectral CS methods on the reconstruction accuracy.
Class Consistent Multi-Modal Fusion With Binary Features
Authors: Ashish Shrivastava, Mohammad Rastegari, Sumit Shekhar, Rama Chellappa, Larry S. Davis
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: Many existing recognition algorithms combine different modalities based on training accuracy but do not consider the possibility of noise at test time. We describe an algorithm that perturbs test features so that all modalities predict the same class. We enforce this perturbation to be as small as possible via a quadratic program (QP) for continuous features, and a mixed integer program (MIP) for binary features. To efficiently solve the MIP, we provide a greedy algorithm and empirically show that its solution is very close to that of a state-of-the-art MIP solver. We evaluate our algorithm on several datasets and show that the method outperforms existing approaches.
R6P - Rolling Shutter Absolute Camera Pose
Authors: Cenek Albl, Zuzana Kukelova, Tomas Pajdla
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015 (Main Conference Track), 2015

Abstract: We present a minimal, non-iterative solution to the absolute pose problem for images from rolling shutter cameras. Absolute pose problem is a key problem in computer vision and rolling shutter is present in a vast majority of today's digital cameras. We propose several rolling shutter camera models and verify their feasibility for a polynomial solver. A solution based on linearized camera model is chosen and verified in several experiments. We use a linear approximation to the camera orientation, which is meaningful only around the identity rotation. We show that the standard P3P algorithm is able to estimate camera orientation within 6 degrees for camera rotation velocity as high as 30deg/frame. Therefore we can use the standard P3P algorithm to estimate camera orientation and to bring the camera rotation matrix close to the identity. Using this solution, camera position, orientation, translational velocity and angular velocity can be computed using six 2D-to-3D correspondences, with orientation error under half a degree and relative position error under 2%. A significant improvement in terms of the number of inliers in RANSAC is demonstrated.