pwc-archive/papers-with-abstracts huggingface dataset:
Outlier-Robust Tensor PCA
Authors: Pan Zhou, Jiashi Feng
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Clustering, Outlier Detection

Low-rank tensor analysis is important for various real applications in computer vision. However, existing methods focus on recovering a low-rank tensor contaminated by Gaussian or gross sparse noise and hence cannot effectively handle outliers that are common in practical tensor data. To solve this issue, we propose an outlier-robust tensor principle component analysis (OR-TPCA) method for simultaneous low-rank tensor recovery and outlier detection. For intrinsically low-rank tensor observations with arbitrary outlier corruption, OR-TPCA is the first method that has provable performance guarantee for exactly recovering the tensor subspace and detecting outliers  under mild conditions.  Since tensor data are naturally high-dimensional and multi-way, we further develop a fast randomized algorithm that requires small sampling size yet can substantially accelerate OR-TPCA without performance drop. Experimental results on four tasks: outlier detection, clustering, semi-supervised and supervised learning, clearly demonstrate the advantages of our method.
Unrolling the Shutter: CNN to Correct Motion Distortions
Authors: Vijay Rengarajan, Yogesh Balaji, A. N. Rajagopalan
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Rolling Shutter Correction

Row-wise exposure delay present in CMOS cameras is responsible for skew and curvature distortions known as the rolling shutter (RS) effect while imaging under camera motion. Existing RS correction methods resort to using multiple images or tailor scene-specific correction schemes. We propose a convolutional neural network (CNN) architecture that automatically learns essential scene features from a single RS image to estimate the row-wise camera motion and undo RS distortions back to the time of first-row exposure. We employ long rectangular kernels to specifically learn the effects produced by the row-wise exposure. Experiments reveal that our proposed architecture performs better than the conventional CNN employing square kernels. Our single-image correction method fares well even operating in a frame-by-frame manner against video-based methods and performs better than scene-specific correction schemes even under challenging situations.
Deep Level Sets for Salient Object Detection
Authors: Ping Hu, Bing Shuai, Jun Liu, Gang Wang
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Saliency Detection, Salient Object Detection

   Deep learning has been applied to saliency detection in recent years. The superior performance has proved that deep networks can model the semantic properties of salient objects. Yet it is difficult for a deep network to discriminate pixels belonging to similar receptive fields around the object boundaries, thus deep networks may output maps with blurred saliency and inaccurate boundaries. To tackle such an issue, in this work, we propose a deep Level Set network to produce compact and uniform saliency maps. Our method drives the network to learn a Level Set function for salient objects so it can output more accurate boundaries and compact saliency. Besides, to propagate saliency information among pixels and recover full resolution saliency map, we extend a superpixel-based guided filter to be a layer in the network. The proposed network has a simple structure and is trained end-to-end. During testing, the network can produce saliency maps by efficiently feedforwarding testing images at a speed over 12FPS on GPUs. Evaluations on benchmark datasets show that the proposed method achieves state-of-the-art performance.
Kernel Square-Loss Exemplar Machines for Image Retrieval
Authors: Rafael S. Rezende, Joaquin Zepeda, Jean Ponce, Francis Bach, Patrick Perez
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Retrieval, Retrieval

Zepeda and Perez have recently demonstrated the promise of the exemplar SVM (ESVM) as a feature encoder for image retrieval. This paper extends this approach in several directions: We first show that replacing the hinge loss by the square loss in the ESVM cost function significantly reduces encoding time with negligible effect on accuracy. We call this model square-loss exemplar machine, or SLEM.  We then introduce a kernelized SLEM which can be implemented efficiently through low-rank matrix decomposition, and displays improved performance.  Both SLEM variants exploit the fact that the negative examples are fixed, so most of the SLEM computational complexity is relegated to an offline process independent of the positive examples.  Our experiments establish the performance and computational advantages of our approach using a large array of base features and standard image retrieval datasets.
Direct Photometric Alignment by Mesh Deformation
Authors: Kaimo Lin, Nianjuan Jiang, Shuaicheng Liu, Loong-Fah Cheong, Minh Do, Jiangbo Lu
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Stitching, Optical Flow Estimation, Video Stabilization

The choice of motion models is vital in applications like image/video stitching and video stabilization. Conventional methods explored different approaches ranging from simple global parametric models to complex per-pixel optical flow. Mesh-based warping methods achieve a good balance between computational complexity and model flexibility. However, they typically require high quality feature correspondences and suffer from mismatches and low-textured image content. In this paper, we propose a mesh-based photometric alignment method that minimizes pixel intensity difference instead of Euclidean distance of known feature correspondences. The proposed method combines the superior performance of dense photometric alignment with the efficiency of mesh-based image warping. It achieves better global alignment quality than the feature-based counterpart in textured images, and more importantly, it is also robust to low-textured image content. Abundant experiments show that our method can handle a variety of images and videos, and outperforms representative state-of-the-art methods in both image stitching and video stabilization tasks.
Semantic Multi-View Stereo: Jointly Estimating Objects and Voxels
Authors: Ali Osman Ulusoy, Michael J. Black, Andreas Geiger
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Reconstruction

Dense 3D reconstruction from RGB images is a highly ill-posed problem due to occlusions, textureless or reflective surfaces, as well as other challenges. We propose object-level shape priors to address these ambiguities. Towards this goal, we formulate a probabilistic model that integrates multi-view image evidence with 3D shape information from multiple objects. Inference in this model yields a dense 3D reconstruction of the scene as well as the existence and precise 3D pose of the objects in it. Our approach is able to recover fine details not captured in the input shapes while defaulting to the input models in occluded regions where image evidence is weak. Due to its probabilistic nature, the approach is able to cope with the approximate geometry of the 3D models as well as input shapes that are not present in the scene. We evaluate the approach quantitatively on several challenging indoor and outdoor datasets.
HOPE: Hierarchical Object Prototype Encoding for Efficient Object Instance Search in Videos
Authors: Tan Yu, Yuwei Wu, Junsong Yuan
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Instance Search, Object

This paper tackles the problem of efficient and effective object instance search in videos. To effectively capture the relevance between a query and video frames and precisely localize the particular object, we leverage the object proposals to improve the quality of object instance search in videos. However, hundreds of object proposals obtained from each frame could result in unaffordable memory and computational cost. To this end, we present a simple yet effective hierarchical object prototype encoding (HOPE) model to accelerate the object instance search without sacrificing accuracy, which exploits both the spatial and temporal self-similarity property existing in object proposals generated from video frames. We design two types of sphere k-means methods, i.e., spatially-constrained sphere k-means and temporally-constrained sphere k-means to learn frame-level object prototypes and dataset-level object prototypes, respectively.  In this way, the object instance search problem is cast to the sparse matrix-vector multiplication problem. Thanks to the sparsity of the codes, both the memory and computational cost are significantly reduced. Experimental results on two video datasets demonstrate that our approach significantly improves the performance of video object instance search over other state-of-the-art fast search schemes.
Learning Adaptive Receptive Fields for Deep Image Parsing Network
Authors: Zhen Wei, Yao Sun, Jinqiao Wang, Hanjiang Lai, Si Liu
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Face Parsing

In this paper, we introduce a novel approach to regulate receptive field in deep image parsing network automatically. Unlike previous works which have stressed much importance on obtaining better receptive fields using manually selected dilated convolutional kernels, our approach uses two affine transformation layers in the network's backbone and operates on feature maps. Feature maps will be inflated/shrinked by the new layer and therefore receptive fields in following layers are changed accordingly. By end-to-end training, the whole framework is data-driven without laborious manual intervention. The proposed method is generic across dataset and different tasks. We conduct extensive experiments on both general parsing task and face parsing task as concrete examples to demonstrate the method's superior regulation ability over manual designs.
Contour-Constrained Superpixels for Image and Video Processing
Authors: Se-Ho Lee, Won-Dong Jang, Chang-Su Kim
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object, Superpixels

A novel contour-constrained superpixel (CCS) algorithm is proposed in this work. We initialize superpixels and regions in a regular grid and then refine the superpixel label of each region hierarchically from block to pixel levels. To make superpixel boundaries compatible with object contours, we propose the notion of contour pattern matching and formulate an objective function including the contour constraint. Furthermore, we extend the CCS algorithm to generate temporal superpixels for video processing. We initialize superpixel labels in each frame by transferring those in the previous frame and refine the labels to make superpixels temporally consistent as well as compatible with object contours. Experimental results demonstrate that the proposed algorithm provides better performance than the state-of-the-art superpixel methods.
Learning to Predict Stereo Reliability Enforcing Local Consistency of Confidence Maps
Authors: Matteo Poggi, Stefano Mattoccia
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Stereo Matching, Stereo Matching Hand

Confidence measures estimate unreliable disparity assignments performed by a stereo matching algorithm and, as recently proved, can be used for several purposes. This paper aims at increasing, by means of a deep network, the effectiveness of state-of-the-art confidence measures exploiting the local consistency assumption. We exhaustively evaluated our proposal on 23 confidence measures, including 5 top-performing ones based on random-forests and CNNs, training our networks with two popular stereo algorithms and a small subset (25 out of 194 frames) of the KITTI 2012 dataset. Experimental results show that our approach dramatically increases the effectiveness of all the 23 confidence measures on the remaining frames. Moreover, without re-training, we report a further cross-evaluation on KITTI 2015 and Middlebury 2014 confirming that our proposal provides remarkable improvements for each confidence measure even when dealing with significantly different input data. To the best of our knowledge, this is the first method to move beyond conventional pixel-wise confidence estimation.
Additive Component Analysis
Authors: Calvin Murdock, Fernando de la Torre
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Additive models, Denoising, Dimensionality Reduction

Principal component analysis (PCA) is one of the most versatile tools for unsupervised learning with applications ranging from dimensionality reduction to exploratory data analysis and visualization. While much effort has been devoted to encouraging meaningful representations through regularization (e.g. non-negativity or sparsity), underlying linearity assumptions can limit their effectiveness. To address this issue, we propose Additive Component Analysis (ACA), a novel nonlinear extension of PCA. Inspired by multivariate nonparametric regression with additive models, ACA fits a smooth manifold to data by learning an explicit mapping from a low-dimensional latent space to the input space, which trivially enables applications like denoising. Furthermore, ACA can be used as a drop-in replacement in many algorithms that use linear component analysis methods as a subroutine via the local tangent space of the learned manifold. Unlike many other nonlinear dimensionality reduction techniques, ACA can be efficiently applied to large datasets since it does not require computing pairwise similarities or storing training data during testing. Multiple ACA layers can also be composed and learned jointly with essentially the same procedure for improved representational power, demonstrating the encouraging potential of nonparametric deep learning. We evaluate ACA on a variety of datasets, showing improved robustness, reconstruction performance, and interpretability.
Attentional Push: A Deep Convolutional Network for Augmenting Image Salience With Shared Attention Modeling in Social Scenes
Authors: Siavash Gorji, James J. Clark
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Transfer Learning

We present a novel visual attention tracking technique based on Shared Attention modeling. By considering the viewer as a participant in the activity occurring in the scene, our model learns the loci of attention of the scene actors and use it to augment image salience. We go beyond image salience and instead of only computing the power of image regions to pull attention, we also consider the strength with which the scene actors push attention to the region in question, thus the term Attentional Push. We present a convolutional neural network (CNN) which augments standard saliency models with Attentional Push. Our model contains two pathways: an Attentional Push pathway which learns the gaze location of the scene actors and a saliency pathway. These are followed by a shallow augmented saliency CNN which combines them and generates the augmented saliency. For training, we use transfer learning to initialize and train the Attentional Push CNN by minimizing the classification error of following the actors' gaze location on a 2-D grid using a large-scale gaze-following dataset. The Attentional Push CNN is then fine-tuned along with the augmented saliency CNN to minimize the Euclidean distance between the augmented saliency and ground truth fixations using an eye-tracking dataset, annotated with the head and the gaze location of the scene actors. We evaluate our model on three challenging eye fixation datasets, SALICON, iSUN and CAT2000, and illustrate significant improvements in predicting viewers' fixations in social scenes.
Fine-Grained Recognition as HSnet Search for Informative Image Parts
Authors: Michael Lam, Behrooz Mahasseni, Sinisa Todorovic
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Fine-Grained Image Classification, image-classification, Image Classification, Informativeness

This work addresses fine-grained image classification. Our work is based on the hypothesis that when dealing with subtle differences among object classes it is critical to identify and only account for a few informative image parts, as the remaining image context may not only be uninformative but may also hurt recognition. This motivates us to formulate our problem as a sequential search for informative parts over a deep feature map produced by a deep Convolutional Neural Network (CNN). A state of this search is a set of proposal bounding boxes in the image, whose "informativeness" is evaluated by the heuristic function (H), and used for generating new candidate states by the successor function (S). The two functions are unified via a Long Short-Term Memory network (LSTM) into a new deep recurrent architecture, called HSnet. Thus, HSnet (i) generates proposals of informative image parts and (ii) fuses all proposals toward final fine-grained recognition. We specify both supervised and weakly supervised training of HSnet depending on the availability of object part annotations. Evaluation on the benchmark Caltech-UCSD Birds 200-2011 and Cars-196 datasets demonstrate our competitive performance relative to the state of the art.
Riemannian Nonlinear Mixed Effects Models: Analyzing Longitudinal Deformations in Neuroimaging
Authors: Hyunwoo J. Kim, Nagesh Adluru, Heemanshu Suri, Baba C. Vemuri, Sterling C. Johnson, Vikas Singh
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Activity Recognition, regression

Statistical machine learning models that operate on manifold-valued data are being extensively studied in vision, motivated by applications in activity recognition, feature tracking and medical imaging. While non-parametric methods have been relatively well studied in the literature, efficient formulations for parametric models (which may offer benefits in small sample size regimes) have only emerged recently. So far, manifold-valued regression models (such as geodesic regression) are restricted to the analysis of cross-sectional data, i.e., the so-called "fixed effects" in statistics. But in most "longitudinal analysis" (e.g., when a participant provides multiple measurements, over time) the application of fixed effects models is problematic. In an effort to answer this need, this paper generalizes non-linear mixed effects model to the regime where the response variable is manifold-valued, i.e., f:R^d -> M. We derive the underlying model and estimation schemes and demonstrate the immediate benefits such a model can provide --- both for group level and individual level analysis --- on longitudinal brain imaging data. The direct consequence of our results is that longitudinal analysis of manifold-valued measurements (especially, the symmetric positive definite manifold) can be conducted in a computationally tractable manner.
Diverse Image Annotation
Authors: Baoyuan Wu, Fan Jia, Wei Liu, Bernard Ghanem
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Diversity, TAG

In this work we study the task of image annotation, of which the goal is to describe an image using a few tags. Instead of predicting the full list of tags, here we target for providing a short list of tags under a limited number (e.g., 3), to cover as much information as possible of the image. The tags in such a short list should be representative and diverse. It means they are required to be not only corresponding to the contents of the image, but also be different to each other.  To this end, we treat the image annotation as a subset selection problem based on the conditional determinantal point process (DPP) model, which formulates the representation and diversity jointly.   We further explore the semantic hierarchy and synonyms among the candidate tags, and require that two tags in a semantic hierarchy or in a pair of synonyms should not be selected simultaneously. This requirement is then embedded into the sampling algorithm according to the learned conditional DPP model.   Besides, we find that traditional metrics for image annotation (e.g., precision, recall and F1 score) only consider the representation, but ignore the diversity. Thus we propose new metrics to evaluate the quality of the selected subset (i.e., the tag list), based on the semantic hierarchy and synonyms. Human study through Amazon Mechanical Turk verifies that the proposed metrics are more close to the human's judgment than traditional metrics. Experiments on two benchmark datasets show that the proposed method can produce more representative and diverse tags, compared with existing image annotation methods.
Factorized Variational Autoencoders for Modeling Audience Reactions to Movies
Authors: Zhiwei Deng, Rajitha Navarathna, Peter Carr, Stephan Mandt, Yisong Yue, Iain Matthews, Greg Mori
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7

Matrix and tensor factorization methods are often used for finding underlying low-dimensional patterns from noisy data. In this paper, we study non-linear tensor factoriza- tion methods based on deep variational autoencoders. Our approach is well-suited for settings where the relationship between the latent representation to be learned and the raw data representation is highly complex. We apply our ap- proach to a large dataset of facial expressions of movie- watching audiences (over 16 million faces). Our experi- ments show that compared to conventional linear factoriza- tion methods, our method achieves better reconstruction of the data, and further discovers interpretable latent factors.
Detecting Masked Faces in the Wild With LLE-CNNs
Authors: Shiming Ge, Jia Li, Qiting Ye, Zhao Luo
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7

Detecting masked faces (i.e., faces with occlusions) is a challenging task due to two main reasons: 1)the absence of large datasets of masked faces, and 2)the absence of facial cues from the masked regions. To address these issues, this paper first introduces a dataset with 30,811 Internet images and 35,806 annotated MAsked FAces, which is denoted as MAFA. Different from many previous datasets, each annotated face in MAFA is partially occluded by mask. By analyzing the characteristics of masked faces, we propose LLE-CNNs that detect masked face via three major modules. The proposal module first combines two pre-trained CNNs to extract candidate facial regions from the input image and represent them with high dimensional descriptors. After that, the embedding module turns such descriptors into vectors of weights with respect to the components in pre-trained dictionaries of representative normal faces and non-faces by using locally linear embedding. In this manner, missing facial cues in the masked regions can be largely recovered, and the influences of noisy cues introduced by diversified masks can be greatly alleviated. Finally, the verification module takes the weight vectors as input and identifies real facial regions as well as their accurate positions by jointly performing the classification and regression tasks within unified CNNs. Experimental results on MAFA show that the proposed approach significantly outperforms 6 state-of-the-arts by at least 15.6% in detecting masked faces.