pwc-archive/papers-with-abstracts huggingface dataset:
Online Summarization via Submodular and Convex Optimization
Authors: Ehsan Elhamifar, M. Clara De Paolis Kaluza
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Video Summarization

We consider the problem of subset selection in the online setting, where data arrive incrementally. Instead of storing and running subset selection on the entire dataset, we propose an incremental subset selection framework that, at each time instant, uses the previously selected set of representatives and the new batch of data in order to update the set of representatives. We cast the problem as an integer binary optimization minimizing the encoding cost of the data via representatives regularized by the number of selected items. As the proposed optimization is, in general, NP-hard and non-convex, we study a greedy approach based on unconstrained submodular optimization and also propose an efficient convex relaxation. We show that, under appropriate conditions, the solution of our proposed convex algorithm achieves the global optimal solution of the non-convex problem. Our results also address the conventional problem of subset selection in the offline setting, as a special case. By extensive experiments on the problem of video summarization, we demonstrate that our proposed online subset selection algorithms perform well on real data, capturing diverse representative events in videos, while they obtain objective function values close to the offline setting.
From Red Wine to Red Tomato: Composition With Context
Authors: Ishan Misra, Abhinav Gupta, Martial Hebert
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7

Compositionality and contextuality are key building blocks of intelligence. They allow us to compose known concepts to generate new and complex ones. However, traditional learning methods do not model both these properties and require copious amounts of labeled data to learn new concepts. A large fraction of existing techniques, e.g., using late fusion, compose concepts but fail to model contextuality. For example, red in red wine is different from red in red tomatoes. In this paper, we present a simple method that respects contextuality in order to compose classifiers of known visual concepts. Our method builds upon the intuition that classifiers lie in a smooth space where compositional transforms can be modeled. We show how it can generalize to unseen combinations of concepts. Our results on composing attributes, objects as well as composing subject, predicate, and objects demonstrate its strong generalization performance compared to baselines. Finally, we present detailed analysis of our method and highlight its properties.
Superpixel-Based Tracking-By-Segmentation Using Markov Chains
Authors: Donghun Yeo, Jeany Son, Bohyung Han, Joon Hee Han
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: graph construction, Segmentation, Superpixels

 We propose a simple but effective tracking-by-segmentation algorithm using Absorbing Markov Chain (AMC) on superpixel segmentation, where target state is estimated by a combination of bottom-up and top-down approaches, and target segmentation is propagated to subsequent frames in a recursive manner. Our algorithm constructs a graph for AMC using the superpixels identified in two consecutive frames, where background superpixels in the previous frame correspond to absorbing vertices while all other superpixels create transient ones. The weight of each edge depends on the similarity of scores in the end superpixels, which are learned by support vector regression. Once graph construction is completed, target segmentation is estimated using the absorption time of each superpixel. The proposed tracking algorithm achieves substantially improved performance compared to the state-of-the-art segmentation-based tracking techniques in multiple challenging datasets.
Jointly Learning Energy Expenditures and Activities Using Egocentric Multimodal Signals
Authors: Katsuyuki Nakamura, Serena Yeung, Alexandre Alahi, Li Fei-Fei
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Video Understanding

Physiological signals such as heart rate can provide valuable information about an individual's state and activity. However, existing work on computer vision has not yet explored leveraging these signals to enhance egocentric video understanding. In this work, we propose a model for reasoning on multimodal data to jointly predict activities and energy expenditures. We use heart rate signals as privileged self-supervision to derive energy expenditure in a training stage. A multitask objective is used to jointly optimize the two tasks. Additionally, we introduce a dataset that contains 31 hours of egocentric video augmented with heart rate and acceleration signals. This study can lead to new applications such as a visual calorie counter.
An Exact Penalty Method for Locally Convergent Maximum Consensus
Authors: Huu Le, Tat-Jun Chin, David Suter
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7

Maximum consensus estimation plays a critically important role in computer vision. Currently, the most prevalent approach draws from the class of non-deterministic hypothesize-and-verify algorithms, which are cheap but do not guarantee solution quality. On the other extreme, there are global algorithms which are exhaustive search in nature and can be costly for practical-sized inputs. This paper aims to fill the gap between the two extremes by proposing a locally convergent maximum consensus algorithm. Our method is based on a formulating the problem with linear complementarity constraints, then defining a penalized version which is provably equivalent to the original problem. Based on the penalty problem, we develop a Frank-Wolfe algorithm that can deterministically solve the maximum consensus problem. Compared to the randomized techniques, our method is deterministic and locally convergent; relative to the global algorithms, our method is much more practical on realistic input sizes. Further, our approach is naturally applicable to problems with geometric residuals.
Semantic Segmentation via Structured Patch Prediction, Context CRF and Guidance CRF
Authors: Falong Shen, Rui Gan, Shuicheng Yan, Gang Zeng
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Segmentation, Scene Parsing, Segmentation, Semantic Segmentation

This paper describes a fast and accurate semantic image segmentation approach that encodes not only segmentation-specified features but also high-order context compatibilities and boundary guidance constraints. We introduce a structured patch prediction technique to make a trade-off between classification discriminability and boundary sensibility for features. Both label and feature contexts are embedded to ensure recognition accuracy and compatibility, while the complexity of the high order cliques is reduced by a distance-aware sampling and pooling strategy. The proposed joint model also employs a guidance CRF to further enhance the segmentation performance. The message passing step is augmented with the guided filtering which enables an efficient and joint training of the whole system in an end-to-end fashion. Our proposed joint model outperforms the state-of-art on Pascal VOC 2012 and Cityscapes, with mIoU(%) of 82.5 and 79.2 respectively. It also reaches a leading performance on ADE20K, which is the dataset of the scene parsing track in ILSVRC 2016.
Analyzing Computer Vision Data - The Good, the Bad and the Ugly
Authors: Oliver Zendel, Katrin Honauer, Markus Murschitz, Martin Humenberger, Gustavo Fernandez Dominguez
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Autonomous Driving, Stereo Matching, Stereo Matching Hand

In recent years, a great number of datasets were published to train and evaluate computer vision (CV) algorithms. These valuable contributions helped to push CV solutions to a level where they can be used for safety-relevant applications, such as autonomous driving. However, major questions concerning quality and usefulness of test data for CV evaluation are still unanswered. Researchers and engineers try to cover all test cases by using as much test data as possible. In this paper, we propose a different solution for this challenge. We introduce a method for dataset analysis which builds upon an improved version of the CV-HAZOP checklist, a list of potential hazards within the CV domain. Picking stereo vision as an example, we provide an extensive survey of 28 datasets covering the last two decades. We create a tailored checklist and apply it to the datasets Middlebury, KITTI, Sintel, Freiburg, and HCI to present a thorough characterization and quantitative comparison. We confirm the usability of our checklist for identification of challenging stereo situations by applying nine state-of-the-art stereo matching algorithms on the analyzed datasets, showing that hazard frames correlate with difficult frames. We show that challenging datasets still allow a meaningful algorithm evaluation even for small subsets. Finally, we provide a list of missing test cases that are still not covered by current datasets as inspiration for researchers who want to participate in future dataset creation.
3D Convolutional Neural Networks for Efficient and Robust Hand Pose Estimation From Single Depth Images
Authors: Liuhao Ge, Hui Liang, Junsong Yuan, Daniel Thalmann
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Hand Pose Estimation, Data Augmentation, GPU, Hand Pose Estimation, Pose Estimation

We propose a simple, yet effective approach for real-time hand pose estimation from single depth images using three-dimensional Convolutional Neural Networks (3D CNNs). Image based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial information. Our proposed 3D CNN taking a 3D volumetric representation of the hand depth image as input can capture the 3D spatial structure of the input and accurately regress full 3D hand pose in a single pass. In order to make the 3D CNN robust to variations in hand sizes and global orientations, we perform 3D data augmentation on the training data. Experiments show that our proposed 3D CNN based approach outperforms state-of-the-art methods on two challenging hand pose datasets, and is very efficient as our implementation runs at over 215 fps on a standard computer with a single GPU.
MCMLSD: A Dynamic Programming Approach to Line Segment Detection
Authors: Emilio J. Almazan, Ron Tal, Yiming Qian, James H. Elder
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Line Segment Detection

Prior approaches to line segment detection typically involve perceptual grouping in the image domain or global accumulation in the Hough domain.  Here we propose a probabilistic algorithm that merges the advantages of both approaches.  In a first stage lines are detected using a global probabilistic Hough approach.  In the second stage each detected line is analyzed in the image domain to localize the line segments that generated the peak in the Hough map.  By limiting search to a line, the distribution of segments over the sequence of points on the line can be modeled as a Markov chain, and a probabilistically optimal labelling can be computed exactly using a standard dynamic programming algorithm, in linear time.  The Markov assumption also leads to an intuitive ranking method that uses the local marginal posterior probabilities to estimate the expected number of correctly labelled points on a segment.  To assess the resulting Markov Chain Marginal Line Segment Detector (MCMLSD) we develop and apply a novel quantitative evaluation methodology that controls for under- and over-segmentation.  Evaluation on the YorkUrbanDB dataset shows that the proposed MCMLSD method outperforms the state-of-the-art by a substantial margin.
Low-Rank Embedded Ensemble Semantic Dictionary for Zero-Shot Learning
Authors: Zhengming Ding, Ming Shao, Yun Fu
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Dictionary Learning, Zero-Shot Learning

Zero-shot learning for visual recognition has received much interest in the most recent years. However, the semantic gap across visual features and their underlying semantics is still the biggest obstacle in zero-shot learning. To fight off this hurdle, we propose an effective Low-rank Embedded Semantic Dictionary learning (LESD) through ensemble strategy. Specifically, we formulate a novel framework to jointly seek a low-rank embedding and semantic dictionary to link visual features with their semantic representations, which manages to capture shared features across different observed classes. Moreover, ensemble strategy is adopted to learn multiple semantic dictionaries to constitute the latent basis for the unseen classes. Consequently, our model could extract a variety of visual characteristics within objects, which can be well generalized to unknown categories. Extensive experiments on several zero-shot benchmarks verify that the proposed model can outperform the state-of-the-art approaches.
BIND: Binary Integrated Net Descriptors for Texture-Less Object Recognition
Authors: Jacob Chan, Jimmy Addison Lee, Qian Kemao
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object, Object Recognition

This paper presents BIND (Binary Integrated Net Descriptor), a texture-less object detector that encodes multi-layered binary-represented nets for high precision edge-based description. Our proposed concept aligns layers of object-sized patches (nets) onto highly fragmented occlusion resistant line-segment midpoints (linelets) to encode regional information into efficient binary strings. These lightweight nets encourage discriminative object description through their high-spatial resolution, enabling highly precise encoding of the object's edges and internal texture-less information. BIND achieved various invariant properties such as rotation, scale and edge-polarity through its unique binary logical-operated encoding and matching techniques, while performing remarkably well in occlusion and clutter. Apart from yielding efficient computational performance, BIND also attained remarkable recognition rates surpassing recent state-of-the-art texture-less object detectors such as BORDER, BOLD and LINE2D.
S2F: Slow-To-Fast Interpolator Flow
Authors: Yanchao Yang, Stefano Soatto
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Optical Flow Estimation

We introduce a method to compute optical flow at multiple scales of motion, without resorting to multi- resolution or combinatorial methods. It addresses the key problem of small objects moving fast, and resolves the artificial binding between how large an object is and how fast it can move before being diffused away by clas- sical scale-space. Even with no learning, it achieves top performance on the most challenging optical flow benchmark. Moreover, the results are interpretable, and indeed we list the assumptions underlying our method explicitly. The key to our approach is the matching pro- gression from slow to fast, as well as the choice of in- terpolation method, or equivalently the prior, to fill in regions where the data allows it. We use several off- the-shelf components, with relatively low sensitivity to parameter tuning. Computational cost is comparable to the state-of-the-art.
Learned Contextual Feature Reweighting for Image Geo-Localization
Authors: Hyo Jin Kim, Enrique Dunn, Jan-Michael Frahm
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: geo-localization

We address the problem of large scale image geo-localization where the location of an image is estimated by identifying geo-tagged reference images depicting the same place. We propose a novel model for learning image representations that integrates context-aware feature reweighting in order to effectively focus on regions that positively contribute to geo-localization. In particular, we introduce a Contextual Reweighting Network (CRN) that predicts the importance of each region in the feature map based on the image context. Our model is learned end-to-end for the image geo-localization task, and requires no annotation other than image geo-tags for training. In experimental results, the proposed approach significantly outperforms the previous state-of-the-art on the standard geo-localization benchmark datasets. We also demonstrate that our CRN discovers task-relevant contexts without any additional supervision.
On the Effectiveness of Visible Watermarks
Authors: Tali Dekel, Michael Rubinstein, Ce Liu, William T. Freeman
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Matting

Visible watermarking is a widely-used technique for marking and protecting copyrights of many millions of images on the web, yet it suffers from an inherent security flaw---watermarks are typically added in a consistent manner to many images. We show that this consistency allows to automatically estimate the watermark and recover the original images with high accuracy. Specifically, we present a generalized multi-image matting algorithm that  takes a watermarked image collection as input and automatically estimates the "foreground" (watermark), its alpha matte, and  the "background" (original) images.  Since such an attack relies on the consistency of watermarks across image collection, we explore and evaluate how it is affected by various types of inconsistencies in the watermark embedding that could potentially be used to make watermarking more secured.  We demonstrate the algorithm on stock imagery available on the web, and provide extensive quantitative analysis on synthetic watermarked data. A key takeaway message of this paper is that visible watermarks should be designed to not only be robust against removal from a single image, but to be more resistant to mass-scale removal from image collections as well.
Designing Illuminant Spectral Power Distributions for Surface Classification
Authors: Henryk Blasinski, Joyce Farrell, Brian Wandell
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Classification, Dimensionality Reduction, General Classification

There are many scientific, medical and industrial imaging applications where users have full control of the scene illumination and color reproduction is not the primary objective For example, it is possible to co-design sensors and spectral illumination in order to classify and detect changes in biological tissues, organic and inorganic materials, and object surface properties.  In this paper, we propose two different approaches to illuminant spectrum selection for surface classification. In the supervised framework we formulate a biconvex optimization problem where we alternate between optimizing support vector classifier weights and optimal illuminants. We also describe a sparse Principal Component Analysis (PCA) dimensionality reduction approach that can be used with unlabeled data. We efficiently solve the non-convex PCA problem using a convex relaxation and Alternating Direction Method of Multipliers (ADMM). We compare the classification accuracy of a monochrome imaging sensor with optimized illuminants to the classification accuracy of conventional RGB cameras with natural broadband illumination.
CLKN: Cascaded Lucas-Kanade Networks for Image Alignment
Authors: Che-Han Chang, Chun-Nan Chou, Edward Y. Chang
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Homography Estimation

This paper proposes a data-driven approach for image alignment. Our main contribution is a novel network architecture that combines the strengths of convolutional neural networks (CNNs) and the Lucas-Kanade algorithm. The main component of this architecture is a Lucas-Kanade layer that performs the inverse compositional algorithm on convolutional feature maps. To train our network, we develop a cascaded feature learning method that incorporates the coarse-to-fine strategy into the training process. This method learns a pyramid representation of convolutional features in a cascaded manner and yields a cascaded network that performs coarse-to-fine alignment on the feature pyramids. We apply our model to the task of homography estimation, and perform training and evaluation on a large labeled dataset generated from the MS-COCO dataset. Experimental results show that the proposed approach significantly outperforms the other methods.
ShapeOdds: Variational Bayesian Learning of Generative Shape Models
Authors: Shireen Elhabian, Ross Whitaker
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Segmentation, object-detection, Object Detection, Semantic Segmentation

Shape models provide a compact parameterization of a class of shapes, and have been shown to be important to a variety of vision problems, including object detection, tracking, and image segmentation. Learning generative shape models from grid-structured representations, aka silhouettes, is usually hindered by (1) data likelihoods with intractable marginals and posteriors, (2) high-dimensional shape spaces with limited training samples (and the associated risk of overfitting), and (3) estimation of hyperparameters relating to model complexity that often entails computationally expensive grid searches. In this paper, we propose a Bayesian treatment that relies on direct probabilistic formulation for learning generative shape models in the silhouettes space. We propose a variational approach for learning a latent variable model in which we make use of, and extend, recent works on variational bounds of logistic-Gaussian integrals to circumvent intractable marginals and posteriors. Spatial coherency and sparsity priors are also incorporated to lend stability to the optimization problem by regularizing the solution space while avoiding overfitting in this high-dimensional, low-sample-size scenario. We deploy a type-II maximum likelihood estimate of the model hyperparameters to avoid grid searches. We demonstrate that the proposed model generates realistic samples, generalizes to unseen examples, and is able to handle missing regions and/or background clutter, while comparing favorably with recent, neural-network-based approaches.
ER3: A Unified Framework for Event Retrieval, Recognition and Recounting
Authors: Zhanning Gao, Gang Hua, Dong-Qing Zhang, Nebojsa Jojic, Le Wang, Jianru Xue, Nanning Zheng
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Language Modeling, Language Modelling, Retrieval

We develop a unified framework for complex event retrieval, recognition and recounting. The framework is based on a compact video representation that exploits the temporal correlations in image features. Our feature alignment procedure identifies and removes the feature redundancies across frames and outputs an intermediate tensor representation we call video imprint. The video imprint is then fed into a reasoning network, whose attention mechanism parallels that of memory networks used in language modeling. The reasoning network simultaneously recognizes the event category and locates the key pieces of evidence for event recounting. In event retrieval tasks, we show that the compact video representation aggregated from the video imprint achieves significantly better retrieval accuracy compared with existing methods. We also set new state of the art results in event recognition tasks with an additional benefit: The latent structure in our reasoning network highlights the areas of the video imprint and can be directly used for event recounting. As video imprint maps back to locations in the video frames, the network allows not only the identification of key frames but also specific areas inside each frame which are most influential to the decision process.