pwc-archive/papers-with-abstracts huggingface dataset:
A General Framework for Curve and Surface Comparison and Registration With Oriented Varifolds
Authors: Irene Kaltenmark, Benjamin Charlier, Nicolas Charon
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Clustering

This paper introduces a general setting for the construction of data fidelity metrics between oriented or non-oriented geometric shapes like curves, curve sets or surfaces. These metrics are based on the representation of shapes as distributions of their local tangent or normal vectors and the definition of reproducing kernels on these spaces. The construction, that combines in one common setting and extends the previous frameworks of currents and varifolds, provides a very large class of kernel metrics which can be easily computed without requiring any kind of parametrization of shapes and which are smooth enough to give robustness to certain imperfections that could result e.g. from bad segmentation. We then give a sense, with synthetic examples, of the versatility and potentialities of such metrics when used in various problems like shape comparison, clustering and diffeomorphic registration.
BranchOut: Regularization for Online Ensemble Tracking With Convolutional Neural Networks
Authors: Bohyung Han, Jack Sim, Hartwig Adam
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Diversity, Visual Tracking

We propose an extremely simple but effective regularization technique of convolutional neural networks (CNNs), referred to as BranchOut, for online ensemble tracking.  Our algorithm employs a CNN for target representation, which has a common convolutional layers but has multiple branches of fully connected layers.  For better regularization, a subset of branches in the CNN are selected randomly for online learning whenever target appearance models need to be updated.  Each branch may have a different number of layers to maintain variable abstraction levels of target appearances. BranchOut with multi-level target representation allows us to learn robust target appearance models with diversity and handle various challenges in visual tracking problem effectively.  The proposed algorithm is evaluated in standard tracking benchmarks and shows the state-of-the-art performance even without additional pretraining on external tracking sequences.
Learning to Align Semantic Segmentation and 2.5D Maps for Geolocalization
Authors: Anil Armagan, Martin Hirzer, Peter M. Roth, Vincent Lepetit
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Semantic Segmentation

We present an efficient method for geolocalization in urban environments starting from a coarse estimate of the location provided by a GPS and using a simple untextured 2.5D model of the surrounding buildings. Our key contribution is a novel efficient and robust method to optimize the pose: We train a Deep Network to predict the best direction to improve a pose estimate, given a semantic segmentation of the input image and a rendering of the buildings from this estimate. We then iteratively apply this CNN until converging to a good pose. This approach avoids the use of reference images of the surroundings, which are difficult to acquire and match, while 2.5D models are broadly available. We can therefore apply it to places unseen during training.
LCR-Net: Localization-Classification-Regression for Human Pose
Authors: Gregory Rogez, Philippe Weinzaepfel, Cordelia Schmid
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Human Pose Estimation, 3D Multi-Person Pose Estimation (absolute), 3D Pose Estimation, Classification, General Classification, Pose Estimation, regression

We propose an end-to-end architecture for joint 2D and 3D human pose estimation in natural images. Key to our approach is the generation and scoring of a number of pose proposals per image, which allows us to predict 2D and 3D pose of multiple people simultaneously. Hence, our approach does not require an approximate localization of the humans for initialization. Our architecture, named LCR-Net, contains 3 main components: 1) the pose proposal generator that suggests potential poses at different locations in the image; 2) a classifier that scores the different pose proposals; and 3) a regressor that refines pose proposals both in 2D and 3D. All three stages share the convolutional feature layers and are trained jointly. The final pose estimation is obtained by integrating over neighboring pose hypotheses, which is shown to improve over a standard non maximum suppression algorithm. Our approach significantly outperforms the state of the art in 3D pose estimation on Human3.6M, a controlled environment. Moreover, it shows promising results on real images for both single and multi-person subsets of the MPII 2D pose benchmark.
Primary Object Segmentation in Videos Based on Region Augmentation and Reduction
Authors: Yeong Jun Koh, Chang-Su Kim
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object, Semantic Segmentation, Unsupervised Video Object Segmentation

A novel algorithm to segment a primary object in a video sequence is proposed in this work. First, we generate candidate regions for the primary object using both color and motion edges. Second,we estimate initial primary object regions, by exploiting the recurrence property of the primary object. Third, we augment the initial regions with missing parts or reducing them by excluding noisy parts repeatedly. This augmentation and reduction process (ARP) identifies the primary object region in each frame. Experimental results demonstrate that the proposed algorithm significantly outperforms the state-of-the-art conventional algorithms on recent benchmark datasets.
Deep 360 Pilot: Learning a Deep Agent for Piloting Through 360deg Sports Videos
Authors: Hou-Ning Hu, Yen-Chen Lin, Ming-Yu Liu, Hsien-Tzu Cheng, Yung-Ju Chang, Min Sun
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object

Watching a 360* sports video requires a viewer to continuously select a viewing angle, either through a sequence of mouse clicks or head movements. To relieve the viewer from this "360 piloting" task, we propose "deep 360 pilot" - a deep learning-based agent for piloting through 360* sports videos automatically. At each frame, the agent observes a panoramic image and has the knowledge of previously selected viewing angles. The task of the agent is to shift the current viewing angle (i.e. action) to the next preferred one (i.e., goal). We propose to directly learn an online policy of the agent from data. Specifically, we leverage a state-of-the-art object detector to propose a few candidate objects of interest (yellow boxes in Fig. 1). Then, a recurrent neural network is used to select the main object (green dash boxes in Fig. 1). Given the main object and previously selected viewing angles, our method regresses a shift in viewing angle to move to the next one. We use the policy gradient technique to jointly train our pipeline, by minimizing: (1) a regression loss measuring the distance between the selected and ground truth viewing angles, (2) a smoothness loss encouraging smooth transition in viewing angle, and (3) maximizing an expected reward of focusing on a foreground object. To evaluate our method, we built a new 360-Sports video dataset consisting of five sports domains. We trained domain-specific agents and achieved the best performance on viewing angle selection accuracy and users' preference compared to [54] and other baselines.
Noise-Blind Image Deblurring
Authors: Meiguang Jin, Stefan Roth, Paolo Favaro
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Blind Image Deblurring, Computational Efficiency, Deblurring, Image Deblurring

   We present a novel approach to noise-blind deblurring, the problem of deblurring an image with known blur, but unknown noise level. We introduce an efficient and robust solution based on a Bayesian framework using a smooth generalization of the 0-1 loss. A novel bound allows the calculation of very high-dimensional integrals in closed form. It avoids the degeneracy of Maximum a-Posteriori (MAP) estimates and leads to an effective noise-adaptive scheme. Moreover, we drastically accelerate our algorithm by using Majorization Minimization (MM) without introducing any approximation or boundary artifacts. We further speed up convergence by turning our algorithm into a neural network termed GradNet, which is highly parallelizable and can be efficiently trained. We demonstrate that our noise-blind formulation can be integrated with different priors and significantly improves existing deblurring algorithms in the noise-blind and in the known-noise case. Furthermore, GradNet leads to state-of-the-art performance across different noise levels, while retaining high computational efficiency.
Multi-Scale FCN With Cascaded Instance Aware Segmentation for Arbitrary Oriented Word Spotting in the Wild
Authors: Dafang He, Xiao Yang, Chen Liang, Zihan Zhou, Alexander G. Ororbi II, Daniel Kifer, C. Lee Giles
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Scene Text Detection, Text Detection

   Scene text detection has attracted great attention these years.  Text potentially exist in a wide variety of images or videos and play an important role in understanding the scene.  In this paper, we present a novel text detection algorithm which is composed of two cascaded steps:  (1) a multi-scale fully convolutional neural network (FCN) is proposed to extract text block regions;  (2) a novel instance (word or line) aware segmentation is designed to further remove false positives and obtain word instances.  The proposed algorithm can accurately localize word or text line in arbitrary orientations, including curved text lines which cannot be handled in a lot of other frameworks.  Our algorithm achieved state-of-the-art performance in ICDAR 2013 (IC13), ICDAR 2015 (IC15) and CUTE80 and Street View Text (SVT) benchmark datasets.
Combining Bottom-Up, Top-Down, and Smoothness Cues for Weakly Supervised Image Segmentation
Authors: Anirban Roy, Sinisa Todorovic
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Segmentation, Object, Segmentation, Semantic Segmentation

This paper addresses the problem of weakly supervised semantic image segmentation. Our goal is to label every pixel in a new image, given only image-level object labels associated with training images. Our problem statement differs from common semantic segmentation, where pixel-wise annotations are typically assumed available in training. We specify a novel deep architecture which fuses three distinct computation processes toward semantic segmentation -- namely, (i) the bottom-up computation of neural activations in a CNN for the image-level prediction of object classes; (ii) the top-down estimation of conditional likelihoods of the CNN's activations given the predicted objects, resulting in probabilistic attention maps per object class; and (iii) the lateral attention-message passing from neighboring neurons at the same CNN layer. The fusion of (i)-(iii) is realized via a conditional random field as recurrent network aimed at generating a smooth and boundary-preserving segmentation. Unlike existing work, we formulate a unified end-to-end learning of all components of our deep architecture. Evaluation on the benchmark PASCAL VOC 2012 dataset demonstrates that we outperform reasonable weakly supervised baselines and state-of-the-art approaches.
Multiple People Tracking by Lifted Multicut and Person Re-Identification
Authors: Siyu Tang, Mykhaylo Andriluka, Bjoern Andres, Bernt Schiele
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Multiple People Tracking, Person Re-Identification, Pose Estimation

Tracking multiple persons in a monocular video of a crowded scene is a challenging task. Humans can master it even if they loose track of a person locally by re-identifying the same person based on their appearance. Care must be taken across long distances, as similar-looking persons need not be identical. In this work, we propose a novel graph-based formulation that links and clusters person hypotheses over time by solving an instance of a minimum cost lifted multicut problem. Our model generalizes previous works by introducing a mechanism for adding long-range attractive connections between nodes in the graph without modifying the original set of feasible solutions. This allows us to reward tracks that assign detections of similar appearance to the same person in a way that does not introduce implausible solutions. To effectively match hypotheses over longer temporal gaps we develop new deep architectures for re-identification of people. They combine holistic representations extracted with deep networks and body pose layout obtained with a state-of-the-art pose estimation model. We demonstrate the effectiveness of our formulation by reporting a new state-of-the-art for the MOT16 benchmark.
Filter Flow Made Practical: Massively Parallel and Lock-Free
Authors: Sathya N. Ravi, Yunyang Xiong, Lopamudra Mukherjee, Vikas Singh
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Optical Flow Estimation

This paper is inspired by a relatively recent work of Seitz and Baker which introduced the so-called Filter Flow model. Filter flow finds the transformation relating a pair of (or multiple) images by identifying a large set of local linear filters; imposing additional constraints on certain structural properties of these filters enables Filter Flow to serve as a general "one stop" construction for a spectrum of problems in vision: from optical flow to defocus to stereo to affine alignment. The idea is beautiful yet the benefits are not borne out in practice because of significant computational challenges. This issue makes most (if not all) deployments for practical vision problems out of reach. The key thrust of our work is to identify mathematically (near) equivalent reformulations of this model that can eliminate this serious limitation. We demonstrate via a detailed optimization-focused development that Filter Flow can indeed be solved fairly efficiently for a wide range of instantiations. We derive efficient algorithms, perform extensive theoretical analysis focused on convergence and parallelization and show how results competitive with the state of the art for many applications can be achieved with negligible application specific adjustments or post-processing. The actual numerical scheme is easy to understand and, implement (30 lines in Matlab) -- this development will enable Filter Flow to be a viable general solver and testbed for numerous applications in the community, going forward.
Acquiring Axially-Symmetric Transparent Objects Using Single-View Transmission Imaging
Authors: Jaewon Kim, Ilya Reshetouski, Abhijeet Ghosh
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Transparent objects

We propose a novel, practical solution for high quality reconstruction of axially-symmetric transparent objects. While a special case, such transparent objects are ubiquitous in the real world. Common examples of these are glasses, goblets, tumblers, carafes, etc., that can have very unique and visually appealing forms making their reconstruction interesting for vision and graphics applications. Our acquisition setup involves imaging such objects from a single viewpoint while illuminating them from directly behind with a few patterns emitted by an LCD panel. Our reconstruction step is then based on optimization of the object's geometry and its refractive index to minimize the difference between observed and simulated transmission/refraction of rays passing through the object. We exploit the object's axial symmetry as a strong shape prior which allows us to achieve robust reconstruction from a single viewpoint using a simple, commodity acquisition setup. We demonstrate high quality reconstruction of several common rotationally symmetric as well as more complex n-fold symmetric transparent objects with our approach.
Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data
Authors: Joel Janai, Fatma Guney, Jonas Wulff, Michael J. Black, Andreas Geiger
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Optical Flow Estimation

Existing optical flow datasets are limited in size and variability due to the difficulty of capturing dense ground truth. In this paper, we tackle this problem by tracking pixels through densely sampled space-time volumes recorded with a high-speed video camera. Our model exploits the linearity of small motions and reasons about occlusions from multiple frames. Using our technique, we are able to establish accurate reference flow fields outside the laboratory in natural environments. Besides, we show how our predictions can be used to augment the input images with realistic motion blur. We demonstrate the quality of the produced flow fields on synthetic and real-world datasets. Finally, we collect a novel challenging optical flow dataset by applying our technique on data from a high-speed camera and analyze the performance of the state-of-the-art in optical flow under various levels of motion blur.
Webly Supervised Semantic Segmentation
Authors: Bin Jin, Maria V. Ortiz Segovia, Sabine Susstrunk
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Segmentation, Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

We propose a weakly supervised semantic segmentation algorithm that uses image tags for supervision. We apply the tags in queries to collect three sets of web images, which encode the clean foregrounds, the common back- grounds, and realistic scenes of the classes. We introduce a novel three-stage training pipeline to progressively learn semantic segmentation models. We first train and refine a class-specific shallow neural network to obtain segmentation masks for each class. The shallow neural networks of all classes are then assembled into one deep convolutional neural network for end-to-end training and testing. Experiments show that our method notably outperforms previous state-of-the-art weakly supervised semantic segmentation approaches on the PASCAL VOC 2012 segmentation bench- mark. We further apply the class-specific shallow neural networks to object segmentation and obtain excellent results.
Non-Contact Full Field Vibration Measurement Based on Phase-Shifting
Authors: Hiroyuki Kayaba, Yuji Kokumai
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7

Vibration measurement systems are widely used in the industry.  A variety of vibration measurement techniques are proposed, including methods using an acceleration sensor, a laser displacement meter, and tracking a marker using a camera. However, these methods have limitations that allow only one point to be measured and require markers. We present a novel, non-contact full field joint measurement technique both of vibrations and shape based on phase-shifting. Our key idea is to acquire the frequency of vibrating objects using FFT to analyze the phase-shift error of vibrating objects. Our proposed algorithm estimates the phase-shift error by iterating frame-to-frame optimization and pixel-to-pixel optimization. A feature of our approach is to measure the surface of vibrating at different frequencies without markers or texture in full fields.  Our developed system is a low cost system, which is composed of a digital-light-processing (DLP) projector and camera (100 frames per second). The results of our experiments show that low frequency vibration of objects can be measured with high accuracy in non-contact.  Also, reconstruction of the vibrating object surface can be performed with high accuracy.
FusionSeg: Learning to Combine Motion and Appearance for Fully Automatic Segmentation of Generic Objects in Videos
Authors: Suyog Dutt Jain, Bo Xiong, Kristen Grauman
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Segmentation, Structured Prediction, Video Segmentation, Video Semantic Segmentation

We propose an end-to-end learning framework for segmenting generic objects in videos. Our method learns to combine appearance and motion information to produce pixel level segmentation masks for all prominent objects in videos. We formulate this task as a structured prediction problem and design a two-stream fully convolutional neural network which fuses together motion and appearance in a unified framework. Since large-scale video datasets with pixel level segmentations are problematic, we show how to bootstrap weakly annotated videos together with existing image recognition datasets for training. Through experiments on three challenging video segmentation benchmarks, our method substantially improves the state-of-the-art for segmenting generic (unseen) objects. Code and pre-trained models are available on the project website.
Task-Driven Dynamic Fusion: Reducing Ambiguity in Video Description
Authors: Xishan Zhang, Ke Gao, Yongdong Zhang, Dongming Zhang, Jintao Li, Qi Tian
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Video Captioning, Video Description

Integrating complementary features from multiple channels is expected to solve the description ambiguity problem in video captioning, whereas inappropriate fusion strategies often harm rather than help the performance. Existing static fusion methods in video captioning such as concatenation and summation cannot attend to appropriate feature channels, thus fail to adaptively support the recognition of various kinds of visual entities such as actions and objects. This paper contributes to: 1)The first in-depth study of the weakness inherent in data-driven static fusion methods for video captioning. 2) The establishment of a task-driven dynamic fusion (TDDF) method. It can adaptively choose different fusion patterns according to model status. 3) The improvement of video captioning. Extensive experiments conducted on two well-known benchmarks demonstrate that our dynamic fusion method outperforms the state-of-the-art results on MSVD with METEOR scores 0.333, and achieves superior METEOR scores 0.278 on MSR-VTT-10K. Compared to single features, the relative improvement derived from our fusion method are 10.0% and 5.7% respectively on two datasets.
Simultaneous Visual Data Completion and Denoising Based on Tensor Rank and Total Variation Minimization and Its Primal-Dual Splitting Algorithm
Authors: Tatsuya Yokota, Hidekata Hontani
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Denoising, Retrieval, Tensor Decomposition

Tensor completion has attracted attention because of its promising ability and generality.  However, there are few studies on noisy scenarios which directly solve an optimization problem consisting of a "noise inequality constraint".  In this paper, we propose a new tensor completion and denoising model including tensor total variation and tensor nuclear norm minimization with a range of values and noise inequalities.  Furthermore, we developed its solution algorithm based on a primal-dual splitting method, which is computationally efficient as compared to tensor decomposition based non-convex optimization.  Lastly, extensive experiments demonstrated the advantages of the proposed method for visual data retrieval such as for color images, movies, and 3D-volumetric data.