pwc-archive/papers-with-abstracts huggingface dataset:
Phase-Only Image Based Kernel Estimation for Single Image Blind Deblurring
Authors: Liyuan Pan,  Richard Hartley,  Miaomiao Liu,  Yuchao Dai
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Blind Image Deblurring, Deblurring, Image Deblurring, Single-Image Blind Deblurring

The image motion blurring process is generally modelled as the convolution of a blur kernel with a latent image. Therefore, the estimation of the blur kernel is essentially important for blind image deblurring. Unlike existing approaches which focus on approaching the problem by enforcing various priors on the blur kernel and the latent image, we are aiming at obtaining a high quality blur kernel directly by studying the problem in the frequency domain. We show that the auto-correlation of the absolute phase-only image 1 can provide faithful information about the motion (e.g., the motion direction and magnitude, we call it the motion pattern in this paper.) that caused the blur, leading to a new and efficient blur kernel estimation approach. The blur kernel is then refined and the sharp image is estimated by solving an optimization problem by enforcing a regularization on the blur kernel and the latent image. We further extend our approach to handle non-uniform blur, which involves spatially varying blur kernels. Our approach is evaluated extensively on synthetic and real data and shows good results compared to the state-of-the-art deblurring approaches.
FOCNet: A Fractional Optimal Control Network for Image Denoising
Authors: Xixi Jia,  Sanyang Liu,  Xiangchu Feng,  Lei Zhang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Denoising, Image Denoising

Deep convolutional neural networks (DCNN) have been successfully used in many low-level vision problems such as image denoising. Recent studies on the mathematical foundation of DCNN has revealed that the forward propagation of DCNN corresponds to a dynamic system, which can be described by an ordinary differential equation (ODE) and solved by the optimal control method. However, most of these methods employ integer-order differential equation, which has local connectivity in time space and cannot describe the long-term memory of the system. Inspired by the fact that the fractional-order differential equation has long-term memory, in this paper we develop an advanced image denoising network, namely FOCNet, by solving a fractional optimal control (FOC) problem. Specifically, the network structure is designed based on the discretization of a fractional-order differential equation, which enjoys long-term memory in both forward and backward passes. Besides, multi-scale feature interactions are introduced into the FOCNet to strengthen the control of the dynamic system. Extensive experiments demonstrate the leading performance of the proposed FOCNet on image denoising. Code will be made available.
Orthogonal Decomposition Network for Pixel-Wise Binary Classification
Authors: Chang Liu,  Fang Wan,  Wei Ke,  Zhuowei Xiao,  Yuan Yao,  Xiaosong Zhang,  Qixiang Ye
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Binary Classification, Classification, Edge Detection, General Classification, Keypoint Detection, Saliency Detection

The weight sharing scheme and spatial pooling operations in Convolutional Neural Networks (CNNs) introduce semantic correlation to neighboring pixels on feature maps and therefore deteriorate their pixel-wise classification performance. In this paper, we implement an Orthogonal Decomposition Unit (ODU) that transforms a convolutional feature map into orthogonal bases targeting at de-correlating neighboring pixels on convolutional features. In theory, complete orthogonal decomposition produces orthogonal bases which can perfectly reconstruct any binary mask (ground-truth). In practice, we further design incomplete orthogonal decomposition focusing on de-correlating local patches which balances the reconstruction performance and computational cost. Fully Convolutional Networks (FCNs) implemented with ODUs, referred to as Orthogonal Decomposition Networks (ODNs), learn de-correlated and complementary convolutional features and fuse such features in a pixel-wise selective manner. Over pixel-wise binary classification tasks for two-dimensional image processing, specifically  skeleton detection, edge detection, and saliency detection, and one-dimensional keypoint detection, specifically S-wave arrival time detection for earthquake localization, ODNs consistently improves the state-of-the-arts with significant margins.
Combinatorial Persistency Criteria for Multicut and Max-Cut
Authors: Jan-Hendrik Lange,  Bjoern Andres,  Paul Swoboda
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Combinatorial Optimization

In combinatorial optimization, partial variable assignments are called persistent if they agree with some optimal solution. We propose persistency criteria for the multicut and max-cut problem as well as fast combinatorial routines to verify them. The criteria that we derive are based on mappings that improve feasible multicuts, respectively cuts. Our elementary criteria can be checked enumeratively. The more advanced ones rely on fast algorithms for upper and lower bounds for the respective cut problems and max-flow techniques for auxiliary min-cut problems. Our methods can be used as a preprocessing technique for reducing problem sizes or for computing partial optimality guarantees for solutions output by heuristic solvers. We show the efficacy of our methods on instances of both problems from computer vision, biomedical image analysis and statistical physics.
Polynomial Representation for Persistence Diagram
Authors: Zhichao Wang,  Qian Li,  Gang Li,  Guandong Xu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: BIG-bench Machine Learning, Topological Data Analysis

Persistence diagram (PD) has been considered as a compact descriptor for topological data analysis (TDA). Unfortunately, PD cannot be directly used in machine learning methods since it is a multiset of points. Recent efforts have been devoted to transforming PDs into vectors to accommodate machine learning methods. However, they share one common shortcoming: the mapping of PDs to a feature representation depends on a pre-defined polynomial. To address this limitation, this paper proposes an algebraic representation for PDs, i.e., polynomial representation. In this work, we discover a set of general polynomials that vanish on vectorized PDs and extract the task-adapted feature representation from these polynomials. We also prove two attractive properties of the proposed polynomial representation, i.e., stability and linear separability. Experiments also show that our method compares favorably with state-of-the-art TDA methods.
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks
Authors: Xiaolong Jiang,  Zehao Xiao,  Baochang Zhang,  Xiantong Zhen,  Xianbin Cao,  David Doermann,  Ling Shao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Crowd Counting, Decoder, Density Estimation

Crowd counting has recently attracted increasing interest in computer vision but remains a challenging problem. In this paper, we propose a trellis encoder-decoder network (TEDnet) for crowd counting, which focuses on generating high-quality density estimation maps. The major contributions are four-fold. First, we develop a new trellis architecture that incorporates multiple decoding paths to hierarchically aggregate features at different encoding stages, which improves the representative capability of convolutional features for large variations in objects. Second, we employ dense skip connections interleaved across paths to facilitate sufficient multi-scale feature fusions, which also helps TEDnet to absorb the supervision information. Third, we propose a new combinatorial loss to enforce similarities in local coherence and spatial correlation between maps. By distributedly imposing this combinatorial loss on intermediate outputs, TEDnet can improve the back-propagation process and alleviate the gradient vanishing problem. Finally, on four widely-used benchmarks, our TEDnet achieves the best overall performance in terms of both density map quality and counting accuracy, with an improvement up to 14% in MAE metric. These results validate the effectiveness of TEDnet for crowd counting.
Cross-Atlas Convolution for Parameterization Invariant Learning on Textured Mesh Surface
Authors: Shiwei Li,  Zixin Luo,  Mingmin Zhen,  Yao Yao,  Tianwei Shen,  Tian Fang,  Long Quan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, Segmentation

We present a convolutional network architecture for direct feature learning on mesh surfaces through their atlases of texture maps. The texture map encodes the parameterization from 3D to 2D domain, rendering not only RGB values but also rasterized geometric features if necessary. Since the parameterization of texture map is not pre-determined, and depends on the surface topologies, we therefore introduce a novel cross-atlas convolution to recover the original mesh geodesic neighborhood, so as to achieve the invariance property to arbitrary parameterization. The proposed module is integrated into classification and segmentation architectures, which takes the input texture map of a mesh, and infers the output predictions. Our method not only shows competitive performances on classification and segmentation public benchmarks, but also paves the way for the broad mesh surfaces learning.
Marginalized Latent Semantic Encoder for Zero-Shot Learning
Authors: Zhengming Ding,  Hongfu Liu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Graph Reconstruction, Zero-Shot Learning

Zero-shot learning has been well explored to precisely identify new unobserved classes through a visual-semantic function obtained from the existing objects. However, there exist two challenging obstacles: one is that the human-annotated semantics are insufficient to fully describe the visual samples; the other is the domain shift across existing and new classes. In this paper, we attempt to exploit the intrinsic relationship in the semantic manifold when given semantics are not enough to describe the visual objects, and enhance the generalization ability of the visual-semantic function with marginalized strategy. Specifically, we design a Marginalized Latent Semantic Encoder (MLSE), which is learned on the augmented seen visual features and the latent semantic representation. Meanwhile, latent semantics are discovered under an adaptive graph reconstruction scheme based on the provided semantics. Consequently, our proposed algorithm could enrich visual characteristics from seen classes, and well generalize to unobserved classes. Experimental results on zero-shot benchmarks demonstrate that the proposed model delivers superior performance over the state-of-the-art zero-shot learning approaches.
A Robust Local Spectral Descriptor for Matching Non-Rigid Shapes With Incompatible Shape Structures
Authors: Yiqun Wang,  Jianwei Guo,  Dong-Ming Yan,  Kai Wang,  Xiaopeng Zhang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Triplet

Constructing a robust and discriminative local descriptor for 3D shape is a key component of many computer vision applications. Although existing learning-based approaches can achieve good performance in some specific benchmarks, they usually fail to learn enough information from shapes with different shape types and structures (e.g., spatial resolution, connectivity, transformations, etc.)  Focusing on this issue, in this paper, we present a more discriminative local descriptor for deformable 3D shapes with incompatible structures. Based on the spectral embedding using the Laplace-Beltrami framework on the surface, we first construct a novel local spectral feature which shows great resilience to change in mesh resolution, triangulation, transformation. Then the multi-scale local spectral features around each vertex are encoded into a `geometry image', called vertex spectral image, in a very compact way. Such vertex spectral images can be efficiently trained to learn local descriptors using a triplet neural network. Finally, for training and evaluation, we present a new benchmark dataset by extending the widely used FAUST dataset. We utilize a remeshing approach to generate modified shapes with different structures. We evaluate the proposed approach thoroughly and make an extensive comparison to demonstrate that our approach outperforms recent state-of-the-art methods on this benchmark.
Spot and Learn: A Maximum-Entropy Patch Sampler for Few-Shot Image Classification
Authors: Wen-Hsuan Chu,  Yu-Jhe Li,  Jing-Cheng Chang,  Yu-Chiang Frank Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Data Augmentation, Few-Shot Image Classification, Few-Shot Learning, General Classification, image-classification, Image Classification, Reinforcement Learning

Few-shot learning (FSL) requires one to learn from object categories with a small amount of training data (as novel classes), while the remaining categories (as base classes) contain a sufficient amount of data for training. It is often desirable to transfer knowledge from the base classes and derive dominant features efficiently for the novel samples. In this work, we propose a sampling method that de-correlates an image based on maximum entropy reinforcement learning, and extracts varying sequences of patches on every forward-pass with discriminative information observed. This can be viewed as a form of "learned" data augmentation in the sense that we search for different sequences of patches within an image and performs classification with aggregation of the extracted features, resulting in improved FSL performances. In addition, our positive and negative sampling policies along with a newly defined reward function would favorably improve the effectiveness of our model. Our experiments on two benchmark datasets confirm the effectiveness of our framework and its superiority over recent FSL approaches.
Synthesizing Environment-Aware Activities via Activity Sketches
Authors: Yuan-Hong Liao,  Xavier Puig,  Marko Boben,  Antonio Torralba,  Sanja Fidler
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

In order to learn to perform activities from demonstrations or descriptions, agents need to distill what the essence of the given activity is, and how it can be adapted to new environments. In this work, we address the problem: environment-aware program generation. Given a visual demonstration or a description of an activity, we generate program sketches representing the essential instructions and propose a model to flesh these into full programs representing the actions needed to perform the activity under the presented environmental constraints. To this end, we build upon VirtualHome, to create a new dataset VirtualHome-Env, where we collect program sketches to represent activities and match programs with environments that can afford them. Furthermore, we construct a knowledge base to sample realistic environments and another knowledge base to seek out the  programs under the sampled environments. Finally, we propose RNN-ResActGraph, a network that generates a program from a given sketch and an environment graph and tracks the changes in the environment induced by the program.
Versatile Multiple Choice Learning and Its Application to Vision Computing
Authors: Kai Tian,  Yi Xu,  Shuigeng Zhou,  Jihong Guan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: image-classification, Image Classification, Image Segmentation, Multiple-choice, Semantic Segmentation

Most existing ensemble methods aim to train the underlying embedded models independently and simply aggregate their final outputs via averaging or weighted voting. As many prediction tasks contain uncertainty, most of these ensemble methods just reduce variance of the predictions without considering the collaborations among the ensembles. Different from these ensemble methods, multiple choice learning (MCL) methods exploit the cooperation among all the embedded models to generate multiple diverse hypotheses. In this paper, a new MCL method, called vMCL (the abbreviation of versatile Multiple Choice Learning), is developed to extend the application scenarios of MCL methods by ensembling deep neural networks. Our vMCL method keeps the advantage of existing MCL methods while overcoming their major drawback, thus achieves better performance. The novelty of our vMCL lies in three aspects: (1) a choice network is designed to learn the confidence level of each specialist which can provide the best prediction base on multiple hypotheses; (2) a hinge loss is introduced to alleviate the overconfidence issue in MCL settings; (3) Easy to be implemented and can be trained in an end-to-end manner, which is a very attractive feature for many real-world applications. Experiments on image classification and image segmentation task show that vMCL outperforms the existing state-of-the-art MCL methods.
EV-Gait: Event-Based Robust Gait Recognition Using Dynamic Vision Sensors
Authors: Yanxiang Wang,  Bowen Du,  Yiran Shen,  Kai Wu,  Guangrong Zhao,  Jianguo Sun,  Hongkai Wen
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Gait Recognition

In this paper, we introduce a new type of sensing modality, the Dynamic Vision Sensors (Event Cameras), for the task of gait recognition. Compared with the traditional RGB sensors, the event cameras have many unique advantages such as ultra low resources consumption, high temporal resolution and much larger dynamic range. However, those cameras only produce noisy and asynchronous events of intensity changes rather than frames, where conventional vision-based gait recognition algorithms can't be directly applied. To address this, we propose a new Event-based Gait Recognition (EV-Gait) approach, which exploits motion consistency to effectively remove noise, and uses a deep neural network to recognise gait from the event streams. To evaluate the performance of EV-Gait, we collect two event-based gait datasets, one from real-world experiments and the other by converting the publicly available RGB gait recognition benchmark CASIA-B. Extensive experiments show that EV-Gait can get nearly 96% recognition accuracy in the real-world settings, while on the CASIA-B benchmark it achieves comparable performance with state-of-the-art RGB-based gait recognition approaches.
ToothNet: Automatic Tooth Instance Segmentation and Identification From Cone Beam CT Images
Authors: Zhiming Cui,  Changjian Li,  Wenping Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: GPU, Instance Segmentation, Region Proposal, Segmentation, Semantic Segmentation

This paper proposes a method that uses deep convolutional neural networks to achieve automatic and accurate tooth instance segmentation and identification from CBCT (cone beam CT) images for digital dentistry. The core of our method is a two-stage network. In the first stage, an edge map is extracted from the input CBCT image to enhance image contrast along shape boundaries. Then this edge map and the input images are passed to the second stage. In the second stage, we build our network upon the 3D region proposal network (RPN) with a novel learned-similarity matrix to help efficiently remove redundant proposals, speed up training and save GPU memory. To resolve the ambiguity in the identification task, we encode teeth spatial relationships as an additional feature input in the identification task, which helps to remarkably improve the identification accuracy. Our evaluation, comparison and comprehensive ablation studies demonstrate that our method produces accurate instance segmentation and identification results automatically and outperforms the state-of-the-art approaches. To the best of our knowledge, our method is the first to use neural networks to achieve automatic tooth segmentation and identification from CBCT images.
L3-Net: Towards Learning Based LiDAR Localization for Autonomous Driving
Authors: Weixin Lu,  Yao Zhou,  Guowei Wan,  Shenhua Hou,  Shiyu Song
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Autonomous Driving

We present L3-Net - a novel learning-based LiDAR localization system that achieves centimeter-level localization accuracy, comparable to prior state-of-the-art systems with hand-crafted pipelines. Rather than relying on these hand-crafted modules, we innovatively implement the use of various deep neural network structures to establish a learning-based approach. L3-Net learns local descriptors specifically optimized for matching in different real-world driving scenarios. 3D convolutions over a cost volume built in the solution space significantly boosts the localization accuracy. RNNs are demonstrated to be effective in modeling the vehicle's dynamics, yielding better temporal smoothness and accuracy. We comprehensively validate the effectiveness of our approach using freshly collected datasets. Multiple trials of repetitive data collection over the same road and areas make our dataset ideal for testing localization systems. The  SunnyvaleBigLoop sequences, with a year's time interval between the collected mapping and testing data, made it quite challenging, but the low localization error of our method in these datasets demonstrates its maturity for real industrial implementation.
Reasoning-RCNN: Unifying Adaptive Global Reasoning Into Large-Scale Object Detection
Authors: Hang Xu,  Chenhan Jiang,  Xiaodan Liang,  Liang Lin,  Zhenguo Li
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Object, object-detection, Object Detection

In this paper, we address the large-scale object detection problem with thousands of categories, which poses severe challenges due to long-tail data distributions, heavy occlusions, and class ambiguities. However, the dominant object detection paradigm is limited by treating each object region separately without considering crucial semantic dependencies among objects. In this work, we introduce a novel Reasoning-RCNN to endow any detection networks the capability of adaptive global reasoning over all object regions by exploiting diverse human commonsense knowledge. Instead of only propagating the visual features on the image directly, we evolve the high-level semantic representations of all categories globally to avoid distracted or poor visual features in the image. Specifically, built on feature representations of basic detection network, the proposed network first generates a global semantic pool by collecting the weights of previous classification layer for each category, and then adaptively enhances each object features via attending different semantic contexts in the global semantic pool. Rather than propagating information from all semantic information that may be noisy, our adaptive global reasoning automatically discovers most relative categories for feature evolving. Our Reasoning-RCNN is light-weight and flexible enough to enhance any detection backbone networks, and extensible for integrating any knowledge resources. Solid experiments on object detection benchmarks show the superiority of our Reasoning-RCNN, e.g. achieving around 16% improvement on VisualGenome, 37% on ADE in terms of mAP and 15% improvement on COCO.