pwc-archive/papers-with-abstracts huggingface dataset:
Balanced Self-Paced Learning for Generative Adversarial Clustering Network
Authors: Kamran Ghasedi,  Xiaoqian Wang,  Cheng Deng,  Heng Huang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Clustering, Deep Clustering, Image Clustering, Image Retrieval, Unsupervised Spatial Clustering

Clustering is an important problem in various machine learning applications, but still a challenging task when dealing with complex real data. The existing clustering algorithms utilize either shallow models with insufficient capacity for capturing the non-linear nature of data, or deep models with large number of parameters prone to overfitting. In this paper, we propose  a deep Generative Adversarial Clustering Network (ClusterGAN), which tackles the problems of training of deep clustering models in unsupervised manner. ClusterGAN consists of three networks, a discriminator, a generator and a clusterer (i.e. a clustering network). We employ an adversarial game between these three players to synthesize realistic samples given discriminative latent variables via the generator, and learn the inverse mapping of the real samples to the discriminative embedding space via the clusterer. Moreover, we utilize a conditional entropy minimization loss to increase/decrease the similarity of intra/inter cluster samples. Since the ground-truth similarities are unknown in clustering task, we propose a novel balanced self-paced learning algorithm to gradually include samples into training from easy to difficult, while considering the diversity of selected samples from all clusters. Therefore, our method makes it possible to efficiently train clusterers with large depth by leveraging the proposed adversarial game and balanced self-paced learning algorithm. According our experiments, ClusterGAN achieves competitive results compared to the state-of-the-art clustering and hashing models on several datasets.
Parallel Optimal Transport GAN
Authors: Gil Avraham,  Yan Zuo,  Tom Drummond
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Diversity, Image Generation

Although Generative Adversarial Networks (GANs) are known for their sharp realism in image generation, they often fail to estimate areas of the data density. This leads to low modal diversity and at times distorted generated samples. These problems essentially arise from poor estimation of the distance metric responsible for training these networks. To address these issues, we introduce an additional regularisation term which performs optimal transport in parallel within a low dimensional representation space. We demonstrate that operating in a low dimension representation of the data distribution benefits from convergence rate gains in estimating the Wasserstein distance, resulting in more stable GAN training. We empirically show that our regulariser achieves a stabilising effect which leads to higher quality of generated samples and increased mode coverage of the given data distribution. Our method achieves significant improvements on the CIFAR-10, Oxford Flowers and CUB Birds datasets over several GAN baselines both qualitatively and quantitatively.
Causes and Corrections for Bimodal Multi-Path Scanning With Structured Light
Authors: Yu Zhang,  Daniel L. Lau,  Ying Yu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Structured light illumination is an active 3D scanning technique based on projecting/capturing a set of striped patterns and measuring the warping of the patterns as they reflect off a target object's surface. As designed, each pixel in the camera sees exactly one pixel from the projector; however, there are multi-path situations when the scanned surface has a complicated geometry with step edges and other discontinuities in depth or where the target surface has specularities that reflect light away from the camera. These situations are generally referred to multi-path where a camera pixel sees light from multiple projector positions.  In the case of bimodal multi-path, the camera pixel receives light from exactly two positions which occurs along a step edge where the edge slices through a pixel so that the pixel sees both a foreground and background surface. In this paper, we present a general mathematical model to address the bimodal multi-path issue in a phase-measuring-profilometry scanner to measure the constructive and destructive interference between the two light paths, and by taking advantage of this interesting cue, separate the paths and make two decoupled phase measurements. We validate our algorithm with  a number of challenging real-world scenarios, outperforming the state-of-the-art method.
3D Shape Reconstruction From Images in the Frequency Domain
Authors: Weichao Shen,  Yunde Jia,  Yuwei Wu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Shape Reconstruction, Computational Efficiency

Reconstructing the high-resolution volumetric 3D shape from images is challenging due to the cubic growth of computational cost. In this paper, we propose a Fourier-based method that reconstructs a 3D shape from images in a 2D space by predicting slices in the frequency domain. According to the Fourier slice projection theorem, we introduce a thickness map to bridge the domain gap between images in the spatial domain and slices in the frequency domain. The thickness map is the 2D spatial projection of the 3D shape, which is easily predicted from the input image by a general convolutional neural network. Each slice in the frequency domain is the Fourier transform of the corresponding thickness map. All slices constitute a 3D descriptor and the 3D shape is the inverse Fourier transform of the descriptor. Using slices in the frequency domain, our method can transfer the 3D shape reconstruction from the 3D space into the 2D space, which significantly reduces the computational cost. The experiment results on the ShapeNet dataset demonstrate that our method achieves competitive reconstruction accuracy and computational efficiency compared with the state-of-the-art reconstruction methods.
Learning Structure-And-Motion-Aware Rolling Shutter Correction
Authors: Bingbing Zhuang,  Quoc-Huy Tran,  Pan Ji,  Loong-Fah Cheong,  Manmohan Chandraker
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Rolling Shutter Correction

An exact method of correcting the rolling shutter (RS) effect requires recovering the underlying geometry, i.e. the scene structures and the camera motions between scanlines or between views. However, the multiple-view geometry for RS cameras is much more complicated than its global shutter (GS) counterpart, with various degeneracies. In this paper, we first make a theoretical contribution by showing that RS two-view geometry is degenerate in the case of pure translational camera motion. In view of the complex RS geometry, we then propose a Convolutional Neural Network (CNN)-based method which learns the underlying geometry (camera motion and scene structure) from just a single RS image and perform RS image correction. We call our method structure-and-motion-aware RS correction because it reasons about the concealed motions between the scanlines as well as the scene structure. Our method learns from a large-scale dataset synthesized in a geometrically meaningful way where the RS effect is generated in a manner consistent with the camera motion and scene structure. In extensive experiments, our method achieves superior performance compared to other state-of-the-art methods for single image RS correction and subsequent Structure from Motion (SfM) applications.
Diverse Generation for Multi-Agent Sports Games
Authors: Raymond A. Yeh,  Alexander G. Schwing,  Jonathan Huang,  Kevin Murphy
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: counterfactual

In this paper, we propose a new generative model for multi-agent trajectory data, focusing on the case of multi-player sports games. Our model leverages graph neural networks (GNNs) and variational recurrent neural networks (VRNNs) to achieve a permutation equivariant model suitable for sports. On two challenging datasets (basketball and soccer), we show that we are able to produce more accurate forecasts than previous methods. We assess accuracy using various metrics, such as log-likelihood and "best of N" loss, based on N different samples of the future. We also measure the distribution of statistics of interest, such as player location or velocity, and show that the distribution induced by our generative model better matches the empirical distribution of the test set. Finally, we show that our model can perform conditional prediction, which lets us answer counterfactual questions such as "how will the players move differently if A passes the ball to B instead of C?"
GFrames: Gradient-Based Local Reference Frame for 3D Shape Matching
Authors: Simone Melzi,  Riccardo Spezialetti,  Federico Tombari,  Michael M. Bronstein,  Luigi Di Stefano,  Emanuele Rodola
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

We introduce GFrames, a novel local reference frame (LRF) construction for 3D meshes and point clouds. GFrames are based on the computation of the intrinsic gradient of a scalar field defined on top of the input shape. The resulting tangent vector field defines a repeatable tangent direction of the local frame at each point; importantly, it directly inherits the properties and invariance classes of the underlying scalar function, making it remarkably robust under strong sampling artifacts, vertex noise, as well as non-rigid deformations. Existing local descriptors can directly benefit from our repeatable frames, as we showcase in a selection of 3D vision and shape analysis applications where we demonstrate state-of-the-art performance in a variety of challenging settings.
Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking
Authors: Andrii Maksai,  Pascal Fua
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Multiple Object Tracking, Object Tracking

Identity  Switching remains  one  of the  main  difficulties Multiple  Object Tracking (MOT) algorithms  have to deal with.  Many state-of-the-art approaches now  use sequence  models  to solve  this  problem but  their  training can  be affected by biases that decrease their  efficiency. In this paper, we introduce a new training procedure that confronts the algorithm to its own mistakes while explicitly  attempting to  minimize the  number of  switches, which  results in better training.  We propose an iterative scheme of building  a rich training set and using it to learn a  scoring function  that is  an explicit proxy  for the  target tracking metric.  Whether using  only simple  geometric features  or more  sophisticated ones  that also  take appearance  into  account, our  approach outperforms  the state-of-the-art on several MOT benchmarks.
Graph Convolutional Tracking
Authors: Junyu. Gao,  Tianzhu Zhang,  Changsheng Xu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Visual Tracking

Tracking by siamese networks has achieved favorable performance in recent years. However, most of existing siamese methods do not take full advantage of spatial-temporal target appearance modeling under different contextual situations. In fact, the spatial-temporal information can provide diverse features to enhance the target representation, and the context information is important for online adaption of target localization. To comprehensively leverage the spatial-temporal structure of historical target exemplars and get benefit from the context information, in this work, we present a novel Graph Convolutional Tracking (GCT) method for high-performance visual tracking. Specifically, the GCT jointly incorporates two types of Graph Convolutional Networks (GCNs) into a siamese framework for target appearance modeling. Here, we adopt a spatial-temporal GCN to model the structured representation of historical target exemplars. Furthermore, a context GCN is designed to utilize the context of the current frame to learn adaptive features for target localization. Extensive results on 4 challenging benchmarks show that our GCT method performs favorably against state-of-the-art trackers while running around 50 frames per second.
Visual Tracking via Adaptive Spatially-Regularized Correlation Filters
Authors: Kenan Dai,  Dong Wang,  Huchuan Lu,  Chong Sun,  Jianhua Li
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Visual Tracking

In this work, we propose a novel adaptive spatially-regularized correlation filters (ASRCF) model to simultaneously optimize the filter coefficients and the spatial regularization weight. First, this adaptive spatial regularization scheme could learn an effective spatial weight for a specific object and its appearance variations, and therefore result in more reliable filter coefficients during the tracking process. Second, our ASRCF model can be effectively optimized based on the alternating direction method of multipliers, where each subproblem has the closed-from solution. Third, our tracker applies two kinds of CF models to estimate the location and scale respectively. The location CF model exploits ensembles of shallow and deep features to determine the optimal position accurately. The scale CF model works on multi-scale shallow features to estimate the optimal scale efficiently. Extensive experiments on five recent benchmarks show that our tracker performs favorably against many state-of-the-art algorithms, with real-time performance of 28fps.
Learning Joint Gait Representation via Quintuplet Loss Minimization
Authors: Kaihao Zhang,  Wenhan Luo,  Lin Ma,  Wei Liu,  Hongdong Li
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Gait Recognition

Gait recognition is an important biometric method popularly used in video surveillance, where the task is to identify people at a distance by their walking patterns from video sequences. Most of the current successful approaches for gait recognition either use a pair of gait images to form a cross-gait representation or rely on a single gait image for unique-gait representation. These two types of representations emperically complement one another. In this paper, we propose a new Joint Unique-gait and Cross-gait Network (JUCNet), to combine the advantages of unique-gait representation with that of cross-gait representation, leading to an significantly improved performance. Another key contribution of this paper is a novel quintuplet loss function, which simultaneously increases the inter-class differences by pushing representations extracted from different subjects apart and decreases the intra-class variations by pulling representations extracted from the same subject together. Experiments show that our method achieves the state-of-the-art performance tested on standard benchmark datasets, demonstrating its superiority over existing methods.
Sensitive-Sample Fingerprinting of Deep Neural Networks
Authors: Zecheng He,  Tianwei Zhang,  Ruby Lee
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Deep Learning

Numerous cloud-based services are provided to help customers develop and deploy deep learning applications. When a customer deploys a deep learning model in the cloud and serves it to end-users, it is important to be able to verify that the deployed model has not been tampered with.  In this paper, we propose a novel and practical methodology to verify the integrity of remote deep learning models, with only black-box access to the target models. Specifically, we define Sensitive-Sample fingerprints, which are a small set of human unnoticeable transformed inputs that make the model outputs sensitive to the model's parameters. Even small model changes can be clearly reflected in the model outputs. Experimental results on different types of model integrity attacks show that we proposed approach is both effective and efficient. It can detect model integrity breaches with high accuracy (>99.95%) and guaranteed zero false positives on all evaluated attacks. Meanwhile, it only requires up to 103X fewer model inferences, compared with non-sensitive samples.
Soft Labels for Ordinal Regression
Authors: Raul Diaz,  Amit Marathe
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Aesthetics Quality Assessment, Age And Gender Classification, Age Estimation, Depth Estimation, General Classification, Monocular Depth Estimation, regression

Ordinal regression attempts to solve classification problems in which categories are not independent, but rather follow a natural order. It is crucial to classify each class correctly while learning adequate interclass ordinal relationships. We present a simple and effective method that constrains these relationships among categories by seamlessly incorporating metric penalties into ground truth label representations. This encoding allows deep neural networks to automatically learn intraclass and interclass relationships without any explicit modification of the network architecture. Our method converts data labels into soft probability distributions that pair well with common categorical loss functions such as cross-entropy. We show that this approach is effective by using off-the-shelf classification and segmentation networks in four wildly different scenarios: image quality ranking, age estimation, horizon line regression, and monocular depth estimation. We demonstrate that our general-purpose method is very competitive with respect to specialized approaches, and adapts well to a variety of different network architectures and metrics.
Local to Global Learning: Gradually Adding Classes for Training Deep Neural Networks
Authors: Hao Cheng,  Dongze Lian,  Bowen Deng,  Shenghua Gao,  Tao Tan,  Yanlin Geng
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification

We propose a new learning paradigm, Local to Global Learning (LGL), for Deep Neural Networks (DNNs) to improve the performance of classification problems. The core of LGL is to learn a DNN model from fewer categories (local) to more categories (global) gradually within the entire training set. LGL is most related to the Self-Paced Learning (SPL) algorithm but its formulation is different from SPL. SPL trains its data from simple to complex, while LGL from local to global. In this paper, we incorporate the idea of LGL into the learning objective of DNNs and explain why LGL works better from an information-theoretic perspective. Experiments on the toy data, CIFAR-10, CIFAR-100, and ImageNet dataset show that LGL outperforms the baseline and SPL-based algorithms.
What Does It Mean to Learn in Deep Networks? And, How Does One Detect Adversarial Attacks?
Authors: Ciprian A. Corneanu,  Meysam Madadi,  Sergio Escalera,  Aleix M. Martinez
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Self-Driving Cars

The flexibility and high-accuracy of Deep Neural Networks (DNNs) has transformed computer vision. But, the fact that we do not know when a specific DNN will work and when it will fail has resulted in a lack of trust. A clear example is self-driving cars; people are uncomfortable sitting in a car driven by algorithms that may fail under some unknown, unpredictable conditions. Interpretability and explainability approaches attempt to address this by uncovering what a DNN models, i.e., what each node (cell) in the network represents and what images are most likely to activate it. This can be used to generate, for example, adversarial attacks. But these approaches do not generally allow us to determine where a DNN will succeed or fail and   why . i.e., does this learned representation   generalize  to unseen samples? Here, we derive a novel approach to define what it means to learn in deep networks, and how to use this knowledge to detect adversarial attacks. We show how this defines the ability of a network to generalize to unseen testing samples and, most importantly,   why  this is the case.
RENAS: Reinforced Evolutionary Neural Architecture Search
Authors: Yukang Chen,  Gaofeng Meng,  Qian Zhang,  Shiming Xiang,  Chang Huang,  Lisen Mu,  Xinggang Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Neural Architecture Search, Semantic Segmentation

Neural Architecture Search (NAS) is an important yet challenging task in network design due to its high computational consumption. To address this issue, we propose the Reinforced Evolutionary Neural Architecture Search (RENAS), which is an evolutionary method with reinforced mutation for NAS. Our method integrates reinforced mutation into an evolution algorithm for neural architecture exploration, in which a mutation controller is introduced to learn the effects of slight modifications and make mutation actions. The reinforced mutation controller guides the model population to evolve efficiently. Furthermore, as child models can inherit parameters from their parents during evolution, our method requires very limited computational resources. In experiments, we conduct the proposed search method on CIFAR-10 and obtain a powerful network architecture, RENASNet. This architecture achieves a competitive result on CIFAR-10. The explored network architecture is transferable to ImageNet and achieves a new state-of-the-art accuracy, i.e., 75.7% top-1 accuracy with 5.36M parameters on mobile ImageNet. We further test its performance on semantic segmentation with DeepLabv3 on the PASCAL VOC. RENASNet outperforms MobileNet-v1, MobileNet-v2 and NASNet. It achieves 75.83% mIOU without being pretrained on COCO.
Co-Occurrence Neural Network
Authors: Irina Shevlev,  Shai Avidan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Convolutional Neural Networks (CNNs) became a very popular tool for image analysis. Convolutions are fast to compute and easy to store, but they also have some limitations. First, they are shift-invariant and, as a result, they do not adapt to different regions of the image. Second, they have a fixed spatial layout, so small geometric deformations in the layout of a patch will completely change the filter response. For these reasons, we need multiple filters to handle the different parts and variations in the input. We augment the standard convolutional tools used in CNNs with a new filter that addresses both issues raised above. Our filter combines two terms, a spatial filter and a term that is based on the co-occurrence statistics of input values in the neighborhood. The proposed filter is differentiable and can therefore be packaged as a layer in CNN and trained using back-propagation. We show how to train the filter as part of the network and report results on several data sets. In particular, we replace a convolutional layer with hundreds of thousands of parameters with a Co-occurrence Layer consisting of only a few hundred parameters with minimal impact on accuracy.
Attribute-Driven Feature Disentangling and Temporal Aggregation for Video Person Re-Identification
Authors: Yiru Zhao,  Xu Shen,  Zhongming Jin,  Hongtao Lu,  Xian-Sheng Hua
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Attribute, Person Re-Identification, Video-Based Person Re-Identification

Video-based person re-identification plays an important role in surveillance video analysis, expanding image-based methods by learning features of multiple frames. Most existing methods fuse features by temporal average-pooling, without exploring the different frame weights caused by various viewpoints, poses, and occlusions. In this paper, we propose an attribute-driven method for feature disentangling and frame re-weighting. The features of single frames are disentangled into groups of sub-features, each corresponds to specific semantic attributes. The sub-features are re-weighted by the confidence of attribute recognition and then aggregated at the temporal dimension as the final representation. By means of this strategy, the most informative regions of each frame are enhanced and contributes to a more discriminative sequence representation. Extensive ablation studies demonstrate the effectiveness of feature disentangling as well as temporal re-weighting. The experimental results on the iLIDS-VID, PRID-2011 and MARS datasets demonstrate that our proposed method outperforms existing state-of-the-art approaches.