pwc-archive/papers-with-abstracts huggingface dataset:
Large-Scale Few-Shot Learning: Knowledge Transfer With Class Hierarchy
Authors: Aoxue Li,  Tiange Luo,  Zhiwu Lu,  Tao Xiang,  Liwei Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Few-Shot Learning, Transfer Learning, Zero-Shot Learning

Recently, large-scale few-shot learning (FSL) becomes topical. It is discovered that, for a large-scale FSL problem with 1,000 classes in the source domain, a strong baseline emerges, that is, simply training a deep feature embedding model using the aggregated source classes and performing nearest neighbor (NN) search using the learned features on the target classes. The state-of-the-art large-scale FSL methods struggle to beat this baseline, indicating intrinsic limitations on scalability. To overcome the challenge, we propose a novel large-scale FSL model by learning transferable visual features with the class hierarchy which encodes the semantic relations between source and target classes. Extensive experiments show that the proposed model significantly outperforms not only the NN baseline but also the state-of-the-art alternatives. Furthermore, we show that the proposed model can be easily extended to the large-scale zero-shot learning (ZSL) problem and also achieves the state-of-the-art results.
Stochastic Class-Based Hard Example Mining for Deep Metric Learning
Authors: Yumin Suh,  Bohyung Han,  Wonsik Kim,  Kyoung Mu Lee
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Retrieval, Metric Learning, Retrieval

Performance of deep metric learning depends heavily on the capability of mining hard negative examples during training.  However, many metric learning algorithms often require intractable computational cost due to frequent feature computations and nearest neighbor searches in a large-scale dataset. As a result, existing approaches often suffer from trade-off between training speed and prediction accuracy. To alleviate this limitation, we propose a stochastic hard negative mining method. Our key idea is to adopt class signatures that keep track of feature embedding online with minor additional cost during training, and identify hard negative example candidates using the signatures. Given an anchor instance, our algorithm first selects a few hard negative classes based on the class-to-sample distances and then performs a refined search in an instance-level only from the selected classes. As most of the classes are discarded at the first step, it is much more efficient than exhaustive search while effectively mining a large number of hard examples. Our experiment shows that the proposed technique improves image retrieval accuracy substantially; it achieves the state-of-the-art performance on the several standard benchmark datasets.
Ensemble Deep Manifold Similarity Learning Using Hard Proxies
Authors: Nicolas Aziere,  Sinisa Todorovic
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Clustering, Image Retrieval, Retrieval, Triplet

This paper is about learning deep representations of images such that images belonging to the same class have more similar representations than those belonging to different classes. For this goal, prior work typically uses the triplet or N-pair loss, specified in terms of either l2-distances or dot-products between deep features. However, such formulations seem poorly  suited to the highly non-Euclidean deep feature space. Our first contribution is in specifying the N-pair loss in terms of manifold similarities between deep features. We introduce a new time- and memory-efficient method for estimating the manifold similarities by using a closed-form convergence solution of the Random Walk algorithm. Our efficiency comes, in part, from following the recent work that randomly partitions the deep feature space, and expresses image distances via representatives of the resulting subspaces, a.k.a. proxies. Our second contribution is aimed at reducing overfitting by estimating hard proxies that are as close to one another as possible, but remain in their respective subspaces. Our evaluation demonstrates that we outperform the state of the art in both image retrieval and clustering on the benchmark  CUB-200-2011, Cars196, and Stanford Online Products datasets.
Occlusion-Net: 2D/3D Occluded Keypoint Localization Using Graph Networks
Authors: N. Dinesh Reddy,  Minh Vo,  Srinivasa G. Narasimhan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Car Instance Understanding, 3D Object Reconstruction From A Single Image, 3D Pose Estimation, Decoder, Pose Estimation, Vehicle Pose Estimation

We present Occlusion-Net, a framework to predict 2D and 3D locations of occluded keypoints for objects, in a largely self-supervised manner. We use an off-the-shelf detector as input (like MaskRCNN) that is trained only on visible key point annotations. This is the only supervision used in this work. A graph encoder network then explicitly classifies invisible edges and a graph decoder network corrects the occluded keypoint locations from the initial detector. Central to this work is a trifocal tensor loss that provides indirect self-supervision for occluded keypoint locations that are visible in other views of the object. The 2D keypoints are then passed into a 3D graph network that estimates the 3D shape and camera pose using the self-supervised re-projection loss. At test time, our approach successfully localizes keypoints in a single view under a diverse set of severe occlusion settings. We demonstrate and evaluate our approach on synthetic CAD data as well as a large image set capturing vehicles at many busy city intersections. As an interesting aside, we compare the accuracy of human labels of invisible keypoints against those obtained from geometric trifocal-tensor loss.
Efficient Featurized Image Pyramid Network for Single Shot Detector
Authors: Yanwei Pang,  Tiancai Wang,  Rao Muhammad Anwer,  Fahad Shahbaz Khan,  Ling Shao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: GPU

Single-stage object detectors have recently gained popularity due to their combined advantage of high detection accuracy and real-time speed. However, while promising results have been achieved by these detectors on standard-sized objects, their performance on small objects is far from satisfactory. To detect very small/large objects, classical pyramid representation can be exploited, where an image pyramid is used to build a feature pyramid (featurized image pyramid), enabling detection across a range of scales. Existing single-stage detectors avoid such a featurized image pyramid representation due to its memory and time complexity. In this paper, we introduce a light-weight architecture to efficiently produce featurized image pyramid in a single-stage detection framework. The resulting multi-scale features are then injected into the prediction layers of the detector using an attention module. The performance of our detector is validated on two benchmarks: PASCAL VOC and MS COCO. For a 300x300 input, our detector operates at 111 frames per second (FPS) on a Titan X GPU, providing state-of-the-art detection accuracy on PASCAL VOC 2007 testset. On the MS COCO testset, our detector achieves state-of-the-art results surpassing all existing single-stage methods in the case of single-scale inference.
MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition
Authors: Weihe Zhang,  Yali Wang,  Yu Qiao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Diversity, Object Recognition

Deep Neural Networks (DNNs) have achieved remarkable successes in large-scale visual recognition. However, they often suffer from overfitting under noisy labels. To alleviate this problem, we propose a conceptually simple but effective MetaCleaner, which can learn to hallucinate a clean representation of an object category, according to a small noisy subset from the same category. Specially, MetaCleaner consists of two flexible submodules. The first submodule, namely Noisy Weighting, can estimate the confidence scores of all the images in the noisy subset, by analyzing their deep features jointly. The second submodule, namely Clean Hallucinating, can generate a clean representation from the noisy subset, by summarizing the noisy images with their confidence scores. Via MetaCleaner, DNNs can strengthen its robustness to noisy labels, as well as enhance its generalization capacity with richer data diversity. Moreover, MetaCleaner can be easily integrated into the standard training procedure of DNNs, which promotes its value for real-life applications. We conduct extensive experiments on two popular benchmarks in noisy-labeled recognition, i.e., Food-101N and Clothing1M. For both datasets, our MetaCleaner significantly outperforms baselines, and achieves the state-of-the-art performance.
Fitting Multiple Heterogeneous Models by Multi-Class Cascaded T-Linkage
Authors: Luca Magri,  Andrea Fusiello
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Model Selection

This paper addresses the problem of multiple models fitting in the general context where the sought structures can be described by a mixture of heterogeneous parametric models drawn from different classes.  To this end, we conceive a multi-model selection framework that extend T-linkage to cope with different nested class of models. Our method, called MCT, compares favourably with the state-of-the-art on publicly available data-sets for various fitting problems: lines and conics, homographies and fundamental matrices, planes and cylinders.
A Late Fusion CNN for Digital Matting
Authors: Yunke Zhang,  Lixue Gong,  Lubin Fan,  Peiran Ren,  Qixing Huang,  Hujun Bao,  Weiwei Xu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, General Classification, Image Matting

This paper studies the structure of a deep convolutional neural network to predict the foreground alpha matte by taking a single RGB image as input. Our network is fully convolutional with two decoder branches for the foreground and background classification respectively. Then a fusion branch is used to integrate the two classification results which gives rise to alpha values as the soft segmentation result. This design provides more degrees of freedom than a single decoder branch for the network to obtain better alpha values during training. The network can implicitly produce trimaps without user interaction, which is easy to use for novices without expertise in digital matting. Experimental results demonstrate that our network can achieve high-quality alpha mattes for various types of objects and outperform the state-of-the-art CNN-based image matting methods on the human image matting task.
BASNet: Boundary-Aware Salient Object Detection
Authors: Xuebin Qin,  Zichen Zhang,  Chenyang Huang,  Chao Gao,  Masood Dehghan,  Martin Jagersand
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Camouflaged Object Segmentation, Decoder, Dichotomous Image Segmentation, GPU, Object, object-detection, Object Detection, RGB Salient Object Detection, Saliency Prediction, Salient Object Detection, SSIM

Deep Convolutional Neural Networks have been adopted for salient object detection and achieved the state-of-the-art performance. Most of the previous works however focus on region accuracy but not on the boundary quality. In this paper, we propose a predict-refine architecture, BASNet, and a new hybrid loss for Boundary-Aware Salient object detection. Specifically, the architecture is composed of a densely supervised Encoder-Decoder network and a residual refinement module, which are respectively in charge of saliency prediction and saliency map refinement. The hybrid loss guides the network to learn the transformation between the input image and the ground truth in a three-level hierarchy -- pixel-, patch- and map- level -- by fusing Binary Cross Entropy (BCE), Structural SIMilarity (SSIM) and Intersection-over-Union (IoU) losses. Equipped with the hybrid loss, the proposed predict-refine architecture is able to effectively segment the salient object regions and accurately predict the fine structures with clear boundaries. Experimental results on six public datasets show that our method outperforms the state-of-the-art methods both in terms of regional and boundary evaluation measures. Our method runs at over 25 fps on a single GPU. The code is available at: https://github.com/NathanUA/BASNet.
ZigZagNet: Fusing Top-Down and Bottom-Up Context for Object Segmentation
Authors: Di Lin,  Dingguo Shen,  Siting Shen,  Yuanfeng Ji,  Dani Lischinski,  Daniel Cohen-Or,  Hui Huang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

Multi-scale context information has proven to be essential for object segmentation tasks. Recent works construct the multi-scale context by aggregating convolutional feature maps extracted by different levels of a deep neural network. This is typically done by propagating and fusing features in a one-directional, top-down and bottom-up, manner. In this work, we introduce ZigZagNet, which aggregates a richer multi-context feature map by using not only dense top-down and bottom-up propagation, but also by introducing pathways crossing between different levels of the top-down and the bottom-up hierarchies, in a zig-zag fashion. Furthermore, the context information is exchanged and aggregated over multiple stages, where the fused feature maps from one stage are fed into the next one, yielding a more comprehensive context for improved segmentation performance. Our extensive evaluation on the public benchmarks demonstrates that ZigZagNet surpasses the state-of-the-art accuracy for both semantic segmentation and instance segmentation tasks.
Object Instance Annotation With Deep Extreme Level Set Evolution
Authors: Zian Wang,  David Acuna,  Huan Ling,  Amlan Kar,  Sanja Fidler
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Object, Segmentation, Semantic Segmentation, Video Segmentation, Video Semantic Segmentation

In this paper, we tackle the task of interactive object segmentation. We revive the old ideas on level set segmentation which framed object annotation as curve evolution. Carefully designed energy functions ensured that the curve was well aligned with image boundaries, and generally "well behaved". The Level Set Method can handle objects with complex shapes and topological changes such as merging and splitting, thus able to deal with occluded objects and objects with holes. We propose Deep Extreme Level Set Evolution that combines powerful CNN models with level set optimization in an end-to-end fashion. Our method learns to predict evolution parameters conditioned on the image and evolves the predicted initial contour to produce the final result. We make our model interactive by incorporating user clicks on the extreme boundary points, following DEXTR. We show that our approach significantly outperforms DEXTR on the static Cityscapes dataset and the video segmentation benchmark DAVIS, and performs on par on PASCAL and SBD.
Adaptive Pyramid Context Network for Semantic Segmentation
Authors: Junjun He,  Zhongying Deng,  Lei Zhou,  Yali Wang,  Yu Qiao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Scene Parsing, Segmentation, Semantic Segmentation, Thermal Image Segmentation

Recent studies witnessed that context features can significantly improve the performance of deep semantic segmentation networks. Current context based segmentation methods differ with each other in how to construct context features and perform differently in practice. This paper firstly introduces three desirable properties of context features in segmentation task. Specially, we find that Global-guided Local Affinity (GLA) can play a vital role in constructing effective context features, while this property has been largely ignored in previous works. Based on this analysis, this paper proposes Adaptive Pyramid Context Network (APCNet) for semantic segmentation. APCNet adaptively constructs multi-scale contextual representations with multiple well-designed Adaptive Context Modules (ACMs). Specifically, each ACM leverages a global image representation as a guidance to estimate the local affinity coefficients for each sub-region, and then calculates a context vector with these affinities. We empirically evaluate our APCNet on three semantic segmentation and scene parsing datasets, including PASCAL VOC 2012, Pascal-Context, and ADE20K dataset.  Experimental results show that APCNet achieves state-of-the-art performance on all three benchmarks, and obtains a new record 84.2% on PASCAL VOC 2012 test set without MS COCO pre-trained and any post-processing.
Audio Visual Scene-Aware Dialog
Authors: Huda Alamri,  Vincent Cartillier,  Abhishek Das,  Jue Wang,  Anoop Cherian,  Irfan Essa,  Dhruv Batra,  Tim K. Marks,  Chiori Hori,  Peter Anderson,  Stefan Lee,  Devi Parikh
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

We introduce the task of scene-aware dialog. Our goal is to generate a complete and natural response to a question about a scene, given video and audio of the scene and the history of previous turns in the dialog. To answer successfully, agents must ground concepts from the question in the video while leveraging contextual cues from the dialog history. To benchmark this task, we introduce the Audio Visual Scene-Aware Dialog (AVSD) Dataset. For each of more than 11,000 videos of human actions from the Charades dataset, our dataset contains a dialog about the video, plus a final summary of the video by one of the dialog participants. We train several baseline systems for this task and evaluate the performance of the trained models using both qualitative and quantitative metrics. Our results indicate that models must utilize all the available inputs (video, audio, question, and dialog history) to perform best on this dataset.
Learning to Minify Photometric Stereo
Authors: Junxuan Li,  Antonio Robles-Kelly,  Shaodi You,  Yasuyuki Matsushita
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Photometric stereo estimates the surface normal given a set of images acquired under different illumination conditions. To deal with diverse factors involved in the image formation process, recent photometric stereo methods demand a large number of images as input. We propose a method that can dramatically decrease the demands on the number of images by learning the most informative ones under different illumination conditions. To this end, we use a deep learning framework to automatically learn the critical illumination conditions required at input. Furthermore, we present an occlusion layer that can synthesize cast shadows, which effectively improves the estimation accuracy. We assess our method on challenging real-world conditions, where we outperform techniques elsewhere in the literature with a significantly reduced number of light conditions.
Reflective and Fluorescent Separation Under Narrow-Band Illumination
Authors: Koji Koyamatsu,  Daichi Hidaka,  Takahiro Okabe,  Hendrik P. A. Lensch
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

In this paper, we address the separation of reflective and fluorescent components in RGB images taken under narrow-band light sources such as LEDs. First, we show that the fluorescent color per pixel can be estimated from at least two images under different light source colors, because the observed color at a surface point is represented by a convex combination of the light source color and the illumination-invariant fluorescent color. Second, we propose a method for robustly estimating the fluorescent color via MAP estimation by taking the prior knowledge with respect to fluorescent colors into consideration. We conducted a number of experiments by using both synthetic and real images, and confirmed that our proposed method works better than the closely related state-of-the-art method and enables us to separate reflective and fluorescent components even from a single image. Furthermore, we demonstrate that our method is effective for applications such as image-based material editing and relighting.
Connecting the Dots: Learning Representations for Active Monocular Depth Estimation
Authors: Gernot Riegler,  Yiyi Liao,  Simon Donne,  Vladlen Koltun,  Andreas Geiger
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Depth Estimation, Depth Prediction, Monocular Depth Estimation

We propose a technique for depth estimation with a monocular structured-light camera, i.e., a calibrated stereo set-up with one camera and one laser projector. Instead of formulating the depth estimation via a correspondence search problem, we show that a simple convolutional architecture is sufficient for high-quality disparity estimates in this setting. As accurate ground-truth is hard to obtain, we train our model in a self-supervised fashion with a combination of photometric and geometric losses. Further, we demonstrate that the projected pattern of the structured light sensor can be reliably separated from the ambient information. This can then be used to improve depth boundaries in a weakly supervised fashion by modeling the joint statistics of image and depth edges. The model trained in this fashion compares favorably to the state-of-the-art on challenging synthetic and real-world datasets. In addition, we contribute a novel simulator, which allows to benchmark active depth prediction algorithms in controlled conditions.
Learning Non-Volumetric Depth Fusion Using Successive Reprojections
Authors: Simon Donne,  Andreas Geiger
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Reconstruction, Depth Estimation

Given a set of input views, multi-view stereopsis techniques estimate depth maps to represent the 3D reconstruction of the scene; these are fused into a single, consistent, reconstruction -- most often a point cloud. In this work we propose to learn an auto-regressive depth refinement directly from data. While deep learning has improved the accuracy and speed of depth estimation significantly, learned MVS techniques remain limited to the planesweeping paradigm. We refine a set of input depth maps by successively reprojecting information from neighbouring views to leverage multi-view constraints. Compared to learning-based volumetric fusion techniques, an image-based representation allows significantly more detailed reconstructions; compared to traditional point-based techniques, our method learns noise suppression and surface completion in a data-driven fashion. Due to the limited availability of high-quality reconstruction datasets with ground truth, we introduce two novel synthetic datasets to (pre-)train our network. Our approach is able to improve both the output depth maps and the reconstructed point cloud, for both learned and traditional depth estimation front-ends, on both synthetic and real data.
MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction
Authors: Hongwei Yi,  Chen Li,  Qiong Cao,  Xiaoyong Shen,  Sheng Li,  Guoping Wang,  Yu-Wing Tai
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Face Reconstruction, regression

We propose to address the face reconstruction in the wild by using a multi-metric regression network, MMFace, to align a 3D face morphable model (3DMM) to an input image. The key idea is to utilize a volumetric sub-network to estimate an intermediate geometry representation, and a parametric sub-network to regress the 3DMM parameters. Our parametric sub-network consists of identity loss, expression loss, and pose loss which greatly improves the aligned geometry details by incorporating high level loss functions directly defined in the 3DMM parametric spaces. Our high-quality reconstruction is robust under large variations of expressions, poses, illumination conditions, and even with large partial occlusions. We evaluate our method by comparing the performance with state-of-the-art approaches on latest 3D face dataset LS3D-W and Florence. We achieve significant improvements both quantitatively and qualitatively. Due to our high-quality reconstruction, our method can be easily extended to generate high-quality geometry sequences for video inputs.