pwc-archive/papers-with-abstracts huggingface dataset:
Joint Map and Symmetry Synchronization
Authors: Yifan Sun, Zhenxiao Liang, Xiangru Huang, Qi-Xing Huang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object

Most existing techniques in map computation (e.g., in the form of feature or dense correspondences) assume that the underlying map between an object pair is unique. This assumption, however, easily breaks when visual objects possess self-symmetries. In this paper, we study the problem of jointly optimizing self-symmetries and pair-wise maps among a collection of similar objects. We introduce a lifting map representation for encoding both symmetry groups and maps between symmetry groups. Based on this representation, we introduce a reweighted non-linear least square framework for joint symmetry and map synchronization. Experimental results show that this approach outperforms state-of-the-art methods for self-symmetry group extraction from a single object as well as joint map optimization among a object collection.
Semi-Supervised Generative Adversarial Hashing for Image Retrieval
Authors: Guan'an Wang, Qinghao Hu, Jian Cheng, Zeng-Guang Hou
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Hashing, Image Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity, Triplet

With explosive growth of image and video data on the Internet, hashing technique has been extensively studied for large-scale visual search. Benefiting from the advance of deep learning, deep hashing methods have achieved promising performance. However, those deep hashing models are usually trained with supervised information, which is rare and expensive in practice, especially class labels. In this paper, inspired by the idea of generative models and the minimax two-player game, we propose a novel semi-supervised generative adversarial hashing (SSGAH) approach. Firstly, we unify a generative model, a discriminative model and a deep hashing model in a framework for making use of triplet-wise information and unlabeled data. Secondly, we design novel structure of the generative model and the discriminative model to learn the distribution of triplet-wise information in a semi-supervised way. In addition, we propose a semi-supervised ranking loss and an adversary ranking loss to learn binary codes which preserve semantic similarity for both labeled data and unlabeled data. Finally, by optimizing the whole model in an adversary training way, the learned binary codes can capture better semantic information of all data. Extensive empirical evaluations on two widely-used benchmark datasets show that our proposed approach significantly outperforms state-of-the-art hashing methods.
Training Binary Weight Networks via Semi-Binary Decomposition
Authors: Qinghao Hu, Gang Li, Peisong Wang, Yifan Zhang, Jian Cheng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Computational Efficiency

Recently binary weight networks have attracted lots of attentions due to their high computational efficiency and small parameter size. Yet they still suffer from large accuracy drops because of their limited representation capacity.  In this paper, we propose a novel semi-binary decomposition method which decomposes a matrix into two binary matrices and a diagonal matrix. Since the matrix product of binary matrices has more numerical values than binary matrix,  the proposed semi-binary decomposition has more representation capacity.  Besides, we propose an alternating optimization method to solve the semi-binary decomposition problem while keeping binary constraints. Extensive experiments on AlexNet, ResNet-18, and ResNet-50 demonstrate that our method outperforms state-of-the-art methods by a large margin (5 percentage higher in top1 accuracy). We also implement binary weight AlexNet on FPGA platform, which shows that our proposed method can achieve $sim 9	imes$ speed-ups while reducing the consumption of on-chip memory and dedicated multipliers significantly.
Part-Activated Deep Reinforcement Learning for Action Prediction
Authors: Lei Chen, Jiwen Lu, Zhanjie Song, Jie zhou
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Reinforcement Learning, Prediction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we propose a part-activated deep reinforcement learning (PA-DRL) for action prediction. Most existing methods for action prediction utilize the evolution of whole frames to model actions, which cannot avoid the noise of the current action, especially in the early prediction. Moreover, the loss of structural information of human body diminishes the capacity of features to describe actions. To address this, we design a PA-DRL to exploit the structure of the human body by extracting skeleton proposals under a deep reinforcement learning framework. Specifically, we extract features from different parts of the human body individually and activate the action-related parts in features to enhance the representation. Our method not only exploits the structure information of the human body, but also considers the saliency part for expressing actions. We evaluate our method on three popular action prediction datasets: UT-Interaction, BIT-Interaction and UCF101. Our experimental results demonstrate that our method achieves the performance with state-of-the-arts.
Lifelong Learning via Progressive Distillation and Retrospection
Authors: Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, Dahua Lin
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Knowledge Distillation, Lifelong learning

Lifelong learning aims at adapting a learned model to new tasks while retaining the knowledge gained earlier. A key challenge for lifelong learning is how to strike a balance between the preservation on old tasks and the adaptation to a new one within a given model. Approaches that combine both objectives in training have been explored in previous works. Yet the performance still suffers from considerable degradation in a long sequence of tasks. In this work, we propose a novel approach to lifelong learning, which tries to seek a better balance between preservation and adaptation via two techniques: Distillation and Retrospection. Specifically, the target model adapts to the new task by knowledge distillation from an intermediate expert, while the previous knowledge is more effectively preserved by caching a small subset of data for old tasks. The combination of Distillation and Retrospection leads to a more gentle learning curve for the target model, and extensive experiments demonstrate that our approach can bring consistent improvements on both old and new tasks.
Focus, Segment and Erase: An Efficient Network for Multi-Label Brain Tumor Segmentation
Authors: Xuan Chen, Jun Hao Liew, Wei Xiong, Chee-Kong Chui, Sim-Heng Ong
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Brain Tumor Segmentation, Segmentation, Tumor Segmentation

In multi-label brain tumor segmentation, class imbalance and inter-class interference are common and challenging problems. In this paper, we propose a novel end-to-end trainable network named FSENet to address the aforementioned issues. The proposed FSENet has a tumor region pooling component to restrict the prediction within the tumor region (``focus"), thus mitigating the influence of the dominant non-tumor region. Furthermore, the network decomposes the more challenging multi-label brain tumor segmentation problem into several simpler binary segmentation tasks (``segment"), where each task focuses on a specific tumor tissue. To alleviate inter-class interference, we adopt a simple yet effective idea in our work: we erase the segmented regions before proceeding to further segmentation of tumor tissue (``erase"), thus reduces competition among different tumor classes. Our single-model FSENet ranks 3rd on the multi-modal brain tumor segmentation benchmark 2015 (BraTS 2015) without relying on ensembles or complicated post-processing steps.
3D Recurrent Neural Networks with Context Fusion for Point Cloud Semantic Segmentation
Authors: Xiaoqing Ye, Jiamao Li, Hexiao Huang, Liang Du, Xiaolin Zhang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Segmentation, Semantic Segmentation

Semantic segmentation of 3D unstructured point clouds remains an open research problem. Recent works predict semantic labels of 3D points by virtue of neural networks but take limited context knowledge into consideration. In this paper, a novel end-to-end approach for unstructured point cloud semantic segmentation is proposed to exploit the inherent contextual features. First the efficient pointwise pyramid pooling module is investigated to capture local structures at various densities by taking multi-scale neighborhood into account. Then the two-dimensional hierarchical recurrent neural networks (RNNs) are utilized to explore long-range spatial dependencies. Each recurrent layer takes as input the local features derived from unrolled cells and sweeps the 3D space along two horizontal directions successively to integrate structure knowledge. On challenging indoor and outdoor 3D datasets, the proposed framework demonstrates robust performance superior to state-of-the-arts.
Transductive Semi-Supervised Deep Learning using Min-Max Features
Authors: Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng MaXiaoyu Tao, Nanning Zheng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning, General Classification, image-classification, Image Classification, Transductive Learning

In this paper, we propose Transductive Semi-Supervised Deep Learning (TSSDL) method that is effective for training Deep Convolutional Neural Network (DCNN) models. The method applies transductive learning principle to DCNN training, introduces confidence levels on unlabeled image samples to overcome unreliable label estimates on outliers and uncertain samples, and develops the Min-Max Feature (MMF) regularization that encourages DCNN to learn feature descriptors with better between-class separability and within-class compactness. TSSDL method is independent of any DCNN architectures and complementary to the latest Semi-Supervised Learning (SSL) methods. Comprehensive experiments on the benchmark datasets CIFAR10 and SVHN have shown that the DCNN model trained by the proposed TSSDL method can produce image classification accuracies compatible to the state-of-the-art SSL methods, and that combining TSSDL with the Mean Teacher method can produce the best classification accuracies on the two benchmark datasets.
Visual Tracking via Spatially Aligned Correlation Filters Network
Authors: Mengdan Zhang, Qiang Wang, Junliang Xing, Jin Gao, Peixi Peng, Weiming Hu, Steve Maybank
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Visual Tracking

Correlation filters based trackers rely on a periodic assumption of the search sample to efficiently distinguish the target from the background. This assumption however yields undesired boundary effects and restricts aspect ratios of search samples. To handle these issues, an end-to-end deep architecture is proposed to incorporate geometric transformations into a correlation filters based network. This architecture introduces a novel spatial alignment module, which provides continuous feedback for transforming the target from the border to the center with a normalized aspect ratio. It enables correlation filters to work on well-aligned samples for better tracking. The whole architecture not only learns a generic relationship between object geometric transformations and object appearances, but also learns robust representations coupled to correlation filters in case of various geometric transformations. This lightweight architecture permits real-time speed. Experiments show our tracker effectively handles boundary effects and aspect ratio variations, achieving state-of-the-art tracking results on three benchmarks.
Online Dictionary Learning for Approximate Archetypal Analysis
Authors: Jieru Mei, Chunyu Wang, Wen-Jun Zeng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Dictionary Learning

Archetypal analysis is an unsupervised learning approach which represents data by convex combinations of a set of archetypes. The archetypes generally correspond to the extremal points in the dataset and are learned by requiring them to be convex combinations of the training data. In spite of its nice property of interpretability, the method is slow. We propose a variant of archetypal analysis which scales gracefully to large datasets. The core idea is to decouple the binding between data and archetypes and require them to be unit normalized. Geometrically, the method learns a convex hull inside the unit sphere and represents the data by their projections on the closest surfaces of the convex hull. By minimizing the representation error, the method pushes the convex hull surfaces close to the regions of the sphere where the data reside. The vertices of the convex hull are the learned archetypes. We apply the method to human faces and poses to validate its effectiveness in the context of reconstructions and classifications.
Video Object Segmentation by Learning Location-Sensitive Embeddings
Authors: Hai Ci, Chunyu Wang, Yizhou Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation

We address the problem of video object segmentation which outputs the masks of a target object throughout a video given only a bounding box in the first frame. There are two main challenges for this task. First, the background may contain similar objects as the target. Second, the appearance of the target object may change drastically over time. To tackle these challenges, we propose an end-to-end training network which accomplishes foreground predictions by leveraging the location-sensitive embeddings which are capable to distinguish the pixels of similar objects. To deal with appearance changes, for a test video, we propose a robust model adaptation method which pre-scans the whole video, generates pseudo foreground/background labels and retrains the model based on the labels. Our method outperforms the state-of-the-art methods on the DAVIS and the SegTrack V2 datasets.
Learning to Capture Light Fields through a Coded Aperture Camera
Authors: Yasutaka Inagaki, Yuto Kobayashi, Keita Takahashi, Toshiaki Fujii, Hajime Nagahara
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We propose a learning-based framework for acquiring a light field through a coded aperture camera. Acquiring a light field is a challenging task due to the amount of data. To make the acquisition process efficient, coded aperture cameras were successfully adopted; using these cameras, a light field is computationally reconstructed from several images that are acquired with different aperture patterns. However, it is still difficult to reconstruct a high-quality light field from only a few acquired images. To tackle this limitation, we formulated the entire pipeline of light field acquisition from the perspective of an auto-encoder. This auto-encoder was implemented as a stack of fully convolutional layers and was trained end-to-end by using a collection of training samples. We experimentally show that our method can successfully learn good image-acquisition and reconstruction strategies. With our method, light fields consisting of 5 x 5 or 8 x 8 images can be successfully reconstructed only from a few acquired images. Moreover, our method achieved superior performance over several state-of-the-art methods. We also applied our method to a real prototype camera to show that it is capable of capturing a real 3-D scene.
Learning to Reconstruct High-quality 3D Shapes with Cascaded Fully Convolutional Networks
Authors: Yan-Pei Cao, Zheng-Ning Liu, Zheng-Fei Kuang, Leif Kobbelt, Shi-Min Hu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We present a data-driven approach to reconstructing high-resolution and detailed volumetric representations of 3D shapes. Although well studied, algorithms for volumetric fusion from multi-view depth scans are still prone to scanning noise and occlusions, making it hard to obtain high-fidelity 3D reconstructions. In this paper, inspired by recent advances in efficient 3D deep learning techniques, we introduce a novel cascaded 3D convolutional network architecture, which learns to reconstruct implicit surface representations from noisy and incomplete depth maps in a progressive, coarse-to-fine manner. To this end, we also develop an algorithm for end-to-end training of the proposed cascaded structure. Qualitative and quantitative experimental results on both simulated and real-world datasets demonstrate that the presented approach outperforms existing state-of-the-art work in terms of quality and fidelity of reconstructed models.
X2Face: A network for controlling face generation using images, audio, and pose codes
Authors: Olivia Wiles, A. Sophia Koepke, Andrew Zisserman
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Generation, Talking Head Generation

 The objective of this paper is a neural network model that controls the pose and expression of a given face, using  another face or modality (e.g. audio).   This model can then be used for lightweight, sophisticated video and image editing.  We make the following three contributions. First,  we introduce a network, X2Face, that can  control a source face (specified by one or more frames)  using  another face in a driving frame to produce a generated frame with the identity of the source frame but the pose and expression of the face in the driving frame.   Second, we propose  a method for training the network fully self-supervised using a large collection of video data.  Third,  we show that  the generation process can be driven by other modalities, such as audio or pose codes,  without any further training of the network.    The generation results for driving a face with another face are compared to state-of-the-art self-supervised/supervised methods.  We show that our approach is more robust than other methods, as it makes fewer  assumptions about the input data. We also show examples of using our framework for video face editing.
Revisiting Autofocus for Smartphone Cameras
Authors: Abdullah Abuolaim, Abhijith Punnappurath, Michael S. Brown
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Autofocus (AF) on smartphones is the process of determining how to move a camera's lens such that certain scene content is in focus.   The underlying algorithms used by AF systems, such as contrast detection and phase differencing, are well established. However, determining a high-level objective regarding how to best focus a particular scene is less clear.  This is evident in part by the fact that different smartphone cameras employ different AF criteria; for example, some attempt to keep items in the center in focus, others give priority to faces while others maximize the sharpness of the entire scene.  The fact that different objectives exist raises the research question of whether there is a preferred objective.  This becomes more interesting when AF is applied to videos of dynamic scenes.  The work in this paper aims to revisit AF for smartphones within the context of temporal image data.  As part of this effort, we describe the capture of a new 4D dataset that provides access to a full focal stack at each time point in a temporal sequence.  Based on this dataset, we have developed a platform and associated application programming interface (API) that mimic real AF systems, restricting lens motion within the constraints of a dynamic environment and frame capture. Using our platform we evaluated several high-level focusing objectives and found interesting insight into what users prefer.  We believe our new temporal focal stack dataset, AF platform, and initial user-study findings will be useful in advancing AF research.
A Dataset of Flash and Ambient Illumination Pairs from the Crowd
Authors: Yagiz Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys, Wojciech Matusik
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Illumination is a critical element of photography and is essential for many computer vision tasks. Flash light is unique in the sense that it is a widely available tool for easily manipulating the scene illumination. We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations. Different than the typical use of crowdsourcing in generating computer vision datasets, we make use of the crowd to directly take the photographs that make up our dataset. As a result, our dataset covers a wide variety of scenes captured by many casual photographers. We detail the advantages and challenges of our approach to crowdsourcing as well as the computational effort to generate completely separate flash illuminations from the ambient light in an uncontrolled setup. We present a brief examination of illumination decomposition, a challenging and underconstrained problem in flash photography, to demonstrate the use of our dataset in a data-driven approach.
ISNN: Impact Sound Neural Network for Audio-Visual Object Classification
Authors: Auston Sterling, Justin Wilson, Sam Lowe, Ming C. Lin
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Classification, General Classification, Object

3D object geometry reconstruction remains a challenge when working with transparent, occluded, or highly reflective surfaces. While recent methods classify shape features using raw audio, we present a multimodal neural network optimized for estimating an object's geometry and material. Our networks use spectrograms of recorded and synthesized object impact sounds and voxelized shape estimates to extend the capabilities of vision-based reconstruction. We evaluate our method on multiple datasets of both recorded and synthesized sounds. We further present an interactive application for real-time scene reconstruction in which a user can strike objects, producing sound that can instantly classify and segment the struck object, even if the object is transparent or visually occluded.
Compositing-aware Image Search
Authors: Hengshuang Zhao, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Brian Price, Jiaya Jia
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Retrieval, Object

We present a new image search technique that, given a background image, returns compatible foreground objects for image compositing tasks. The compatibility of a foreground object and a background scene depends on various aspects such as semantics, surrounding context, geometry, style and color. However, existing image search techniques measure the similarities on only a few aspects, and may return many results that are not suitable for compositing. Moreover, the importance of each factor may vary for different object categories and image content, making it difficult to manually define the matching criteria. In this paper, we propose to learn feature representations for foreground objects and background scenes respectively, where image content and object category information are jointly encoded during training. As a result, the learned features can adaptively encode the most important compatibility factors. We project the features to a common embedding space, so that the compatibility scores can be easily measured using the cosine similarity, enabling very efficient search. We collect an evaluation set consisting of eight object categories commonly used in compositing tasks, on which we demonstrate that our approach significantly outperforms other search techniques.