pwc-archive/papers-with-abstracts huggingface dataset:
Neural Procedural Reconstruction for Residential Buildings
Authors: Huayi Zeng, Jiaye Wu, Yasutaka Furukawa
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Reconstruction

This paper proposes a novel 3D reconstruction approach, dubbed Neural Procedural Reconstruction (NPR), which trains deep neural networks to procedurally apply shape grammar rules and reconstruct CAD-quality models from 3D points. In contrast to Procedural Modeling (PM), which randomly applies shape grammar rules to synthesize 3D models, NPR classifies a rule branch to explore and regresses geometric parameters at each rule application. We demonstrate the proposed system for residential buildings with aerial LiDAR as the input. Our 3D models boast extremely compact geometry and semantically segmented architectural components. Qualitative and quantitative evaluations on hundreds of houses show that our approach robustly generates CAD-quality 3D models from raw sensor data, making significant improvements over the existing state-of-the-art.
View-graph Selection Framework for SfM
Authors: Rajvi Shah, Visesh Chari, P. J. Narayanan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Model Selection

View-graph is an essential input to large-scale structure from motion (SfM) pipelines. Accuracy and efficiency of large-scale SfM is crucially dependent on the input view-graph. Inconsistent or inaccurate edges can lead to inferior or wrong reconstruction. Most SfM methods remove `undesirable' images and pairs using several fixed heuristic criteria, and propose tailor-made solutions to achieve specific reconstruction objectives such as efficiency, accuracy, or disambiguation. In contrast to these disparate solutions, we propose an optimization based formulation that can be used to achieve these different reconstruction objectives with task-specific cost modeling that uses and construct a very efficient network flow based formulation for its approximate solution. The abstraction brought on by this selection mechanism separates the challenges specific to datasets and reconstruction objectives from the standard SfM pipeline and improves its generalization. This paper mainly focuses on application of this framework with standard SfM pipeline for accurate and ghost-free reconstructions of highly ambiguous datasets. To model selection costs for this task, we introduce new disambiguation priors based on local geometry. We further demonstrate versatility of the method by using it for the general objective of accurate and efficient reconstruction of large-scale Internet datasets using costs based on well-known SfM priors.
DFT-based Transformation Invariant Pooling Layer for Visual Classification
Authors: Jongbin Ryu, Ming-Hsuan Yang, Jongwoo Lim
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Classification, General Classification, Translation

We propose a novel discrete Fourier transform-based pooling layer for convolutional neural networks. The DFT magnitude pooling replaces the traditional max/average pooling layer between the convolution and fully-connected layers to retain translation invariance and shape preserving (aware of shape difference) properties based on the shift theorem of the Fourier transform. Thanks to the ability to handle image misalignment while keeping important structural information in the pooling stage, the DFT magnitude pooling improves the classification accuracy significantly. In addition, we propose the DFT+ method for ensemble networks using the middle convolution layer outputs. The proposed methods are extensively evaluated on various classification tasks using the ImageNet, CUB 2010-2011, MIT Indoors, Caltech 101, FMD and DTD datasets. The AlexNet, VGG-VD 16, Inception-v3, and ResNet are used as the base networks, upon which DFT and DFT+ methods are implemented. Experimental results show that the proposed methods improve the classification performance in all networks and datasets.
Learning Compression from Limited Unlabeled Data
Authors: Xiangyu He,  Jian Cheng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: CPU, GPU, Model Compression, Quantization

Convolutional neural networks (CNNs) have dramatically advanced the state-of-art in a number of domains. However, most models are both computation and memory intensive, which arouse the interest of network compression. While existing compression methods achieve good performance, they suffer from three limitations: 1) the inevitable retraining with enormous labeled data; 2) the massive GPU hours for retraining; 3) the training tricks for model compression. Especially the requirement of retraining on original datasets makes it difficult to apply in many real-world scenarios, where training data is not publicly available. In this paper, we reveal that re-normalization is the practical and effective way to alleviate the above limitations. Through quantization or pruning, most methods may compress a large number of parameters but ignore the core role in performance degradation, which is the Gaussian conjugate prior induced by batch normalization. By employing the re-estimated statistics in batch normalization, we significantly improve the accuracy of compressed CNNs. Extensive experiments on ImageNet show it outperforms baselines by a large margin and is comparable to label-based methods. Besides, the fine-tuning process takes less than 5 minutes on CPU, using 1000 unlabeled images.
3D Vehicle Trajectory Reconstruction in Monocular Video Data Using Environment Structure Constraints
Authors: Sebastian Bullinger, Christoph Bodensteiner, Michael Arens, Rainer Stiefelhagen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Optical Flow Estimation, Semantic Segmentation

We present a framework to reconstruct three-dimensional vehicle trajectories using monocular video data. We track two-dimensional vehicle shapes on pixel level exploiting instance-aware semantic segmentation techniques and optical flow cues. We apply Structure from Motion techniques to vehicle and background images to determine for each frame camera poses relative to vehicle instances and background structures. By combining vehicle and background camera pose information, we restrict the vehicle trajectory to a one-parameter family of possible solutions. We compute a ground representation by fusing background structures and corresponding semantic segmentations. We propose a novel method to determine vehicle trajectories consistent to image observations and reconstructed environment structures as well as a criterion to identify frames suitable for scale ratio estimation. We show qualitative results using drone imagery as well as driving sequences from the Cityscape dataset. Due to the lack of suitable benchmark datasets we present a new dataset to evaluate the quality of reconstructed three-dimensional vehicle trajectories. The video sequences show vehicles in urban areas and are rendered using the path-tracing render engine Cycles. In contrast to previous work, we perform a quantitative evaluation of the presented approach. Our algorithm achieves an average reconstruction-to-ground-truth-trajectory distance of 0.31 meter using this dataset. The dataset including evaluation scripts will be publicly available on our website.
Appearance-Based Gaze Estimation via Evaluation-Guided Asymmetric Regression
Authors: Yihua Cheng, Feng Lu, Xucong Zhang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Gaze Estimation, regression

Eye gaze estimation has been increasingly demanded by recent intelligent systems to accomplish a range of interaction-related tasks, by using simple eye images as input. However, learning the highly complex regression between eye images and gaze directions is nontrivial, and thus the problem is yet to be solved efficiently. In this paper, we propose the Asymmetric Regression-Evaluation Network (ARE-Net), and try to improve the gaze estimation performance to its full extent. At the core of our method is the notion of ``two eye asymmetry'' observed during gaze estimation for the left and right eyes. Inspired by this, we design the multi-stream ARE-Net; one asymmetric regression network (AR-Net) predicts 3D gaze directions for both eyes with a novel asymmetric strategy, and the evaluation network (E-Net) adaptively adjusts the strategy by evaluating the two eyes in terms of their performance during optimization. By training the whole network, our method achieves promising results and surpasses the state-of-the-art methods on multiple public datasets.
Deep Attention Neural Tensor Network for Visual Question Answering
Authors: Yalong Bai, Jianlong Fu, Tiejun Zhao, Tao Mei
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Attention, Question Answering, Triplet, Visual Question Answering, Visual Question Answering (VQA)

Visual question answering (VQA) has drawn great attention in cross-modal learning problems, which enables a machine to answer a natural language question given a reference image. Significant progress has been made by learning rich embedding features from images and questions by bilinear models, while neglects the key role from answers. In this paper, we propose a novel deep attention neural tensor network (DA-NTN) for visual question answering, which can discover the joint correlations over images, questions and answers with tensor-based representations. First, we model one of the pairwise interaction (e.g., image and question) by bilinear features, which is further encoded with the third dimension (e.g., answer) to be a triplet by bilinear tensor product. Second, we decompose the correlation of different triplets by different answer and question types, and further propose a slice-wise attention module on tensor to select the most discriminative reasoning process for inference. Third, we optimize the proposed DA-NTN by learning a label regression with KL-divergence losses. Such a design enables scalable training and fast convergence over a large number of answer set. We integrate the proposed DA-NTN structure into the state-of-the-art VQA models (e.g., MLB and MUTAN). Extensive experiments demonstrate the superior accuracy than the original MLB and MUTAN models, with 1.98%, 1.70% relative increases on VQA-2.0 dataset, respectively.
Normalized Blind Deconvolution
Authors: Meiguang Jin, Stefan Roth, Paolo Favaro
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We introduce a family of novel approaches to single-image blind deconvolution, ie , the problem of recovering a sharp image and a blur kernel from a single blurry input. This problem is highly ill-posed, because infinite (image, blur) pairs produce the same blurry image. Most research effort has been devoted to the design of priors for natural images and blur kernels, which can drastically prune the set of possible solutions. Unfortunately, these priors are usually not sufficient to favor the sharp solution.  In this paper we address this issue by looking at a much less studied aspect: the relative scale ambiguity between the sharp image and the blur.  Most prior work eliminates this ambiguity by fixing the $L^1$ norm of the blur kernel. In principle, however, this choice is arbitrary. We show that a careful design of the blur normalization yields a blind deconvolution formulation with remarkable accuracy and robustness to noise.  Specifically, we show that using the Frobenius norm to fix the scale ambiguity enables convex image priors, such as the total variation, to achieve state-of-the-art performance on both synthetic and real datasets.
Selfie Video Stabilization
Authors: Jiyang Yu, Ravi Ramamoorthi
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Model, Optical Flow Estimation, Video Stabilization

We propose a novel algorithm for stabilizing selfie videos. Our goal is to automatically generate stabilized video that has optimal smooth motion in the sense of both foreground and background. The key insight is that non-rigid foreground motion in selfie videos can be analyzed using a 3D face model, and background motion can be analyzed using optical flow. We use second derivative of temporal trajectory of selected pixels as the measure of smoothness. Our algorithm stabilizes selfie videos by minimizing the smoothness measure of the back-ground, regularized by the motion of the foreground. Experiments show that our method outperforms state-of-the-art general video stabilization techniques in selfie videos.
Simultaneous 3D Reconstruction for Water Surface and Underwater Scene
Authors: Yiming Qian, Yinqiang Zheng, Minglun Gong, Yee-Hong Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Reconstruction, Optical Flow Estimation

This paper presents the first approach for simultaneously recovering the 3D shape of both the wavy water surface and the moving underwater scene. A portable camera array system is constructed, which captures the scene from multiple viewpoints above the water. The correspondences across these cameras are estimated using an optical flow method and are used to infer the shape of the water surface and the underwater scene. We assume that there is only one refraction occurring at the water interface. Under this assumption, two estimates of the water surface normals should agree: one from Snell's law of light refraction and another from local surface structure. The experimental results using both synthetic and real data demonstrate the effectiveness of the presented approach.
Proximal Dehaze-Net: A Prior Learning-Based Deep Network for Single Image Dehazing
Authors:  Dong Yang, Jian Sun
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning, Image Dehazing, Single Image Dehazing

Photos taken in hazy weather are usually covered with white masks and often lose important details. In this paper, we propose a novel deep learning approach for single image dehazing by learning dark channel and transmission priors. First, we build an energy model for dehazing using dark channel and transmission priors and design an iterative optimization algorithm using proximal operators for these two priors. Second, we unfold the iterative algorithm to be a deep network, dubbed as 	extit{proximal dehaze-net}, by learning the proximal operators using convolutional neural networks. Our network combines the advantages of  traditional prior-based dehazing methods and deep learning methods by incorporating haze-related prior learning into deep network. Experiments show that our method achieves state-of-the-art  performance for single image dehazing.
SDC-Net: Video prediction using spatially-displaced convolution
Authors: Fitsum A. Reda, Guilin Liu, Kevin J. Shih, Robert Kirby, Jon Barker, David Tarjan, Andrew Tao, Bryan Catanzaro
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Optical Flow Estimation, Prediction, SSIM, Video Prediction

We present an approach for high-resolution video frame prediction by conditioning on both past frames and past optical flows. Previous approaches rely on resampling past frames, guided by a learned future optical flow, or on direct generation of pixels. Resampling based on flow is insufficient because it cannot deal with disocclusions. Generative models currently lead to blurry results. Recent approaches synthesis a pixel by convolving input patches with a predicted kernel. However, their memory requirement increases with kernel size. Here, we present spatially-displaced convolution (SDC) module for video frame prediction. We learn a motion vector and a kernel for each pixel and synthesize a pixel by applying the kernel at a displaced location in the source image, defined by the predicted motion vector. Our approach inherits the merits of both vector-based and kernel-based approaches, while ameliorating their respective disadvantages. We train our model on 428K unlabelled 1080p video game frames. Our approach produces state-of-the-art results, achieving an SSIM score of 0.904 on high-definition YouTube-8M videos, 0.918 on Caltech Pedestrian videos. Our model handles large motion effectively and synthesizes crisp frames with consistent motion.
Exploiting temporal information for 3D human pose estimation
Authors: Mir Rayat Imtiaz Hossain, James J. Little
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation

In this work, we address the problem of 3D human pose estimation from a sequence of 2D human poses. Although the recent success of deep networks has led many state-of-the-art methods for 3D pose estimation to train deep networks end-to-end to predict from images directly, the top-performing approaches have shown the effectiveness of dividing the task of 3D pose estimation into two steps: using a state-of-the-art 2D pose estimator to estimate the 2D pose from images and then mapping them into 3D space. They also showed that a low-dimensional representation like 2D locations of a set of joints can be discriminative enough to estimate 3D pose with high accuracy. However, estimation of 3D pose for individual frames leads to temporally incoherent estimates due to independent error in each frame causing jitter. Therefore, in this work we utilize the temporal information across a sequence of 2D joint locations to estimate a sequence of 3D poses. We designed a sequence-to-sequence network composed of layer-normalized LSTM units with shortcut connections connecting the input to the output on the decoder side and imposed temporal smoothness constraint during training. We found that the knowledge of temporal consistency improves the best reported result on Human3.6M dataset by approximately $12.2%$ and helps our network to recover temporally consistent 3D poses over a sequence of images even when the 2D pose detector fails.
Joint Camera Spectral Sensitivity Selection and Hyperspectral Image Recovery
Authors: Ying Fu, Tao Zhang, Yinqiang Zheng, Debing Zhang, Hua Huang 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Sensitivity

 Hyperspectral image (HSI) recovery from a single RGB image has attracted much   attention, whose performance has recently been shown to be sensitive to the   camera spectral sensitivity (CSS). In this paper, we present an efficient   convolutional neural network (CNN) based method, which can jointly select the   optimal CSS from a candidate dataset and learn a mapping to recover HSI from a   single RGB image captured with this algorithmically selected camera. Given a   specific CSS, we first present a HSI recovery network, which accounts for the   underlying characteristics of the HSI, including spectral nonlinear mapping and spatial similarity. Later, we append a CSS selection layer onto the recovery network, and the optimal CSS can thus be automatically determined from the network weights under the nonnegative sparse constraint. Experimental results show that our HSI recovery network outperforms state-of-the-art methods in terms of both quantitative metrics and perceptive quality, and the selection layer always returns a CSS consistent to the best one determined by exhaustive search.
Weakly-supervised Video Summarization using Variational Encoder-Decoder and Web Prior
Authors: Sijia Cai, WangMeng Zuo, Larry S. Davis, Lei Zhang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Decoder, Saliency Prediction, Supervised Video Summarization, Video Summarization

Video summarization is a challenging under-constrained problem because the underlying summary of a single video strongly depends on users' subjective understandings. Data-driven approaches, such as deep neural networks, can deal with the ambiguity inherent in this task to some extent, but it is extremely expensive to acquire the temporal annotations of a large-scale video dataset. To leverage the plentiful web-crawled videos to improve the performance of video summarization, we present a generative modelling framework to learn the latent semantic video representations to bridge the benchmark data and web data. Specifically, our framework couples two important components: a variational autoencoder for learning the latent semantics from web videos, and an encoder-attention-decoder for saliency estimation of raw video and summary generation. A loss term to learn the semantic matching between the generated summaries and web videos is presented, and the overall framework is further formulated into a unified conditional variational encoder-decoder, called variational encoder-summarizer-decoder (VESD). Experiments conducted on the challenging datasets CoSum and TVSum demonstrate the superior performance of the proposed VESD to existing state-of-the-art methods. The source code of this work can be found at https://github.com/cssjcai/vesd.
Compound Memory Networks for Few-shot Video Classification
Authors: Linchao Zhu, Yi Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Classification, General Classification, Video Classification

In this paper, we propose a new memory network structure for few-shot video classification by making the following contributions. First, we propose a compound memory network (CMN) structure under the key-value memory network paradigm, in which each key memory involves multiple constituent keys. These constituent keys work collaboratively for training, which enables the CMN to obtain an optimal video representation in a larger space. Second, we introduce a multi-saliency embedding algorithm which encodes a variable-length video sequence into a fixed-size matrix representation by discovering multiple saliencies of interest.  For example,  given a video of car auction, some people are interested in the car,  while others are interested in the auction activities. Third,  we design an abstract memory on top of the constituent keys. The abstract memory and constituent keys  form a layered structure, which makes the CMN more efficient and capable of being scaled, while also retaining the representation capability of the multiple keys. We compare CMN with several state-of-the-art baselines on a new few-shot video classification dataset and show the effectiveness of our approach.
Polarimetric Three-View Geometry
Authors: Lixiong Chen, Yinqiang Zheng, Art Subpa-Asa, Imari Sato
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Translation

This paper theorizes the connection between polarization and three-view geometry. It presents a ubiquitous polarization-induced constraint that regulates the relative pose of a system of three cameras. We demonstrate that, in a multi-view system, the polarization phase obtained for a surface point is induced from one of the two pencils of planes: one by specular reflections with its axis aligned with the incident light; one by diffusive reflections with its axis aligned with the surface normal. Differing from the traditional three-view geometry, we show that this constraint directly encodes camera rotation and projection, and is independent of camera translation. In theory, six polarized diffusive point-point-point correspondences suffice to determine the camera rotations. In practise, a cross-validation mechanism using correspondences of specularites can effectively resolve the ambiguities caused by mixed polarization. The experiments on real world scenes validate our proposed theory.