pwc-archive/papers-with-abstracts huggingface dataset:
AutoLoc: Weakly-supervised Temporal Action Localization in Untrimmed Videos
Authors: Zheng Shou, Hang Gao, Lei Zhang, Kazuyuki Miyazawa, Shih-Fu Chang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Localization, Temporal Action Localization, Weakly Supervised Action Localization, Weakly-supervised Temporal Action Localization

Temporal Action Localization (TAL) in untrimmed video is important for many applications. But it is very expensive to annotate the segment-level ground truth (action class and temporal boundary). This raises the interest of addressing TAL with weak supervision, namely only video-level annotations are available during training). However, the state-of-the-art weakly-supervised TAL methods only focus on generating good Class Activation Sequence (CAS) over time but conduct simple thresholding on CAS to localize actions. In this paper, we first develop a novel weakly-supervised TAL framework called AutoLoc to directly predict the temporal boundary of each action instance. We propose a novel Outer-Inner-Contrastive (OIC) loss to automatically discover the needed segment-level supervision for training such a boundary predictor. Our method achieves dramatically improved performance: under the IoU threshold 0.5, our method improves mAP on THUMOS'14 from 13.7% to 21.2% and mAP on ActivityNet from 7.4% to 27.3%. It is also very encouraging to see that our weakly-supervised method achieves comparable results with some fully-supervised methods.
Practical Black-box Attacks on Deep Neural Networks using Efficient Query Mechanisms
Authors: Arjun Nitin Bhagoji, Warren He, Bo Li, Dawn Song
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Existing black-box attacks on deep neural networks (DNNs) have largely focused on transferability, where an adversarial instance generated for a locally trained model can âtransferâ to attack other learning models. In this paper, we propose novel Gradient Estimation black-box attacks for adversaries with query access to the target modelâs class probabilities, which do not rely on transferability. We also propose strategies to decouple the number of queries required to generate each adversarial sample from the dimensionality of the input. An iterative variant of our attack achieves close to 100% attack success rates for both targeted and untargeted attacks on DNNs. We carry out a thorough comparative evaluation of black-box attacks and show that Gradient Estimation attacks achieve attack success rates similar to state-of-the-art white-box attacks on the MNIST and CIFAR-10 datasets. We also apply the Gradient Estimation attacks successfully against real-world classiï¬ers hosted by Clarifai. Further, we evaluate black-box attacks against state-of-the-art defenses based on adversarial training and show that the Gradient Estimation attacks are very eï¬ective even against these defenses.
Context Refinement for Object Detection
Authors: Zhe Chen, Shaoli Huang, DaCheng Tao
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, object-detection, Object Detection, Region Proposal

Current two-stage object detectors, which consists of a region proposal stage and a refinement stage, may produce unreliable results due to ill-localized proposed regions. To address this problem, we propose a context refinement algorithm that explores rich contextual information to better refine each proposed region. In particular, we first identify neighboring regions that may contain useful contexts and then perform refinement based on the extracted and unified contextual information. In practice, our method effectively improves the quality of the final detection results as well as region proposals. Empirical studies show that context refinement yields substantial and consistent improvements over different baseline detectors. Moreover, the proposed algorithm brings around 3% performance gain on PASCAL VOC benchmark and around 6% gain on MS COCO benchmark respectively.
Exploiting Vector Fields for Geometric Rectification of Distorted Document Images
Authors: Gaofeng MENG, Yuanqi SU, Ying Wu, Shiming Xiang, Chunhong Pan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

This paper proposes a segment-free method for geometric rectification of a distorted document image captured by a hand-held camera. The method can recover the 3D page shape by exploiting the intrinsic vector fields of the image. Based on the assumption that the curled page shape is a general cylindrical surface, we estimate the parameters related to the camera and the 3D shape model through weighted majority voting on the vector fields. Then the spatial directrix of the surface is recovered by solving an ordinary differential equation (ODE) through the Euler method. Finally, the geometric distortions in images can be rectified by flattening the estimated 3D page surface onto a plane. Our method can exploit diverse types of visual cues available in a distorted document image to estimate its vector fields for 3D page shape recovery. In comparison to the state-of-the-art methods, the great advantage is that it is a segment-free method and does not have to extract curved text lines or textual blocks, which is still a very challenging problem especially for a distorted document image. Our method can therefore be freely applied to document images with extremely complicated page layouts and severe image quality degradation. Extensive experiments are implemented to demonstrate the effectiveness of the proposed method.
Task-driven Webpage Saliency
Authors: Quanlong Zheng, Jianbo Jiao, Ying Cao, Rynson W. H. Lau
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Prediction, Saliency Detection, Saliency Prediction

In this paper, we present an end-to-end learning framework for predicting task-driven visual saliency on webpages. Given a webpage, we propose a convolutional neural network to predict where people look at it under different task conditions. Inspired by the observation that given a specific task, human attention is strongly correlated with certain semantic components on a webpage (e.g., images, buttons and input boxes), our network explicitly disentangles saliency prediction into two independent sub-tasks: task-specific attention shift prediction and task-free saliency prediction. The task-specific branch estimates task-driven attention shift over a webpage from its semantic components, while the task-free branch infers visual saliency induced by visual features of the webpage. The outputs of the two branches are combined to produce the final prediction. Such a task decomposition framework allows us to efficiently learn our model from a small-scale task-driven saliency dataset with sparse labels (captured under a single task condition). Experimental results show that our method outperforms the baselines and prior works, achieving state-of-the-art performance on a newly collected benchmark dataset for task-driven webpage saliency detection.
Coded Two-Bucket Cameras for Computer Vision
Authors: Mian Wei, Navid Sarhangnejad, Zhengfan Xia, Nikita Gusev, Nikola Katic, Roman Genov, Kiriakos N. Kutulakos
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Vocal Bursts Valence Prediction

We introduce coded two-bucket (C2B) imaging, a new operating principle for computational sensors with applications in active 3D shape estimation and coded-exposure imaging. A C2B sensor modulates the light arriving at each pixel by  controlling which of the pixel's two "buckets" should integrate it. C2B sensors output two images per video frame---one per bucket---and allow rapid, fully-programmable, per-pixel control of the active bucket. Using these properties as a starting point, we (1) develop an image formation model for these sensors, (2) couple them with programmable light sources to acquire illumination mosaics, i.e., images of a scene under many different  illumination conditions whose pixels have been multiplexed onto the sensor plane and acquired in one shot, and (3) show how to process illumination  mosaics to  acquire time-varying depth or normal maps of dynamic scenes at the sensor's native resolution. We present the first experimental demonstration of these capabilities, using a fully functional C2B camera prototype. Key to this prototype is  a C2B sensor that was designed by us, fabricated in a standard CMOS imaging technology, and demonstrated for the first time in this paper.
Egocentric Activity Prediction via Event Modulated Attention
Authors: Yang Shen, Bingbing Ni, Zefan Li, Ning Zhuang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Activity Prediction, Event Extraction, Prediction

Predicting future activities from an egocentric viewpoint is of particular interest in assisted living. However, state-of-the-art egocentric activity understanding techniques are mostly NOT capable of predictive tasks, as their synchronous processing architecture performs poorly in either modeling event dependency or pruning temporal redundant features. This work explicitly addresses these issues by proposing an asynchronous gaze-event driven attentive activity prediction network. This network is built on a gaze-event extraction module inspired by the fact that gaze moving in/out a certain object most probably indicates the occurrence/ending of a certain activity. The extracted gaze events are input to: 1) an asynchronous module which reasons about the temporal dependency between events and 2) a synchronous module which softly attends to informative temporal durations for more compact and discriminative feature extraction. Both modules are seamlessly integrated for collaborative prediction. Extensive experimental results on egocentric activity prediction as well as recognition well demonstrate the effectiveness of the proposed method.
Real-Time Hair Rendering using Sequential Adversarial Networks
Authors: Lingyu Wei, Liwen Hu, Vladimir Kim, Ersin Yumer, Hao Li
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Diversity

We present an adversarial network for rendering photorealistic hair as an alternative to conventional computer graphics pipelines. Our deep learning approach does not require low-level parameter tuning nor ad-hoc asset design. Our method simply takes a strand-based 3D hair model as input and provides intuitive user-control for color and lighting through reference images. To handle the diversity of hairstyles and its appearance complexity, we disentangle hair structure, color, and illumination properties using a sequential GAN architecture and a semi-supervised training approach. We also introduce an intermediate edge activation map to orientation field conversion step to ensure a successful CG-to-photoreal transition, while preserving the hair structures of the original input data. As we only require a feed-forward pass through the network, our rendering performs in real-time. We demonstrate the synthesis of photorealistic hair images on a wide range of intricate hairstyles and compare our technique with state-of-the-art hair rendering methods.
Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation
Authors: Zhen-Yu Zhang, Zhen Cui, Chunyan Xu, Zequn Jie, Xiang Li, Jian Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Depth Estimation, Monocular Depth Estimation, Segmentation, Semantic Segmentation

In this paper, we propose a novel joint Task-Recursive Learning (TRL) framework for the closing-loop semantic segmentation and monocular depth estimation tasks. TRL can recursively refine the results of both tasks through serialized task-level interactions. In order to mutually-boost for each other, we encapsulate the interaction into a specific Task-Attentional Module (TAM) to adaptively enhance some counterpart patterns of both tasks. Further, to make the inference more credible, we propagate previous learning experiences on both tasks into the next network evolution by explicitly concatenating previous responses. The sequence of task-level interactions are finally evolved along a coarse-to-fine scale space such that the required details may be reconstructed progressively. Extensive experiments on NYU-Depth v2 and SUN RGB-D datasets demonstrate that our method achieves state-of-the-art results for monocular depth estimation and semantic segmentation.
Pose Partition Networks for Multi-Person Pose Estimation
Authors: Xuecheng Nie, Jiashi Feng, Junliang Xing, Shuicheng Yan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Human Detection, Multi-Person Pose Estimation, Pose Estimation

This paper proposes a novel Pose Partition Network (PPN) to address the challenging multi-person pose estimation problem. The proposed PPN is favorably featured by low complexity and high accuracy of joint detection and partition. In particular, PPN  performs  dense regressions from global joint candidates within a specific embedding space, which is  parameterized by centroids of persons, to efficiently generate robust person detection and joint partitions. Then, PPN infers    joint configurations of person poses through  conducting   graph partition  for each  person detection locally, utilizing reliable global affinity cues. In this way, PPN reduces computation complexity and  improves multi-person pose estimation significantly. We implement PPN  with the Hourglass architecture as the backbone network to simultaneously learn joint detector and dense regressor. Extensive experiments on benchmarks MPII Human Pose Multi-Person, extended PASCAL-Person-Part, and WAF, show the efficiency of PPN with new state-of-the-art performance.
3D-CODED: 3D Correspondences by Deep Deformation
Authors: Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, Mathieu Aubry
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We present a new deep learning approach for matching deformable shapes by introducing Shape Deformation Networks which jointly encode  3D  shapes  and  correspondences.  This  is  achieved  by  factoring the  surface  representation  into  (i)  a  template,  that  parameterizes  the surface,  and  (ii)  a  learnt  global  feature  vector  that  parameterizes  the transformation of the template into the input surface. By predicting this feature for a new shape, we implicitly predict correspondences between this shape and the template. We show that these correspondences can be improved by an additional step which improves the shape feature by minimizing  the  Chamfer  distance  between  the  input  and  transformed template. We demonstrate that our simple approach improves on state-of-the-art results on the difficult FAUST-inter challenge, with an average correspondence error of 2.88cm. We show, on the TOSCA dataset, that our  method  is  robust  to  many  types  of  perturbations,  and  generalizes to non-human shapes. This robustness allows it to perform well on real unclean, meshes from the the SCAPE dataset.
Interpretable Basis Decomposition for Visual Explanation
Authors: Bolei Zhou, Yiyou Sun, David Bau, Antonio Torralba
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Explanations of the decisions made by a deep neural network are important for human end-users to be able to understand and diagnose the trustworthiness of the system. Current neural networks used for visual recognition are generally used as black boxes that do not provide any human interpretable justification for a prediction. In this work we propose a new framework called Interpretable Basis Decomposition for providing visual explanations for classification networks. By decomposing the neural activations of the input image into semantically interpretable components pre-trained from a large concept corpus, the proposed framework is able to disentangle the evidence encoded in the activation feature vector, and quantify the contribution of each piece of evidence to the final prediction. We apply our framework for providing explanations to several popular networks for visual recognition, and show it is able to explain the predictions given by the networks in human-interpretable way. The human interpretability of the visual explanations provided by our framework and other recent explanation methods is evaluated through Amazon Mechanical Turk, showing that our framework generates more faithful and interpretable explanations.
HandMap: Robust Hand Pose Estimation via Intermediate Dense Guidance Map Supervision
Authors: Xiaokun Wu, Daniel Finnegan, Eamonn O'Neill, Yong-Liang Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Hand Pose Estimation, Pose Estimation, regression

This work presents a novel hand pose estimation framework via intermediate dense guidance map supervision. By leveraging the advantage of predicting heat maps of hand joints in detection-based methods, we propose to use dense feature maps through intermediate supervision in a regression-based framework that is not limited to the resolution of the heat map. Our dense feature maps are delicately designed to encode the hand geometry and the spatial relation between local joint and global hand. The proposed framework significantly improves the state-of-the-art in both 2D and 3D on the recent benchmark datasets.
Understanding Degeneracies and Ambiguities in Attribute Transfer
Authors: Attila Szabo, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, Paolo Favaro
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Attribute

We study the problem of building models that can transfer selected attributes from one image to another without affecting the other attributes. Towards this goal, we develop analysis and a training methodology for autoencoding models, whose encoded features aim to disentangle attributes. These features are explicitly split into two components: one that should represent attributes in common between pairs of images, and another that should represent attributes that change between pairs of images. We show that achieving this objective faces two main challenges: One is that the model may learn degenerate mappings, which we call shortcut problem, and the other is that the attribute representation for an image is not guaranteed to follow the same interpretation on another image, which we call reference ambiguity. To address the shortcut problem, we introduce novel constraints on image pairs and triplets and show their effectiveness both analytically and experimentally. In the case of the reference ambiguity, we formally prove that a model that guarantees an ideal feature separation cannot be built. We validate our findings on several datasets and show that, surprisingly, trained neural networks often do not exhibit the reference ambiguity.
Relaxation-Free Deep Hashing via Policy Gradient
Authors: Xin Yuan, Liangliang Ren, Jiwen Lu, Jie zhou
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Hashing, Image Retrieval, Retrieval

In this paper, we propose a simple yet effective relaxation-free method to learn more effective binary codes via policy gradient for scalable image search. While a variety of deep hashing methods have been proposed in recent years, most of them are confronted by the dilemma to obtain optimal binary codes in a truly end-to-end manner with non-smooth sign activations. Unlike existing methods which usually employ a general relaxation framework to adapt to the gradient-based algorithms, our approach formulates the non-smooth part of the hashing network as sampling with a stochastic policy, so that the retrieval performance degradation caused by the relaxation can be avoided. Specifically, our method directly generates the binary codes and maximizes the expectation of rewards for similarity preservation, where the network can be trained directly via policy gradient. Hence, the differentiation challenge for discrete optimization can be naturally addressed, which leads to effective gradients and binary codes. Extensive experimental results on three benchmark datasets validate the effectiveness of the proposed method.
A New Large Scale Dynamic Texture Dataset with Application to ConvNet Understanding
Authors: Isma Hadji, Richard P. Wildes
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Optical Flow Estimation, Scene Recognition, Transfer Learning

This paper introduces a new large scale dynamic texture dataset. The dataset is provided with two complementary organizations, one based on dynamics independent of spatial appearance and one based on spatial appearance independent of dynamics. With over 10,000 videos, the proposed Dynamic Texture DataBase (DTDB) is two orders of magnitude larger than any previously available dynamic texture dataset. The complementary organizations of the dataset allow for uniquely insightful experiments regarding the abilities of major classes of spatiotemporal ConvNet architectures to exploit appearance vs. dynamic information. We also present a novel two-stream ConvNet that provides an alterna- tive to the standard optical-flow-based motion stream to broaden the range of dynamic patterns that can be encompassed. The resulting motion stream is shown to outperform the traditional optical flow stream by considerable margins. Finally, the utility of the dataset as a pre-training substrate is demonstrated via transfer learning experiments with a different dynamic texture dataset as well as the companion task of dynamic scene recognition resulting in a new state-of-the-art.
Dynamic Task Prioritization for Multitask Learning
Authors: Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, Li Fei-Fei
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We propose dynamic task prioritization for multitask learning. This allows a model to dynamically prioritize difficult tasks during training, where difficulty is inversely proportional to performance, and where difficulty changes over time. In contrast to curriculum learning, where easy tasks are prioritized above difficult tasks, we present several studies showing the importance of prioritizing difficult tasks first. We observe that imbalances in task difficulty can lead to unnecessary emphasis on easier tasks, thus neglecting and slowing progress on difficult tasks. Motivated by this finding, we introduce a notion of dynamic task prioritization to automatically prioritize more difficult tasks by adaptively adjusting the mixing weight of each task's loss objective. Additional ablation studies show the impact of the task hierarchy, or the task ordering, when explicitly encoded in the network architecture. Our method outperforms existing multitask methods and demonstrates competitive results with modern single-task models on the COCO and MPII datasets.
Deep Regression Tracking with Shrinkage Loss
Authors: Xiankai Lu, Chao Ma, Bingbing Ni, Xiaokang Yang, Ian Reid, Ming-Hsuan Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: regression

Regression trackers directly learn a mapping from regularly dense samples of target objects to soft labels, which are usually generated by a Gaussian function, to estimate target positions. Due to the potential for fast-tracking and easy implementation, regression trackers have received increasing attention recently. However, state-of-the-art deep regression trackers do not perform as well as discriminative correlation filters (DCFs) trackers. We identify the main bottleneck of training regression networks as extreme foreground-background data imbalance. To balance training data, we propose a novel shrinkage loss to penalize the importance of easy training data.  Additionally, we apply residual connections to fuse multiple convolutional layers as well as their output response maps. Without bells and whistles, the proposed deep regression tracking method performs favorably against state-of-the-art trackers, especially in comparison with DCFs trackers, on five benchmark datasets including OTB-2013, OTB-2015, Temple-128, UAV-123 and VOT-2016.