pwc-archive/papers-with-abstracts huggingface dataset:
Weakly-supervised 3D Hand Pose Estimation from Monocular RGB Images
Authors: Yujun Cai, Liuhao Ge, Jianfei Cai, Junsong Yuan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Hand Pose Estimation, Hand Pose Estimation, Pose Estimation

	Compared with depth-based 3D hand pose estimation, it is more challenging to infer 3D hand pose from monocular RGB images, due to substantial depth ambiguity and the difficulty of obtaining fully-annotated training data. Different from existing learning-based monocular RGB-input approaches that require accurate 3D annotations for training, we propose to leverage the depth images that can be easily obtained from commodity RGB-D cameras during training, while during testing we take only RGB inputs for 3D joint predictions. In this way, we alleviate the burden of the costly 3D annotations in real-world dataset. Particularly, we  propose a weakly-supervised method, adaptating from fully-annotated synthetic dataset to weakly-labeled real-world dataset with the aid of a depth regularizer, which generates depth maps from predicted 3D pose and serves as weak supervision for 3D pose regression. Extensive experiments on benchmark datasets validate the effectiveness of the proposed depth regularizer in both weakly-supervised and fully-supervised settings.
Graph Adaptive Knowledge Transfer for Unsupervised Domain Adaptation
Authors: Zhengming Ding, Sheng Li, Ming Shao, Yun Fu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, Transfer Learning, Unsupervised Domain Adaptation

Unsupervised domain adaptation has caught appealing attentions as it facilitates the unlabeled target learning by borrowing existing well-established source domain knowledge. Recent practice on domain adaptation manages to extract effective features by incorporating the pseudo labels for the target domain to better solve cross-domain distribution divergences. However, existing approaches separate target label optimization and domain-invariant feature learning as different steps. To address that issue, we develop a novel Graph Adaptive Knowledge Transfer (GAKT) model to jointly optimize target labels and domain-free features in a unified framework. Specifically, semi-supervised knowledge adaptation and label propagation on target data are coupled to benefit each other, and hence the marginal and conditional disparities across different domains will be better alleviated. Experimental evaluation on two cross-domain visual datasets demonstrates the effectiveness of our designed approach on facilitating the unlabeled target task learning, compared to the state-of-the-art domain adaptation approaches.
Single Image Intrinsic Decomposition without a Single Intrinsic Image
Authors: Wei-Chiu Ma, Hang Chu, Bolei Zhou, Raquel Urtasun, Antonio Torralba
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Intrinsic Image Decomposition

Intrinsic image decomposition---decomposing a natural image into a set of images corresponding to different physical causes---is one of the key and fundamental problems of computer vision. Previous intrinsic decomposition approaches either address the problem in a fully supervised manner, or require multiple images of the same scene as input. These approaches are less desirable in practice, as ground truth intrinsic images are extremely difficult to acquire, and requirement of multiple images pose severe limitation on applicable scenarios. In this paper, we propose to bring the best of both worlds. We present a two stream convolutional neural network framework that is capable of learning the decomposition effectively in the absence of any ground truth intrinsic images, and can be easily extended to a (semi-)supervised setup. At inference time, our model can be easily reduced to a single stream module that performs intrinsic decomposition on a single input image. We demonstrate the effectiveness of our framework through extensive experimental study on both synthetic and real-world datasets, showing superior performance over previous approaches in both single-image and multi-image settings. Notably, our approach outperforms previous state-of-the-art single image methods while using only 50% of ground truth supervision.
Deep Multi-Task Learning to Recognise Subtle Facial Expressions of Mental States
Authors: Guosheng Hu, Li Liu, Yang Yuan, Zehao Yu, Yang Hua, Zhihong Zhang, Fumin Shen, Ling Shao, Timothy Hospedales, Neil Robertson, Yongxin Yang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deception Detection, Facial Expression Recognition, Facial Expression Recognition (FER), Facial Landmark Detection, Multi-Task Learning, Representation Learning, Transfer Learning

Facial expression recognition is a topical task. However, very little research investigates subtle expression recognition, which is important for mental activity analysis, deception detection, etc. We address subtle expression recognition through convolutional neural networks (CNNs) by developing multi-task learning (MTL) to effectively leverage a side task: facial landmark detection. Existing MTL methods follow a design pattern of shared bottom CNN layers and task-specific top layers. However, the sharing architecture is usually heuristically chosen, as it is difficult to decide which layers should be shared. Our approach is composed of (1) a novel MTL framework that automatically learns which layers to share through optimisation under tensor trace norm regularisation and (2) an invariant representation learning approach that allows the CNN to leverage tasks defined on disjoint datasets without suffering from dataset distribution shift. To advance subtle expression recognition, we contribute a Large-scale Subtle Emotions and Mental States in the Wild database (LSEMSW). LSEMSW includes a variety of cognitive states as well as basic emotions. It contains 176K images, manually annotated with 13 emotions, and thus provides the first subtle expression dataset large enough for training deep CNNs. Evaluations on LSEMSW and 300-W (landmark) databases show the effectiveness of the proposed methods. In addition, we investigate transferring knowledge learned from LSEMSW database to traditional (non-subtle) expression recognition. We achieve very competitive performance on Oulu-Casia NIR&Vis and CK+ databases via transfer learning.
Recovering 3D Planes from a Single Image via Convolutional Neural Networks
Authors: Fengting Yang, Zihan Zhou
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

In this paper, we study the problem of recovering 3D planar surfaces from a single image of man-made environment. We show that it is possible to directly train a deep neural network to achieve this goal. A novel plane structure-induced loss is proposed to train the network to simultaneously predict a plane segmentation map and the parameters of the 3D planes. Further, to avoid the tedious manual labeling process, we show how to leverage existing large-scale RGB-D dataset to train our network without explicit 3D plane annotations, and how to take advantage of the semantic labels come with the dataset for accurate planar and non-planar classification. Experiment results demonstrate that our method significantly outperforms existing methods, both qualitatively and quantitatively. The recovered planes could potentially benefit many important visual tasks such as vision-based navigation and human-robot interaction.
A Geometric Perspective on Structured Light Coding
Authors: Mohit Gupta, Nikhil Nakhate
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We present a mathematical framework for analysis and design of high performance structured light (SL) coding schemes. Using this framework, we design Hamiltonian SL coding, a novel family of SL coding schemes that can recover 3D shape with extreme precision, with a small number (as few as three) of images. We establish structural similarity between popular discrete (binary) SL coding methods, and Hamiltonian coding, which is a continuous coding approach. Based on this similarity, and by leveraging design principles from several different SL coding families, we propose a general recipe for designing Hamiltonian coding patterns with specific desirable properties, such as patterns with high spatial frequencies for dealing with global illumination. We perform several experiments to evaluate the proposed approach, and demonstrate that Hamiltonian coding based SL approaches outperform existing methods in challenging scenarios, including scenes with dark albedos, strong ambient light, and interreflections.
Learning to Dodge A Bullet: Concyclic View Morphing via Deep Learning
Authors: Shi Jin, Ruiynag Liu, Yu Ji, Jinwei Ye, Jingyi Yu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

The bullet-time effect, presented in feature film ``The Matrix", has been widely adopted in feature films and TV commercials to create an amazing stopping-time illusion. Producing such visual effects, however, typically requires using a large number of cameras/images surrounding the subject. In this paper, we present a learning-based solution that is capable of producing the bullet-time effect from only a small set of images. Specifically, we present a view morphing framework that can synthesize smooth and realistic transitions along 	extit{a circular view path} using as few as three reference images. We apply a novel cyclic rectification technique to align the reference images onto a common circle and then feed the rectified results into a deep network to predict its motion field and per-pixel visibility for new view interpolation. Comprehensive experiments on synthetic and real data show that our new framework outperforms the state-of-the-art and provides an inexpensive and practical solution for producing the bullet-time effects.
Learning Discriminative Video Representations Using Adversarial Perturbations
Authors: Jue Wang, Anoop Cherian
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Binary Classification, Riemannian optimization, Video Recognition

Adversarial perturbations are noise-like patterns that can subtly change the data, while failing an otherwise accurate classifier. In this paper, we propose to use such perturbations for improving the robustness of video representations. To this end, given a well-trained deep model for per-frame video recognition, we first generate adversarial noise adapted to this model. Using the original data features from the full video sequence and their perturbed counterparts, as two separate bags, we develop a binary classification problem that learns a set of discriminative hyperplanes -- as a subspace -- that will separate the two bags from each other. This subspace is then used as a descriptor for the video, dubbed discriminative subspace pooling. As the perturbed features belong to data classes that are likely to be confused with the original features, the discriminative subspace will characterize parts of the feature space that are more representative of the original data, and thus may provide robust video representations. To learn such descriptors, we formulate a subspace learning objective on the Stiefel manifold and resort to Riemannian optimization methods for solving it efficiently. We provide experiments on several video datasets and demonstrate state-of-the-art results.
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video
Authors: Yin Li, Miao Liu, James M. Rehg
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Recognition, Gaze Estimation, Temporal Action Localization

We address the task of jointly determining what a person is doing and where they are looking based on the analysis of video captured by a headworn camera. We propose a novel deep model for joint gaze estimation and action recognition in First Person Vision. Our method describes the participant's gaze as a probabilistic variable and models its distribution using stochastic units in a deep network. We sample from these stochastic units to generate an attention map. This attention map guides the aggregation of visual features in action recognition, thereby providing coupling between gaze and action. We evaluate our method on the standard EGTEA dataset and demonstrate performance that exceeds the state-of-the-art by a significant margin of 3.5%.
Compositional Learning for Human Object Interaction
Authors: Keizo Kato, Yin Li, Abhinav Gupta
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Human-Object Interaction Detection, Object, Zero-Shot Learning

The world of human-object interactions is rich. While generally we sit on chairs and sofas, if need be we can even sit on TVs or top of shelves. In recent years, there has been progress in modeling actions and human-object interactions. However, most of these approaches require lots of data. It is not clear if the learned representations of actions are generalizable to new categories. In this paper, we explore the problem of zero-shot learning of human-object interactions. Given limited verb-noun interactions in training data, we want to learn a model than can work even on unseen combinations. To deal with this problem, In this paper, we propose  a novel method using external knowledge graph and graph convolutional networks which learns how to compose classifiers for verb-noun pairs. We also provide benchmarks on several dataset for zero-shot learning including both image and video.  We hope our method, dataset and baselines will facilitate future research in this direction.
stagNet: An Attentive Semantic RNN for Group Activity Recognition
Authors: Mengshi Qi, Jie Qin, Annan Li, Yunhong Wang, Jiebo Luo, Luc van Gool
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Activity Recognition, Group Activity Recognition

Group activity recognition plays a fundamental role in a variety of applications, e.g. sports video analysis and intelligent surveillance. How to model the spatio-temporal contextual information in a scene still remains a crucial yet challenging issue. We propose a novel attentive semantic recurrent neural network (RNN), namely stagNet, for understanding group activities in videos, based on the spatio-temporal attention and semantic graph. A semantic graph is explicitly modeled to describe the spatial context of the whole scene, which is further integrated with the temporal factor via structural-RNN. Benefiting from the 'factor sharing' and 'message passing' mechanisms, our model is able to extract discriminative spatio-temporal features and to capture inter-group relationships. Moreover, we adopt a spatio-temporal attention model to attend to key persons/frames for improved performance. Two widely-used datasets are employed for performance evaluation, and the extensive results demonstrate the superiority of our method.
Double JPEG Detection in Mixed JPEG Quality Factors using Deep Convolutional Neural Network
Authors: Jinseok Park, Donghyeon Cho, Wonhyuk Ahn, Heung-Kyu Lee
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Quantization

Double JPEG detection is essential for detecting various image manipulations. This paper proposes a novel deep convolutional neural network for double JPEG detection using statistical histogram features from each block with a vectorized quantization table. In contrast to previous methods, the proposed approach handles mixed JPEG quality factors and is suitable for real-world situations. We collected real-world JPEG images from the image forensic service and generated a new double JPEG dataset with 1120 quantization tables to train the network. The proposed approach was verified experimentally to produce a state-of-the-art performance, successfully detecting various image manipulations.
Learning 3D Keypoint Descriptors for Non-Rigid Shape Matching
Authors: Hanyu Wang, Jianwei Guo, Dong-Ming Yan, Weize Quan, Xiaopeng Zhang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Metric Learning, Triplet

In this paper, we present a novel deep learning framework that derives discriminative local descriptors for 3D surface shapes. In contrast to previous convolutional neural networks (CNNs) that rely on rendering multi-view images or extracting intrinsic shape properties, we parameterize the multi-scale localized neighborhoods of a keypoint into regular 2D grids, which are termed as `geometry images'. The benefits of such geometry images include retaining sufficient geometric information, as well as allowing the usage of standard CNNs. Specifically, we leverage a triplet network to perform deep metric learning, which takes a set of triplets as input, and a newly designed triplet loss function is minimized to distinguish between similar and dissimilar pairs of keypoints. At the testing stage, given a geometry image of a point of interest, our network outputs a discriminative local descriptor for it. Experimental results for non-rigid shape matching on several benchmarks demonstrate the superior performance of our learned descriptors over traditional descriptors and the state-of-the-art learning-based alternatives.
DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs
Authors: Shi Yan, Chenglei Wu, Lizhen Wang, Feng Xu, Liang An, Kaiwen Guo, Yebin Liu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Denoising

Consumer depth sensors are more and more popular and come to our daily lives marked by its recent integration in the latest Iphone X. However, they still suffer from heavy noises which limit their applications. Although plenty of progresses have been made to reduce the noises and boost geometric details, due to the inherent illness and the real-time requirement, the problem is still far from been solved. We propose a cascaded Depth Denoising and Refinement Network (DDRNet) to tackle this problem by leveraging the multi-frame fused geometry and the accompanying high quality color image through a joint training strategy. The rendering equation is exploited in our network in an unsupervised manner. In detail, we impose an unsupervised loss based on the light transport to extract the high-frequency geometry. Experimental results indicate that our network achieves real-time single depth enhancement on various categories of scenes. Thanks to the well decoupling of the low and high frequency information in the cascaded network, we achieve superior performance over the state-of-the-art techniques.
GAL: Geometric Adversarial Loss for Single-View 3D-Object Reconstruction
Authors: Li Jiang, Shaoshuai Shi, Xiaojuan Qi, Jiaya Jia
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Object Reconstruction, Object Reconstruction

In this paper, we present a framework for reconstructing a point-based 3D model of an object from a single view image. Distance metrics, like Chamfer distance, were used in previous work to measure the difference of two point sets and serve as the loss function in point-based reconstruction. However, such point-point loss does not constrain the 3D model from a global perspective. We propose to add geometric adversarial loss (GAL). It is composed of two terms where the geometric loss ensures consistent shape of reconstructed 3D models close to ground-truth from different viewpoints, and the conditional adversarial loss generates a semantically-meaningful point cloud. GAL benefits predicting the obscured part of objects and maintaining geometric structure of the predicted 3D model. Both the qualitative results and quantitative analysis manifest the generality and suitability of our method.
Viewpoint Estimation---Insights & Model
Authors: Gilad Divon, Ayellet Tal
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, model, Viewpoint Estimation

This paper addresses the problem of viewpoint estimation of an object in a given image.  It presents five key insights and a CNN that is based on them. The network's major properties are as follows. (i) The architecture jointly solves detection, classification, and viewpoint estimation. (ii) New types of data are added and trained on. (iii) A novel loss function, which takes into account both the geometry of the problem and the new types of data, is propose. Our network allows a substantial boost in performance: from 36.1% gained by SOTA algorithms to 45.9%.
Super-Resolution and Sparse View CT Reconstruction
Authors: Guangming Zang, Mohamed Aly, Ramzi Idoughi, Peter Wonka, Wolfgang Heidrich
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Computed Tomography (CT), CT Reconstruction, Super-Resolution

We present a flexible framework for robust computed tomography (CT) reconstruction with a specific emphasis on recovering thin 1D and 2D manifolds embedded in 3D volumes. To reconstruct such structures at resolutions below the Nyquist limit of the CT image sensor, we devise a new 3D structure tensor prior, which can be incorporated as a regularizer into more traditional proximal optimization methods for CT reconstruction. As a second, smaller contribution, we also show that when using such a proximal reconstruction framework, it is beneficial to employ the Simultaneous Algebraic Reconstruction Technique (SART) instead of the commonly used Conjugate Gradient (CG) method in the solution of the data term proximal operator. We show empirically that CG often does not converge to the global optimum for tomography problem even though the underlying problem is convex. We demonstrate that using SART provides better reconstruction results in sparse-view settings using fewer projection images. We provide extensive experimental results for both contributions on both simulated and real data. Moreover, our code will also be made publicly available.
NNEval: Neural Network based Evaluation Metric for Image Captioning
Authors: Naeha Sharif, Lyndon White, Mohammed Bennamoun, Syed Afaq Ali Shah
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Captioning, Sentence

The automatic evaluation of image descriptions is an intricate task, and it is highly important in the development and fine-grained analysis of captioning systems. Existing metrics to automatically evaluate image captioning systems fail to achieve a satisfactory level of correlation with human judgements at the sentence-level. Moreover, these metrics, unlike humans, tend to focus on specific aspects of quality, such as the n-gram overlap or the semantic meaning. In this paper, we present the first learning-based metric to evaluate image captions. Our proposed framework enables us to incorporate both lexical and semantic information into a single learned metric. This results in an evaluator that takes into account various linguistic features to assess the caption quality. The experiments we performed to assess the proposed metric, show improvements upon the state of the art in terms of correlation with human judgements and demonstrate its superior robustness to distractions.
Monocular Depth Estimation Using Whole Strip Masking and Reliability-Based Refinement
Authors: Minhyeok Heo, Jae-Han Lee, Kyung-Rae Kim, Han-Ul Kim, Chang-Su Kim
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Depth Estimation, Monocular Depth Estimation

We propose a monocular depth estimation algorithm, which extracts a depth map from a single image, based on whole strip masking (WSM) and reliability-based refinement. First, we develop a convolutional neural network (CNN) tailored for the depth estimation. Specifically, we design a novel filter, called WSM, to exploit the tendency that a scene has similar depths in horizonal or vertical directions. The proposed CNN combines WSM upsampling blocks with the ResNet encoder. Second, we measure the reliability of an estimated depth, by appending additional layers to the main CNN. Using the reliability information, we perform conditional random field (CRF) optimization to refine the estimated depth map. Extensive experimental results demonstrate that the proposed algorithm provides the state-of-the-art depth estimation performance, outperforming conventional algorithms significantly.