pwc-archive/papers-with-abstracts huggingface dataset:
Sequential Clique Optimization for Video Object Segmentation
Authors: Yeong Jun Koh, Young-Yoon Lee, Chang-Su Kim
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection, Segmentation, Semantic Segmentation, Video Object Segmentation, Video Salient Object Detection, Video Semantic Segmentation

 A novel algorithm to segment out objects in a video sequence is proposed in this work. First, we extract object instances in each frame. Then, we select a visually important object instance in each frame to construct the salient object track through the sequence. This can be formulated as finding the maximal weight clique in a complete k-partite graph, which is NP hard. Therefore, we develop the sequential clique optimization (SCO) technique to efficiently determine the cliques corresponding to salient object tracks. We convert these tracks into video object segmentation results. Experimental results show that the proposed algorithm significantly outperforms the state-of-the-art video object segmentation and video salient object detection algorithms on recent benchmark datasets.
Constraint-Aware Deep Neural Network Compression
Authors: Changan Chen, Frederick Tung, Naveen Vedula, Greg Mori
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Bayesian Optimization, Neural Network Compression, Pedestrian Detection

Deep neural network compression has the potential to bring modern resource-hungry deep networks to resource-limited devices. However, in many of the most compelling deployment scenarios of compressed deep networks, the operational constraints matter: for example, a pedestrian detection network on a self-driving car may have to satisfy a latency constraint for safe operation. We propose the first principled treatment of deep network compression under operational constraints. We formulate the compression learning problem from the perspective of constrained Bayesian optimization, and introduce a cooling (annealing) strategy to guide the network compression towards the target constraints. Experiments on ImageNet demonstrate the value of modelling constraints directly in network compression.
Linear RGB-D SLAM for Planar Environments
Authors: Pyojin Kim, Brian Coltin, H. Jin Kim
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We propose a new formulation for including orthogonal planar features as a global model into a linear SLAM approach based on sequential Bayesian filtering. Previous planar SLAM algorithms estimate the camera poses and multiple landmark planes in a pose graph optimization. However, since it is formulated as a high dimensional nonlinear optimization problem, there is no guarantee the algorithm will converge to the global optimum. To overcome these limitations, we present a new SLAM method that jointly estimates camera position and planar landmarks in the map within a linear Kalman filter framework. It is rotations that make the SLAM problem highly nonlinear. Therefore, we solve for the rotational motion of the camera using structural regularities in the Manhattan world (MW), resulting in a linear SLAM formulation. We test our algorithm on standard RGB-D benchmarks as well as additional large indoor environments, demonstrating comparable performance to other state-of-the-art SLAM methods without the use of expensive nonlinear optimization.
Key-Word-Aware Network for Referring Expression Image Segmentation
Authors: Hengcan Shi, Hongliang Li, Fanman Meng, Qingbo Wu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Segmentation, Object, Referring Expression, Semantic Segmentation

Referring expression image segmentation aims to segment out the object referred by a natural language query expression. Without considering the specific properties of visual and textual information, existing works usually deal with this task by directly feeding a foreground/background classifier with cascaded image and text features, which are extracted from each image region and the whole query, respectively. On the one hand, they ignore that each word in a query expression makes different contributions to identify the desired object, which requires a differential treatment in extracting text feature. On the other hand, the relationships of different image regions are not considered as well, even though they are greatly important to eliminate the undesired foreground object in accordance with specific query. To address aforementioned issues, in this paper, we propose a key-word-aware network, which contains a query attention model and a key-word-aware visual context model. In extracting text features, the query attention model attends to assign higher weights for the words which are more important for identifying object. Meanwhile, the key-word-aware visual context model describes the relationships among different image regions, according to corresponding query. Our proposed method outperforms state-of-the-art methods on two referring expression image segmentation databases.
Few-Shot Human Motion Prediction via Meta-Learning
Authors: Liang-Yan Gui, Yu-Xiong Wang, Deva Ramanan, Jose M. F.  Moura
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Few-Shot Learning, Human motion prediction, Meta-Learning, motion prediction, Prediction

Human motion prediction, forecasting human motion in a few milliseconds conditioning on a historical 3D skeleton sequence, is a long-standing problem in computer vision and robotic vision. Existing forecasting algorithms rely on extensive annotated motion capture data and are brittle to novel actions. This paper addresses the problem of few-shot human motion prediction, in the spirit of the recent progress on few-shot learning and meta-learning. More precisely, our approach is based on the insight that having a good generalization from few examples relies on both a generic initial model and an effective strategy for adapting this model to novel tasks. To accomplish this, we propose proactive and adaptive meta-learning (PAML) that introduces a novel combination of model-agnostic meta-learning and model regression networks and unifies them into an integrated, end-to-end framework. By doing so, our meta-learner produces a generic model initialization through aggregating contextual information from a variety of prediction tasks, while this model can effectively adapt to a specific task by leveraging learning-to-learn knowledge about how to transform few-shot model parameters to many-shot model parameters. The resulting PAML predictor model significantly improves the prediction performance on the heavily benchmarked H3.6M dataset in the small-sample size regime.
Correcting the Triplet Selection Bias for Triplet Loss
Authors: Baosheng Yu, Tongliang Liu, Mingming Gong, Changxing Ding, DaCheng Tao
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Recognition, Fine-Grained Image Classification, General Classification, image-classification, Image Classification, Image Retrieval, Metric Learning, Retrieval, Selection bias, Triplet

Triplet loss, popular for metric learning, has made a great success in many computer vision tasks, such as fine-grained image classification, image retrieval, and face recognition. Considering that the number of triplets grows cubically with the size of training data, triplet mining is thus indispensable for efficiently training with triplet loss. However, in practice, the training is usually very sensitive to the selected triplets, e.g., it almost does not converge with randomly selected triplets and selecting hardest triplets also leads to bad local minima. We argue that the bias in sampling of triplets degrades the performance of learning with triplet loss. In this paper, we propose a new variant of triplet loss, which tries to reduce the bias in triplet sampling by adaptively correcting the distribution shift on sampled triplets. We refer to this new triplet loss as adapted triplet loss. We conduct a number of experiments on MNIST and Fashion-MNIST for image classification, and on CARS196, CUB200-2011, and Stanford Online Products for image retrieval. The experimental results demonstrate the effectiveness of the proposed method.
Good Line Cutting: towards Accurate Pose Tracking of Line-assisted VO/VSLAM
Authors: Yipu Zhao, Patricio A. Vela
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Pose Estimation, Pose Tracking

This paper tackles a problem in line-assisted VO/VSLAM:  accurately solving the least squares pose optimization with unreliable 3D  line input. The solution we present is good line cutting, which extracts the most-informative sub-segment from each 3D line for use within the pose optimization formulation. By studying the impact of line cutting towards the information gain of pose estimation in line-based least squares problem, we demonstrate the applicability of improving pose estimation accuracy with good line cutting. To that end, we describe an efficient algorithm that approximately  approaches the joint optimization problem of good line cutting. The proposed algorithm is integrated into a state-of-the-art  line-assisted VSLAM system.   When evaluated in two target scenarios of line-assisted VO/VSLAM,  low-texture and motion blur, the accuracy of pose tracking is  improved, while the robustness is preserved.
``Factual'' or ``Emotional'': Stylized Image Captioning with Adaptive Learning and Attention
Authors: Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, Jiebo Luo
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Captioning

Generating stylized captions for an image is an emerging topic in image captioning. Given an image as input, it requires the system to generate a caption that has a specific style (e.g., humorous, romantic, positive, and negative) while describing the image content semantically accurately. In this paper, we propose a novel stylized image captioning model that effectively takes both requirements into consideration. To this end, we first devise a new variant of LSTM, named style-factual LSTM, as the building block of our model. It  uses two groups of matrices to capture the factual and stylized knowledge, respectively, and automatically learns the word-level weights of the two groups based on previous context. In addition, when we train the model to capture stylized elements, we propose an adaptive learning approach based on a reference factual model, it provides factual knowledge to the model as the model learns from stylized caption labels, and can adaptively compute how much information to supply at each time step. We evaluate our model on two stylized image captioning datasets, which contain humorous/romantic captions and positive/negative captions, respectively. Experiments shows that our proposed model outperforms the state-of-the-art approaches, without using extra ground truth supervision.
Single Image Water Hazard Detection using FCN with Reflection Attention Units
Authors: Xiaofeng Han, Chuong Nguyen, ShaoDi You, Jianfeng Lu
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Water bodies, such as puddles and flooded areas, on and off road pose significant risks to autonomous cars. Detecting water from moving camera is a challenging task as water surface is highly refractive, and its appearance varies with viewing angle, surrounding scene, weather conditions. In this paper, we present a water puddle detection method based on a Fully Convolutional Network (FCN) with our newly proposed Reflection Attention Units (RAUs). An RAU is a deep network unit designed to embody the physics of reflection on water surface from sky and nearby scene. To verify the performance of our proposed method, we collect 11455 color stereo images with polarizers, and 985 of left images are annotated and divided into 2 datasets: On Road (ONR) dataset and Off Road (OFR) dataset. We show that FCN-8s with RAUs improves significantly precision and recall metrics as compared to FCN-8s, DeepLab V2 and Gaussian Mixture Model (GMM). We also show that focal loss function can improve the performance of FCN-8s network due to the extreme imbalance of water versus ground classification problem.
Bidirectional Feature Pyramid Network with Recurrent Attention Residual Modules for Shadow Detection
Authors: Lei Zhu, Zijun Deng, Xiao-Wei Hu, Chi-Wing Fu, Xuemiao Xu, Jing Qin, Pheng-Ann Heng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Shadow Detection

This paper presents a network to detect shadows by exploring and combining global context in deep layers and local context in shallow layers of a deep convolutional neural network (CNN). There are two technical contributions in our network design. First, we formulate the recurrent attention residual (RAR) module to combine the contexts in two adjacent CNN layers and learn an attention map to select a residual and then refine the context features. Second, we develop a bidirectional feature pyramid network (BFPN) to aggregate shadow contexts spanned across different CNN layers by deploying two series of RAR modules in the network to iteratively combine and refine context features: one series to refine context features from deep to shallow layers, and another series from shallow to deep layers. Hence, we can better suppress false detections and enhance shadow details at the same time. We evaluate our network on two common shadow detection benchmark datasets: SBU and UCF. Experimental results show that our network outperforms the best existing method with 34.88% reduction on SBU and 34.57% reduction on UCF for the balance error rate.
Dense Semantic and Topological Correspondence of 3D Faces without Landmarks
Authors: Zhenfeng Fan, Xiyuan Hu, Chen Chen, Silong Peng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Semantic correspondence

Many previous literatures use landmarks to guide the cor- respondence of 3D faces. However, these landmarks, either manually or automatically annotated, are hard to define consistently across differ- ent faces in many circumstances. We propose a general framework for dense correspondence of 3D faces without landmarks in this paper. The dense correspondence goal is revisited in two perspectives: semantic and topological correspondence. Starting from a template facial mesh, we sequentially perform global alignment, primary correspondence by tem- plate warping, and contextual mesh refinement, to reach the final cor- respondence result. The semantic correspondence is achieved by a local iterative closest point (ICP) algorithm of kernelized version, allowing accurate matching of local features. Then, robust deformation from the template to the target face is formulated as a minimization problem. Fur- thermore, this problem leads to a well-posed sparse linear system such that the solution is unique and efficient. Finally, a contextual mesh re- fining algorithm is applied to ensure topological correspondence. In the experiment, the proposed method is evaluated both qualitatively and quantitatively on two datasets including a publicly available FRGC v2.0 dataset, demonstrating reasonable and reliable correspondence results.
Mancs: A Multi-task Attentional Network with Curriculum Sampling for Person Re-identification
Authors: Cheng Wang, Qian Zhang, Chang Huang, Wenyu Liu, Xinggang Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Person Re-Identification

We propose a novel deep network called Mancs that solves the person re-identification problem from the following aspects: fully utilizing the attention mechanism for the person misalignment problem and properly sampling for the ranking loss to obtain more stable person representation. Technically, we contribute a novel fully attentional block which is deeply supervised and can be plugged into any CNN, and a novel curriculum sampling method which is effective for training ranking losses. The learning tasks are integrated into a unified framework and jointly optimized. Experiments have been carried out on Market1501, CUHK03 and DukeMTMC. All the results show that Mancs can significantly outperform the previous state-of-the-arts. In addition, the effectiveness of the newly proposed ideas has been confirmed by extensive ablation studies.
Efficient Relative Attribute Learning using Graph Neural Networks
Authors: Zihang Meng, Nagesh Adluru, Hyunwoo J. Kim, Glenn Fung, Vikas Singh
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Attribute, Clothing Attribute Recognition

A sizable body of work on relative attributes provides compelling evidence that relating pairs of images along a continuum of strength pertaining to a visual attribute yields significant improvements in a wide variety of tasks in vision. In this paper, we show how emerging ideas in graph neural networks can yield a unified solution to various problems that broadly fall under relative attribute learning. Our main idea is the realization that relative attribute learning naturally benefits from exploiting the graphical structure of dependencies among the different relative attributes of images, especially when only partial ordering of the relative attributes is provided in the training data. We use message passing on a probabilistic graphical model to perform end to end learning of appropriate representations of the images, their relationships as well as the interplay between different attributes to best align with provided annotations.  Our experiments demonstrate that this simple end-to-end learning framework using GNNs is very effective in achieving competitive accuracy with specialized methods for both relative attribute learning and binary attribute prediction, while significantly relaxing the requirements on the training data and/or the number of parameters or both.
Using Object Information for Spotting Text
Authors: Shitala Prasad, Adams Wai Kin Kong
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, Text Detection, Text Spotting

Text spotting, also called text detection, is a challenging computer vision task because of cluttered backgrounds, diverse imaging environments, various text sizes and similarity between some objects and characters, e.g., tyre and âoâ. However, text spotting is a vital step in numerous AI and computer vision systems, such as autonomous robots and systems for visually impaired. Due to its potential applications and commercial values, researchers have proposed various deep architectures and methods for text spotting. These methods and architectures concentrate only on text in images, but neglect other information related to text. There exists a strong relationship between certain objects and the presence of text, such as signboards or the absence of text, such as trees. In this paper, a text spotting algorithm based on text and object dependency is proposed. The proposed algorithm consists of two sub-convolutional neural networks and three training stages. For this study, a new NTU-UTOI dataset containing over 22k non-synthetic images with 277k bounding boxes for text and 42 text-related object classes is established. According to our best knowledge, it is the second largest non-synthetic text image database. Experimental results on three benchmark datasets with clutter backgrounds, COCO-Text, MSRA-TD500 and SVT show that the proposed algorithm provides comparable performance to state-of-the-art text spotting methods. Experiments are also performed on our newly established dataset to investigate the effectiveness of object information for text spotting. The experimental results indicate that the object information contributes significantly on the performance gain.
Deep Recursive HDRI: Inverse Tone Mapping using Generative Adversarial Networks
Authors: Siyeong Lee, Gwon Hwan An, Suk-Ju Kang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Generative Adversarial Network, inverse tone mapping, Inverse-Tone-Mapping, Tone Mapping

High dynamic range images contain luminance information of the physical world and provide more realistic experience than conventional low dynamic range images. Because most images have a low dynamic range, recovering the lost dynamic range from a single low dynamic range image is still prevalent. We propose a novel method for restoring the lost dynamic range from a single low dynamic range image through a deep neural network. The proposed method is the first framework to create high dynamic range images based on the estimated multi-exposure stack using the conditional generative adversarial network structure. In this architecture, we train the network by setting an objective function that is a combination of L1 loss and generative adversarial network loss. In addition, this architecture has a simplified structure than the existing networks. In the experimental results, the proposed network generated a multi-exposure stack consisting of realistic images with varying exposure values while avoiding artifacts on public benchmarks, compared with the existing methods. In addition, both the multi-exposure stacks and high dynamic range images estimated by the proposed method are significantly similar to the ground truth than other state-of-the-art algorithms.
Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection
Authors: Si-Qi Liu, Xiangyuan Lan, Pong C. Yuen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Presentation Attack Detection, Face Recognition

3D mask face presentation attack, as a new challenge in face recognition, has been attracting increasing attention. Recently, remote Photoplethysmography (rPPG) is employed as an intrinsic liveness cue which is independent of the mask appearance. Although existing rPPG-based methods achieve promising results on both intra and cross dataset scenarios, they may not be robust enough when rPPG signals are contaminated by noise. In this paper, we propose a new liveness feature, called rPPG correspondence feature (CFrPPG) to precisely identify the heartbeat vestige from the observed noisy rPPG signals. To further overcome the global interferences, we propose a novel learning strategy which incorporates the global noise within the CFrPPG feature. Extensive experiments indicate that the proposed feature not only outperforms the state-of-the-art rPPG based methods on 3D mask attacks but also be able to handle the practical scenarios with dim light and camera motion.
Fast Light Field Reconstruction With Deep Coarse-To-Fine Modeling of Spatial-Angular Clues
Authors: Henry Wing Fung Yeung, Junhui Hou, Jie Chen, Yuk Ying Chung, Xiaoming Chen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

Densely-sampled light fields (LFs) are beneficial to many applications such as depth inference and post-capture refocusing. However, it is costly and challenging to capture them. In this paper, we propose a learning based algorithm to reconstruct a densely-sampled LF fast and accurately from a sparsely-sampled LF in one forward pass. Our method uses computationally efficient convolutions to deeply characterize the high dimensional spatial-angular clues in a coarse-tofine manner. Specifically, our end-to-end model first synthesizes a set of intermediate novel sub-aperture images (SAIs) by exploring the coarse characteristics of the sparsely-sampled LF input with spatial-angular alternating convolutions. Then, the synthesized intermediate novel SAIs are efficiently refined by further recovering the fine relations from all SAIs via guided residual learning and stride-2 4-D convolutions. Experimental results on extensive real-world and synthetic LF images show that our model can provide more than 3 dB advantage in reconstruction quality in average than the state-of-the-art methods while being computationally faster by a factor of 30. Besides, more accurate depth can be inferred from the reconstructed densely-sampled LFs by our method.