pwc-archive/papers-with-abstracts huggingface dataset:
Light Structure from Pin Motion: Simple and Accurate Point Light Calibration for Physics-based Modeling
Authors: Hiroaki Santo, Michael Waechter, Masaki Samejima, Yusuke Sugano, Yasuyuki Matsushita
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Position

We present a practical method for geometric point light source calibration. Unlike in prior works that use Lambertian spheres, mirror spheres, or mirror planes, our calibration target consists of a Lambertian plane and small shadow casters at unknown positions above the plane. Due to their small size, the casters' shadows can be localized more precisely than highlights on mirrors. We show that, given shadow observations from a moving calibration target and a fixed camera, the shadow caster positions and the light position or direction can be simultaneously recovered in a structure from motion framework. Our evaluation on simulated and real scenes shows that our method yields light estimates that are stable and more accurate than existing techniques while having a considerably simpler setup and requiring less manual labor.
Find and Focus: Retrieve and Localize Video Events with Natural Language Queries
Authors: Dian Shao, Yu Xiong, Yue Zhao, Qingqiu Huang, Yu Qiao, Dahua Lin
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Diversity, Natural Language Queries, Retrieval, Sentence, Video Retrieval

The thriving of video sharing services brings new challenges to video retrieval, e.g. the rapid growth in video duration and content diversity. Meeting such challenges calls for new techniques that can effectively retrieve videos with natural language queries. Existing methods along this line, which mostly rely on embedding videos as a whole, remain far from satisfactory for real-world applications due to the limited expressive power. In this work, we aim to move beyond this limitation by delving into the internal structures of both sides, the queries and the videos. Specifically, we propose a new framework called Find and Focus (FIFO), which not only performs top-level matching (paragraph vs. video), but also makes part-level associations, localizing a video clip for each sentence in the query with the help of a focusing guide. These levels are complementary - the top-level matching narrows the search while the part-level localization refines the results. On both ActivityNet Captions and modified LSMDC datasets, the proposed framework achieves remarkable performance gains.
Evaluating Capability of Deep Neural Networks for Image Classification via Information Plane
Authors: Hao Cheng, Dongze Lian, Shenghua Gao, Yanlin Geng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, image-classification, Image Classification, Information Plane

Inspired by the pioneering work of information bottleneck principle for Deep Neural Networks (DNNs) analysis, we design an information plane based framework to evaluate the capability of DNNs for image classification tasks, which not only helps understand the capability of DNNs, but also helps us choose a neural network which leads to higher classification accuracy more efficiently. Further, with experiments, the relationship among the model accuracy, I(X;T) and I(T;Y) are analyzed, where I(X;T) and I(T;Y) are the mutual information of DNN's output T with input X and label Y. We also show the information plane is more informative than loss curve and apply mutual information to infer the model's capability for recognizing objects of each class. Our studies would facilitate a better understanding of DNNs.
SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network
Authors: Yancheng Bai, Yongqiang Zhang, Mingli Ding, Bernard Ghanem
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Generative Adversarial Network, Object, object-detection, Object Detection, regression, Small Object Detection, Super-Resolution

Object detection is a fundamental and important problem in computer vision. Although impressive results have been achieved on large/medium sized objects on large-scale detection benchmarks (e.g. the COCO dataset), the performance on small objects is far from satisfaction. The reason is that small objects lack sufficient detailed appearance information, which can distinguish them from the background or similar objects. To deal with small object detection problem, we propose an end-to-end multi-task generative adversarial network (MTGAN). In the MTGAN, the generator is a super-resolution network, which can up-sample small blurred images into fine-scale ones and recover detailed information for more accurate detection. The discriminator is a multi task network, which describes each super-resolution image patch with a real/fake score, object category scores, and bounding box regression off sets. Furthermore, to make the generator recover more details for easier detection, the classification and regression losses in the discriminator are back-propagated into the generator during training. Extensive experiments on the challenging COCO dataset demonstrate the effectiveness of the proposed method in restoring a clear super-resolution image from a blurred small one, and show that the detection performance, especially for small sized objects, improves over state-of-the-art methods.
Face Super-resolution Guided by Facial Component Heatmaps
Authors: Xin Yu, Basura Fernando, Bernard Ghanem, Fatih Porikli, Richard Hartley
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Face Hallucination, Hallucination, Super-Resolution

State-of-the-art face super-resolution methods use deep convolutional neural networks to learn a mapping between low-resolution (LR) facial patterns and their corresponding high-resolution (HR) counterparts by exploring local information. However, most of them do not account for face structure and suffer from degradations due to large pose variations and misalignments of faces. Our method incorporates structural information of faces explicitly into face super-resolution by using a multi-task convolutional neural network (CNN).  Our CNN has two branches: one for super-resolving face images and the other branch for predicting salient regions of a face coined facial component heatmaps.  These heatmaps guide the up-sampling stream for generating better super-resolved faces with high-quality details. Our method uses not only the low-level information (ie intensity similarity), but also middle-level information (ie face structure) to further explore spatial constraints of facial components from LR inputs images. Therefore, we are able to super-resolve very small unaligned face images (16$	imes$16 pixels) with a large upscaling factor of 8$	imes$ while preserving face structure.  Extensive experiments demonstrate that our network achieves superior face hallucination results and outperforms the state-of-the-art.
ML-LocNet: Improving Object Localization with Multi-view Learning Network
Authors: Xiaopeng Zhang, Yang Yang, Jiashi Feng
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: MULTI-VIEW LEARNING, Object Localization, Weakly-Supervised Object Localization

This paper addresses Weakly Supervised Object Localization (WSOL) with only image-level supervision. We propose a Multi-view Learning Localization Network (ML-LocNet) by incorporating multi-view learning into a two-phase WSOL model. The multi-view learning would benefit localization due to the complementary relationships among the learned features from different views and the consensus property among the mined instances from each view. In the first phase, the representation is augmented by integrating features learned from multiple views, and in the second phase, the model performs multi-view co-training to enhance localization performance of one view with the help of instances mined from other views, which thus effectively avoids early fitting. ML-LocNet can be easily combined with existing WSOL models to further improve the localization accuracy. Its effectiveness has been proved experimentally. Notably, it achieves 68.6% CorLoc and 49.7% mAP on PASCAL VOC 2007, surpassing the state-of-the-arts by a large margin.
Facial Expression Recognition with Inconsistently Annotated Datasets
Authors: Jiabei Zeng, Shiguang Shan, Xilin Chen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

Annotation errors and bias are inevitable among different facial expression datasets due to the subjectiveness of annotating facial expressions. Ascribe to the inconsistent annotations, performance of existing facial expression recognition (FER) methods cannot keep improving when the training set is enlarged by merging multiple datasets. To address the inconsistency, we propose an Inconsistent Pseudo Annotations to Latent Truth(IPA2LT) framework to train a FER model from multiple inconsistently labeled datasets and large scale unlabeled data. In IPA2LT, we assign each sample more than one labels with human annotations or model predictions. Then, we propose an end-to-end LTNet with a scheme of discovering the latent truth from the inconsistent pseudo labels and the input face images. To our knowledge, IPA2LT serves as the first work to solve the training problem with inconsistently labeled FER datasets. Experiments on synthetic data validate the effectiveness of the proposed method in learning from inconsistent labels. We also conduct extensive experiments in FER and show that our method outperforms other state-of-the-art and optional methods under a rigorous evaluation protocol involving 7 FER datasets.
Deformable Pose Traversal Convolution for 3D Action and Gesture Recognition
Authors: Junwu Weng, Mengyuan Liu, Xudong Jiang, Junsong Yuan
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

The representation of 3D pose plays a critical role for 3D body action and hand gesture recognition. Rather than directly representing the 3D pose using its joint locations, in this paper, we propose Deformable Pose Traversal Convolution which applies one-dimensional convolution to traverse the 3D pose to represent it. Instead of fixing the reception field when performing traversal convolution, it optimizes the convolutional kernel for each joint, by considering contextual joints with various weights. This deformable convolution can better utilize contextual joints for action and gesture recognition and is more robust to noisy joints. Moreover, by feeding the learned pose feature to a LSTM, we can perform end-to-end training which jointly optimizes 3D pose representation and temporal sequence recognition. Experiments on three benchmark datasets validate the competitive performance of our proposed method, as well as its efficiency and robustness to handle noisy pose.
What do I Annotate Next? An Empirical Study of Active Learning for Action Localization
Authors: Fabian Caba Heilbron, Joon-Young Lee, Hailin Jin, Bernard Ghanem 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Localization, Active Learning, Temporal Action Localization, Temporal Localization

Despite tremendous progress achieved in temporal action localization, state-of-the-art methods still struggle to train accurate models when annotated data is scarce. In this paper, we introduce a novel active learning framework for temporal localization that aims to mitigate this data dependency issue. We equip our framework with active selection functions that can reuse knowledge from previously annotated datasets. We study the performance of two state-of-the-art active selection functions as well as two widely used active learning baselines. To validate the effectiveness of each one of these selection functions, we conduct simulated experiments on ActivityNet. We find that using previously acquired knowledge as a bootstrapping source is crucial for active learners aiming to localize actions. When equipped with the right selection function, our proposed framework exhibits significantly better performance than standard active learning strategies, such as uncertainty sampling. Finally, we employ our framework to augment the newly compiled Kinetics action dataset with ground-truth temporal annotations. As a result, we collect Kinetics-Localization, a novel large-scale dataset for temporal action localization, which contains more than 15K YouTube videos.
Self-Calibrating Isometric Non-Rigid Structure-from-Motion
Authors: Shaifali Parashar, Adrien Bartoli, Daniel Pizarro
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

We present self-calibrating isometric non-rigid structure- from-motion (SCIso-NRSfM), the first method to reconstruct a non-rigid object from at least three monocular images with constant but unknown focal length. The majority of NRSfM methods using the perspective cam- era simply assume that the calibration is known. SCIso-NRSfM leverages the recent powerful differential approaches to NRSfM, based on formu- lating local polynomial constraints, where local means correspondence- wise. In NRSfM, the local shape may be solved from these constraints. In SCIso-NRSfM, the difficulty is to also solve for the focal length as a global variable. We propose to eliminate the shape using resultants, obtaining univariate polynomials for the focal length only, whose sum of squares can then be globally minimized. SCIso-NRSfM thus solves for the focal length by integrating the constraints for all correspondences and the whole image set. Once this is done, the local shape is easily re- covered. Our experiments show that its performance is very close to the state-of-the-art methods that use a calibrated camera.
Transductive Centroid Projection for Semi-supervised Large-scale Recognition
Authors: Yu Liu, Guanglu Song, Jing Shao, Xiao Jin, Xiaogang Wang 
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Clustering, General Classification

Conventional deep semi-supervised learning methods, such as recursive clustering and training process, suffer from cumulative error and high computational complexity when collaborating with Convolutional Neural Networks. To this end, we design a simple but effective learning mechanism that merely substitutes  the last fully-connected layer with the proposed Transductive Centroid Projection (TCP) module. It is inspired by the observation of the weights in classification layer (called 	extit{anchors}) converge to the central direction of each class in hyperspace. Specifically, we design the TCP module by dynamically adding an 	extit{ad hoc anchor} for each cluster in one mini-batch. It essentially reduces the probability of the inter-class conflict and enables the unlabelled data functioning as labelled data. We inspect its effectiveness with elaborate ablation study on seven public face/person classification benchmarks. Without any bells and whistles, TCP can achieve significant performance gains over most state-of-the-art methods in both fully-supervised and semi-supervised manners.
PSANet: Point-wise Spatial Attention Network for Scene Parsing
Authors: Hengshuang Zhao, Yi Zhang, Shu Liu, Jianping Shi, Chen Change Loy, Dahua Lin, Jiaya Jia
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Position, Scene Parsing, Semantic Segmentation

We notice information flow in convolutional neural networks is restricted inside local neighborhood regions due to the physical design of convolutional filters, which limits the overall understanding of complex scenes. In this paper, we propose the point-wise spatial attention network (PSANet) to relax the local neighborhood constraint. Each position on the feature map is connected to all the other ones through a self-adaptively learned attention mask. Moreover, information propagation in bi-direction for scene parsing is enabled. Information at other positions can be collected to help the prediction of the current position and vice versa, information at the current position can be distributed to assist the prediction of other ones. Our proposed approach achieves top performance on various competitive scene parsing datasets, including ADE20K, PASCAL VOC 2012 and Cityscapes, demonstrating its effectiveness and generality.
Robust Anchor Embedding for Unsupervised Video Person Re-Identification in the Wild
Authors: Mang Ye, Xiangyuan Lan, Pong C. Yuen
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Person Re-Identification, Representation Learning, Video-Based Person Re-Identification

This paper addresses the scalability and robustness issues of estimating labels from imbalanced unlabeled data for unsupervised video-based person re-identification (re-ID). To achieve it, we propose a novel Robust AnChor Embedding (RACE) framework via deep feature representation learning for large-scale unsupervised video re-ID. Within this framework, anchor sequences representing different persons are firstly selected to formulate an anchor graph which also initializes the CNN model to get discriminative feature representations for later label estimation. To accurately estimate labels from unlabeled sequences with noisy frames, robust anchor embedding is introduced based on the regularized affine hull. Efficiency is ensured with kNN anchors embedding instead of the whole anchor set under manifold assumptions. After that, a robust and efficient top-k counts label prediction strategy is proposed to predict the labels of unlabeled image sequences. With the newly estimated labeled sequences, the unified anchor embedding framework enables the feature learning process to be further facilitated. Extensive experimental results on the large-scale dataset show that the proposed method outperforms existing unsupervised video re-ID methods.
Semi-Supervised Deep Learning with Memory
Authors: Yanbei Chen, Xiatian Zhu, Shaogang Gong
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning, General Classification, image-classification, Image Classification, Multi-class Classification, Semi-Supervised Image Classification

We consider the semi-supervised multi-class classification problem of learning from sparse labelled and abundant unlabelled training data. To address this problem, existing semi-supervised deep learning methods often rely on the up-to-date ânetwork-in-trainingâ to formulate the semi-supervised learning objective. This ignores both the discriminative feature representation and the model inference uncertainty revealed by the network in the preceding learning iterations, referred to as the memory of model learning. In this work, we propose a novel Memory-Assisted Deep Neural Network (MA-DNN) capable of exploiting the memory of model learning to enable semi-supervised learning. Specifically, we introduce a memory mechanism into the network training process as an assimilation-accommodation interaction between the network and an external memory module. Experiments demonstrate the advantages of the proposed MA-DNN model over the state-of-the-art semi-supervised deep learning methods on three image classification benchmark datasets: SVHN, CIFAR10, and CIFAR100.
Towards End-to-End License Plate Detection and Recognition: A Large Dataset and Baseline
Authors: Zhenbo Xu, Wei Yang, Ajin Meng, Nanxue Lu, Huan Huang, Changchun Ying, Liusheng Huang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: License Plate Detection, Management, object-detection, Object Detection

Most current license plate (LP) detection and recognition approaches are evaluated on a small and usually unrepresentative dataset since there are no publicly available large diverse datasets. In this paper, we introduce CCPD, a large and comprehensive LP dataset. All images are taken manually by workers of a roadside parking management company and are annotated carefully. To our best knowledge, CCPD is the largest publicly available LP dataset to date with over 250k unique car images, and the only one provides vertices location annotations. With CCPD, we present a novel network model which can predict the bounding box and recognize the corresponding LP number simultaneously with high speed and accuracy. Through comparative experiments, we demonstrate our model outperforms current object detection and recognition approaches in both accuracy and speed. In real-world applications, our model recognizes LP numbers directly from relatively high-resolution images at over 61 fps and 98.5% accuracy.
Deep Fundamental Matrix Estimation
Authors: Rene Ranftl, Vladlen Koltun
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

 We present an approach to robust estimation of fundamental matrices from noisy data contaminated by outliers. The problem is cast as a series of weighted homogeneous least-squares problems, where robust weights are estimated using deep networks. The presented formulation acts directly on putative correspondences and thus fits into standard 3D vision pipelines that perform feature extraction, matching, and model fitting. The approach can be trained end-to-end and yields computationally efficient robust estimators. Our experiments indicate that the presented approach is able to train robust estimators that outperform classic approaches on real data by a significant margin.
Improved Structure from Motion Using Fiducial Marker Matching
Authors: Joseph DeGol, Timothy Bretl, Derek Hoiem
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9

In this paper, we present an incremental structure from motion (SfM) algorithm that signiï¬cantly outperforms existing algorithms when ï¬ducial markers are present in the scene, and that matches the performance of existing algorithms when no markers are present. Our algorithm uses markers to limit potential incorrect image matches, change the order in which images are added to the reconstruction, and enforce new bundle adjustment constraints. To validate our algorithm, we introduce a new dataset with 16 image collections of large indoor scenes with challenging characteristics (e.g., blank hallways, glass facades, brick walls) and with markers placed throughout. We show that our algorithm produces complete, accurate reconstructions on all 16 image collections, most of which cause other algorithms to fail. Further, by selectively masking ï¬ducial markers, we show that the presence of even a small number of markers can improve the results of our algorithm.
Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training
Authors: Yang Zou, Zhiding Yu, B. V. K. Vijaya Kumar, Jinsong Wang
Date: 2018-09-01 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, Image-to-Image Translation, Pseudo Label, Semantic Segmentation, Semi-Supervised Semantic Segmentation, Unsupervised Domain Adaptation

Recent deep networks achieved state of the art performanceon a variety of semantic segmentation tasks. Despite such progress, thesemodels often face challenges in real world âwild tasksâ where large differ-ence between labeled training/source data and unseen test/target dataexists. In particular, such difference is often referred to as âdomain gapâ,and  could  cause  significantly  decreased  performance  which  cannot  beeasily remedied by further increasing the representation power. Unsuper-vised domain adaptation (UDA) seeks to overcome such problem withouttarget domain labels. In this paper, we propose a novel UDA frameworkbased  on  an  iterative  self-training  (ST)  procedure,  where  the  problemis formulated as latent variable loss minimization, and can be solved byalternatively generating pseudo labels on target data and re-training themodel with these labels. On top of ST, we also propose a novel class-balanced  self-training  (CBST)  framework  to  avoid  the  gradual  domi-nance of large classes on pseudo-label generation, and introduce spatialpriors to refine generated labels. Comprehensive experiments show thatthe  proposed  methods  achieve  state  of  the  art  semantic  segmentationperformance under multiple major UDA settings.