pwc-archive/papers-with-abstracts huggingface dataset:
Task-conditioned Domain Adaptation for Pedestrian Detection in Thermal Imagery
Authors: My Kieu, Andrew D. Bagdanov, Marco Bertini, Alberto del Bimbo
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Domain Adaptation, Pedestrian Detection

Pedestrian detection is a core problem in computer vision that sees broad application in video surveillance and, more recently, in advanced driving assistance systems. Despite its broad application and interest, it remains a challenging problem in part due to the vast range of conditions under which it must be robust. Pedestrian detection at nighttime and during adverse weather conditions is particularly challenging, which is one of the reasons why thermal and multispectral approaches have been become popular in recent years. In this paper, we propose a novel approach to domain adaptation that significantly improves pedestrian detection performance in the thermal domain. The key idea behind our technique is to adapt an RGB-trained detection network to simultaneously solve two related tasks. An auxiliary classification task that distinguishes between daytime and nighttime thermal images is added to the main detection task during domain adaptation. The internal representation learned to perform this classification task is used to condition a YOLOv3 detector at multiple points in order to improve its adaptation to the thermal domain. We validate the effectiveness of task-conditioned domain adaptation by comparing with the state-of-the-art on the KAIST Multispectral Pedestrian Detection Benchmark. To the best of our knowledge, our proposed task-conditioned approach achieves the best single-modality detection results.
Splitting vs. Merging: Mining Object Regions with Discrepancy and Intersection Loss for Weakly Supervised Semantic Segmentation
Authors: Tianyi Zhang, Guosheng Lin, Weide Liu, Jianfei Cai, Alex Kot
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Segmentation, Semantic Segmentation, Weakly supervised segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

In this paper we focus on the task of weakly-supervised semantic segmentation supervised with image-level labels. Since the pixel-level annotation is not available in the training process, we rely on region mining models to estimate the pseudo-masks from the image-level labels. Thus, in order to improve the final segmentation results, we aim to train a region-mining model which could accurately and completely highlight the target object regions for generating high-quality pseudo-masks. However, the region mining models are likely to only highlight the most discriminative regions instead of the entire objects. In this paper, we aim to tackle this problem from a novel perspective of optimization process. We propose a Splitting vs. Merging optimization strategy, which is mainly composed of the Discrepancy loss and the Intersection loss. The proposed Discrepancy loss aims at mining out regions of diﬀerent spatial patterns instead of only the most discriminative region, which leads to the splitting eﬀect. The Intersection loss aims at mining the common regions of the diﬀerent maps, which leads to the merging eﬀect. Our Splitting vs. Merging strategy helps to expand the output heatmap of the region mining model to the object scale. Finally, by training the segmentation model with the masks generated by our Splitting vs Merging strategy, we achieve the state-of-the-art weakly-supervised segmentation results on the Pascal VOC 2012 benchmark.
Temporal Keypoint Matching and Refinement Network for Pose Estimation and Tracking
Authors: Chunluan Zhou Zhou Ren Gang Hua
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Human Detection, Multi-Person Pose Estimation, Multi-Person Pose Estimation and Tracking, Pose Estimation

Multi-person pose estimation and tracking in realistic videos is very challenging due to factors such as occlusions, fast motion and pose variations. Top-down approaches are commonly used for this task, which involves three stages: person detection, single-person pose estimation, and pose association across time. Recently, significant progress has been made in person detection and single-person pose estimation. In this paper, we mainly focus on improving pose association and estimation in a video to build a strong pose estimator and tracker. To this end, we propose a novel temporal keypoint matching and refinement network. Specifically, we propose two network modules, temporal keypoint matching and temporal keypoint refinement, which are incorporated into a single-person pose estimatin network. The temporal keypoint matching module learns a simialrity metric for matching keypoints across frames. Pose matching is performed by aggregating keypoint similarities between poses in adjacent frames. The temporal keypoint refinement module serves to correct individual poses by utilizing their associated poses in neighboring frames as temporal context. We validate the effectiveness of our proposed network on two benchmark datasets: PoseTrack 2017 and PoseTrack 2018. Exprimental results show that our approach achieves state-of-the-art performance on both datasets.
FHDe²Net: Full High Definition Demoireing Network
Authors: Bin He, Ce Wang, Boxin Shi, Ling-Yu Duan
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Diversity, Vocal Bursts Intensity Prediction

Frequency aliasing in the digital capture of display screens leads to the moir´e pattern, appearing as stripe-shaped distortions in images. Efforts to demoir´eing have been made recently in a learning fashion due to the complexity and diversity of the pattern appearance. However, existing methods cannot satisfy the practical demand of demoir´eing on camera phone capturing more pixels than a full high definition (FHD) image, which poses additional challenges of wider pattern scale range and fine detail preservation. We propose the Full High Definition Demoir´eing Network (FHDe$^2$Net) to solve such problems. The framework consists of a global to local cascaded removal branch to eradicate multi-scale moir´e patterns and a frequency based high resolution content separation branch to retain fine details. We further collect an FHD moir´e image dataset as a new benchmark for training and evaluation. Comparison experiments and ablation studies have verified the effectiveness of the proposed framework and each functional module both quantitatively and qualitatively in practical application scenarios.
Learning Structural Similarity of User Interface Layouts using Graph Networks
Authors: Dipu Manandhar, Dan Ruta, John Collomosse
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Decoder, Representation Learning, Triplet

We propose a novel representation learning technique for measuring the similarity of user interface designs. A triplet network is used to learn a search embedding for layout similarity, with a hybrid encoder-decoder backbone comprising a graph convolutional network (GCN) and convolutional decoder (CNN). The properties of interface components and their spatial relationships are encoded via a graph which also models the containment (nesting) relationships of interface components. We supervise the training of a dual reconstruction and pair-wise loss using an auxiliary measure of layout similarity based on intersection over union (IoU) distance. The resulting embedding is shown to exceed state of the art performance for visual search of user interface layouts over the public Rico dataset, and an auto-annotated dataset of interface layouts collected from the web.
Margin-Mix: Semi–Supervised Learning for Face Expression Recognition
Authors: Corneliu Florea, Mihai Badea, Laura Florea, Andrei Racoviteanu, Constantin Vertan
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8

In this paper, as we aim to construct a semi-supervised learning algorithm, we exploit the characteristics of the Deep Convolutional Networks to provide, for an input image, both an embedding descriptor and a prediction. The unlabeled data is combined with the labeled one in order to provide synthetic data, which describes better the input space. The network is asked to provide a large margin between clusters, while new data is self-labeled by the distance to class centroids, in the embedding space. The method is tested on standard benchmarks for semi--supervised learning, where it matches state of the art performance and on the problem of face expression recognition where it increases the accuracy by a noticeable margin.
Principal Feature Visualisation in Convolutional Neural Networks
Authors: Marianne Bakken, Johannes Kvam, Alexey A. Stepanov, Asbjørn Berge
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Transfer Learning

We introduce a new visualisation technique for CNNs called Principal Feature Visualisation (PFV). It uses a single forward pass of the original network to map principal features from the final convolutional layer to the original image space as RGB channels. By working on a batch of images we can extract contrasting features, not just the most dominant ones with respect to the classification. This allows us to differentiate between several features in one image in an unsupervised manner. This enables us to assess the feasibility of transfer learning and to debug a pre-trained classifier by localising misleading or missing features.
Progressive Refinement Network for Occluded Pedestrian Detection
Authors: Xiaolin Song Kaili Zhao Wen-Sheng Chu Honggang Zhang Jun Guo
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Body Detection, Pedestrian Detection

We present Progressive Refinement Network (PRNet), a novel single-stage detector that tackles occluded pedestrian detection. Motivated by human's progressive process on annotating occluded pedestrians, PRNet achieves sequential refinement by three phases: Finding high-confident anchors of visible parts, calibrating such anchors to a full-body template derived from occlusion statistics, and then adjusting the calibrated anchors to final full-body regions. Unlike conventional methods that exploit predefined anchors, the confidence-aware calibration offers adaptive anchor initialization for detection with occlusions, and helps reduce the gap between visible-part and full-body detection. In addition, we introduce an occlusion loss to up-weigh hard examples, and a Receptive Field Backfeed (RFB) module to diversify receptive fields in early layers that commonly fire only on visible parts or small-size full-body regions. Experiments were performed within and across CityPersons, ETH, and Caltech datasets. Results show that PRNet can match the speed of existing single-stage detectors, consistently outperforms alternatives in terms of overall miss rate, and offers significantly better cross-dataset generalization. Code is available.
SEN: A Novel Feature Normalization Dissimilarity Measure for Prototypical Few-Shot Learning Networks
Authors: Van Nhan Nguyen, Sigurd Løkse, Kristoffer Wickstrøm, Michael Kampffmeyer, Davide Roverso, Robert Jenssen
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Few-Shot Learning

In this paper, we equip Prototypical Networks (PNs) with a novel dissimilarity measure to enable discriminative feature normalization for few-shot learning. The embedding onto the hypersphere requires no direct normalization and is easy to optimize. Our theoretical analysis shows that the proposed dissimilarity measure, denoted the Squared root of the Euclidean distance and the Norm distance (SEN), forces embedding points to be attracted to its correct prototype, while being repelled from all other prototypes, keeping the norm of all points the same. The resulting SEN PN outperforms the regular PN with a considerable margin, with no additional parameters as well as with negligible computational overhead.
SACA Net: Cybersickness Assessment of Individual Viewers for VR Content via Graph-based Symptom Relation Embedding
Authors: Sangmin Lee, Jung Uk Kim, Hak Gu Kim, Seongyeop Kim, Yong Man Ro
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Relation

Recently, cybersickness assessment for VR content is required to deal with viewing safety issues. Assessing physical symptoms of individual viewers is challenging but important to provide detailed and personalized guides for viewing safety. In this paper, we propose a novel symptom-aware cybersickness assessment network (SACA Net) that quantifies physical symptom levels for assessing cybersickness of individual viewers. The SACA Net is designed to utilize the relational characteristics of symptoms for complementary effects among relevant symptoms. The proposed network consists of three main parts: a stimulus symptom context guider, a physiological symptom guider, and a symptom relation embedder. The stimulus symptom context guider and the physiological symptom guider extract symptom features from VR content and human physiology, respectively. The symptom relation embedder refines the stimulus-response symptom features to effectively predict cybersickness by embedding relational characteristics with graph formulation. For validation, we utilize two public 360-degree video datasets that contain cybersickness scores and physiological signals. Experimental results show that the proposed method is effective in predicting human cybersickness with physical symptoms. Further, latent relations among symptoms are interpretable by analyzing relational weights in the proposed network.
End-to-End Low Cost Compressive Spectral Imaging with Spatial-Spectral Self-Attention
Authors: Ziyi Meng, Jiawei Ma, Xin Yuan
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Decoder, Spectral Reconstruction

Coded aperture snapshot spectral imaging (CASSI) is an effective tool to capture real-world 3D hyperspectral images. While a number of existing work has been conducted for hardware and algorithm design, we make a step towards the low-cost solution that enjoys video-rate high-quality reconstruction. To make solid progress on this challenging yet under-investigated task, we reproduce a stable single disperser (SD) CASSI system to gather large-scale real-world CASSI data and propose a novel deep convolutional network to carry out the real-time reconstruction by using self-attention. In order to jointly capture the self-attention across different dimensions in hyperspectral images (i.e., channel-wise spectral correlation and non-local spatial regions), we propose Spatial-Spectral Self-Attention (TSA) to process each dimension sequentially, yet in an order-independent manner. We employ TSA in an encoder-decoder network, dubbed TSA-Net, to reconstruct the desired 3D cube. Furthermore, we investigate how noise affects the results and propose to add shot noise in model training, which improves the real data results significantly. We hope our large-scale CASSI data serve as a benchmark in future research and our TSA model as a baseline in deep learning based reconstruction algorithms.
DeepLandscape: Adversarial Modeling of Landscape Videos
Authors: Elizaveta Logacheva, Roman Suvorov, Oleg Khomenko, Anton Mashikhin, Victor Lempitsky
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Video Generation

We build a new model of landscape videos that can be trained on a mixture of static landscape images as well as landscape animations. Our architecture extends StyleGAN model by augmenting it with parts that allow to model dynamic changes in a scene. Once trained, our model can be used to generate realistic time-lapse landscape videos with moving objects and time-of-the-day changes. Furthermore, by fitting the learned models to a static landscape image, the latter can be reenacted in a realistic way. We propose simple but necessary modifications to StyleGAN inversion procedure, which lead to in-domain latent codes and allow to manipulate real images. Quantitative comparisons and user studies suggest that our model produces more compelling animations of given photographs than previously proposed methods. The results of our approach including comparisons with prior art can be seen in supplementary materials and on the project page https://saic-mdal.github.io/deep-landscape/.
Spatial-Angular Interaction for Light Field Image Super-Resolution
Authors: Yingqian Wang,  Longguang Wang,  Jungang Yang, Wei An,  Jingyi Yu,  Yulan Guo
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Image Super-Resolution, SSIM, Super-Resolution

Light field (LF) cameras record both intensity and directions of light rays, and capture scenes from a number of viewpoints. Both information within each perspective (i.e., spatial information) and among different perspectives (i.e., angular information) is beneficial to image super-resolution (SR). In this paper, we propose a spatial-angular interactive network (namely, LF-InterNet) for LF image SR. Specifically, spatial and angular features are first separately extracted from input LFs, and then repetitively interacted to progressively incorporate spatial and angular information. Finally, the interacted features are fused to superresolve each sub-aperture image. Experimental results demonstrate the superiority of LF-InterNet over the state-of-the-art methods, i.e., our method can achieve high PSNR and SSIM scores with low computational cost, and recover faithful details in the reconstructed images.
Adaptive Variance Based Label Distribution Learning For Facial Age Estimation
Authors: Xin Wen, Biying Li, Haiyun Guo, Zhiwei Liu, Guosheng Hu, Ming Tang, Jinqiao Wang
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Age Estimation, Meta-Learning, MORPH

Estimating age from a single facial image is a classic and challenging topic in computer vision. One of its most intractable issues is label ambiguity, i.e., face images from adjacent age of the same person are often indistinguishable. Some existing methods adopt distribution learning to tackle this issue by exploiting the semantic correlation between age labels. Actually, most of them set a fixed value to the variance of Gaussian label distribution for all the images. However, the variance is closely related to the correlation between adjacent ages and should vary across ages and identities. To model a sample-specific variance, in this paper, we propose an adaptive variance based distribution learning (AVDL) method for facial age estimation. AVDL introduces the data-driven optimization framework, meta-learning, to achieve this. Specifically, AVDL performs a meta gradient descent step on the variable (i.e. variance) to minimize the loss on a clean unbiased validation set. By adaptively learning proper variance for each sample, our method can approximate the true age probability distribution more effectively. Extensive experiments on FG-NET and MORPH II datasets show the superiority of our proposed approach to the existing state-of-the-art methods.
VarSR: Variational Super-Resolution Network for Very Low Resolution Images
Authors: Sangeek Hyun, Jae-Pil Heo
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Image Super-Resolution, Super-Resolution

As is well known, single image super-resolution (SR) is an ill-posed problem where multiple high resolution (HR) images can be matched to one low resolution (LR) image due to the difference of their representation capabilities. Such many-to-one nature is particularly magnified when super-resolving with large upscaling factors from very low dimensional domains such as 8$ imes$8 resolution where detailed information of HR is hardly discovered. Most existing methods are optimized for deterministic generation of SR images under pre-defined objectives such as pixel-level reconstruction and thus limited to the one-to-one correspondence between LR and SR images against the nature. In this paper, we propose VarSR, Variational Super Resolution Network, that matches latent distributions of LR and HR images to recover the missing details. Specifically, we draw samples from the learned common latent distribution of LR and HR to generate diverse SR images as the many-to-one relationship. Experimental results validate that our method can produce more accurate and perceptually plausible SR images from very low resolutions compared to the deterministic techniques.
From Image to Stability: Learning Dynamics from Human Pose
Authors: Jesse Scott, Bharadwaj Ravichandran, Christopher Funk, Robert T. Collins, Yanxi Liu
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8

We propose and validate two end-to-end deep learning architectures to learn foot pressure distribution maps (dynamics) from 2D or 3D human pose (kinematics). The networks are trained using 1.36 million synchronized pose+pressure data pairs from 10 subjects performing multiple takes of a 5-minute long choreographed Taiji sequence. Using leave-one-subject-out cross validation, we demonstrate reliable and repeatable foot pressure prediction, setting the first baseline for solving a non-obvious pose to pressure cross-modality mapping problem in computer vision. Furthermore, we compute and quantitatively validate Center of Pressure (CoP) and Base of Support (BoS), two key components for stability analysis, from the predicted foot pressure distributions.
LevelSet R-CNN: A Deep Variational Method for Instance Segmentation
Authors: Namdar Homayounfar Yuwen Xiong Justin Liang Wei-Chiu Ma Raquel Urtasun {namdar,yuwen,justin.liang,weichiu,urtasun}@uber.com
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Autonomous Driving, Instance Segmentation, Segmentation, Semantic Segmentation

Obtaining precise instance segmentation masks is of high importance in many modern applications such as robotic manipulation and autonomous driving. Currently, many state of the art models are based on the Mask R-CNN framework which, while very powerful, outputs masks at low resolutions which could result in imprecise boundaries. On the other hand, classic variational methods for segmentation impose desirable global and local data and geometry constraints on the masks by optimizing an energy functional. While mathematically elegant, their direct dependence on good initialization, non-robust image cues and manual setting of hyperparameters renders them unsuitable for modern applications. We propose LevelSet R-CNN, which combines the best of both worlds by obtaining powerful feature representations that are combined in an end-to-end manner with a variational segmentation framework. We demonstrate the effectiveness of our approach on COCO and Cityscapes datasets.
Reducing Distributional Uncertainty by Mutual Information Maximisation and Transferable Feature Learning
Authors: Jian Gao, Yang Hua, Guosheng Hu, Chi Wang, Neil M. Robertson
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Domain Adaptation

Distributional uncertainty exists broadly in many real-world applications, one of which in the form of domain discrepancy. Yet in the existing literature, the mathematical definition of it is missing. In this paper, we propose to formulate the distributional uncertainty both between the source(s) and target domain(s) and within each domain using mutual information. Further, to reduce distributional uncertainty (e.g. domain discrepancy), we (1) maximise the mutual information between source and target domains and (2) propose a transferable feature learning scheme, balancing two complementary and discriminative feature learning processes (general texture learning and self-supervised transferable shape learning) according to the uncertainty. We conduct extensive experiments on both domain adaption and domain generalisation using challenging common benchmarks: Office-Home and DomainNet. Results show the great effectiveness of the proposed method and its superiority over the state-of-the-art methods.