pwc-archive/papers-with-abstracts huggingface dataset:
Points2Surf Learning Implicit Surfaces from Point Clouds
Authors: Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Niloy J. Mitra, Michael Wimmer
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Surface Reconstruction

A key step in any scanning-based asset creation workflow is to convert unordered point clouds to a surface. Classical methods (e.g. Poisson reconstruction) start to degrade in the presence of noisy and partial scans. Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans. However, such data-driven methods struggle to generalize to new shapes with large geometric and topological variations. We present Points2Surf, a novel patch-based learning framework that produces accurate surfaces directly from raw scans without normals. Learning a prior over a combination of detailed local patches and coarse global information improves generalization performance and reconstruction accuracy. Our extensive comparison on both synthetic and real data demonstrates a clear advantage of our method over state-of-the-art alternatives on previously unseen classes (on average, Points2Surf brings down reconstruction error by 30% over SPR and by 270%+ over deep learning based SotA methods) at the cost of longer computation times and a slight increase in small-scale topological noise in some cases. Our source code, pre-trained model, and dataset are available at: https://github.com/ErlerPhilipp/points2surf
Entropy Minimisation Framework for Event-based Vision Model Estimation
Authors: Urbano Miguel Nunes, Yiannis Demiris
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Event-based vision, Motion Compensation, Motion Estimation, Optical Flow Estimation

We propose a novel EMin framework for event-based vision model estimation. The framework extends previous event-based motion compensation algorithms to handle models whose outputs have arbitrary dimensions. The main motivation comes from estimating motion from events directly in 3D space (e. g. events augmented with depth), without projecting them onto an image plane. This is achieved by modelling the event alignment according to candidate parameters and minimising the resultant dispersion. We provide a family of suitable entropy loss functions and an efficient approximation whose complexity is only linear with the number of events (e. g. the complexity does not depend on the number of image pixels). The framework is evaluated on several motion estimation problems, including optical flow and rotational motion. As proof of concept, we also test our framework on 6-DOF estimation by performing the optimisation directly in 3D space.
Hallucinating Visual Instances in Total Absentia
Authors: Jiayan Qiu, Yiding Yang, Xinchao Wang, DaCheng Tao
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Hallucination, Image Inpainting, Object

In this paper, we investigate a new visual restoration task, termed as hallucinating visual instances in total absentia (HVITA). Unlike conventional image inpainting task that works on images with only part of a visual instance missing, HVITA concerns scenarios where an object is completely absent from the scene. This seemingly minor difference in fact makes the HVITA a much challenging task, as the restoration algorithm would have to not only infer the category of the object in total absentia, but also hallucinate an object of which the appearance is consistent with the background. Towards solving HVITA, we propose an end-to-end deep approach that explicitly looks into the global semantics within the image. Specifically, we transform the input image to a semantic graph, wherein each node corresponds to a detected object in the scene. We then adopt a Graph Convolutional Network on top of the scene graph to estimate the category of the missing object in the masked region, and finally introduce a Generative Adversarial Module to carry out the hallucination. Experiments on COCO, Visual Genome and NYU Depth v2 datasets demonstrate that the proposed approach yields truly encouraging and visually plausible results.
CLIFFNet for Monocular Depth Estimation with Hierarchical Embedding Loss
Authors: Lijun Wang, Jianming Zhang, Yifan Wang, Huchuan Lu, Xiang Ruan
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Depth Estimation, Monocular Depth Estimation

This paper proposes a hierarchical loss for monocular depth estimation, which measures the differences between the prediction and ground truth in hierarchical embedding spaces of depth maps. In order to find an appropriate embedding space, we design different architectures for hierarchical embedding generators (HEGs) and explore relevant tasks to train their parameters. Compared to conventional depth losses manually defined on a per-pixel basis, the proposed hierarchical loss can be learned in a data-driven manner. As verified by our experiments, the hierarchical loss even learned without additional labels can capture multi-scale context information, is more robust to local outliers, and thus delivers superior performance. To further improve depth accuracy, a cross level identity feature fusion network (CLIFFNet) is proposed, where low-level features with finer details are refined using more reliable high-level cues. Through end-to-end training, CLIFFNet can learn to select the optimal combinations of low-level and high-level features, leading to more effective cross level feature fusion. When trained using the proposed hierarchical loss, CLIFFNet sets a new state of the art on popular depth estimation benchmarks.
Adaptive Margin Diversity Regularizer for handling Data Imbalance in Zero-Shot SBIR
Authors: Titir Dutta, Anurag Singh, Soma Biswas
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Diversity, Image Retrieval, Retrieval, Sketch-Based Image Retrieval

Data from new categories are continuously being discovered, which has sparked significant amount of research in developing approaches which generalizes to previously unseen categories, i.e. zero-shot setting. Zero-shot sketch-based image retrieval~(ZS-SBIR) is one such problem in the context of cross-domain retrieval, which has received lot of attention due to its various real-life applications. Since most real-world training data have a fair amount of imbalance; in this work, for the first time in literature, we extensively study the effect of training data imbalance on the generalization to unseen categories, with ZS-SBIR as the application area. We evaluate several state-of-the-art data imbalance mitigating techniques and analyze their results. Furthermore, we propose a novel framework AMDReg (Adaptive Margin Diversity Regularizer), which ensures that the embeddings of the sketch and images in the latent space are not only semantically meaningful, but they also are separated according to their class-representations in the training set. The proposed approach is model-independent, and it can be incorporated seamlessly with several state-of-the-art ZS-SBIR methods to improve their performance under imbalanced condition. Extensive experiments and analysis justifies the effectiveness of the proposed AMDReg for mitigating the effect of data imbalance for generalization to unseen classes in ZS-SBIR.
Calibration-free Structure-from-Motion with Calibrated Radial Trifocal Tensors
Authors: Viktor Larsson, Nicolas Zobernig, Kasim Taskin, Marc Pollefeys
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8

In this paper we consider the problem of Structure-from-Motion from images with unknown intrinsic calibration. Instead of estimating the internal camera parameters through some self-calibration procedure, we propose to use a subset of the reprojection constraints that is invariant to radial displacement. This allows us to recover metric 3D reconstructions without explicitly estimating the cameras' focal length or radial distortion parameters. The weaker projection model makes initializing the reconstruction especially difficult. To handle this additional challenge we propose two novel minimal solvers for radial trifocal tensor estimation. We evaluate our approach on real images and show that even for extreme optical systems, such as fisheye or catadioptric, we are able to get accurate reconstructions without performing any calibration.
A Generalization of Otsu’s Method and Minimum Error Thresholding
Authors: Jonathan T. Barron
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Binarization

We present Generalized Histogram Thresholding (GHT), a simple, fast, and effective technique for histogram-based image thresholding. GHT works by performing approximate maximum a posteriori estimation of a mixture of Gaussians with appropriate priors. We demonstrate that GHT subsumes three classic thresholding techniques as special cases: Otsu's method, Minimum Error Thresholding (MET), and weighted percentile thresholding. GHT thereby enables the continuous interpolation between those three algorithms, which allows thresholding accuracy to be improved significantly. GHT also provides a clarifying interpretation of the common practice of coarsening a histogram's bin width during thresholding. We show that GHT outperforms or matches the performance of all algorithms on a recent challenge for handwritten document image binarization (including deep neural networks trained to produce per-pixel binarizations), and can be implemented in a dozen lines of code or as a trivial modification to Otsu's method or MET.
Character Grounding and Re-Identification in Story of Videos and Text Descriptions
Authors: Youngjae Yu, Jongseok Kim, Heeseung Yun, Jiwan Chung, Gunhee Kim
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Gender Prediction

We address character grounding and re-identification in multiple story-based videos like movies and associated text descriptions. In order to solve these related tasks in a mutually rewarding way, we propose a model named Character in Story Identification Network (CiSIN). Our method builds two semantically informative representations via joint training of multiple objectives for character grounding, video/text re-identification and gender prediction: Visual Track Embedding from videos and Textual Character Embedding from text context. These two representations are learned to retain rich semantic multimodal information that enables even simple MLPs to achieve the state-of-the-art performance on the target tasks. More specifically, our CiSIN model achieves the best performance in the Fill-in the Characters task of LSMDC 2019 challenges. Moreover, it outperforms previous state-of-the-art models in M-VAD Names dataset as a benchmark of multimodal character grounding and re-identification.
Bounding-box Channels for Visual Relationship Detection
Authors: Sho Inayoshi, Keita Otani, Antonio Tejero-de-Pablos, Tatsuya Harada
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Relationship Detection, Visual Relationship Detection

Recognizing the relationship between multiple objects in an image is essential for a deeper understanding of the meaning of the image. However, current visual recognition methods are still far from reaching human-level accuracy. Recent approaches have tackled this task by combining image features with semantic and spatial features, but the way they relate them to each other is weak, mostly because the spatial context in the image feature is lost. In this paper, we propose the bounding-box channels, a novel architecture capable of relating the semantic, spatial, and image features strongly. Our network learns bounding-box channels, which are initialized according to the position of the objects and the label of objects, and concatenated to the image features extracted from the objects. Then, they are input together to the relationship estimator. Our method can retain the spatial information in the image features, and strongly associate them with the semantic and spatial features. This way, our method is capable of effectively emphasizing the features in the object area for a better modeling of the relationships within objects. In addition, we experimentally show that our bounding-box channels have a high generalization ability. Our evaluation results show the efficacy of our architecture outperforming previous works in visual relationship detection.
Semi-supervised Semantic Segmentation via Strong-weak Dual-branch Network
Authors: Wenfeng Luo, Meng Yang
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Segmentation, Semantic Segmentation, Semi-Supervised Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

While existing works have explored a variety of techniques to push the envelop of weakly-supervised semantic segmentation, there is still a significant gap compared to the supervised methods. In real-world application, besides massive amount of weakly-supervised data there are usually a few available pixel-level annotations, based on which semi-supervised track becomes a promising way for semantic segmentation. Current methods simply bundle these two different sets of annotations together to train a segmentation network. However, we discover that such treatment is problematic and achieves even worse results than just using strong labels, which indicates the misuse of the weak ones. To fully explore the potential of the weak labels, we propose to impose separate treatments of strong and weak annotations via a strong-weak dual-branch network, which discriminates the massive inaccurate weak supervisions from those strong ones. We design a shared network component to exploit the joint discrimination of strong and weak annotations; meanwhile, the proposed dual branches separately handle full and weak supervised learning and effectively eliminate their mutual interference. This simple architecture requires only slight additional computational costs during training yet brings significant improvements over the previous methods. Experiments on two standard benchmark datasets show the effectiveness of the proposed method.
Erasing Appearance Preservation in Optimization-based Smoothing
Authors: Lvmin Zhang, Chengze Li, Yi Ji, Chunping Liu, Tien-Tsin Wong
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Image Manipulation, image smoothing

Optimization-based image smoothing is routinely formulated as the game between a smoothing energy and an appearance preservation energy. Achieving adequate smoothing is a fundamental goal of these image smoothing algorithms. We show that partially ""erasing"" the appearance preservation facilitate adequate image smoothing. In this paper, we call this manipulation as Erasing Appearance Preservation (EAP). We conduct an user study, allowing users to indicate the ""erasing"" positions by drawing scribbles interactively, to verify the correctness and effectiveness of EAP. We observe the characteristics of human-indicated ""erasing"" positions, and then formulate a simple and effective 0-1 knapsack to automatically synthesize the ""erasing"" positions. We test our synthesized erasing positions in a majority of image smoothing methods. Experimental results and large-scale perceptual human judgments show that the EAP solution tends to encourage the pattern separation or elimination capabilities of image smoothing algorithms. We further study the performance of the EAP solution in many image decomposition problems to decompose textures, shadows, and the challenging specular reflections. We also present examinations of diversiform image manipulation applications like texture removal, retexturing, intrinsic decomposition, layer extraction, recoloring, material manipulation, etc. Due to the widespread applicability of image smoothing, the EAP is also likely to be used in more image editing applications.
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampler
Authors: Tsu-Jui Fu, Xin Eric Wang, Matthew F. Peterson,Scott T. Grafton, Miguel P. Eckstein, William Yang Wang
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: counterfactual, Counterfactual Reasoning, Data Augmentation, Vision and Language Navigation

Vision-and-Language Navigation (VLN) is a task where agents must decide how to move through a 3D environment to reach a goal by grounding natural language instructions to the visual surroundings. One of the problems of the VLN task is data scarcity since it is difficult to collect enough navigation paths with human-annotated instructions for interactive environments. In this paper, we explore the use of counterfactual thinking as a human-inspired data augmentation method that results in robust models. Counterfactual thinking is a concept that describes the human propensity to create possible alternatives to life events that have already occurred. We propose an adversarial-driven counterfactual reasoning model that can consider effective conditions instead of low-quality augmented data. In particular, we present a model-agnostic adversarial path sampler (APS) that learns to sample challenging paths that force the navigator to improve based on the navigation performance. APS also serves to do pre-exploration of unseen environments to strengthen the model's ability to generalize. We evaluate the influence of APS on the performance of different VLN baseline models using the room-to-room dataset (R2R). The results show that the adversarial training process with our proposed APS benefits VLN models under both seen and unseen environments. And the pre-exploration process can further gain additional improvements under unseen environments.
RD-GAN: Few/Zero-Shot Chinese Character Style Transfer via Radical Decomposition and Rendering
Authors: Yaoxiong Huang, Mengchao He, Lianwen Jin, Yongpan Wang
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Style Transfer, Zero-shot Generalization

Style transfer has attracted much interest owing to its various applications. Compared with English character or general artistic style transfer, Chinese character style transfer remains a challenge owing to the large size of the vocabulary(70224 characters in GB18010-2005) and the complexity of the structure. Recently some GAN-based methods were proposed for style transfer; however, they treated Chinese characters as a whole, ignoring the structures and radicals that compose characters. In this paper, a novel radical decomposition-and-rendering-based GAN(RD-GAN) is proposed to utilize the radical-level compositions of Chinese characters and achieves few-shot/zero-shot Chinese character style transfer. The RD-GAN consists of three components: a radical extraction module (REM), radical rendering module (RRM), and multi-level discriminator (MLD). Experiments demonstrate that our method has a powerful few-shot/zero-shot generalization ability by using the radical-level compositions of Chinese characters.
Joint Semantic Instance Segmentation on Graphs with the Semantic Mutex Watershed
Authors: Steffen Wolf, Yuyan Li, Constantin Pape, Alberto Bailoni, Anna Kreshuk, Fred A. Hamprecht
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: graph partitioning, Instance Segmentation, Segmentation, Semantic Segmentation, Superpixels

Semantic instance segmentation is the task of simultaneously partitioning an image into distinct segments while associating each pixel with a class label. In commonly used pipelines, segmentation and label assignment are solved separately since joint optimization is computationally expensive. We propose a greedy algorithm for joint graph partitioning and labeling derived from the efficient Mutex Watershed partitioning algorithm. It optimizes an objective function closely related to the Symmetric Multiway Cut objective and empirically shows efficient scaling behavior. Due to the algorithm's efficiency, it can operate directly on pixels without prior over-segmentation of the image into superpixels. We evaluate the performance on the Cityscapes dataset (2D urban scenes) and on a 3D microscopy volume. In urban scenes, the proposed algorithm combined with current deep neural networks outperforms the strong baseline of `Panoptic Feature Pyramid Networks' by Kirillov et al. (2019). In the 3D electron microscopy images, we show explicitly that our joint formulation outperforms a separate optimization of the partitioning and labeling problems.
Photon-Efficient 3D Imaging with A Non-Local Neural Network
Authors: Jiayong Peng, Zhiwei Xiong, Xin Huang, Zheng-Ping Li, Dong Liu, Feihu Xu
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8

Photon-efficient imaging has enabled a number of applications relying on single-photon sensors that can capture a 3D image with as few as one photon per pixel. In practice, however, measurements of low photon counts are often mixed with heavy background noise, which poses a great challenge for existing computational reconstruction algorithms. In this paper, we first analyze the long-range correlations in both spatial and temporal dimensions of the measurements. Then we propose a non-local neural network for depth reconstruction by exploiting the long-range correlations. The proposed network achieves decent reconstruction fidelity even under photon counts (and signal-to-background ratio, SBR) as low as 1 photon/pixel (and 0.01 SBR), which significantly surpasses the state-of-the-art. Moreover, our non-local network trained on simulated data can be well generalized to different real-world imaging systems, which could extend the application scope of photon-efficient imaging in challenging scenarios with a strict limit on optical flux. Code is available at https://github.com/JiayongO-O/PENonLocal.
Shonan Rotation Averaging: Global Optimality by Surfing SO(p)(n)
Authors: Frank Dellaert, David M. Rosen, Jing Wu, Robert Mahony, Luca Carlone
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8

Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise. Our method employs semidefinite relaxation in order to recover provably globally optimal solutions of the rotation averaging problem. In contrast to prior work, we show how to solve large-scale instances of these relaxations using manifold minimization on (only slightly) higher-dimensional rotation manifolds, re-using existing high-performance (but local) structure-from-motion pipelines. Our method thus preserves the speed and scalability of those, while enabling the recovery of globally optimal solutions.
ProgressFace: Scale-Aware Progressive Learning for Face Detection
Authors: Jiashu Zhu, Dong Li, Tiantian Han, Lu Tian, Yi Shan
Date: 2020-08-01 00:00:00
Proceeding: ECCV 2020 8
Tasks: Face Detection

Scale variation stands out as one of key challenges in face detection. Recent attempts have been made to cope with this issue by incorporating image / feature pyramids or adjusting anchor sampling / matching strategies. In this work, we propose a novel scale-aware progressive training mechanism to address large scale variations across faces. Inspired by curriculum learning, our method gradually learns large-to-small face instances. The preceding models learned with easier samples (i.e., large faces) can provide good initialization for succeeding learning with harder samples (i.e., small faces), ultimately deriving a better optimum of face detectors. Moreover, we propose an auxiliary anchor-free enhancement module to facilitate the learning of small faces by supplying positive anchors that may be not covered according to the criterion of IoU overlap. Such anchor-free module will be removed during inference and hence no extra computation cost is introduced. Extensive experimental results demonstrate the superiority of our method compared to the state-of-the-arts on the standard FDDB and WIDER FACE benchmarks. Especially, our ProgressFace-Light with MobileNet-0.25 backbone achieves 87.9% AP on the hard set of WIDER FACE, surpassing largely RetinaFace with the same backbone by 9.7%. Code and our trained face detection models are available at https://github.com/jiashu-zhu/ProgressFace.