GenAI4ELab/papercli-papers huggingface dataset:
Can You Read Me Now? Content Aware Rectification using Angle Supervision
Authors: Amir Markovitz; Inbal Lavi; Or Perel; Shai Mazor; Roee Litman
Venue: ECCV 2020 (main)

Abstract: The ubiquity of smartphone cameras has led to more and more documents being captured by cameras rather than scanned. Unlike flatbed scanners, photographed documents are often folded and crumpled, resulting in large local variance in text structure. The problem of document rectification is fundamental to the Optical Character Recognition (OCR) process on documents, and its ability to overcome geometric distortions significantly affects recognition accuracy. Despite the great progress in recent OCR systems, most still rely on a pre-process that ensures the text lines are straight and axis aligned. Recent works have tackled the problem of rectifying document images taken in-the-wild using various supervision signals and alignment means. However, they focused on global features that can be extracted from the document's boundaries, ignoring various signals that could be obtained from the document's content. We present CREASE: Content Aware Rectification using Angle Supervision, the first learned method for document rectification that relies on the document's content, the location of the words and specifically their orientation, as hints to assist in the rectification process. We utilize a novel pixel-wise angle regression approach and a curvature estimation side-task for optimizing our rectification model. Our method surpasses previous approaches in terms of OCR accuracy, geometric error and visual similarity."
Momentum Batch Normalization for Deep Learning with Small Batch Size
Authors: Hongwei Yong; Jianqiang Huang; Deyu Meng; Xiansheng Hua; Lei Zhang
Venue: ECCV 2020 (main)

Abstract: Normalization layers play an important role in deep network training. As one of the most popular normalization techniques, batch normalization (BN) has shown its eﬀectiveness in accelerating the model training speed and improving model generalization capability. The success of BN has been explained from diﬀerent views, such as reducing internal covariate shift, allowing the use of large learning rate, smoothing optimization landscape, etc. To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size. Such a noise generation mechanism of BN regularizes the training process, and we present an explicit regularizer formulation of BN. Since the regularization strength of BN is determined by the batch size, a small batch size may cause the under-ﬁtting problem, resulting in a less eﬀective model. To reduce the dependency of BN on batch size, we propose a momentum BN (MBN) scheme by averaging the mean and variance of current mini-batch with the historical means and variances. With a dynamic momentum parameter, we can automatically control the noise level in the training process. As a result, MBN works very well even when the batch size is very small (e.g., 2), which is hard to achieve by traditional BN."
AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds
Authors: Abdullah Hamdi; Sara Rojas; Ali Thabet; Bernard Ghanem
Venue: ECCV 2020 (main)

Abstract: Deep neural networks are vulnerable to adversarial attacks, in which imperceptible perturbations to their input lead to erroneous network predictions. This phenomenon has been extensively studied in the image domain, and has only recently been extended to 3D point clouds. In this work, we present novel data-driven adversarial attacks against 3D point cloud networks. We aim to address the following problems in current 3D point cloud adversarial attacks: they do not transfer well between different networks, and they are easy to defend against via simple statistical methods. To this extent, we develop a new point cloud attack (dubbed AdvPC) that exploits the input data distribution by adding an adversarial loss, after Auto-Encoder reconstruction, to the objective it optimizes. AdvPC leads to perturbations that are resilient against current defenses, while remaining highly transferable compared to state-of-the-art attacks. We test AdvPC using four popular point cloud networks: PointNet, PointNet++ (MSG and SSG), and DGCNN. Our proposed attack increases the attack success rate by up to 40% for those transferred to unseen networks (transferability), while maintaining a high success rate on the attacked network. AdvPC also increases the ability to break defenses by up to 38% as compared to other baselines on the ModelNet40 dataset. The code is available at https://github.com/ajhamdi/AdvPC ."
Edge-aware Graph Representation Learning and Reasoning for Face Parsing
Authors: Gusi Te; Yinglu Liu; Wei Hu; Hailin Shi; Tao Mei
Venue: ECCV 2020 (main)

Abstract: Face parsing infers a pixel-wise label to each facial component, which has drawn much attention recently. Previous methods have shown their efficiency in face parsing, which however overlook the correlation among different face regions. The correlation is a critical clue about the facial appearance, pose, expression etc., and should be taken into account for face parsing. To this end, we propose to model and reason the region-wise relations by learning graph representations, and leverage the edge information between regions for optimized abstraction. Specifically, we encode a facial image onto a global graph representation where a collection of pixels (""regions"") with similar features are projected to each vertex. Our model learns and reasons over relations between the regions by propagating information across vertices on the graph. Furthermore, we incorporate the edge information to aggregate the pixel-wise features onto vertices, which emphasizes on the features around edges for fine segmentation along edges. The finally learned graph representation is projected back to pixel grids for parsing. Experiments demonstrate that our model outperforms state-of-the-art methods on the widely used Helen dataset, and also exhibits the superior performance on the large-scale CelebAMask-HQ dataset."
BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network
Authors: Deng-Ping Fan; Yingjie Zhai; Ali Borji; Jufeng Yang; Ling Shao
Venue: ECCV 2020 (main)

Abstract: Multi-level feature fusion is a fundamental topic in computer vision for detecting, segmenting, and classifying objects at various scales. When multi-level features meet multi-modal cues, the optimal fusion problem becomes a hot potato. In this paper, we make the first attempt to leverage the inherent multi-modal and multi-level nature of RGB-D salient object detection to develop a novel cascaded refinement network. In particular, we 1) propose a bifurcated backbone strategy (BBS) to split the multi-level features into teacher and student features, and 2) utilize a depth-enhanced module (DEM) to excavate informative parts of depth cues from the channel and spatial views. This fuses RGB and depth modalities in a complementary way. Our simple yet efficient architecture, dubbed Bifurcated Backbone Strategy Network (BBS-Net), is backbone independent, and outperforms 18 SOTAs on seven challenging datasets using four metrics."
G-LBM:Generative Low-dimensional Background Model Estimation from Video Sequences
Authors: Behnaz Rezaei; Amirreza Farnoosh; Sarah Ostadabbas
Venue: ECCV 2020 (main)

Abstract: In this paper, we propose a computationally tractable and theoretically supported non-linear low-dimensional generative model to represent real-world data in the presence of noise and sparse outliers. The non-linear low-dimensional manifold discovery of data is done through describing a joint distribution over observations, and their low-dimensional representations (i.e. manifold coordinates). Our model, called generative low-dimensional background model (G-LBM) admits variational operations on the distribution of the manifold coordinates and simultaneously generates a low-rank structure of the latent manifold given the data. Therefore, our probabilistic model contains the intuition of the non-probabilistic low-dimensional manifold learning. G-LBM selects the intrinsic dimensionality of the underling manifold of the observations, and its probabilistic nature models the noise in the observation data. G-LBM has direct application in the background scenes model estimation from video sequences and we have evaluated its performance on SBMnet-2016 and BMC2012 datasets, where it achieved a performance higher or comparable to other state-of-the-art methods while being agnostic to the background scenes in videos. Besides, in challenges such as camera jitter and background motion, G-LBM is able to robustly estimate the background by effectively modeling the uncertainties in video observations in these scenarios."
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives
Authors: Zaiwei Zhang; Bo Sun; Haitao Yang; Qixing Huang
Venue: ECCV 2020 (main)

Abstract: We introduce H3DNet, which takes a colorless 3D point cloud as input and outputs a collection of oriented object bounding boxes (or BB) and their semantic labels. The critical idea of H3DNet is to predict a hybrid set of geometric primitives, i.e., BB centers, BB face centers, and BB edge centers. We show how to convert the predicted geometric primitives into object proposals by defining a distance function between an object and the geometric primitives. This distance function enables continuous optimization of object proposals, and its local minimums provide high-fidelity object proposals. H3DNet then utilizes a matching and refinement module to classify object proposals into detected objects and fine-tune the geometric parameters of the detected objects. The hybrid set of geometric primitives not only provides more accurate signals for object detection than using a single type of geometric primitives, but it also provides an overcomplete set of constraints on the resulting 3D layout. Therefore, H3DNet can tolerate outliers in predicted geometric primitives. Our model achieves state-of-the-art 3D detection results on two large datasets with real 3D scans, ScanNet and SUN RGB-D."
Expressive Telepresence via Modular Codec Avatars
Authors: Hang Chu; Shugao Ma; Fernando De la Torre; Sanja Fidler; Yaser Sheikh
Venue: ECCV 2020 (main)

Abstract: VR telepresence consists of interacting with another human in a virtual space represented by an avatar. Today most avatars are cartoon-like, but soon the technology will allow video-realistic ones. This paper aims in this direction and presents Modular Codec Avatars (MCA), a method to generate hyper-realistic faces driven by the cameras in the VR headset. MCA extends traditional Codec Avatars (CA) by replacing the holistic models with a learned modular representation. It is important to note that traditional person-specific CAs are learned from few training samples, and typically lack robustness as well as limited expressiveness when transferring facial expressions. MCAs solve these issues by learning a modulated adaptive blending of different facial components as well as an exemplar-based latent alignment. We demonstrate that MCA achieves improved expressiveness and robustness w.r.t to CA in a variety of real-world datasets and practical scenarios. Finally, we showcase new applications in VR telepresence enabled by the proposed model."
Cascade Graph Neural Networks for RGB-D Salient Object Detection
Authors: Ao Luo; Xin Li; Fan Yang; Zhicheng Jiao; Hong Cheng; Siwei Lyu
Venue: ECCV 2020 (main)

Abstract: In this paper, we study the problem of salient object detection for RGB-D images by using both color and depth information. A major technical challenge for detecting salient objects in RGB-D images is to fully leverage the two complementary data sources. The existing works either simply distill prior knowledge from the corresponding depth map to handle the RGB-image or blindly fuse color and geometric information to generate the depth-aware representations, hindering the performance of RGB-D saliency detectors. In this work, we introduce Cascade Graph Neural Networks (Cas-Gnn), a unified framework which is capable of comprehensively distilling and reasoning the mutual benefit between these two data sources through a set of cascade graphs, to learn powerful representations for RGB-D salient object detection. Cas-Gnn processes the two data sources separately and employs a novel Cascade Graph Reasoning (CGR) module to learn the powerful dense feature embeddings so that the saliency map can be easily inferred. Different from previous approaches, Cas-Gnn, by explicitly modeling and reasoning high-level relations between complementary data sources, can overcome many challenges like occlusions and ambiguities. Extensive experiments on several widely-used benchmarks demonstrate that CasGnn achieves significantly better performance than all existing RGB-D SOD approaches."
FairALM: Augmented Lagrangian Method for Training Fair Models with Little Regret
Authors: Vishnu Suresh Lokhande; Aditya Kumar Akash; Sathya N. Ravi; Vikas Singh
Venue: ECCV 2020 (main)

Abstract: Algorithmic decision making based on computer vision and machine learning technologies continue to permeate our lives. But issues related to biases of these models and the extent to which they treat certain segments of the population unfairly, have led to concern in the general public. It is now accepted that because of biases in the datasets we present to the models, a fairness-oblivious training will lead to unfair models. An interesting topic is the study of mechanisms via which the de novo design or training of the model can be informed by fairness measures. Here, we study mechanisms that impose fairness concurrently while training the model. While existing fairness based approaches in vision have largely relied on training adversarial modules together with the primary classification/regression task, in an effort to remove the influence of the protected attribute or variable, we show how ideas based on well-known optimization concepts can provide a simpler alternative. In our proposed scheme, imposing fairness just requires specifying the protected attribute and utilizing our optimization routine. We provide a detailed technical analysis and present experiments demonstrating that various fairness measures from the literature can be reliably imposed on a number of training tasks in vision in a manner that is interpretable."
Generating Videos of Zero-Shot Compositions of Actions and Objects
Authors: Megha Nawhal; Mengyao Zhai; Andreas Lehrmann; Leonid Sigal; Greg Mori
Venue: ECCV 2020 (main)

Abstract: Human activity videos involve rich, varied interactions between people and objects. In this paper we develop methods for generating such videos -- making progress toward addressing the important, open problem of video generation in complex scenes. In particular, we introduce the task of generating human-object interaction videos in a zero-shot compositional setting, i.e., generating videos for action-object compositions that are unseen during training, having seen the target action and target object separately. This setting is particularly important for generalization in human activity video generation, obviating the need to observe every possible action-object combination in training and thus avoiding the combinatorial explosion involved in modeling complex scenes. To generate human-object interaction videos, we propose a novel adversarial framework HOI-GAN which includes multiple discriminators focusing on different aspects of a video. To demonstrate the effectiveness of our proposed framework, we perform extensive quantitative and qualitative evaluation on two challenging datasets: EPIC-Kitchens and 20BN-Something-Something v2."
ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language
Authors: Zhe Wang; Zhiyuan Fang; Jun Wang; Yezhou Yang
Venue: ECCV 2020 (main)

Abstract: Person search by natural language aims at retrieving a specific person in a large-scale image pool that matches given textual descriptions. While most of the current methods treat the task as a holistic visual and textual feature matching one, we approach it from an attribute-aligning perspective that allows grounding specific attribute phrases to the corresponding visual regions. We achieve success as well as a performance boost by a robust feature learning that the referred identity can be accurately bundled by multiple attribute cues. To be concrete, our Visual-Textual Attribute Alignment model (dubbed as ViTAA) learns to disentangle the feature space of a person into sub-spaces corresponding to attributes using a light auxiliary attribute segmentation layer. It then aligns these visual features with the textual attributes parsed from the sentences via a novel contrastive learning loss. We validate our ViTAA framework through extensive experiments on tasks of person search by natural language and by attribute-phrase queries, on which our system achieves state-of-the-art performances."
Renovating Parsing R-CNN for Accurate Multiple Human Parsing
Authors: Lu Yang; Qing Song; Zhihui Wang; Mengjie Hu; Chun Liu; Xueshi Xin; Wenhe Jia; Songcen Xu
Venue: ECCV 2020 (main)

Abstract: Multiple human parsing aims to segment various human parts and associate each part with the corresponding instance simultaneously. This is a very challenging task due to the diverse human appearance, semantic ambiguity of different body parts and clothing, and complex background. Through analysis of human parsing task, we observe that human-centric context perception and accurate instance-level parsing scoring are particularly important for obtaining high-quality results. But the most state-of-the-art methods have not paid enough attention to these problems. To reverse this phenomenon, we present Renovating Parsing R-CNN (RP R-CNN), which introduces a global semantic enhanced feature pyramid network and a parsing re-scoring network into the existing high-performance pipeline. The proposed RP R-CNN adopts global semantic feature to enhance multi-scale features for generating human parsing, and regresses a confidence score to represent its quality. Extensive experiments show that RP R-CNN performs favorably against state-of-the-art methods on CIHP and MHP-v2 datasets. Code and models will be publicly available."
Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning
Authors: Qing Yu; Daiki Ikami; Go Irie; Kiyoharu Aizawa
Venue: ECCV 2020 (main)

Abstract: Semi-supervised learning (SSL) has been proposed to leverage unlabeled data for training powerful models when only limited labeled data is available. While existing SSL methods assume that samples in the labeled and unlabeled data share the classes of their samples, we address a more complex novel scenario named open-set SSL, where out-of-distribution (OOD) samples are contained in unlabeled data. Instead of training an OOD detector and SSL separately, we propose a multi-task curriculum learning framework. First, to detect the OOD samples in unlabeled data, we estimate the probability of the sample belonging to OOD. We use a joint optimization framework, which updates the network parameters and the OOD score alternately. Simultaneously, to achieve high performance on the classification of in-distribution (ID) data, we select ID samples in unlabeled data having small OOD scores, and use these data with labeled data for training the deep neural networks to classify ID samples in a semi-supervised manner. We conduct several experiments, and our method achieves state-of-the-art results by successfully eliminating the effect of OOD samples."
Gradient-Induced Co-Saliency Detection
Authors: Zhao Zhang; Wenda Jin; Jun Xu; Ming-Ming Cheng
Venue: ECCV 2020 (main)

Abstract: Co-saliency detection (Co-SOD) aims to segment the common salient foreground in a group of relevant images. In this paper, inspired by human behavior, we propose a gradient-induced co-saliency detection (GICD) method. We first abstract a consensus representation for the grouped images in the embedding space; then, by comparing the single image with consensus representation, we utilize the feedback gradient information to induce more attention to the discriminative co-salient features. In addition, due to the lack of Co-SOD training data, we design a jigsaw training strategy, with which Co-SOD networks can be trained on general saliency datasets without extra pixel-level annotations. To evaluate the performance of Co-SOD methods on discovering the co-salient object among multiple foregrounds, we construct a challenging CoCA dataset, where each image contains at least one extraneous foreground along with the co-salient object. Experiments demonstrate that our GICD achieves state-of-the-art performance. Our codes and dataset are available at https://mmcheng.net/gicd/."
Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks
Authors: Wending Yan; Robby T. Tan; Dengxin Dai
Venue: ECCV 2020 (main)

Abstract: In this paper, we address the problem of nighttime defogging from a single image. We propose a framework consisting of two main modules: grayscale and color modules. Given an RGB foggy nighttime image, our grayscale module takes the grayscale version of the image as input, and decomposes it into high and low frequency layers. The high frequency layer contains the scene texture information, which is less affected by fog; and, the low frequency layer contains the scene layout/structure information including fog and glow. Our grayscale module then enhances the visibility of the textures in the high frequency layers, and removes the presence of glow and fog in the low frequency layers. Having processed the high/low frequency information, it fuses the layers to obtain a grayscale defogged image. Our second module, the color module, takes the original RGB image as input. The module has similar operations to those of the grayscale module. However, to obtain fog-free high and low frequency information, the module is guided by the grayscale module. The reason of doing this is because grayscale images are less affected by multiple colors of atmospheric light, which are commonly present in nighttime scenes. Moreover, having the grayscale module allows us to have consistency losses between the outputs of the two modules, which is critical to our framework, since we do not have paired ground-truths for our real data. Our extensive experiments on real foggy nighttime images show the effectiveness of our method."
SegFix: Model-Agnostic Boundary Refinement for Segmentation
Authors: Yuhui Yuan; Jingyi Xie; Xilin Chen; Jingdong Wang
Venue: ECCV 2020 (main)

Abstract: We present a model-agnostic post-processing scheme to improve the boundary quality for the segmentation result that is generated by any existing segmentation model. Motivated by the empirical observation that the label predictions of interior pixels are more reliable, we propose to replace the originally unreliable predictions of boundary pixels by the predictions of interior pixels. Our approach processes only the input image through two steps: (i) localize the boundary pixels and (ii) identify the corresponding interior pixel for each boundary pixel. We build the correspondence by learning a direction away from the boundary pixel to an interior pixel. Our method requires no prior information of the segmentation models and achieves nearly real-time speed. We empirically verify that our SegFix consistently reduces the boundary errors for segmentation results generated from various state-of-the-art models on Cityscapes, ADE20K and GTA5. "