pwc-archive/papers-with-abstracts huggingface dataset:
Explainable Deep Reinforcement Learning Using Introspection in a Non-episodic Task
Authors: Angel Ayala, Francisco Cruz, Bruno Fernandes, Richard Dazeley
Date: 2021-08-18 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Explainable reinforcement learning allows artificial agents to explain their behavior in a human-like manner aiming at non-expert end-users. An efficient alternative of creating explanations is to use an introspection-based method that transforms Q-values into probabilities of success used as the base to explain the agent's decision-making process. This approach has been effectively used in episodic and discrete scenarios, however, to compute the probability of success in non-episodic and more complex environments has not been addressed yet. In this work, we adapt the introspection method to be used in a non-episodic task and try it in a continuous Atari game scenario solved with the Rainbow algorithm. Our initial results show that the probability of success can be computed directly from the Q-values for all possible actions.
Discriminative Region-based Multi-Label Zero-Shot Learning
Authors: Sanath Narayan, Akshita Gupta, Salman Khan, Fahad Shahbaz Khan, Ling Shao, Mubarak Shah
Date: 2021-08-20 00:00:00
Proceeding: ICCV 2021 10
Tasks: Image Retrieval, Multi-label zero-shot learning, zero-shot-classification, Zero-Shot Learning

Multi-label zero-shot learning (ZSL) is a more realistic counter-part of standard single-label ZSL since several objects can co-exist in a natural image. However, the occurrence of multiple objects complicates the reasoning and requires region-specific processing of visual features to preserve their contextual cues. We note that the best existing multi-label ZSL method takes a shared approach towards attending to region features with a common set of attention maps for all the classes. Such shared maps lead to diffused attention, which does not discriminatively focus on relevant locations when the number of classes are large. Moreover, mapping spatially-pooled visual features to the class semantics leads to inter-class feature entanglement, thus hampering the classification. Here, we propose an alternate approach towards region-based discriminability-preserving multi-label zero-shot classification. Our approach maintains the spatial resolution to preserve region-level characteristics and utilizes a bi-level attention module (BiAM) to enrich the features by incorporating both region and scene context information. The enriched region-level features are then mapped to the class semantics and only their class predictions are spatially pooled to obtain image-level predictions, thereby keeping the multi-class features disentangled. Our approach sets a new state of the art on two large-scale multi-label zero-shot benchmarks: NUS-WIDE and Open Images. On NUS-WIDE, our approach achieves an absolute gain of 6.9% mAP for ZSL, compared to the best published results.
Zoom, Enhance! Measuring Surveillance GAN Up-sampling
Authors: Jake Sparkman, Abdalla Al-Ayrot, Utkarsh Contractor
Date: 2021-08-20 00:00:00
Tasks: Image Generation, Image Quality Assessment

Deep Neural Networks have been very successfully used for many computer vision and pattern recognition applications. While Convolutional Neural Networks(CNNs) have shown the path to state of art image classifications, Generative Adversarial Networks or GANs have provided state of art capabilities in image generation. In this paper we extend the applications of CNNs and GANs to experiment with up-sampling techniques in the domains of security and surveillance. Through this work we evaluate, compare and contrast the state of art techniques in both CNN and GAN based image and video up-sampling in the surveillance domain. As a result of this study we also provide experimental evidence to establish DISTS as a stronger Image Quality Assessment(IQA) metric for comparing GAN Based Image Up-sampling in the surveillance domain.
MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction
Authors: Patrick Dendorfer, Sven Elflein, Laura Leal-Taixe
Date: 2021-08-20 00:00:00
Proceeding: ICCV 2021 10
Tasks: Pedestrian Trajectory Prediction, Trajectory Prediction

Pedestrian trajectory prediction is challenging due to its uncertain and multimodal nature. While generative adversarial networks can learn a distribution over future trajectories, they tend to predict out-of-distribution samples when the distribution of future trajectories is a mixture of multiple, possibly disconnected modes. To address this issue, we propose a multi-generator model for pedestrian trajectory prediction. Each generator specializes in learning a distribution over trajectories routing towards one of the primary modes in the scene, while a second network learns a categorical distribution over these generators, conditioned on the dynamics and scene input. This architecture allows us to effectively sample from specialized generators and to significantly reduce the out-of-distribution samples compared to single generator methods.
Approximate Bayesian Neural Doppler Imaging
Authors: A. Asensio Ramos, C. Diaz Baso, O. Kochukhov
Date: 2021-08-20 00:00:00
Tasks: Bayesian Inference

The non-uniform surface temperature distribution of rotating active stars is routinely mapped with the Doppler Imaging technique. Inhomogeneities in the surface produce features in high-resolution spectroscopic observations that shift in wavelength depending on their position on the visible hemisphere. The inversion problem has been systematically solved using maximum a-posteriori regularized methods assuming smoothness or maximum entropy. Our aim in this work is to solve the full Bayesian inference problem, by providing access to the posterior distribution of the surface temperature in the star. We use amortized neural posterior estimation to produce a model that approximates the high-dimensional posterior distribution for spectroscopic observations of selected spectral ranges sampled at arbitrary rotation phases. The posterior distribution is approximated with conditional normalizing flows, which are flexible, tractable and easy to sample approximations to arbitrary distributions. When conditioned on the spectroscopic observations, they provide a very efficient way of obtaining samples from the posterior distribution. The conditioning on observations is obtained through the use of Transformer encoders, which can deal with arbitrary wavelength sampling and rotation phases. Our model can produce thousands of posterior samples per second. Our validation of the model for very high signal-to-noise observations shows that it correctly approximates the posterior, although with some overestimation of the broadening. We apply the model to the moderately fast rotator II Peg, producing the first Bayesian map of its temperature inhomogenities. We conclude that conditional normalizing flows are a very promising tool to carry out approximate Bayesian inference in more complex problems in stellar physics, like constraining the magnetic properties.
PowerLinear Activation Functions with application to the first layer of CNNs
Authors: Kamyar Nasiri, Kamaledin Ghiasi-Shirazi
Date: 2021-08-20 00:00:00

Convolutional neural networks (CNNs) have become the state-of-the-art tool for dealing with unsolved problems in computer vision and image processing. Since the convolution operator is a linear operator, several generalizations have been proposed to improve the performance of CNNs. One way to increase the capability of the convolution operator is by applying activation functions on the inner product operator. In this paper, we will introduce PowerLinear activation functions, which are based on the polynomial kernel generalization of the convolution operator. EvenPowLin functions are the main branch of the PowerLinear activation functions. This class of activation functions is saturated neither in the positive input region nor in the negative one. Also, the negative inputs are activated with the same magnitude as the positive inputs. These features made the EvenPowLin activation functions able to be utilized in the first layer of CNN architectures and learn complex features of input images. Additionally, EvenPowLin activation functions are used in CNN models to classify the inversion of grayscale images as accurately as the original grayscale images, which is significantly better than commonly used activation functions.
Dual-Neighborhood Deep Fusion Network for Point Cloud Analysis
Authors: Guoquan Xu, Hezhi Cao, Yifan Zhang, Jianwei Wan, Ke Xu, Yanxin Ma
Date: 2021-08-20 00:00:00
Tasks: 3D Point Cloud Classification, Classification, Point Cloud Classification, Representation Learning

Recently, deep neural networks have made remarkable achievements in 3D point cloud classification. However, existing classification methods are mainly implemented on idealized point clouds and suffer heavy degradation of per-formance on non-idealized scenarios. To handle this prob-lem, a feature representation learning method, named Dual-Neighborhood Deep Fusion Network (DNDFN), is proposed to serve as an improved point cloud encoder for the task of non-idealized point cloud classification. DNDFN utilizes a trainable neighborhood learning method called TN-Learning to capture the global key neighborhood. Then, the global neighborhood is fused with the local neighbor-hood to help the network achieve more powerful reasoning ability. Besides, an Information Transfer Convolution (IT-Conv) is proposed for DNDFN to learn the edge infor-mation between point-pairs and benefits the feature transfer procedure. The transmission of information in IT-Conv is similar to the propagation of information in the graph which makes DNDFN closer to the human reasoning mode. Extensive experiments on existing benchmarks especially non-idealized datasets verify the effectiveness of DNDFN and DNDFN achieves the state of the arts.
Video Ads Content Structuring by Combining Scene Confidence Prediction and Tagging
Authors: Tomoyuki Suzuki, Antonio Tejero-de-Pablos
Date: 2021-08-20 00:00:00
Tasks: TAG

Video ads segmentation and tagging is a challenging task due to two main reasons: (1) the video scene structure is complex and (2) it includes multiple modalities (e.g., visual, audio, text.). While previous work focuses mostly on activity videos (e.g. "cooking", "sports"), it is not clear how they can be leveraged to tackle the task of video ads content structuring. In this paper, we propose a two-stage method that first provides the boundaries of the scenes, and then combines a confidence score for each segmented scene and the tag classes predicted for that scene. We provide extensive experimental results on the network architectures and modalities used for the proposed method. Our combined method improves the previous baselines on the challenging "Tencent Advertisement Video" dataset.
Exploring Data Aggregation and Transformations to Generalize across Visual Domains
Authors: Antono D'Innocente
Date: 2021-08-20 00:00:00
Tasks: Data Augmentation, Domain Adaptation, Domain Generalization, object-detection, Object Detection

Computer vision has flourished in recent years thanks to Deep Learning advancements, fast and scalable hardware solutions and large availability of structured image data. Convolutional Neural Networks trained on supervised tasks with backpropagation learn to extract meaningful representations from raw pixels automatically, and surpass shallow methods in image understanding. Though convenient, data-driven feature learning is prone to dataset bias: a network learns its parameters from training signals alone, and will usually perform poorly if train and test distribution differ. To alleviate this problem, research on Domain Generalization (DG), Domain Adaptation (DA) and their variations is increasing. This thesis contributes to these research topics by presenting novel and effective ways to solve the dataset bias problem in its various settings. We propose new frameworks for Domain Generalization and Domain Adaptation which make use of feature aggregation strategies and visual transformations via data-augmentation and multi-task integration of self-supervision. We also design an algorithm that adapts an object detection model to any out of distribution sample at test time. With through experimentation, we show how our proposed solutions outperform competitive state-of-the-art approaches in established DG and DA benchmarks.
Towards Photorealistic Colorization by Imagination
Authors: Chenyang Lei, Yue Wu, Qifeng Chen
Date: 2021-08-20 00:00:00
Tasks: Colorization, Image Colorization, Image Generation

We present a novel approach to automatic image colorization by imitating the imagination process of human experts. Our imagination module is designed to generate color images that are context-correlated with black-and-white photos. Given a black-and-white image, our imagination module firstly extracts the context information, which is then used to synthesize colorful and diverse images using a conditional image synthesis network (e.g., semantic image synthesis model). We then design a colorization module to colorize the black-and-white images with the guidance of imagination for photorealistic colorization. Experimental results show that our work produces more colorful and diverse results than state-of-the-art image colorization methods. Our source codes will be publicly available.
Region-level Active Detector Learning
Authors: Michael Laielli, Giscard Biamby, Dian Chen, Ritwik Gupta, Adam Loeffler, Phat Dat Nguyen, Ross Luo, Trevor Darrell, Sayna Ebrahimi
Date: 2021-08-20 00:00:00
Tasks: Active Learning, Diversity, Object, object-detection, Object Detection

Active learning for object detection is conventionally achieved by applying techniques developed for classification in a way that aggregates individual detections into image-level selection criteria. This is typically coupled with the costly assumption that every image selected for labelling must be exhaustively annotated. This yields incremental improvements on well-curated vision datasets and struggles in the presence of data imbalance and visual clutter that occurs in real-world imagery. Alternatives to the image-level approach are surprisingly under-explored in the literature. In this work, we introduce a new strategy that subsumes previous Image-level and Object-level approaches into a generalized, Region-level approach that promotes spatial-diversity by avoiding nearby redundant queries from the same image and minimizes context-switching for the labeler. We show that this approach significantly decreases labeling effort and improves rare object search on realistic data with inherent class-imbalance and cluttered scenes.
Boosting of Head Pose Estimation by Knowledge Distillation
Authors: Andrey Sheka, Victor Samun
Date: 2021-08-20 00:00:00
Tasks: Head Pose Estimation, Knowledge Distillation, Pose Estimation, regression

We propose a response-based method of knowledge distillation (KD) for the head pose estimation problem. A student model trained by the proposed KD achieves results better than a teacher model, which is atypical for the response-based method. Our method consists of two stages. In the first stage, we trained the base neural network (NN), which has one regression head and four regression via classification (RvC) heads. We build the convolutional ensemble over the base NN using offsets of face bounding boxes over a regular grid. In the second stage, we perform KD from the convolutional ensemble into the final NN with one RvC head. The KD improves the results by an average of 7.7\% compared to base NN. This feature makes it possible to use KD as a booster and effectively train deeper NNs. NNs trained by our KD method partially improved the state-of-the-art results. KD-ResNet152 has the best results, and KD-ResNet18 has a better result on the AFLW2000 dataset than any previous method.We have made publicly available trained NNs and face bounding boxes for the 300W-LP, AFLW, AFLW2000, and BIWI datasets.Our method potentially can be effective for other regression problems.
Self-Rule to Multi-Adapt: Generalized Multi-source Feature Learning Using Unsupervised Domain Adaptation for Colorectal Cancer Tissue Detection
Authors: Christian Abbet, Linda Studer, Andreas Fischer, Heather Dawson, Inti Zlobec, Behzad Bozorgtabar, Jean-Philippe Thiran
Date: 2021-08-20 00:00:00
Tasks: Domain Adaptation, Self-Supervised Learning, Unsupervised Domain Adaptation

Supervised learning is constrained by the availability of labeled data, which are especially expensive to acquire in the field of digital pathology. Making use of open-source data for pre-training or using domain adaptation can be a way to overcome this issue. However, pre-trained networks often fail to generalize to new test domains that are not distributed identically due to tissue stainings, types, and textures variations. Additionally, current domain adaptation methods mainly rely on fully-labeled source datasets. In this work, we propose Self-Rule to Multi-Adapt (SRMA), which takes advantage of self-supervised learning to perform domain adaptation, and removes the necessity of fully-labeled source datasets. SRMA can effectively transfer the discriminative knowledge obtained from a few labeled source domain's data to a new target domain without requiring additional tissue annotations. Our method harnesses both domains' structures by capturing visual similarity with intra-domain and cross-domain self-supervision. Moreover, we present a generalized formulation of our approach that allows the framework to learn from multiple source domains. We show that our proposed method outperforms baselines for domain adaptation of colorectal tissue type classification \new{in single and multi-source settings}, and further validate our approach on an in-house clinical cohort. The code and trained models are available open-source: https://github.com/christianabbet/SRA.
Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
Authors: Jiaming Zhang, Kailun Yang, Angela Constantinescu, Kunyu Peng, Karin Müller, Rainer Stiefelhagen
Date: 2021-08-20 00:00:00
Tasks: Decoder, GPU, Navigate, Scene Segmentation, Semantic Segmentation, Transparent objects

Transparent objects, such as glass walls and doors, constitute architectural obstacles hindering the mobility of people with low vision or blindness. For instance, the open space behind glass doors is inaccessible, unless it is correctly perceived and interacted with. However, traditional assistive technologies rarely cover the segmentation of these safety-critical transparent objects. In this paper, we build a wearable system with a novel dual-head Transformer for Transparency (Trans4Trans) perception model, which can segment general- and transparent objects. The two dense segmentation results are further combined with depth information in the system to help users navigate safely and assist them to negotiate transparent obstacles. We propose a lightweight Transformer Parsing Module (TPM) to perform multi-scale feature interpretation in the transformer-based decoder. Benefiting from TPM, the double decoders can perform joint learning from corresponding datasets to pursue robustness, meanwhile maintain efficiency on a portable GPU, with negligible calculation increase. The entire Trans4Trans model is constructed in a symmetrical encoder-decoder architecture, which outperforms state-of-the-art methods on the test sets of Stanford2D3D and Trans10K-v2 datasets, obtaining mIoU of 45.13% and 75.14%, respectively. Through a user study and various pre-tests conducted in indoor and outdoor scenes, the usability and reliability of our assistive system have been extensively verified. Meanwhile, the Tran4Trans model has outstanding performances on driving scene datasets. On Cityscapes, ACDC, and DADA-seg datasets corresponding to common environments, adverse weather, and traffic accident scenarios, mIoU scores of 81.5%, 76.3%, and 39.2% are obtained, demonstrating its high efficiency and robustness for real-world transportation applications.
Geometry-Aware Self-Training for Unsupervised Domain Adaptationon Object Point Clouds
Authors: Longkun Zou, Hui Tang, Ke Chen, Kui Jia
Date: 2021-08-20 00:00:00
Tasks: Domain Adaptation, Point Cloud Classification, Representation Learning, Unsupervised Domain Adaptation

The point cloud representation of an object can have a large geometric variation in view of inconsistent data acquisition procedure, which thus leads to domain discrepancy due to diverse and uncontrollable shape representation cross datasets. To improve discrimination on unseen distribution of point-based geometries in a practical and feasible perspective, this paper proposes a new method of geometry-aware self-training (GAST) for unsupervised domain adaptation of object point cloud classification. Specifically, this paper aims to learn a domain-shared representation of semantic categories, via two novel self-supervised geometric learning tasks as feature regularization. On one hand, the representation learning is empowered by a linear mixup of point cloud samples with their self-generated rotation labels, to capture a global topological configuration of local geometries. On the other hand, a diverse point distribution across datasets can be normalized with a novel curvature-aware distortion localization. Experiments on the PointDA-10 dataset show that our GAST method can significantly outperform the state-of-the-art methods.
Contrastive Representations for Label Noise Require Fine-Tuning
Authors: Pierre Nodet, Vincent Lemaire, Alexis Bondu, Antoine Cornuéjols
Date: 2021-08-20 00:00:00
Tasks: Robust classification

In this paper we show that the combination of a Contrastive representation with a label noise-robust classification head requires fine-tuning the representation in order to achieve state-of-the-art performances. Since fine-tuned representations are shown to outperform frozen ones, one can conclude that noise-robust classification heads are indeed able to promote meaningful representations if provided with a suitable starting point. Experiments are conducted to draw a comprehensive picture of performances by featuring six methods and nine noise instances of three different kinds (none, symmetric, and asymmetric). In presence of noise the experiments show that fine tuning of Contrastive representation allows the six methods to achieve better results than end-to-end learning and represent a new reference compare to the recent state of art. Results are also remarkable stable versus the noise level.