pwc-archive/papers-with-abstracts huggingface dataset:
Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise
Authors: Jannik Schmitt, Stefan Roth
Date: 2021-03-15 00:00:00
Tasks: image-classification, Image Classification, Variational Inference

To adopt neural networks in safety critical domains, knowing whether we can trust their predictions is crucial. Bayesian neural networks (BNNs) provide uncertainty estimates by averaging predictions with respect to the posterior weight distribution. Variational inference methods for BNNs approximate the intractable weight posterior with a tractable distribution, yet mostly rely on sampling from the variational distribution during training and inference. Recent sampling-free approaches offer an alternative, but incur a significant parameter overhead. We here propose a more efficient parameterization of the posterior approximation for sampling-free variational inference that relies on the distribution induced by multiplicative Gaussian activation noise. This allows us to combine parameter efficiency with the benefits of sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks including ImageNet.
Visualizing Data Velocity using DSNE
Authors: Songting Shi
Date: 2021-03-15 00:00:00

We present a new technique called "DSNE" which learns the velocity embeddings of low dimensional map points when given the high-dimensional data points with its velocities. The technique is a variation of Stochastic Neighbor Embedding, which uses the Euclidean distance on the unit sphere between the unit-length velocity of the point and the unit-length direction from the point to its near neighbors to define similarities, and try to match the two kinds of similarities in the high dimension space and low dimension space to find the velocity embeddings on the low dimension space. DSNE can help to visualize how the data points move in the high dimension space by presenting the movements in two or three dimensions space. It is helpful for understanding the mechanism of cell differentiation and embryo development.
Meta-Solver for Neural Ordinary Differential Equations
Authors: Julia Gusak, Alexandr Katrutsa, Talgat Daulbaev, Andrzej Cichocki, Ivan Oseledets
Date: 2021-03-15 00:00:00

A conventional approach to train neural ordinary differential equations (ODEs) is to fix an ODE solver and then learn the neural network's weights to optimize a target loss function. However, such an approach is tailored for a specific discretization method and its properties, which may not be optimal for the selected application and yield the overfitting to the given solver. In our paper, we investigate how the variability in solvers' space can improve neural ODEs performance. We consider a family of Runge-Kutta methods that are parameterized by no more than two scalar variables. Based on the solvers' properties, we propose an approach to decrease neural ODEs overfitting to the pre-defined solver, along with a criterion to evaluate such behaviour. Moreover, we show that the right choice of solver parameterization can significantly affect neural ODEs models in terms of robustness to adversarial attacks. Recently it was shown that neural ODEs demonstrate superiority over conventional CNNs in terms of robustness. Our work demonstrates that the model robustness can be further improved by optimizing solver choice for a given task. The source code to reproduce our experiments is available at https://github.com/juliagusak/neural-ode-metasolver.
Efficient Intrusion Detection Using Evidence Theory
Authors: Islam Debicha, Thibault Debatty, Wim Mees, Jean-Michel Dricot
Date: 2021-03-15 00:00:00
Tasks: Intrusion Detection

Intrusion Detection Systems (IDS) are now an essential element when it comes to securing computers and networks. Despite the huge research efforts done in the field, handling sources' reliability remains an open issue. To address this problem, this paper proposes a novel contextual discounting method based on sources' reliability and their distinguishing ability between normal and abnormal behavior. Dempster-Shafer theory, a general framework for reasoning under uncertainty, is used to construct an evidential classifier. The NSL-KDD dataset, a significantly revised and improved version of the existing KDDCUP'99 dataset, provides the basis for assessing the performance of our new detection approach. While giving comparable results on the KDDTest+ dataset, our approach outperformed some other state-of-the-art methods on the KDDTest-21 dataset which is more challenging.
Cross-Domain Similarity Learning for Face Recognition in Unseen Domains
Authors: Masoud Faraki, Xiang Yu, Yi-Hsuan Tsai, Yumin Suh, Manmohan Chandraker
Date: 2021-03-12 00:00:00
Proceeding: CVPR 2021 1
Tasks: Face Recognition, Metric Learning, Triplet

Face recognition models trained under the assumption of identical training and test distributions often suffer from poor generalization when faced with unknown variations, such as a novel ethnicity or unpredictable individual make-ups during test time. In this paper, we introduce a novel cross-domain metric learning loss, which we dub Cross-Domain Triplet (CDT) loss, to improve face recognition in unseen domains. The CDT loss encourages learning semantically meaningful features by enforcing compact feature clusters of identities from one domain, where the compactness is measured by underlying similarity metrics that belong to another training domain with different statistics. Intuitively, it discriminatively correlates explicit metrics derived from one domain, with triplet samples from another domain in a unified loss function to be minimized within a network, which leads to better alignment of the training domains. The network parameters are further enforced to learn generalized features under domain shift, in a model-agnostic learning pipeline. Unlike the recent work of Meta Face Recognition, our method does not require careful hard-pair sample mining and filtering strategy during training. Extensive experiments on various face recognition benchmarks show the superiority of our method in handling variations, compared to baseline and the state-of-the-art methods.
On Incorporating Forecasts into Linear State Space Model Markov Decision Processes
Authors: Jacques A. de Chalendar, Peter W. Glynn
Date: 2021-03-12 00:00:00

Weather forecast information will very likely find increasing application in the control of future energy systems. In this paper, we introduce an augmented state space model formulation with linear dynamics, within which one can incorporate forecast information that is dynamically revealed alongside the evolution of the underlying state variable. We use the martingale model for forecast evolution (MMFE) to enforce the necessary consistency properties that must govern the joint evolution of forecasts with the underlying state. The formulation also generates jointly Markovian dynamics that give rise to Markov decision processes (MDPs) that remain computationally tractable. This paper is the first to enforce MMFE consistency requirements within an MDP formulation that preserves tractability.
How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events
Authors: Azim Ahmadzadeh, Berkay Aydin, Manolis K. Georgoulis, Dustin J. Kempton, Sushant S. Mahajan, Rafal A. Angryk
Date: 2021-03-12 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

We present a case study of solar flare forecasting by means of metadata feature time series, by treating it as a prominent class-imbalance and temporally coherent problem. Taking full advantage of pre-flare time series in solar active regions is made possible via the Space Weather Analytics for Solar Flares (SWAN-SF) benchmark dataset; a partitioned collection of multivariate time series of active region properties comprising 4075 regions and spanning over 9 years of the Solar Dynamics Observatory (SDO) period of operations. We showcase the general concept of temporal coherence triggered by the demand of continuity in time series forecasting and show that lack of proper understanding of this effect may spuriously enhance models' performance. We further address another well-known challenge in rare event prediction, namely, the class-imbalance issue. The SWAN-SF is an appropriate dataset for this, with a 60:1 imbalance ratio for GOES M- and X-class flares and a 800:1 for X-class flares against flare-quiet instances. We revisit the main remedies for these challenges and present several experiments to illustrate the exact impact that each of these remedies may have on performance. Moreover, we acknowledge that some basic data manipulation tasks such as data normalization and cross validation may also impact the performance -- we discuss these problems as well. In this framework we also review the primary advantages and disadvantages of using true skill statistic and Heidke skill score, as two widely used performance verification metrics for the flare forecasting task. In conclusion, we show and advocate for the benefits of time series vs. point-in-time forecasting, provided that the above challenges are measurably and quantitatively addressed.
Causal Markov Boundaries
Authors: Sofia Triantafillou, Fattaneh Jabbari, Greg Cooper
Date: 2021-03-12 00:00:00
Tasks: feature selection

Feature selection is an important problem in machine learning, which aims to select variables that lead to an optimal predictive model. In this paper, we focus on feature selection for post-intervention outcome prediction from pre-intervention variables. We are motivated by healthcare settings, where the goal is often to select the treatment that will maximize a specific patient's outcome; however, we often do not have sufficient randomized control trial data to identify well the conditional treatment effect. We show how we can use observational data to improve feature selection and effect estimation in two cases: (a) using observational data when we know the causal graph, and (b) when we do not know the causal graph but have observational and limited experimental data. Our paper extends the notion of Markov boundary to treatment-outcome pairs. We provide theoretical guarantees for the methods we introduce. In simulated data, we show that combining observational and experimental data improves feature selection and effect estimation.
Slip-Based Autonomous ZUPT through Gaussian Process to Improve Planetary Rover Localization
Authors: Cagri Kilic, Nicholas Ohi, Yu Gu, Jason N. Gross
Date: 2021-03-13 00:00:00

The zero-velocity update (ZUPT) algorithm provides valuable state information to maintain the inertial navigation system (INS) reliability when stationary conditions are satisfied. Employing ZUPT along with leveraging non-holonomic constraints can greatly benefit wheeled mobile robot dead-reckoning localization accuracy. However, determining how often they should be employed requires consideration to balance localization accuracy and traversal rate for planetary rovers. To address this, we investigate when to autonomously initiate stops to improve wheel-inertial odometry (WIO) localization performance with ZUPT. To do this, we propose a 3D dead-reckoning approach that predicts wheel slippage while the rover is in motion and forecasts the appropriate time to stop without changing any rover hardware or major rover operations. We validate with field tests that our approach is viable on different terrain types and achieves a 3D localization accuracy of more than 97% over 650 m drives on rough terrain.
RL-Controller: a reinforcement learning framework for active structural control
Authors: Soheila Sadeghi Eshkevari, Soheil Sadeghi Eshkevari, Debarshi Sen, Shamim N. Pakzad
Date: 2021-03-13 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

To maintain structural integrity and functionality during the designed life cycle of a structure, engineers are expected to accommodate for natural hazards as well as operational load levels. Active control systems are an efficient solution for structural response control when a structure is subjected to unexpected extreme loads. However, development of these systems through traditional means is limited by their model dependent nature. Recent advancements in adaptive learning methods, in particular, reinforcement learning (RL), for real-time decision making problems, along with rapid growth in high-performance computational resources, help structural engineers to transform the classic model-based active control problem to a purely data-driven one. In this paper, we present a novel RL-based approach for designing active controllers by introducing RL-Controller, a flexible and scalable simulation environment. The RL-Controller includes attributes and functionalities that are defined to model active structural control mechanisms in detail. We show that the proposed framework is easily trainable for a five story benchmark building with 65% reductions on average in inter story drifts (ISD) when subjected to strong ground motions. In a comparative study with LQG active control method, we demonstrate that the proposed model-free algorithm learns more optimal actuator forcing strategies that yield higher performance, e.g., 25% more ISD reductions on average with respect to LQG, without using prior information about the mechanical properties of the system.
Early Prediction and Diagnosis of Retinoblastoma Using Deep Learning Techniques
Authors: C. Anand Deva Durai, T Jemima Jebaseeli, Salem Alelyani, Azath Mubharakali
Date: 2021-03-13 00:00:00
Tasks: Deep Learning

Retinoblastoma is the most prominent childhood primary intraocular malignancy that impacts the vision of children and adults worldwide. In contrasting and comparing with adults it is uveal melanoma. It is an aggressive tumor that can fill and destroy the eye and the surrounding structures. Therefore early detection of retinoblastoma in childhood is the key. The major impact of the research is to identify the tumor cells in the retina. Also is to find out the stages of the tumor and its corresponding group. The proposed systems assist the ophthalmologists for accurate prediction and diagnosis of retinoblastoma cancer disease at the earliest. The contribution of the proposed approach is to save the life of infants and the grown-up children from vision impairment. The proposed methodology consists of three phases namely, preprocessing, segmentation, and classification. Initially, the fundus images are preprocessed using the Liner Predictive Decision based Median Filter (LPDMF). It removes the noise introduced in the image due to illumination while capturing or scanning the eye of the patients. The preprocessed images are segmented using the Convolutional Neural Network (CNN) to distinguish the foreground tumor cells from the background.
Generating Unrestricted Adversarial Examples via Three Parameters
Authors: Hanieh Naderi, Leili Goli, Shohreh Kasaei
Date: 2021-03-13 00:00:00
Tasks: Adversarial Attack

Deep neural networks have been shown to be vulnerable to adversarial examples deliberately constructed to misclassify victim models. As most adversarial examples have restricted their perturbations to $L_{p}$-norm, existing defense methods have focused on these types of perturbations and less attention has been paid to unrestricted adversarial examples; which can create more realistic attacks, able to deceive models without affecting human predictions. To address this problem, the proposed adversarial attack generates an unrestricted adversarial example with a limited number of parameters. The attack selects three points on the input image and based on their locations transforms the image into an adversarial example. By limiting the range of movement and location of these three points and using a discriminatory network, the proposed unrestricted adversarial example preserves the image appearance. Experimental results show that the proposed adversarial examples obtain an average success rate of 93.5% in terms of human evaluation on the MNIST and SVHN datasets. It also reduces the model accuracy by an average of 73% on six datasets MNIST, FMNIST, SVHN, CIFAR10, CIFAR100, and ImageNet. It should be noted that, in the case of attacks, lower accuracy in the victim model denotes a more successful attack. The adversarial train of the attack also improves model robustness against a randomly transformed image.
PhotoApp: Photorealistic Appearance Editing of Head Portraits
Authors: Mallikarjun B R, Ayush Tewari, Abdallah Dib, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Louis Chevallier, Mohamed Elgharib, Christian Theobalt
Date: 2021-03-13 00:00:00

Photorealistic editing of portraits is a challenging task as humans are very sensitive to inconsistencies in faces. We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image. This requires our method to capture and control the full reflectance field of the person in the image. Most editing approaches rely on supervised learning using training data captured with setups such as light and camera stages. Such datasets are expensive to acquire, not readily available and do not capture all the rich variations of in-the-wild portrait images. In addition, most supervised approaches only focus on relighting, and do not allow camera viewpoint editing. Thus, they only capture and control a subset of the reflectance field. Recently, portrait editing has been demonstrated by operating in the generative model space of StyleGAN. While such approaches do not require direct supervision, there is a significant loss of quality when compared to the supervised approaches. In this paper, we present a method which learns from limited supervised training data. The training images only include people in a fixed neutral expression with eyes closed, without much hair or background variations. Each person is captured under 150 one-light-at-a-time conditions and under 8 camera poses. Instead of training directly in the image space, we design a supervised problem which learns transformations in the latent space of StyleGAN. This combines the best of supervised learning and generative adversarial modeling. We show that the StyleGAN prior allows for generalisation to different expressions, hairstyles and backgrounds. This produces high-quality photorealistic results for in-the-wild images and significantly outperforms existing methods. Our approach can edit the illumination and pose simultaneously, and runs at interactive rates.
uTHCD: A New Benchmarking for Tamil Handwritten OCR
Authors: Noushath Shaffi, Faizal Hajamohideen
Date: 2021-03-13 00:00:00
Tasks: Benchmarking, Optical Character Recognition (OCR)

Handwritten character recognition is a challenging research in the field of document image analysis over many decades due to numerous reasons such as large writing styles variation, inherent noise in data, expansive applications it offers, non-availability of benchmark databases etc. There has been considerable work reported in literature about creation of the database for several Indic scripts but the Tamil script is still in its infancy as it has been reported only in one database [5]. In this paper, we present the work done in the creation of an exhaustive and large unconstrained Tamil Handwritten Character Database (uTHCD). Database consists of around 91000 samples with nearly 600 samples in each of 156 classes. The database is a unified collection of both online and offline samples. Offline samples were collected by asking volunteers to write samples on a form inside a specified grid. For online samples, we made the volunteers write in a similar grid using a digital writing pad. The samples collected encompass a vast variety of writing styles, inherent distortions arising from offline scanning process viz stroke discontinuity, variable thickness of stroke, distortion etc. Algorithms which are resilient to such data can be practically deployed for real time applications. The samples were generated from around 650 native Tamil volunteers including school going kids, homemakers, university students and faculty. The isolated character database will be made publicly available as raw images and Hierarchical Data File (HDF) compressed file. With this database, we expect to set a new benchmark in Tamil handwritten character recognition and serve as a launchpad for many avenues in document image analysis domain. Paper also presents an ideal experimental set-up using the database on convolutional neural networks (CNN) with a baseline accuracy of 88% on test data.
Error-Aware Policy Learning: Zero-Shot Generalization in Partially Observable Dynamic Environments
Authors: Visak Kumar, Sehoon Ha, C. Karen Liu
Date: 2021-03-13 00:00:00
Tasks: Zero-shot Generalization

Simulation provides a safe and efficient way to generate useful data for learning complex robotic tasks. However, matching simulation and real-world dynamics can be quite challenging, especially for systems that have a large number of unobserved or unmeasurable parameters, which may lie in the robot dynamics itself or in the environment with which the robot interacts. We introduce a novel approach to tackle such a sim-to-real problem by developing policies capable of adapting to new environments, in a zero-shot manner. Key to our approach is an error-aware policy (EAP) that is explicitly made aware of the effect of unobservable factors during training. An EAP takes as input the predicted future state error in the target environment, which is provided by an error-prediction function, simultaneously trained with the EAP. We validate our approach on an assistive walking device trained to help the human user recover from external pushes. We show that a trained EAP for a hip-torque assistive device can be transferred to different human agents with unseen biomechanical characteristics. In addition, we show that our method can be applied to other standard RL control tasks.
Reconsidering Representation Alignment for Multi-view Clustering
Authors: Daniel J. Trosten, Sigurd Løkse, Robert Jenssen, Michael Kampffmeyer
Date: 2021-03-13 00:00:00
Proceeding: CVPR 2021 1
Tasks: Clustering, Contrastive Learning

Aligning distributions of view representations is a core component of today's state of the art models for deep multi-view clustering. However, we identify several drawbacks with na\"ively aligning representation distributions. We demonstrate that these drawbacks both lead to less separable clusters in the representation space, and inhibit the model's ability to prioritize views. Based on these observations, we develop a simple baseline model for deep multi-view clustering. Our baseline model avoids representation alignment altogether, while performing similar to, or better than, the current state of the art. We also expand our baseline model by adding a contrastive learning component. This introduces a selective alignment procedure that preserves the model's ability to prioritize views. Our experiments show that the contrastive learning component enhances the baseline model, improving on the current state of the art by a large margin on several datasets.
Learning Novel Objects Continually Through Curiosity
Authors: Ali Ayub, Alan R. Wagner
Date: 2021-03-13 00:00:00
Tasks: Active Learning, Continual Learning

Children learn continually by asking questions about the concepts they are most curious about. With robots becoming an integral part of our society, they must also learn unknown concepts continually by asking humans questions. The paper analyzes a recent state-of-the-art approach for continual learning. The paper further develops a self-supervised technique to find most of the uncertain objects in an environment by utilizing the cluster representation of the previously learned classes. We test our approach on a benchmark dataset for continual learning on robots. Our results show that our curiosity-driven continual learning approach beats random sampling and softmax-based uncertainty sampling in terms of classification accuracy and the total number of classes learned.