pwc-archive/papers-with-abstracts huggingface dataset:
DeepVir -- Graphical Deep Matrix Factorization for "In Silico" Antiviral Repositioning: Application to COVID-19
Authors: Aanchal Mongia, Stuti Jain, Emilie Chouzenoux, Angshul Majumda
Date: 2020-09-22 00:00:00
Tasks: Matrix Completion

This work formulates antiviral repositioning as a matrix completion problem where the antiviral drugs are along the rows and the viruses along the columns. The input matrix is partially filled, with ones in positions where the antiviral has been known to be effective against a virus. The curated metadata for antivirals (chemical structure and pathways) and viruses (genomic structure and symptoms) is encoded into our matrix completion framework as graph Laplacian regularization. We then frame the resulting multiple graph regularized matrix completion problem as deep matrix factorization. This is solved by using a novel optimization method called HyPALM (Hybrid Proximal Alternating Linearized Minimization). Results on our curated RNA drug virus association (DVA) dataset shows that the proposed approach excels over state-of-the-art graph regularized matrix completion techniques. When applied to "in silico" prediction of antivirals for COVID-19, our approach returns antivirals that are either used for treating patients or are under for trials for the same.
Learning Task-Agnostic Action Spaces for Movement Optimization
Authors: Amin Babadi, Michiel Van de Panne, C. Karen Liu, Perttu Hämäläinen
Date: 2020-09-22 00:00:00

We propose a novel method for exploring the dynamics of physically based animated characters, and learning a task-agnostic action space that makes movement optimization easier. Like several previous papers, we parameterize actions as target states, and learn a short-horizon goal-conditioned low-level control policy that drives the agent's state towards the targets. Our novel contribution is that with our exploration data, we are able to learn the low-level policy in a generic manner and without any reference movement data. Trained once for each agent or simulation environment, the policy improves the efficiency of optimizing both trajectories and high-level policies across multiple tasks and optimization algorithms. We also contribute novel visualizations that show how using target states as actions makes optimized trajectories more robust to disturbances; this manifests as wider optima that are easy to find. Due to its simplicity and generality, our proposed approach should provide a building block that can improve a large variety of movement optimization methods and applications.
Gamma distribution-based sampling for imbalanced data
Authors: Firuz Kamalov, Dmitry Denisov
Date: 2020-09-22 00:00:00
Tasks: Fraud Detection

Imbalanced class distribution is a common problem in a number of fields including medical diagnostics, fraud detection, and others. It causes bias in classification algorithms leading to poor performance on the minority class data. In this paper, we propose a novel method for balancing the class distribution in data through intelligent resampling of the minority class instances. The proposed method is based on generating new minority instances in the neighborhood of the existing minority points via a gamma distribution. Our method offers a natural and coherent approach to balancing the data. We conduct a comprehensive numerical analysis of the new sampling technique. The experimental results show that the proposed method outperforms the existing state-of-the-art methods for imbalanced data. Concretely, the new sampling technique produces the best results on 12 out of 24 real life as well as synthetic datasets. For comparison, the SMOTE method achieves the top score on only 1 dataset. We conclude that the new technique offers a simple yet effective sampling approach to balance data.
Inter-database validation of a deep learning approach for automatic sleep scoring
Authors: Diego Alvarez-Estevez, Roselyne M. Rijsman
Date: 2020-09-22 00:00:00
Tasks: Sleep Staging

In this work we describe a new deep learning approach for automatic sleep staging, and carry out its validation by addressing its generalization capabilities on a wide range of sleep staging databases. Prediction capabilities are evaluated in the context of independent local and external generalization scenarios. Effectively, by comparing both procedures it is possible to better extrapolate the expected performance of the method on the general reference task of sleep staging, regardless of data from a specific database. In addition, we examine the suitability of a novel approach based on the use of an ensemble of individual local models and evaluate its impact on the resulting inter-database generalization performance. Validation results show good general performance, as compared to the expected levels of human expert agreement, as well as state-of-the-art automatic sleep staging approaches
Explainable, Stable, and Scalable Graph Convolutional Networks for Learning Graph Representation
Authors: Ping-En Lu, Cheng-Shang Chang
Date: 2020-09-22 00:00:00
Tasks: Clustering, Dimensionality Reduction, Network Embedding

The network embedding problem that maps nodes in a graph to vectors in Euclidean space can be very useful for addressing several important tasks on a graph. Recently, graph neural networks (GNNs) have been proposed for solving such a problem. However, most embedding algorithms and GNNs are difficult to interpret and do not scale well to handle millions of nodes. In this paper, we tackle the problem from a new perspective based on the equivalence of three constrained optimization problems: the network embedding problem, the trace maximization problem of the modularity matrix in a sampled graph, and the matrix factorization problem of the modularity matrix in a sampled graph. The optimal solutions to these three problems are the dominant eigenvectors of the modularity matrix. We proposed two algorithms that belong to a special class of graph convolutional networks (GCNs) for solving these problems: (i) Clustering As Feature Embedding GCN (CAFE-GCN) and (ii) sphere-GCN. Both algorithms are stable trace maximization algorithms, and they yield good approximations of dominant eigenvectors. Moreover, there are linear-time implementations for sparse graphs. In addition to solving the network embedding problem, both proposed GCNs are capable of performing dimensionality reduction. Various experiments are conducted to evaluate our proposed GCNs and show that our proposed GCNs outperform almost all the baseline methods. Moreover, CAFE-GCN could be benefited from the labeled data and have tremendous improvements in various performance metrics.
PS8-Net: A Deep Convolutional Neural Network to Predict the Eight-State Protein Secondary Structure
Authors: Md. Aminur Rab Ratul, Maryam Tavakol Elahi, M. Hamed Mozaffari, Won-Sook Lee
Date: 2020-09-22 00:00:00
Tasks: Prediction, Protein Secondary Structure Prediction

Protein secondary structure is crucial to creating an information bridge between the primary and tertiary (3D) structures. Precise prediction of eight-state protein secondary structure (PSS) has significantly utilized in the structural and functional analysis of proteins in bioinformatics. Deep learning techniques have been recently applied in this research area and raised the eight-state (Q8) protein secondary structure prediction accuracy remarkably. Nevertheless, from a theoretical standpoint, there are still lots of rooms for improvement, specifically in the eight-state PSS prediction. In this study, we have presented a new deep convolutional neural network (DCNN), namely PS8-Net, to enhance the accuracy of eight-class PSS prediction. The input of this architecture is a carefully constructed feature matrix from the proteins sequence features and profile features. We introduce a new PS8 module in the network, which is applied with skip connection to extracting the long-term inter-dependencies from higher layers, obtaining local contexts in earlier layers, and achieving global information during secondary structure prediction. Our proposed PS8-Net achieves 76.89%, 71.94%, 76.86%, and 75.26% Q8 accuracy respectively on benchmark CullPdb6133, CB513, CASP10, and CASP11 datasets. This architecture enables the efficient processing of local and global interdependencies between amino acids to make an accurate prediction of each class. To the best of our knowledge, PS8-Net experiment results demonstrate that it outperforms all the state-of-the-art methods on the aforementioned benchmark datasets.
Is Q-Learning Provably Efficient? An Extended Analysis
Authors: Kushagra Rastogi, Jonathan Lee, Fabrice Harel-Canada, Aditya Joglekar
Date: 2020-09-22 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning (RL)

This work extends the analysis of the theoretical results presented within the paper Is Q-Learning Provably Efficient? by Jin et al. We include a survey of related research to contextualize the need for strengthening the theoretical guarantees related to perhaps the most important threads of model-free reinforcement learning. We also expound upon the reasoning used in the proofs to highlight the critical steps leading to the main result showing that Q-learning with UCB exploration achieves a sample efficiency that matches the optimal regret that can be achieved by any model-based approach.
Privacy Preserving K-Means Clustering: A Secure Multi-Party Computation Approach
Authors: Daniel Hurtado Ramírez, J. M. Auñón
Date: 2020-09-22 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Privacy Preserving

Knowledge discovery is one of the main goals of Artificial Intelligence. This Knowledge is usually stored in databases spread in different environments, being a tedious (or impossible) task to access and extract data from them. To this difficulty we must add that these datasources may contain private data, therefore the information can never leave the source. Privacy Preserving Machine Learning (PPML) helps to overcome this difficulty, employing cryptographic techniques, allowing knowledge discovery while ensuring data privacy. K-means is one of the data mining techniques used in order to discover knowledge, grouping data points in clusters that contain similar features. This paper focuses in Privacy Preserving Machine Learning applied to K-means using recent protocols from the field of criptography. The algorithm is applied to different scenarios where data may be distributed either horizontally or vertically.
Local Post-Hoc Explanations for Predictive Process Monitoring in Manufacturing
Authors: Nijat Mehdiyev, Peter Fettke
Date: 2020-09-22 00:00:00
Tasks: Binary Classification, Decision Making, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Predictive Process Monitoring

This study proposes an innovative explainable predictive quality analytics solution to facilitate data-driven decision-making for process planning in manufacturing by combining process mining, machine learning, and explainable artificial intelligence (XAI) methods. For this purpose, after integrating the top-floor and shop-floor data obtained from various enterprise information systems, a deep learning model was applied to predict the process outcomes. Since this study aims to operationalize the delivered predictive insights by embedding them into decision-making processes, it is essential to generate relevant explanations for domain experts. To this end, two complementary local post-hoc explanation approaches, Shapley values and Individual Conditional Expectation (ICE) plots are adopted, which are expected to enhance the decision-making capabilities by enabling experts to examine explanations from different perspectives. After assessing the predictive strength of the applied deep neural network with relevant binary classification evaluation measures, a discussion of the generated explanations is provided.
Adversarial Training with Stochastic Weight Average
Authors: Joong-won Hwang, Youngwan Lee, Sungchan Oh, Yuseok Bae
Date: 2020-09-21 00:00:00

Adversarial training deep neural networks often experience serious overfitting problem. Recently, it is explained that the overfitting happens because the sample complexity of training data is insufficient to generalize robustness. In traditional machine learning, one way to relieve overfitting from the lack of data is to use ensemble methods. However, adversarial training multiple networks is extremely expensive. Moreover, we found that there is a dilemma on choosing target model to generate adversarial examples. Optimizing attack to the members of ensemble will be suboptimal attack to the ensemble and incurs covariate shift, while attack to ensemble will weaken the members and lose the benefit from ensembling. In this paper, we propose adversarial training with Stochastic weight average (SWA); while performing adversarial training, we aggregate the temporal weight states in the trajectory of training. By adopting SWA, the benefit of ensemble can be gained without tremendous computational increment and without facing the dilemma. Moreover, we further improved SWA to be adequate to adversarial training. The empirical results on CIFAR-10, CIFAR-100 and SVHN show that our method can improve the robustness of models.
A Centralised Soft Actor Critic Deep Reinforcement Learning Approach to District Demand Side Management through CityLearn
Authors: Anjukan Kathirgamanathan, Kacper Twardowski, Eleni Mangina, Donal Finn
Date: 2020-09-22 00:00:00
Tasks: Deep Reinforcement Learning, Management, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning is a promising model-free and adaptive controller for demand side management, as part of the future smart grid, at the district level. This paper presents the results of the algorithm that was submitted for the CityLearn Challenge, which was hosted in early 2020 with the aim of designing and tuning a reinforcement learning agent to flatten and smooth the aggregated curve of electrical demand of a district of diverse buildings. The proposed solution secured second place in the challenge using a centralised 'Soft Actor Critic' deep reinforcement learning agent that was able to handle continuous action spaces. The controller was able to achieve an averaged score of 0.967 on the challenge dataset comprising of different buildings and climates. This highlights the potential application of deep reinforcement learning as a plug-and-play style controller, that is capable of handling different climates and a heterogenous building stock, for district demand side management of buildings.
Anomalous diffusion dynamics of learning in deep neural networks
Authors: Guozhang Chen, Cheng Kevin Qu, Pulin Gong
Date: 2020-09-22 00:00:00

Learning in deep neural networks (DNNs) is implemented through minimizing a highly non-convex loss function, typically by a stochastic gradient descent (SGD) method. This learning process can effectively find good wide minima without being trapped in poor local ones. We present a novel account of how such effective deep learning emerges through the interactions of the SGD and the geometrical structure of the loss landscape. Rather than being a normal diffusion process (i.e. Brownian motion) as often assumed, we find that the SGD exhibits rich, complex dynamics when navigating through the loss landscape; initially, the SGD exhibits anomalous superdiffusion, which attenuates gradually and changes to subdiffusion at long times when the solution is reached. Such learning dynamics happen ubiquitously in different DNNs such as ResNet and VGG-like networks and are insensitive to batch size and learning rate. The anomalous superdiffusion process during the initial learning phase indicates that the motion of SGD along the loss landscape possesses intermittent, big jumps; this non-equilibrium property enables the SGD to escape from sharp local minima. By adapting the methods developed for studying energy landscapes in complex physical systems, we find that such superdiffusive learning dynamics are due to the interactions of the SGD and the fractal-like structure of the loss landscape. We further develop a simple model to demonstrate the mechanistic role of the fractal loss landscape in enabling the SGD to effectively find global minima. Our results thus reveal the effectiveness of deep learning from a novel perspective and have implications for designing efficient deep neural networks.
Automating Outlier Detection via Meta-Learning
Authors: Yue Zhao, Ryan A. Rossi, Leman Akoglu
Date: 2020-09-22 00:00:00
Tasks: Anomaly Detection, AutoML, Meta-Learning, Model Selection, Outlier Detection

Given an unsupervised outlier detection (OD) task on a new dataset, how can we automatically select a good outlier detection method and its hyperparameter(s) (collectively called a model)? Thus far, model selection for OD has been a "black art"; as any model evaluation is infeasible due to the lack of (i) hold-out data with labels, and (ii) a universal objective function. In this work, we develop the first principled data-driven approach to model selection for OD, called MetaOD, based on meta-learning. MetaOD capitalizes on the past performances of a large body of detection models on existing outlier detection benchmark datasets, and carries over this prior experience to automatically select an effective model to be employed on a new dataset without using any labels. To capture task similarity, we introduce specialized meta-features that quantify outlying characteristics of a dataset. Through comprehensive experiments, we show the effectiveness of MetaOD in selecting a detection model that significantly outperforms the most popular outlier detectors (e.g., LOF and iForest) as well as various state-of-the-art unsupervised meta-learners while being extremely fast. To foster reproducibility and further research on this new problem, we open-source our entire meta-learning system, benchmark environment, and testbed datasets.
Property-Directed Verification of Recurrent Neural Networks
Authors: Igor Khmelnitsky, Daniel Neider, Rajarshi Roy, Benoît Barbot, Benedikt Bollig, Alain Finkel, Serge Haddad, Martin Leucker, Lina Ye
Date: 2020-09-22 00:00:00

This paper presents a property-directed approach to verifying recurrent neural networks (RNNs). To this end, we learn a deterministic finite automaton as a surrogate model from a given RNN using active automata learning. This model may then be analyzed using model checking as verification technique. The term property-directed reflects the idea that our procedure is guided and controlled by the given property rather than performing the two steps separately. We show that this not only allows us to discover small counterexamples fast, but also to generalize them by pumping towards faulty flows hinting at the underlying error in the RNN.
An Exponential Factorization Machine with Percentage Error Minimization to Retail Sales Forecasting
Authors: Chongshou Li, Brenda Cheang, Zhixing Luo, Andrew Lim
Date: 2020-09-22 00:00:00
Tasks: Attribute, Demand Forecasting, Feature Engineering

This paper proposes a new approach to sales forecasting for new products with long lead time but short product life cycle. These SKUs are usually sold for one season only, without any replenishments. An exponential factorization machine (EFM) sales forecast model is developed to solve this problem which not only considers SKU attributes, but also pairwise interactions. The EFM model is significantly different from the original Factorization Machines (FM) from two-fold: (1) the attribute-level formulation for explanatory variables and (2) exponential formulation for the positive response variable. The attribute-level formation excludes infeasible intra-attribute interactions and results in more efficient feature engineering comparing with the conventional one-hot encoding, while the exponential formulation is demonstrated more effective than the log-transformation for the positive but not skewed distributed responses. In order to estimate the parameters, percentage error squares (PES) and error squares (ES) are minimized by a proposed adaptive batch gradient descent method over the training set. Real-world data provided by a footwear retailer in Singapore is used for testing the proposed approach. The forecasting performance in terms of both mean absolute percentage error (MAPE) and mean absolute error (MAE) compares favourably with not only off-the-shelf models but also results reported by extant sales and demand forecasting studies. The effectiveness of the proposed approach is also demonstrated by two external public datasets. Moreover, we prove the theoretical relationships between PES and ES minimization, and present an important property of the PES minimization for regression models; that it trains models to underestimate data. This property fits the situation of sales forecasting where unit-holding cost is much greater than the unit-shortage cost.
Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS
Authors: Lin Chen, Sheng Xu
Date: 2020-09-22 00:00:00
Proceeding: ICLR 2021 1

We prove that the reproducing kernel Hilbert spaces (RKHS) of a deep neural tangent kernel and the Laplace kernel include the same set of functions, when both kernels are restricted to the sphere $\mathbb{S}^{d-1}$. Additionally, we prove that the exponential power kernel with a smaller power (making the kernel less smooth) leads to a larger RKHS, when it is restricted to the sphere $\mathbb{S}^{d-1}$ and when it is defined on the entire $\mathbb{R}^d$.
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
Authors: Weinan E, Chao Ma, Stephan Wojtowytsch, Lei Wu
Date: 2020-09-22 00:00:00

The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.
Asynchronous Distributed Optimization with Stochastic Delays
Authors: Margalit Glasgow, Mary Wootters
Date: 2020-09-22 00:00:00
Tasks: Distributed Optimization

We study asynchronous finite sum minimization in a distributed-data setting with a central parameter server. While asynchrony is well understood in parallel settings where the data is accessible by all machines -- e.g., modifications of variance-reduced gradient algorithms like SAGA work well -- little is known for the distributed-data setting. We develop an algorithm ADSAGA based on SAGA for the distributed-data setting, in which the data is partitioned between many machines. We show that with $m$ machines, under a natural stochastic delay model with an mean delay of $m$, ADSAGA converges in $\tilde{O}\left(\left(n + \sqrt{m}\kappa\right)\log(1/\epsilon)\right)$ iterations, where $n$ is the number of component functions, and $\kappa$ is a condition number. This complexity sits squarely between the complexity $\tilde{O}\left(\left(n + \kappa\right)\log(1/\epsilon)\right)$ of SAGA \textit{without delays} and the complexity $\tilde{O}\left(\left(n + m\kappa\right)\log(1/\epsilon)\right)$ of parallel asynchronous algorithms where the delays are \textit{arbitrary} (but bounded by $O(m)$), and the data is accessible by all. Existing asynchronous algorithms with distributed-data setting and arbitrary delays have only been shown to converge in $\tilde{O}(n^2\kappa\log(1/\epsilon))$ iterations. We empirically compare on least-squares problems the iteration complexity and wallclock performance of ADSAGA to existing parallel and distributed algorithms, including synchronous minibatch algorithms. Our results demonstrate the wallclock advantage of variance-reduced asynchronous approaches over SGD or synchronous approaches.