pwc-archive/papers-with-abstracts huggingface dataset:
HDMI: High-order Deep Multiplex Infomax
Authors: Baoyu Jing, Chanyoung Park, Hanghang Tong
Date: 2021-02-15 00:00:00
Tasks: Node Classification, Representation Learning, Vocal Bursts Intensity Prediction

Networks have been widely used to represent the relations between objects such as academic networks and social networks, and learning embedding for networks has thus garnered plenty of research attention. Self-supervised network representation learning aims at extracting node embedding without external supervision. Recently, maximizing the mutual information between the local node embedding and the global summary (e.g. Deep Graph Infomax, or DGI for short) has shown promising results on many downstream tasks such as node classification. However, there are two major limitations of DGI. Firstly, DGI merely considers the extrinsic supervision signal (i.e., the mutual information between node embedding and global summary) while ignores the intrinsic signal (i.e., the mutual dependence between node embedding and node attributes). Secondly, nodes in a real-world network are usually connected by multiple edges with different relations, while DGI does not fully explore the various relations among nodes. To address the above-mentioned problems, we propose a novel framework, called High-order Deep Multiplex Infomax (HDMI), for learning node embedding on multiplex networks in a self-supervised way. To be more specific, we first design a joint supervision signal containing both extrinsic and intrinsic mutual information by high-order mutual information, and we propose a High-order Deep Infomax (HDI) to optimize the proposed supervision signal. Then we propose an attention based fusion module to combine node embedding from different layers of the multiplex network. Finally, we evaluate the proposed HDMI on various downstream tasks such as unsupervised clustering and supervised classification. The experimental results show that HDMI achieves state-of-the-art performance on these tasks.
Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring
Authors: Sam Sattarzadeh, Mahesh Sudhakar, Konstantinos N. Plataniotis, Jongseong Jang, Yeonjeong Jeong, Hyunwoo Kim
Date: 2021-02-15 00:00:00
Tasks: Object Localization, Sensitivity

Visualizing the features captured by Convolutional Neural Networks (CNNs) is one of the conventional approaches to interpret the predictions made by these models in numerous image recognition applications. Grad-CAM is a popular solution that provides such a visualization by combining the activation maps obtained from the model. However, the average gradient-based terms deployed in this method underestimates the contribution of the representations discovered by the model to its predictions. Addressing this problem, we introduce a solution to tackle this issue by computing the path integral of the gradient-based terms in Grad-CAM. We conduct a thorough analysis to demonstrate the improvement achieved by our method in measuring the importance of the extracted representations for the CNN's predictions, which yields to our method's administration in object localization and model interpretation.
Scaling Up Exact Neural Network Compression by ReLU Stability
Authors: Thiago Serra, Xin Yu, Abhinav Kumar, Srikumar Ramalingam
Date: 2021-02-15 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Neural Network Compression

We can compress a rectifier network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable. However, current approaches to determine the stability of neurons with Rectified Linear Unit (ReLU) activations require solving or finding a good approximation to multiple discrete optimization problems. In this work, we introduce an algorithm based on solving a single optimization problem to identify all stable neurons. Our approach is on median 183 times faster than the state-of-art method on CIFAR-10, which allows us to explore exact compression on deeper (5 x 100) and wider (2 x 800) networks within minutes. For classifiers trained under an amount of L1 regularization that does not worsen accuracy, we can remove up to 56% of the connections on the CIFAR-10 dataset. The code is available at the following link, https://github.com/yuxwind/ExactCompression.
Efficient Learning with Arbitrary Covariate Shift
Authors: Adam Kalai, Varun Kanade
Date: 2021-02-15 00:00:00

We give an efficient algorithm for learning a binary function in a given class C of bounded VC dimension, with training data distributed according to P and test data according to Q, where P and Q may be arbitrary distributions over X. This is the generic form of what is called covariate shift, which is impossible in general as arbitrary P and Q may not even overlap. However, recently guarantees were given in a model called PQ-learning (Goldwasser et al., 2020) where the learner has: (a) access to unlabeled test examples from Q (in addition to labeled samples from P, i.e., semi-supervised learning); and (b) the option to reject any example and abstain from classifying it (i.e., selective classification). The algorithm of Goldwasser et al. (2020) requires an (agnostic) noise tolerant learner for C. The present work gives a polynomial-time PQ-learning algorithm that uses an oracle to a "reliable" learner for C, where reliable learning (Kalai et al., 2012) is a model of learning with one-sided noise. Furthermore, our reduction is optimal in the sense that we show the equivalence of reliable and PQ learning.
Enhancing the Spatio-temporal Observability of Grid-Edge Resources in Distribution Grids
Authors: Shanny Lin, Hao Zhu
Date: 2021-02-15 00:00:00

Enhancing the spatio-temporal observability of distributed energy resources (DERs) is crucial for achieving secure and efficient operations in distribution grids. This paper puts forth a joint recovery framework for residential loads by leveraging the complimentary strengths of heterogeneous measurements in real time. The proposed framework integrates low-resolution smart meter data collected at every load node with fast-sampled feeder-level measurements from limited number of distribution phasor measurement units. To address the lack of data, we exploit two key characteristics for the loads and DERs, namely the sparse changes due to infrequent activities of appliances and electric vehicles (EVs) and the locational dependence of solar photovoltaic (PV) generation. Accordingly, meaningful regularization terms are introduced to cast a convex load recovery problem, which will be further simplified to reduce the computational complexity. The load recovery solutions can be utilized to identify the EV charging events at each load node and to infer the total behind-the-meter PV output. Numerical tests using real-world data have demonstrated the effectiveness of the proposed approaches in enhancing the visibility of grid-edge DERs.
Top-$k$ eXtreme Contextual Bandits with Arm Hierarchy
Authors: Rajat Sen, Alexander Rakhlin, Lexing Ying, Rahul Kidambi, Dean Foster, Daniel Hill, Inderjit Dhillon
Date: 2021-02-15 00:00:00
Tasks: Computational Efficiency, Extreme Multi-Label Classification, Multi-Armed Bandits, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Recommendation Systems

Motivated by modern applications, such as online advertisement and recommender systems, we study the top-$k$ extreme contextual bandits problem, where the total number of arms can be enormous, and the learner is allowed to select $k$ arms and observe all or some of the rewards for the chosen arms. We first propose an algorithm for the non-extreme realizable setting, utilizing the Inverse Gap Weighting strategy for selecting multiple arms. We show that our algorithm has a regret guarantee of $O(k\sqrt{(A-k+1)T \log (|\mathcal{F}|T)})$, where $A$ is the total number of arms and $\mathcal{F}$ is the class containing the regression function, while only requiring $\tilde{O}(A)$ computation per time step. In the extreme setting, where the total number of arms can be in the millions, we propose a practically-motivated arm hierarchy model that induces a certain structure in mean rewards to ensure statistical and computational efficiency. The hierarchical structure allows for an exponential reduction in the number of relevant arms for each context, thus resulting in a regret guarantee of $O(k\sqrt{(\log A-k+1)T \log (|\mathcal{F}|T)})$. Finally, we implement our algorithm using a hierarchical linear function class and show superior performance with respect to well-known benchmarks on simulated bandit feedback experiments using extreme multi-label classification datasets. On a dataset with three million arms, our reduction scheme has an average inference time of only 7.9 milliseconds, which is a 100x improvement.
Ada-SISE: Adaptive Semantic Input Sampling for Efficient Explanation of Convolutional Neural Networks
Authors: Mahesh Sudhakar, Sam Sattarzadeh, Konstantinos N. Plataniotis, Jongseong Jang, Yeonjeong Jeong, Hyunwoo Kim
Date: 2021-02-15 00:00:00
Tasks: Computational Efficiency, Explainable Artificial Intelligence (XAI)

Explainable AI (XAI) is an active research area to interpret a neural network's decision by ensuring transparency and trust in the task-specified learned models. Recently, perturbation-based model analysis has shown better interpretation, but backpropagation techniques are still prevailing because of their computational efficiency. In this work, we combine both approaches as a hybrid visual explanation algorithm and propose an efficient interpretation method for convolutional neural networks. Our method adaptively selects the most critical features that mainly contribute towards a prediction to probe the model by finding the activated features. Experimental results show that the proposed method can reduce the execution time up to 30% while enhancing competitive interpretability without compromising the quality of explanation generated.
Universal Adversarial Examples and Perturbations for Quantum Classifiers
Authors: Weiyuan Gong, Dong-Ling Deng
Date: 2021-02-15 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning

Quantum machine learning explores the interplay between machine learning and quantum physics, which may lead to unprecedented perspectives for both fields. In fact, recent works have shown strong evidences that quantum computers could outperform classical computers in solving certain notable machine learning tasks. Yet, quantum learning systems may also suffer from the vulnerability problem: adding a tiny carefully-crafted perturbation to the legitimate input data would cause the systems to make incorrect predictions at a notably high confidence level. In this paper, we study the universality of adversarial examples and perturbations for quantum classifiers. Through concrete examples involving classifications of real-life images and quantum phases of matter, we show that there exist universal adversarial examples that can fool a set of different quantum classifiers. We prove that for a set of $k$ classifiers with each receiving input data of $n$ qubits, an $O(\frac{\ln k} {2^n})$ increase of the perturbation strength is enough to ensure a moderate universal adversarial risk. In addition, for a given quantum classifier we show that there exist universal adversarial perturbations, which can be added to different legitimate samples and make them to be adversarial examples for the classifier. Our results reveal the universality perspective of adversarial attacks for quantum machine learning systems, which would be crucial for practical applications of both near-term and future quantum technologies in solving machine learning problems.
PeriodNet: A non-autoregressive waveform generation model with a structure separating periodic and aperiodic components
Authors: Yukiya Hono, Shinji Takaki, Kei Hashimoto, Keiichiro Oura, Yoshihiko Nankaku, Keiichi Tokuda
Date: 2021-02-15 00:00:00

We propose PeriodNet, a non-autoregressive (non-AR) waveform generation model with a new model structure for modeling periodic and aperiodic components in speech waveforms. The non-AR waveform generation models can generate speech waveforms parallelly and can be used as a speech vocoder by conditioning an acoustic feature. Since a speech waveform contains periodic and aperiodic components, both components should be appropriately modeled to generate a high-quality speech waveform. However, it is difficult to decompose the components from a natural speech waveform in advance. To address this issue, we propose a parallel model and a series model structure separating periodic and aperiodic components. The features of our proposed models are that explicit periodic and aperiodic signals are taken as input, and external periodic/aperiodic decomposition is not needed in training. Experiments using a singing voice corpus show that our proposed structure improves the naturalness of the generated waveform. We also show that the speech waveforms with a pitch outside of the training data range can be generated with more naturalness.
Online learning of Riemannian hidden Markov models in homogeneous Hadamard spaces
Authors: Quinten Tupker, Salem Said, Cyrus Mostajeran
Date: 2021-02-15 00:00:00

Hidden Markov models with observations in a Euclidean space play an important role in signal and image processing. Previous work extending to models where observations lie in Riemannian manifolds based on the Baum-Welch algorithm suffered from high memory usage and slow speed. Here we present an algorithm that is online, more accurate, and offers dramatic improvements in speed and efficiency.
Communication-efficient Distributed Cooperative Learning with Compressed Beliefs
Authors: Mohammad Taha Toghani, César A. Uribe
Date: 2021-02-14 00:00:00

We study the problem of distributed cooperative learning, where a group of agents seeks to agree on a set of hypotheses that best describes a sequence of private observations. In the scenario where the set of hypotheses is large, we propose a belief update rule where agents share compressed (either sparse or quantized) beliefs with an arbitrary positive compression rate. Our algorithm leverages a unified communication rule that enables agents to access wide-ranging compression operators as black-box modules. We prove the almost sure asymptotic exponential convergence of beliefs around the set of optimal hypotheses. Additionally, we show a non-asymptotic, explicit, and linear concentration rate in probability of the beliefs on the optimal hypothesis set. We provide numerical experiments to illustrate the communication benefits of our method. The simulation results show that the number of transmitted bits can be reduced to 5-10% of the non-compressed method in the studied scenarios.
On Technical Trading and Social Media Indicators in Cryptocurrencies' Price Classification Through Deep Learning
Authors: Marco Ortu, Nicola Uras, Claudio Conversano, Giuseppe Destefanis, Silvia Bartolucci
Date: 2021-02-13 00:00:00
Tasks: General Classification, Time Series Analysis, Time Series Classification

This work aims to analyse the predictability of price movements of cryptocurrencies on both hourly and daily data observed from January 2017 to January 2021, using deep learning algorithms. For our experiments, we used three sets of features: technical, trading and social media indicators, considering a restricted model of only technical indicators and an unrestricted model with technical, trading and social media indicators. We verified whether the consideration of trading and social media indicators, along with the classic technical variables (such as price's returns), leads to a significative improvement in the prediction of cryptocurrencies price's changes. We conducted the study on the two highest cryptocurrencies in volume and value (at the time of the study): Bitcoin and Ethereum. We implemented four different machine learning algorithms typically used in time-series classification problems: Multi Layers Perceptron (MLP), Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) neural network and Attention Long Short Term Memory (ALSTM). We devised the experiments using the advanced bootstrap technique to consider the variance problem on test samples, which allowed us to evaluate a more reliable estimate of the model's performance. Furthermore, the Grid Search technique was used to find the best hyperparameters values for each implemented algorithm. The study shows that, based on the hourly frequency results, the unrestricted model outperforms the restricted one. The addition of the trading indicators to the classic technical indicators improves the accuracy of Bitcoin and Ethereum price's changes prediction, with an increase of accuracy from a range of 51-55% for the restricted model, to 67-84% for the unrestricted model.
Variable importance scores
Authors: Wei-Yin Loh, Peigen Zhou
Date: 2021-02-13 00:00:00
Tasks: Missing Values

Scoring of variables for importance in predicting a response is an ill-defined concept. Several methods have been proposed but little is known of their performance. This paper fills the gap with a comparative evaluation of eleven methods and an updated one based on the GUIDE algorithm. For data without missing values, eight of the methods are shown to be biased in that they give higher or lower scores to different types of variables, even when all are independent of the response. Of the remaining four methods, only two are applicable to data with missing values, with GUIDE the only unbiased one. GUIDE achieves unbiasedness by using a self-calibrating step that is applicable to other methods for score de-biasing. GUIDE also yields a threshold for distinguishing important from unimportant variables at 95 and 99 percent confidence levels; the technique is applicable to other methods as well. Finally, the paper studies the relationship of the scores to predictive power in three data sets. It is found that the scores of many methods are more consistent with marginal predictive power than conditional predictive power.
A Comparison of Deep-Learning Methods for Analysing and Predicting Business Processes
Authors: Ishwar Venugopal, Jessica Töllich, Michael Fairbank, Ansgar Scherp
Date: 2021-02-11 00:00:00
Tasks: Prediction

Deep-learning models such as Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) have been successfully used for process-mining tasks. They have achieved better performance for different predictive tasks than traditional approaches. We extend the existing body of research by testing four different variants of Graph Neural Networks (GNN) and a fully connected Multi-layer Perceptron (MLP) with dropout for the tasks of predicting the nature and timestamp of the next process activity. In contrast to existing studies, we evaluate our models' performance at different stages of a process, determined by quartiles of the number of events and normalized quarters of the case duration. This provides new insights into the performance of a prediction model, as they behave differently at different stages of a business-process. Interestingly, our experiments show that the simple MLP often outperforms more sophisticated deep-learning models in both prediction tasks. We argue that care needs to be taken when applying automated process-prediction techniques at different stages of a process. We further argue that researchers should reflect their results with strong baselines methods like simple MLPs.
Four Generations of Control Theory Development ?
Authors: Tai Cheng Yang
Date: 2021-02-11 00:00:00

This short article presents an opinion that control system study up to date can be divided into four generations; namely, 1 transfer function based; 2 state-space based; 3 networked control systems; and 4 control in the new AI era.
Differentiating Surgeon Expertise Solely by Eye Movement Features
Authors: Benedikt Hosp, Myat Su Yin, Peter Haddawy, Paphon Sa-Ngasoongsong, Enkelejda Kasneci
Date: 2021-02-11 00:00:00
Tasks: Diagnostic

Developments in computer science in recent years are moving into hospitals. Surgeons are faced with ever new technical challenges. Visual perception plays a key role in most of these. Diagnostic and training models are needed to optimize the training of young surgeons. In this study, we present a model for classifying experts, 4th-year residents and 3rd-year residents, using only eye movements. We show a model that uses a minimal set of features and still achieve a robust accuracy of 76.46 % to classify eye movements into the correct class. Likewise, in this study, we address the evolutionary steps of visual perception between three expertise classes, forming a first step towards a diagnostic model for expertise.
Convergence of Nonlinear Observers on R^n with a Riemannian Metric (Part III)
Authors: Ricardo G. Sanfelice, Laurent Praly
Date: 2021-02-16 00:00:00

This paper is the third and final component of a three-part effort on observers contracting a Riemannian distance between the state of the system and its estimate. In Part I, we showed that such a contraction property holds if the system dynamics and the Riemannian metric satisfy two key conditions: a differential detectability property and a geodesic monotonicity property. With the former condition being the focus of Part II, in this Part III, we study the latter condition in relationship to the nullity of the second fundamental form of the output function. We formulate sufficient and necessary conditions for it to hold. We establish a link between it and the infinite gain margin property, and we provide a systematic way for constructing a metric satisfying this condition. Finally, we illustrate cases where both conditions hold and propose ways to facilitate the satisfaction of these two conditions together.
Multilayer heat equations: application to finance
Authors: A. Itkin, A. Lipton, D. Muravey
Date: 2021-02-16 00:00:00

In this paper, we develop a Multilayer (ML) method for solving one-factor parabolic equations. Our approach provides a powerful alternative to the well-known finite difference and Monte Carlo methods. We discuss various advantages of this approach, which judiciously combines semi-analytical and numerical techniques and provides a fast and accurate way of finding solutions to the corresponding equations. To introduce the core of the method, we consider multilayer heat equations, known in physics for a relatively long time but never used when solving financial problems. Thus, we expand the analytic machinery of quantitative finance by augmenting it with the ML method. We demonstrate how one can solve various problems of mathematical finance by using our approach. Specifically, we develop efficient algorithms for pricing barrier options for time-dependent one-factor short-rate models, such as Black-Karasinski and Verhulst. Besides, we show how to solve the well-known Dupire equation quickly and accurately. Numerical examples confirm that our approach is considerably more efficient for solving the corresponding partial differential equations than the conventional finite difference method by being much faster and more accurate than the known alternatives.