pwc-archive/papers-with-abstracts huggingface dataset:
SEMULATOR: Emulating the Dynamics of Crossbar Array-based Analog Neural System with Regression Neural Networks
Authors: Chaeun Lee, Seyoung Kim
Date: 2021-01-19 00:00:00
Tasks: regression

As deep neural networks require tremendous amount of computation and memory, analog computing with emerging memory devices is a promising alternative to digital computing for edge devices. However, because of the increasing simulation time for analog computing system, it has not been explored. To overcome this issue, analytically approximated simulators are developed, but these models are inaccurate and narrow down the options for peripheral circuits for multiply-accumulate operation (MAC). In this sense, we propose a methodology, SEMULATOR (SiMULATOR by Emulating the analog computing block) which uses a deep neural network to emulate the behavior of crossbar-based analog computing system. With the proposed neural architecture, we experimentally and theoretically shows that it emulates a MAC unit for neural computation. In addition, the simulation time is incomparably reduced when it compared to the circuit simulators such as SPICE.
Near-Optimal Regret Bounds for Contextual Combinatorial Semi-Bandits with Linear Payoff Functions
Authors: Kei Takemura, Shinji Ito, Daisuke Hatano, Hanna Sumita, Takuro Fukunaga, Naonori Kakimura, Ken-ichi Kawarabayashi
Date: 2021-01-20 00:00:00
Tasks: Decision Making, Recommendation Systems

The contextual combinatorial semi-bandit problem with linear payoff functions is a decision-making problem in which a learner chooses a set of arms with the feature vectors in each round under given constraints so as to maximize the sum of rewards of arms. Several existing algorithms have regret bounds that are optimal with respect to the number of rounds $T$. However, there is a gap of $\tilde{O}(\max(\sqrt{d}, \sqrt{k}))$ between the current best upper and lower bounds, where $d$ is the dimension of the feature vectors, $k$ is the number of the chosen arms in a round, and $\tilde{O}(\cdot)$ ignores the logarithmic factors. The dependence of $k$ and $d$ is of practical importance because $k$ may be larger than $T$ in real-world applications such as recommender systems. In this paper, we fill the gap by improving the upper and lower bounds. More precisely, we show that the C${}^2$UCB algorithm proposed by Qin, Chen, and Zhu (2014) has the optimal regret bound $\tilde{O}(d\sqrt{kT} + dk)$ for the partition matroid constraints. For general constraints, we propose an algorithm that modifies the reward estimates of arms in the C${}^2$UCB algorithm and demonstrate that it enjoys the optimal regret bound for a more general problem that can take into account other objectives simultaneously. We also show that our technique would be applicable to related problems. Numerical experiments support our theoretical results and considerations.
Robust W-GAN-Based Estimation Under Wasserstein Contamination
Authors: Zheng Liu, Po-Ling Loh
Date: 2021-01-20 00:00:00
Tasks: regression

Robust estimation is an important problem in statistics which aims at providing a reasonable estimator when the data-generating distribution lies within an appropriately defined ball around an uncontaminated distribution. Although minimax rates of estimation have been established in recent years, many existing robust estimators with provably optimal convergence rates are also computationally intractable. In this paper, we study several estimation problems under a Wasserstein contamination model and present computationally tractable estimators motivated by generative adversarial networks (GANs). Specifically, we analyze properties of Wasserstein GAN-based estimators for location estimation, covariance matrix estimation, and linear regression and show that our proposed estimators are minimax optimal in many scenarios. Finally, we present numerical results which demonstrate the effectiveness of our estimators.
On the Non-Monotonicity of a Non-Differentially Mismeasured Binary Confounder
Authors: Jose M. Peña
Date: 2021-01-20 00:00:00

Suppose that we are interested in the average causal effect of a binary treatment on an outcome when this relationship is confounded by a binary confounder. Suppose that the confounder is unobserved but a non-differential binary proxy of it is observed. We identify conditions under which adjusting for the proxy comes closer to the incomputable true average causal effect than not adjusting at all. Unlike other works, we do not assume that the average causal effect of the confounder on the outcome is in the same direction among treated and untreated.
Evaluating uncertainties in electrochemical impedance spectra of solid oxide fuel cells
Authors: Luka Žnidarič, Gjorgji Nusev, Bertrand Morel, Julie Mougin, Đani Juričić, Pavle Boškoski
Date: 2021-01-20 00:00:00
Tasks: Diagnostic

Electrochemical impedance spectroscopy (EIS) is a widely used tool for characterization of fuel cells and other electrochemical conversion systems. When applied to the on-line monitoring in the context of in-field applications, the disturbances, drifts and sensor noise may cause severe distortions in the evaluated spectra, especially in the low-frequency part. Failure to ignore the random effects can result in misinterpreted spectra and, consequently, in misleading diagnostic reasoning. This fact has not been often addressed in the research so far. In this paper, we propose an approach to the quantification of the spectral uncertainty, which relies on evaluating the uncertainty of the equivalent circuit model (ECM). We apply the computationally efficient variational Bayes (VB) method and compare the quality of the results with those obtained with the Markov chain Monte Carlo (MCMC) algorithm. Namely, MCMC algorithm returns accurate distributions of the estimated model parameters, while VB approach provides the approximate distributions. By using simulated and real data we show that approximate results provided by VB approach, although slightly over-optimistic, are still close to the more realistic MCMC estimates. A great advantage of the VB method for online monitoring is low computational load, which is several orders of magnitude lower compared to MCMC. The performance of VB algorithm is demonstrated on a case of ECM parameters estimation in a 6 cell solid oxide fuel cell (SOFC) stack. The complete numerical implementation for recreating the results can be found at https://repo.ijs.si/lznidaric/variational-bayes-supplementary-material.
A Similarity Measure of Gaussian Process Predictive Distributions
Authors: Lucia Asencio-Martín, Eduardo C. Garrido-Merchán
Date: 2021-01-20 00:00:00

Some scenarios require the computation of a predictive distribution of a new value evaluated on an objective function conditioned on previous observations. We are interested on using a model that makes valid assumptions on the objective function whose values we are trying to predict. Some of these assumptions may be smoothness or stationarity. Gaussian process (GPs) are probabilistic models that can be interpreted as flexible distributions over functions. They encode the assumptions through covariance functions, making hypotheses about new data through a predictive distribution by being fitted to old observations. We can face the case where several GPs are used to model different objective functions. GPs are non-parametric models whose complexity is cubic on the number of observations. A measure that represents how similar is one GP predictive distribution with respect to another would be useful to stop using one GP when they are modelling functions of the same input space. We are really inferring that two objective functions are correlated, so one GP is enough to model both of them by performing a transformation of the prediction of the other function in case of inverse correlation. We show empirical evidence in a set of synthetic and benchmark experiments that GPs predictive distributions can be compared and that one of them is enough to predict two correlated functions in the same input space. This similarity metric could be extremely useful used to discard objectives in Bayesian many-objective optimization.
Machine learning applications for COVID-19: A state-of-the-art review
Authors: Firuz Kamalov, Aswani Cherukuri, Hana Sulieman, Fadi Thabtah, Akbar Hossain
Date: 2021-01-19 00:00:00
Tasks: BIG-bench Machine Learning

The COVID-19 pandemic has galvanized the machine learning community to create new solutions that can help in the fight against the virus. The body of literature related to applications of machine learning and artificial intelligence to COVID-19 is constantly growing. The goal of this article is to present the latest advances in machine learning research applied to COVID-19. We cover four major areas of research: forecasting, medical diagnostics, drug development, and contact tracing. We review and analyze the most successful state of the art studies. In contrast to other existing surveys on the subject, our article presents a high level overview of the current research that is sufficiently detailed to provide an informed insight.
Learning Abstract Task Representations
Authors: Mikhail M. Meskhi, Adriano Rivolli, Rafael G. Mantovani, Ricardo Vilalta
Date: 2021-01-19 00:00:00
Tasks: Meta-Learning

A proper form of data characterization can guide the process of learning-algorithm selection and model-performance estimation. The field of meta-learning has provided a rich body of work describing effective forms of data characterization using different families of meta-features (statistical, model-based, information-theoretic, topological, etc.). In this paper, we start with the abundant set of existing meta-features and propose a method to induce new abstract meta-features as latent variables in a deep neural network. We discuss the pitfalls of using traditional meta-features directly and argue for the importance of learning high-level task properties. We demonstrate our methodology using a deep neural network as a feature extractor. We demonstrate that 1) induced meta-models mapping abstract meta-features to generalization performance outperform other methods by ~18% on average, and 2) abstract meta-features attain high feature-relevance scores.
Intelligent Icing Detection Model of Wind Turbine Blades Based on SCADA data
Authors: Wenqian Jiang, Junyang Jin
Date: 2021-01-20 00:00:00
Tasks: Domain Adaptation, Feature Engineering

Diagnosis of ice accretion on wind turbine blades is all the time a hard nut to crack in condition monitoring of wind farms. Existing methods focus on mechanism analysis of icing process, deviation degree analysis of feature engineering. However, there have not been deep researches of neural networks applied in this field at present. Supervisory control and data acquisition (SCADA) makes it possible to train networks through continuously providing not only operation parameters and performance parameters of wind turbines but also environmental parameters and operation modes. This paper explores the possibility that using convolutional neural networks (CNNs), generative adversarial networks (GANs) and domain adaption learning to establish intelligent diagnosis frameworks under different training scenarios. Specifically, PGANC and PGANT are proposed for sufficient and non-sufficient target wind turbine labeled data, respectively. The basic idea is that we consider a two-stage training with parallel GANs, which are aimed at capturing intrinsic features for normal and icing samples, followed by classification CNN or domain adaption module in various training cases. Model validation on three wind turbine SCADA data shows that two-stage training can effectively improve the model performance. Besides, if there is no sufficient labeled data for a target turbine, which is an extremely common phenomenon in real industrial practices, the addition of domain adaption learning makes the trained model show better performance. Overall, our proposed intelligent diagnosis frameworks can achieve more accurate detection on the same wind turbine and more generalized capability on a new wind turbine, compared with other machine learning models and conventional CNNs.
SparseDNN: Fast Sparse Deep Learning Inference on CPUs
Authors: Ziheng Wang
Date: 2021-01-20 00:00:00
Tasks: Deep Learning, Quantization

The last few years have seen gigantic leaps in algorithms and systems to support efficient deep learning inference. Pruning and quantization algorithms can now consistently compress neural networks by an order of magnitude. For a compressed neural network, a multitude of inference frameworks have been designed to maximize the performance of the target hardware. While we find mature support for quantized neural networks in production frameworks such as OpenVINO and MNN, support for pruned sparse neural networks is still lacking. To tackle this challenge, we present SparseDNN, a sparse deep learning inference engine targeting CPUs. We present both kernel-level optimizations with a sparse code generator to accelerate sparse operators and novel network-level optimizations catering to sparse networks. We show that our sparse code generator can achieve significant speedups over state-of-the-art sparse and dense libraries. On end-to-end benchmarks such as Huggingface pruneBERT, SparseDNN achieves up to 5x throughput improvement over dense inference with state-of-the-art OpenVINO. Open source library at: https://github.com/marsupialtail/sparsednn.
Convolutional conditional neural processes for local climate downscaling
Authors: Anna Vaughan, Will Tebbutt, J. Scott Hosking, Richard E. Turner
Date: 2021-01-20 00:00:00
Tasks: Gaussian Processes

A new model is presented for multisite statistical downscaling of temperature and precipitation using convolutional conditional neural processes (convCNPs). ConvCNPs are a recently developed class of models that allow deep learning techniques to be applied to off-the-grid spatio-temporal data. This model has a substantial advantage over existing downscaling methods in that the trained model can be used to generate multisite predictions at an arbitrary set of locations, regardless of the availability of training data. The convCNP model is shown to outperform an ensemble of existing downscaling techniques over Europe for both temperature and precipitation taken from the VALUE intercomparison project. The model also outperforms an approach that uses Gaussian processes to interpolate single-site downscaling models at unseen locations. Importantly, substantial improvement is seen in the representation of extreme precipitation events. These results indicate that the convCNP is a robust downscaling model suitable for generating localised projections for use in climate impact studies, and motivates further research into applications of deep learning techniques in statistical downscaling.
PyTorch-Direct: Enabling GPU Centric Data Access for Very Large Graph Neural Network Training with Irregular Accesses
Authors: Seung Won Min, Kun Wu, Sitao Huang, Mert Hidayetoğlu, JinJun Xiong, Eiman Ebrahimi, Deming Chen, Wen-mei Hwu
Date: 2021-01-20 00:00:00
Tasks: CPU, GPU, Graph Neural Network

With the increasing adoption of graph neural networks (GNNs) in the machine learning community, GPUs have become an essential tool to accelerate GNN training. However, training GNNs on very large graphs that do not fit in GPU memory is still a challenging task. Unlike conventional neural networks, mini-batching input samples in GNNs requires complicated tasks such as traversing neighboring nodes and gathering their feature values. While this process accounts for a significant portion of the training time, we find existing GNN implementations using popular deep neural network (DNN) libraries such as PyTorch are limited to a CPU-centric approach for the entire data preparation step. This "all-in-CPU" approach has negative impact on the overall GNN training performance as it over-utilizes CPU resources and hinders GPU acceleration of GNN training. To overcome such limitations, we introduce PyTorch-Direct, which enables a GPU-centric data accessing paradigm for GNN training. In PyTorch-Direct, GPUs are capable of efficiently accessing complicated data structures in host memory directly without CPU intervention. Our microbenchmark and end-to-end GNN training results show that PyTorch-Direct reduces data transfer time by 47.1% on average and speeds up GNN training by up to 1.6x. Furthermore, by reducing CPU utilization, PyTorch-Direct also saves system power by 12.4% to 17.5% during training. To minimize programmer effort, we introduce a new "unified tensor" type along with necessary changes to the PyTorch memory allocator, dispatch logic, and placement rules. As a result, users need to change at most two lines of their PyTorch GNN training code for each tensor object to take advantage of PyTorch-Direct.
Representation Evaluation Block-based Teacher-Student Network for the Industrial Quality-relevant Performance Modeling and Monitoring
Authors: Dan Yang, Xin Peng, Yusheng Lu, Haojie Huang, Weimin Zhong
Date: 2021-01-20 00:00:00
Tasks: Fault Detection

Quality-relevant fault detection plays an important role in industrial processes, while the current quality-related fault detection methods based on neural networks main concentrate on process-relevant variables and ignore quality-relevant variables, which restrict the application of process monitoring. Therefore, in this paper, a fault detection scheme based on the improved teacher-student network is proposed for quality-relevant fault detection. In the traditional teacher-student network, as the features differences between the teacher network and the student network will cause performance degradation on the student network, representation evaluation block (REB) is proposed to quantify the features differences between the teacher and the student networks, and uncertainty modeling is used to add this difference in modeling process, which are beneficial to reduce the features differences and improve the performance of the student network. Accordingly, REB and uncertainty modeling is applied in the teacher-student network named as uncertainty modeling teacher-student uncertainty autoencoder (TSUAE). Then, the proposed TSUAE is applied to process monitoring, which can effectively detect faults in the process-relevant subspace and quality-relevant subspace simultaneously. The proposed TSUAE-based fault detection method is verified in two simulation experiments illustrating that it has satisfactory fault detection performance compared to other fault detection methods.
Deep Learning for Intelligent Demand Response and Smart Grids: A Comprehensive Survey
Authors: Prabadevi B, Quoc-Viet Pham, Madhusanka Liyanage, N Deepa, Mounik VVSS, Shivani Reddy, Praveen Kumar Reddy Maddikunta, Neelu Khare, Thippa Reddy Gadekallu, Won-Joo Hwang
Date: 2021-01-20 00:00:00
Tasks: Load Forecasting, Management, State Estimation

Electricity is one of the mandatory commodities for mankind today. To address challenges and issues in the transmission of electricity through the traditional grid, the concepts of smart grids and demand response have been developed. In such systems, a large amount of data is generated daily from various sources such as power generation (e.g., wind turbines), transmission and distribution (microgrids and fault detectors), load management (smart meters and smart electric appliances). Thanks to recent advancements in big data and computing technologies, Deep Learning (DL) can be leveraged to learn the patterns from the generated data and predict the demand for electricity and peak hours. Motivated by the advantages of deep learning in smart grids, this paper sets to provide a comprehensive survey on the application of DL for intelligent smart grids and demand response. Firstly, we present the fundamental of DL, smart grids, demand response, and the motivation behind the use of DL. Secondly, we review the state-of-the-art applications of DL in smart grids and demand response, including electric load forecasting, state estimation, energy theft detection, energy sharing and trading. Furthermore, we illustrate the practicality of DL via various use cases and projects. Finally, we highlight the challenges presented in existing research works and highlight important issues and potential directions in the use of DL for smart grids and demand response.
Riemannian Manifold Optimization for Discriminant Subspace Learning
Authors: Wanguang Yin, Zhengming Ma, Quanying Liu
Date: 2021-01-20 00:00:00
Tasks: General Classification, image-classification, Image Classification, Tensor Decomposition

Linear discriminant analysis (LDA) is a widely used algorithm in machine learning to extract a low-dimensional representation of high-dimensional data, it features to find the orthogonal discriminant projection subspace by using the Fisher discriminant criterion. However, the traditional Euclidean-based methods for solving LDA are easily convergent to spurious local minima and hardly obtain an optimal solution. To address such a problem, in this paper, we propose a novel algorithm namely Riemannian-based discriminant analysis (RDA) for subspace learning. In order to obtain an explicit solution, we transform the traditional Euclidean-based methods to the Riemannian manifold space and use the trust-region method to learn the discriminant projection subspace. We compare the proposed algorithm to existing variants of LDA, as well as the unsupervised tensor decomposition methods on image classification tasks. The numerical results suggest that RDA achieves state-of-the-art performance in classification accuracy.
A Generalized Weisfeiler-Lehman Graph Kernel
Authors: Till Hendrik Schulz, Tamás Horváth, Pascal Welke, Stefan Wrobel
Date: 2021-01-20 00:00:00

The Weisfeiler-Lehman graph kernels are among the most prevalent graph kernels due to their remarkable time complexity and predictive performance. Their key concept is based on an implicit comparison of neighborhood representing trees with respect to equality (i.e., isomorphism). This binary valued comparison is, however, arguably too rigid for defining suitable similarity measures over graphs. To overcome this limitation, we propose a generalization of Weisfeiler-Lehman graph kernels which takes into account the similarity between trees rather than equality. We achieve this using a specifically fitted variation of the well-known tree edit distance which can efficiently be calculated. We empirically show that our approach significantly outperforms state-of-the-art methods in terms of predictive performance on datasets containing structurally more complex graphs beyond the typically considered molecular graphs.