pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Agent Reinforcement Learning Based Coded Computation for Mobile Ad Hoc Computing
Authors: Baoqian Wang, Junfei Xie, Kejie Lu, Yan Wan, Shengli Fu
Date: 2021-04-15 00:00:00
Tasks: Distributed Computing, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Mobile ad hoc computing (MAHC), which allows mobile devices to directly share their computing resources, is a promising solution to address the growing demands for computing resources required by mobile devices. However, offloading a computation task from a mobile device to other mobile devices is a challenging task due to frequent topology changes and link failures because of node mobility, unstable and unknown communication environments, and the heterogeneous nature of these devices. To address these challenges, in this paper, we introduce a novel coded computation scheme based on multi-agent reinforcement learning (MARL), which has many promising features such as adaptability to network changes, high efficiency and robustness to uncertain system disturbances, consideration of node heterogeneity, and decentralized load allocation. Comprehensive simulation studies demonstrate that the proposed approach can outperform state-of-the-art distributed computing schemes.
On Energy-Based Models with Overparametrized Shallow Neural Networks
Authors: Carles Domingo-Enrich, Alberto Bietti, Eric Vanden-Eijnden, Joan Bruna
Date: 2021-04-15 00:00:00

Energy-based models (EBMs) are a simple yet powerful framework for generative modeling. They are based on a trainable energy function which defines an associated Gibbs measure, and they can be trained and sampled from via well-established statistical tools, such as MCMC. Neural networks may be used as energy function approximators, providing both a rich class of expressive models as well as a flexible device to incorporate data structure. In this work we focus on shallow neural networks. Building from the incipient theory of overparametrized neural networks, we show that models trained in the so-called "active" regime provide a statistical advantage over their associated "lazy" or kernel regime, leading to improved adaptivity to hidden low-dimensional structure in the data distribution, as already observed in supervised learning. Our study covers both maximum likelihood and Stein Discrepancy estimators, and we validate our theoretical results with numerical experiments on synthetic data.
QuickLoc: Adaptive Deep-Learning for Fast Indoor Localization with Mobile Devices
Authors: Saideep Tiku, Prathmesh Kale, Sudeep Pasricha
Date: 2021-04-15 00:00:00
Tasks: Deep Learning, Indoor Localization

Indoor localization services are a crucial aspect for the realization of smart cyber-physical systems within cities of the future. Such services are poised to reinvent the process of navigation and tracking of people and assets in a variety of indoor and subterranean environments. The growing ownership of computationally capable smartphones has laid the foundations of portable fingerprinting-based indoor localization through deep learning. However, as the demand for accurate localization increases, the computational complexity of the associated deep learning models increases as well. We present an approach for reducing the computational requirements of a deep learning-based indoor localization framework while maintaining localization accuracy targets. Our proposed methodology is deployed and validated across multiple smartphones and is shown to deliver up to 42% reduction in prediction latency and 45% reduction in prediction energy as compared to the best-known baseline deep learning-based indoor localization model.
FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems
Authors: Li Li, Moming Duan, Duo Liu, Yu Zhang, Ao Ren, Xianzhang Chen, Yujuan Tan, Chengliang Wang
Date: 2021-04-15 00:00:00
Tasks: Active Learning, Federated Learning

Federated Learning (FL) is a novel distributed machine learning which allows thousands of edge devices to train model locally without uploading data concentrically to the server. But since real federated settings are resource-constrained, FL is encountered with systems heterogeneity which causes a lot of stragglers directly and then leads to significantly accuracy reduction indirectly. To solve the problems caused by systems heterogeneity, we introduce a novel self-adaptive federated framework FedSAE which adjusts the training task of devices automatically and selects participants actively to alleviate the performance degradation. In this work, we 1) propose FedSAE which leverages the complete information of devices' historical training tasks to predict the affordable training workloads for each device. In this way, FedSAE can estimate the reliability of each device and self-adaptively adjust the amount of training load per client in each round. 2) combine our framework with Active Learning to self-adaptively select participants. Then the framework accelerates the convergence of the global model. In our framework, the server evaluates devices' value of training based on their training loss. Then the server selects those clients with bigger value for the global model to reduce communication overhead. The experimental result indicates that in a highly heterogeneous system, FedSAE converges faster than FedAvg, the vanilla FL framework. Furthermore, FedSAE outperforms than FedAvg on several federated datasets - FedSAE improves test accuracy by 26.7% and reduces stragglers by 90.3% on average.
Cross-domain Speech Recognition with Unsupervised Character-level Distribution Matching
Authors: Wenxin Hou, Jindong Wang, Xu Tan, Tao Qin, Takahiro Shinozaki
Date: 2021-04-15 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Cross-device ASR, Cross-environment ASR, Domain Adaptation, speech-recognition, Speech Recognition, Unsupervised Domain Adaptation

End-to-end automatic speech recognition (ASR) can achieve promising performance with large-scale training data. However, it is known that domain mismatch between training and testing data often leads to a degradation of recognition accuracy. In this work, we focus on the unsupervised domain adaptation for ASR and propose CMatch, a Character-level distribution matching method to perform fine-grained adaptation between each character in two domains. First, to obtain labels for the features belonging to each character, we achieve frame-level label assignment using the Connectionist Temporal Classification (CTC) pseudo labels. Then, we match the character-level distributions using Maximum Mean Discrepancy. We train our algorithm using the self-training technique. Experiments on the Libri-Adapt dataset show that our proposed approach achieves 14.39% and 16.50% relative Word Error Rate (WER) reduction on both cross-device and cross-environment ASR. We also comprehensively analyze the different strategies for frame-level label assignment and Transformer adaptations.
Lorentzian Graph Convolutional Networks
Authors: Yiding Zhang, Xiao Wang, Chuan Shi, Nian Liu, Guojie Song
Date: 2021-04-15 00:00:00

Graph convolutional networks (GCNs) have received considerable research attention recently. Most GCNs learn the node representations in Euclidean geometry, but that could have a high distortion in the case of embedding graphs with scale-free or hierarchical structure. Recently, some GCNs are proposed to deal with this problem in non-Euclidean geometry, e.g., hyperbolic geometry. Although hyperbolic GCNs achieve promising performance, existing hyperbolic graph operations actually cannot rigorously follow the hyperbolic geometry, which may limit the ability of hyperbolic geometry and thus hurt the performance of hyperbolic GCNs. In this paper, we propose a novel hyperbolic GCN named Lorentzian graph convolutional network (LGCN), which rigorously guarantees the learned node features follow the hyperbolic geometry. Specifically, we rebuild the graph operations of hyperbolic GCNs with Lorentzian version, e.g., the feature transformation and non-linear activation. Also, an elegant neighborhood aggregation method is designed based on the centroid of Lorentzian distance. Moreover, we prove some proposed graph operations are equivalent in different types of hyperbolic geometry, which fundamentally indicates their correctness. Experiments on six datasets show that LGCN performs better than the state-of-the-art methods. LGCN has lower distortion to learn the representation of tree-likeness graphs compared with existing hyperbolic GCNs. We also find that the performance of some hyperbolic GCNs can be improved by simply replacing the graph operations with those we defined in this paper.
Cross-Domain Label-Adaptive Stance Detection
Authors: Momchil Hardalov, Arnav Arora, Preslav Nakov, Isabelle Augenstein
Date: 2021-04-15 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Domain Adaptation, Mixture-of-Experts, Stance Detection

Stance detection concerns the classification of a writer's viewpoint towards a target. There are different task variants, e.g., stance of a tweet vs. a full article, or stance with respect to a claim vs. an (implicit) topic. Moreover, task definitions vary, which includes the label inventory, the data collection, and the annotation protocol. All these aspects hinder cross-domain studies, as they require changes to standard domain adaptation approaches. In this paper, we perform an in-depth analysis of 16 stance detection datasets, and we explore the possibility for cross-domain learning from them. Moreover, we propose an end-to-end unsupervised framework for out-of-domain prediction of unseen, user-defined labels. In particular, we combine domain adaptation techniques such as mixture of experts and domain-adversarial training with label embeddings, and we demonstrate sizable performance gains over strong baselines, both (i) in-domain, i.e., for seen targets, and (ii) out-of-domain, i.e., for unseen targets. Finally, we perform an exhaustive analysis of the cross-domain results, and we highlight the important factors influencing the model performance.
Bayesian and Dempster-Shafer models for combining multiple sources of evidence in a fraud detection system
Authors: Fabrice Daniel
Date: 2021-04-15 00:00:00
Tasks: Fraud Detection

Combining evidence from different sources can be achieved with Bayesian or Dempster-Shafer methods. The first requires an estimate of the priors and likelihoods while the second only needs an estimate of the posterior probabilities and enables reasoning with uncertain information due to imprecision of the sources and with the degree of conflict between them. This paper describes the two methods and how they can be applied to the estimation of a global score in the context of fraud detection.
Estimation of atrial fibrillation from lead-I ECGs: Comparison with cardiologists and machine learning model (CurAlive), a clinical validation study
Authors: N. Korucuk, C. Polat, E. S. Gunduz, O. Karaman, V. Tosun, M. Onac, N. Yildirim, Y. Cete, K. Polat
Date: 2021-04-15 00:00:00
Tasks: Medical Diagnosis, Rhythm

Electrocardiogram recognition of cardiac arrhythmias is critical for cardiac abnormality diagnosis. Because of their strong prediction characteristics, artificial neural networks are the preferred method in medical diagnosis systems. This study presents a method to detect atrial fibrillation with lead-I ECGs using artificial intelligence. The aim of the study is to compare the accuracy of the diagnoses estimated by cardiologists and artificial intelligence over lead-I ECGs using 12-lead ECGs as references. To evaluate the performance of the proposed model, dataset were collected from China Physiological Signal Challenge 2018. In the study, diagnoses were examined in three groups as normal sinus rhythm, atrial fibrillation and OTHER. All rhythm and beat types except NSR and AFIB were labeled as OTHER super-class. OTHER contains First-degree atrioventricular blocks, Conduction disturbances, Left bundle branch block, Right bundle branch block, Premature atrial contraction, Premature ventricular contraction, ST-segment depression and ST-segment elevated type ECGs. CurAlive A.I. model which is using DenseNet as a CNN architecture and continuous wavelet transform as feature extraction method, showed a great performance on classifying ECGs from only lead-I compared to cardiologists. The AI model reached the weighted average precision, recall, F1-score and total accuracy 94.1%, 93.6%, 93.7% and 93.6% respectively, and the average of each of the three cardiologists has reached weighted average precision, recall, F1-score and total accuracy 82.2%, 54.6%, 57.5% and 54.6% respectively. This study showed that the proposed CNN model CurAlive, can be used to accurately diagnose AFIB, NSR, and OTHER rhythm using lead-I ECGs to accelerate the early detection of AFIB as a cardiologist assistant. It is also able to identify patients into different risk groups as part of remote patient monitoring systems.
Ransomware Detection Using Deep Learning in the SCADA System of Electric Vehicle Charging Station
Authors: Manoj Basnet, Subash Poudyal, Mohd. Hasan Ali, Dipankar Dasgupta
Date: 2021-04-15 00:00:00
Tasks: Deep Learning

The Supervisory control and data acquisition (SCADA) systems have been continuously leveraging the evolution of network architecture, communication protocols, next-generation communication techniques (5G, 6G, Wi-Fi 6), and the internet of things (IoT). However, SCADA system has become the most profitable and alluring target for ransomware attackers. This paper proposes the deep learning-based novel ransomware detection framework in the SCADA controlled electric vehicle charging station (EVCS) with the performance analysis of three deep learning algorithms, namely deep neural network (DNN), 1D convolution neural network (CNN), and long short-term memory (LSTM) recurrent neural network. All three-deep learning-based simulated frameworks achieve around 97% average accuracy (ACC), more than 98% of the average area under the curve (AUC), and an average F1-score under 10-fold stratified cross-validation with an average false alarm rate (FAR) less than 1.88%. Ransomware driven distributed denial of service (DDoS) attack tends to shift the SOC profile by exceeding the SOC control thresholds. The severity has been found to increase as the attack progress and penetration increases. Also, ransomware driven false data injection (FDI) attack has the potential to damage the entire BES or physical system by manipulating the SOC control thresholds. It's a design choice and optimization issue that a deep learning algorithm can deploy based on the tradeoffs between performance metrics.
Node Co-occurrence based Graph Neural Networks for Knowledge Graph Link Prediction
Authors: Dai Quoc Nguyen, Vinh Tong, Dinh Phung, Dat Quoc Nguyen
Date: 2021-04-15 00:00:00
Tasks: Knowledge Graph Completion, Link Prediction

We introduce a novel embedding model, named NoGE, which aims to integrate co-occurrence among entities and relations into graph neural networks to improve knowledge graph completion (i.e., link prediction). Given a knowledge graph, NoGE constructs a single graph considering entities and relations as individual nodes. NoGE then computes weights for edges among nodes based on the co-occurrence of entities and relations. Next, NoGE proposes Dual Quaternion Graph Neural Networks (DualQGNN) and utilizes DualQGNN to update vector representations for entity and relation nodes. NoGE then adopts a score function to produce the triple scores. Comprehensive experimental results show that NoGE obtains state-of-the-art results on three new and difficult benchmark datasets CoDEx for knowledge graph completion.
Conditional independence for pretext task selection in Self-supervised speech representation learning
Authors: Salah Zaiem, Titouan Parcollet, Slim Essid
Date: 2021-04-15 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Pseudo Label, Representation Learning, Self-Supervised Learning, Speaker Recognition, speech-recognition, Speech Recognition, Speech Representation Learning

Through solving pretext tasks, self-supervised learning (SSL) leverages unlabeled data to extract useful latent representations replacing traditional input features in the downstream task. A common pretext task consists in pretraining a SSL model on pseudo-labels derived from the original signal. This technique is particularly relevant for speech data where various meaningful signal processing features may serve as pseudo-labels. However, the process of selecting pseudo-labels, for speech or other types of data, remains mostly unexplored and currently relies on observing the results on the final downstream task. Nevertheless, this methodology is not sustainable at scale due to substantial computational (hence carbon) costs. Thus, this paper introduces a practical and theoretical framework to select relevant pseudo-labels with respect to a given downstream task. More precisely, we propose a functional estimator of the pseudo-label utility grounded in the conditional independence theory, which does not require any training. The experiments conducted on speaker recognition and automatic speech recognition validate our estimator, showing a significant correlation between the performance observed on the downstream task and the utility estimates obtained with our approach, facilitating the prospection of relevant pseudo-labels for self-supervised speech representation learning.
Facilitating Machine Learning Model Comparison and Explanation Through A Radial Visualisation
Authors: Jianlong Zhou, Weidong Huang, Fang Chen
Date: 2021-04-15 00:00:00
Tasks: BIG-bench Machine Learning, Feature Importance

Building an effective Machine Learning (ML) model for a data set is a difficult task involving various steps. One of the most important steps is to compare generated substantial amounts of ML models to find the optimal one for the deployment. It is challenging to compare such models with dynamic number of features. Comparison is more than just finding differences of ML model performance, users are also interested in the relations between features and model performance such as feature importance for ML explanations. This paper proposes RadialNet Chart, a novel visualisation approach to compare ML models trained with a different number of features of a given data set while revealing implicit dependent relations. In RadialNet Chart, ML models and features are represented by lines and arcs respectively. These lines are generated effectively using a recursive function. The dependence of ML models with dynamic number of features is encoded into the structure of visualisation, where ML models and their dependent features are directly revealed from related line connections. ML model performance information is encoded with colour and line width in RadialNet Chart. Together with the structure of visualisation, feature importance can be directly discerned in RadialNet Chart for ML explanations.
Scale Invariant Monte Carlo under Linear Function Approximation with Curvature based step-size
Authors: Rahul Madhavan, Hemanta Makwana
Date: 2021-04-15 00:00:00

We study the feature-scaled version of the Monte Carlo algorithm with linear function approximation. This algorithm converges to a scale-invariant solution, which is not unduly affected by states having feature vectors with large norms. The usual versions of the MCMC algorithm, obtained by minimizing the least-squares criterion, do not produce solutions that give equal importance to all states irrespective of feature-vector norm -- a requirement that may be critical in many reinforcement learning contexts. To speed up convergence in our algorithm, we introduce an adaptive step-size based on the curvature of the iterate convergence path -- a novelty that may be useful in more general optimization contexts as well. A key contribution of this paper is to prove convergence, in the presence of adaptive curvature based step-size and heavy-ball momentum. We provide rigorous theoretical guarantees and use simulations to demonstrate the efficacy of our ideas.
Fast Private Parameter Learning and Inference for Sum-Product Networks
Authors: Ernst Althaus, Mohammad Sadeq Dousti, Stefan Kramer, Nick Johannes Peter Rassau
Date: 2021-04-15 00:00:00

A sum-product network (SPN) is a graphical model that allows several types of inferences to be drawn efficiently. There are two types of learning for SPNs: Learning the architecture of the model, and learning the parameters. In this paper, we tackle the second problem: We show how to learn the weights for the sum nodes, assuming the architecture is fixed, and the data is horizontally partitioned between multiple parties. The computations will preserve the privacy of each participant. Furthermore, we will use secret sharing instead of (homomorphic) encryption, which allows fast computations and requires little computational resources. To this end, we use a novel integer division to compute approximate real divisions. We also show how simple and private inferences can be performed using the learned SPN.
OneLog: Towards End-to-End Training in Software Log Anomaly Detection
Authors: Shayan Hashemi, Mika Mäntylä
Date: 2021-04-15 00:00:00
Tasks: Anomaly Detection

With the growth of online services, IoT devices, and DevOps-oriented software development, software log anomaly detection is becoming increasingly important. Prior works mainly follow a traditional four-staged architecture (Preprocessor, Parser, Vectorizer, and Classifier). This paper proposes OneLog, which utilizes a single Deep Neural Network (DNN) instead of multiple separate components. OneLog harnesses Convolutional Neural Networks (CNN) at the character level to take digits, numbers, and punctuations, which were removed in prior works, into account alongside the main natural language text. We evaluate our approach in six message- and sequence-based data sets: HDFS, Hadoop, BGL, Thunderbird, Spirit, and Liberty. We experiment with Onelog with single-, multi-, and cross-project setups. Onelog offers state-of-the-art performance in our datasets. Onelog can utilize multi-project datasets simultaneously during training, which suggests our model can generalize between datasets. Multi-project training also improves Onelog performance making it ideal when limited training data is available for an individual project. We also found that cross-project anomaly detection is possible with a single project pair (Liberty and Spirit). Analysis of model internals shows that one log has multiple modes of detecting anomalies and that the model learns manually validated parsing rules for the log messages. We conclude that character-based CNNs are a promising approach toward end-to-end learning in log anomaly detection. They offer good performance and generalization over multiple datasets. We will make our scripts publicly available upon the acceptance of this paper.
Speaker Attentive Speech Emotion Recognition
Authors: Clément Le Moine, Nicolas Obin, Axel Roebel
Date: 2021-04-15 00:00:00
Tasks: Emotion Recognition, Speech Emotion Recognition

Speech Emotion Recognition (SER) task has known significant improvements over the last years with the advent of Deep Neural Networks (DNNs). However, even the most successful methods are still rather failing when adaptation to specific speakers and scenarios is needed, inevitably leading to poorer performances when compared to humans. In this paper, we present novel work based on the idea of teaching the emotion recognition network about speaker identity. Our system is a combination of two ACRNN classifiers respectively dedicated to speaker and emotion recognition. The first informs the latter through a Self Speaker Attention (SSA) mechanism that is shown to considerably help to focus on emotional information of the speech signal. Experiments on social attitudes database Att-HACK and IEMOCAP corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.