pwc-archive/papers-with-abstracts huggingface dataset:
Deepsleep: Fast and Accurate Delineation of Sleep Arousals at Millisecond Resolution by Deep Learning
Authors: Hongyang Li, Yuanfang Guan
Date: 2019-09-07 00:00:00
Proceeding: Lancet 2019 9
Tasks: Sleep Arousal Detection, Sleep Micro-event detection, Sleep Quality

Background: Sleep arousals are transient periods of wakefulness punctuated into sleep. Excessive sleep arousals are associated with many negative effects including daytime sleepiness and sleep disorders. High-quality annotation of polysomnographic recordings is crucial for the diagnosis of sleep arousal disorders. Currently, sleep arousals are mainly annotated by human experts through looking at millions of data points manually, which requires considerable time and effort.

Methods: We used the polysomnograms of 2,994 individuals from two independent datasets (i) PhysioNet Challenge dataset (n=994), and (ii) Sleep Heart Health Study dataset (n=2000) for model training (60%), validation (15%), and testing (25%). We developed a deep convolutional neural network approach, DeepSleep, to automatically segment sleep arousal events. Our method captured the long-range and short-range interactions among physiological signals at multiple time scales to empower the detection of sleep arousals. A novel augmentation strategy by randomly swapping similar physiological channels was further applied to improve the prediction accuracy.

Findings: Compared with other computational methods in sleep study, DeepSleep features accurate (area under receiver operating characteristic curve of 0.93 and area under the precision recall curve of 0.55), high-resolution (5-millisecond resolution), and fast (10 seconds per sleep record) delineation of sleep arousals. This method ranked first in segmenting non-apenic arousals when evaluated on a large held-out dataset (n=989) in the 2018 PhysioNet Challenge. We found that DeepSleep provided more detailed delineations than humans, especially at the low-confident boundary regions between arousal and non-arousal events. This indicates that in silico annotations is a complement to human annotations and potentially advances the current binary label system and scoring criteria for sleep arousals.

Interpretation: The proposed deep learning model achieved state-of-the-art performance in detection of sleep arousals. By introducing the probability of annotation confidence, this model would provide more accurate information for the diagnosis of sleep disorders and the evaluation of sleep quality.
Decoding P300 Variability using Convolutional Neural Networks
Authors: Amelia J. Solon, Vernon J. Lawhern, Jonathan Touryan, Jonathan R. McDaniel, Anthony J. Ries, Stephen M. Gordon
Date: 2019-06-14 00:00:00
Proceeding: Frontiers in Human Neuroscience 2019 6
Tasks: EEG, Eeg Decoding, Electroencephalogram (EEG), speech-recognition, Speech Recognition

Deep convolutional neural networks (CNN) have previously been shown to be useful tools for signal decoding and analysis in a variety of complex domains, such as image processing and speech recognition. By learning from large amounts of data, the representations encoded by these deep networks are often invariant to moderate changes in the underlying feature spaces. Recently, we proposed a CNN architecture that could be applied to electroencephalogram (EEG) decoding and analysis. In this article, we train our CNN model using data from prior experiments in order to later decode the P300 evoked response from an unseen, hold-out experiment. We analyze the CNN output as a function of the underlying variability in the P300 response and demonstrate that the CNN output is sensitive to the experiment-induced changes in the neural response. We then assess the utility of our approach as a means of improving the overall signal-to-noise ratio in the EEG record. Finally, we show an example of how CNN-based decoding can be applied to the analysis of complex data.
Amplifying the Neural Power Spectrum
Authors: J. Andrew Doyle, Paule-Joanne Toussaint, Alan C. Evans
Date: 2019-06-04 00:00:00
Proceeding: bioRxiv 2019 6
Tasks: EEG, Electroencephalogram (EEG)

We introduce a novel method that employs a parametric model of human electroen-cephalographic (EEG) brain signal power spectra to evaluate cognitive science experiments and test scientific hypotheses. We develop the Neural Power Amplifier (NPA), a data-driven approach to EEG pre-processing that can replace current filtering strategies with a principled method based on combining filters with log-arithmic and Gaussian magnitude responses. Presenting the first time domain evidence to validate an increasingly popular model for neural power spectra, we show that filtering out the 1/f background signal and selecting peaks improves a time-domain decoding experiment for visual stimulus of human faces versus random noise.
DeepEMGNet: An Application for Efficient Discrimination of ALS and Normal EMG Signals
Authors: Abdulkadir Sengur, Mehmet Gedikpinar, Yaman Akbulut, Erkan Deniz, Varun Bajaj, Yanhui Guo
Date: 2018-01-01 00:00:00
Proceeding: MECHATRONICS 2017. Advances in Intelligent Systems and Computing 2018 1
Tasks: ALS Detection, Electromyography (EMG), General Classification

This paper proposes a deep learning application for efficient classification of amyotrophic lateral sclerosis (ALS) and normal Electromyogram (EMG) signals. EMG signals are helpful in analyzing of the neuromuscular diseases like ALS. ALS is a well-known brain disease, which progressively degenerates the motor neurons. Most of the previous works about EMG signal classification covers a dozen of basic signal processing methodologies such as statistical signal processing, wavelet analysis, and empirical mode decomposition (EMD). In this work, a different application is implemented which is based on time-frequency (TF) representation of EMG signals and convolutional neural networks (CNN). Short Time Fourier Transform (STFT) is considered for TF representation. Two convolution layers, two pooling layer, a fully connected layer and a lost function layer is considered in CNN architecture. The efficiency of the proposed implementation is tested on publicly available EMG dataset. The dataset contains 89 ALS and 133 normal EMG signals with 24 kHz sampling frequency. Experimental results show 96.69% accuracy. The obtained results are also compared with other methods, which show the superiority of the proposed method.
Deep Learning for Musculoskeletal Force Prediction
Authors: Lance Rane, Ziyun Ding, Alison H. McGregor, Anthony M. J. Bull
Date: 2018-12-13 00:00:00
Proceeding: Annals of Biomedical Engineering 2018 12
Tasks: Deep Learning, Electromyography (EMG), EMG Signal Prediction, Medial knee JRF Prediction, Muscle Force Prediction, Prediction

Musculoskeletal models permit the determination of internal forces acting during dynamic movement, which is clinically useful, but traditional methods may suffer from slowness and a need for extensive input data. Recently, there has been interest in the use of supervised learning to build approximate models for computationally demanding processes, with benefits in speed and flexibility. Here, we use a deep neural network to learn the mapping from movement space to muscle space. Trained on a set of kinematic, kinetic and electromyographic measurements from 156 subjects during gait, the network’s predictions of internal force magnitudes show good concordance with those derived by musculoskeletal modelling. In a separate set of experiments, training on data from the most widely known benchmarks of modelling performance, the international Grand Challenge competitions, generates predictions that better those of the winning submissions in four of the six competitions. Computational speedup facilitates incorporation into a lab-based system permitting real-time estimation of forces, and interrogation of the trained neural networks provides novel insights into population-level relationships between kinematic and kinetic factors.
Intracerebral EEG Artifact Identification Using Convolutional Neural Networks
Authors: Petr Nejedly, Jan Cimbalnik, Petr Klimes, Filip Plesinger, Josef Halamek, Vaclav Kremen, Ivo Viscor, Benjamin H. Brinkmann, Martin Pail, Milan Brazdil, Gregory Worrell, Pavel Jurak
Date: 2018-08-13 00:00:00
Proceeding: Neuroinformatics 2018 8
Tasks: Artifact Detection, EEG, EEG Artifact Removal, Electroencephalogram (EEG), Transfer Learning

Manual and semi-automatic identification of artifacts and unwanted physiological signals in large intracerebral electroencephalographic (iEEG) recordings is time consuming and inaccurate. To date, unsupervised methods to accurately detect iEEG artifacts are not available. This study introduces a novel machine-learning approach for detection of artifacts in iEEG signals in clinically controlled conditions using convolutional neural networks (CNN) and benchmarks the method’s performance against expert annotations. The method was trained and tested on data obtained from St Anne’s University Hospital (Brno, Czech Republic) and validated on data from Mayo Clinic (Rochester, Minnesota, U.S.A). We show that the proposed technique can be used as a generalized model for iEEG artifact detection. Moreover, a transfer learning process might be used for retraining of the generalized version to form a data-specific model. The generalized model can be efficiently retrained for use with different EEG acquisition systems and noise environments. The generalized and specialized model F1 scores on the testing dataset were 0.81 and 0.96, respectively. The CNN model provides faster, more objective, and more reproducible iEEG artifact detection compared to manual approaches.
Properties of Laplacian Pyramids for Extension and Denoising
Authors: William Leeb
Date: 2019-09-17 00:00:00
Tasks: Denoising

We analyze the Laplacian pyramids algorithm of Rabin and Coifman for extending and denoising a function sampled on a discrete set of points. We provide mild conditions under which the algorithm converges, and prove stability bounds on the extended function. We also consider the iterative application of truncated Laplacian pyramids kernels for denoising signals by non-local means.
Relaxed Softmax for learning from Positive and Unlabeled data
Authors: Ugo Tanielian, Flavian vasile
Date: 2019-09-17 00:00:00
Tasks: Density Estimation, Language Modeling, Language Modelling

In recent years, the softmax model and its fast approximations have become the de-facto loss functions for deep neural networks when dealing with multi-class prediction. This loss has been extended to language modeling and recommendation, two fields that fall into the framework of learning from Positive and Unlabeled data. In this paper, we stress the different drawbacks of the current family of softmax losses and sampling schemes when applied in a Positive and Unlabeled learning setup. We propose both a Relaxed Softmax loss (RS) and a new negative sampling scheme based on Boltzmann formulation. We show that the new training objective is better suited for the tasks of density estimation, item similarity and next-event prediction by driving uplifts in performance on textual and recommendation datasets against classical softmax.
A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Authors: Mingzhe Chen, Zhaohui Yang, Walid Saad, Changchuan Yin, H. Vincent Poor, Shuguang Cui
Date: 2019-09-17 00:00:00
Tasks: Federated Learning

In this paper, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In particular, in the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station (BS) that will generate a global FL model and send it back to the users. Since all training parameters are transmitted over wireless links, the quality of the training will be affected by wireless factors such as packet errors and the availability of wireless resources. Meanwhile, due to the limited wireless bandwidth, the BS must select an appropriate subset of users to execute the FL algorithm so as to build a global FL model accurately. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize an FL loss function that captures the performance of the FL algorithm. To address this problem, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of wireless factors on FL. Then, based on the expected convergence rate of the FL algorithm, the optimal transmit power for each user is derived, under a given user selection and uplink resource block (RB) allocation scheme. Finally, the user selection and uplink RB allocation is optimized so as to minimize the FL loss function. Simulation results show that the proposed joint federated learning and communication framework can reduce the FL loss function value by up to 10% and 16%, respectively, compared to: 1) An optimal user selection algorithm with random resource allocation and 2) a standard FL algorithm with random user selection and resource allocation.
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review
Authors: Han Xu, Yao Ma, Haochen Liu, Debayan Deb, Hui Liu, Jiliang Tang, Anil K. Jain
Date: 2019-09-17 00:00:00
Tasks: Adversarial Attack

Deep neural networks (DNN) have achieved unprecedented success in numerous machine learning tasks in various domains. However, the existence of adversarial examples has raised concerns about applying deep learning to safety-critical applications. As a result, we have witnessed increasing interests in studying attack and defense mechanisms for DNN models on different data types, such as images, graphs and text. Thus, it is necessary to provide a systematic and comprehensive overview of the main threats of attacks and the success of corresponding countermeasures. In this survey, we review the state of the art algorithms for generating adversarial examples and the countermeasures against adversarial examples, for the three popular data types, i.e., images, graphs and text.
MER-SDN: Machine Learning Framework for Traffic Aware Energy Efficient Routing in SDN
Authors: Beakal Gizachew Assefa, Oznur Ozkasap
Date: 2019-08-27 00:00:00
Tasks: BIG-bench Machine Learning, Management, Parameter Prediction, Reinforcement Learning, Traffic Classification

Software Defined Networking (SDN) achieves programmability of a network through separation of the control and data planes. It enables flexibility in network management and control. Energy efficiency is one of the challenging global problems which has both economic and environmental impact. A massive amount of information is generated in the controller of an SDN based network. Machine learning gives the ability to computers to progressively learn from data without having to write specific instructions. In this work, we propose MER-SDN: a machine learning framework for traffic-aware energy efficient routing in SDN. Feature extraction, training, and testing are the three main stages of the learning machine. Experiments are conducted on Mininet and POX controller using real-world network topology and dynamic traffic traces from SNDlib. Results show that our approach achieves more than 65\% feature size reduction, more than 70% accuracy in parameter prediction of an energy efficient heuristics algorithm, also our prediction refine heuristics converges the predicted value to the optimal parameters values with up to 25X speedup as compared to the brute force method.
A Distributed Fair Machine Learning Framework with Private Demographic Data Protection
Authors: Hui Hu, Yijun Liu, Zhen Wang, Chao Lan
Date: 2019-09-17 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

Fair machine learning has become a significant research topic with broad societal impact. However, most fair learning methods require direct access to personal demographic data, which is increasingly restricted to use for protecting user privacy (e.g. by the EU General Data Protection Regulation). In this paper, we propose a distributed fair learning framework for protecting the privacy of demographic data. We assume this data is privately held by a third party, which can communicate with the data center (responsible for model development) without revealing the demographic information. We propose a principled approach to design fair learning methods under this framework, exemplify four methods and show they consistently outperform their existing counterparts in both fairness and accuracy across three real-world data sets. We theoretically analyze the framework, and prove it can learn models with high fairness or high accuracy, with their trade-offs balanced by a threshold variable.
Q-Learning Based Aerial Base Station Placement for Fairness Enhancement in Mobile Networks
Authors: Rozhina Ghanavi, Maryam Sabbaghian, Halim Yanikomeroglu
Date: 2019-09-10 00:00:00
Tasks: Fairness, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we use an aerial base station (aerial-BS) to enhance fairness in a dynamic environment with user mobility. The problem of optimally placing the aerial-BS is a non-deterministic polynomial-time hard (NP-hard) problem. Moreover, the network topology is subject to continuous changes due to the user mobility. These issues intensify the quest to develop an adaptive and fast algorithm for 3D placement of the aerial-BS. To this end, we propose a method based on reinforcement learning to achieve these goals. Simulation results show that our method increases fairness among users in a reasonable computing time, while the solution is comparatively close to the optimal solution obtained by exhaustive search.
The Explanation Game: Explaining Machine Learning Models Using Shapley Values
Authors: Luke Merrick, Ankur Taly
Date: 2019-09-17 00:00:00
Tasks: BIG-bench Machine Learning

A number of techniques have been proposed to explain a machine learning model's prediction by attributing it to the corresponding input features. Popular among these are techniques that apply the Shapley value method from cooperative game theory. While existing papers focus on the axiomatic motivation of Shapley values, and efficient techniques for computing them, they offer little justification for the game formulations used, and do not address the uncertainty implicit in their methods' outputs. For instance, the popular SHAP algorithm's formulation may give substantial attributions to features that play no role in the model. In this work, we illustrate how subtle differences in the underlying game formulations of existing methods can cause large differences in the attributions for a prediction. We then present a general game formulation that unifies existing methods, and enables straightforward confidence intervals on their attributions. Furthermore, it allows us to interpret the attributions as contrastive explanations of an input relative to a distribution of reference inputs. We tie this idea to classic research in cognitive psychology on contrastive explanations, and propose a conceptual framework for generating and interpreting explanations for ML models, called formulate, approximate, explain (FAE). We apply this framework to explain black-box models trained on two UCI datasets and a Lending Club dataset.
Dynamics of Deep Neural Networks and Neural Tangent Hierarchy
Authors: Jiaoyang Huang, Horng-Tzer Yau
Date: 2019-09-18 00:00:00
Proceeding: ICML 2020 1
Tasks: regression

The evolution of a deep neural network trained by the gradient descent can be described by its neural tangent kernel (NTK) as introduced in [20], where it was proven that in the infinite width limit the NTK converges to an explicit limiting kernel and it stays constant during training. The NTK was also implicit in some other recent papers [6,13,14]. In the overparametrization regime, a fully-trained deep neural network is indeed equivalent to the kernel regression predictor using the limiting NTK. And the gradient descent achieves zero training loss for a deep overparameterized neural network. However, it was observed in [5] that there is a performance gap between the kernel regression using the limiting NTK and the deep neural networks. This performance gap is likely to originate from the change of the NTK along training due to the finite width effect. The change of the NTK along the training is central to describe the generalization features of deep neural networks. In the current paper, we study the dynamic of the NTK for finite width deep fully-connected neural networks. We derive an infinite hierarchy of ordinary differential equations, the neural tangent hierarchy (NTH) which captures the gradient descent dynamic of the deep neural network. Moreover, under certain conditions on the neural network width and the data set dimension, we prove that the truncated hierarchy of NTH approximates the dynamic of the NTK up to arbitrary precision. This description makes it possible to directly study the change of the NTK for deep neural networks, and sheds light on the observation that deep neural networks outperform kernel regressions using the corresponding limiting NTK.
Decision-Directed Data Decomposition
Authors: Brent D. Davis, Ethan Jackson, Daniel J. Lizotte
Date: 2019-09-18 00:00:00
Tasks: Word Embeddings

We present an algorithm, Decision-Directed Data Decomposition (D4), which decomposes a dataset into two components. The first contains most of the useful information for a specified supervised learning task. The second orthogonal component contains little information about the task but retains associations and information that were not targeted. The algorithm is simple and scalable. We illustrate its application in image and text processing domains. Our results show that 1) post-hoc application of D4 to an image representation space can remove information about specified concepts without impacting other concepts, 2) D4 is able to improve predictive generalization in certain settings, and 3) applying D4 to word embedding representations produces state-of-the-art results in debiasing.
Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis
Authors: Minlong Peng, Qi Zhang, Xuanjing Huang
Date: 2019-09-18 00:00:00
Proceeding: COLING 2020 8
Tasks: Domain Adaptation, Representation Learning, Sentiment Analysis

Cross-domain sentiment analysis is currently a hot topic in the research and engineering areas. One of the most popular frameworks in this field is the domain-invariant representation learning (DIRL) paradigm, which aims to learn a distribution-invariant feature representation across domains. However, in this work, we find out that applying DIRL may harm domain adaptation when the label distribution $\rm{P}(\rm{Y})$ changes across domains. To address this problem, we propose a modification to DIRL, obtaining a novel weighted domain-invariant representation learning (WDIRL) framework. We show that it is easy to transfer existing SOTA DIRL models to WDIRL. Empirical studies on extensive cross-domain sentiment analysis tasks verified our statements and showed the effectiveness of our proposed solution.