pwc-archive/papers-with-abstracts huggingface dataset:
Information Theoretic Feature Transformation Learning for Brain Interfaces
Authors: Ozan Ozdenizci, Deniz Erdogmus
Date: 2019-03-28 00:00:00
Tasks: Brain Computer Interface, Dimensionality Reduction, EEG, Electroencephalogram (EEG), feature selection, Motor Imagery

Objective: A variety of pattern analysis techniques for model training in
brain interfaces exploit neural feature dimensionality reduction based on
feature ranking and selection heuristics. In the light of broad evidence
demonstrating the potential sub-optimality of ranking based feature selection
by any criterion, we propose to extend this focus with an information theoretic
learning driven feature transformation concept. Methods: We present a maximum
mutual information linear transformation (MMI-LinT), and a nonlinear
transformation (MMI-NonLinT) framework derived by a general definition of the
feature transformation learning problem. Empirical assessments are performed
based on electroencephalographic (EEG) data recorded during a four class motor
imagery brain-computer interface (BCI) task. Exploiting state-of-the-art
methods for initial feature vector construction, we compare the proposed
approaches with conventional feature selection based dimensionality reduction
techniques which are widely used in brain interfaces. Furthermore, for the
multi-class problem, we present and exploit a hierarchical graphical model
based BCI decoding system. Results: Both binary and multi-class decoding
analyses demonstrate significantly better performances with the proposed
methods. Conclusion: Information theoretic feature transformations are capable
of tackling potential confounders of conventional approaches in various
settings. Significance: We argue that this concept provides significant
insights to extend the focus on feature selection heuristics to a broader
definition of feature transformation learning in brain interfaces.
Adversarial Approximate Inference for Speech to Electroglottograph Conversion
Authors: Prathosh A. P., Varun Srivastava, Mayank Mishra
Date: 2019-03-28 00:00:00

Speech produced by human vocal apparatus conveys substantial non-semantic information including the gender of the speaker, voice quality, affective state, abnormalities in the vocal apparatus etc. Such information is attributed to the properties of the voice source signal, which is usually estimated from the speech signal. However, most of the source estimation techniques depend heavily on the goodness of the model assumptions and are prone to noise. A popular alternative is to indirectly obtain the source information through the Electroglottographic (EGG) signal that measures the electrical admittance around the vocal folds using dedicated hardware. In this paper, we address the problem of estimating the EGG signal directly from the speech signal, devoid of any hardware. Sampling from the intractable conditional distribution of the EGG signal given the speech signal is accomplished through optimization of an evidence lower bound. This is constructed via minimization of the KL-divergence between the true and the approximated posteriors of a latent variable learned using a deep neural auto-encoder that serves an informative prior. We demonstrate the efficacy of the method at generating the EGG signal by conducting several experiments on datasets comprising multiple speakers, voice qualities, noise settings and speech pathologies. The proposed method is evaluated on many benchmark metrics and is found to agree with the gold standard while proving better than the state-of-the-art algorithms on a few tasks such as epoch extraction.
Using Deep Learning Neural Networks and Candlestick Chart Representation to Predict Stock Market
Authors: Rosdyana Mangir Irawan Kusuma, Trang-Thi Ho, Wei-Chun Kao, Yu-Yen Ou, Kai-Lung Hua
Date: 2019-02-26 00:00:00
Tasks: Stock Market Prediction

Stock market prediction is still a challenging problem because there are many
factors effect to the stock market price such as company news and performance,
industry performance, investor sentiment, social media sentiment and economic
factors. This work explores the predictability in the stock market using Deep
Convolutional Network and candlestick charts. The outcome is utilized to design
a decision support framework that can be used by traders to provide suggested
indications of future stock price direction. We perform this work using various
types of neural networks like convolutional neural network, residual network
and visual geometry group network. From stock market historical data, we
converted it to candlestick charts. Finally, these candlestick charts will be
feed as input for training a Convolutional Neural Network model. This
Convolutional Neural Network model will help us to analyze the patterns inside
the candlestick chart and predict the future movements of stock market. The
effectiveness of our method is evaluated in stock market prediction with a
promising results 92.2% and 92.1% accuracy for Taiwan and Indonesian stock
market dataset respectively. The constructed model have been implemented as a
web-based system freely available at http://140.138.155.216/deepcandle/ for
predicting stock market using candlestick chart and deep learning neural
networks.
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Authors: Dan Hendrycks, Thomas Dietterich
Date: 2019-03-28 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Defense, Benchmarking, Domain Generalization

In this paper we establish rigorous benchmarks for image classifier
robustness. Our first benchmark, ImageNet-C, standardizes and expands the
corruption robustness topic, while showing which classifiers are preferable in
safety-critical applications. Then we propose a new dataset called ImageNet-P
which enables researchers to benchmark a classifier's robustness to common
perturbations. Unlike recent robustness research, this benchmark evaluates
performance on common corruptions and perturbations not worst-case adversarial
perturbations. We find that there are negligible changes in relative corruption
robustness from AlexNet classifiers to ResNet classifiers. Afterward we
discover ways to enhance corruption and perturbation robustness. We even find
that a bypassed adversarial defense provides substantial common perturbation
robustness. Together our benchmarks may aid future work toward networks that
robustly generalize.
Towards Standardization of Data Licenses: The Montreal Data License
Authors: Misha Benjamin, Paul Gagnon, Negar Rostamzadeh, Chris Pal, Yoshua Bengio, Alex Shee
Date: 2019-03-21 00:00:00

This paper provides a taxonomy for the licensing of data in the fields of
artificial intelligence and machine learning. The paper's goal is to build
towards a common framework for data licensing akin to the licensing of open
source software. Increased transparency and resolving conceptual ambiguities in
existing licensing language are two noted benefits of the approach proposed in
the paper. In parallel, such benefits may help foster fairer and more efficient
markets for data through bringing about clearer tools and concepts that better
define how data can be used in the fields of AI and ML. The paper's approach is
summarized in a new family of data license language - \textit{the Montreal Data
License (MDL)}. Alongside this new license, the authors and their collaborators
have developed a web-based tool to generate license language espousing the
taxonomies articulated in this paper.
Validation of a recommender system for prompting omitted foods in online dietary assessment surveys
Authors: Timur Osadchiy, Ivan Poliakov, Patrick Olivier, Maisie Rowland, Emma Foster
Date: 2019-03-20 00:00:00
Tasks: Recommendation Systems

Recall assistance methods are among the key aspects that improve the accuracy
of online dietary assessment surveys. These methods still mainly rely on
experience of trained interviewers with nutritional background, but data driven
approaches could improve cost-efficiency and scalability of automated dietary
assessment. We evaluated the effectiveness of a recommender algorithm developed
for an online dietary assessment system called Intake24, that automates the
multiple-pass 24-hour recall method. The recommender builds a model of eating
behavior from recalls collected in past surveys. Based on foods they have
already selected, the model is used to remind respondents of associated foods
that they may have omitted to report. The performance of prompts generated by
the model was compared to that of prompts hand-coded by nutritionists in two
dietary studies. The results of our studies demonstrate that the recommender
system is able to capture a higher number of foods omitted by respondents of
online dietary surveys than prompts hand-coded by nutritionists. However, the
considerably lower precision of generated prompts indicates an opportunity for
further improvement of the system.
Generative Adversarial Networks: recent developments
Authors: Maciej Zamorski, Adrian Zdobylak, Maciej Zięba, Jerzy Świątek
Date: 2019-03-16 00:00:00
Tasks: BIG-bench Machine Learning

In traditional generative modeling, good data representation is very often a
base for a good machine learning model. It can be linked to good
representations encoding more explanatory factors that are hidden in the
original data. With the invention of Generative Adversarial Networks (GANs), a
subclass of generative models that are able to learn representations in an
unsupervised and semi-supervised fashion, we are now able to adversarially
learn good mappings from a simple prior distribution to a target data
distribution. This paper presents an overview of recent developments in GANs
with a focus on learning latent space representations.
PyTorch-BigGraph: A Large-scale Graph Embedding System
Authors: Adam Lerer, Ledell Wu, Jiajun Shen, Timothee Lacroix, Luca Wehrstedt, Abhijit Bose, Alex Peysakhovich
Date: 2019-03-28 00:00:00
Tasks: Graph Embedding, graph partitioning, Link Prediction

Graph embedding methods produce unsupervised node features from graphs that
can then be used for a variety of machine learning tasks. Modern graphs,
particularly in industrial applications, contain billions of nodes and
trillions of edges, which exceeds the capability of existing embedding systems.
We present PyTorch-BigGraph (PBG), an embedding system that incorporates
several modifications to traditional multi-relation embedding systems that
allow it to scale to graphs with billions of nodes and trillions of edges. PBG
uses graph partitioning to train arbitrarily large embeddings on either a
single machine or in a distributed environment. We demonstrate comparable
performance with existing embedding systems on common benchmarks, while
allowing for scaling to arbitrarily large graphs and parallelization on
multiple machines. We train and evaluate embeddings on several large social
network graphs as well as the full Freebase dataset, which contains over 100
million nodes and 2 billion edges.
Improved Reinforcement Learning with Curriculum
Authors: Joseph West, Frederic Maire, Cameron Browne, Simon Denman
Date: 2019-03-29 00:00:00
Tasks: Board Games, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Humans tend to learn complex abstract concepts faster if examples are presented in a structured manner. For instance, when learning how to play a board game, usually one of the first concepts learned is how the game ends, i.e. the actions that lead to a terminal state (win, lose or draw). The advantage of learning end-games first is that once the actions which lead to a terminal state are understood, it becomes possible to incrementally learn the consequences of actions that are further away from a terminal state - we call this an end-game-first curriculum. Currently the state-of-the-art machine learning player for general board games, AlphaZero by Google DeepMind, does not employ a structured training curriculum; instead learning from the entire game at all times. By employing an end-game-first training curriculum to train an AlphaZero inspired player, we empirically show that the rate of learning of an artificial player can be improved during the early stages of training when compared to a player not using a training curriculum.
Learning Good Representation via Continuous Attention
Authors: Liang Zhao, Wei Xu
Date: 2019-03-29 00:00:00
Tasks: Object, Object Recognition, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), scientific discovery

In this paper we present our scientific discovery that good representation
can be learned via continuous attention during the interaction between
Unsupervised Learning(UL) and Reinforcement Learning(RL) modules driven by
intrinsic motivation. Specifically, we designed intrinsic rewards generated
from UL modules for driving the RL agent to focus on objects for a period of
time and to learn good representations of objects for later object recognition
task. We evaluate our proposed algorithm in both with and without extrinsic
reward settings. Experiments with end-to-end training in simulated environments
with applications to few-shot object recognition demonstrated the effectiveness
of the proposed algorithm.
The Challenge of Predicting Meal-to-meal Blood Glucose Concentrations for Patients with Type I Diabetes
Authors: Neil C. Borle, Edmond A. Ryan, Russell Greiner
Date: 2019-03-29 00:00:00

Patients with Type I Diabetes (T1D) must take insulin injections to prevent
the serious long term effects of hyperglycemia - high blood glucose (BG).
Patients must also be careful not to inject too much insulin because this could
induce hypoglycemia (low BG), which can potentially be fatal. Patients
therefore follow a "regimen" that determines how much insulin to inject at
certain times. Current methods for managing this disease require adjusting the
patient's regimen over time based on the disease's behavior (recorded in the
patient's diabetes diary). If we can accurately predict a patient's future BG
values from his/her current features (e.g., predicting today's lunch BG value
given today's diabetes diary entry for breakfast, including insulin injections,
and perhaps earlier entries), then it is relatively easy to produce an
effective regimen. This study explores the challenges of BG modeling by
applying several machine learning algorithms and various data preprocessing
variations (corresponding to 312 [learner, preprocessed-dataset] combinations),
to a new T1D dataset containing 29 601 entries from 47 different patients. Our
most accurate predictor is a weighted ensemble of two Gaussian Process
Regression models, which achieved an errL1 loss of 2.70 mmol/L (48.65 mg/dl).
This was an unexpectedly poor result given that one can obtain an errL1 of 2.91
mmol/L (52.43 mg/dl) using the naive approach of simply predicting the
patient's average BG. For each of data-variant/model combination we report
several evaluation metrics, including glucose-specific metrics, and find
similarly disappointing results (the best model was only incrementally better
than the simplest measure). These results suggest that the diabetes diary data
that is typically collected may not be sufficient to produce accurate BG
prediction models; additional data may be necessary to build accurate BG
prediction models.
Deep Representation with ReLU Neural Networks
Authors: Andreas Heinecke, Wen-Liang Hwang
Date: 2019-03-29 00:00:00

We consider deep feedforward neural networks with rectified linear units from
a signal processing perspective. In this view, such representations mark the
transition from using a single (data-driven) linear representation to utilizing
a large collection of affine linear representations tailored to particular
regions of the signal space. This paper provides a precise description of the
individual affine linear representations and corresponding domain regions that
the (data-driven) neural network associates to each signal of the input space.
In particular, we describe atomic decompositions of the representations and,
based on estimating their Lipschitz regularity, suggest some conditions that
can stabilize learning independent of the network depth. Such an analysis may
promote further theoretical insight from both the signal processing and machine
learning communities.
Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet
Authors: Mingyang Zhang, Xin Wang, Fuming Fang, Haizhou Li, Junichi Yamagishi
Date: 2019-03-29 00:00:00
Tasks: Decoder, Speech Synthesis, text-to-speech, Text to Speech, Voice Conversion

We investigated the training of a shared model for both text-to-speech (TTS)
and voice conversion (VC) tasks. We propose using an extended model
architecture of Tacotron, that is a multi-source sequence-to-sequence model
with a dual attention mechanism as the shared model for both the TTS and VC
tasks. This model can accomplish these two different tasks respectively
according to the type of input. An end-to-end speech synthesis task is
conducted when the model is given text as the input while a
sequence-to-sequence voice conversion task is conducted when it is given the
speech of a source speaker as the input. Waveform signals are generated by
using WaveNet, which is conditioned by using a predicted mel-spectrogram. We
propose jointly training a shared model as a decoder for a target speaker that
supports multiple sources. Listening experiments show that our proposed
multi-source encoder-decoder model can efficiently achieve both the TTS and VC
tasks.
Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform
Authors: Shinji Takaki, Hirokazu Kameoka, Junichi Yamagishi
Date: 2019-03-29 00:00:00

Recently, we proposed short-time Fourier transform (STFT)-based loss
functions for training a neural speech waveform model. In this paper, we
generalize the above framework and propose a training scheme for such models
based on spectral amplitude and phase losses obtained by either STFT or
continuous wavelet transform (CWT), or both of them. Since CWT is capable of
having time and frequency resolutions different from those of STFT and is cable
of considering those closer to human auditory scales, the proposed loss
functions could provide complementary information on speech signals.
Experimental results showed that it is possible to train a high-quality model
by using the proposed CWT spectral loss and is as good as one using STFT-based
loss.
Online Variance Reduction with Mixtures
Authors: Zalán Borsos, Sebastian Curi, Kfir. Y. Levy, Andreas Krause
Date: 2019-03-29 00:00:00
Tasks: Stochastic Optimization

Adaptive importance sampling for stochastic optimization is a promising
approach that offers improved convergence through variance reduction. In this
work, we propose a new framework for variance reduction that enables the use of
mixtures over predefined sampling distributions, which can naturally encode
prior knowledge about the data. While these sampling distributions are fixed,
the mixture weights are adapted during the optimization process. We propose
VRM, a novel and efficient adaptive scheme that asymptotically recovers the
best mixture weights in hindsight and can also accommodate sampling
distributions over sets of points. We empirically demonstrate the versatility
of VRM in a range of applications.
From Variational to Deterministic Autoencoders
Authors: Partha Ghosh, Mehdi S. M. Sajjadi, Antonio Vergari, Michael Black, Bernhard Schölkopf
Date: 2019-03-29 00:00:00
Proceeding: ICLR 2020 1
Tasks: Decoder, Density Estimation

Variational Autoencoders (VAEs) provide a theoretically-backed and popular framework for deep generative models. However, learning a VAE from data poses still unanswered theoretical questions and considerable practical challenges. In this work, we propose an alternative framework for generative modeling that is simpler, easier to train, and deterministic, yet has many of the advantages of VAEs. We observe that sampling a stochastic encoder in a Gaussian VAE can be interpreted as simply injecting noise into the input of a deterministic decoder. We investigate how substituting this kind of stochasticity, with other explicit and implicit regularization schemes, can lead to an equally smooth and meaningful latent space without forcing it to conform to an arbitrarily chosen prior. To retrieve a generative mechanism to sample new data, we introduce an ex-post density estimation step that can be readily applied also to existing VAEs, improving their sample quality. We show, in a rigorous empirical study, that the proposed regularized deterministic autoencoders are able to generate samples that are comparable to, or better than, those of VAEs and more powerful alternatives when applied to images as well as to structured data such as molecules. \footnote{An implementation is available at: \url{https://github.com/ParthaEth/Regularized_autoencoders-RAE-}}
Online Multi-target regression trees with stacked leaf models
Authors: Saulo Martiello Mastelini, Sylvio Barbon Jr., André Carlos Ponce de Leon Ferreira de Carvalho
Date: 2019-03-29 00:00:00
Tasks: Multi-target regression, Neural Network Compression, regression

One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, most of the existing solutions do not consider the occurrence of inter-target correlations when making predictions. In this work, we propose a novel online learning strategy for multi-target regression in data streams. The proposed strategy extends existing online decision tree learning algorithm to explore inter-target dependencies while making predictions. For such, the proposed strategy, called Stacked Single-target Hoeffding Tree (SST-HT), uses the inter-target dependencies as an additional information source to enhance predictive accuracy. Throughout an extensive experimental setup, we evaluate our proposal against state-of-the-art decision tree-based algorithms for online multi-target regression. According to the experimental results, SST-HT presents superior predictive accuracy, with a small increase in the processing time and memory requirements.
Cross-Subject Transfer Learning in Human Activity Recognition Systems using Generative Adversarial Networks
Authors: Elnaz Soleimania, Ehsan Nazerfard
Date: 2019-03-29 00:00:00
Tasks: Activity Recognition, Generative Adversarial Network, Human Activity Recognition, Subject Transfer, Transfer Learning

Application of intelligent systems especially in smart homes and
health-related topics has been drawing more attention in the last decades.
Training Human Activity Recognition (HAR) models -- as a major module --
requires a fair amount of labeled data. Despite training with large datasets,
most of the existing models will face a dramatic performance drop when they are
tested against unseen data from new users. Moreover, recording enough data for
each new user is unviable due to the limitations and challenges of working with
human users. Transfer learning techniques aim to transfer the knowledge which
has been learned from the source domain (subject) to the target domain in order
to decrease the models' performance loss in the target domain. This paper
presents a novel method of adversarial knowledge transfer named SA-GAN stands
for Subject Adaptor GAN which utilizes Generative Adversarial Network framework
to perform cross-subject transfer learning in the domain of wearable
sensor-based Human Activity Recognition. SA-GAN outperformed other
state-of-the-art methods in more than 66% of experiments and showed the second
best performance in the remaining 25% of experiments. In some cases, it reached
up to 90% of the accuracy which can be obtained by supervised training over the
same domain data.