pwc-archive/papers-with-abstracts huggingface dataset:
Emotion Recognition from Speech based on Relevant Feature and Majority Voting
Authors: Md. Kamruzzaman Sarker, Kazi Md. Rokibul Alam, Md. Arifuzzaman
Date: 2018-07-11 00:00:00
Tasks: BIG-bench Machine Learning, Emotion Recognition, feature selection

This paper proposes an approach to detect emotion from human speech employing
majority voting technique over several machine learning techniques. The
contribution of this work is in two folds: firstly it selects those features of
speech which is most promising for classification and secondly it uses the
majority voting technique that selects the exact class of emotion. Here,
majority voting technique has been applied over Neural Network (NN), Decision
Tree (DT), Support Vector Machine (SVM) and K-Nearest Neighbor (KNN). Input
vector of NN, DT, SVM and KNN consists of various acoustic and prosodic
features like Pitch, Mel-Frequency Cepstral coefficients etc. From speech
signal many feature have been extracted and only promising features have been
selected. To consider a feature as promising, Fast Correlation based feature
selection (FCBF) and Fisher score algorithms have been used and only those
features are selected which are highly ranked by both of them. The proposed
approach has been tested on Berlin dataset of emotional speech [3] and
Electromagnetic Articulography (EMA) dataset [4]. The experimental result shows
that majority voting technique attains better accuracy over individual machine
learning techniques. The employment of the proposed approach can effectively
recognize the emotion of human beings in case of social robot, intelligent chat
client, call-center of a company etc.
The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization
Authors: Constantinos Daskalakis, Ioannis Panageas
Date: 2018-07-11 00:00:00
Proceeding: NeurIPS 2018 12

Motivated by applications in Optimization, Game Theory, and the training of
Generative Adversarial Networks, the convergence properties of first order
methods in min-max problems have received extensive study. It has been
recognized that they may cycle, and there is no good understanding of their
limit points when they do not. When they converge, do they converge to local
min-max solutions? We characterize the limit points of two basic first order
methods, namely Gradient Descent/Ascent (GDA) and Optimistic Gradient Descent
Ascent (OGDA). We show that both dynamics avoid unstable critical points for
almost all initializations. Moreover, for small step sizes and under mild
assumptions, the set of \{OGDA\}-stable critical points is a superset of
\{GDA\}-stable critical points, which is a superset of local min-max solutions
(strict in some cases). The connecting thread is that the behavior of these
dynamics can be studied from a dynamical systems perspective.
Deep Imbalanced Attribute Classification using Visual Attention Aggregation
Authors: Nikolaos Sarafianos, Xiang Xu, Ioannis A. Kakadiaris
Date: 2018-07-10 00:00:00
Proceeding: ECCV 2018 9
Tasks: Attribute, Classification, General Classification, Image Description

For many computer vision applications, such as image description and human
identification, recognizing the visual attributes of humans is an essential yet
challenging problem. Its challenges originate from its multi-label nature, the
large underlying class imbalance and the lack of spatial annotations. Existing
methods follow either a computer vision approach while failing to account for
class imbalance, or explore machine learning solutions, which disregard the
spatial and semantic relations that exist in the images. With that in mind, we
propose an effective method that extracts and aggregates visual attention masks
at different scales. We introduce a loss function to handle class imbalance
both at class and at an instance level and further demonstrate that penalizing
attention masks with high prediction variance accounts for the weak supervision
of the attention mechanism. By identifying and addressing these challenges, we
achieve state-of-the-art results with a simple attention mechanism in both PETA
and WIDER-Attribute datasets without additional context or side information.
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Authors: Kimin Lee, Kibok Lee, Honglak Lee, Jinwoo Shin
Date: 2018-07-10 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: class-incremental learning, Class Incremental Learning, Incremental Learning, Out-of-Distribution Detection

Detecting test samples drawn sufficiently far away from the training
distribution statistically or adversarially is a fundamental requirement for
deploying a good classifier in many real-world machine learning applications.
However, deep neural networks with the softmax classifier are known to produce
highly overconfident posterior distributions even for such abnormal samples. In
this paper, we propose a simple yet effective method for detecting any abnormal
samples, which is applicable to any pre-trained softmax neural classifier. We
obtain the class conditional Gaussian distributions with respect to (low- and
upper-level) features of the deep models under Gaussian discriminant analysis,
which result in a confidence score based on the Mahalanobis distance. While
most prior methods have been evaluated for detecting either out-of-distribution
or adversarial samples, but not both, the proposed method achieves the
state-of-the-art performances for both cases in our experiments. Moreover, we
found that our proposed method is more robust in harsh cases, e.g., when the
training dataset has noisy labels or small number of samples. Finally, we show
that the proposed method enjoys broader usage by applying it to
class-incremental learning: whenever out-of-distribution samples are detected,
our classification rule can incorporate new classes well without further
training deep models.
Vision System for AGI: Problems and Directions
Authors: Alexey Potapov, Sergey Rodionov, Maxim Peterson, Oleg Shcherbakov, Innokentii Zhdanov, Nikolai Skorobogatko
Date: 2018-07-10 00:00:00

What frameworks and architectures are necessary to create a vision system for
AGI? In this paper, we propose a formal model that states the task of
perception within AGI. We show the role of discriminative and generative models
in achieving efficient and general solution of this task, thus specifying the
task in more detail. We discuss some existing generative and discriminative
models and demonstrate their insufficiency for our purposes. Finally, we
discuss some architectural dilemmas and open questions.
DeepDiff: Deep-learning for predicting Differential gene expression from histone modifications
Authors: Arshdeep Sekhon, Ritambhara Singh, Yanjun Qi
Date: 2018-07-10 00:00:00
Tasks: Prediction

Computational methods that predict differential gene expression from histone
modification signals are highly desirable for understanding how histone
modifications control the functional heterogeneity of cells through influencing
differential gene regulation. Recent studies either failed to capture
combinatorial effects on differential prediction or primarily only focused on
cell type-specific analysis. In this paper, we develop a novel attention-based
deep learning architecture, DeepDiff, that provides a unified and end-to-end
solution to model and to interpret how dependencies among histone modifications
control the differential patterns of gene regulation. DeepDiff uses a hierarchy
of multiple Long short-term memory (LSTM) modules to encode the spatial
structure of input signals and to model how various histone modifications
cooperate automatically. We introduce and train two levels of attention jointly
with the target prediction, enabling DeepDiff to attend differentially to
relevant modifications and to locate important genome positions for each
modification. Additionally, DeepDiff introduces a novel deep-learning based
multi-task formulation to use the cell-type-specific gene expression
predictions as auxiliary tasks, encouraging richer feature embeddings in our
primary task of differential expression prediction. Using data from Roadmap
Epigenomics Project (REMC) for ten different pairs of cell types, we show that
DeepDiff significantly outperforms the state-of-the-art baselines for
differential gene expression prediction. The learned attention weights are
validated by observations from previous studies about how epigenetic mechanisms
connect to differential gene expression. Codes and results are available at
\url{deepchrome.org}
Deep Structured Generative Models
Authors: Kun Xu, Haoyu Liang, Jun Zhu, Hang Su, Bo Zhang
Date: 2018-07-10 00:00:00

Deep generative models have shown promising results in generating realistic
images, but it is still non-trivial to generate images with complicated
structures. The main reason is that most of the current generative models fail
to explore the structures in the images including spatial layout and semantic
relations between objects. To address this issue, we propose a novel deep
structured generative model which boosts generative adversarial networks (GANs)
with the aid of structure information. In particular, the layout or structure
of the scene is encoded by a stochastic and-or graph (sAOG), in which the
terminal nodes represent single objects and edges represent relations between
objects. With the sAOG appropriately harnessed, our model can successfully
capture the intrinsic structure in the scenes and generate images of
complicated scenes accordingly. Furthermore, a detection network is introduced
to infer scene structures from a image. Experimental results demonstrate the
effectiveness of our proposed method on both modeling the intrinsic structures,
and generating realistic images.
Deep learning for comprehensive forecasting of Alzheimer's Disease progression
Authors: Charles K. Fisher, Aaron M. Smith, Jonathan R. Walsh, the Coalition Against Major Diseases
Date: 2018-07-10 00:00:00
Tasks: Deep Learning

Most approaches to machine learning from electronic health data can only
predict a single endpoint. Here, we present an alternative that uses
unsupervised deep learning to simulate detailed patient trajectories. We use
data comprising 18-month trajectories of 44 clinical variables from 1908
patients with Mild Cognitive Impairment or Alzheimer's Disease to train a model
for personalized forecasting of disease progression. We simulate synthetic
patient data including the evolution of each sub-component of cognitive exams,
laboratory tests, and their associations with baseline clinical
characteristics, generating both predictions and their confidence intervals.
Our unsupervised model predicts changes in total ADAS-Cog scores with the same
accuracy as specifically trained supervised models and identifies
sub-components associated with word recall as predictive of progression. The
ability to simultaneously simulate dozens of patient characteristics is a
crucial step towards personalized medicine for Alzheimer's Disease.
Automatic Gradient Boosting
Authors: Janek Thomas, Stefan Coors, Bernd Bischl
Date: 2018-07-10 00:00:00
Tasks: AutoML, BIG-bench Machine Learning, Hyperparameter Optimization, Model Selection

Automatic machine learning performs predictive modeling with high performing
machine learning tools without human interference. This is achieved by making
machine learning applications parameter-free, i.e. only a dataset is provided
while the complete model selection and model building process is handled
internally through (often meta) optimization. Projects like Auto-WEKA and
auto-sklearn aim to solve the Combined Algorithm Selection and Hyperparameter
optimization (CASH) problem resulting in huge configuration spaces. However,
for most real-world applications, the optimization over only a few different
key learning algorithms can not only be sufficient, but also potentially
beneficial. The latter becomes apparent when one considers that models have to
be validated, explained, deployed and maintained. Here, less complex model are
often preferred, for validation or efficiency reasons, or even a strict
requirement. Automatic gradient boosting simplifies this idea one step further,
using only gradient boosting as a single learning algorithm in combination with
model-based hyperparameter tuning, threshold optimization and encoding of
categorical features. We introduce this general framework as well as a concrete
implementation called autoxgboost. It is compared to current AutoML projects on
16 datasets and despite its simplicity is able to achieve comparable results on
about half of the datasets as well as performing best on two.
"Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention
Authors: Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, Jiebo Luo
Date: 2018-07-10 00:00:00
Tasks: Image Captioning

Generating stylized captions for an image is an emerging topic in image
captioning. Given an image as input, it requires the system to generate a
caption that has a specific style (e.g., humorous, romantic, positive, and
negative) while describing the image content semantically accurately. In this
paper, we propose a novel stylized image captioning model that effectively
takes both requirements into consideration. To this end, we first devise a new
variant of LSTM, named style-factual LSTM, as the building block of our model.
It uses two groups of matrices to capture the factual and stylized knowledge,
respectively, and automatically learns the word-level weights of the two groups
based on previous context. In addition, when we train the model to capture
stylized elements, we propose an adaptive learning approach based on a
reference factual model, it provides factual knowledge to the model as the
model learns from stylized caption labels, and can adaptively compute how much
information to supply at each time step. We evaluate our model on two stylized
image captioning datasets, which contain humorous/romantic captions and
positive/negative captions, respectively. Experiments shows that our proposed
model outperforms the state-of-the-art approaches, without using extra ground
truth supervision.
Learning Implicit Generative Models by Teaching Explicit Ones
Authors: Chao Du, Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang
Date: 2018-07-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: Bilevel Optimization, Rolling Shutter Correction

Implicit generative models are difficult to train as no explicit density functions are defined. Generative adversarial nets (GANs) present a minimax framework to train such models, which however can suffer from mode collapse due to the nature of the JS-divergence. This paper presents a learning by teaching (LBT) approach to learning implicit models, which intrinsically avoids the mode collapse problem by optimizing a KL-divergence rather than the JS-divergence in GANs. In LBT, an auxiliary density estimator is introduced to fit the implicit model's distribution while the implicit model teaches the density estimator to match the data distribution. LBT is formulated as a bilevel optimization problem, whose optimal generator matches the true data distribution. LBT can be naturally integrated with GANs to derive a hybrid LBT-GAN that enjoys complimentary benefits. Finally, we present a stochastic gradient ascent algorithm with unrolling to solve the challenging learning problems. Experimental results demonstrate the effectiveness of our method.
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
Authors: Yuping Luo, Huazhe Xu, Yuanzhi Li, Yuandong Tian, Trevor Darrell, Tengyu Ma
Date: 2018-07-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: continuous-control, Continuous Control, Deep Reinforcement Learning, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Uncertainty Quantification

Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding of such methods has been rather limited. This paper introduces a novel algorithmic framework for designing and analyzing model-based RL algorithms with theoretical guarantees. We design a meta-algorithm with a theoretical guarantee of monotone improvement to a local maximum of the expected reward. The meta-algorithm iteratively builds a lower bound of the expected reward based on the estimated dynamical model and sample trajectories, and then maximizes the lower bound jointly over the policy and the model. The framework extends the optimism-in-face-of-uncertainty principle to non-linear dynamical models in a way that requires \textit{no explicit} uncertainty quantification. Instantiating our framework with simplification gives a variant of model-based RL algorithms Stochastic Lower Bounds Optimization (SLBO). Experiments demonstrate that SLBO achieves state-of-the-art performance when only one million or fewer samples are permitted on a range of continuous control benchmark tasks.
Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition
Authors: Chun-Fu Chen, Quanfu Fan, Neil Mallinar, Tom Sercu, Rogerio Feris
Date: 2018-07-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: Object, Object Recognition, speech-recognition, Speech Recognition

In this paper, we propose a novel Convolutional Neural Network (CNN) architecture for learning multi-scale feature representations with good tradeoffs between speed and accuracy. This is achieved by using a multi-branch network, which has different computational complexity at different branches. Through frequent merging of features from branches at distinct scales, our model obtains multi-scale features while using less computation. The proposed approach demonstrates improvement of model efficiency and performance on both object recognition and speech recognition tasks,using popular architectures including ResNet and ResNeXt. For object recognition, our approach reduces computation by 33% on object recognition while improving accuracy with 0.9%. Furthermore, our model surpasses state-of-the-art CNN acceleration approaches by a large margin in accuracy and FLOPs reduction. On the task of speech recognition, our proposed multi-scale CNNs save 30% FLOPs with slightly better word error rates, showing good generalization across domains. The codes are available at https://github.com/IBM/BigLittleNet
Model-based free-breathing cardiac MRI reconstruction using deep learned \& STORM priors: MoDL-STORM
Authors: Sampurna Biswas, Hemant K. Aggarwal, Sunrita Poddar, Mathews Jacob
Date: 2018-07-10 00:00:00
Tasks: Deep Learning, MRI Reconstruction

We introduce a model-based reconstruction framework with deep learned (DL)
and smoothness regularization on manifolds (STORM) priors to recover free
breathing and ungated (FBU) cardiac MRI from highly undersampled measurements.
The DL priors enable us to exploit the local correlations, while the STORM
prior enables us to make use of the extensive non-local similarities that are
subject dependent. We introduce a novel model-based formulation that allows the
seamless integration of deep learning methods with available prior information,
which current deep learning algorithms are not capable of. The experimental
results demonstrate the preliminary potential of this work in accelerating FBU
cardiac MRI.
Universal Transformers
Authors: Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, Łukasz Kaiser
Date: 2018-07-10 00:00:00
Proceeding: ICLR 2019 5
Tasks: Inductive Bias, LAMBADA, Language Modeling, Language Modelling, Learning to Execute, Machine Translation, Translation

Recurrent neural networks (RNNs) sequentially process data by updating their
state with each new data point, and have long been the de facto choice for
sequence modeling tasks. However, their inherently sequential computation makes
them slow to train. Feed-forward and convolutional architectures have recently
been shown to achieve superior results on some sequence modeling tasks such as
machine translation, with the added advantage that they concurrently process
all inputs in the sequence, leading to easy parallelization and faster training
times. Despite these successes, however, popular feed-forward sequence models
like the Transformer fail to generalize in many simple tasks that recurrent
models handle with ease, e.g. copying strings or even simple logical inference
when the string or formula lengths exceed those observed at training time. We
propose the Universal Transformer (UT), a parallel-in-time self-attentive
recurrent sequence model which can be cast as a generalization of the
Transformer model and which addresses these issues. UTs combine the
parallelizability and global receptive field of feed-forward sequence models
like the Transformer with the recurrent inductive bias of RNNs. We also add a
dynamic per-position halting mechanism and find that it improves accuracy on
several tasks. In contrast to the standard Transformer, under certain
assumptions, UTs can be shown to be Turing-complete. Our experiments show that
UTs outperform standard Transformers on a wide range of algorithmic and
language understanding tasks, including the challenging LAMBADA language
modeling task where UTs achieve a new state of the art, and machine translation
where UTs achieve a 0.9 BLEU improvement over Transformers on the WMT14 En-De
dataset.
Attack and defence in cellular decision-making: lessons from machine learning
Authors: Thomas J. Rademaker, Emmanuel Bengio, Paul François
Date: 2018-07-10 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

Machine learning algorithms can be fooled by small well-designed adversarial perturbations. This is reminiscent of cellular decision-making where ligands (called antagonists) prevent correct signalling, like in early immune recognition. We draw a formal analogy between neural networks used in machine learning and models of cellular decision-making (adaptive proofreading). We apply attacks from machine learning to simple decision-making models, and show explicitly the correspondence to antagonism by weakly bound ligands. Such antagonism is absent in more nonlinear models, which inspired us to implement a biomimetic defence in neural networks filtering out adversarial perturbations. We then apply a gradient-descent approach from machine learning to different cellular decision-making models, and we reveal the existence of two regimes characterized by the presence or absence of a critical point for the gradient. This critical point causes the strongest antagonists to lie close to the decision boundary. This is validated in the loss landscapes of robust neural networks and cellular decision-making models, and observed experimentally for immune cells. For both regimes, we explain how associated defence mechanisms shape the geometry of the loss landscape, and why different adversarial attacks are effective in different regimes. Our work connects evolved cellular decision-making to machine learning, and motivates the design of a general theory of adversarial perturbations, both for in vivo and in silico systems.