pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning
Authors: Ulysse Côté-Allard, Cheikh Latyr Fall, Alexandre Drouin, Alexandre Campeau-Lecours, Clément Gosselin, Kyrre Glette, François Laviolette, Benoit Gosselin
Date: 2018-01-10 00:00:00
Tasks: Deep Learning, EMG Gesture Recognition, General Classification, Gesture Recognition, Transfer Learning

In recent years, deep learning algorithms have become increasingly more
prominent for their unparalleled ability to automatically learn discriminant
features from large amounts of data. However, within the field of
electromyography-based gesture recognition, deep learning algorithms are seldom
employed as they require an unreasonable amount of effort from a single person,
to generate tens of thousands of examples.
  This work's hypothesis is that general, informative features can be learned
from the large amounts of data generated by aggregating the signals of multiple
users, thus reducing the recording burden while enhancing gesture recognition.
Consequently, this paper proposes applying transfer learning on aggregated data
from multiple users, while leveraging the capacity of deep learning algorithms
to learn discriminant features from large datasets. Two datasets comprised of
19 and 17 able-bodied participants respectively (the first one is employed for
pre-training) were recorded for this work, using the Myo Armband. A third Myo
Armband dataset was taken from the NinaPro database and is comprised of 10
able-bodied participants. Three different deep learning networks employing
three different modalities as input (raw EMG, Spectrograms and Continuous
Wavelet Transform (CWT)) are tested on the second and third dataset. The
proposed transfer learning scheme is shown to systematically and significantly
enhance the performance for all three networks on the two datasets, achieving
an offline accuracy of 98.31% for 7 gestures over 17 participants for the
CWT-based ConvNet and 68.98% for 18 gestures over 10 participants for the raw
EMG-based ConvNet. Finally, a use-case study employing eight able-bodied
participants suggests that real-time feedback allows users to adapt their
muscle activation strategy which reduces the degradation in accuracy normally
experienced over time.
Explaining and Generalizing Back-Translation through Wake-Sleep
Authors: Ryan Cotterell, Julia Kreutzer
Date: 2018-06-12 00:00:00
Tasks: Machine Translation, Translation

Back-translation has become a commonly employed heuristic for semi-supervised
neural machine translation. The technique is both straightforward to apply and
has led to state-of-the-art results. In this work, we offer a principled
interpretation of back-translation as approximate inference in a generative
model of bitext and show how the standard implementation of back-translation
corresponds to a single iteration of the wake-sleep algorithm in our proposed
model. Moreover, this interpretation suggests a natural iterative
generalization, which we demonstrate leads to further improvement of up to 1.6
BLEU.
Quantum classification of the MNIST dataset with Slow Feature Analysis
Authors: Iordanis Kerenidis, Alessandro Luongo
Date: 2018-05-22 00:00:00
Tasks: Benchmarking, Classification, Dimensionality Reduction, General Classification, Quantum Machine Learning

Quantum machine learning carries the promise to revolutionize information and communication technologies. While a number of quantum algorithms with potential exponential speedups have been proposed already, it is quite difficult to provide convincing evidence that quantum computers with quantum memories will be in fact useful to solve real-world problems. Our work makes considerable progress towards this goal. We design quantum techniques for Dimensionality Reduction and for Classification, and combine them to provide an efficient and high accuracy quantum classifier that we test on the MNIST dataset. More precisely, we propose a quantum version of Slow Feature Analysis (QSFA), a dimensionality reduction technique that maps the dataset in a lower dimensional space where we can apply a novel quantum classification procedure, the Quantum Frobenius Distance (QFD). We simulate the quantum classifier (including errors) and show that it can provide classification of the MNIST handwritten digit dataset, a widely used dataset for benchmarking classification algorithms, with $98.5\%$ accuracy, similar to the classical case. The running time of the quantum classifier is polylogarithmic in the dimension and number of data points. We also provide evidence that the other parameters on which the running time depends (condition number, Frobenius norm, error threshold, etc.) scale favorably in practice, thus ascertaining the efficiency of our algorithm.
Attentive cross-modal paratope prediction
Authors: Andreea Deac, Petar Veličković, Pietro Sormanni
Date: 2018-06-12 00:00:00
Tasks: Antibody-antigen binding prediction, Computational Efficiency, Prediction

Antibodies are a critical part of the immune system, having the function of
directly neutralising or tagging undesirable objects (the antigens) for future
destruction. Being able to predict which amino acids belong to the paratope,
the region on the antibody which binds to the antigen, can facilitate antibody
design and contribute to the development of personalised medicine. The
suitability of deep neural networks has recently been confirmed for this task,
with Parapred outperforming all prior physical models. Our contribution is
twofold: first, we significantly outperform the computational efficiency of
Parapred by leveraging \`a trous convolutions and self-attention. Secondly, we
implement cross-modal attention by allowing the antibody residues to attend
over antigen residues. This leads to new state-of-the-art results on this task,
along with insightful interpretations.
Community Detection in Partially Observable Social Networks
Authors: Cong Tran, Won-Yong Shin, Andreas Spitz
Date: 2017-12-30 00:00:00
Tasks: Community Detection

The discovery of community structures in social networks has gained significant attention since it is a fundamental problem in understanding the networks' topology and functions. However, most social network data are collected from partially observable networks with both missing nodes and edges. In this paper, we address a new problem of detecting overlapping community structures in the context of such an incomplete network, where communities in the network are allowed to overlap since nodes belong to multiple communities at once. To solve this problem, we introduce KroMFac, a new framework that conducts community detection via regularized nonnegative matrix factorization (NMF) based on the Kronecker graph model. Specifically, from an inferred Kronecker generative parameter matrix, we first estimate the missing part of the network. As our major contribution to the proposed framework, to improve community detection accuracy, we then characterize and select influential nodes (which tend to have high degrees) by ranking, and add them to the existing graph. Finally, we uncover the community structures by solving the regularized NMF-aided optimization problem in terms of maximizing the likelihood of the underlying graph. Furthermore, adopting normalized mutual information (NMI), we empirically show superiority of our KroMFac approach over two baseline schemes by using both synthetic and real-world networks.
Qiniu Submission to ActivityNet Challenge 2018
Authors: Xiaoteng Zhang, Yixin Bao, Feiyun Zhang, Kai Hu, Yicheng Wang, Liang Zhu, Qinzhu He, Yining Lin, Jie Shao, Yao Peng
Date: 2018-06-12 00:00:00
Tasks: Activity Recognition, Optical Flow Estimation

In this paper, we introduce our submissions for the tasks of trimmed activity
recognition (Kinetics) and trimmed event recognition (Moments in Time) for
Activitynet Challenge 2018. In the two tasks, non-local neural networks and
temporal segment networks are implemented as our base models. Multi-modal cues
such as RGB image, optical flow and acoustic signal have also been used in our
method. We also propose new non-local-based models for further improvement on
the recognition accuracy. The final submissions after ensembling the models
achieve 83.5% top-1 accuracy and 96.8% top-5 accuracy on the Kinetics
validation set, 35.81% top-1 accuracy and 62.59% top-5 accuracy on the MIT
validation set.
The Weighted Kendall and High-order Kernels for Permutations
Authors: Yunlong Jiao, Jean-Philippe Vert
Date: 2018-02-23 00:00:00
Proceeding: ICML 2018 7
Tasks: Vocal Bursts Intensity Prediction

We propose new positive definite kernels for permutations. First we introduce
a weighted version of the Kendall kernel, which allows to weight unequally the
contributions of different item pairs in the permutations depending on their
ranks. Like the Kendall kernel, we show that the weighted version is invariant
to relabeling of items and can be computed efficiently in $O(n \ln(n))$
operations, where $n$ is the number of items in the permutation. Second, we
propose a supervised approach to learn the weights by jointly optimizing them
with the function estimated by a kernel machine. Third, while the Kendall
kernel considers pairwise comparison between items, we extend it by considering
higher-order comparisons among tuples of items and show that the supervised
approach of learning the weights can be systematically generalized to
higher-order permutation kernels.
How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD
Authors: Zeyuan Allen-Zhu
Date: 2018-01-08 00:00:00
Proceeding: NeurIPS 2018 12

Stochastic gradient descent (SGD) gives an optimal convergence rate when minimizing convex stochastic objectives $f(x)$. However, in terms of making the gradients small, the original SGD does not give an optimal rate, even when $f(x)$ is convex. If $f(x)$ is convex, to find a point with gradient norm $\varepsilon$, we design an algorithm SGD3 with a near-optimal rate $\tilde{O}(\varepsilon^{-2})$, improving the best known rate $O(\varepsilon^{-8/3})$ of [18]. If $f(x)$ is nonconvex, to find its $\varepsilon$-approximate local minimum, we design an algorithm SGD5 with rate $\tilde{O}(\varepsilon^{-3.5})$, where previously SGD variants only achieve $\tilde{O}(\varepsilon^{-4})$ [6, 15, 33]. This is no slower than the best known stochastic version of Newton's method in all parameter regimes [30].
Fast Rotational Sparse Coding
Authors: Michael T. McCann, Vincent Andrearczyk, Michael Unser, Adrien Depeursinge
Date: 2018-06-12 00:00:00
Tasks: Dictionary Learning, Texture Classification

We propose an algorithm for rotational sparse coding along with an efficient implementation using steerability. Sparse coding (also called dictionary learning) is an important technique in image processing, useful in inverse problems, compression, and analysis; however, the usual formulation fails to capture an important aspect of the structure of images: images are formed from building blocks, e.g., edges, lines, or points, that appear at different locations, orientations, and scales. The sparse coding problem can be reformulated to explicitly account for these transforms, at the cost of increased computation. In this work, we propose an algorithm for a rotational version of sparse coding that is based on K-SVD with additional rotation operations. We then propose a method to accelerate these rotations by learning the dictionary in a steerable basis. Our experiments on patch coding and texture classification demonstrate that the proposed algorithm is fast enough for practical use and compares favorably to standard sparse coding.
Kernel Recursive ABC: Point Estimation with Intractable Likelihood
Authors: Takafumi Kajihara, Motonobu Kanagawa, Keisuke Yamazaki, Kenji Fukumizu
Date: 2018-02-23 00:00:00
Proceeding: ICML 2018 7
Tasks: parameter estimation

We propose a novel approach to parameter estimation for simulator-based
statistical models with intractable likelihood. Our proposed method involves
recursive application of kernel ABC and kernel herding to the same observed
data. We provide a theoretical explanation regarding why the approach works,
showing (for the population setting) that, under a certain assumption, point
estimates obtained with this method converge to the true parameter, as
recursion proceeds. We have conducted a variety of numerical experiments,
including parameter estimation for a real-world pedestrian flow simulator, and
show that in most cases our method outperforms existing approaches.
Initialize globally before acting locally: Enabling Landmark-free 3D US to MRI Registration
Authors: Julia Rackerseder, Maximilian Baust, Rüdiger Göbl, Nassir Navab, Christoph Hennersperger
Date: 2018-06-12 00:00:00
Tasks: Image Registration, Position

Registration of partial-view 3D US volumes with MRI data is influenced by
initialization. The standard of practice is using extrinsic or intrinsic
landmarks, which can be very tedious to obtain. To overcome the limitations of
registration initialization, we present a novel approach that is based on
Euclidean distance maps derived from easily obtainable coarse segmentations. We
evaluate our approach quantitatively on the publicly available RESECT dataset
and show that it is robust regarding overlap of target area and initial
position. Furthermore, our method provides initializations that are suitable
for state-of-the-art nonlinear, deformable image registration algorithm's
capture ranges.
Mean Field Multi-Agent Reinforcement Learning
Authors: Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Wei-Nan Zhang, Jun Wang
Date: 2018-02-15 00:00:00
Proceeding: ICML 2018 7
Tasks: Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Existing multi-agent reinforcement learning methods are limited typically to a small number of agents. When the agent number increases largely, the learning becomes intractable due to the curse of the dimensionality and the exponential growth of agent interactions. In this paper, we present \emph{Mean Field Reinforcement Learning} where the interactions within the population of agents are approximated by those between a single agent and the average effect from the overall population or neighboring agents; the interplay between the two entities is mutually reinforced: the learning of the individual agent's optimal policy depends on the dynamics of the population, while the dynamics of the population change according to the collective patterns of the individual policies. We develop practical mean field Q-learning and mean field Actor-Critic algorithms and analyze the convergence of the solution to Nash equilibrium. Experiments on Gaussian squeeze, Ising model, and battle games justify the learning effectiveness of our mean field approaches. In addition, we report the first result to solve the Ising model via model-free reinforcement learning methods.
Drift Analysis
Authors: Johannes Lengler
Date: 2017-12-04 00:00:00
Tasks: Evolutionary Algorithms

Drift analysis is one of the major tools for analysing evolutionary
algorithms and nature-inspired search heuristics. In this chapter we give an
introduction to drift analysis and give some examples of how to use it for the
analysis of evolutionary algorithms.
MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning
Authors: Bo Zhao, Xinwei Sun, Yanwei Fu, Yuan YAO, Yizhou Wang
Date: 2018-06-12 00:00:00
Proceeding: ICML 2018 7
Tasks: feature selection, Zero-Shot Learning

It is one typical and general topic of learning a good embedding model to
efficiently learn the representation coefficients between two spaces/subspaces.
To solve this task, $L_{1}$ regularization is widely used for the pursuit of
feature selection and avoiding overfitting, and yet the sparse estimation of
features in $L_{1}$ regularization may cause the underfitting of training data.
$L_{2}$ regularization is also frequently used, but it is a biased estimator.
In this paper, we propose the idea that the features consist of three
orthogonal parts, \emph{namely} sparse strong signals, dense weak signals and
random noise, in which both strong and weak signals contribute to the fitting
of data. To facilitate such novel decomposition, \emph{MSplit} LBI is for the
first time proposed to realize feature selection and dense estimation
simultaneously. We provide theoretical and simulational verification that our
method exceeds $L_{1}$ and $L_{2}$ regularization, and extensive experimental
results show that our method achieves state-of-the-art performance in the
few-shot and zero-shot learning.
Multi-Task Neural Models for Translating Between Styles Within and Across Languages
Authors: Xing Niu, Sudha Rao, Marine Carpuat
Date: 2018-06-12 00:00:00
Proceeding: COLING 2018 8
Tasks: Machine Translation, Multi-Task Learning, Translation

Generating natural language requires conveying content in an appropriate
style. We explore two related tasks on generating text of varying formality:
monolingual formality transfer and formality-sensitive machine translation. We
propose to solve these tasks jointly using multi-task learning, and show that
our models achieve state-of-the-art performance for formality transfer and are
able to perform formality-sensitive translation without being explicitly
trained on style-annotated translation examples.
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Authors: Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, Andrew Rabinovich
Date: 2017-11-07 00:00:00
Proceeding: ICML 2018 7

Deep multitask networks, in which one neural network produces multiple
predictive outputs, can offer better speed and performance than their
single-task counterparts but are challenging to train properly. We present a
gradient normalization (GradNorm) algorithm that automatically balances
training in deep multitask models by dynamically tuning gradient magnitudes. We
show that for various network architectures, for both regression and
classification tasks, and on both synthetic and real datasets, GradNorm
improves accuracy and reduces overfitting across multiple tasks when compared
to single-task networks, static baselines, and other adaptive multitask loss
balancing techniques. GradNorm also matches or surpasses the performance of
exhaustive grid search methods, despite only involving a single asymmetry
hyperparameter $\alpha$. Thus, what was once a tedious search process that
incurred exponentially more compute for each task added can now be accomplished
within a few training runs, irrespective of the number of tasks. Ultimately, we
will demonstrate that gradient manipulation affords us great control over the
training dynamics of multitask networks and may be one of the keys to unlocking
the potential of multitask learning.
Exploiting Document Knowledge for Aspect-level Sentiment Classification
Authors: Ruidan He, Wee Sun Lee, Hwee Tou Ng, Daniel Dahlmeier
Date: 2018-06-12 00:00:00
Proceeding: ACL 2018 7
Tasks: Aspect-Based Sentiment Analysis (ABSA), Classification, General Classification, Sentiment Analysis, Sentiment Classification

Attention-based long short-term memory (LSTM) networks have proven to be
useful in aspect-level sentiment classification. However, due to the
difficulties in annotating aspect-level data, existing public datasets for this
task are all relatively small, which largely limits the effectiveness of those
neural models. In this paper, we explore two approaches that transfer knowledge
from document- level data, which is much less expensive to obtain, to improve
the performance of aspect-level sentiment classification. We demonstrate the
effectiveness of our approaches on 4 public datasets from SemEval 2014, 2015,
and 2016, and we show that attention-based LSTM benefits from document-level
knowledge in multiple ways.
Optimizing Variational Quantum Circuits using Evolution Strategies
Authors: Johannes S. Otterbach
Date: 2018-06-12 00:00:00

This version withdrawn by arXiv administrators because the submitter did not
have the right to agree to our license at the time of submission.
Focused Hierarchical RNNs for Conditional Sequence Processing
Authors: Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
Date: 2018-06-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Open-Domain Question Answering, Policy Gradient Methods, Question Answering

Recurrent Neural Networks (RNNs) with attention mechanisms have obtained
state-of-the-art results for many sequence processing tasks. Most of these
models use a simple form of encoder with attention that looks over the entire
sequence and assigns a weight to each token independently. We present a
mechanism for focusing RNN encoders for sequence modelling tasks which allows
them to attend to key parts of the input as needed. We formulate this using a
multi-layer conditional sequence encoder that reads in one token at a time and
makes a discrete decision on whether the token is relevant to the context or
question being asked. The discrete gating mechanism takes in the context
embedding and the current hidden state as inputs and controls information flow
into the layer above. We train it using policy gradient methods. We evaluate
this method on several types of tasks with different attributes. First, we
evaluate the method on synthetic tasks which allow us to evaluate the model for
its generalization ability and probe the behavior of the gates in more
controlled settings. We then evaluate this approach on large scale Question
Answering tasks including the challenging MS MARCO and SearchQA tasks. Our
models shows consistent improvements for both tasks over prior work and our
baselines. It has also shown to generalize significantly better on synthetic
tasks as compared to the baselines.
When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?
Authors: Tengyu Xu, Yi Zhou, Kaiyi Ji, Yingbin Liang
Date: 2018-06-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Binary Classification

We study the implicit bias of gradient descent methods in solving a binary
classification problem over a linearly separable dataset. The classifier is
described by a nonlinear ReLU model and the objective function adopts the
exponential loss function. We first characterize the landscape of the loss
function and show that there can exist spurious asymptotic local minima besides
asymptotic global minima. We then show that gradient descent (GD) can converge
to either a global or a local max-margin direction, or may diverge from the
desired max-margin direction in a general context. For stochastic gradient
descent (SGD), we show that it converges in expectation to either the global or
the local max-margin direction if SGD converges. We further explore the
implicit bias of these algorithms in learning a multi-neuron network under
certain stationary conditions, and show that the learned classifier maximizes
the margins of each sample pattern partition under the ReLU activation.