pwc-archive/papers-with-abstracts huggingface dataset:
Optimizing Segmentation Granularity for Neural Machine Translation
Authors: Elizabeth Salesky, Andrew Runge, Alex Coda, Jan Niehues, Graham Neubig
Date: 2018-10-19 00:00:00
Tasks: Machine Translation, NMT, Translation

In neural machine translation (NMT), it is has become standard to translate
using subword units to allow for an open vocabulary and improve accuracy on
infrequent words. Byte-pair encoding (BPE) and its variants are the predominant
approach to generating these subwords, as they are unsupervised, resource-free,
and empirically effective. However, the granularity of these subword units is a
hyperparameter to be tuned for each language and task, using methods such as
grid search. Tuning may be done inexhaustively or skipped entirely due to
resource constraints, leading to sub-optimal performance. In this paper, we
propose a method to automatically tune this parameter using only one training
pass. We incrementally introduce new vocabulary online based on the held-out
validation loss, beginning with smaller, general subwords and adding larger,
more specific units over the course of training. Our method matches the results
found with grid search, optimizing segmentation granularity without any
additional training time. We also show benefits in training efficiency and
performance improvements for rare words due to the way embeddings for larger
units are incrementally constructed by combining those from smaller units.
On Extensions of CLEVER: A Neural Network Robustness Evaluation Algorithm
Authors: Tsui-Wei Weng, huan zhang, Pin-Yu Chen, Aurelie Lozano, Cho-Jui Hsieh, Luca Daniel
Date: 2018-10-19 00:00:00

CLEVER (Cross-Lipschitz Extreme Value for nEtwork Robustness) is an Extreme
Value Theory (EVT) based robustness score for large-scale deep neural networks
(DNNs). In this paper, we propose two extensions on this robustness score.
First, we provide a new formal robustness guarantee for classifier functions
that are twice differentiable. We apply extreme value theory on the new formal
robustness guarantee and the estimated robustness is called second-order CLEVER
score. Second, we discuss how to handle gradient masking, a common defensive
technique, using CLEVER with Backward Pass Differentiable Approximation (BPDA).
With BPDA applied, CLEVER can evaluate the intrinsic robustness of neural
networks of a broader class -- networks with non-differentiable input
transformations. We demonstrate the effectiveness of CLEVER with BPDA in
experiments on a 121-layer Densenet model trained on the ImageNet dataset.
Fast and Robust Multiple ColorChecker Detection using Deep Convolutional Neural Networks
Authors: Pedro D. Marrero Fernandez, Fidel A. Guerrero-Peña, Tsang Ing Ren, Jorge J. G. Leandro
Date: 2018-10-19 00:00:00

ColorCheckers are reference standards that professional photographers and
filmmakers use to ensure predictable results under every lighting condition.
The objective of this work is to propose a new fast and robust method for
automatic ColorChecker detection. The process is divided into two steps: (1)
ColorCheckers localization and (2) ColorChecker patches recognition. For the
ColorChecker localization, we trained a detection convolutional neural network
using synthetic images. The synthetic images are created with the 3D models of
the ColorChecker and different background images. The output of the neural
networks are the bounding box of each possible ColorChecker candidates in the
input image. Each bounding box defines a cropped image which is evaluated by a
recognition system, and each image is canonized with regards to color and
dimensions. Subsequently, all possible color patches are extracted and grouped
with respect to the center's distance. Each group is evaluated as a candidate
for a ColorChecker part, and its position in the scene is estimated. Finally, a
cost function is applied to evaluate the accuracy of the estimation. The method
is tested using real and synthetic images. The proposed method is fast, robust
to overlaps and invariant to affine projections. The algorithm also performs
well in case of multiple ColorCheckers detection.
Population and Empirical PR Curves for Assessment of Ranking Algorithms
Authors: Jacqueline M. Hughes-Oliver
Date: 2018-10-19 00:00:00

The ROC curve is widely used to assess the quality of
prediction/classification/ranking algorithms, and its properties have been
extensively studied. The precision-recall (PR) curve has become the de facto
replacement for the ROC curve in the presence of imbalance, namely where one
class is far more likely than the other class. While the PR and ROC curves tend
to be used interchangeably, they have some very different properties.
Properties of the PR curve are the focus of this paper. We consider: (1)
population PR curves, where complete distributional assumptions are specified
for scores from both classes; and (2) empirical estimators of the PR curve,
where we observe scores and no distributional assumptions are made. The
properties have direct consequence on how the PR curve should, and should not,
be used. For example, the empirical PR curve is not consistent when scores in
the class of primary interest come from discrete distributions. On the other
hand, a normal approximation can fit quite well for points on the empirical PR
curve from continuously-defined scores, but convergence can be heavily
influenced by the distributional setting, the amount of imbalance, and the
point of interest on the PR curve.
A Model Parallel Proximal Stochastic Gradient Algorithm for Partially Asynchronous Systems
Authors: Rui Zhu, Di Niu
Date: 2018-10-19 00:00:00
Tasks: BIG-bench Machine Learning, Recommendation Systems

Large models are prevalent in modern machine learning scenarios, including
deep learning, recommender systems, etc., which can have millions or even
billions of parameters. Parallel algorithms have become an essential solution
technique to many large-scale machine learning jobs. In this paper, we propose
a model parallel proximal stochastic gradient algorithm, AsyB-ProxSGD, to deal
with large models using model parallel blockwise updates while in the meantime
handling a large amount of training data using proximal stochastic gradient
descent (ProxSGD). In our algorithm, worker nodes communicate with the
parameter servers asynchronously, and each worker performs proximal stochastic
gradient for only one block of model parameters during each iteration. Our
proposed algorithm generalizes ProxSGD to the asynchronous and model parallel
setting. We prove that AsyB-ProxSGD achieves a convergence rate of
$O(1/\sqrt{K})$ to stationary points for nonconvex problems under
\emph{constant} minibatch sizes, where $K$ is the total number of block
updates. This rate matches the best-known rates of convergence for a wide range
of gradient-like algorithms. Furthermore, we show that when the number of
workers is bounded by $O(K^{1/4})$, we can expect AsyB-ProxSGD to achieve
linear speedup as the number of workers increases. We implement the proposed
algorithm on MXNet and demonstrate its convergence behavior and near-linear
speedup on a real-world dataset involving both a large model size and large
amounts of data.
Autonomous Functional Locomotion in a Tendon-Driven Limb via Limited Experience
Authors: Ali Marjaninejad, Darío Urbina-Meléndez, Brian A. Cohn, Francisco J. Valero-Cuevas
Date: 2018-10-19 00:00:00

Robots will become ubiquitously useful only when they can use few attempts to
teach themselves to perform different tasks, even with complex bodies and in
dynamical environments. Vertebrates, in fact, successfully use trial-and-error
to learn multiple tasks in spite of their intricate tendon-driven anatomies.
Roboticists find such tendon-driven systems particularly hard to control
because they are simultaneously nonlinear, under-determined (many tendon
tensions combine to produce few net joint torques), and over-determined (few
joint rotations define how many tendons need to be reeled-in/payed-out). We
demonstrate---for the first time in simulation and in hardware---how a
model-free approach allows few-shot autonomous learning to produce effective
locomotion in a 3-tendon/2-joint tendon-driven leg. Initially, an artificial
neural network fed by sparsely sampled data collected using motor babbling
creates an inverse map from limb kinematics to motor activations, which is
analogous to juvenile vertebrates playing during development. Thereafter,
iterative reward-driven exploration of candidate motor activations
simultaneously refines the inverse map and finds a functional locomotor
limit-cycle autonomously. This biologically-inspired algorithm, which we call
G2P (General to Particular), enables versatile adaptation of robots to changes
in the target task, mechanics of their bodies, and environment. Moreover, this
work empowers future studies of few-shot autonomous learning in biological
systems, which is the foundation of their enviable functional versatility.
Gradient target propagation
Authors: Tiago de Souza Farias, Jonas Maziero
Date: 2018-10-19 00:00:00

We report a learning rule for neural networks that computes how much each
neuron should contribute to minimize a giving cost function via the estimation
of its target value. By theoretical analysis, we show that this learning rule
contains backpropagation, Hebian learning, and additional terms. We also give a
general technique for weights initialization. Our results are at least as good
as those obtained with backpropagation. The neural networks are trained and
tested in three problems: MNIST, MNIST-Fashion, and CIFAR-10 datasets. The
associated code is available at https://github.com/tiago939/target.
CNN inference acceleration using dictionary of centroids
Authors: D. Babin, I. Mazurenko, D. Parkhomenko, A. Voloshko
Date: 2018-10-19 00:00:00
Tasks: Quantization, Tensor Decomposition

It is well known that multiplication operations in convolutional layers of
common CNNs consume a lot of time during inference stage. In this article we
present a flexible method to decrease both computational complexity of
convolutional layers in inference as well as amount of space to store them. The
method is based on centroid filter quantization and outperforms approaches
based on tensor decomposition by a large margin. We performed comparative
analysis of the proposed method and series of CP tensor decomposition on
ImageNet benchmark and found that our method provide almost 2.9 times better
computational gain. Despite the simplicity of our method it cannot be applied
directly in inference stage in modern frameworks, but could be useful for cases
calculation flow could be changed, e.g. for CNN-chip designers.
Real-time Neural-based Input Method
Authors: Jiali Yao, Raphael Shu, Xinjian Li, Katsutoshi Ohtsuki, Hideki Nakayama
Date: 2018-10-19 00:00:00
Proceeding: ICLR 2019 5
Tasks: CPU, Language Modeling, Language Modelling

The input method is an essential service on every mobile and desktop devices
that provides text suggestions. It converts sequential keyboard inputs to the
characters in its target language, which is indispensable for Japanese and
Chinese users. Due to critical resource constraints and limited network
bandwidth of the target devices, applying neural models to input method is not
well explored. In this work, we apply a LSTM-based language model to input
method and evaluate its performance for both prediction and conversion tasks
with Japanese BCCWJ corpus. We articulate the bottleneck to be the slow softmax
computation during conversion. To solve the issue, we propose incremental
softmax approximation approach, which computes softmax with a selected subset
vocabulary and fix the stale probabilities when the vocabulary is updated in
future steps. We refer to this method as incremental selective softmax. The
results show a two order speedup for the softmax computation when converting
Japanese input sequences with a large vocabulary, reaching real-time speed on
commodity CPU. We also exploit the model compressing potential to achieve a 92%
model size reduction without losing accuracy.
A database linking piano and orchestral MIDI scores with application to automatic projective orchestration
Authors: Léopold Crestel, Philippe Esling, Lena Heng, Stephen McAdams
Date: 2018-10-19 00:00:00

This article introduces the Projective Orchestral Database (POD), a
collection of MIDI scores composed of pairs linking piano scores to their
corresponding orchestrations. To the best of our knowledge, this is the first
database of its kind, which performs piano or orchestral prediction, but more
importantly which tries to learn the correlations between piano and orchestral
scores. Hence, we also introduce the projective orchestration task, which
consists in learning how to perform the automatic orchestration of a piano
score. We show how this task can be addressed using learning methods and also
provide methodological guidelines in order to properly use this database.
Revisiting Distributional Correspondence Indexing: A Python Reimplementation and New Experiments
Authors: Alejandro Moreo, Andrea Esuli, Fabrizio Sebastiani
Date: 2018-10-19 00:00:00
Tasks: Domain Adaptation, General Classification, Sentiment Analysis, text-classification, Text Classification, Transfer Learning

This paper introduces PyDCI, a new implementation of Distributional
Correspondence Indexing (DCI) written in Python. DCI is a transfer learning
method for cross-domain and cross-lingual text classification for which we had
provided an implementation (here called JaDCI) built on top of JaTeCS, a Java
framework for text classification. PyDCI is a stand-alone version of DCI that
exploits scikit-learn and the SciPy stack. We here report on new experiments
that we have carried out in order to test PyDCI, and in which we use as
baselines new high-performing methods that have appeared after DCI was
originally proposed. These experiments show that, thanks to a few subtle ways
in which we have improved DCI, PyDCI outperforms both JaDCI and the
above-mentioned high-performing methods, and delivers the best known results on
the two popular benchmarks on which we had tested DCI, i.e.,
MultiDomainSentiment (a.k.a. MDS -- for cross-domain adaptation) and
Webis-CLS-10 (for cross-lingual adaptation). PyDCI, together with the code
allowing to replicate our experiments, is available at
https://github.com/AlexMoreo/pydci .
A Stacked Autoencoder Neural Network based Automated Feature Extraction Method for Anomaly detection in On-line Condition Monitoring
Authors: Mohendra Roy, Sumon Kumar Bose, Bapi Kar, Pradeep Kumar Gopalakrishnan, Arindam Basu
Date: 2018-10-19 00:00:00
Tasks: Anomaly Detection

Condition monitoring is one of the routine tasks in all major process
industries. The mechanical parts such as a motor, gear, bearings are the major
components of a process industry and any fault in them may cause a total
shutdown of the whole process, which may result in serious losses. Therefore,
it is very crucial to predict any approaching defects before its occurrence.
Several methods exist for this purpose and many research are being carried out
for better and efficient models. However, most of them are based on the
processing of raw sensor signals, which is tedious and expensive. Recently,
there has been an increase in the feature based condition monitoring, where
only the useful features are extracted from the raw signals and interpreted for
the prediction of the fault. Most of these are handcrafted features, where
these are manually obtained based on the nature of the raw data. This of course
requires the prior knowledge of the nature of data and related processes. This
limits the feature extraction process. However, recent development in the
autoencoder based feature extraction method provides an alternative to the
traditional handcrafted approaches; however, they have mostly been confined in
the area of image and audio processing. In this work, we have developed an
automated feature extraction method for on-line condition monitoring based on
the stack of the traditional autoencoder and an on-line sequential extreme
learning machine(OSELM) network. The performance of this method is comparable
to that of the traditional feature extraction approaches. The method can
achieve 100% detection accuracy for determining the bearing health states of
NASA bearing dataset. The simple design of this method is promising for the
easy hardware implementation of Internet of Things(IoT) based prognostics
solutions.
Improving Annotation for 3D Pose Dataset of Fine-Grained Object Categories
Authors: Yaming Wang, Xiao Tan, Yi Yang, Ziyu Li, Xiao Liu, Feng Zhou, Larry S. Davis
Date: 2018-10-19 00:00:00
Tasks: 3D Pose Estimation, Object, Object Recognition, Pose Estimation

Existing 3D pose datasets of object categories are limited to generic object
types and lack of fine-grained information. In this work, we introduce a new
large-scale dataset that consists of 409 fine-grained categories and 31,881
images with accurate 3D pose annotation. Specifically, we augment three
existing fine-grained object recognition datasets (StanfordCars, CompCars and
FGVC-Aircraft) by finding a specific 3D model for each sub-category from
ShapeNet and manually annotating each 2D image by adjusting a full set of 7
continuous perspective parameters. Since the fine-grained shapes allow 3D
models to better fit the images, we further improve the annotation quality by
initializing from the human annotation and conducting local search of the pose
parameters with the objective of maximizing the IoUs between the projected mask
and the segmentation reference estimated from state-of-the-art deep
Convolutional Neural Networks (CNNs). We provide full statistics of the
annotations with qualitative and quantitative comparisons suggesting that our
dataset can be a complementary source for studying 3D pose estimation. The
dataset can be downloaded at http://users.umiacs.umd.edu/~wym/3dpose.html.
Infinite Factorial Finite State Machine for Blind Multiuser Channel Estimation
Authors: Francisco J. R. Ruiz, Isabel Valera, Lennart Svensson, Fernando Perez-Cruz
Date: 2018-10-18 00:00:00

New communication standards need to deal with machine-to-machine
communications, in which users may start or stop transmitting at any time in an
asynchronous manner. Thus, the number of users is an unknown and time-varying
parameter that needs to be accurately estimated in order to properly recover
the symbols transmitted by all users in the system. In this paper, we address
the problem of joint channel parameter and data estimation in a multiuser
communication channel in which the number of transmitters is not known. For
that purpose, we develop the infinite factorial finite state machine model, a
Bayesian nonparametric model based on the Markov Indian buffet that allows for
an unbounded number of transmitters with arbitrary channel length. We propose
an inference algorithm that makes use of slice sampling and particle Gibbs with
ancestor sampling. Our approach is fully blind as it does not require a prior
channel estimation step, prior knowledge of the number of transmitters, or any
signaling information. Our experimental results, loosely based on the LTE
random access channel, show that the proposed approach can effectively recover
the data-generating process for a wide range of scenarios, with varying number
of transmitters, number of receivers, constellation order, channel length, and
signal-to-noise ratio.
WikiHow: A Large Scale Text Summarization Dataset
Authors: Mahnaz Koupaee, William Yang Wang
Date: 2018-10-18 00:00:00
Tasks: Articles, Diversity, Text Summarization

Sequence-to-sequence models have recently gained the state of the art
performance in summarization. However, not too many large-scale high-quality
datasets are available and almost all the available ones are mainly news
articles with specific writing style. Moreover, abstractive human-style systems
involving description of the content at a deeper level require data with higher
levels of abstraction. In this paper, we present WikiHow, a dataset of more
than 230,000 article and summary pairs extracted and constructed from an online
knowledge base written by different human authors. The articles span a wide
range of topics and therefore represent high diversity styles. We evaluate the
performance of the existing methods on WikiHow to present its challenges and
set some baselines to further improve it.
Analyzing and Interpreting Convolutional Neural Networks in NLP
Authors: Mahnaz Koupaee, William Yang Wang
Date: 2018-10-18 00:00:00
Tasks: Decision Making, Negation

Convolutional neural networks have been successfully applied to various NLP
tasks. However, it is not obvious whether they model different linguistic
patterns such as negation, intensification, and clause compositionality to help
the decision-making process. In this paper, we apply visualization techniques
to observe how the model can capture different linguistic features and how
these features can affect the performance of the model. Later on, we try to
identify the model errors and their sources. We believe that interpreting CNNs
is the first step to understand the underlying semantic features which can
raise awareness to further improve the performance and explainability of CNN
models.
Classification of normal/abnormal heart sound recordings based on multi-domain features and back propagation neural network
Authors: Hong Tang, Huaming Chen, Ting Li, Mingjun Zhong
Date: 2018-10-17 00:00:00
Tasks: General Classification, Sensitivity, Specificity

This paper aims to classify a single PCG recording as normal or abnormal for
computer-aided diagnosis. The proposed framework for this challenge has four
steps: preprocessing, feature extraction, training and validation. In the
preprocessing step, a recording is segmented into four states, i.e., the first
heart sound, systolic interval, the second heart sound, and diastolic interval
by the Springer Segmentation algorithm. In the feature extraction step, the
authors extract 324 features from multi-domains to perform classification. A
back propagation neural network is used as predication model. The optimal
threshold for distinguishing normal and abnormal is determined by the
statistics of model output for both normal and abnormal. The performance of the
proposed predictor tested by the six training sets is sensitivity 0.812 and
specificity 0.860 (overall accuracy is 0.836). However, the performance reduces
to sensitivity 0.807 and specificity 0.829 (overall accuracy is 0.818) for the
hidden test set.
How to train your MAML
Authors: Antreas Antoniou, Harrison Edwards, Amos Storkey
Date: 2018-10-22 00:00:00
Proceeding: ICLR 2019 5
Tasks: Few-Shot Image Classification, Few-Shot Learning, Meta-Learning

The field of few-shot learning has recently seen substantial advancements.
Most of these advancements came from casting few-shot learning as a
meta-learning problem. Model Agnostic Meta Learning or MAML is currently one of
the best approaches for few-shot learning via meta-learning. MAML is simple,
elegant and very powerful, however, it has a variety of issues, such as being
very sensitive to neural network architectures, often leading to instability
during training, requiring arduous hyperparameter searches to stabilize
training and achieve high generalization and being very computationally
expensive at both training and inference times. In this paper, we propose
various modifications to MAML that not only stabilize the system, but also
substantially improve the generalization performance, convergence speed and
computational overhead of MAML, which we call MAML++.