pwc-archive/papers-with-abstracts huggingface dataset:
Combining Advanced Methods in Japanese-Vietnamese Neural Machine Translation
Authors: Thi-Vinh Ngo, Thanh-Le Ha, Phuong-Thai Nguyen, Le-Minh Nguyen
Date: 2018-05-18 00:00:00
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) systems have recently obtained state-of-the
art in many machine translation systems between popular language pairs because
of the availability of data. For low-resourced language pairs, there are few
researches in this field due to the lack of bilingual data. In this paper, we
attempt to build the first NMT systems for a low-resourced language
pairs:Japanese-Vietnamese. We have also shown significant improvements when
combining advanced methods to reduce the adverse impacts of data sparsity and
improve the quality of NMT systems. In addition, we proposed a variant of
Byte-Pair Encoding algorithm to perform effective word segmentation for
Vietnamese texts and alleviate the rare-word problem that persists in NMT
systems.
Tree Edit Distance Learning via Adaptive Symbol Embeddings: Supplementary Materials and Results
Authors: Benjamin Paaßen
Date: 2018-05-18 00:00:00
Tasks: Metric Learning

Metric learning has the aim to improve classification accuracy by learning a
distance measure which brings data points from the same class closer together
and pushes data points from different classes further apart. Recent research
has demonstrated that metric learning approaches can also be applied to trees,
such as molecular structures, abstract syntax trees of computer programs, or
syntax trees of natural language, by learning the cost function of an edit
distance, i.e. the costs of replacing, deleting, or inserting nodes in a tree.
However, learning such costs directly may yield an edit distance which violates
metric axioms, is challenging to interpret, and may not generalize well. In
this contribution, we propose a novel metric learning approach for trees which
learns an edit distance indirectly by embedding the tree nodes as vectors, such
that the Euclidean distance between those vectors supports class
discrimination. We learn such embeddings by reducing the distance to
prototypical trees from the same class and increasing the distance to
prototypical trees from different classes. In our experiments, we show that our
proposed metric learning approach improves upon the state-of-the-art in metric
learning for trees on six benchmark data sets, ranging from computer science
over biomedical data to a natural-language processing data set containing over
300,000 nodes.
Change Point Methods on a Sequence of Graphs
Authors: Daniele Zambon, Cesare Alippi, Lorenzo Livi
Date: 2018-05-18 00:00:00
Tasks: Graph Classification, Seizure Detection

Given a finite sequence of graphs, e.g., coming from technological,
biological, and social networks, the paper proposes a methodology to identify
possible changes in stationarity in the stochastic process generating the
graphs. In order to cover a large class of applications, we consider the
general family of attributed graphs where both topology (number of vertexes and
edge configuration) and related attributes are allowed to change also in the
stationary case. Novel Change Point Methods (CPMs) are proposed, that (i) map
graphs into a vector domain; (ii) apply a suitable statistical test in the
vector space; (iii) detect the change --if any-- according to a confidence
level and provide an estimate for its time occurrence. Two specific
multivariate CPMs have been designed: one that detects shifts in the
distribution mean, the other addressing generic changes affecting the
distribution. We ground our proposal with theoretical results showing how to
relate the inference attained in the numerical vector space to the graph
domain, and vice versa. We also show how to extend the methodology for handling
multiple change points in the same sequence. Finally, the proposed CPMs have
been validated on real data sets coming from epileptic-seizure detection
problems and on labeled data sets for graph classification. Results show the
effectiveness of what proposed in relevant application scenarios.
Adaptive Recurrent Neural Network Based on Mixture Layer
Authors: Kui Zhao, Yuechuan Li, Chi Zhang, Cheng Yang, Huan Xu
Date: 2018-01-24 00:00:00

Although Recurrent Neural Network (RNN) has been a powerful tool for modeling
sequential data, its performance is inadequate when processing sequences with
multiple patterns. In this paper, we address this challenge by introducing a
novel mixture layer and constructing an adaptive RNN. The mixture layer
augmented RNN (termed as M-RNN) partitions patterns in training sequences into
several clusters and stores the principle patterns as prototype vectors of
components in a mixture model. By leveraging the mixture layer, the proposed
method can adaptively update states according to the similarities between
encoded inputs and prototype vectors, leading to a stronger capacity in
assimilating sequences with multiple patterns. Moreover, our approach can be
further extended by taking advantage of prior knowledge about data. Experiments
on both synthetic and real datasets demonstrate the effectiveness of the
proposed method.
Improving Image Captioning with Conditional Generative Adversarial Nets
Authors: Chen Chen, Shuai Mu, Wanpeng Xiao, Zexiong Ye, Liesi Wu, Qi Ju
Date: 2018-05-18 00:00:00
Tasks: Decoder, Image Captioning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we propose a novel
conditional-generative-adversarial-nets-based image captioning framework as an
extension of traditional reinforcement-learning (RL)-based encoder-decoder
architecture. To deal with the inconsistent evaluation problem among different
objective language metrics, we are motivated to design some "discriminator"
networks to automatically and progressively determine whether generated caption
is human described or machine generated. Two kinds of discriminator
architectures (CNN and RNN-based structures) are introduced since each has its
own advantages. The proposed algorithm is generic so that it can enhance any
existing RL-based image captioning framework and we show that the conventional
RL training method is just a special case of our approach. Empirically, we show
consistent improvements over all language evaluation metrics for different
state-of-the-art image captioning models. In addition, the well-trained
discriminators can also be viewed as objective image captioning evaluators
Extending Dynamic Bayesian Networks for Anomaly Detection in Complex Logs
Authors: Stephen Pauwels, Toon Calders
Date: 2018-05-18 00:00:00
Tasks: Anomaly Detection

Checking various log files from different processes can be a tedious task as
these logs contain lots of events, each with a (possibly large) number of
attributes. We developed a way to automatically model log files and detect
outlier traces in the data. For that we extend Dynamic Bayesian Networks to
model the normal behavior found in log files. We introduce a new algorithm that
is able to learn a model of a log file starting from the data itself. The model
is capable of scoring traces even when new values or new combinations of values
appear in the log file.
TractSeg - Fast and accurate white matter tract segmentation
Authors: Jakob Wasserthal, Peter Neher, Klaus H. Maier-Hein
Date: 2018-05-18 00:00:00
Tasks: Clustering, Image Registration

The individual course of white matter fiber tracts is an important key for
analysis of white matter characteristics in healthy and diseased brains.
Uniquely, diffusion-weighted MRI tractography in combination with region-based
or clustering-based selection of streamlines allows for the in-vivo delineation
and analysis of anatomically well known tracts. This, however, currently
requires complex, computationally intensive and tedious-to-set-up processing
pipelines. TractSeg is a novel convolutional neural network-based approach that
directly segments tracts in the field of fiber orientation distribution
function (fODF) peaks without requiring tractography, image registration or
parcellation. We demonstrate in 105 subjects from the Human Connectome Project
that the proposed approach is much faster than existing methods while providing
unprecedented accuracy. The code and data are openly available at
https://github.com/MIC-DKFZ/TractSeg/ and
https://doi.org/10.5281/zenodo.1088277, respectively.
An attention-based Bi-GRU-CapsNet model for hypernymy detection between compound entities
Authors: Qi. Wang, Chenming Xu, Yangming Zhou, Tong Ruan, Daqi Gao, Ping He
Date: 2018-05-13 00:00:00

Named entities are usually composable and extensible. Typical examples are
names of symptoms and diseases in medical areas. To distinguish these entities
from general entities, we name them \textit{compound entities}. In this paper,
we present an attention-based Bi-GRU-CapsNet model to detect hypernymy
relationship between compound entities. Our model consists of several important
components. To avoid the out-of-vocabulary problem, English words or Chinese
characters in compound entities are fed into the bidirectional gated recurrent
units. An attention mechanism is designed to focus on the differences between
the two compound entities. Since there are some different cases in hypernymy
relationship between compound entities, capsule network is finally employed to
decide whether the hypernymy relationship exists or not. Experimental results
demonstrate
Deep learning improved by biological activation functions
Authors: Gardave S Bhumbra
Date: 2018-03-19 00:00:00
Tasks: Deep Learning

`Biologically inspired' activation functions, such as the logistic sigmoid,
have been instrumental in the historical advancement of machine learning.
However in the field of deep learning, they have been largely displaced by
rectified linear units (ReLU) or similar functions, such as its exponential
linear unit (ELU) variant, to mitigate the effects of vanishing gradients
associated with error back-propagation. The logistic sigmoid however does not
represent the true input-output relation in neuronal cells under physiological
conditions. Here, bionodal root unit (BRU) activation functions are introduced,
exhibiting input-output non-linearities that are substantially more
biologically plausible since their functional form is based on known
biophysical properties of neuronal cells.
  In order to evaluate the learning performance of BRU activations, deep
networks are constructed with identical architectures except differing in their
transfer functions (ReLU, ELU, and BRU). Multilayer perceptrons, stacked
auto-encoders, and convolutional networks are used to test supervised and
unsupervised learning based on the MNIST and CIFAR-10/100 datasets. Comparisons
of learning performance, quantified using loss and error measurements,
demonstrate that bionodal networks both train faster than their ReLU and ELU
counterparts and result in the best generalised models even in the absence of
formal regularisation. These results therefore suggest that revisiting the
detailed properties of biological neurones and their circuitry might prove
invaluable in the field of deep learning for the future.
Tropical Geometry of Deep Neural Networks
Authors: Liwen Zhang, Gregory Naitzat, Lek-Heng Lim
Date: 2018-05-18 00:00:00
Proceeding: ICML 2018 7

We establish, for the first time, connections between feedforward neural
networks with ReLU activation and tropical geometry --- we show that the family
of such neural networks is equivalent to the family of tropical rational maps.
Among other things, we deduce that feedforward ReLU neural networks with one
hidden layer can be characterized by zonotopes, which serve as building blocks
for deeper networks; we relate decision boundaries of such neural networks to
tropical hypersurfaces, a major object of study in tropical geometry; and we
prove that linear regions of such neural networks correspond to vertices of
polytopes associated with tropical rational functions. An insight from our
tropical formulation is that a deeper network is exponentially more expressive
than a shallow network.
Trusted Neural Networks for Safety-Constrained Autonomous Control
Authors: Shalini Ghosh, Amaury Mercier, Dheeraj Pichapati, Susmit Jha, Vinod Yegneswaran, Patrick Lincoln
Date: 2018-05-18 00:00:00
Tasks: Self-Driving Cars

We propose Trusted Neural Network (TNN) models, which are deep neural network
models that satisfy safety constraints critical to the application domain. We
investigate different mechanisms for incorporating rule-based knowledge in the
form of first-order logic constraints into a TNN model, where rules that encode
safety are accompanied by weights indicating their relative importance. This
framework allows the TNN model to learn from knowledge available in form of
data as well as logical rules. We propose multiple approaches for solving this
problem: (a) a multi-headed model structure that allows trade-off between
satisfying logical constraints and fitting training data in a unified training
framework, and (b) creating a constrained optimization problem and solving it
in dual formulation by posing a new constrained loss function and using a
proximal gradient descent algorithm. We demonstrate the efficacy of our TNN
framework through experiments using the open-source TORCS~\cite{BernhardCAA15}
3D simulator for self-driving cars. Experiments using our first approach of a
multi-headed TNN model, on a dataset generated by a customized version of
TORCS, show that (1) adding safety constraints to a neural network model
results in increased performance and safety, and (2) the improvement increases
with increasing importance of the safety constraints. Experiments were also
performed using the second approach of proximal algorithm for constrained
optimization --- they demonstrate how the proposed method ensures that (1) the
overall TNN model satisfies the constraints even when the training data
violates some of the constraints, and (2) the proximal gradient descent
algorithm on the constrained objective converges faster than the unconstrained
version.
Optimizing for Generalization in Machine Learning with Cross-Validation Gradients
Authors: Shane Barratt, Rishi Sharma
Date: 2018-05-18 00:00:00
Tasks: BIG-bench Machine Learning, Hyperparameter Optimization, regression

Cross-validation is the workhorse of modern applied statistics and machine
learning, as it provides a principled framework for selecting the model that
maximizes generalization performance. In this paper, we show that the
cross-validation risk is differentiable with respect to the hyperparameters and
training data for many common machine learning algorithms, including logistic
regression, elastic-net regression, and support vector machines. Leveraging
this property of differentiability, we propose a cross-validation gradient
method (CVGM) for hyperparameter optimization. Our method enables efficient
optimization in high-dimensional hyperparameter spaces of the cross-validation
risk, the best surrogate of the true generalization ability of our learning
algorithm.
Multifunction Cognitive Radar Task Scheduling Using Monte Carlo Tree Search and Policy Networks
Authors: Mahdi Shaghaghi, Raviraj S. Adve, Zhen Ding
Date: 2018-05-18 00:00:00
Tasks: Management, Scheduling

A modern radar may be designed to perform multiple functions, such as
surveillance, tracking, and fire control. Each function requires the radar to
execute a number of transmit-receive tasks. A radar resource management (RRM)
module makes decisions on parameter selection, prioritization, and scheduling
of such tasks. RRM becomes especially challenging in overload situations, where
some tasks may need to be delayed or even dropped. In general, task scheduling
is an NP-hard problem. In this work, we develop the branch-and-bound (B&B)
method which obtains the optimal solution but at exponential computational
complexity. On the other hand, heuristic methods have low complexity but
provide relatively poor performance. We resort to machine learning-based
techniques to address this issue; specifically we propose an approximate
algorithm based on the Monte Carlo tree search method. Along with using bound
and dominance rules to eliminate nodes from the search tree, we use a policy
network to help to reduce the width of the search. Such a network can be
trained using solutions obtained by running the B&B method offline on problems
with feasible complexity. We show that the proposed method provides
near-optimal performance, but with computational complexity orders of magnitude
smaller than the B&B algorithm.
Bayesian Joint Spike-and-Slab Graphical Lasso
Authors: Zehang Richard Li, Tyler H. McCormick, Samuel J. Clark
Date: 2018-05-18 00:00:00
Tasks: Bayesian Inference, Model Selection

In this article, we propose a new class of priors for Bayesian inference with multiple Gaussian graphical models. We introduce fully Bayesian treatments of two popular procedures, the group graphical lasso and the fused graphical lasso, and extend them to a continuous spike-and-slab framework to allow self-adaptive shrinkage and model selection simultaneously. We develop an EM algorithm that performs fast and dynamic explorations of posterior modes. Our approach selects sparse models efficiently with substantially smaller bias than would be induced by alternative regularization procedures. The performance of the proposed methods are demonstrated through simulation and two real data examples.
SNU_IDS at SemEval-2018 Task 12: Sentence Encoder with Contextualized Vectors for Argument Reasoning Comprehension
Authors: Taeuk Kim, Jihun Choi, Sang-goo Lee
Date: 2018-05-18 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Machine Translation, Sentence, Transfer Learning, Translation

We present a novel neural architecture for the Argument Reasoning
Comprehension task of SemEval 2018. It is a simple neural network consisting of
three parts, collectively judging whether the logic built on a set of given
sentences (a claim, reason, and warrant) is plausible or not. The model
utilizes contextualized word vectors pre-trained on large machine translation
(MT) datasets as a form of transfer learning, which can help to mitigate the
lack of training data. Quantitative analysis shows that simply leveraging LSTMs
trained on MT datasets outperforms several baselines and non-transferred
models, achieving accuracies of about 70% on the development set and about 60%
on the test set.
Asynch-SGBDT: Asynchronous Parallel Stochastic Gradient Boosting Decision Tree based on Parameters Server
Authors: Cheng Daning, Xia Fen, Li Shigang, Zhang Yunquan
Date: 2018-04-12 00:00:00
Tasks: BIG-bench Machine Learning, Diversity

In AI research and industry, machine learning is the most widely used tool. One of the most important machine learning algorithms is Gradient Boosting Decision Tree, i.e. GBDT whose training process needs considerable computational resources and time. To shorten GBDT training time, many works tried to apply GBDT on Parameter Server. However, those GBDT algorithms are synchronous parallel algorithms which fail to make full use of Parameter Server. In this paper, we examine the possibility of using asynchronous parallel methods to train GBDT model and name this algorithm as asynch-SGBDT (asynchronous parallel stochastic gradient boosting decision tree). Our theoretical and experimental results indicate that the scalability of asynch-SGBDT is influenced by the sample diversity of datasets, sampling rate, step length and the setting of GBDT tree. Experimental results also show asynch-SGBDT training process reaches a linear speedup in asynchronous parallel manner when datasets and GBDT trees meet high scalability requirements.
Aspect Based Sentiment Analysis with Gated Convolutional Networks
Authors: Wei Xue, Tao Li
Date: 2018-05-18 00:00:00
Proceeding: ACL 2018 7
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect Category Sentiment Analysis, Sentiment Analysis

Aspect based sentiment analysis (ABSA) can provide more detailed information
than general sentiment analysis, because it aims to predict the sentiment
polarities of the given aspects or entities in text. We summarize previous
approaches into two subtasks: aspect-category sentiment analysis (ACSA) and
aspect-term sentiment analysis (ATSA). Most previous approaches employ long
short-term memory and attention mechanisms to predict the sentiment polarity of
the concerned targets, which are often complicated and need more training time.
We propose a model based on convolutional neural networks and gating
mechanisms, which is more accurate and efficient. First, the novel Gated
Tanh-ReLU Units can selectively output the sentiment features according to the
given aspect or entity. The architecture is much simpler than attention layer
used in the existing models. Second, the computations of our model could be
easily parallelized during training, because convolutional layers do not have
time dependency as in LSTM layers, and gating units also work independently.
The experiments on SemEval datasets demonstrate the efficiency and
effectiveness of our models.
Semi-Supervised Learning with Declaratively Specified Entropy Constraints
Authors: Haitian Sun, William W. Cohen, Lidong Bing
Date: 2018-04-24 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Bayesian Optimization, Relation Extraction

We propose a technique for declaratively specifying strategies for
semi-supervised learning (SSL). The proposed method can be used to specify
ensembles of semi-supervised learning, as well as agreement constraints and
entropic regularization constraints between these learners, and can be used to
model both well-known heuristics such as co-training and novel domain-specific
heuristics. In addition to representing individual SSL heuristics, we show that
multiple heuristics can also be automatically combined using Bayesian
optimization methods. We show consistent improvements on a suite of
well-studied SSL benchmarks, including a new state-of-the-art result on a
difficult relation extraction task.
A Self-Replication Basis for Designing Complex Agents
Authors: Thommen George Karimpanal
Date: 2018-05-18 00:00:00

In this work, we describe a self-replication-based mechanism for designing
agents of increasing complexity. We demonstrate the validity of this approach
by solving simple, standard evolutionary computation problems in simulation. In
the context of these simulation results, we describe the fundamental
differences of this approach when compared to traditional approaches. Further,
we highlight the possible advantages of applying this approach to the problem
of designing complex artificial agents, along with the potential drawbacks and
issues to be addressed in the future.
Domain Adaptation from Synthesis to Reality in Single-model Detector for Video Smoke Detection
Authors: Gao Xu, Yongming Zhang, Qixing Zhang, Gaohua Lin, Jinjun Wang
Date: 2017-09-24 00:00:00
Tasks: Domain Adaptation, object-detection, Object Detection

This paper proposes a method for video smoke detection using synthetic smoke
samples. The virtual data can automatically offer precise and rich annotated
samples. However, the learning of smoke representations will be hurt by the
appearance gap between real and synthetic smoke samples. The existed researches
mainly work on the adaptation to samples extracted from original annotated
samples. These methods take the object detection and domain adaptation as two
independent parts. To train a strong detector with rich synthetic samples, we
construct the adaptation to the detection layer of state-of-the-art
single-model detectors (SSD and MS-CNN). The training procedure is an
end-to-end stage. The classification, location and adaptation are combined in
the learning. The performance of the proposed model surpasses the original
baseline in our experiments. Meanwhile, our results show that the detectors
based on the adversarial adaptation are superior to the detectors based on the
discrepancy adaptation. Code will be made publicly available on
http://smoke.ustc.edu.cn. Moreover, the domain adaptation for two-stage
detector is described in Appendix A.