pwc-archive/papers-with-abstracts huggingface dataset:
Neural Machine Translation for Bilingually Scarce Scenarios: A Deep Multi-task Learning Approach
Authors: Poorya Zaremoodi, Gholamreza Haffari
Date: 2018-05-11 00:00:00
Proceeding: NAACL 2018 6
Tasks: Machine Translation, Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Semantic Parsing, Translation

Neural machine translation requires large amounts of parallel training text
to learn a reasonable-quality translation model. This is particularly
inconvenient for language pairs for which enough parallel text is not
available. In this paper, we use monolingual linguistic resources in the source
side to address this challenging problem based on a multi-task learning
approach. More specifically, we scaffold the machine translation task on
auxiliary tasks including semantic parsing, syntactic parsing, and named-entity
recognition. This effectively injects semantic and/or syntactic knowledge into
the translation model, which would otherwise require a large amount of training
bitext. We empirically evaluate and show the effectiveness of our multi-task
learning approach on three translation tasks: English-to-French,
English-to-Farsi, and English-to-Vietnamese.
Retinal Vessel Segmentation Based on Conditional Deep Convolutional Generative Adversarial Networks
Authors: Yun Jiang, Ning Tan
Date: 2018-05-11 00:00:00
Tasks: Retinal Vessel Segmentation, Segmentation, Sensitivity

The segmentation of retinal vessels is of significance for doctors to
diagnose the fundus diseases. However, existing methods have various problems
in the segmentation of the retinal vessels, such as insufficient segmentation
of retinal vessels, weak anti-noise interference ability, and sensitivity to
lesions, etc. Aiming to the shortcomings of existed methods, this paper
proposes the use of conditional deep convolutional generative adversarial
networks to segment the retinal vessels. We mainly improve the network
structure of the generator. The introduction of the residual module at the
convolutional layer for residual learning makes the network structure sensitive
to changes in the output, as to better adjust the weight of the generator. In
order to reduce the number of parameters and calculations, using a small
convolution to halve the number of channels in the input signature before using
a large convolution kernel. By used skip connection to connect the output of
the convolutional layer with the output of the deconvolution layer to avoid
low-level information sharing. By verifying the method on the DRIVE and STARE
datasets, the segmentation accuracy rate is 96.08% and 97.71%, the sensitivity
reaches 82.74% and 85.34% respectively, and the F-measure reaches 82.08% and
85.02% respectively. The sensitivity is 4.82% and 2.4% higher than that of
R2U-Net.
Human-Machine Collaborative Optimization via Apprenticeship Scheduling
Authors: Matthew Gombolay, Reed Jensen, Jessica Stigile, Toni Golen, Neel Shah, Sung-Hyun Son, Julie Shah
Date: 2018-05-11 00:00:00
Tasks: Decision Making, Job Shop Scheduling, Scheduling

Coordinating agents to complete a set of tasks with intercoupled temporal and
resource constraints is computationally challenging, yet human domain experts
can solve these difficult scheduling problems using paradigms learned through
years of apprenticeship. A process for manually codifying this domain knowledge
within a computational framework is necessary to scale beyond the
``single-expert, single-trainee" apprenticeship model. However, human domain
experts often have difficulty describing their decision-making processes,
causing the codification of this knowledge to become laborious. We propose a
new approach for capturing domain-expert heuristics through a pairwise ranking
formulation. Our approach is model-free and does not require enumerating or
iterating through a large state space. We empirically demonstrate that this
approach accurately learns multifaceted heuristics on a synthetic data set
incorporating job-shop scheduling and vehicle routing problems, as well as on
two real-world data sets consisting of demonstrations of experts solving a
weapon-to-target assignment problem and a hospital resource allocation problem.
We also demonstrate that policies learned from human scheduling demonstration
via apprenticeship learning can substantially improve the efficiency of a
branch-and-bound search for an optimal schedule. We employ this human-machine
collaborative optimization technique on a variant of the weapon-to-target
assignment problem. We demonstrate that this technique generates solutions
substantially superior to those produced by human domain experts at a rate up
to 9.5 times faster than an optimization approach and can be applied to
optimally solve problems twice as complex as those solved by a human
demonstrator.
Structure-from-Motion using Dense CNN Features with Keypoint Relocalization
Authors: Aji Resindra Widya, Akihiko Torii, Masatoshi Okutomi
Date: 2018-05-10 00:00:00

Structure from Motion (SfM) using imagery that involves extreme appearance
changes is yet a challenging task due to a loss of feature repeatability. Using
feature correspondences obtained by matching densely extracted convolutional
neural network (CNN) features significantly improves the SfM reconstruction
capability. However, the reconstruction accuracy is limited by the spatial
resolution of the extracted CNN features which is not even pixel-level accuracy
in the existing approach. Providing dense feature matches with precise keypoint
positions is not trivial because of memory limitation and computational burden
of dense features. To achieve accurate SfM reconstruction with highly
repeatable dense features, we propose an SfM pipeline that uses dense CNN
features with relocalization of keypoint position that can efficiently and
accurately provide pixel-level feature correspondences. Then, we demonstrate on
the Aachen Day-Night dataset that the proposed SfM using dense CNN features
with the keypoint relocalization outperforms a state-of-the-art SfM (COLMAP
using RootSIFT) by a large margin.
Deep RNNs Encode Soft Hierarchical Syntax
Authors: Terra Blevins, Omer Levy, Luke Zettlemoyer
Date: 2018-05-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Dependency Parsing, Language Modeling, Language Modelling, Machine Translation, Semantic Role Labeling, Translation

We present a set of experiments to demonstrate that deep recurrent neural
networks (RNNs) learn internal representations that capture soft hierarchical
notions of syntax from highly varied supervision. We consider four syntax tasks
at different depths of the parse tree; for each word, we predict its part of
speech as well as the first (parent), second (grandparent) and third level
(great-grandparent) constituent labels that appear above it. These predictions
are made from representations produced at different depths in networks that are
pretrained with one of four objectives: dependency parsing, semantic role
labeling, machine translation, or language modeling. In every case, we find a
correspondence between network depth and syntactic depth, suggesting that a
soft syntactic hierarchy emerges. This effect is robust across all conditions,
indicating that the models encode significant amounts of syntax even in the
absence of an explicit syntactic training supervision.
An Adaptive Population Size Differential Evolution with Novel Mutation Strategy for Constrained Optimization
Authors: Yuan Fu, Hu Wang, Meng-Zhu Yang
Date: 2018-05-11 00:00:00

Differential evolution (DE) has competitive performance on constrained
optimization problems (COPs), which targets at searching for global optimal
solution without violating the constraints. Generally, researchers pay more
attention on avoiding violating the constraints than better objective function
value. To achieve the aim of searching the feasible solutions accurately, an
adaptive population size method and an adaptive mutation strategy are proposed
in the paper. The adaptive population method is similar to a state switch which
controls the exploring state and exploiting state according to the situation of
feasible solution search. The novel mutation strategy is designed to enhance
the effect of status switch based on adaptive population size, which is useful
to reduce the constraint violations. Moreover, a mechanism based on
multipopulation competition and a more precise method of constraint control are
adopted in the proposed algorithm. The proposed differential evolution
algorithm, APDE-NS, is evaluated on the benchmark problems from CEC2017
constrained real parameter optimization. The experimental results show the
effectiveness of the proposed method is competitive compared to other
state-of-the-art algorithms.
Learning Disentangled Joint Continuous and Discrete Representations
Authors: Emilien Dupont
Date: 2018-03-31 00:00:00
Proceeding: NeurIPS 2018 12

We present a framework for learning disentangled and interpretable jointly
continuous and discrete representations in an unsupervised manner. By
augmenting the continuous latent distribution of variational autoencoders with
a relaxed discrete distribution and controlling the amount of information
encoded in each latent unit, we show how continuous and categorical factors of
variation can be discovered automatically from data. Experiments show that the
framework disentangles continuous and discrete generative factors on various
datasets and outperforms current disentangling methods when a discrete
generative factor is prominent.
Pushing the Limits of Unconstrained Face Detection: a Challenge Dataset and Baseline Results
Authors: Hajime Nada, Vishwanath A. Sindagi, He Zhang, Vishal M. Patel
Date: 2018-04-26 00:00:00
Tasks: Face Detection, Robust Face Recognition

Face detection has witnessed immense progress in the last few years, with new
milestones being surpassed every year. While many challenges such as large
variations in scale, pose, appearance are successfully addressed, there still
exist several issues which are not specifically captured by existing methods or
datasets. In this work, we identify the next set of challenges that requires
attention from the research community and collect a new dataset of face images
that involve these issues such as weather-based degradations, motion blur,
focus blur and several others. We demonstrate that there is a considerable gap
in the performance of state-of-the-art detectors and real-world requirements.
Hence, in an attempt to fuel further research in unconstrained face detection,
we present a new annotated Unconstrained Face Detection Dataset (UFDD) with
several challenges and benchmark recent methods. Additionally, we provide an
in-depth analysis of the results and failure cases of these methods. The
dataset as well as baseline results will be made publicly available in due
time. The UFDD dataset as well as baseline results are available at:
www.ufdd.info/
Automatic Article Commenting: the Task and Dataset
Authors: Lianhui Qin, Lemao Liu, Victoria Bi, Yan Wang, Xiaojiang Liu, Zhiting Hu, Hai Zhao, Shuming Shi
Date: 2018-05-09 00:00:00
Proceeding: ACL 2018 7
Tasks: Articles, Comment Generation

Comments of online articles provide extended views and improve user
engagement. Automatically making comments thus become a valuable functionality
for online forums, intelligent chatbots, etc. This paper proposes the new task
of automatic article commenting, and introduces a large-scale Chinese dataset
with millions of real comments and a human-annotated subset characterizing the
comments' varying quality. Incorporating the human bias of comment quality, we
further develop automatic metrics that generalize a broad set of popular
reference-based metrics and exhibit greatly improved correlations with human
evaluations.
Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness
Authors: Vicente Ivan Sanchez Carmona, Jeff Mitchell, Sebastian Riedel
Date: 2018-05-11 00:00:00
Proceeding: NAACL 2018 6
Tasks: Natural Language Inference, valid

Natural Language Inference is a challenging task that has received
substantial attention, and state-of-the-art models now achieve impressive test
set performance in the form of accuracy scores. Here, we go beyond this single
evaluation metric to examine robustness to semantically-valid alterations to
the input data. We identify three factors - insensitivity, polarity and unseen
pairs - and compare their impact on three SNLI models under a variety of
conditions. Our results demonstrate a number of strengths and weaknesses in the
models' ability to generalise to new in-domain instances. In particular, while
strong performance is possible on unseen hypernyms, unseen antonyms are more
challenging for all the models. More generally, the models suffer from an
insensitivity to certain small but semantically significant alterations, and
are also often influenced by simple statistical correlations between words and
training labels. Overall, we show that evaluations of NLI models can benefit
from studying the influence of factors intrinsic to the models or found in the
dataset used.
Learning to Grasp Without Seeing
Authors: Adithyavairavan Murali, Yin Li, Dhiraj Gandhi, Abhinav Gupta
Date: 2018-05-10 00:00:00
Tasks: Object Localization

Can a robot grasp an unknown object without seeing it? In this paper, we
present a tactile-sensing based approach to this challenging problem of
grasping novel objects without prior knowledge of their location or physical
properties. Our key idea is to combine touch based object localization with
tactile based re-grasping. To train our learning models, we created a
large-scale grasping dataset, including more than 30 RGB frames and over 2.8
million tactile samples from 7800 grasp interactions of 52 objects. To learn a
representation of tactile signals, we propose an unsupervised auto-encoding
scheme, which shows a significant improvement of 4-9% over prior methods on a
variety of tactile perception tasks. Our system consists of two steps. First,
our touch localization model sequentially 'touch-scans' the workspace and uses
a particle filter to aggregate beliefs from multiple hits of the target. It
outputs an estimate of the object's location, from which an initial grasp is
established. Next, our re-grasping model learns to progressively improve grasps
with tactile feedback based on the learned features. This network learns to
estimate grasp stability and predict adjustment for the next grasp. Re-grasping
thus is performed iteratively until our model identifies a stable grasp.
Finally, we demonstrate extensive experimental results on grasping a large set
of novel objects using tactile sensing alone. Furthermore, when applied on top
of a vision-based policy, our re-grasping model significantly boosts the
overall accuracy by 10.6%. We believe this is the first attempt at learning to
grasp with only tactile sensing and without any prior object knowledge.
GANAX: A Unified MIMD-SIMD Acceleration for Generative Adversarial Networks
Authors: Amir Yazdanbakhsh, Hajar Falahati, Philip J. Wolfe, Kambiz Samadi, Nam Sung Kim, Hadi Esmaeilzadeh
Date: 2018-05-10 00:00:00

Generative Adversarial Networks (GANs) are one of the most recent deep
learning models that generate synthetic data from limited genuine datasets.
GANs are on the frontier as further extension of deep learning into many
domains (e.g., medicine, robotics, content synthesis) requires massive sets of
labeled data that is generally either unavailable or prohibitively costly to
collect. Although GANs are gaining prominence in various fields, there are no
accelerators for these new models. In fact, GANs leverage a new operator,
called transposed convolution, that exposes unique challenges for hardware
acceleration. This operator first inserts zeros within the multidimensional
input, then convolves a kernel over this expanded array to add information to
the embedded zeros. Even though there is a convolution stage in this operator,
the inserted zeros lead to underutilization of the compute resources when a
conventional convolution accelerator is employed. We propose the GANAX
architecture to alleviate the sources of inefficiency associated with the
acceleration of GANs using conventional convolution accelerators, making the
first GAN accelerator design possible. We propose a reorganization of the
output computations to allocate compute rows with similar patterns of zeros to
adjacent processing engines, which also avoids inconsequential multiply-adds on
the zeros. This compulsory adjacency reclaims data reuse across these
neighboring processing engines, which had otherwise diminished due to the
inserted zeros. The reordering breaks the full SIMD execution model, which is
prominent in convolution accelerators. Therefore, we propose a unified
MIMD-SIMD design for GANAX that leverages repeated patterns in the computation
to create distinct microprograms that execute concurrently in SIMD mode.
An Unsupervised Clustering-Based Short-Term Solar Forecasting Methodology Using Multi-Model Machine Learning Blending
Authors: Cong Feng, Mingjian Cui, Bri-Mathias Hodge, Siyuan Lu, Hendrik F. Hamann, Jie Zhang
Date: 2018-05-10 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Time Series, Time Series Analysis

Solar forecasting accuracy is affected by weather conditions, and weather
awareness forecasting models are expected to improve the performance. However,
it may not be available and reliable to classify different forecasting tasks by
using only meteorological weather categorization. In this paper, an
unsupervised clustering-based (UC-based) solar forecasting methodology is
developed for short-term (1-hour-ahead) global horizontal irradiance (GHI)
forecasting. This methodology consists of three parts: GHI time series
unsupervised clustering, pattern recognition, and UC-based forecasting. The
daily GHI time series is first clustered by an Optimized Cross-validated
ClUsteRing (OCCUR) method, which determines the optimal number of clusters and
best clustering results. Then, support vector machine pattern recognition
(SVM-PR) is adopted to recognize the category of a certain day using the first
few hours' data in the forecasting stage. GHI forecasts are generated by the
most suitable models in different clusters, which are built by a two-layer
Machine learning based Multi-Model (M3) forecasting framework. The developed
UC-based methodology is validated by using 1-year of data with six solar
features. Numerical results show that (i) UC-based models outperform non-UC
(all-in-one) models with the same M3 architecture by approximately 20%; (ii)
M3-based models also outperform the single-algorithm machine learning (SAML)
models by approximately 20%.
Survey and cross-benchmark comparison of remaining time prediction methods in business process monitoring
Authors: Ilya Verenich, Marlon Dumas, Marcello La Rosa, Fabrizio Maggi, Irene Teinemaa
Date: 2018-05-08 00:00:00
Tasks: Prediction, Systematic Literature Review

Predictive business process monitoring methods exploit historical process
execution logs to generate predictions about running instances (called cases)
of a business process, such as the prediction of the outcome, next activity or
remaining cycle time of a given process case. These insights could be used to
support operational managers in taking remedial actions as business processes
unfold, e.g. shifting resources from one case onto another to ensure this
latter is completed on time. A number of methods to tackle the remaining cycle
time prediction problem have been proposed in the literature. However, due to
differences in their experimental setup, choice of datasets, evaluation
measures and baselines, the relative merits of each method remain unclear. This
article presents a systematic literature review and taxonomy of methods for
remaining time prediction in the context of business processes, as well as a
cross-benchmark comparison of 16 such methods based on 16 real-life datasets
originating from different industry domains.
Deep Neural Machine Translation with Weakly-Recurrent Units
Authors: Mattia Antonino Di Gangi, Marcello Federico
Date: 2018-05-10 00:00:00
Tasks: Machine Translation, NMT, Translation, valid

Recurrent neural networks (RNNs) have represented for years the state of the
art in neural machine translation. Recently, new architectures have been
proposed, which can leverage parallel computation on GPUs better than classical
RNNs. Faster training and inference combined with different
sequence-to-sequence modeling also lead to performance improvements. While the
new models completely depart from the original recurrent architecture, we
decided to investigate how to make RNNs more efficient. In this work, we
propose a new recurrent NMT architecture, called Simple Recurrent NMT, built on
a class of fast and weakly-recurrent units that use layer normalization and
multiple attentions. Our experiments on the WMT14 English-to-German and WMT16
English-Romanian benchmarks show that our model represents a valid alternative
to LSTMs, as it can achieve better results at a significantly lower
computational cost.
Joint Embedding of Words and Labels for Text Classification
Authors: Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin
Date: 2018-05-10 00:00:00
Proceeding: ACL 2018 7
Tasks: Classification, General Classification, Sentiment Analysis, text-classification, Text Classification

Word embeddings are effective intermediate representations for capturing
semantic regularities between words, when learning the representations of text
sequences. We propose to view text classification as a label-word joint
embedding problem: each label is embedded in the same space with the word
vectors. We introduce an attention framework that measures the compatibility of
embeddings between text sequences and labels. The attention is learned on a
training set of labeled samples to ensure that, given a text sequence, the
relevant words are weighted higher than the irrelevant ones. Our method
maintains the interpretability of word embeddings, and enjoys a built-in
ability to leverage alternative sources of information, in addition to input
text sequences. Extensive results on the several large text datasets show that
the proposed framework outperforms the state-of-the-art methods by a large
margin, in terms of both accuracy and speed.
Unifying Data, Model and Hybrid Parallelism in Deep Learning via Tensor Tiling
Authors: Minjie Wang, Chien-chin Huang, Jinyang Li
Date: 2018-05-10 00:00:00

Deep learning systems have become vital tools across many fields, but the
increasing model sizes mean that training must be accelerated to maintain such
systems' utility. Current systems like Tensorflow and MXNet focus on one
specific parallelization strategy, data parallelism, which requires large
training batch sizes in order to scale. We cast the problem of finding the best
parallelization strategy as the problem of finding the best tiling to partition
tensors with the least overall communication. We propose an algorithm that can
find the optimal tiling. Our resulting parallelization solution is a hybrid of
data parallelism and model parallelism. We build the SoyBean system that
performs automatic parallelization. SoyBean automatically transforms a serial
dataflow graph captured by an existing deep learning system frontend into a
parallel dataflow graph based on the optimal tiling it has found. Our
evaluations show that SoyBean is 1.5x-4x faster than pure data parallelism for
AlexNet and VGG. We present this automatic tiling in a new system, SoyBean,
that can act as a backend for Tensorflow, MXNet, and others.