pwc-archive/papers-with-abstracts huggingface dataset:
Learning Domain-Sensitive and Sentiment-Aware Word Embeddings
Authors: Bei Shi, Zihao Fu, Lidong Bing, Wai Lam
Date: 2018-05-10 00:00:00
Proceeding: ACL 2018 7
Tasks: Data Augmentation, General Classification, Sentence, Sentiment Analysis, Sentiment Classification, Word Embeddings

Word embeddings have been widely used in sentiment classification because of
their efficacy for semantic representations of words. Given reviews from
different domains, some existing methods for word embeddings exploit sentiment
information, but they cannot produce domain-sensitive embeddings. On the other
hand, some other existing methods can generate domain-sensitive word
embeddings, but they cannot distinguish words with similar contexts but
opposite sentiment polarity. We propose a new method for learning
domain-sensitive and sentiment-aware embeddings that simultaneously capture the
information of sentiment semantics and domain sensitivity of individual words.
Our method can automatically determine and produce domain-common embeddings and
domain-specific embeddings. The differentiation of domain-common and
domain-specific words enables the advantage of data augmentation of common
semantics from multiple domains and capture the varied semantics of specific
words from different domains at the same time. Experimental results show that
our model provides an effective way to learn domain-sensitive and
sentiment-aware word embeddings which benefit sentiment classification at both
sentence level and lexicon term level.
hyperdoc2vec: Distributed Representations of Hypertext Documents
Authors: Jialong Han, Yan Song, Wayne Xin Zhao, Shuming Shi, Haisong Zhang
Date: 2018-05-10 00:00:00
Proceeding: ACL 2018 7
Tasks: Citation Recommendation, Document Embedding, General Classification

Hypertext documents, such as web pages and academic papers, are of great
importance in delivering information in our daily life. Although being
effective on plain documents, conventional text embedding methods suffer from
information loss if directly adapted to hyper-documents. In this paper, we
propose a general embedding approach for hyper-documents, namely, hyperdoc2vec,
along with four criteria characterizing necessary information that
hyper-document embedding models should preserve. Systematic comparisons are
conducted between hyperdoc2vec and several competitors on two tasks, i.e.,
paper classification and citation recommendation, in the academic paper domain.
Analyses and experiments both validate the superiority of hyperdoc2vec to other
models w.r.t. the four criteria.
Toward `verifying' a Water Treatment System
Authors: Jingyi Wang, Jun Sun, Yifan Jia, Shengchao Qin, Zhiwu Xu
Date: 2017-12-12 00:00:00

Modeling and verifying real-world cyber-physical systems is challenging,
which is especially so for complex systems where manually modeling is
infeasible. In this work, we report our experience on combining model learning
and abstraction refinement to analyze a challenging system, i.e., a real-world
Secure Water Treatment system (SWaT). Given a set of safety requirements, the
objective is to either show that the system is safe with a high probability (so
that a system shutdown is rarely triggered due to safety violation) or not. As
the system is too complicated to be manually modeled, we apply latest automatic
model learning techniques to construct a set of Markov chains through
abstraction and refinement, based on two long system execution logs (one for
training and the other for testing). For each probabilistic safety property, we
either report it does not hold with a certain level of probabilistic
confidence, or report that it holds by showing the evidence in the form of an
abstract Markov chain. The Markov chains can subsequently be implemented as
runtime monitors in SWaT.
OFF-ApexNet on Micro-expression Recognition System
Authors: Sze-Teng Liong, Y. S. Gan, Wei-Chuen Yau, Yen-Chang Huang, Tan Lit Ken
Date: 2018-05-10 00:00:00
Tasks: Micro Expression Recognition, Micro-Expression Recognition, Optical Flow Estimation

When a person attempts to conceal an emotion, the genuine emotion is manifest
as a micro-expression. Exploration of automatic facial micro-expression
recognition systems is relatively new in the computer vision domain. This is
due to the difficulty in implementing optimal feature extraction methods to
cope with the subtlety and brief motion characteristics of the expression. Most
of the existing approaches extract the subtle facial movements based on
hand-crafted features. In this paper, we address the micro-expression
recognition task with a convolutional neural network (CNN) architecture, which
well integrates the features extracted from each video. A new feature
descriptor, Optical Flow Features from Apex frame Network (OFF-ApexNet) is
introduced. This feature descriptor combines the optical ow guided context with
the CNN. Firstly, we obtain the location of the apex frame from each video
sequence as it portrays the highest intensity of facial motion among all
frames. Then, the optical ow information are attained from the apex frame and a
reference frame (i.e., onset frame). Finally, the optical flow features are fed
into a pre-designed CNN model for further feature enhancement as well as to
carry out the expression classification. To evaluate the effectiveness of
OFF-ApexNet, comprehensive evaluations are conducted on three public
spontaneous micro-expression datasets (i.e., SMIC, CASME II and SAMM). The
promising recognition result suggests that the proposed method can optimally
describe the significant micro-expression details. In particular, we report
that, in a multi-database with leave-one-subject-out cross-validation
experimental protocol, the recognition performance reaches 74.60% of
recognition accuracy and F-measure of 71.04%. We also note that this is the
first work that performs cross-dataset validation on three databases in this
domain.
Dust concentration vision measurement based on moment of inertia in gray level-rank co-occurrence matrix
Authors: Zhiwen Luo, Guo-Hui Li, Junfeng Du, Jieping Wu
Date: 2018-05-10 00:00:00

To improve the accuracy of existing dust concentration measurements, a dust
concentration measurement based on Moment of inertia in Gray level-Rank
Co-occurrence Matrix (GRCM), which is from the dust image sample measured by a
machine vision system is proposed in this paper. Firstly, a Polynomial
computational model between dust Concentration and Moment of inertia (PCM) is
established by experimental methods and fitting methods. Then computing methods
for GRCM and its Moment of inertia are constructed by theoretical and
mathematical analysis methods. And then developing an on-line dust
concentration vision measurement experimental system, the cement dust
concentration measurement in a cement production workshop is taken as a
practice example with the system and the PCM measurement. The results show that
measurement error is within 9%, and the measurement range is 0.5-1000 mg/m3.
Finally, comparing with the filter membrane weighing measurement, light
scattering measurement and laser measurement, the proposed PCM measurement has
advantages on error and cost, which can be provided a valuable reference for
the dust concentration vision measurements.
Deep Learning of Geometric Constellation Shaping including Fiber Nonlinearities
Authors: Rasmus T. Jones, Tobias A. Eriksson, Metodi P. Yankov, Darko Zibar
Date: 2018-05-10 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning

A new geometric shaping method is proposed, leveraging unsupervised machine
learning to optimize the constellation design. The learned constellation
mitigates nonlinear effects with gains up to 0.13 bit/4D when trained with a
simplified fiber channel model.
SlugNERDS: A Named Entity Recognition Tool for Open Domain Dialogue Systems
Authors: Kevin K. Bowden, Jiaqi Wu, Shereen Oraby, Amita Misra, Marilyn Walker
Date: 2018-05-10 00:00:00
Proceeding: LREC 2018 5
Tasks: Entity Linking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In dialogue systems, the tasks of named entity recognition (NER) and named
entity linking (NEL) are vital preprocessing steps for understanding user
intent, especially in open domain interaction where we cannot rely on
domain-specific inference. UCSC's effort as one of the funded teams in the 2017
Amazon Alexa Prize Contest has yielded Slugbot, an open domain social bot,
aimed at casual conversation. We discovered several challenges specifically
associated with both NER and NEL when building Slugbot, such as that the NE
labels are too coarse-grained or the entity types are not linked to a useful
ontology. Moreover, we have discovered that traditional approaches do not
perform well in our context: even systems designed to operate on tweets or
other social media data do not work well in dialogue systems. In this paper, we
introduce Slugbot's Named Entity Recognition for dialogue Systems (SlugNERDS),
a NER and NEL tool which is optimized to address these issues. We describe two
new resources that we are building as part of this work: SlugEntityDB and
SchemaActuator. We believe these resources will be useful for the research
community.
k-Space Deep Learning for Accelerated MRI
Authors: Yoseob Han, Leonard Sunwoo, Jong Chul Ye
Date: 2018-05-10 00:00:00
Tasks: compressed sensing, Deep Learning, Denoising, Matrix Completion

The annihilating filter-based low-rank Hankel matrix approach (ALOHA) is one of the state-of-the-art compressed sensing approaches that directly interpolates the missing k-space data using low-rank Hankel matrix completion. The success of ALOHA is due to the concise signal representation in the k-space domain thanks to the duality between structured low-rankness in the k-space domain and the image domain sparsity. Inspired by the recent mathematical discovery that links convolutional neural networks to Hankel matrix decomposition using data-driven framelet basis, here we propose a fully data-driven deep learning algorithm for k-space interpolation. Our network can be also easily applied to non-Cartesian k-space trajectories by simply adding an additional regridding layer. Extensive numerical experiments show that the proposed deep learning method consistently outperforms the existing image-domain deep learning approaches.
Deep Reinforcement Learning for Optimal Control of Space Heating
Authors: Adam Nagy, Hussain Kazmi, Farah Cheaib, Johan Driesen
Date: 2018-05-10 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Classical methods to control heating systems are often marred by suboptimal
performance, inability to adapt to dynamic conditions and unreasonable
assumptions e.g. existence of building models. This paper presents a novel deep
reinforcement learning algorithm which can control space heating in buildings
in a computationally efficient manner, and benchmarks it against other known
techniques. The proposed algorithm outperforms rule based control by between
5-10% in a simulation environment for a number of price signals. We conclude
that, while not optimal, the proposed algorithm offers additional practical
advantages such as faster computation times and increased robustness to
non-stationarities in building dynamics.
The Evolution of Popularity and Images of Characters in Marvel Cinematic Universe Fanfictions
Authors: Fan Bu
Date: 2018-05-10 00:00:00

This analysis proposes a new topic model to study the yearly trends in Marvel
Cinematic Universe fanfictions on three levels: character popularity, character
images/topics, and vocabulary pattern of topics. It is found that character
appearances in fanfictions have become more diverse over the years thanks to
constant introduction of new characters in feature films, and in the case of
Captain America, multi-dimensional character development is well-received by
the fanfiction world.
Discourse-Aware Neural Rewards for Coherent Text Generation
Authors: Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, Yejin Choi
Date: 2018-05-10 00:00:00
Proceeding: NAACL 2018 6
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Sentence Ordering, Text Generation

In this paper, we investigate the use of discourse-aware rewards with
reinforcement learning to guide a model to generate long, coherent text. In
particular, we propose to learn neural rewards to model cross-sentence ordering
as a means to approximate desired discourse structure. Empirical results
demonstrate that a generator trained with the learned reward produces more
coherent and less repetitive text than models trained with cross-entropy or
with reinforcement learning with commonly used scores as rewards.
Generalizing Bottleneck Problems
Authors: Hsiang Hsu, Shahab Asoodeh, Salman Salamatian, Flavio P. Calmon
Date: 2018-02-16 00:00:00
Tasks: LEMMA

Given a pair of random variables $(X,Y)\sim P_{XY}$ and two convex functions
$f_1$ and $f_2$, we introduce two bottleneck functionals as the lower and upper
boundaries of the two-dimensional convex set that consists of the pairs
$\left(I_{f_1}(W; X), I_{f_2}(W; Y)\right)$, where $I_f$ denotes
$f$-information and $W$ varies over the set of all discrete random variables
satisfying the Markov condition $W \to X \to Y$. Applying Witsenhausen and
Wyner's approach, we provide an algorithm for computing boundaries of this set
for $f_1$, $f_2$, and discrete $P_{XY}$. In the binary symmetric case, we fully
characterize the set when (i) $f_1(t)=f_2(t)=t\log t$, (ii)
$f_1(t)=f_2(t)=t^2-1$, and (iii) $f_1$ and $f_2$ are both $\ell^\beta$ norm
function for $\beta \geq 2$. We then argue that upper and lower boundaries in
(i) correspond to Mrs. Gerber's Lemma and its inverse (which we call Mr.
Gerber's Lemma), in (ii) correspond to estimation-theoretic variants of
Information Bottleneck and Privacy Funnel, and in (iii) correspond to Arimoto
Information Bottleneck and Privacy Funnel.
Expectation Propagation for Approximate Inference: Free Probability Framework
Authors: Burak Çakmak, Manfred Opper
Date: 2018-01-16 00:00:00

We study asymptotic properties of expectation propagation (EP) -- a method
for approximate inference originally developed in the field of machine
learning. Applied to generalized linear models, EP iteratively computes a
multivariate Gaussian approximation to the exact posterior distribution. The
computational complexity of the repeated update of covariance matrices severely
limits the application of EP to large problem sizes. In this study, we present
a rigorous analysis by means of free probability theory that allows us to
overcome this computational bottleneck if specific data matrices in the problem
fulfill certain properties of asymptotic freeness. We demonstrate the relevance
of our approach on the gene selection problem of a microarray dataset.
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD
Authors: Sanghamitra Dutta, Gauri Joshi, Soumyadip Ghosh, Parijat Dube, Priya Nagpurkar
Date: 2018-03-03 00:00:00

Distributed Stochastic Gradient Descent (SGD) when run in a synchronous
manner, suffers from delays in waiting for the slowest learners (stragglers).
Asynchronous methods can alleviate stragglers, but cause gradient staleness
that can adversely affect convergence. In this work we present a novel
theoretical characterization of the speed-up offered by asynchronous methods by
analyzing the trade-off between the error in the trained model and the actual
training runtime (wallclock time). The novelty in our work is that our runtime
analysis considers random straggler delays, which helps us design and compare
distributed SGD algorithms that strike a balance between stragglers and
staleness. We also present a new convergence analysis of asynchronous SGD
variants without bounded or exponential delay assumptions, and a novel learning
rate schedule to compensate for gradient staleness.
Whiteout: Gaussian Adaptive Noise Regularization in Deep Neural Networks
Authors: Yi-Nan Li, Fang Liu
Date: 2016-12-05 00:00:00

Noise injection (NI) is an efficient technique to mitigate over-fitting in
neural networks (NNs). The Bernoulli NI procedure as implemented in dropout and
shakeout has connections with $l_1$ and $l_2$ regularization for the NN model
parameters. We propose whiteout, a family NI regularization techniques (NIRT)
through injecting adaptive Gaussian noises during the training of NNs. Whiteout
is the first NIRT than imposes a broad range of the $l_{\gamma}$ sparsity
regularization $(\gamma\in(0,2))$ without having to involving the $l_2$
regularization. Whiteout can also be extended to offer regularizations similar
to the adaptive lasso and group lasso. We establish the regularization effect
of whiteout in the framework of generalized linear models with closed-form
penalty terms and show that whiteout stabilizes the training of NNs with
decreased sensitivity to small perturbations in the input. We establish that
the noise-perturbed empirical loss function (pelf) with whiteout converges
almost surely to the ideal loss function (ilf), and the minimizer of the pelf
is consistent for the minimizer of the ilf. We derive the tail bound on the
pelf to establish the practical feasibility in its minimization. The
superiority of whiteout over the Bernoulli NIRTs, dropout and shakeout, in
learning NNs with relatively small-sized training sets and non-inferiority in
large-sized training sets is demonstrated in both simulated and real-life data
sets. This work represents the first in-depth theoretical, methodological, and
practical examination of the regularization effects of both additive and
multiplicative Gaussian NI in deep NNs.
Neural Machine Translation Decoding with Terminology Constraints
Authors: Eva Hasler, Adrià De Gispert, Gonzalo Iglesias, Bill Byrne
Date: 2018-05-09 00:00:00
Proceeding: NAACL 2018 6
Tasks: Machine Translation, NMT, Translation

Despite the impressive quality improvements yielded by neural machine
translation (NMT) systems, controlling their translation output to adhere to
user-provided terminology constraints remains an open problem. We describe our
approach to constrained neural decoding based on finite-state machines and
multi-stack decoding which supports target-side constraints as well as
constraints with corresponding aligned input text spans. We demonstrate the
performance of our framework on multiple translation tasks and motivate the
need for constrained decoding with attentions as a means of reducing
misplacement and duplication when translating user constraints.
Riemannian Optimization via Frank-Wolfe Methods
Authors: Melanie Weber, Suvrit Sra
Date: 2017-10-30 00:00:00
Tasks: Riemannian optimization

We study projection-free methods for constrained Riemannian optimization. In particular, we propose the Riemannian Frank-Wolfe (RFW) method. We analyze non-asymptotic convergence rates of RFW to an optimum for (geodesically) convex problems, and to a critical point for nonconvex objectives. We also present a practical setting under which RFW can attain a linear convergence rate. As a concrete example, we specialize RFW to the manifold of positive definite matrices and apply it to two tasks: (i) computing the matrix geometric mean (Riemannian centroid); and (ii) computing the Bures-Wasserstein barycenter. Both tasks involve geodesically convex interval constraints, for which we show that the Riemannian "linear" oracle required by RFW admits a closed-form solution; this result may be of independent interest. We further specialize RFW to the special orthogonal group and show that here too, the Riemannian "linear" oracle can be solved in closed form. Here, we describe an application to the synchronization of data matrices (Procrustes problem). We complement our theoretical results with an empirical comparison of RFW against state-of-the-art Riemannian optimization methods and observe that RFW performs competitively on the task of computing Riemannian centroids.
Graph Neural Networks for Learning Robot Team Coordination
Authors: Amanda Prorok
Date: 2018-05-09 00:00:00

This paper shows how Graph Neural Networks can be used for learning
distributed coordination mechanisms in connected teams of robots. We capture
the relational aspect of robot coordination by modeling the robot team as a
graph, where each robot is a node, and edges represent communication links.
During training, robots learn how to pass messages and update internal states,
so that a target behavior is reached. As a proxy for more complex problems,
this short paper considers the problem where each robot must locally estimate
the algebraic connectivity of the team's network topology.
Most Ligand-Based Classification Benchmarks Reward Memorization Rather than Generalization
Authors: Izhar Wallach, Abraham Heifets
Date: 2017-06-20 00:00:00
Tasks: Classification, General Classification, Memorization

Undetected overfitting can occur when there are significant redundancies
between training and validation data. We describe AVE, a new measure of
training-validation redundancy for ligand-based classification problems that
accounts for the similarity amongst inactive molecules as well as active. We
investigated seven widely-used benchmarks for virtual screening and
classification, and show that the amount of AVE bias strongly correlates with
the performance of ligand-based predictive methods irrespective of the
predicted property, chemical fingerprint, similarity measure, or
previously-applied unbiasing techniques. Therefore, it may be that the
previously-reported performance of most ligand-based methods can be explained
by overfitting to benchmarks rather than good prospective accuracy.
Variational Inference: A Review for Statisticians
Authors: David M. Blei, Alp Kucukelbir, Jon D. McAuliffe
Date: 2016-01-04 00:00:00
Tasks: Stochastic Optimization, Variational Inference

One of the core problems of modern statistics is to approximate
difficult-to-compute probability densities. This problem is especially
important in Bayesian statistics, which frames all inference about unknown
quantities as a calculation involving the posterior density. In this paper, we
review variational inference (VI), a method from machine learning that
approximates probability densities through optimization. VI has been used in
many applications and tends to be faster than classical methods, such as Markov
chain Monte Carlo sampling. The idea behind VI is to first posit a family of
densities and then to find the member of that family which is close to the
target. Closeness is measured by Kullback-Leibler divergence. We review the
ideas behind mean-field variational inference, discuss the special case of VI
applied to exponential family models, present a full example with a Bayesian
mixture of Gaussians, and derive a variant that uses stochastic optimization to
scale up to massive data. We discuss modern research in VI and highlight
important open problems. VI is powerful, but it is not yet well understood. Our
hope in writing this paper is to catalyze statistical research on this class of
algorithms.
Evaluating ResNeXt Model Architecture for Image Classification
Authors: Saifuddin Hitawala
Date: 2018-05-09 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, model

In recent years, deep learning methods have been successfully applied to
image classification tasks. Many such deep neural networks exist today that can
easily differentiate cats from dogs. One such model is the ResNeXt model that
uses a homogeneous, multi-branch architecture for image classification. This
paper aims at implementing and evaluating the ResNeXt model architecture on
subsets of the CIFAR-10 dataset. It also tweaks the original ResNeXt
hyper-parameters such as cardinality, depth and base-width and compares the
performance of the modified model with the original. Analysis of the
experiments performed in this paper show that a slight decrease in depth or
base-width does not affect the performance of the model much leading to
comparable results.