pwc-archive/papers-with-abstracts huggingface dataset:
Bootstrapping Method for Developing Part-of-Speech Tagged Corpus in Low Resource Languages Tagset - A Focus on an African Igbo
Authors: Onyenwe Ikechukwu E, Onyedinma Ebele G, Aniegwu Godwin E, Ezeani Ignatius M
Date: 2019-03-12 00:00:00
Tasks: Machine Translation, POS, POS Tagging, speech-recognition, Speech Recognition, Word Alignment

Most languages, especially in Africa, have fewer or no established
part-of-speech (POS) tagged corpus. However, POS tagged corpus is essential for
natural language processing (NLP) to support advanced researches such as
machine translation, speech recognition, etc. Even in cases where there is no
POS tagged corpus, there are some languages for which parallel texts are
available online. The task of POS tagging a new language corpus with a new
tagset usually face a bootstrapping problem at the initial stages of the
annotation process. The unavailability of automatic taggers to help the human
annotator makes the annotation process to appear infeasible to quickly produce
adequate amounts of POS tagged corpus for advanced NLP research and training
the taggers. In this paper, we demonstrate the efficacy of a POS annotation
method that employed the services of two automatic approaches to assist POS
tagged corpus creation for a novel language in NLP. The two approaches are
cross-lingual and monolingual POS tags projection. We used cross-lingual to
automatically create an initial 'errorful' tagged corpus for a target language
via word-alignment. The resources for creating this are derived from a source
language rich in NLP resources. A monolingual method is applied to clean the
induce noise via an alignment process and to transform the source language tags
to the target language tags. We used English and Igbo as our case study. This
is possible because there are parallel texts that exist between English and
Igbo, and the source language English has available NLP resources. The results
of the experiment show a steady improvement in accuracy and rate of tags
transformation with score ranges of 6.13% to 83.79% and 8.67% to 98.37%
respectively. The rate of tags transformation evaluates the rate at which
source language tags are translated to target language tags.
Syntax-aware Neural Semantic Role Labeling with Supertags
Authors: Jungo Kasai, Dan Friedman, Robert Frank, Dragomir Radev, Owen Rambow
Date: 2019-03-12 00:00:00
Proceeding: NAACL 2019 6
Tasks: Semantic Role Labeling, TAG

We introduce a new syntax-aware model for dependency-based semantic role
labeling that outperforms syntax-agnostic models for English and Spanish. We
use a BiLSTM to tag the text with supertags extracted from dependency parses,
and we feed these supertags, along with words and parts of speech, into a deep
highway BiLSTM for semantic role labeling. Our model combines the strengths of
earlier models that performed SRL on the basis of a full dependency parse with
more recent models that use no syntactic information at all. Our local and
non-ensemble model achieves state-of-the-art performance on the CoNLL 09
English and Spanish datasets. SRL models benefit from syntactic information,
and we show that supertagging is a simple, powerful, and robust way to
incorporate syntax into a neural SRL system.
End-To-End Speech Recognition Using A High Rank LSTM-CTC Based Model
Authors: Yangyang Shi, Mei-Yuh Hwang, Xin Lei
Date: 2019-03-12 00:00:00
Tasks: Data Augmentation, speech-recognition, Speech Recognition

Long Short Term Memory Connectionist Temporal Classification (LSTM-CTC) based
end-to-end models are widely used in speech recognition due to its simplicity
in training and efficiency in decoding. In conventional LSTM-CTC based models,
a bottleneck projection matrix maps the hidden feature vectors obtained from
LSTM to softmax output layer. In this paper, we propose to use a high rank
projection layer to replace the projection matrix. The output from the high
rank projection layer is a weighted combination of vectors that are projected
from the hidden feature vectors via different projection matrices and
non-linear activation function. The high rank projection layer is able to
improve the expressiveness of LSTM-CTC models. The experimental results show
that on Wall Street Journal (WSJ) corpus and LibriSpeech data set, the proposed
method achieves 4%-6% relative word error rate (WER) reduction over the
baseline CTC system. They outperform other published CTC based end-to-end (E2E)
models under the condition that no external data or data augmentation is
applied. Code has been made available at https://github.com/mobvoi/lstm_ctc.
Sub-event detection from Twitter streams as a sequence labeling problem
Authors: Giannis Bekoulis, Johannes Deleu, Thomas Demeester, Chris Develder
Date: 2019-03-13 00:00:00
Proceeding: NAACL 2019 6
Tasks: Event Detection

This paper introduces improved methods for sub-event detection in social
media streams, by applying neural sequence models not only on the level of
individual posts, but also directly on the stream level. Current approaches to
identify sub-events within a given event, such as a goal during a soccer match,
essentially do not exploit the sequential nature of social media streams. We
address this shortcoming by framing the sub-event detection problem in social
media streams as a sequence labeling task and adopt a neural sequence
architecture that explicitly accounts for the chronological order of posts.
Specifically, we (i) establish a neural baseline that outperforms a graph-based
state-of-the-art method for binary sub-event detection (2.7% micro-F1
improvement), as well as (ii) demonstrate superiority of a recurrent neural
network model on the posts sequence level for labeled sub-events (2.4%
bin-level F1 improvement over non-sequential models).
Market Trend Prediction using Sentiment Analysis: Lessons Learned and Paths Forward
Authors: Andrius Mudinas, Dell Zhang, Mark Levene
Date: 2019-03-13 00:00:00
Tasks: Sentiment Analysis

Financial market forecasting is one of the most attractive practical
applications of sentiment analysis. In this paper, we investigate the potential
of using sentiment \emph{attitudes} (positive vs negative) and also sentiment
\emph{emotions} (joy, sadness, etc.) extracted from financial news or tweets to
help predict stock price movements. Our extensive experiments using the
\emph{Granger-causality} test have revealed that (i) in general sentiment
attitudes do not seem to Granger-cause stock price changes; and (ii) while on
some specific occasions sentiment emotions do seem to Granger-cause stock price
changes, the exhibited pattern is not universal and must be looked at on a case
by case basis. Furthermore, it has been observed that at least for certain
stocks, integrating sentiment emotions as additional features into the machine
learning based market trend prediction model could improve its accuracy.
Overview of the Ugglan Entity Discovery and Linking System
Authors: Marcus Klang, Firas Dib, Pierre Nugues
Date: 2019-03-13 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Ugglan is a system designed to discover named entities and link them to
unique identifiers in a knowledge base. It is based on a combination of a name
and nominal dictionary derived from Wikipedia and Wikidata, a named entity
recognition module (NER) using fixed ordinally-forgetting encoding (FOFE)
trained on the TAC EDL data from 2014-2016, a candidate generation module from
the Wikipedia link graph across multiple editions, a PageRank link and
cooccurrence graph disambiguator, and finally a reranker trained on the TAC EDL
2015-2016 data.
Adversarial attacks against Fact Extraction and VERification
Authors: James Thorne, Andreas Vlachos
Date: 2019-03-13 00:00:00
Tasks: Fact Checking, Information Retrieval, Natural Language Inference, Retrieval

This paper describes a baseline for the second iteration of the Fact
Extraction and VERification shared task (FEVER2.0) which explores the
resilience of systems through adversarial evaluation. We present a collection
of simple adversarial attacks against systems that participated in the first
FEVER shared task. FEVER modeled the assessment of truthfulness of written
claims as a joint information retrieval and natural language inference task
using evidence from Wikipedia. A large number of participants made use of deep
neural networks in their submissions to the shared task. The extent as to
whether such models understand language has been the subject of a number of
recent investigations and discussion in literature. In this paper, we present a
simple method of generating entailment-preserving and entailment-altering
perturbations of instances by common patterns within the training data. We find
that a number of systems are greatly affected with absolute losses in
classification accuracy of up to $29\%$ on the newly perturbed instances. Using
these newly generated instances, we construct a sample submission for the
FEVER2.0 shared task. Addressing these types of attacks will aid in building
more robust fact-checking models, as well as suggest directions to expand the
datasets.
MMKG: Multi-Modal Knowledge Graphs
Authors: Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, David S. Rosenblum
Date: 2019-03-13 00:00:00
Tasks: Knowledge Graphs, Link Prediction, Prediction

We present MMKG, a collection of three knowledge graphs that contain both
numerical features and (links to) images for all entities as well as entity
alignments between pairs of KGs. Therefore, multi-relational link prediction
and entity matching communities can benefit from this resource. We believe this
data set has the potential to facilitate the development of novel multi-modal
learning approaches for knowledge graphs.We validate the utility ofMMKG in the
sameAs link prediction task with an extensive set of experiments. These
experiments show that the task at hand benefits from learning of multiple
feature types.
Hypothesis-based Belief Planning for Dexterous Grasping
Authors: Claudio Zito, Valerio Ortenzi, Maxime Adjigble, Marek Kopicki, Rustam Stolkin, Jeremy L. Wyatt
Date: 2019-03-13 00:00:00
Tasks: Object, Robot Manipulation

Belief space planning is a viable alternative to formalise partially
observable control problems and, in the recent years, its application to robot
manipulation problems has grown. However, this planning approach was tried
successfully only on simplified control problems. In this paper, we apply
belief space planning to the problem of planning dexterous reach-to-grasp
trajectories under object pose uncertainty. In our framework, the robot
perceives the object to be grasped on-the-fly as a point cloud and compute a
full 6D, non-Gaussian distribution over the object's pose (our belief space).
The system has no limitations on the geometry of the object, i.e., non-convex
objects can be represented, nor assumes that the point cloud is a complete
representation of the object. A plan in the belief space is then created to
reach and grasp the object, such that the information value of expected
contacts along the trajectory is maximised to compensate for the pose
uncertainty. If an unexpected contact occurs when performing the action, such
information is used to refine the pose distribution and triggers a re-planning.
Experimental results show that our planner (IR3ne) improves grasp reliability
and compensates for the pose uncertainty such that it doubles the proportion of
grasps that succeed on a first attempt.
Semantic Structure-based Unsupervised Deep Hashing
Authors: Erkun Yang, Cheng Deng, Tongliang Liu, Wei Liu, DaCheng Tao
Date: 2018-05-01 00:00:00
Proceeding: IJCAI2018 2018 5
Tasks: Deep Hashing, Semantic Similarity, Semantic Textual Similarity

Hashing is becoming increasingly popular for approximate nearest neighbor searching in massive databases due to its storage and search efficiency. Recent supervised hashing methods, which usually construct semantic similarity matrices to guide hash code learning using label information, have shown promising results. However, it is relatively difficult to capture and utilize the semantic relationships between points in unsupervised settings. To address this problem, we propose a novel unsupervised deep framework called Semantic Structure-based unsupervised Deep Hashing (SSDH). We first empirically study the deep feature statistics, and find that the distribution of the cosine distance for point pairs can be estimated by two half Gaussian distributions. Based on this observation, we construct the semantic structure by considering points with distances obviously smaller than the others as semantically similar and points with distances obviously larger than the others as semantically dissimilar. We then design a deep architecture and a pair-wise loss function to preserve this semantic structure in Hamming space. Extensive experiments show that SSDH significantly outperforms current state-of-the-art methods.
Learning compact binary descriptors with unsupervised deep neural networks
Authors: Kevin Lin; Jiwen Lu; Chu-Song Chen; Jie Zhou
Date: 2016-06-01 00:00:00
Proceeding: CVPR2016 2016 6
Tasks: Image Retrieval, Object, Object Recognition, Quantization, Retrieval

In this paper, we propose a new unsupervised deep learning approach called DeepBit to learn compact binary descriptor for efficient visual object matching. Unlike most existing binary descriptors which were designed with random projections or linear hash functions, we develop a deep neural network to learn binary descriptors in an unsupervised manner. We enforce three criterions on binary codes which are learned at the top layer of our network: 1) minimal loss quantization, 2) evenly distributed codes and 3) uncorrelated bits. Then, we learn the parameters of the networks with a back-propagation technique. Experimental results on three different visual analysis tasks including image matching, image retrieval, and object recognition clearly demonstrate the effectiveness of the proposed approach.
Learning pseudo-Boolean k-DNF and Submodular Functions
Authors: Sofya Raskhodnikova, Grigory Yaroslavtsev
Date: 2012-08-10 00:00:00
Tasks: 2k, LEMMA, PAC learning

We prove that any submodular function f: {0,1}^n -> {0,1,...,k} can be
represented as a pseudo-Boolean 2k-DNF formula. Pseudo-Boolean DNFs are a
natural generalization of DNF representation for functions with integer range.
Each term in such a formula has an associated integral constant. We show that
an analog of Hastad's switching lemma holds for pseudo-Boolean k-DNFs if all
constants associated with the terms of the formula are bounded.
  This allows us to generalize Mansour's PAC-learning algorithm for k-DNFs to
pseudo-Boolean k-DNFs, and hence gives a PAC-learning algorithm with membership
queries under the uniform distribution for submodular functions of the form
f:{0,1}^n -> {0,1,...,k}. Our algorithm runs in time polynomial in n, k^{O(k
\log k / \epsilon)}, 1/\epsilon and log(1/\delta) and works even in the
agnostic setting. The line of previous work on learning submodular functions
[Balcan, Harvey (STOC '11), Gupta, Hardt, Roth, Ullman (STOC '11), Cheraghchi,
Klivans, Kothari, Lee (SODA '12)] implies only n^{O(k)} query complexity for
learning submodular functions in this setting, for fixed epsilon and delta.
  Our learning algorithm implies a property tester for submodularity of
functions f:{0,1}^n -> {0, ..., k} with query complexity polynomial in n for
k=O((\log n/ \loglog n)^{1/2}) and constant proximity parameter \epsilon.
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
Authors: Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, Jack Xin
Date: 2019-03-13 00:00:00
Proceeding: ICLR 2019 5
Tasks: Negation

Training activation quantized neural networks involves minimizing a piecewise constant function whose gradient vanishes almost everywhere, which is undesirable for the standard back-propagation or chain rule. An empirical way around this issue is to use a straight-through estimator (STE) (Bengio et al., 2013) in the backward pass only, so that the "gradient" through the modified chain rule becomes non-trivial. Since this unusual "gradient" is certainly not the gradient of loss function, the following question arises: why searching in its negative direction minimizes the training loss? In this paper, we provide the theoretical justification of the concept of STE by answering this question. We consider the problem of learning a two-linear-layer network with binarized ReLU activation and Gaussian input data. We shall refer to the unusual "gradient" given by the STE-modifed chain rule as coarse gradient. The choice of STE is not unique. We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss. We further show the associated coarse gradient descent algorithm converges to a critical point of the population loss minimization problem. Moreover, we show that a poor choice of STE leads to instability of the training algorithm near certain local minima, which is verified with CIFAR-10 experiments.
Fuzzy Rough Set Feature Selection to Enhance Phishing Attack Detection
Authors: Mahdieh Zabihimayvan, Derek Doran
Date: 2019-03-13 00:00:00
Tasks: feature selection

Phishing as one of the most well-known cybercrime activities is a deception
of online users to steal their personal or confidential information by
impersonating a legitimate website. Several machine learning-based strategies
have been proposed to detect phishing websites. These techniques are dependent
on the features extracted from the website samples. However, few studies have
actually considered efficient feature selection for detecting phishing attacks.
In this work, we investigate an agreement on the definitive features which
should be used in phishing detection. We apply Fuzzy Rough Set (FRS) theory as
a tool to select most effective features from three benchmarked data sets. The
selected features are fed into three often used classifiers for phishing
detection. To evaluate the FRS feature selection in developing a generalizable
phishing detection, the classifiers are trained by a separate out-of-sample
data set of 14,000 website samples. The maximum F-measure gained by FRS feature
selection is 95% using Random Forest classification. Also, there are 9
universal features selected by FRS over all the three data sets. The F-measure
value using this universal feature set is approximately 93% which is a
comparable result in contrast to the FRS performance. Since the universal
feature set contains no features from third-part services, this finding implies
that with no inquiry from external sources, we can gain a faster phishing
detection which is also robust toward zero-day attacks.
Improving Prostate Cancer Detection with Breast Histopathology Images
Authors: Umair Akhtar Hasan Khan, Carolin Stürenberg, Oguzhan Gencoglu, Kevin Sandeman, Timo Heikkinen, Antti Rannikko, Tuomas Mirtti
Date: 2019-03-14 00:00:00
Tasks: General Classification, Transfer Learning, whole slide images

Deep neural networks have introduced significant advancements in the field of
machine learning-based analysis of digital pathology images including prostate
tissue images. With the help of transfer learning, classification and
segmentation performance of neural network models have been further increased.
However, due to the absence of large, extensively annotated, publicly available
prostate histopathology datasets, several previous studies employ datasets from
well-studied computer vision tasks such as ImageNet dataset. In this work, we
propose a transfer learning scheme from breast histopathology images to improve
prostate cancer detection performance. We validate our approach on annotated
prostate whole slide images by using a publicly available breast histopathology
dataset as pre-training. We show that the proposed cross-cancer approach
outperforms transfer learning from ImageNet dataset.
Functional Variational Bayesian Neural Networks
Authors: Shengyang Sun, Guodong Zhang, Jiaxin Shi, Roger Grosse
Date: 2019-03-14 00:00:00
Proceeding: ICLR 2019 5
Tasks: Bayesian Inference, Gaussian Processes, Variational Inference

Variational Bayesian neural networks (BNNs) perform variational inference
over weights, but it is difficult to specify meaningful priors and approximate
posteriors in a high-dimensional weight space. We introduce functional
variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower
BOund (ELBO) defined directly on stochastic processes, i.e. distributions over
functions. We prove that the KL divergence between stochastic processes equals
the supremum of marginal KL divergences over all finite sets of inputs. Based
on this, we introduce a practical training objective which approximates the
functional ELBO using finite measurement sets and the spectral Stein gradient
estimator. With fBNNs, we can specify priors entailing rich structures,
including Gaussian processes and implicit stochastic processes. Empirically, we
find fBNNs extrapolate well using various structured priors, provide reliable
uncertainty estimates, and scale to large datasets.
Diagnosing and Enhancing VAE Models
Authors: Bin Dai, David Wipf
Date: 2019-03-14 00:00:00
Proceeding: ICLR 2019 5
Tasks: Decoder

Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In this regard, we rigorously analyze the VAE objective, differentiating situations where this belief is and is not actually true. We then leverage the corresponding insights to develop a simple VAE enhancement that requires no additional hyperparameters or sensitive tuning. Quantitatively, this proposal produces crisp samples and stable FID scores that are actually competitive with a variety of GAN models, all while retaining desirable attributes of the original VAE architecture. A shorter version of this work will appear in the ICLR 2019 conference proceedings (Dai and Wipf, 2019). The code for our model is available at https://github.com/daib13/ TwoStageVAE.
On Applications of Bootstrap in Continuous Space Reinforcement Learning
Authors: Mohamad Kazem Shirani Faradonbeh, Ambuj Tewari, George Michailidis
Date: 2019-03-14 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In decision making problems for continuous state and action spaces, linear
dynamical models are widely employed. Specifically, policies for stochastic
linear systems subject to quadratic cost functions capture a large number of
applications in reinforcement learning. Selected randomized policies have been
studied in the literature recently that address the trade-off between
identification and control. However, little is known about policies based on
bootstrapping observed states and actions. In this work, we show that
bootstrap-based policies achieve a square root scaling of regret with respect
to time. We also obtain results on the accuracy of learning the model's
dynamics. Corroborative numerical analysis that illustrates the technical
results is also provided.