pwc-archive/papers-with-abstracts huggingface dataset:
From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing
Authors: Juan Andrés Laura, Gabriel Masi, Luis Argerich
Date: 2017-05-01 00:00:00
Tasks: Data Compression

In recent studies [1][13][12] Recurrent Neural Networks were used for
generative processes and their surprising performance can be explained by their
ability to create good predictions. In addition, data compression is also based
on predictions. What the problem comes down to is whether a data compressor
could be used to perform as well as recurrent neural networks in natural
language processing tasks. If this is possible,then the problem comes down to
determining if a compression algorithm is even more intelligent than a neural
network in specific tasks related to human language. In our journey we
discovered what we think is the fundamental difference between a Data
Compression Algorithm and a Recurrent Neural Network.
Convex-constrained Sparse Additive Modeling and Its Extensions
Authors: Junming Yin, Yao-Liang Yu
Date: 2017-05-01 00:00:00
Tasks: Additive models, regression

Sparse additive modeling is a class of effective methods for performing
high-dimensional nonparametric regression. In this work we show how shape
constraints such as convexity/concavity and their extensions, can be integrated
into additive models. The proposed sparse difference of convex additive models
(SDCAM) can estimate most continuous functions without any a priori smoothness
assumption. Motivated by a characterization of difference of convex functions,
our method incorporates a natural regularization functional to avoid
overfitting and to reduce model complexity. Computationally, we develop an
efficient backfitting algorithm with linear per-iteration complexity.
Experiments on both synthetic and real data verify that our method is
competitive against state-of-the-art sparse additive models, with improved
performance in most scenarios.
Efficient Natural Language Response Suggestion for Smart Reply
Authors: Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-Hsuan Sung, Laszlo Lukacs, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, Ray Kurzweil
Date: 2017-05-01 00:00:00

This paper presents a computationally efficient machine-learned method for
natural language response suggestion. Feed-forward neural networks using n-gram
embedding features encode messages into vectors which are optimized to give
message-response pairs a high dot-product value. An optimized search finds
response suggestions. The method is evaluated in a large-scale commercial
e-mail application, Inbox by Gmail. Compared to a sequence-to-sequence
approach, the new system achieves the same quality at a small fraction of the
computational requirements and latency.
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Authors: William Yang Wang
Date: 2017-05-01 00:00:00
Proceeding: ACL 2017 7
Tasks: Deception Detection, Fact Checking, Fake News Detection

Automatic fake news detection is a challenging problem in deception
detection, and it has tremendous real-world political and social impacts.
However, statistical approaches to combating fake news has been dramatically
limited by the lack of labeled benchmark datasets. In this paper, we present
liar: a new, publicly available dataset for fake news detection. We collected a
decade-long, 12.8K manually labeled short statements in various contexts from
PolitiFact.com, which provides detailed analysis report and links to source
documents for each case. This dataset can be used for fact-checking research as
well. Notably, this new dataset is an order of magnitude larger than previously
largest public fake news datasets of similar type. Empirically, we investigate
automatic fake news detection based on surface-level linguistic patterns. We
have designed a novel, hybrid convolutional neural network to integrate
meta-data with text. We show that this hybrid approach can improve a text-only
deep learning model.
Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation
Authors: Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, WangMeng Zuo
Date: 2017-05-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

In domain adaptation, maximum mean discrepancy (MMD) has been widely adopted
as a discrepancy metric between the distributions of source and target domains.
However, existing MMD-based domain adaptation methods generally ignore the
changes of class prior distributions, i.e., class weight bias across domains.
This remains an open problem but ubiquitous for domain adaptation, which can be
caused by changes in sample selection criteria and application scenarios. We
show that MMD cannot account for class weight bias and results in degraded
domain adaptation performance. To address this issue, a weighted MMD model is
proposed in this paper. Specifically, we introduce class-specific auxiliary
weights into the original MMD for exploiting the class prior probability on
source and target domains, whose challenge lies in the fact that the class
label in target domain is unavailable. To account for it, our proposed weighted
MMD model is defined by introducing an auxiliary weight for each class in the
source domain, and a classification EM algorithm is suggested by alternating
between assigning the pseudo-labels, estimating auxiliary weights and updating
model parameters. Extensive experiments demonstrate the superiority of our
weighted MMD over conventional MMD for domain adaptation.
Spotting the Difference: Context Retrieval and Analysis for Improved Forgery Detection and Localization
Authors: Joel Brogan, Paolo Bestagini, Aparna Bharati, Allan Pinto, Daniel Moreira, Kevin Bowyer, Patrick Flynn, Anderson Rocha, Walter Scheirer
Date: 2017-05-01 00:00:00
Tasks: Image Forensics, Image Retrieval, Retrieval

As image tampering becomes ever more sophisticated and commonplace, the need
for image forensics algorithms that can accurately and quickly detect forgeries
grows. In this paper, we revisit the ideas of image querying and retrieval to
provide clues to better localize forgeries. We propose a method to perform
large-scale image forensics on the order of one million images using the help
of an image search algorithm and database to gather contextual clues as to
where tampering may have taken place. In this vein, we introduce five new
strongly invariant image comparison methods and test their effectiveness under
heavy noise, rotation, and color space changes. Lastly, we show the
effectiveness of these methods compared to passive image forensics using Nimble
[https://www.nist.gov/itl/iad/mig/nimble-challenge], a new, state-of-the-art
dataset from the National Institute of Standards and Technology (NIST).
SANet: Structure-Aware Network for Visual Tracking
Authors: Heng Fan, Haibin Ling
Date: 2016-11-21 00:00:00
Tasks: General Classification, Object, Visual Tracking

Convolutional neural network (CNN) has drawn increasing interest in visual
tracking owing to its powerfulness in feature extraction. Most existing
CNN-based trackers treat tracking as a classification problem. However, these
trackers are sensitive to similar distractors because their CNN models mainly
focus on inter-class classification. To address this problem, we use
self-structure information of object to distinguish it from distractors.
Specifically, we utilize recurrent neural network (RNN) to model object
structure, and incorporate it into CNN to improve its robustness to similar
distractors. Considering that convolutional layers in different levels
characterize the object from different perspectives, we use multiple RNNs to
model object structure in different levels respectively. Extensive experiments
on three benchmarks, OTB100, TC-128 and VOT2015, show that the proposed
algorithm outperforms other methods. Code is released at
http://www.dabi.temple.edu/~hbling/code/SANet/SANet.html.
Towards well-specified semi-supervised model-based classifiers via structural adaptation
Authors: Zhaocai Sun, William K. Cheung, Xiaofeng Zhang, Jun Yang
Date: 2017-05-01 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, image-classification, Image Classification

Semi-supervised learning plays an important role in large-scale machine
learning. Properly using additional unlabeled data (largely available nowadays)
often can improve the machine learning accuracy. However, if the machine
learning model is misspecified for the underlying true data distribution, the
model performance could be seriously jeopardized. This issue is known as model
misspecification. To address this issue, we focus on generative models and
propose a criterion to detect the onset of model misspecification by measuring
the performance difference between models obtained using supervised and
semi-supervised learning. Then, we propose to automatically modify the
generative models during model training to achieve an unbiased generative
model. Rigorous experiments were carried out to evaluate the proposed method
using two image classification data sets PASCAL VOC'07 and MIR Flickr. Our
proposed method has been demonstrated to outperform a number of
state-of-the-art semi-supervised learning approaches for the classification
task.
Forced to Learn: Discovering Disentangled Representations Without Exhaustive Labels
Authors: Alexey Romanov, Anna Rumshisky
Date: 2017-05-01 00:00:00
Tasks: Clustering

Learning a better representation with neural networks is a challenging
problem, which was tackled extensively from different prospectives in the past
few years. In this work, we focus on learning a representation that could be
used for a clustering task and introduce two novel loss components that
substantially improve the quality of produced clusters, are simple to apply to
an arbitrary model and cost function, and do not require a complicated training
procedure. We evaluate them on two most common types of models, Recurrent
Neural Networks and Convolutional Neural Networks, showing that the approach we
propose consistently improves the quality of KMeans clustering in terms of
Adjusted Mutual Information score and outperforms previously proposed methods.
Lancaster A at SemEval-2017 Task 5: Evaluation metrics matter: predicting sentiment from financial news headlines
Authors: Andrew Moore, Paul Rayson
Date: 2017-05-01 00:00:00
Proceeding: SEMEVAL 2017 8
Tasks: regression

This paper describes our participation in Task 5 track 2 of SemEval 2017 to
predict the sentiment of financial news headlines for a specific company on a
continuous scale between -1 and 1. We tackled the problem using a number of
approaches, utilising a Support Vector Regression (SVR) and a Bidirectional
Long Short-Term Memory (BLSTM). We found an improvement of 4-6% using the LSTM
model over the SVR and came fourth in the track. We report a number of
different evaluations using a finance specific word embedding model and reflect
on the effects of using different evaluation metrics.
Optimal dictionary for least squares representation
Authors: Mohammed Rayyan Sheriff, Debasish Chatterjee
Date: 2016-03-07 00:00:00

Dictionaries are collections of vectors used for representations of random
vectors in Euclidean spaces. Recent research on optimal dictionaries is focused
on constructing dictionaries that offer sparse representations, i.e.,
$\ell_0$-optimal representations. Here we consider the problem of finding
optimal dictionaries with which representations of samples of a random vector
are optimal in an $\ell_2$-sense: optimality of representation is defined as
attaining the minimal average $\ell_2$-norm of the coefficients used to
represent the random vector. With the help of recent results on rank-$1$
decompositions of symmetric positive semidefinite matrices, we provide an
explicit description of $\ell_2$-optimal dictionaries as well as their
algorithmic constructions in polynomial time.
WebAPIRec: Recommending Web APIs to Software Projects via Personalized Ranking
Authors: Ferdian Thung, Richard J. Oentaryo, David Lo, Yuan Tian
Date: 2017-05-01 00:00:00

Application programming interfaces (APIs) offer a plethora of functionalities
for developers to reuse without reinventing the wheel. Identifying the
appropriate APIs given a project requirement is critical for the success of a
project, as many functionalities can be reused to achieve faster development.
However, the massive number of APIs would often hinder the developers' ability
to quickly find the right APIs. In this light, we propose a new, automated
approach called WebAPIRec that takes as input a project profile and outputs a
ranked list of {web} APIs that can be used to implement the project. At its
heart, WebAPIRec employs a personalized ranking model that ranks web APIs
specific (personalized) to a project. Based on the historical data of {web} API
usages, WebAPIRec learns a model that minimizes the incorrect ordering of web
APIs, i.e., when a used {web} API is ranked lower than an unused (or a
not-yet-used) web API. We have evaluated our approach on a dataset comprising
9,883 web APIs and 4,315 web application projects from ProgrammableWeb with
promising results. For 84.0% of the projects, WebAPIRec is able to successfully
return correct APIs that are used to implement the projects in the top-5
positions. This is substantially better than the recommendations provided by
ProgrammableWeb's native search functionality. WebAPIRec also outperforms
McMillan et al.'s application search engine and popularity-based
recommendation.
Nonsymbolic Text Representation
Authors: Hinrich Schuetze, Heike Adel, Ehsaneddin Asgari
Date: 2016-10-03 00:00:00
Tasks: Denoising

We introduce the first generic text representation model that is completely
nonsymbolic, i.e., it does not require the availability of a segmentation or
tokenization method that attempts to identify words or other symbolic units in
text. This applies to training the parameters of the model on a training corpus
as well as to applying it when computing the representation of a new text. We
show that our model performs better than prior work on an information
extraction and a text denoising task.
Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks
Authors: Rajarshi Das, Arvind Neelakantan, David Belanger, Andrew McCallum
Date: 2016-07-05 00:00:00
Proceeding: EACL 2017 4
Tasks: Logical Reasoning

Our goal is to combine the rich multistep inference of symbolic logical
reasoning with the generalization capabilities of neural networks. We are
particularly interested in complex reasoning about entities and relations in
text and large-scale knowledge bases (KBs). Neelakantan et al. (2015) use RNNs
to compose the distributed semantics of multi-hop paths in KBs; however for
multiple reasons, the approach lacks accuracy and practicality. This paper
proposes three significant modeling advances: (1) we learn to jointly reason
about relations, entities, and entity-types; (2) we use neural attention
modeling to incorporate multiple paths; (3) we learn to share strength in a
single RNN that represents logical composition across all relations. On a
largescale Freebase+ClueWeb prediction task, we achieve 25% error reduction,
and a 53% error reduction on sparse relations due to shared strength. On chains
of reasoning in WordNet we reduce error in mean quantile by 84% versus previous
state-of-the-art. The code and data are available at
https://rajarshd.github.io/ChainsofReasoning
Error bounds for approximations with deep ReLU networks
Authors: Dmitry Yarotsky
Date: 2016-10-03 00:00:00

We study expressive power of shallow and deep neural networks with piece-wise
linear activation functions. We establish new rigorous upper and lower bounds
for the network complexity in the setting of approximations in Sobolev spaces.
In particular, we prove that deep ReLU networks more efficiently approximate
smooth functions than shallow networks. In the case of approximations of 1D
Lipschitz functions we describe adaptive depth-6 network architectures more
efficient than the standard shallow architecture.
Regularized Residual Quantization: a multi-layer sparse dictionary learning approach
Authors: Sohrab Ferdowsi, Slava Voloshynovskiy, Dimche Kostadinov
Date: 2017-05-01 00:00:00
Tasks: Dictionary Learning, Quantization, Super-Resolution

The Residual Quantization (RQ) framework is revisited where the quantization
distortion is being successively reduced in multi-layers. Inspired by the
reverse-water-filling paradigm in rate-distortion theory, an efficient
regularization on the variances of the codewords is introduced which allows to
extend the RQ for very large numbers of layers and also for high dimensional
data, without getting over-trained. The proposed Regularized Residual
Quantization (RRQ) results in multi-layer dictionaries which are additionally
sparse, thanks to the soft-thresholding nature of the regularization when
applied to variance-decaying data which can arise from de-correlating
transformations applied to correlated data. Furthermore, we also propose a
general-purpose pre-processing for natural images which makes them suitable for
such quantization. The RRQ framework is first tested on synthetic
variance-decaying data to show its efficiency in quantization of
high-dimensional data. Next, we use the RRQ in super-resolution of a database
of facial images where it is shown that low-resolution facial images from the
test set quantized with codebooks trained on high-resolution images from the
training set show relevant high-frequency content when reconstructed with those
codebooks.
Least square ellipsoid fitting using iterative orthogonal transformations
Authors: Amit Reza, Anand S. Sengupta
Date: 2017-04-17 00:00:00
Tasks: Retrieval

We describe a generalised method for ellipsoid fitting against a minimum set
of data points. The proposed method is numerically stable and applies to a wide
range of ellipsoidal shapes, including highly elongated and arbitrarily
oriented ellipsoids. This new method also provides for the retrieval of
rotational angle and length of semi-axes of the fitted ellipsoids accurately.
We demonstrate the efficacy of this algorithm on simulated data sets and also
indicate its potential use in gravitational wave data analysis.
MORSE: Semantic-ally Drive-n MORpheme SEgment-er
Authors: Tarek Sakakini, Suma Bhat, Pramod Viswanath
Date: 2017-02-07 00:00:00
Proceeding: ACL 2017 7
Tasks: Benchmarking

We present in this paper a novel framework for morpheme segmentation which
uses the morpho-syntactic regularities preserved by word representations, in
addition to orthographic features, to segment words into morphemes. This
framework is the first to consider vocabulary-wide syntactico-semantic
information for this task. We also analyze the deficiencies of available
benchmarking datasets and introduce our own dataset that was created on the
basis of compositionality. We validate our algorithm across datasets and
present state-of-the-art results.
A Labeling-Free Approach to Supervising Deep Neural Networks for Retinal Blood Vessel Segmentation
Authors: Yongliang Chen
Date: 2017-04-25 00:00:00
Tasks: Medical Diagnosis

Segmenting blood vessels in fundus imaging plays an important role in medical
diagnosis. Many algorithms have been proposed. While deep Neural Networks have
been attracting enormous attention from computer vision community recent years
and several novel works have been done in terms of its application in retinal
blood vessel segmentation, most of them are based on supervised learning which
requires amount of labeled data, which is both scarce and expensive to obtain.
We leverage the power of Deep Convolutional Neural Networks (DCNN) in feature
learning, in this work, to achieve this ultimate goal. The highly efficient
feature learning of DCNN inspires our novel approach that trains the networks
with automatically-generated samples to achieve desirable performance on
real-world fundus images. For this, we design a set of rules abstracted from
the domain-specific prior knowledge to generate these samples. We argue that,
with the high efficiency of DCNN in feature learning, one can achieve this goal
by constructing the training dataset with prior knowledge, no manual labeling
is needed. This approach allows us to take advantages of supervised learning
without labeling. We also build a naive DCNN model to test it. The results on
standard benchmarks of fundus imaging show it is competitive to the
state-of-the-art methods which implies a potential way to leverage the power of
DCNN in feature learning.
Shearlet-based compressed sensing for fast 3D cardiac MR imaging using iterative reweighting
Authors: Jackie Ma, Maximilian März, Stephanie Funk, Jeanette Schulz-Menger, Gitta Kutyniok, Tobias Schaeffter, Christoph Kolbitsch
Date: 2017-05-01 00:00:00
Tasks: Anatomy, compressed sensing, Image Reconstruction

High-resolution three-dimensional (3D) cardiovascular magnetic resonance
(CMR) is a valuable medical imaging technique, but its widespread application
in clinical practice is hampered by long acquisition times. Here we present a
novel compressed sensing (CS) reconstruction approach using shearlets as a
sparsifying transform allowing for fast 3D CMR (3DShearCS). Shearlets are
mathematically optimal for a simplified model of natural images and have been
proven to be more efficient than classical systems such as wavelets. Data is
acquired with a 3D Radial Phase Encoding (RPE) trajectory and an iterative
reweighting scheme is used during image reconstruction to ensure fast
convergence and high image quality. In our in-vivo cardiac MRI experiments we
show that the proposed method 3DShearCS has lower relative errors and higher
structural similarity compared to the other reconstruction techniques
especially for high undersampling factors, i.e. short scan times. In this
paper, we further show that 3DShearCS provides improved depiction of cardiac
anatomy (measured by assessing the sharpness of coronary arteries) and two
clinical experts qualitatively analyzed the image quality.
Spectrum Monitoring for Radar Bands using Deep Convolutional Neural Networks
Authors: Ahmed Selim, Francisco Paisana, Jerome A. Arokkiam, Yi Zhang, Linda Doyle, Luiz A. DaSilva
Date: 2017-05-01 00:00:00

In this paper, we present a spectrum monitoring framework for the detection
of radar signals in spectrum sharing scenarios. The core of our framework is a
deep convolutional neural network (CNN) model that enables Measurement Capable
Devices to identify the presence of radar signals in the radio spectrum, even
when these signals are overlapped with other sources of interference, such as
commercial LTE and WLAN. We collected a large dataset of RF measurements, which
include the transmissions of multiple radar pulse waveforms, downlink LTE,
WLAN, and thermal noise. We propose a pre-processing data representation that
leverages the amplitude and phase shifts of the collected samples. This
representation allows our CNN model to achieve a classification accuracy of
99.6% on our testing dataset. The trained CNN model is then tested under
various SNR values, outperforming other models, such as spectrogram-based CNN
models.