pwc-archive/papers-with-abstracts huggingface dataset:
Constrained-CNN losses for weakly supervised segmentation
Authors: Hoel Kervadec, Jose Dolz, Meng Tang, Eric Granger, Yuri Boykov, Ismail Ben Ayed
Date: 2018-05-12 00:00:00
Tasks: Medical Image Segmentation, Segmentation, Semantic Segmentation, Weakly-supervised Learning, Weakly supervised segmentation, Weakly-Supervised Semantic Segmentation

Weakly-supervised learning based on, e.g., partially labelled images or
image-tags, is currently attracting significant attention in CNN segmentation
as it can mitigate the need for full and laborious pixel/voxel annotations.
Enforcing high-order (global) inequality constraints on the network output (for
instance, to constrain the size of the target region) can leverage unlabeled
data, guiding the training process with domain-specific knowledge. Inequality
constraints are very flexible because they do not assume exact prior knowledge.
However, constrained Lagrangian dual optimization has been largely avoided in
deep networks, mainly for computational tractability reasons. To the best of
our knowledge, the method of [Pathak et al., 2015] is the only prior work that
addresses deep CNNs with linear constraints in weakly supervised segmentation.
It uses the constraints to synthesize fully-labeled training masks (proposals)
from weak labels, mimicking full supervision and facilitating dual
optimization. We propose to introduce a differentiable penalty, which enforces
inequality constraints directly in the loss function, avoiding expensive
Lagrangian dual iterates and proposal generation. From constrained-optimization
perspective, our simple penalty-based approach is not optimal as there is no
guarantee that the constraints are satisfied. However, surprisingly, it yields
substantially better results than the Lagrangian-based constrained CNNs in
[Pathak et al., 2015], while reducing the computational demand for training. By
annotating only a small fraction of the pixels, the proposed approach can reach
a level of segmentation performance that is comparable to full supervision on
three separate tasks. While our experiments focused on basic linear constraints
such as the target-region size and image tags, our framework can be easily
extended to other non-linear constraints.
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
Authors: Urvashi Khandelwal, He He, Peng Qi, Dan Jurafsky
Date: 2018-05-12 00:00:00
Proceeding: ACL 2018 7
Tasks: Sentence

We know very little about how neural language models (LM) use prior
linguistic context. In this paper, we investigate the role of context in an
LSTM LM, through ablation studies. Specifically, we analyze the increase in
perplexity when prior context words are shuffled, replaced, or dropped. On two
standard datasets, Penn Treebank and WikiText-2, we find that the model is
capable of using about 200 tokens of context on average, but sharply
distinguishes nearby context (recent 50 tokens) from the distant history. The
model is highly sensitive to the order of words within the most recent
sentence, but ignores word order in the long-range context (beyond 50 tokens),
suggesting the distant past is modeled only as a rough semantic field or topic.
We further find that the neural caching model (Grave et al., 2017b) especially
helps the LSTM to copy words from within this distant context. Overall, our
analysis not only provides a better understanding of how neural LMs use their
context, but also sheds light on recent success from cache-based models.
TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation
Authors: Alexander R. Fabbri, Irene Li, Prawat Trairatvorakul, Yijiao He, Wei Tai Ting, Robert Tung, Caitlin Westerfield, Dragomir R. Radev
Date: 2018-05-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Information Retrieval, Retrieval

The field of Natural Language Processing (NLP) is growing rapidly, with new
research published daily along with an abundance of tutorials, codebases and
other online resources. In order to learn this dynamic field or stay up-to-date
on the latest research, students as well as educators and researchers must
constantly sift through multiple sources to find valuable, relevant
information. To address this situation, we introduce TutorialBank, a new,
publicly available dataset which aims to facilitate NLP education and research.
We have manually collected and categorized over 6,300 resources on NLP as well
as the related fields of Artificial Intelligence (AI), Machine Learning (ML)
and Information Retrieval (IR). Our dataset is notably the largest
manually-picked corpus of resources intended for NLP education which does not
include only academic papers. Additionally, we have created both a search
engine and a command-line tool for the resources and have annotated the corpus
to include lists of research topics, relevant resources for each topic,
prerequisite relations among topics, relevant sub-parts of individual
resources, among other annotations. We are releasing the dataset and present
several avenues for further research.
Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality
Authors: Alexandre Salle, Aline Villavicencio
Date: 2017-04-03 00:00:00
Proceeding: ACL 2018 7
Tasks: Language Modeling, Language Modelling, Tensor Networks

Increasing the capacity of recurrent neural networks (RNN) usually involves
augmenting the size of the hidden layer, with significant increase of
computational cost. Recurrent neural tensor networks (RNTN) increase capacity
using distinct hidden layer weights for each word, but with greater costs in
memory usage. In this paper, we introduce restricted recurrent neural tensor
networks (r-RNTN) which reserve distinct hidden layer weights for frequent
vocabulary words while sharing a single set of weights for infrequent words.
Perplexity evaluations show that for fixed hidden layer sizes, r-RNTNs improve
language model performance over RNNs using only a small fraction of the
parameters of unrestricted RNTNs. These results hold for r-RNTNs using Gated
Recurrent Units and Long Short-Term Memory.
Twitter User Geolocation using Deep Multiview Learning
Authors: Tien Huu Do, Duc Minh Nguyen, Evaggelia Tsiligianni, Bruno Cornelis, Nikos Deligiannis
Date: 2018-05-11 00:00:00
Tasks: Multiview Learning

Predicting the geographical location of users on social networks like Twitter
is an active research topic with plenty of methods proposed so far. Most of the
existing work follows either a content-based or a network-based approach. The
former is based on user-generated content while the latter exploits the
structure of the network of users. In this paper, we propose a more generic
approach, which incorporates not only both content-based and network-based
features, but also other available information into a unified model. Our
approach, named Multi-Entry Neural Network (MENET), leverages the latest
advances in deep learning and multiview learning. A realization of MENET with
textual, network and metadata features results in an effective method for
Twitter user geolocation, achieving the state of the art on two well-known
datasets.
Textual Membership Queries
Authors: Jonathan Zarecki, Shaul Markovitch
Date: 2018-05-11 00:00:00
Tasks: Active Learning, General Classification, text-classification, Text Classification

Human labeling of data can be very time-consuming and expensive, yet, in many cases it is critical for the success of the learning process. In order to minimize human labeling efforts, we propose a novel active learning solution that does not rely on existing sources of unlabeled data. It uses a small amount of labeled data as the core set for the synthesis of useful membership queries (MQs) - unlabeled instances generated by an algorithm for human labeling. Our solution uses modification operators, functions that modify instances to some extent. We apply the operators on a small set of instances (core set), creating a set of new membership queries. Using this framework, we look at the instance space as a search space and apply search algorithms in order to generate new examples highly relevant to the learner. We implement this framework in the textual domain and test it on several text classification tasks and show improved classifier performance as more MQs are labeled and incorporated into the training set. To the best of our knowledge, this is the first work on membership queries in the textual domain.
Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration
Authors: Adrian V. Dalca, Guha Balakrishnan, John Guttag, Mert R. Sabuncu
Date: 2018-05-11 00:00:00

Traditional deformable registration techniques achieve impressive results and
offer a rigorous theoretical treatment, but are computationally intensive since
they solve an optimization problem for each image pair. Recently,
learning-based methods have facilitated fast registration by learning spatial
deformation functions. However, these approaches use restricted deformation
models, require supervised labels, or do not guarantee a diffeomorphic
(topology-preserving) registration. Furthermore, learning-based registration
tools have not been derived from a probabilistic framework that can offer
uncertainty estimates. In this paper, we present a probabilistic generative
model and derive an unsupervised learning-based inference algorithm that makes
use of recent developments in convolutional neural networks (CNNs). We
demonstrate our method on a 3D brain registration task, and provide an
empirical analysis of the algorithm. Our approach results in state of the art
accuracy and very fast runtimes, while providing diffeomorphic guarantees and
uncertainty estimates. Our implementation is available online at
http://voxelmorph.csail.mit.edu .
Confidence Modeling for Neural Semantic Parsing
Authors: Li Dong, Chris Quirk, Mirella Lapata
Date: 2018-05-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Semantic Parsing

In this work we focus on confidence modeling for neural semantic parsers
which are built upon sequence-to-sequence models. We outline three major causes
of uncertainty, and design various metrics to quantify these factors. These
metrics are then used to estimate confidence scores that indicate whether model
predictions are likely to be correct. Beyond confidence estimation, we identify
which parts of the input contribute to uncertain predictions allowing users to
interpret their model, and verify or refine its input. Experimental results
show that our confidence model significantly outperforms a widely used method
that relies on posterior probability, and improves the quality of
interpretation compared to simply relying on attention scores.
Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction
Authors: Hu Xu, Bing Liu, Lei Shu, Philip S. Yu
Date: 2018-05-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Aspect-Based Sentiment Analysis (ABSA), Aspect Extraction, Deep Learning, Sentiment Analysis

One key task of fine-grained sentiment analysis of product reviews is to
extract product aspects or features that users have expressed opinions on. This
paper focuses on supervised aspect extraction using deep learning. Unlike other
highly sophisticated supervised deep learning models, this paper proposes a
novel and yet simple CNN model employing two types of pre-trained embeddings
for aspect extraction: general-purpose embeddings and domain-specific
embeddings. Without using any additional supervision, this model achieves
surprisingly good results, outperforming state-of-the-art sophisticated
existing methods. To our knowledge, this paper is the first to report such
double embeddings based CNN model for aspect extraction and achieve very good
results.
Behavioral Cloning from Observation
Authors: Faraz Torabi, Garrett Warnell, Peter Stone
Date: 2018-05-04 00:00:00
Tasks: Imitation Learning

Humans often learn how to perform tasks via imitation: they observe others
perform a task, and then very quickly infer the appropriate actions to take
based on their observations. While extending this paradigm to autonomous agents
is a well-studied problem in general, there are two particular aspects that
have largely been overlooked: (1) that the learning is done from observation
only (i.e., without explicit action information), and (2) that the learning is
typically done very quickly. In this work, we propose a two-phase, autonomous
imitation learning technique called behavioral cloning from observation (BCO),
that aims to provide improved performance with respect to both of these
aspects. First, we allow the agent to acquire experience in a self-supervised
fashion. This experience is used to develop a model which is then utilized to
learn a particular task by observing an expert perform that task without the
knowledge of the specific actions taken. We experimentally compare BCO to
imitation learning methods, including the state-of-the-art, generative
adversarial imitation learning (GAIL) technique, and we show comparable task
performance in several different simulation domains while exhibiting increased
learning speed after expert trajectories become available.
Joint Flow: Temporal Flow Fields for Multi Person Tracking
Authors: Andreas Doering, Umar Iqbal, Juergen Gall
Date: 2018-05-11 00:00:00
Tasks: Multi-Person Pose Estimation, Pose Estimation, Pose Tracking

In this work we propose an online multi person pose tracking approach which
works on two consecutive frames $I_{t-1}$ and $I_t$. The general formulation of
our temporal network allows to rely on any multi person pose estimation
approach as spatial network. From the spatial network we extract image features
and pose features for both frames. These features serve as input for our
temporal model that predicts Temporal Flow Fields (TFF). These TFF are vector
fields which indicate the direction in which each body joint is going to move
from frame $I_{t-1}$ to frame $I_t$. This novel representation allows to
formulate a similarity measure of detected joints. These similarities are used
as binary potentials in a bipartite graph optimization problem in order to
perform tracking of multiple poses. We show that these TFF can be learned by a
relative small CNN network whilst achieving state-of-the-art multi person pose
tracking results.
The Domain Transform Solver
Authors: Akash Bapat, Jan-Michael Frahm
Date: 2018-05-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Super-Resolution

We present a framework for edge-aware optimization that is an order of
magnitude faster than the state of the art while having comparable performance.
Our key insight is that the optimization can be formulated by leveraging
properties of the domain transform, a method for edge-aware filtering that
defines a distance-preserving 1D mapping of the input space. This enables our
method to improve performance for a variety of problems including stereo, depth
super-resolution, and render from defocus, while keeping the computational
complexity linear in the number of pixels. Our method is highly parallelizable
and adaptable, and it has demonstrable scalability with respect to image
resolution.
TensOrMachine: Probabilistic Boolean Tensor Decomposition
Authors: Tammo Rukat, Chris C. Holmes, Christopher Yau
Date: 2018-05-11 00:00:00
Tasks: Model Selection, Tensor Decomposition

Boolean tensor decomposition approximates data of multi-way binary
relationships as product of interpretable low-rank binary factors, following
the rules of Boolean algebra. Here, we present its first probabilistic
treatment. We facilitate scalable sampling-based posterior inference by
exploitation of the combinatorial structure of the factor conditionals. Maximum
a posteriori decompositions feature higher accuracies than existing techniques
throughout a wide range of simulated conditions. Moreover, the probabilistic
approach facilitates the treatment of missing data and enables model selection
with much greater accuracy. We investigate three real-world data-sets. First,
temporal interaction networks in a hospital ward and behavioural data of
university students demonstrate the inference of instructive latent patterns.
Next, we decompose a tensor with more than 10 billion data points, indicating
relations of gene expression in cancer patients. Not only does this demonstrate
scalability, it also provides an entirely novel perspective on relational
properties of continuous data and, in the present example, on the molecular
heterogeneity of cancer. Our implementation is available on GitHub:
https://github.com/TammoR/LogicalFactorisationMachines.
Adversarial Contrastive Estimation
Authors: Avishek Joey Bose, Huan Ling, Yanshuai Cao
Date: 2018-05-09 00:00:00
Proceeding: ACL 2018 7
Tasks: Contrastive Learning, Knowledge Graph Embeddings, Knowledge Graphs, Learning Word Embeddings, Word Embeddings

Learning by contrasting positive and negative samples is a general strategy
adopted by many methods. Noise contrastive estimation (NCE) for word embeddings
and translating embeddings for knowledge graphs are examples in NLP employing
this approach. In this work, we view contrastive learning as an abstraction of
all such methods and augment the negative sampler into a mixture distribution
containing an adversarially learned sampler. The resulting adaptive sampler
finds harder negative examples, which forces the main model to learn a better
representation of the data. We evaluate our proposal on learning word
embeddings, order embeddings and knowledge graph embeddings and observe both
faster convergence and improved results on multiple metrics.
Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions
Authors: Mingrui Liu, Xiaoxuan Zhang, Lijun Zhang, Rong Jin, Tianbao Yang
Date: 2018-05-11 00:00:00
Proceeding: NeurIPS 2018 12

Error bound conditions (EBC) are properties that characterize the growth of
an objective function when a point is moved away from the optimal set. They
have recently received increasing attention in the field of optimization for
developing optimization algorithms with fast convergence. However, the studies
of EBC in statistical learning are hitherto still limited. The main
contributions of this paper are two-fold. First, we develop fast and
intermediate rates of empirical risk minimization (ERM) under EBC for risk
minimization with Lipschitz continuous, and smooth convex random functions.
Second, we establish fast and intermediate rates of an efficient stochastic
approximation (SA) algorithm for risk minimization with Lipschitz continuous
random functions, which requires only one pass of $n$ samples and adapts to
EBC. For both approaches, the convergence rates span a full spectrum between
$\widetilde O(1/\sqrt{n})$ and $\widetilde O(1/n)$ depending on the power
constant in EBC, and could be even faster than $O(1/n)$ in special cases for
ERM. Moreover, these convergence rates are automatically adaptive without using
any knowledge of EBC. Overall, this work not only strengthens the understanding
of ERM for statistical learning but also brings new fast stochastic algorithms
for solving a broad range of statistical learning problems.
Domain Adapted Word Embeddings for Improved Sentiment Classification
Authors: Prathusha K Sarma, YIngyu Liang, William A. Sethares
Date: 2018-05-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Classification, General Classification, Sentence, Sentiment Analysis, Sentiment Classification, Specificity, Word Embeddings

Generic word embeddings are trained on large-scale generic corpora; Domain
Specific (DS) word embeddings are trained only on data from a domain of
interest. This paper proposes a method to combine the breadth of generic
embeddings with the specificity of domain specific embeddings. The resulting
embeddings, called Domain Adapted (DA) word embeddings, are formed by aligning
corresponding word vectors using Canonical Correlation Analysis (CCA) or the
related nonlinear Kernel CCA. Evaluation results on sentiment classification
tasks show that the DA embeddings substantially outperform both generic and DS
embeddings when used as input features to standard or state-of-the-art sentence
encoding algorithms for classification.
Vision-based Real Estate Price Estimation
Authors: Omid Poursaeed, Tomas Matera, Serge Belongie
Date: 2017-07-18 00:00:00

Since the advent of online real estate database companies like Zillow, Trulia
and Redfin, the problem of automatic estimation of market values for houses has
received considerable attention. Several real estate websites provide such
estimates using a proprietary formula. Although these estimates are often close
to the actual sale prices, in some cases they are highly inaccurate. One of the
key factors that affects the value of a house is its interior and exterior
appearance, which is not considered in calculating automatic value estimates.
In this paper, we evaluate the impact of visual characteristics of a house on
its market value. Using deep convolutional neural networks on a large dataset
of photos of home interiors and exteriors, we develop a method for estimating
the luxury level of real estate photos. We also develop a novel framework for
automated value assessment using the above photos in addition to home
characteristics including size, offered price and number of bedrooms. Finally,
by applying our proposed method for price estimation to a new dataset of real
estate photos and metadata, we show that it outperforms Zillow's estimates.
Learning-induced categorical perception in a neural network model
Authors: Christian Thériault, Fernanda Pérez-Gay, Dan Rivas, Stevan Harnad
Date: 2018-05-11 00:00:00

In human cognition, the expansion of perceived between-category distances and
compression of within-category distances is known as categorical perception
(CP). There are several hypotheses about the causes of CP (e.g., language,
learning, evolution) but no functional model. Whether CP is essential to
categorisation or simply a by-product of it is not yet clear, but evidence is
accumulating that CP can be induced by category learning. We provide a model
for learning-induced CP as expansion and compression of distances in
hidden-unit space in neural nets. Basic conditions from which the current model
predicts CP are described, and clues as to how these conditions might
generalize to more complex kinds of categorization begin to emerge.
Classification of Protein Crystallization X-Ray Images Using Major Convolutional Neural Network Architectures
Authors: Soheil Ghafurian, Peter Orth, Corey Strickland, Hua Su, Sangita Patel, Steven Soisson, Belma Dogdas
Date: 2018-05-11 00:00:00
Tasks: General Classification

The generation of protein crystals is necessary for the study of protein
molecular function and structure. This is done empirically by processing large
numbers of crystallization trials and inspecting them regularly in search of
those with forming crystals. To avoid missing the hard-gained crystals, this
visual inspection of the trial X-ray images is done manually as opposed to the
existing less accurate machine learning methods. To achieve higher accuracy for
automation, we applied some of the most successful convolutional neural
networks (ResNet, Inception, VGG, and AlexNet) for 10-way classification of the
X-ray images. We showed that substantial classification accuracy is gained by
using such networks compared to two simpler ones previously proposed for this
purpose. The best accuracy was obtained from ResNet (81.43%), which corresponds
to a missed crystal rate of 5.9%. This rate could be lowered to less than 0.1%
by using a top-3 classification strategy. Our dataset consisted of 486,000
internally annotated images, which was augmented to more than a million to
address class imbalance. We also provide a label-wise analysis of the results,
identifying the main sources of error and inaccuracy.