pwc-archive/papers-with-abstracts huggingface dataset:
Low-memory convolutional neural networks through incremental depth-first processing
Authors: Jonathan Binas, Yoshua Bengio
Date: 2018-04-28 00:00:00

We introduce an incremental processing scheme for convolutional neural network (CNN) inference, targeted at embedded applications with limited memory budgets. Instead of processing layers one by one, individual input pixels are propagated through all parts of the network they can influence under the given structural constraints. This depth-first updating scheme comes with hard bounds on the memory footprint: the memory required is constant in the case of 1D input and proportional to the square root of the input dimension in the case of 2D input.
Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation
Authors: Xiao-Feng Gong, Qiu-Hua Lin, Feng-Yu Cong, Lieven De Lathauwer
Date: 2016-12-30 00:00:00
Tasks: blind source separation

Joint blind source separation (J-BSS) is an emerging data-driven technique
for multi-set data-fusion. In this paper, J-BSS is addressed from a tensorial
perspective. We show how, by using second-order multi-set statistics in J-BSS,
a specific double coupled canonical polyadic decomposition (DC-CPD) problem can
be formulated. We propose an algebraic DC-CPD algorithm based on a coupled
rank-1 detection mapping. This algorithm converts a possibly underdetermined
DC-CPD to a set of overdetermined CPDs. The latter can be solved algebraically
via a generalized eigenvalue decomposition based scheme. Therefore, this
algorithm is deterministic and returns the exact solution in the noiseless
case. In the noisy case, it can be used to effectively initialize optimization
based DC-CPD algorithms. In addition, we obtain the determini- stic and generic
uniqueness conditions for DC-CPD, which are shown to be more relaxed than their
CPD counterpart. Experiment results are given to illustrate the superiority of
DC-CPD over standard CPD based BSS methods and several existing J-BSS methods,
with regards to uniqueness and accuracy.
Data-Driven Methods for Solving Algebra Word Problems
Authors: Benjamin Robaidek, Rik Koncel-Kedziorski, Hannaneh Hajishirzi
Date: 2018-04-28 00:00:00
Tasks: Math, World Knowledge

We explore contemporary, data-driven techniques for solving math word
problems over recent large-scale datasets. We show that well-tuned neural
equation classifiers can outperform more sophisticated models such as sequence
to sequence and self-attention across these datasets. Our error analysis
indicates that, while fully data driven models show some promise, semantic and
world knowledge is necessary for further advances.
Modified Apriori Graph Algorithm for Frequent Pattern Mining
Authors: Pritish Yuvraj, Suneetha K. R
Date: 2018-04-27 00:00:00

Web Usage Mining is an application of Data Mining Techniques to discover
interesting usage patterns from web data in order to understand and better
serve the needs of web-based applications. The paper proposes an algorithm for
finding these usage patterns using a modified version of Apriori Algorithm
called Apriori-Graph. These rules will help service providers to predict, which
web pages, the user is likely to visit next. This will optimize the website in
terms of efficiency, bandwidth and will have positive economic benefits for
them. The proposed Apriori Graph Algorithm O((V)(E)) works faster compared to
the existing Apriori Algorithm and is well suitable for real-time application.
Extracting Lungs from CT Images using Fully Convolutional Networks
Authors: Jeovane Honório Alves, Pedro Martins Moreira Neto, Lucas Ferrari de Oliveira
Date: 2018-04-27 00:00:00
Tasks: Computed Tomography (CT), Segmentation

Analysis of cancer and other pathological diseases, like the interstitial
lung diseases (ILDs), is usually possible through Computed Tomography (CT)
scans. To aid this, a preprocessing step of segmentation is performed to reduce
the area to be analyzed, segmenting the lungs and removing unimportant regions.
Generally, complex methods are developed to extract the lung region, also using
hand-made feature extractors to enhance segmentation. With the popularity of
deep learning techniques and its automated feature learning, we propose a lung
segmentation approach using fully convolutional networks (FCNs) combined with
fully connected conditional random fields (CRF), employed in many
state-of-the-art segmentation works. Aiming to develop a generalized approach,
the publicly available datasets from University Hospitals of Geneva (HUG) and
VESSEL12 challenge were studied, including many healthy and pathological CT
scans for evaluation. Experiments using the dataset individually, its trained
model on the other dataset and a combination of both datasets were employed.
Dice scores of $98.67\%\pm0.94\%$ for the HUG-ILD dataset and
$99.19\%\pm0.37\%$ for the VESSEL12 dataset were achieved, outperforming works
in the former and obtaining similar state-of-the-art results in the latter
dataset, showing the capability in using deep learning approaches.
Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
Authors: W. James Murdoch, Peter J. Liu, Bin Yu
Date: 2018-01-16 00:00:00
Proceeding: ICLR 2018 1
Tasks: Sentiment Analysis

The driving force behind the recent success of LSTMs has been their ability
to learn complex and non-linear relationships. Consequently, our inability to
describe these relationships has led to LSTMs being characterized as black
boxes. To this end, we introduce contextual decomposition (CD), an
interpretation algorithm for analysing individual predictions made by standard
LSTMs, without any changes to the underlying model. By decomposing the output
of a LSTM, CD captures the contributions of combinations of words or variables
to the final prediction of an LSTM. On the task of sentiment analysis with the
Yelp and SST data sets, we show that CD is able to reliably identify words and
phrases of contrasting sentiment, and how they are combined to yield the LSTM's
final prediction. Using the phrase-level labels in SST, we also demonstrate
that CD is able to successfully extract positive and negative negations from an
LSTM, something which has not previously been done.
Bandits and Experts in Metric Spaces
Authors: Robert Kleinberg, Aleksandrs Slivkins, Eli Upfal
Date: 2013-12-04 00:00:00

In a multi-armed bandit problem, an online algorithm chooses from a set of
strategies in a sequence of trials so as to maximize the total payoff of the
chosen strategies. While the performance of bandit algorithms with a small
finite strategy set is quite well understood, bandit problems with large
strategy sets are still a topic of very active investigation, motivated by
practical applications such as online auctions and web advertisement. The goal
of such research is to identify broad and natural classes of strategy sets and
payoff functions which enable the design of efficient solutions.
  In this work we study a very general setting for the multi-armed bandit
problem in which the strategies form a metric space, and the payoff function
satisfies a Lipschitz condition with respect to the metric. We refer to this
problem as the "Lipschitz MAB problem". We present a solution for the
multi-armed bandit problem in this setting. That is, for every metric space we
define an isometry invariant which bounds from below the performance of
Lipschitz MAB algorithms for this metric space, and we present an algorithm
which comes arbitrarily close to meeting this bound. Furthermore, our technique
gives even better results for benign payoff functions. We also address the
full-feedback ("best expert") version of the problem, where after every round
the payoffs from all arms are revealed.
mixup: Beyond Empirical Risk Minimization
Authors: Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Paz
Date: 2017-10-25 00:00:00
Proceeding: ICLR 2018 1
Tasks: Domain Generalization, Image Classification, Memorization, Out-of-Distribution Generalization, Semi-Supervised Image Classification

Large deep neural networks are powerful, but exhibit undesirable behaviors
such as memorization and sensitivity to adversarial examples. In this work, we
propose mixup, a simple learning principle to alleviate these issues. In
essence, mixup trains a neural network on convex combinations of pairs of
examples and their labels. By doing so, mixup regularizes the neural network to
favor simple linear behavior in-between training examples. Our experiments on
the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show
that mixup improves the generalization of state-of-the-art neural network
architectures. We also find that mixup reduces the memorization of corrupt
labels, increases the robustness to adversarial examples, and stabilizes the
training of generative adversarial networks.
Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code
Authors: Riyadh Baghdadi, Jessica Ray, Malek Ben Romdhane, Emanuele Del Sozzo, Abdurrahman Akkas, Yunming Zhang, Patricia Suriana, Shoaib Kamil, Saman Amarasinghe
Date: 2018-04-27 00:00:00
Tasks: Scheduling

This paper introduces Tiramisu, a polyhedral framework designed to generate
high performance code for multiple platforms including multicores, GPUs, and
distributed machines. Tiramisu introduces a scheduling language with novel
extensions to explicitly manage the complexities that arise when targeting
these systems. The framework is designed for the areas of image processing,
stencils, linear algebra and deep learning. Tiramisu has two main features: it
relies on a flexible representation based on the polyhedral model and it has a
rich scheduling language allowing fine-grained control of optimizations.
Tiramisu uses a four-level intermediate representation that allows full
separation between the algorithms, loop transformations, data layouts, and
communication. This separation simplifies targeting multiple hardware
architectures with the same algorithm. We evaluate Tiramisu by writing a set of
image processing, deep learning, and linear algebra benchmarks and compare them
with state-of-the-art compilers and hand-tuned libraries. We show that Tiramisu
matches or outperforms existing compilers and libraries on different hardware
architectures, including multicore CPUs, GPUs, and distributed machines.
Reward Learning from Narrated Demonstrations
Authors: Hsiao-Yu Fish Tung, Adam W. Harley, Liang-Kang Huang, Katerina Fragkiadaki
Date: 2018-04-27 00:00:00
Proceeding: CVPR 2018 6

Humans effortlessly "program" one another by communicating goals and desires
in natural language. In contrast, humans program robotic behaviours by
indicating desired object locations and poses to be achieved, by providing RGB
images of goal configurations, or supplying a demonstration to be imitated.
None of these methods generalize across environment variations, and they convey
the goal in awkward technical terms. This work proposes joint learning of
natural language grounding and instructable behavioural policies reinforced by
perceptual detectors of natural language expressions, grounded to the sensory
inputs of the robotic agent. Our supervision is narrated visual
demonstrations(NVD), which are visual demonstrations paired with verbal
narration (as opposed to being silent). We introduce a dataset of NVD where
teachers perform activities while describing them in detail. We map the
teachers' descriptions to perceptual reward detectors, and use them to train
corresponding behavioural policies in simulation.We empirically show that our
instructable agents (i) learn visual reward detectors using a small number of
examples by exploiting hard negative mined configurations from demonstration
dynamics, (ii) develop pick-and place policies using learned visual reward
detectors, (iii) benefit from object-factorized state representations that
mimic the syntactic structure of natural language goal expressions, and (iv)
can execute behaviours that involve novel objects in novel locations at test
time, instructed by natural language.
Negative Log Likelihood Ratio Loss for Deep Neural Network Classification
Authors: Donglai Zhu, Hengshuai Yao, Bei Jiang, Peng Yu
Date: 2018-04-27 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

In deep neural network, the cross-entropy loss function is commonly used for
classification. Minimizing cross-entropy is equivalent to maximizing likelihood
under assumptions of uniform feature and class distributions. It belongs to
generative training criteria which does not directly discriminate correct class
from competing classes. We propose a discriminative loss function with negative
log likelihood ratio between correct and competing classes. It significantly
outperforms the cross-entropy loss on the CIFAR-10 image classification task.
RIDDLE: Race and ethnicity Imputation from Disease history with Deep LEarning
Authors: Ji-Sung Kim, Xin Gao, Andrey Rzhetsky
Date: 2017-07-06 00:00:00
Tasks: Imputation

Anonymized electronic medical records are an increasingly popular source of
research data. However, these datasets often lack race and ethnicity
information. This creates problems for researchers modeling human disease, as
race and ethnicity are powerful confounders for many health exposures and
treatment outcomes; race and ethnicity are closely linked to
population-specific genetic variation. We showed that deep neural networks
generate more accurate estimates for missing racial and ethnic information than
competing methods (e.g., logistic regression, random forest). RIDDLE yielded
significantly better classification performance across all metrics that were
considered: accuracy, cross-entropy loss (error), and area under the curve for
receiver operating characteristic plots (all $p < 10^{-6}$). We made specific
efforts to interpret the trained neural network models to identify, quantify,
and visualize medical features which are predictive of race and ethnicity. We
used these characterizations of informative features to perform a systematic
comparison of differential disease patterns by race and ethnicity. The fact
that clinical histories are informative for imputing race and ethnicity could
reflect (1) a skewed distribution of blue- and white-collar professions across
racial and ethnic groups, (2) uneven accessibility and subjective importance of
prophylactic health, (3) possible variation in lifestyle, such as dietary
habits, and (4) differences in background genetic variation which predispose to
diseases.
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Authors: Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, Rob Fergus
Date: 2017-03-15 00:00:00
Proceeding: ICLR 2018 1

We describe a simple scheme that allows an agent to learn about its
environment in an unsupervised manner. Our scheme pits two versions of the same
agent, Alice and Bob, against one another. Alice proposes a task for Bob to
complete; and then Bob attempts to complete the task. In this work we will
focus on two kinds of environments: (nearly) reversible environments and
environments that can be reset. Alice will "propose" the task by doing a
sequence of actions and then Bob must undo or repeat them, respectively. Via an
appropriate reward structure, Alice and Bob automatically generate a curriculum
of exploration, enabling unsupervised training of the agent. When Bob is
deployed on an RL task within the environment, this unsupervised training
reduces the number of supervised episodes needed to learn, and in some cases
converges to a higher reward.
An Unsupervised Word Sense Disambiguation System for Under-Resourced Languages
Authors: Dmitry Ustalov, Denis Teslenko, Alexander Panchenko, Mikhail Chernoskutov, Chris Biemann, Simone Paolo Ponzetto
Date: 2018-04-27 00:00:00
Proceeding: LREC 2018 5
Tasks: Semantic Similarity, Semantic Textual Similarity, Sentence, Word Sense Disambiguation

In this paper, we present Watasense, an unsupervised system for word sense
disambiguation. Given a sentence, the system chooses the most relevant sense of
each input word with respect to the semantic similarity between the given
sentence and the synset constituting the sense of the target word. Watasense
has two modes of operation. The sparse mode uses the traditional vector space
model to estimate the most similar word sense corresponding to its context. The
dense mode, instead, uses synset embeddings to cope with the sparsity problem.
We describe the architecture of the present system and also conduct its
evaluation on three different lexical semantic resources for Russian. We found
that the dense mode substantially outperforms the sparse one on all datasets
according to the adjusted Rand index.
Joint Shape Representation and Classification for Detecting PDAC
Authors: Fengze Liu, Lingxi Xie, Yingda Xia, Elliot K. Fishman, Alan L. Yuille
Date: 2018-04-27 00:00:00
Tasks: Classification, General Classification, Specificity

We aim to detect pancreatic ductal adenocarcinoma (PDAC) in abdominal CT scans, which sheds light on early diagnosis of pancreatic cancer. This is a 3D volume classification task with little training data. We propose a two-stage framework, which first segments the pancreas into a binary mask, then compresses the mask into a shape vector and performs abnormality classification. Shape representation and classification are performed in a joint manner, both to exploit the knowledge that PDAC often changes the shape of the pancreas and to prevent over-fitting. Experiments are performed on 300 normal scans and 136 PDAC cases. We achieve a specificity of 90.2% (false alarm occurs on less than 1/10 normal cases) at a sensitivity of 80.2% (less than 1/5 PDAC cases are not detected), which show promise for clinical applications.
Deep Speech Denoising with Vector Space Projections
Authors: Jeff Hetherly, Paul Gamble, Maria Barrios, Cory Stephenson, Karl Ni
Date: 2018-04-27 00:00:00
Tasks: Denoising, Speech Denoising

We propose an algorithm to denoise speakers from a single microphone in the
presence of non-stationary and dynamic noise. Our approach is inspired by the
recent success of neural network models separating speakers from other speakers
and singers from instrumental accompaniment. Unlike prior art, we leverage
embedding spaces produced with source-contrastive estimation, a technique
derived from negative sampling techniques in natural language processing, while
simultaneously obtaining a continuous inference mask. Our embedding space
directly optimizes for the discrimination of speaker and noise by jointly
modeling their characteristics. This space is generalizable in that it is not
speaker or noise specific and is capable of denoising speech even if the model
has not seen the speaker in the training set. Parameters are trained with dual
objectives: one that promotes a selective bandpass filter that eliminates noise
at time-frequency positions that exceed signal power, and another that
proportionally splits time-frequency content between signal and noise. We
compare to state of the art algorithms as well as traditional sparse
non-negative matrix factorization solutions. The resulting algorithm avoids
severe computational burden by providing a more intuitive and easily optimized
approach, while achieving competitive accuracy.
Mapping Road Lanes Using Laser Remission and Deep Neural Networks
Authors: Raphael V. Carneiro, Rafael C. Nascimento, Rânik Guidolini, Vinicius B. Cardoso, Thiago Oliveira-Santos, Claudine Badue, Alberto F. de Souza
Date: 2018-04-27 00:00:00
Tasks: Efficient Neural Network

We propose the use of deep neural networks (DNN) for solving the problem of
inferring the position and relevant properties of lanes of urban roads with
poor or absent horizontal signalization, in order to allow the operation of
autonomous cars in such situations. We take a segmentation approach to the
problem and use the Efficient Neural Network (ENet) DNN for segmenting LiDAR
remission grid maps into road maps. We represent road maps using what we called
road grid maps. Road grid maps are square matrixes and each element of these
matrixes represents a small square region of real-world space. The value of
each element is a code associated with the semantics of the road map. Our road
grid maps contain all information about the roads' lanes required for building
the Road Definition Data Files (RDDFs) that are necessary for the operation of
our autonomous car, IARA (Intelligent Autonomous Robotic Automobile). We have
built a dataset of tens of kilometers of manually marked road lanes and used
part of it to train ENet to segment road grid maps from remission grid maps.
After being trained, ENet achieved an average segmentation accuracy of 83.7%.
We have tested the use of inferred road grid maps in the real world using IARA
on a stretch of 3.7 km of urban roads and it has shown performance equivalent
to that of the previous IARA's subsystem that uses a manually generated RDDF.
Large-Scale Visual Relationship Understanding
Authors: Ji Zhang, Yannis Kalantidis, Marcus Rohrbach, Manohar Paluri, Ahmed Elgammal, Mohamed Elhoseiny
Date: 2018-04-27 00:00:00
Tasks: Relationship Detection

Large scale visual understanding is challenging, as it requires a model to handle the widely-spread and imbalanced distribution of <subject, relation, object> triples. In real-world scenarios with large numbers of objects and relations, some are seen very commonly while others are barely seen. We develop a new relationship detection model that embeds objects and relations into two vector spaces where both discriminative capability and semantic affinity are preserved. We learn both a visual and a semantic module that map features from the two modalities into a shared space, where matched pairs of features have to discriminate against those unmatched, but also maintain close distances to semantically similar ones. Benefiting from that, our model can achieve superior performance even when the visual entity categories scale up to more than 80,000, with extremely skewed class distribution. We demonstrate the efficacy of our model on a large and imbalanced benchmark based of Visual Genome that comprises 53,000+ objects and 29,000+ relations, a scale at which no previous work has ever been evaluated at. We show superiority of our model over carefully designed baselines on the original Visual Genome dataset with 80,000+ categories. We also show state-of-the-art performance on the VRD dataset and the scene graph dataset which is a subset of Visual Genome with 200 categories.
Sparse Group Inductive Matrix Completion
Authors: Ivan Nazarov, Boris Shirokikh, Maria Burkina, Gennady Fedonin, Maxim Panov
Date: 2018-04-27 00:00:00
Tasks: feature selection, Low-Rank Matrix Completion, Matrix Completion

We consider the problem of matrix completion with side information
(\textit{inductive matrix completion}). In real-world applications many
side-channel features are typically non-informative making feature selection an
important part of the problem. We incorporate feature selection into inductive
matrix completion by proposing a matrix factorization framework with
group-lasso regularization on side feature parameter matrices. We demonstrate,
that the theoretical sample complexity for the proposed method is much lower
compared to its competitors in sparse problems, and propose an efficient
optimization algorithm for the resulting low-rank matrix completion problem
with sparsifying regularizers. Experiments on synthetic and real-world datasets
show that the proposed approach outperforms other methods.