pwc-archive/papers-with-abstracts huggingface dataset:
Nonlinear Collaborative Scheme for Deep Neural Networks
Authors: Hui-Ling Zhen, Xi Lin, Alan Z. Tang, Zhenhua Li, Qingfu Zhang, Sam Kwong
Date: 2018-11-04 00:00:00

Conventional research attributes the improvements of generalization ability
of deep neural networks either to powerful optimizers or the new network
design. Different from them, in this paper, we aim to link the generalization
ability of a deep network to optimizing a new objective function. To this end,
we propose a \textit{nonlinear collaborative scheme} for deep network training,
with the key technique as combining different loss functions in a nonlinear
manner. We find that after adaptively tuning the weights of different loss
functions, the proposed objective function can efficiently guide the
optimization process. What is more, we demonstrate that, from the mathematical
perspective, the nonlinear collaborative scheme can lead to (i) smaller KL
divergence with respect to optimal solutions; (ii) data-driven stochastic
gradient descent; (iii) tighter PAC-Bayes bound. We also prove that its
advantage can be strengthened by nonlinearity increasing. To some extent, we
bridge the gap between learning (i.e., minimizing the new objective function)
and generalization (i.e., minimizing a PAC-Bayes bound) in the new scheme. We
also interpret our findings through the experiments on Residual Networks and
DenseNet, showing that our new scheme performs superior to single-loss and
multi-loss schemes no matter with randomization or not.
Modeling Stated Preference for Mobility-on-Demand Transit: A Comparison of Machine Learning and Logit Models
Authors: Xilei Zhao, Xiang Yan, Alan Yu, Pascal Van Hentenryck
Date: 2018-11-04 00:00:00
Tasks: BIG-bench Machine Learning

Logit models are usually applied when studying individual travel behavior,
i.e., to predict travel mode choice and to gain behavioral insights on traveler
preferences. Recently, some studies have applied machine learning to model
travel mode choice and reported higher out-of-sample predictive accuracy than
traditional logit models (e.g., multinomial logit). However, little research
focuses on comparing the interpretability of machine learning with logit
models. In other words, how to draw behavioral insights from the
high-performance "black-box" machine-learning models remains largely unsolved
in the field of travel behavior modeling.
  This paper aims at providing a comprehensive comparison between the two
approaches by examining the key similarities and differences in model
development, evaluation, and behavioral interpretation between logit and
machine-learning models for travel mode choice modeling. To complement the
theoretical discussions, the paper also empirically evaluates the two
approaches on the stated-preference survey data for a new type of transit
system integrating high-frequency fixed-route services and ridesourcing. The
results show that machine learning can produce significantly higher predictive
accuracy than logit models. Moreover, machine learning and logit models largely
agree on many aspects of behavioral interpretations. In addition, machine
learning can automatically capture the nonlinear relationship between the input
features and choice outcomes. The paper concludes that there is great potential
in merging ideas from machine learning and conventional statistical methods to
develop refined models for travel behavior research and suggests some new
research directions.
Adversarial Black-Box Attacks on Automatic Speech Recognition Systems using Multi-Objective Evolutionary Optimization
Authors: Shreya Khare, Rahul Aralikatte, Senthil Mani
Date: 2018-11-04 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Fooling deep neural networks with adversarial input have exposed a significant vulnerability in the current state-of-the-art systems in multiple domains. Both black-box and white-box approaches have been used to either replicate the model itself or to craft examples which cause the model to fail. In this work, we propose a framework which uses multi-objective evolutionary optimization to perform both targeted and un-targeted black-box attacks on Automatic Speech Recognition (ASR) systems. We apply this framework on two ASR systems: Deepspeech and Kaldi-ASR, which increases the Word Error Rates (WER) of these systems by upto 980%, indicating the potency of our approach. During both un-targeted and targeted attacks, the adversarial samples maintain a high acoustic similarity of 0.98 and 0.97 with the original audio.
Towards Unsupervised Speech-to-Text Translation
Authors: Yu-An Chung, Wei-Hung Weng, Schrasing Tong, James Glass
Date: 2018-11-04 00:00:00
Tasks: Denoising, Language Modeling, Language Modelling, Speech-to-Text, Speech-to-Text Translation, Translation, Word Translation

We present a framework for building speech-to-text translation (ST) systems
using only monolingual speech and text corpora, in other words, speech
utterances from a source language and independent text from a target language.
As opposed to traditional cascaded systems and end-to-end architectures, our
system does not require any labeled data (i.e., transcribed source audio or
parallel source and target text corpora) during training, making it especially
applicable to language pairs with very few or even zero bilingual resources.
The framework initializes the ST system with a cross-modal bilingual dictionary
inferred from the monolingual corpora, that maps every source speech segment
corresponding to a spoken word to its target text translation. For unseen
source speech utterances, the system first performs word-by-word translation on
each speech segment in the utterance. The translation is improved by leveraging
a language model and a sequence denoising autoencoder to provide prior
knowledge about the target language. Experimental results show that our
unsupervised system achieves comparable BLEU scores to supervised end-to-end
models despite the lack of supervision. We also provide an ablation analysis to
examine the utility of each component in our system.
A dataset for benchmarking vision-based localization at intersections
Authors: Augusto L. Ballardini, Daniele Cattaneo, Domenico G. Sorrenti
Date: 2018-11-04 00:00:00
Tasks: Benchmarking

In this report we present the work performed in order to build a dataset for
benchmarking vision-based localization at intersections, i.e., a set of stereo
video sequences taken from a road vehicle that is approaching an intersection,
altogether with a reliable measure of the observer position. This report is
meant to complement our paper "Vision-Based Localization at Intersections using
Digital Maps" submitted to ICRA2019. It complements the paper because the paper
uses the dataset, but it had no space for describing the work done to obtain
it. Moreover, the dataset is of interest for all those tackling the task of
online localization at intersections for road vehicles, e.g., for a
quantitative comparison with the proposal in our submitted paper, and it is
therefore appropriate to put the dataset description in a separate report. We
considered all datasets from road vehicles that we could find as for the end of
August 2018. After our evaluation, we kept only sub-sequences from the KITTI
dataset. In the future we will increase the collection of sequences with data
from our vehicle.
Block-wise Partitioning for Extreme Multi-label Classification
Authors: Yuefeng Liang, Cho-Jui Hsieh, Thomas C. M. Lee
Date: 2018-11-04 00:00:00
Tasks: Classification, Extreme Multi-Label Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Extreme multi-label classification aims to learn a classifier that annotates
an instance with a relevant subset of labels from an extremely large label set.
Many existing solutions embed the label matrix to a low-dimensional linear
subspace, or examine the relevance of a test instance to every label via a
linear scan. In practice, however, those approaches can be computationally
exorbitant. To alleviate this drawback, we propose a Block-wise Partitioning
(BP) pretreatment that divides all instances into disjoint clusters, to each of
which the most frequently tagged label subset is attached. One multi-label
classifier is trained on one pair of instance and label clusters, and the label
set of a test instance is predicted by first delivering it to the most
appropriate instance cluster. Experiments on benchmark multi-label data sets
reveal that BP pretreatment significantly reduces prediction time, and retains
almost the same level of prediction accuracy.
ColNet: Embedding the Semantics of Web Tables for Column Type Prediction
Authors: Jiaoyan Chen, Ernesto Jimenez-Ruiz, Ian Horrocks, Charles Sutton
Date: 2018-11-04 00:00:00
Tasks: Column Type Annotation, Table annotation, Type prediction, Vocal Bursts Type Prediction

Automatically annotating column types with knowledge base (KB) concepts is a
critical task to gain a basic understanding of web tables. Current methods rely
on either table metadata like column name or entity correspondences of cells in
the KB, and may fail to deal with growing web tables with incomplete meta
information. In this paper we propose a neural network based column type
annotation framework named ColNet which is able to integrate KB reasoning and
lookup with machine learning and can automatically train Convolutional Neural
Networks for prediction. The prediction model not only considers the contextual
semantics within a cell using word representation, but also embeds the
semantics of a column by learning locality features from multiple cells. The
method is evaluated with DBPedia and two different web table datasets, T2Dv2
from the general Web and Limaye from Wikipedia pages, and achieves higher
performance than the state-of-the-art approaches.
Adversarial Gain
Authors: Peter Henderson, Koustuv Sinha, Rosemary Nan Ke, Joelle Pineau
Date: 2018-11-04 00:00:00
Tasks: General Classification

Adversarial examples can be defined as inputs to a model which induce a
mistake - where the model output is different than that of an oracle, perhaps
in surprising or malicious ways. Original models of adversarial attacks are
primarily studied in the context of classification and computer vision tasks.
While several attacks have been proposed in natural language processing (NLP)
settings, they often vary in defining the parameters of an attack and what a
successful attack would look like. The goal of this work is to propose a
unifying model of adversarial examples suitable for NLP tasks in both
generative and classification settings. We define the notion of adversarial
gain: based in control theory, it is a measure of the change in the output of a
system relative to the perturbation of the input (caused by the so-called
adversary) presented to the learner. This definition, as we show, can be used
under different feature spaces and distance conditions to determine attack or
defense effectiveness across different intuitive manifolds. This notion of
adversarial gain not only provides a useful way for evaluating adversaries and
defenses, but can act as a building block for future work in robustness under
adversaries due to its rooted nature in stability and manifold theory.
SimplerVoice: A Key Message & Visual Description Generator System for Illiteracy
Authors: Minh N. B. Nguyen, Samuel Thomas, Anne E. Gattiker, Sujatha Kashyap, Kush R. Varshney
Date: 2018-11-03 00:00:00
Tasks: Navigate

We introduce SimplerVoice: a key message and visual description generator
system to help low-literate adults navigate the information-dense world with
confidence, on their own. SimplerVoice can automatically generate sensible
sentences describing an unknown object, extract semantic meanings of the object
usage in the form of a query string, then, represent the string as multiple
types of visual guidance (pictures, pictographs, etc.). We demonstrate
SimplerVoice system in a case study of generating grocery products' manuals
through a mobile application. To evaluate, we conducted a user study on
SimplerVoice's generated description in comparison to the information
interpreted by users from other methods: the original product package and
search engines' top result, in which SimplerVoice achieved the highest
performance score: 4.82 on 5-point mean opinion score scale. Our result shows
that SimplerVoice is able to provide low-literate end-users with simple yet
informative components to help them understand how to use the grocery products,
and that the system may potentially provide benefits in other real-world use
cases
Strategies for Training Stain Invariant CNNs
Authors: Thomas Lampert, Odyssée Merveille, Jessica Schmitz, Germain Forestier, Friedrich Feuerhake, Cédric Wemmert
Date: 2018-10-17 00:00:00
Tasks: whole slide images

An important part of Digital Pathology is the analysis of multiple digitised
whole slide images from differently stained tissue sections. It is common
practice to mount consecutive sections containing corresponding microscopic
structures on glass slides, and to stain them differently to highlight specific
tissue components. These multiple staining modalities result in very different
images but include a significant amount of consistent image information. Deep
learning approaches have recently been proposed to analyse these images in
order to automatically identify objects of interest for pathologists. These
supervised approaches require a vast amount of annotations, which are difficult
and expensive to acquire---a problem that is multiplied with multiple
stainings. This article presents several training strategies that make progress
towards stain invariant networks. By training the network on one commonly used
staining modality and applying it to images that include corresponding but
differently stained tissue structures, the presented unsupervised strategies
demonstrate significant improvements over standard training strategies.
Learning Contextual Hierarchical Structure of Medical Concepts with Poincairé Embeddings to Clarify Phenotypes
Authors: Brett K. Beaulieu-Jones, Isaac S. Kohane, Andrew L. Beam
Date: 2018-11-03 00:00:00
Tasks: BIG-bench Machine Learning, Diagnostic

Biomedical association studies are increasingly done using clinical concepts,
and in particular diagnostic codes from clinical data repositories as
phenotypes. Clinical concepts can be represented in a meaningful, vector space
using word embedding models. These embeddings allow for comparison between
clinical concepts or for straightforward input to machine learning models.
Using traditional approaches, good representations require high dimensionality,
making downstream tasks such as visualization more difficult. We applied
Poincar\'e embeddings in a 2-dimensional hyperbolic space to a large-scale
administrative claims database and show performance comparable to
100-dimensional embeddings in a euclidean space. We then examine disease
relationships under different disease contexts to better understand potential
phenotypes.
Geometry-Aware Recurrent Neural Networks for Active Visual Recognition
Authors: Ricson Cheng, Ziyan Wang, Katerina Fragkiadaki
Date: 2018-11-03 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: 3D Reconstruction, Object, object-detection, Object Detection, Semantic Segmentation

We present recurrent geometry-aware neural networks that integrate visual
information across multiple views of a scene into 3D latent feature tensors,
while maintaining an one-to-one mapping between 3D physical locations in the
world scene and latent feature locations. Object detection, object
segmentation, and 3D reconstruction is then carried out directly using the
constructed 3D feature memory, as opposed to any of the input 2D images. The
proposed models are equipped with differentiable egomotion-aware feature
warping and (learned) depth-aware unprojection operations to achieve
geometrically consistent mapping between the features in the input frame and
the constructed latent model of the scene. We empirically show the proposed
model generalizes much better than geometryunaware LSTM/GRU networks,
especially under the presence of multiple objects and cross-object occlusions.
Combined with active view selection policies, our model learns to select
informative viewpoints to integrate information from by "undoing" cross-object
occlusions, seamlessly combining geometry with learning from experience.
Auto-ML Deep Learning for Rashi Scripts OCR
Authors: Shahar Mahpod, Yosi Keller
Date: 2018-11-03 00:00:00
Tasks: AutoML, Deep Learning, Optical Character Recognition (OCR)

In this work we propose an OCR scheme for manuscripts printed in Rashi font that is an ancient Hebrew font and corresponding dialect used in religious Jewish literature, for more than 600 years. The proposed scheme utilizes a convolution neural network (CNN) for visual inference and Long-Short Term Memory (LSTM) to learn the Rashi scripts dialect. In particular, we derive an AutoML scheme to optimize the CNN architecture, and a book-specific CNN training to improve the OCR accuracy. The proposed scheme achieved an accuracy of more than 99.8% using a dataset of more than 3M annotated letters from the Responsa Project dataset.
Towards Sparse Hierarchical Graph Classifiers
Authors: Cătălina Cangea, Petar Veličković, Nikola Jovanović, Thomas Kipf, Pietro Liò
Date: 2018-11-03 00:00:00
Tasks: Classification, General Classification, Graph Classification, Graph Neural Network, Link Prediction, Node Classification, Representation Learning

Recent advances in representation learning on graphs, mainly leveraging graph
convolutional networks, have brought a substantial improvement on many
graph-based benchmark tasks. While novel approaches to learning node embeddings
are highly suitable for node classification and link prediction, their
application to graph classification (predicting a single label for the entire
graph) remains mostly rudimentary, typically using a single global pooling step
to aggregate node features or a hand-designed, fixed heuristic for hierarchical
coarsening of the graph structure. An important step towards ameliorating this
is differentiable graph coarsening---the ability to reduce the size of the
graph in an adaptive, data-dependent manner within a graph neural network
pipeline, analogous to image downsampling within CNNs. However, the previous
prominent approach to pooling has quadratic memory requirements during training
and is therefore not scalable to large graphs. Here we combine several recent
advances in graph neural network design to demonstrate that competitive
hierarchical graph classification results are possible without sacrificing
sparsity. Our results are verified on several established graph classification
benchmarks, and highlight an important direction for future research in
graph-based neural networks.
Challenges in detecting evolutionary forces in language change using diachronic corpora
Authors: Andres Karjus, Richard A. Blythe, Simon Kirby, Kenny Smith
Date: 2018-11-03 00:00:00
Tasks: Time Series, Time Series Analysis

Newberry et al. (Detecting evolutionary forces in language change, Nature 551, 2017) tackle an important but difficult problem in linguistics, the testing of selective theories of language change against a null model of drift. Having applied a test from population genetics (the Frequency Increment Test) to a number of relevant examples, they suggest stochasticity has a previously under-appreciated role in language evolution. We replicate their results and find that while the overall observation holds, results produced by this approach on individual time series can be sensitive to how the corpus is organized into temporal segments (binning). Furthermore, we use a large set of simulations in conjunction with binning to systematically explore the range of applicability of the Frequency Increment Test. We conclude that care should be exercised with interpreting results of tests like the Frequency Increment Test on individual series, given the researcher degrees of freedom available when applying the test to corpus data, and fundamental differences between genetic and linguistic data. Our findings have implications for selection testing and temporal binning in general, as well as demonstrating the usefulness of simulations for evaluating methods newly introduced to the field.
ReXCam: Resource-Efficient, Cross-Camera Video Analytics at Scale
Authors: Samvit Jain, Xun Zhang, Yuhao Zhou, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Joseph Gonzalez
Date: 2018-11-03 00:00:00

Enterprises are increasingly deploying large camera networks for video analytics. Many target applications entail a common problem template: searching for and tracking an object or activity of interest (e.g. a speeding vehicle, a break-in) through a large camera network in live video. Such cross-camera analytics is compute and data intensive, with cost growing with the number of cameras and time. To address this cost challenge, we present ReXCam, a new system for efficient cross-camera video analytics. ReXCam exploits spatial and temporal locality in the dynamics of real camera networks to guide its inference-time search for a query identity. In an offline profiling phase, ReXCam builds a cross-camera correlation model that encodes the locality observed in historical traffic patterns. At inference time, ReXCam applies this model to filter frames that are not spatially and temporally correlated with the query identity's current position. In the cases of occasional missed detections, ReXCam performs a fast-replay search on recently filtered video frames, enabling gracefully recovery. Together, these techniques allow ReXCam to reduce compute workload by 8.3x on an 8-camera dataset, and by 23x - 38x on a simulated 130-camera dataset. ReXCam has been implemented and deployed on a testbed of 5 AWS DeepLens cameras.
Legible Normativity for AI Alignment: The Value of Silly Rules
Authors: Dylan Hadfield-Menell, McKane Andrus, Gillian K. Hadfield
Date: 2018-11-03 00:00:00

It has become commonplace to assert that autonomous agents will have to be
built to follow human rules of behavior--social norms and laws. But human laws
and norms are complex and culturally varied systems, in many cases agents will
have to learn the rules. This requires autonomous agents to have models of how
human rule systems work so that they can make reliable predictions about rules.
In this paper we contribute to the building of such models by analyzing an
overlooked distinction between important rules and what we call silly
rules--rules with no discernible direct impact on welfare. We show that silly
rules render a normative system both more robust and more adaptable in response
to shocks to perceived stability. They make normativity more legible for
humans, and can increase legibility for AI systems as well. For AI systems to
integrate into human normative systems, we suggest, it may be important for
them to have models that include representations of silly rules.