pwc-archive/papers-with-abstracts huggingface dataset:
Artificial neural networks condensation: A strategy to facilitate adaption of machine learning in medical settings by reducing computational burden
Authors: Dianbo Liu, Nestor Sepulveda, Ming Zheng
Date: 2018-12-23 00:00:00
Tasks: Computational Efficiency, Mortality Prediction, Quantization

Machine Learning (ML) applications on healthcare can have a great impact on
people's lives helping deliver better and timely treatment to those in need. At
the same time, medical data is usually big and sparse requiring important
computational resources. Although it might not be a problem for wide-adoption
of ML tools in developed nations, availability of computational resource can
very well be limited in third-world nations. This can prevent the less favored
people from benefiting of the advancement in ML applications for healthcare. In
this project we explored methods to increase computational efficiency of ML
algorithms, in particular Artificial Neural Nets (NN), while not compromising
the accuracy of the predicted results. We used in-hospital mortality prediction
as our case analysis based on the MIMIC III publicly available dataset. We
explored three methods on two different NN architectures. We reduced the size
of recurrent neural net (RNN) and dense neural net (DNN) by applying pruning of
"unused" neurons. Additionally, we modified the RNN structure by adding a
hidden-layer to the LSTM cell allowing to use less recurrent layers for the
model. Finally, we implemented quantization on DNN forcing the weights to be
8-bits instead of 32-bits. We found that all our methods increased
computational efficiency without compromising accuracy and some of them even
achieved higher accuracy than the pre-condensed baseline models.
Learning finite-dimensional coding schemes with nonlinear reconstruction maps
Authors: Jaeho Lee, Maxim Raginsky
Date: 2018-12-23 00:00:00
Tasks: Generalization Bounds, Representation Learning

This paper generalizes the Maurer--Pontil framework of finite-dimensional
lossy coding schemes to the setting where a high-dimensional random vector is
mapped to an element of a compact set of latent representations in a
lower-dimensional Euclidean space, and the reconstruction map belongs to a
given class of nonlinear maps. Under this setup, which encompasses a broad
class of unsupervised representation learning problems, we establish a
connection to approximate generative modeling under structural constraints
using the tools from the theory of optimal transportation. Next, we consider
problem of learning a coding scheme on the basis of a finite collection of
training samples and present generalization bounds that hold with high
probability. We then illustrate the general theory in the setting where the
reconstruction maps are implemented by deep neural nets.
Supervised Sentiment Classification with CNNs for Diverse SE Datasets
Authors: Achyudh Ram, Meiyappan Nagappan
Date: 2018-12-23 00:00:00
Tasks: Classification, General Classification, Opinion Mining, Sentiment Analysis, Sentiment Classification

Sentiment analysis, a popular technique for opinion mining, has been used by
the software engineering research community for tasks such as assessing app
reviews, developer emotions in issue trackers and developer opinions on APIs.
Past research indicates that state-of-the-art sentiment analysis techniques
have poor performance on SE data. This is because sentiment analysis tools are
often designed to work on non-technical documents such as movie reviews. In
this study, we attempt to solve the issues with existing sentiment analysis
techniques for SE texts by proposing a hierarchical model based on
convolutional neural networks (CNN) and long short-term memory (LSTM) trained
on top of pre-trained word vectors. We assessed our model's performance and
reliability by comparing it with a number of frequently used sentiment analysis
tools on five gold standard datasets. Our results show that our model pushes
the state of the art further on all datasets in terms of accuracy. We also show
that it is possible to get better accuracy after labelling a small sample of
the dataset and re-training our model rather than using an unsupervised
classifier.
A Cross-Architecture Instruction Embedding Model for Natural Language Processing-Inspired Binary Code Analysis
Authors: Kimberly Redmond, Lannan Luo, Qiang Zeng
Date: 2018-12-23 00:00:00
Tasks: Malware Analysis

Given a closed-source program, such as most of proprietary software and
viruses, binary code analysis is indispensable for many tasks, such as code
plagiarism detection and malware analysis. Today, source code is very often
compiled for various architectures, making cross-architecture binary code
analysis increasingly important. A binary, after being disassembled, is
expressed in an assembly languages. Thus, recent work starts exploring Natural
Language Processing (NLP) inspired binary code analysis. In NLP, words are
usually represented in high-dimensional vectors (i.e., embeddings) to
facilitate further processing, which is one of the most common and critical
steps in many NLP tasks. We regard instructions as words in NLP-inspired binary
code analysis, and aim to represent instructions as embeddings as well.
  To facilitate cross-architecture binary code analysis, our goal is that
similar instructions, regardless of their architectures, have embeddings close
to each other. To this end, we propose a joint learning approach to generating
instruction embeddings that capture not only the semantics of instructions
within an architecture, but also their semantic relationships across
architectures. To the best of our knowledge, this is the first work on building
cross-architecture instruction embedding model. As a showcase, we apply the
model to resolving one of the most fundamental problems for binary code
similarity comparison---semantics-based basic block comparison, and the
solution outperforms the code statistics based approach. It demonstrates that
it is promising to apply the model to other cross-architecture binary code
analysis tasks.
Improving Context-Aware Semantic Relationships in Sparse Mobile Datasets
Authors: Peter Hansel, Nik Marda, William Yin
Date: 2018-12-23 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Embeddings

Traditional semantic similarity models often fail to encapsulate the external
context in which texts are situated. However, textual datasets generated on
mobile platforms can help us build a truer representation of semantic
similarity by introducing multimodal data. This is especially important in
sparse datasets, making solely text-driven interpretation of context more
difficult. In this paper, we develop new algorithms for building external
features into sentence embeddings and semantic similarity scores. Then, we test
them on embedding spaces on data from Twitter, using each tweet's time and
geolocation to better understand its context. Ultimately, we show that applying
PCA with eight components to the embedding space and appending multimodal
features yields the best outcomes. This yields a considerable improvement over
pure text-based approaches for discovering similar tweets. Our results suggest
that our new algorithm can help improve semantic understanding in various
settings.
Chinese Herbal Recognition based on Competitive Attentional Fusion of Multi-hierarchies Pyramid Features
Authors: Yingxue Xu, Guihua Wen, Yang Hu, Mingnan Luo, Dan Dai, Yishan Zhuang
Date: 2018-12-23 00:00:00

Convolution neural netwotks (CNNs) are successfully applied in image
recognition task. In this study, we explore the approach of automatic herbal
recognition with CNNs and build the standard Chinese herbs datasets firstly.
According to the characteristics of herbal images, we proposed the competitive
attentional fusion pyramid networks to model the features of herbal image,
which mdoels the relationship of feature maps from different levels, and
re-weights multi-level channels with channel-wise attention mechanism. In this
way, we can dynamically adjust the weight of feature maps from various layers,
according to the visual characteristics of each herbal image. Moreover, we also
introduce the spatial attention to recalibrate the misaligned features caused
by sampling in features amalgamation. Extensive experiments are conducted on
our proposed datasets and validate the superior performance of our proposed
models. The Chinese herbs datasets will be released upon acceptance to
facilitate the research of Chinese herbal recognition.
Mixed Membership Recurrent Neural Networks
Authors: Ghazal Fazelnia, Mark Ibrahim, Ceena Modarres, Kevin Wu, John Paisley
Date: 2018-12-23 00:00:00
Tasks: Dynamic Topic Modeling

Models for sequential data such as the recurrent neural network (RNN) often
implicitly model a sequence as having a fixed time interval between
observations and do not account for group-level effects when multiple sequences
are observed. We propose a model for grouped sequential data based on the RNN
that accounts for varying time intervals between observations in a sequence by
learning a group-level base parameter to which each sequence can revert. Our
approach is motivated by the mixed membership framework, and we show how it can
be used for dynamic topic modeling in which the distribution on topics (not the
topics themselves) are evolving in time. We demonstrate our approach on a
dataset of 3.4 million online grocery shopping orders made by 206K customers.
Estimating Rationally Inattentive Utility Functions with Deep Clustering for Framing - Applications in YouTube Engagement Dynamics
Authors: William Hoiles, Vikram Krishnamurthy
Date: 2018-12-23 00:00:00
Tasks: Clustering, Deep Clustering, Reinforcement Learning

We consider a framework involving behavioral economics and machine learning.
Rationally inattentive Bayesian agents make decisions based on their posterior
distribution, utility function and information acquisition cost Renyi
divergence which generalizes Shannon mutual information). By observing these
decisions, how can an observer estimate the utility function and information
acquisition cost? Using deep learning, we estimate framing information
(essential extrinsic features) that determines the agent's attention strategy.
Then we present a preference based inverse reinforcement learning algorithm to
test for rational inattention: is the agent an utility maximizer, attention
maximizer, and does an information cost function exist that rationalizes the
data? The test imposes a Renyi mutual information constraint which impacts how
the agent can select attention strategies to maximize their expected utility.
The test provides constructive estimates of the utility function and
information acquisition cost of the agent. We illustrate these methods on a
massive YouTube dataset for characterizing the commenting behavior of users.
Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching
Authors: Chih-Kuan Yeh, Jianshu Chen, Chengzhu Yu, Dong Yu
Date: 2018-12-23 00:00:00
Proceeding: ICLR 2019 5
Tasks: Language Modeling, Language Modelling, Phoneme Recognition, speech-recognition, Speech Recognition, Unsupervised Speech Recognition

We consider the problem of training speech recognition systems without using
any labeled data, under the assumption that the learner can only access to the
input utterances and a phoneme language model estimated from a non-overlapping
corpus. We propose a fully unsupervised learning algorithm that alternates
between solving two sub-problems: (i) learn a phoneme classifier for a given
set of phoneme segmentation boundaries, and (ii) refining the phoneme
boundaries based on a given classifier. To solve the first sub-problem, we
introduce a novel unsupervised cost function named Segmental Empirical Output
Distribution Matching, which generalizes the work in (Liu et al., 2017) to
segmental structures. For the second sub-problem, we develop an approximate MAP
approach to refining the boundaries obtained from Wang et al. (2017).
Experimental results on TIMIT dataset demonstrate the success of this fully
unsupervised phoneme recognition system, which achieves a phone error rate
(PER) of 41.6%. Although it is still far away from the state-of-the-art
supervised systems, we show that with oracle boundaries and matching language
model, the PER could be improved to 32.5%.This performance approaches the
supervised system of the same model architecture, demonstrating the great
potential of the proposed method.
Distribution-Free Uncertainty Quantification for Kernel Methods by Gradient Perturbations
Authors: Balázs Csanád Csáji, Krisztián Balázs Kis
Date: 2018-12-23 00:00:00
Tasks: Gaussian Processes, Uncertainty Quantification

We propose a data-driven approach to quantify the uncertainty of models constructed by kernel methods. Our approach minimizes the needed distributional assumptions, hence, instead of working with, for example, Gaussian processes or exponential families, it only requires knowledge about some mild regularity of the measurement noise, such as it is being symmetric or exchangeable. We show, by building on recent results from finite-sample system identification, that by perturbing the residuals in the gradient of the objective function, information can be extracted about the amount of uncertainty our model has. Particularly, we provide an algorithm to build exact, non-asymptotically guaranteed, distribution-free confidence regions for ideal, noise-free representations of the function we try to estimate. For the typical convex quadratic problems and symmetric noises, the regions are star convex centered around a given nominal estimate, and have efficient ellipsoidal outer approximations. Finally, we illustrate the ideas on typical kernel methods, such as LS-SVC, KRR, $\varepsilon$-SVR and kernelized LASSO.
Estimation and Restoration of Compositional Degradation Using Convolutional Neural Networks
Authors: Kazutaka Uchida, Masayuki Tanaka, Masatoshi Okutomi
Date: 2018-12-23 00:00:00
Tasks: Image Restoration

Image restoration from a single image degradation type, such as blurring,
hazing, random noise, and compression has been investigated for decades.
However, image degradations in practice are often a mixture of several types of
degradation. Such compositional degradations complicate restoration because
they require the differentiation of different degradation types and levels. In
this paper, we propose a convolutional neural network (CNN) model for
estimating the degradation properties of a given degraded image. Furthermore,
we introduce an image restoration CNN model that adopts the estimated
degradation properties as its input. Experimental results show that the
proposed degradation estimation model can successfully infer the degradation
properties of compositionally degraded images. The proposed restoration model
can restore degraded images by exploiting the estimated degradation properties
and can achieve both blind and nonblind image restorations.
Can VAEs Generate Novel Examples?
Authors: Alican Bozkurt, Babak Esmaeili, Dana H. Brooks, Jennifer G. Dy, Jan-Willem van de Meent
Date: 2018-12-22 00:00:00
Tasks: Decoder

An implicit goal in works on deep generative models is that such models
should be able to generate novel examples that were not previously seen in the
training data. In this paper, we investigate to what extent this property holds
for widely employed variational autoencoder (VAE) architectures. VAEs maximize
a lower bound on the log marginal likelihood, which implies that they will in
principle overfit the training data when provided with a sufficiently
expressive decoder. In the limit of an infinite capacity decoder, the optimal
generative model is a uniform mixture over the training data. More generally,
an optimal decoder should output a weighted average over the examples in the
training data, where the magnitude of the weights is determined by the
proximity in the latent space. This leads to the hypothesis that, for a
sufficiently high capacity encoder and decoder, the VAE decoder will perform
nearest-neighbor matching according to the coordinates in the latent space. To
test this hypothesis, we investigate generalization on the MNIST dataset. We
consider both generalization to new examples of previously seen classes, and
generalization to the classes that were withheld from the training set. In both
cases, we find that reconstructions are closely approximated by nearest
neighbors for higher-dimensional parameterizations. When generalizing to unseen
classes however, lower-dimensional parameterizations offer a clear advantage.
Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification
Authors: Mozhi Zhang, Yoshinari Fujinuma, Jordan Boyd-Graber
Date: 2018-12-22 00:00:00
Tasks: Classification, Cross-Lingual Document Classification, Document Classification, General Classification, text-classification, Text Classification, Transfer Learning

Text classification must sometimes be applied in a low-resource language with no labeled training data. However, training data may be available in a related language. We investigate whether character-level knowledge transfer from a related language helps text classification. We present a cross-lingual document classification framework (CACO) that exploits cross-lingual subword similarity by jointly training a character-based embedder and a word-based classifier. The embedder derives vector representations for input words from their written forms, and the classifier makes predictions based on the word vectors. We use a joint character representation for both the source language and the target language, which allows the embedder to generalize knowledge about source language words to target language words with similar forms. We propose a multi-task objective that can further improve the model if additional cross-lingual or monolingual resources are available. Experiments confirm that character-level knowledge transfer is more data-efficient than word-level transfer between related languages.
Search-Guided, Lightly-supervised Training of Structured Prediction Energy Networks
Authors: Amirmohammad Rooshenas, Dongxu Zhang, Gopal Sharma, Andrew McCallum
Date: 2018-12-22 00:00:00
Tasks: Prediction, Structured Prediction

In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching through the entire output space to find the best output with respect to this reward function is typically intractable. In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction. In particular, this truncated randomized search in the reward function yields previously unknown local improvements, providing effective supervision to SPENs, avoiding their traditional need for labeled training data.
Inference and Sampling of $K_{33}$-free Ising Models
Authors: Valerii Likhosherstov, Yury Maximov, Michael Chertkov
Date: 2018-12-22 00:00:00

We call an Ising model tractable when it is possible to compute its partition function value (statistical inference) in polynomial time. The tractability also implies an ability to sample configurations of this model in polynomial time. The notion of tractability extends the basic case of planar zero-field Ising models. Our starting point is to describe algorithms for the basic case computing partition function and sampling efficiently. To derive the algorithms, we use an equivalent linear transition to perfect matching counting and sampling on an expanded dual graph. Then, we extend our tractable inference and sampling algorithms to models, whose triconnected components are either planar or graphs of $O(1)$ size. In particular, it results in a polynomial-time inference and sampling algorithms for $K_{33}$ (minor) free topologies of zero-field Ising models - a generalization of planar graphs with a potentially unbounded genus.
Meta Architecture Search
Authors: Albert Shaw, Wei Wei, Weiyang Liu, Le Song, Bo Dai
Date: 2018-12-22 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Bayesian Inference, Few-Shot Learning, GPU, Meta-Learning, Neural Architecture Search

Neural Architecture Search (NAS) has been quite successful in constructing state-of-the-art models on a variety of tasks. Unfortunately, the computational cost can make it difficult to scale. In this paper, we make the first attempt to study Meta Architecture Search which aims at learning a task-agnostic representation that can be used to speed up the process of architecture search on a large number of tasks. We propose the Bayesian Meta Architecture SEarch (BASE) framework which takes advantage of a Bayesian formulation of the architecture search problem to learn over an entire set of tasks simultaneously. We show that on Imagenet classification, we can find a model that achieves 25.7% top-1 error and 8.1% top-5 error by adapting the architecture in less than an hour from an 8 GPU days pretrained meta-network. By learning a good prior for NAS, our method dramatically decreases the required computation cost while achieving comparable performance to current state-of-the-art methods - even finding competitive models for unseen datasets with very quick adaptation. We believe our framework will open up new possibilities for efficient and massively scalable architecture search research across multiple tasks.
EgoReID Dataset: Person Re-identification in Videos Acquired by Mobile Devices with First-Person Point-of-View
Authors: Emrah Basaran, Yonatan Tariku Tesfaye, Mubarak Shah
Date: 2018-12-22 00:00:00
Tasks: Person Re-Identification, Video-Based Person Re-Identification

In recent years, we have seen the performance of video-based person Re-Identification (ReID) methods have improved considerably. However, most of the work in this area has dealt with videos acquired by fixed cameras with wider field of view. Recently, widespread use of wearable cameras and recording devices such as cellphones have opened the door to interesting research in first-person Point-of-view (POV) videos (egocentric videos). Nonetheless, analysis of such videos is challenging due to factors such as poor video quality due to ego-motion, blurriness, severe changes in lighting conditions and perspective distortions. To facilitate the research towards conquering these challenges, this paper contributes a new dataset called EgoReID. The dataset is captured using 3 mobile cellphones with non-overlapping field-of-view. It contains 900 IDs and around 10,200 tracks with a total of 176,000 detections. The dataset also contains 12-sensor meta data e.g. camera orientation pitch and rotation for each video. In addition, we propose a new framework which takes advantage of both visual and sensor meta data to successfully perform Person ReID. We extend image-based re-ID method employing human body parsing trained on ten datasets to video-based re-ID. In our method, first frame level local features are extracted for each semantic region, then 3D convolutions are applied to encode the temporal information in each sequence of semantic regions. Additionally, we employ sensor meta data to predict targets' next camera and their estimated time of arrival, which considerably improves our ReID performance as it significantly reduces our search space.