pwc-archive/papers-with-abstracts huggingface dataset:
Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces
Authors: Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi
Date: 2018-08-22 00:00:00

Statistical neurodynamics studies macroscopic behaviors of randomly connected
neural networks. We consider a deep layered feedforward network where input
signals are processed layer by layer. The manifold of input signals is embedded
in a higher dimensional manifold of the next layer as a curved submanifold,
provided the number of neurons is larger than that of inputs. We show
geometrical features of the embedded manifold, proving that the manifold
enlarges or shrinks locally isotropically so that it is always embedded
conformally. We study the curvature of the embedded manifold. The scalar
curvature converges to a constant or diverges to infinity slowly. The distance
between two signals also changes, converging eventually to a stable fixed
value, provided both the number of neurons in a layer and the number of layers
tend to infinity. This causes a problem, since when we consider a curve in the
input space, it is mapped as a continuous curve of fractal nature, but our
theory contradictorily suggests that the curve eventually converges to a
discrete set of equally spaced points. In reality, the numbers of neurons and
layers are finite and thus, it is expected that the finite size effect causes
the discrepancies between our theory and reality. We need to further study the
discrepancies to understand their implications on information processing.
On Deep Neural Networks for Detecting Heart Disease
Authors: Nathalie-Sofia Tomov, Stanimire Tomov
Date: 2018-08-22 00:00:00

Heart disease is the leading cause of death, and experts estimate that
approximately half of all heart attacks and strokes occur in people who have
not been flagged as "at risk." Thus, there is an urgent need to improve the
accuracy of heart disease diagnosis. To this end, we investigate the potential
of using data analysis, and in particular the design and use of deep neural
networks (DNNs) for detecting heart disease based on routine clinical data. Our
main contribution is the design, evaluation, and optimization of DNN
architectures of increasing depth for heart disease diagnosis. This work led to
the discovery of a novel five layer DNN architecture - named Heart Evaluation
for Algorithmic Risk-reduction and Optimization Five (HEARO-5) -- that yields
best prediction accuracy. HEARO-5's design employs regularization optimization
and automatically deals with missing data and/or data outliers. To evaluate and
tune the architectures we use k-way cross-validation as well as Matthews
correlation coefficient (MCC) to measure the quality of our classifications.
The study is performed on the publicly available Cleveland dataset of medical
information, and we are making our developments open source, to further
facilitate openness and research on the use of DNNs in medicine. The HEARO-5
architecture, yielding 99% accuracy and 0.98 MCC, significantly outperforms
currently published research in the area.
Deciding the status of controversial phonemes using frequency distributions; an application to semiconsonants in Spanish
Authors: Manuel Ortega-Rodríguez, Hugo Solís-Sánchez, Ricardo Gamboa-Alfaro
Date: 2018-08-22 00:00:00

Exploiting the fact that natural languages are complex systems, the present
exploratory article proposes a direct method based on frequency distributions
that may be useful when making a decision on the status of problematic
phonemes, an open problem in linguistics. The main notion is that natural
languages, which can be considered from a complex outlook as information
processing machines, and which somehow manage to set appropriate levels of
redundancy, already "made the choice" whether a linguistic unit is a phoneme or
not, and this would be reflected in a greater smoothness in a frequency versus
rank graph. For the particular case we chose to study, we conclude that it is
reasonable to consider the Spanish semiconsonant /w/ as a separate phoneme from
its vowel counterpart /u/, on the one hand, and possibly also the semiconsonant
/j/ as a separate phoneme from its vowel counterpart /i/, on the other. As
language has been so central a topic in the study of complexity, this
discussion grants us, in addition, an opportunity to gain insight into emerging
properties in the broader complex systems debate.
Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning
Authors: Matthias J. Ehrhardt, Pawel Markiewicz, Carola-Bibiane Schönlieb
Date: 2018-08-21 00:00:00

Uncompressed clinical data from modern positron emission tomography (PET) scanners are very large, exceeding 350 million data points (projection bins). The last decades have seen tremendous advancements in mathematical imaging tools many of which lead to non-smooth (i.e. non-differentiable) optimization problems which are much harder to solve than smooth optimization problems. Most of these tools have not been translated to clinical PET data, as the state-of-the-art algorithms for non-smooth problems do not scale well to large data. In this work, inspired by big data machine learning applications, we use advanced randomized optimization algorithms to solve the PET reconstruction problem for a very large class of non-smooth priors which includes for example total variation, total generalized variation, directional total variation and various different physical constraints. The proposed algorithm randomly uses subsets of the data and only updates the variables associated with these. While this idea often leads to divergent algorithms, we show that the proposed algorithm does indeed converge for any proper subset selection. Numerically, we show on real PET data (FDG and florbetapir) from a Siemens Biograph mMR that about ten projections and backprojections are sufficient to solve the MAP optimisation problem related to many popular non-smooth priors; thus showing that the proposed algorithm is fast enough to bring these models into routine clinical practice.
Language Identification in Code-Mixed Data using Multichannel Neural Networks and Context Capture
Authors: Soumil Mandal, Anil Kumar Singh
Date: 2018-08-21 00:00:00
Proceeding: WS 2018 11
Tasks: Language Identification

An accurate language identification tool is an absolute necessity for
building complex NLP systems to be used on code-mixed data. Lot of work has
been recently done on the same, but there's still room for improvement.
Inspired from the recent advancements in neural network architectures for
computer vision tasks, we have implemented multichannel neural networks
combining CNN and LSTM for word level language identification of code-mixed
data. Combining this with a Bi-LSTM-CRF context capture module, accuracies of
93.28% and 93.32% is achieved on our two testing sets.
Improving Super-Resolution Methods via Incremental Residual Learning
Authors: Muneeb Aadil, Rafia Rahim, Sibt Ul Hussain
Date: 2018-08-21 00:00:00
Tasks: Super-Resolution

Recently, Convolutional Neural Networks (CNNs) have shown promising performance in super-resolution (SR). However, these methods operate primarily on Low Resolution (LR) inputs for memory efficiency but this limits, as we demonstrate, their ability to (i) model high frequency information; and (ii) smoothly translate from LR to High Resolution (HR) space. To this end, we propose a novel Incremental Residual Learning (IRL) framework to address these mentioned issues. In IRL, first we select a typical SR pre-trained network as a master branch. Next we sequentially train and add residual branches to the main branch, where each residual branch is learned to model accumulated residuals of all previous branches. We plug state of the art methods in IRL framework and demonstrate consistent performance improvement on public benchmark datasets to set a new state of the art for SR at only approximately 20% increase in training time.
Non-asymptotic bounds for sampling algorithms without log-concavity
Authors: Mateusz B. Majka, Aleksandar Mijatović, Lukasz Szpruch
Date: 2018-08-21 00:00:00

Discrete time analogues of ergodic stochastic differential equations (SDEs) are one of the most popular and flexible tools for sampling high-dimensional probability measures. Non-asymptotic analysis in the $L^2$ Wasserstein distance of sampling algorithms based on Euler discretisations of SDEs has been recently developed by several authors for log-concave probability distributions. In this work we replace the log-concavity assumption with a log-concavity at infinity condition. We provide novel $L^2$ convergence rates for Euler schemes, expressed explicitly in terms of problem parameters. From there we derive non-asymptotic bounds on the distance between the laws induced by Euler schemes and the invariant laws of SDEs, both for schemes with standard and with randomised (inaccurate) drifts. We also obtain bounds for the hierarchy of discretisation, which enables us to deploy a multi-level Monte Carlo estimator. Our proof relies on a novel construction of a coupling for the Markov chains that can be used to control both the $L^1$ and $L^2$ Wasserstein distances simultaneously. Finally, we provide a weak convergence analysis that covers both the standard and the randomised (inaccurate) drift case. In particular, we reveal that the variance of the randomised drift does not influence the rate of weak convergence of the Euler scheme to the SDE.
Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner
Authors: Yury Zemlyanskiy, Fei Sha
Date: 2018-08-21 00:00:00
Proceeding: CONLL 2018 10

There have been several attempts to define a plausible motivation for a
chit-chat dialogue agent that can lead to engaging conversations. In this work,
we explore a new direction where the agent specifically focuses on discovering
information about its interlocutor. We formalize this approach by defining a
quantitative metric. We propose an algorithm for the agent to maximize it. We
validate the idea with human evaluation where our system outperforms various
baselines. We demonstrate that the metric indeed correlates with the human
judgments of engagingness.
Three Efficient, Low-Complexity Algorithms for Automatic Color Trapping
Authors: Haiyin Wang, Mireille Boutin, Jeffrey Trask, Jan Allebach
Date: 2018-08-21 00:00:00

Color separations (most often cyan, magenta, yellow, and black) are commonly
used in printing to reproduce multi-color images. For mechanical reasons, these
color separations are generally not perfectly aligned with respect to each
other when they are rendered by their respective imaging stations. This
phenomenon, called color plane misregistration, causes gap and halo artifacts
in the printed image. Color trapping is an image processing technique that aims
to reduce these artifacts by modifying the susceptible edge boundaries to
create small, unnoticeable overlaps between the color planes. We propose three
low-complexity algorithms for automatic color trapping which hide the effects
of small color plane mis-registrations. Our algorithms are designed for
software or embedded firmware implementation. The trapping method they follow
is based on a hardware-friendly technique proposed by J. Trask (JTHBCT03) which
is too computationally expensive for software or firmware implementation. The
first two algorithms are based on the use of look-up tables (LUTs). The first
LUT-based algorithm corrects all registration errors of one pixel in extent and
reduces several cases of misregistration errors of two pixels in extent using
only 727 Kbytes of storage space. This algorithm is particularly attractive for
implementation in the embedded firmware of low-cost formatter-based printers.
The second LUT-based algorithm corrects all types of misregistration errors of
up to two pixels in extent using 3.7 Mbytes of storage space. The third
algorithm is a hybrid one that combines LUTs and feature extraction to minimize
the storage requirements (724 Kbytes) while still correcting all
misregistration errors of up to two pixels in extent. This algorithm is
suitable for both embedded firmware implementation on low-cost formatter-based
printers and software implementation on host-based printers.
CoBaR: Confidence-Based Recommender
Authors: Fernando S. Aguiar Neto, Arthur F. da Costa, Marcelo G. Manzato
Date: 2018-08-21 00:00:00
Tasks: Clustering, Collaborative Filtering

Neighborhood-based collaborative filtering algorithms usually adopt a fixed
neighborhood size for every user or item, although groups of users or items may
have different lengths depending on users' preferences. In this paper, we
propose an extension to a non-personalized recommender based on confidence
intervals and hierarchical clustering to generate groups of users with optimal
sizes. The evaluation shows that the proposed technique outperformed the
traditional recommender algorithms in four publicly available datasets.
Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss
Authors: Rishi Sharma, Amir Barati Farimani, Joe Gomes, Peter Eastman, Vijay Pande
Date: 2018-07-24 00:00:00
Tasks: BIG-bench Machine Learning

In typical machine learning tasks and applications, it is necessary to obtain
or create large labeled datasets in order to to achieve high performance.
Unfortunately, large labeled datasets are not always available and can be
expensive to source, creating a bottleneck towards more widely applicable
machine learning. The paradigm of weak supervision offers an alternative that
allows for integration of domain-specific knowledge by enforcing constraints
that a correct solution to the learning problem will obey over the output
space. In this work, we explore the application of this paradigm to 2-D
physical systems governed by non-linear differential equations. We demonstrate
that knowledge of the partial differential equations governing a system can be
encoded into the loss function of a neural network via an appropriately chosen
convolutional kernel. We demonstrate this by showing that the steady-state
solution to the 2-D heat equation can be learned directly from initial
conditions by a convolutional neural network, in the absence of labeled
training data. We also extend recent work in the progressive growing of fully
convolutional networks to achieve high accuracy (< 1.5% error) at multiple
scales of the heat-flow problem, including at the very large scale (1024x1024).
Finally, we demonstrate that this method can be used to speed up exact
calculation of the solution to the differential equations via finite
difference.
The What, the Why, and the How of Artificial Explanations in Automated Decision-Making
Authors: Tarek R. Besold, Sara L. Uckelman
Date: 2018-08-21 00:00:00
Tasks: Decision Making

The increasing incorporation of Artificial Intelligence in the form of
automated systems into decision-making procedures highlights not only the
importance of decision theory for automated systems but also the need for these
decision procedures to be explainable to the people involved in them.
Traditional realist accounts of explanation, wherein explanation is a relation
that holds (or does not hold) eternally between an explanans and an
explanandum, are not adequate to account for the notion of explanation required
for artificial decision procedures. We offer an alternative account of
explanation as used in the context of automated decision-making that makes
explanation an epistemic phenomenon, and one that is dependent on context. This
account of explanation better accounts for the way that we talk about, and use,
explanations and derived concepts, such as `explanatory power', and also allows
us to differentiate between reasons or causes on the one hand, which do not
need to have an epistemic aspect, and explanations on the other, which do have
such an aspect. Against this theoretical backdrop we then review existing
approaches to explanation in Artificial Intelligence and Machine Learning, and
suggest desiderata which truly explainable decision systems should fulfill.
Machine learning non-local correlations
Authors: Askery Canabarro, Samuraí Brito, Rafael Chaves
Date: 2018-08-21 00:00:00
Tasks: BIG-bench Machine Learning

The ability to witness non-local correlations lies at the core of
foundational aspects of quantum mechanics and its application in the processing
of information. Commonly, this is achieved via the violation of Bell
inequalities. Unfortunately, however, their systematic derivation quickly
becomes unfeasible as the scenario of interest grows in complexity. To cope
with that, we propose here a machine learning approach for the detection and
quantification of non-locality. It consists of an ensemble of multilayer
perceptrons blended with genetic algorithms achieving a high performance in a
number of relevant Bell scenarios. Our results offer a novel method and a
proof-of-principle for the relevance of machine learning for understanding
non-locality.
Isometric Transformation Invariant Graph-based Deep Neural Network
Authors: Renata Khasanova, Pascal Frossard
Date: 2018-08-21 00:00:00
Tasks: General Classification, Translation, Video Classification

Learning transformation invariant representations of visual data is an
important problem in computer vision. Deep convolutional networks have
demonstrated remarkable results for image and video classification tasks.
However, they have achieved only limited success in the classification of
images that undergo geometric transformations. In this work we present a novel
Transformation Invariant Graph-based Network (TIGraNet), which learns
graph-based features that are inherently invariant to isometric transformations
such as rotation and translation of input images. In particular, images are
represented as signals on graphs, which permits to replace classical
convolution and pooling layers in deep networks with graph spectral convolution
and dynamic graph pooling layers that together contribute to invariance to
isometric transformation. Our experiments show high performance on rotated and
translated images from the test set compared to classical architectures that
are very sensitive to transformations in the data. The inherent invariance
properties of our framework provide key advantages, such as increased
resiliency to data variability and sustained performance with limited training
sets. Our code is available online.
Vicious Circle Principle and Logic Programs with Aggregates
Authors: Michael Gelfond, Yuanlin Zhang
Date: 2018-08-21 00:00:00

The paper presents a knowledge representation language $\mathcal{A}log$ which
extends ASP with aggregates. The goal is to have a language based on simple
syntax and clear intuitive and mathematical semantics. We give some properties
of $\mathcal{A}log$, an algorithm for computing its answer sets, and comparison
with other approaches.
Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networks
Authors: Charith Mendis, Alex Renda, Saman Amarasinghe, Michael Carbin
Date: 2018-08-21 00:00:00

Predicting the number of clock cycles a processor takes to execute a block of assembly instructions in steady state (the throughput) is important for both compiler designers and performance engineers. Building an analytical model to do so is especially complicated in modern x86-64 Complex Instruction Set Computer (CISC) machines with sophisticated processor microarchitectures in that it is tedious, error prone, and must be performed from scratch for each processor generation. In this paper we present Ithemal, the first tool which learns to predict the throughput of a set of instructions. Ithemal uses a hierarchical LSTM--based approach to predict throughput based on the opcodes and operands of instructions in a basic block. We show that Ithemal is more accurate than state-of-the-art hand-written tools currently used in compiler backends and static machine code analyzers. In particular, our model has less than half the error of state-of-the-art analytical models (LLVM's llvm-mca and Intel's IACA). Ithemal is also able to predict these throughput values just as fast as the aforementioned tools, and is easily ported across a variety of processor microarchitectures with minimal developer effort.
On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning
Authors: Bolin Gao, Lacra Pavel
Date: 2017-04-03 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we utilize results from convex analysis and monotone operator
theory to derive additional properties of the softmax function that have not
yet been covered in the existing literature. In particular, we show that the
softmax function is the monotone gradient map of the log-sum-exp function. By
exploiting this connection, we show that the inverse temperature parameter
determines the Lipschitz and co-coercivity properties of the softmax function.
We then demonstrate the usefulness of these properties through an application
in game-theoretic reinforcement learning.
Catastrophic Importance of Catastrophic Forgetting
Authors: Albert Ierusalem
Date: 2018-08-20 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper describes some of the possibilities of artificial neural networks
that open up after solving the problem of catastrophic forgetting. A simple
model and reinforcement learning applications of existing methods are also
proposed.
On the Predictability of non-CGM Diabetes Data for Personalized Recommendation
Authors: Tu Nguyen, Markus Rokicki
Date: 2018-08-19 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

With continuous glucose monitoring (CGM), data-driven models on blood glucose prediction have been shown to be effective in related work. However, such (CGM) systems are not always available, e.g., for a patient at home. In this work, we conduct a study on 9 patients and examine the online predictability of data-driven (aka. machine learning) based models on patient-level blood glucose prediction; with measurements are taken only periodically (i.e., after several hours). To this end, we propose several post-prediction methods to account for the noise nature of these data, that marginally improves the performance of the end system.
Privacy Mining from IoT-based Smart Homes
Authors: Ming-Chang Lee, Jia-Chun Lin, Olaf Owe
Date: 2018-08-18 00:00:00

Recently, a wide range of smart devices are deployed in a variety of
environments to improve the quality of human life. One of the important
IoT-based applications is smart homes for healthcare, especially for elders.
IoT-based smart homes enable elders' health to be properly monitored and taken
care of. However, elders' privacy might be disclosed from smart homes due to
non-fully protected network communication or other reasons. To demonstrate how
serious this issue is, we introduce in this paper a Privacy Mining Approach
(PMA) to mine privacy from smart homes by conducting a series of deductions and
analyses on sensor datasets generated by smart homes. The experimental results
demonstrate that PMA is able to deduce a global sensor topology for a smart
home and disclose elders' privacy in terms of their house layouts.
Sentiment Index of the Russian Speaking Facebook
Authors: Alexander Panchenko
Date: 2018-08-23 00:00:00

A sentiment index measures the average emotional level in a corpus. We
introduce four such indexes and use them to gauge average "positiveness" of a
population during some period based on posts in a social network. This article
for the first time presents a text-, rather than word-based sentiment index.
Furthermore, this study presents the first large-scale study of the sentiment
index of the Russian-speaking Facebook. Our results are consistent with the
prior experiments for the English language.