pwc-archive/papers-with-abstracts huggingface dataset:
Learning by Unsupervised Nonlinear Diffusion
Authors: Mauro Maggioni, James M. Murphy
Date: 2018-10-15 00:00:00
Tasks: Clustering

This paper proposes and analyzes a novel clustering algorithm that combines
graph-based diffusion geometry with techniques based on density and mode
estimation. The proposed method is suitable for data generated from mixtures of
distributions with densities that are both multimodal and have nonlinear
shapes. A crucial aspect of this algorithm is the use of time of a data-adapted
diffusion process as a scale parameter that is different from the local spatial
scale parameter used in many clustering algorithms. We prove estimates for the
behavior of diffusion distances with respect to this time parameter under a
flexible nonparametric data model, identifying a range of times in which the
mesoscopic equilibria of the underlying process are revealed, corresponding to
a gap between within-cluster and between-cluster diffusion distances. These
structures can be missed by the top eigenvectors of the graph Laplacian,
commonly used in spectral clustering. This analysis is leveraged to prove
sufficient conditions guaranteeing the accuracy of the proposed \emph{learning
by unsupervised nonlinear diffusion (LUND)} procedure. We implement LUND and
confirm its theoretical properties on illustrative datasets, demonstrating the
theoretical and empirical advantages over both spectral clustering and
density-based clustering techniques.
Lesion Focused Super-Resolution
Authors: Jin Zhu, Guang Yang, Pietro Lio
Date: 2018-10-15 00:00:00
Tasks: Brain Tumor Segmentation, Diagnostic, Image Enhancement, Lesion Detection, Super-Resolution, Tumor Segmentation

Super-resolution (SR) for image enhancement has great importance in medical
image applications. Broadly speaking, there are two types of SR, one requires
multiple low resolution (LR) images from different views of the same object to
be reconstructed to the high resolution (HR) output, and the other one relies
on the learning from a large amount of training datasets, i.e., LR-HR pairs. In
real clinical environment, acquiring images from multi-views is expensive and
sometimes infeasible. In this paper, we present a novel Generative Adversarial
Networks (GAN) based learning framework to achieve SR from its LR version. By
performing simulation based studies on the Multimodal Brain Tumor Segmentation
Challenge (BraTS) datasets, we demonstrate the efficacy of our method in
application of brain tumor MRI enhancement. Compared to bilinear interpolation
and other state-of-the-art SR methods, our model is lesion focused, which is
not only resulted in better perceptual image quality without blurring, but also
more efficient and directly benefit for the following clinical tasks, e.g.,
lesion detection and abnormality enhancement. Therefore, we can envisage the
application of our SR method to boost image spatial resolution while
maintaining crucial diagnostic information for further clinical tasks.
Column generation based math-heuristic for classification trees
Authors: Murat Firat, Guillaume Crognier, Adriana F. Gabor, C. A. J. Hurkens, Yingqian Zhang
Date: 2018-10-15 00:00:00
Tasks: Classification, General Classification, Math

This paper explores the use of Column Generation (CG) techniques in constructing univariate binary decision trees for classification tasks. We propose a novel Integer Linear Programming (ILP) formulation, based on root-to-leaf paths in decision trees. The model is solved via a Column Generation based heuristic. To speed up the heuristic, we use a restricted instance data by considering a subset of decision splits, sampled from the solutions of the well-known CART algorithm. Extensive numerical experiments show that our approach is competitive with the state-of-the-art ILP-based algorithms. In particular, the proposed approach is capable of handling big data sets with tens of thousands of data rows. Moreover, for large data sets, it finds solutions competitive to CART.
Trellis Networks for Sequence Modeling
Authors: Shaojie Bai, J. Zico Kolter, Vladlen Koltun
Date: 2018-10-15 00:00:00
Proceeding: ICLR 2019 5
Tasks: Language Modeling, Language Modelling, Sequential Image Classification

We present trellis networks, a new architecture for sequence modeling. On the
one hand, a trellis network is a temporal convolutional network with special
structure, characterized by weight tying across depth and direct injection of
the input into deep layers. On the other hand, we show that truncated recurrent
networks are equivalent to trellis networks with special sparsity structure in
their weight matrices. Thus trellis networks with general weight matrices
generalize truncated recurrent networks. We leverage these connections to
design high-performing trellis networks that absorb structural and algorithmic
elements from both recurrent and convolutional models. Experiments demonstrate
that trellis networks outperform the current state of the art methods on a
variety of challenging benchmarks, including word-level language modeling and
character-level language modeling tasks, and stress tests designed to evaluate
long-term memory retention. The code is available at
https://github.com/locuslab/trellisnet .
Named-Entity Linking Using Deep Learning For Legal Documents: A Transfer Learning Approach
Authors: Ahmed Elnaggar, Robin Otto, Florian Matthes
Date: 2018-10-15 00:00:00
Tasks: Entity Linking, Transfer Learning

In the legal domain it is important to differentiate between words in
general, and afterwards to link the occurrences of the same entities. The topic
to solve these challenges is called Named-Entity Linking (NEL). Current
supervised neural networks designed for NEL use publicly available datasets for
training and testing. However, this paper focuses especially on the aspect of
applying transfer learning approach using networks trained for NEL to legal
documents. Experiments show consistent improvement in the legal datasets that
were created from the European Union law in the scope of this research. Using
transfer learning approach, we reached F1-score of 98.90\% and 98.01\% on the
legal small and large test dataset.
Deep Transfer Reinforcement Learning for Text Summarization
Authors: Yaser Keneshloo, Naren Ramakrishnan, Chandan K. Reddy
Date: 2018-10-15 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Summarization, Transfer Learning, Transfer Reinforcement Learning

Deep neural networks are data hungry models and thus face difficulties when
attempting to train on small text datasets. Transfer learning is a potential
solution but their effectiveness in the text domain is not as explored as in
areas such as image analysis. In this paper, we study the problem of transfer
learning for text summarization and discuss why existing state-of-the-art
models fail to generalize well on other (unseen) datasets. We propose a
reinforcement learning framework based on a self-critic policy gradient
approach which achieves good generalization and state-of-the-art results on a
variety of datasets. Through an extensive set of experiments, we also show the
ability of our proposed framework to fine-tune the text summarization model
using only a few training samples. To the best of our knowledge, this is the
first work that studies transfer learning in text summarization and provides a
generic solution that works well on unseen data.
Stop Illegal Comments: A Multi-Task Deep Learning Approach
Authors: Ahmed Elnaggar, Bernhard Waltl, Ingo Glaser, Jörg Landthaler, Elena Scepankova, Florian Matthes
Date: 2018-10-15 00:00:00
Tasks: Deep Learning, General Classification, Transfer Learning, Translation

Deep learning methods are often difficult to apply in the legal domain due to
the large amount of labeled data required by deep learning methods. A recent
new trend in the deep learning community is the application of multi-task
models that enable single deep neural networks to perform more than one task at
the same time, for example classification and translation tasks. These powerful
novel models are capable of transferring knowledge among different tasks or
training sets and therefore could open up the legal domain for many deep
learning applications. In this paper, we investigate the transfer learning
capabilities of such a multi-task model on a classification task on the
publicly available Kaggle toxic comment dataset for classifying illegal
comments and we can report promising results.
UNIQUE: Unsupervised Image Quality Estimation
Authors: D. Temel, M. Prabhushankar, G. AlRegib
Date: 2018-10-15 00:00:00
Tasks: Decoder, Image Quality Assessment, Video Quality Assessment

In this paper, we estimate perceived image quality using sparse
representations obtained from generic image databases through an unsupervised
learning approach. A color space transformation, a mean subtraction, and a
whitening operation are used to enhance descriptiveness of images by reducing
spatial redundancy; a linear decoder is used to obtain sparse representations;
and a thresholding stage is used to formulate suppression mechanisms in a
visual system. A linear decoder is trained with 7 GB worth of data, which
corresponds to 100,000 8x8 image patches randomly obtained from nearly 1,000
images in the ImageNet 2013 database. A patch-wise training approach is
preferred to maintain local information. The proposed quality estimator UNIQUE
is tested on the LIVE, the Multiply Distorted LIVE, and the TID 2013 databases
and compared with thirteen quality estimators. Experimental results show that
UNIQUE is generally a top performing quality estimator in terms of accuracy,
consistency, linearity, and monotonic behavior.
Adversarial Inpainting of Medical Image Modalities
Authors: Karim Armanious, Youssef Mecky, Sergios Gatidis, Bin Yang
Date: 2018-10-15 00:00:00
Tasks: Image Inpainting

Numerous factors could lead to partial deteriorations of medical images. For
example, metallic implants will lead to localized perturbations in MRI scans.
This will affect further post-processing tasks such as attenuation correction
in PET/MRI or radiation therapy planning. In this work, we propose the
inpainting of medical images via Generative Adversarial Networks (GANs). The
proposed framework incorporates two patch-based discriminator networks with
additional style and perceptual losses for the inpainting of missing
information in realistically detailed and contextually consistent manner. The
proposed framework outperformed other natural image inpainting techniques both
qualitatively and quantitatively on two different medical modalities.
Diacritization of Maghrebi Arabic Sub-Dialects
Authors: Ahmed Abdelali, Mohammed Attia, Younes Samih, Kareem Darwish, Hamdy Mubarak
Date: 2018-10-15 00:00:00
Tasks: text-to-speech, Text to Speech

Diacritization process attempt to restore the short vowels in Arabic written text; which typically are omitted. This process is essential for applications such as Text-to-Speech (TTS). While diacritization of Modern Standard Arabic (MSA) still holds the lion share, research on dialectal Arabic (DA) diacritization is very limited. In this paper, we present our contribution and results on the automatic diacritization of two sub-dialects of Maghrebi Arabic, namely Tunisian and Moroccan, using a character-level deep neural network architecture that stacks two bi-LSTM layers over a CRF output layer. The model achieves word error rate of 2.7% and 3.6% for Moroccan and Tunisian respectively and is capable of implicitly identifying the sub-dialect of the input.
Optimizing Heuristics for Tableau-based OWL Reasoners
Authors: Razieh Mehri, Volker Haarslev, Hamidreza Chinaei
Date: 2018-10-15 00:00:00

Optimization techniques play a significant role in improving description
logic reasoners covering the Web Ontology Language (OWL). These techniques are
essential to speed up these reasoners. Many of the optimization techniques are
based on heuristic choices. Optimal heuristic selection makes these techniques
more effective. The FaCT++ OWL reasoner and its Java version JFact implement an
optimization technique called ToDo list which is a substitute for a traditional
top-down approach in tableau-based reasoners. The ToDo list mechanism allows
one to arrange the order of applying different rules by giving each a priority.
Compared to a top-down approach, the ToDo list technique has a better control
over the application of expansion rules. Learning the proper heuristic order
for applying rules in ToDo lis} will have a great impact on reasoning speed. We
use a binary SVM technique to build our learning model. The model can help to
choose ontology-specific order sets to speed up OWL reasoning. On average, our
learning approach tested with 40 selected ontologies achieves a speedup of two
orders of magnitude when compared to the worst rule ordering choice.
Learning to Segment Corneal Tissue Interfaces in OCT Images
Authors: Tejas Sudharshan Mathai, Kira Lathrop, John Galeotti
Date: 2018-10-15 00:00:00
Tasks: Deep Learning

Accurate and repeatable delineation of corneal tissue interfaces is necessary
for surgical planning during anterior segment interventions, such as
Keratoplasty. Designing an approach to identify interfaces, which generalizes
to datasets acquired from different Optical Coherence Tomographic (OCT)
scanners, is paramount. In this paper, we present a Convolutional Neural
Network (CNN) based framework called CorNet that can accurately segment three
corneal interfaces across datasets obtained with different scan settings from
different OCT scanners. Extensive validation of the approach was conducted
across all imaged datasets. To the best of our knowledge, this is the first
deep learning based approach to segment both anterior and posterior corneal
tissue interfaces. Our errors are 2x lower than non-proprietary
state-of-the-art corneal tissue interface segmentation algorithms, which
include image analysis-based and deep learning approaches.
Deep learning-based super-resolution in coherent imaging systems
Authors: Tairan Liu, Kevin De Haan, Yair Rivenson, Zhensong Wei, Xin Zeng, Yibo Zhang, Aydogan Ozcan
Date: 2018-10-15 00:00:00
Tasks: Deep Learning, Generative Adversarial Network, Image Reconstruction, Super-Resolution

We present a deep learning framework based on a generative adversarial
network (GAN) to perform super-resolution in coherent imaging systems. We
demonstrate that this framework can enhance the resolution of both pixel
size-limited and diffraction-limited coherent imaging systems. We
experimentally validated the capabilities of this deep learning-based coherent
imaging approach by super-resolving complex images acquired using a lensfree
on-chip holographic microscope, the resolution of which was pixel size-limited.
Using the same GAN-based approach, we also improved the resolution of a
lens-based holographic imaging system that was limited in resolution by the
numerical aperture of its objective lens. This deep learning-based
super-resolution framework can be broadly applied to enhance the
space-bandwidth product of coherent imaging systems using image data and
convolutional neural networks, and provides a rapid, non-iterative method for
solving inverse image reconstruction or enhancement problems in optics.
Modeling of nonlinear audio effects with end-to-end deep neural networks
Authors: Marco Martínez, Joshua D. Reiss
Date: 2018-10-15 00:00:00

In the context of music production, distortion effects are mainly used for
aesthetic reasons and are usually applied to electric musical instruments. Most
existing methods for nonlinear modeling are often either simplified or
optimized to a very specific circuit. In this work, we investigate deep
learning architectures for audio processing and we aim to find a general
purpose end-to-end deep neural network to perform modeling of nonlinear audio
effects. We show the network modeling various nonlinearities and we discuss the
generalization capabilities among different instruments.
Concise Explanations of Neural Networks using Adversarial Training
Authors: Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Somesh Jha, Xi Wu
Date: 2018-10-15 00:00:00
Proceeding: ICML 2020 1
Tasks: Multi-class Classification

We show new connections between adversarial learning and explainability for deep neural networks (DNNs). One form of explanation of the output of a neural network model in terms of its input features, is a vector of feature-attributions. Two desirable characteristics of an attribution-based explanation are: (1) $\textit{sparseness}$: the attributions of irrelevant or weakly relevant features should be negligible, thus resulting in $\textit{concise}$ explanations in terms of the significant features, and (2) $\textit{stability}$: it should not vary significantly within a small local neighborhood of the input. Our first contribution is a theoretical exploration of how these two properties (when using attributions based on Integrated Gradients, or IG) are related to adversarial training, for a class of 1-layer networks (which includes logistic regression models for binary and multi-class classification); for these networks we show that (a) adversarial training using an $\ell_\infty$-bounded adversary produces models with sparse attribution vectors, and (b) natural model-training while encouraging stable explanations (via an extra term in the loss function), is equivalent to adversarial training. Our second contribution is an empirical verification of phenomenon (a), which we show, somewhat surprisingly, occurs $\textit{not only}$ $\textit{in 1-layer networks}$, $\textit{but also DNNs}$ $\textit{trained on }$ $\textit{standard image datasets}$, and extends beyond IG-based attributions, to those based on DeepSHAP: adversarial training with $\ell_\infty$-bounded perturbations yields significantly sparser attribution vectors, with little degradation in performance on natural test data, compared to natural training. Moreover, the sparseness of the attribution vectors is significantly better than that achievable via $\ell_1$-regularized natural training.
The Trajectory of Voice Onset Time with Vocal Aging
Authors: Xuanda Chen, Ziyu Xiong, Jian Hu
Date: 2018-10-15 00:00:00
Tasks: Human Aging

Vocal aging, a universal process of human aging, can largely affect one's
language use, possibly including some subtle acoustic features of one's
utterances like Voice Onset Time. To figure out the time effects, Queen
Elizabeth's Christmas speeches are documented and analyzed in the long-term
trend. We build statistical models of time dependence in Voice Onset Time,
controlling a wide range of other fixed factors, to present annual variations
and the simulated trajectory. It is revealed that the variation range of Voice
Onset Time has been narrowing over fifty years with a slight reduction in the
mean value, which, possibly, is an effect of diminishing exertion, resulting
from subdued muscle contraction, transcending other non-linguistic factors in
forming Voice Onset Time patterns over a long time.
Revisit Batch Normalization: New Understanding from an Optimization View and a Refinement via Composition Optimization
Authors: Xiangru Lian, Ji Liu
Date: 2018-10-15 00:00:00

Batch Normalization (BN) has been used extensively in deep learning to
achieve faster training process and better resulting models. However, whether
BN works strongly depends on how the batches are constructed during training
and it may not converge to a desired solution if the statistics on a batch are
not close to the statistics over the whole dataset. In this paper, we try to
understand BN from an optimization perspective by formulating the optimization
problem which motivates BN. We show when BN works and when BN does not work by
analyzing the optimization problem. We then propose a refinement of BN based on
compositional optimization techniques called Full Normalization (FN) to
alleviate the issues of BN when the batches are not constructed ideally. We
provide convergence analysis for FN and empirically study its effectiveness to
refine BN.
Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images
Authors: Javier Marin, Aritro Biswas, Ferda Ofli, Nicholas Hynes, Amaia Salvador, Yusuf Aytar, Ingmar Weber, Antonio Torralba
Date: 2018-10-14 00:00:00
Tasks: Cross-Modal Retrieval, General Classification, Retrieval

In this paper, we introduce Recipe1M+, a new large-scale, structured corpus of over one million cooking recipes and 13 million food images. As the largest publicly available collection of recipe data, Recipe1M+ affords the ability to train high-capacity modelson aligned, multimodal data. Using these data, we train a neural network to learn a joint embedding of recipes and images that yields impressive results on an image-recipe retrieval task. Moreover, we demonstrate that regularization via the addition of a high-level classification objective both improves retrieval performance to rival that of humans and enables semantic vector arithmetic. We postulate that these embeddings will provide a basis for further exploration of the Recipe1M+ dataset and food and cooking in general. Code, data and models are publicly available.
Tentacular Artificial Intelligence, and the Architecture Thereof, Introduced
Authors: Selmer Bringsjord, Naveen Sundar Govindarajulu, Atriya Sen, Matthew Peveler, Biplav Srivastava, Kartik Talamadupula
Date: 2018-10-14 00:00:00

We briefly introduce herein a new form of distributed, multi-agent artificial
intelligence, which we refer to as "tentacular." Tentacular AI is distinguished
by six attributes, which among other things entail a capacity for reasoning and
planning based in highly expressive calculi (logics), and which enlists
subsidiary agents across distances circumscribed only by the reach of one or
more given networks.
Topographic Representation for Quantum Machine Learning
Authors: Bruce MacLennan
Date: 2018-10-13 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning

This paper proposes a brain-inspired approach to quantum machine learning with the goal of circumventing many of the complications of other approaches. The fact that quantum processes are unitary presents both opportunities and challenges. A principal opportunity is that a large number of computations can be carried out in parallel in linear superposition, that is, quantum parallelism. The challenge is that the process is linear, and most approaches to machine learning depend significantly on nonlinear processes. Fortunately, the situation is not hopeless, for we know that nonlinear processes can be embedded in unitary processes, as is familiar from the circuit model of quantum computation. This paper explores an approach to the quantum implementation of machine learning involving nonlinear functions operating on information represented topographically (by computational maps), as common in neural cortex.
Non-computability of human intelligence
Authors: Yasha Savelyev
Date: 2018-10-12 00:00:00

We revisit the question (most famously) initiated by Turing: can human intelligence be completely modeled by a Turing machine? We show that the answer is \emph{no}, assuming a certain weak soundness hypothesis. More specifically we show that at least some meaningful thought processes of the brain cannot be Turing computable. In particular some physical processes are not Turing computable, which is not entirely expected. There are some similarities of our argument with the well known Lucas-Penrose argument, but we work purely on the level of Turing machines, and do not use G\"odel's incompleteness theorem or any direct analogue. Instead we construct directly and use a weak analogue of a G\"odel statement for a certain system which involves our human, this allows us to side-step some (possible) meta-logical issues with their argument.