pwc-archive/papers-with-abstracts huggingface dataset:
Bandana: Using Non-volatile Memory for Storing Deep Learning Models
Authors: Assaf Eisenman, Maxim Naumov, Darryl Gardner, Misha Smelyanskiy, Sergey Pupyrev, Kim Hazelwood, Asaf Cidon, Sachin Katti
Date: 2018-11-14 00:00:00
Tasks: Deep Learning, hypergraph partitioning, Recommendation Systems

Typical large-scale recommender systems use deep learning models that are
stored on a large amount of DRAM. These models often rely on embeddings, which
consume most of the required memory. We present Bandana, a storage system that
reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as
the primary storage medium, with a small amount of DRAM as cache. The main
challenge in storing embeddings on NVM is its limited read bandwidth compared
to DRAM. Bandana uses two primary techniques to address this limitation: first,
it stores embedding vectors that are likely to be read together in the same
physical location, using hypergraph partitioning, and second, it decides the
number of embedding vectors to cache in DRAM by simulating dozens of small
caches. These techniques allow Bandana to increase the effective read bandwidth
of NVM by 2-3x and thereby significantly reduce the total cost of ownership.
The Greedy Dirichlet Process Filter - An Online Clustering Multi-Target Tracker
Authors: Benjamin Naujoks, Patrick Burger, Hans-Joachim Wuensche
Date: 2018-11-14 00:00:00
Tasks: Autonomous Driving, Clustering, Collision Avoidance, Online Clustering

Reliable collision avoidance is one of the main requirements for autonomous
driving. Hence, it is important to correctly estimate the states of an unknown
number of static and dynamic objects in real-time. Here, data association is a
major challenge for every multi-target tracker. We propose a novel multi-target
tracker called Greedy Dirichlet Process Filter (GDPF) based on the
non-parametric Bayesian model called Dirichlet Processes and the fast posterior
computation algorithm Sequential Updating and Greedy Search (SUGS). By adding a
temporal dependence we get a real-time capable tracking framework without the
need of a previous clustering or data association step. Real-world tests show
that GDPF outperforms other multi-target tracker in terms of accuracy and
stability.
Deep Bayesian Inversion
Authors: Jonas Adler, Ozan Öktem
Date: 2018-11-14 00:00:00
Tasks: Decision Making, Image Reconstruction

Characterizing statistical properties of solutions of inverse problems is
essential for decision making. Bayesian inversion offers a tractable framework
for this purpose, but current approaches are computationally unfeasible for
most realistic imaging applications in the clinic. We introduce two novel deep
learning based methods for solving large-scale inverse problems using Bayesian
inversion: a sampling based method using a WGAN with a novel mini-discriminator
and a direct approach that trains a neural network using a novel loss function.
The performance of both methods is demonstrated on image reconstruction in
ultra low dose 3D helical CT. We compute the posterior mean and standard
deviation of the 3D images followed by a hypothesis test to assess whether a
"dark spot" in the liver of a cancer stricken patient is present. Both methods
are computationally efficient and our evaluation shows very promising
performance that clearly supports the claim that Bayesian inversion is usable
for 3D imaging in time critical applications.
The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task
Authors: Alberto Poncelas, Andy Way, Kepa Sarasola
Date: 2018-11-14 00:00:00
Proceeding: IWSLT (EMNLP) 2018 10
Tasks: Machine Translation, Translation

In this paper we present the ADAPT system built for the Basque to English Low
Resource MT Evaluation Campaign. Basque is a low-resourced,
morphologically-rich language. This poses a challenge for Neural Machine
Translation models which usually achieve better performance when trained with
large sets of data.
  Accordingly, we used synthetic data to improve the translation quality
produced by a model built using only authentic data. Our proposal uses
back-translated data to: (a) create new sentences, so the system can be trained
with more data; and (b) translate sentences that are close to the test set, so
the model can be fine-tuned to the document to be translated.
QUENN: QUantization Engine for low-power Neural Networks
Authors: Miguel de Prado, Maurizio Denna, Luca Benini, Nuria Pazos
Date: 2018-11-14 00:00:00
Tasks: Clustering, Quantization

Deep Learning is moving to edge devices, ushering in a new age of distributed
Artificial Intelligence (AI). The high demand of computational resources
required by deep neural networks may be alleviated by approximate computing
techniques, and most notably reduced-precision arithmetic with coarsely
quantized numerical representations. In this context, Bonseyes comes in as an
initiative to enable stakeholders to bring AI to low-power and autonomous
environments such as: Automotive, Medical Healthcare and Consumer Electronics.
To achieve this, we introduce LPDNN, a framework for optimized deployment of
Deep Neural Networks on heterogeneous embedded devices. In this work, we detail
the quantization engine that is integrated in LPDNN. The engine depends on a
fine-grained workflow which enables a Neural Network Design Exploration and a
sensitivity analysis of each layer for quantization. We demonstrate the engine
with a case study on Alexnet and VGG16 for three different techniques for
direct quantization: standard fixed-point, dynamic fixed-point and k-means
clustering, and demonstrate the potential of the latter. We argue that using a
Gaussian quantizer with k-means clustering can achieve better performance than
linear quantizers. Without retraining, we achieve over 55.64\% saving for
weights' storage and 69.17\% for run-time memory accesses with less than 1\%
drop in top5 accuracy in Imagenet.
Development of Real-time ADAS Object Detector for Deployment on CPU
Authors: Alexander Kozlov, Daniil Osokin
Date: 2018-11-14 00:00:00
Tasks: CPU, Object, object-detection, Object Detection

In this work, we outline the set of problems, which any Object Detection CNN
faces when its development comes to the deployment stage and propose methods to
deal with such difficulties. We show that these practices allow one to get
Object Detection network, which can recognize two classes: vehicles and
pedestrians and achieves more than 60 frames per second inference speed on
Core$^{TM}$ i5-6500 CPU. The proposed model is built on top of the popular
Single Shot MultiBox Object Detection framework but with substantial
improvements, which were inspired by the discovered problems. The network has
just 1.96 GMAC complexity and less than 7 MB model size. It is publicly
available as a part of Intel$\circledR$ OpenVINO$^{TM}$ Toolkit.
Dependency Grammar Induction with a Neural Variational Transition-based Parser
Authors: Bowen Li, Jianpeng Cheng, Yang Liu, Frank Keller
Date: 2018-11-14 00:00:00
Tasks: Dependency Grammar Induction, Variational Inference

Dependency grammar induction is the task of learning dependency syntax
without annotated training data. Traditional graph-based models with global
inference achieve state-of-the-art results on this task but they require
$O(n^3)$ run time. Transition-based models enable faster inference with $O(n)$
time complexity, but their performance still lags behind. In this work, we
propose a neural transition-based parser for dependency grammar induction,
whose inference procedure utilizes rich neural features with $O(n)$ time
complexity. We train the parser with an integration of variational inference,
posterior regularization and variance reduction techniques. The resulting
framework outperforms previous unsupervised transition-based dependency parsers
and achieves performance comparable to graph-based models, both on the English
Penn Treebank and on the Universal Dependency Treebank. In an empirical
comparison, we show that our approach substantially increases parsing speed
over graph-based models.
Large-scale Interactive Recommendation with Tree-structured Policy Gradient
Authors: Haokun Chen, Xinyi Dai, Han Cai, Wei-Nan Zhang, Xuejian Wang, Ruiming Tang, Yuzhou Zhang, Yong Yu
Date: 2018-11-14 00:00:00
Tasks: Clustering, Interactive Recommendation, Recommendation Systems, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) has recently been introduced to interactive
recommender systems (IRS) because of its nature of learning from dynamic
interactions and planning for long-run performance. As IRS is always with
thousands of items to recommend (i.e., thousands of actions), most existing
RL-based methods, however, fail to handle such a large discrete action space
problem and thus become inefficient. The existing work that tries to deal with
the large discrete action space problem by utilizing the deep deterministic
policy gradient framework suffers from the inconsistency between the continuous
action representation (the output of the actor network) and the real discrete
action. To avoid such inconsistency and achieve high efficiency and
recommendation effectiveness, in this paper, we propose a Tree-structured
Policy Gradient Recommendation (TPGR) framework, where a balanced hierarchical
clustering tree is built over the items and picking an item is formulated as
seeking a path from the root to a certain leaf of the tree. Extensive
experiments on carefully-designed environments based on two real-world datasets
demonstrate that our model provides superior recommendation performance and
significant efficiency improvement over state-of-the-art methods.
Pitfalls of Graph Neural Network Evaluation
Authors: Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann
Date: 2018-11-14 00:00:00
Tasks: Graph Mining, Graph Neural Network, Node Classification

Semi-supervised node classification in graphs is a fundamental problem in graph mining, and the recently proposed graph neural networks (GNNs) have achieved unparalleled results on this task. Due to their massive success, GNNs have attracted a lot of attention, and many novel architectures have been put forward. In this paper we show that existing evaluation strategies for GNN models have serious shortcomings. We show that using the same train/validation/test splits of the same datasets, as well as making significant changes to the training procedure (e.g. early stopping criteria) precludes a fair comparison of different architectures. We perform a thorough empirical evaluation of four prominent GNN models and show that considering different splits of the data leads to dramatically different rankings of models. Even more importantly, our findings suggest that simpler GNN architectures are able to outperform the more sophisticated ones if the hyperparameters and the training procedure are tuned fairly for all models.
Robust low-rank multilinear tensor approximation for a joint estimation of the multilinear rank and the loading matrices
Authors: Xu Han, Laurent Albera, Amar Kachenoura, Huazhong Shu, Lotfi Senhadji
Date: 2018-11-14 00:00:00
Tasks: Tensor Decomposition

In order to compute the best low-rank tensor approximation using the Multilinear Tensor Decomposition (MTD) model, it is essential to estimate the rank of the underlying multilinear tensor from the noisy observation tensor. In this paper, we propose a Robust MTD (R-MTD) method, which jointly estimates the multilinear rank and the loading matrices. Based on the low-rank property and an over-estimation of the core tensor, this joint estimation problem is solved by promoting (group) sparsity of the over-estimated core tensor. Group sparsity is promoted using mixed-norms. Then we establish a link between the mixed-norms and the nuclear norm, showing that mixed-norms are better candidates for a convex envelope of the rank. After several iterations of the Alternating Direction Method of Multipliers (ADMM), the Minimum Description Length (MDL) criterion computed from the eigenvalues of the unfolding matrices of the estimated core tensor is minimized in order to estimate the multilinear rank. The latter is then used to estimate more accurately the loading matrices. We further develop another R-MTD method, called R-OMTD, by imposing an orthonormality constraint on each loading matrix in order to decrease the computation complexity. A series of simulated noisy tensor and real-world data are used to show the effectiveness of the proposed methods compared with state-of-the-art methods.
Predicting the time-evolution of multi-physics systems with sequence-to-sequence models
Authors: K. D. Humbird, J. L. Peterson, R. G. McClarren
Date: 2018-11-14 00:00:00
Tasks: Translation

In this work, sequence-to-sequence (seq2seq) models, originally developed for
language translation, are used to predict the temporal evolution of complex,
multi-physics computer simulations. The predictive performance of seq2seq
models is compared to state transition models for datasets generated with
multi-physics codes with varying levels of complexity - from simple 1D
diffusion calculations to simulations of inertial confinement fusion
implosions. Seq2seq models demonstrate the ability to accurately emulate
complex systems, enabling the rapid estimation of the evolution of quantities
of interest in computationally expensive simulations.
Drop-Activation: Implicit Parameter Reduction and Harmonic Regularization
Authors: Senwei Liang, Yuehaw Khoo, Haizhao Yang
Date: 2018-11-14 00:00:00
Tasks: image-classification, Image Classification

Overfitting frequently occurs in deep learning. In this paper, we propose a novel regularization method called Drop-Activation to reduce overfitting and improve generalization. The key idea is to drop nonlinear activation functions by setting them to be identity functions randomly during training time. During testing, we use a deterministic network with a new activation function to encode the average effect of dropping activations randomly. Our theoretical analyses support the regularization effect of Drop-Activation as implicit parameter reduction and verify its capability to be used together with Batch Normalization (Ioffe and Szegedy 2015). The experimental results on CIFAR-10, CIFAR-100, SVHN, EMNIST, and ImageNet show that Drop-Activation generally improves the performance of popular neural network architectures for the image classification task. Furthermore, as a regularizer Drop-Activation can be used in harmony with standard training and regularization techniques such as Batch Normalization and Auto Augment (Cubuk et al. 2019). The code is available at \url{https://github.com/LeungSamWai/Drop-Activation}.
A Learning-Based Framework for Line-Spectra Super-resolution
Authors: Gautier Izacard, Brett Bernstein, Carlos Fernandez-Granda
Date: 2018-11-14 00:00:00
Tasks: Super-Resolution

We propose a learning-based approach for estimating the spectrum of a multisinusoidal signal from a finite number of samples. A neural-network is trained to approximate the spectra of such signals on simulated data. The proposed methodology is very flexible: adapting to different signal and noise models only requires modifying the training data accordingly. Numerical experiments show that the approach performs competitively with classical methods designed for additive Gaussian noise at a range of noise levels, and is also effective in the presence of impulsive noise.
Revisiting Projection-Free Optimization for Strongly Convex Constraint Sets
Authors: Jarrid Rector-Brooks, Jun-Kun Wang, Barzan Mozafari
Date: 2018-11-14 00:00:00

We revisit the Frank-Wolfe (FW) optimization under strongly convex constraint
sets. We provide a faster convergence rate for FW without line search, showing
that a previously overlooked variant of FW is indeed faster than the standard
variant. With line search, we show that FW can converge to the global optimum,
even for smooth functions that are not convex, but are quasi-convex and
locally-Lipschitz. We also show that, for the general case of (smooth)
non-convex functions, FW with line search converges with high probability to a
stationary point at a rate of $O\left(\frac{1}{t}\right)$, as long as the
constraint set is strongly convex -- one of the fastest convergence rates in
non-convex optimization.
Distortion Robust Image Classification using Deep Convolutional Neural Network with Discrete Cosine Transform
Authors: Md Tahmid Hossain, Shyh Wei Teng, Dengsheng Zhang, Suryani Lim, Guojun Lu
Date: 2018-11-14 00:00:00
Tasks: General Classification, image-classification, Image Classification

Convolutional Neural Network is good at image classification. However, it is found to be vulnerable to image quality degradation. Even a small amount of distortion such as noise or blur can severely hamper the performance of these CNN architectures. Most of the work in the literature strives to mitigate this problem simply by fine-tuning a pre-trained CNN on mutually exclusive or a union set of distorted training data. This iterative fine-tuning process with all known types of distortion is exhaustive and the network struggles to handle unseen distortions. In this work, we propose distortion robust DCT-Net, a Discrete Cosine Transform based module integrated into a deep network which is built on top of VGG16. Unlike other works in the literature, DCT-Net is "blind" to the distortion type and level in an image both during training and testing. As a part of the training process, the proposed DCT module discards input information which mostly represents the contribution of high frequencies. The DCT-Net is trained "blindly" only once and applied in generic situation without further retraining. We also extend the idea of traditional dropout and present a training adaptive version of the same. We evaluate our proposed method against Gaussian blur, motion blur, salt and pepper noise, Gaussian noise and speckle noise added to CIFAR-10/100 and ImageNet test sets. Experimental results demonstrate that once trained, DCT-Net not only generalizes well to a variety of unseen image distortions but also outperforms other methods in the literature.
ProstateGAN: Mitigating Data Bias via Prostate Diffusion Imaging Synthesis with Generative Adversarial Networks
Authors: Xiaodan Hu, Audrey G. Chung, Paul Fieguth, Farzad Khalvati, Masoom A. Haider, Alexander Wong
Date: 2018-11-14 00:00:00
Tasks: Data Augmentation, Image Generation, Medical Image Analysis

Generative Adversarial Networks (GANs) have shown considerable promise for
mitigating the challenge of data scarcity when building machine learning-driven
analysis algorithms. Specifically, a number of studies have shown that
GAN-based image synthesis for data augmentation can aid in improving
classification accuracy in a number of medical image analysis tasks, such as
brain and liver image analysis. However, the efficacy of leveraging GANs for
tackling prostate cancer analysis has not been previously explored. Motivated
by this, in this study we introduce ProstateGAN, a GAN-based model for
synthesizing realistic prostate diffusion imaging data. More specifically, in
order to generate new diffusion imaging data corresponding to a particular
cancer grade (Gleason score), we propose a conditional deep convolutional GAN
architecture that takes Gleason scores into consideration during the training
process. Experimental results show that high-quality synthetic prostate
diffusion imaging data can be generated using the proposed ProstateGAN for
specified Gleason scores.
Creatures great and SMAL: Recovering the shape and motion of animals from video
Authors: Benjamin Biggs, Thomas Roddick, Andrew Fitzgibbon, Roberto Cipolla
Date: 2018-11-14 00:00:00
Tasks: Video Segmentation, Video Semantic Segmentation

We present a system to recover the 3D shape and motion of a wide variety of
quadrupeds from video. The system comprises a machine learning front-end which
predicts candidate 2D joint positions, a discrete optimization which finds
kinematically plausible joint correspondences, and an energy minimization stage
which fits a detailed 3D model to the image. In order to overcome the limited
availability of motion capture training data from animals, and the difficulty
of generating realistic synthetic training images, the system is designed to
work on silhouette data. The joint candidate predictor is trained on
synthetically generated silhouette images, and at test time, deep learning
methods or standard video segmentation tools are used to extract silhouettes
from real data. The system is tested on animal videos from several species, and
shows accurate reconstructions of 3D shape and pose.
LoANs: Weakly Supervised Object Detection with Localizer Assessor Networks
Authors: Christian Bartz, Haojin Yang, Joseph Bethge, Christoph Meinel
Date: 2018-11-14 00:00:00
Tasks: Object, object-detection, Object Detection, Weakly Supervised Object Detection

Recently, deep neural networks have achieved remarkable performance on the
task of object detection and recognition. The reason for this success is mainly
grounded in the availability of large scale, fully annotated datasets, but the
creation of such a dataset is a complicated and costly task. In this paper, we
propose a novel method for weakly supervised object detection that simplifies
the process of gathering data for training an object detector. We train an
ensemble of two models that work together in a student-teacher fashion. Our
student (localizer) is a model that learns to localize an object, the teacher
(assessor) assesses the quality of the localization and provides feedback to
the student. The student uses this feedback to learn how to localize objects
and is thus entirely supervised by the teacher, as we are using no labels for
training the localizer. In our experiments, we show that our model is very
robust to noise and reaches competitive performance compared to a
state-of-the-art fully supervised approach. We also show the simplicity of
creating a new dataset, based on a few videos (e.g. downloaded from YouTube)
and artificially generated data.
Neural Based Statement Classification for Biased Language
Authors: Christoph Hube, Besnik Fetahu
Date: 2018-11-14 00:00:00
Tasks: Classification, General Classification

Biased language commonly occurs around topics which are of controversial
nature, thus, stirring disagreement between the different involved parties of a
discussion. This is due to the fact that for language and its use,
specifically, the understanding and use of phrases, the stances are cohesive
within the particular groups. However, such cohesiveness does not hold across
groups.
  In collaborative environments or environments where impartial language is
desired (e.g. Wikipedia, news media), statements and the language therein
should represent equally the involved parties and be neutrally phrased. Biased
language is introduced through the presence of inflammatory words or phrases,
or statements that may be incorrect or one-sided, thus violating such
consensus.
  In this work, we focus on the specific case of phrasing bias, which may be
introduced through specific inflammatory words or phrases in a statement. For
this purpose, we propose an approach that relies on a recurrent neural networks
in order to capture the inter-dependencies between words in a phrase that
introduced bias.
  We perform a thorough experimental evaluation, where we show the advantages
of a neural based approach over competitors that rely on word lexicons and
other hand-crafted features in detecting biased language. We are able to
distinguish biased statements with a precision of P=0.92, thus significantly
outperforming baseline models with an improvement of over 30%. Finally, we
release the largest corpus of statements annotated for biased language.
An Introduction to Fuzzy & Annotated Semantic Web Languages
Authors: Umberto Straccia
Date: 2018-11-14 00:00:00

We present the state of the art in representing and reasoning with fuzzy
knowledge in Semantic Web Languages such as triple languages RDF/RDFS,
conceptual languages of the OWL 2 family and rule languages. We further show
how one may generalise them to so-called annotation domains, that cover also
e.g. temporal and provenance extensions.