pwc-archive/papers-with-abstracts huggingface dataset:
Sequential Embedding Induced Text Clustering, a Non-parametric Bayesian Approach
Authors: Tiehang Duan, Qi Lou, Sargur N. Srihari, Xiaohui Xie
Date: 2018-11-29 00:00:00
Tasks: Clustering, Decoder, Text Clustering, Word Embeddings

Current state-of-the-art nonparametric Bayesian text clustering methods model
documents through multinomial distribution on bags of words. Although these
methods can effectively utilize the word burstiness representation of documents
and achieve decent performance, they do not explore the sequential information
of text and relationships among synonyms. In this paper, the documents are
modeled as the joint of bags of words, sequential features and word embeddings.
We proposed Sequential Embedding induced Dirichlet Process Mixture Model
(SiDPMM) to effectively exploit this joint document representation in text
clustering. The sequential features are extracted by the encoder-decoder
component. Word embeddings produced by the continuous-bag-of-words (CBOW) model
are introduced to handle synonyms. Experimental results demonstrate the
benefits of our model in two major aspects: 1) improved performance across
multiple diverse text datasets in terms of the normalized mutual information
(NMI); 2) more accurate inference of ground truth cluster numbers with
regularization effect on tiny outlier clusters.
On Implicit Filter Level Sparsity in Convolutional Neural Networks
Authors: Dushyant Mehta, Kwang In Kim, Christian Theobalt
Date: 2018-11-29 00:00:00
Proceeding: CVPR 2019 6
Tasks: L2 Regularization

We investigate filter level sparsity that emerges in convolutional neural
networks (CNNs) which employ Batch Normalization and ReLU activation, and are
trained with adaptive gradient descent techniques and L2 regularization or
weight decay. We conduct an extensive experimental study casting our initial
findings into hypotheses and conclusions about the mechanisms underlying the
emergent filter level sparsity. This study allows new insight into the
performance gap obeserved between adapative and non-adaptive gradient descent
methods in practice. Further, analysis of the effect of training strategies and
hyperparameters on the sparsity leads to practical suggestions in designing CNN
training strategies enabling us to explore the tradeoffs between feature
selectivity, network capacity, and generalization performance. Lastly, we show
that the implicit sparsity can be harnessed for neural network speedup at par
or better than explicit sparsification / pruning approaches, with no
modifications to the typical training pipeline required.
Playing Soccer without Colors in the SPL: A Convolutional Neural Network Approach
Authors: Francisco Leiva, Nicolás Cruz, Ignacio Bugueño, Javier Ruiz-del-Solar
Date: 2018-11-29 00:00:00

The goal of this paper is to propose a vision system for humanoid robotic
soccer that does not use any color information. The main features of this
system are: (i) real-time operation in the NAO robot, and (ii) the ability to
detect the ball, the robots, their orientations, the lines and key field
features robustly. Our ball detector, robot detector, and robot's orientation
detector obtain the highest reported detection rates. The proposed vision
system is tested in a SPL field with several NAO robots under realistic and
highly demanding conditions. The obtained results are: robot detection rate of
94.90%, ball detection rate of 97.10%, and a completely perceived orientation
rate of 99.88% when the observed robot is static, and 95.52% when the observed
robot is moving.
Leveraging Deep Stein's Unbiased Risk Estimator for Unsupervised X-ray Denoising
Authors: Fahad Shamshad, Muhammad Awais, Muhammad Asim, Zain ul Aabidin Lodhi, Muhammad Umair, Ali Ahmed
Date: 2018-11-29 00:00:00
Tasks: Deep Learning, Denoising

Among the plethora of techniques devised to curb the prevalence of noise in
medical images, deep learning based approaches have shown the most promise.
However, one critical limitation of these deep learning based denoisers is the
requirement of high-quality noiseless ground truth images that are difficult to
obtain in many medical imaging applications such as X-rays. To circumvent this
issue, we leverage recently proposed approach of [7] that incorporates Stein's
Unbiased Risk Estimator (SURE) to train a deep convolutional neural network
without requiring denoised ground truth X-ray data. Our experimental results
demonstrate the effectiveness of SURE based approach for denoising X-ray
images.
Learning to Separate Multiple Illuminants in a Single Image
Authors: Zhuo Hui, Ayan Chakrabarti, Kalyan Sunkavalli, Aswin C. Sankaranarayanan
Date: 2018-11-29 00:00:00
Proceeding: CVPR 2019 6

We present a method to separate a single image captured under two
illuminants, with different spectra, into the two images corresponding to the
appearance of the scene under each individual illuminant. We do this by
training a deep neural network to predict the per-pixel reflectance
chromaticity of the scene, which we use in conjunction with a previous
flash/no-flash image-based separation algorithm to produce the final two output
images. We design our reflectance chromaticity network and loss functions by
incorporating intuitions from the physics of image formation. We show that this
leads to significantly better performance than other single image techniques
and even approaches the quality of the two image separation method.
Unlabeled Compression Schemes Exceeding the VC-dimension
Authors: Dömötör Pálvölgyi, Gábor Tardos
Date: 2018-11-29 00:00:00

In this note we disprove a conjecture of Kuzmin and Warmuth claiming that every family whose VC-dimension is at most d admits an unlabeled compression scheme to a sample of size at most d. We also study the unlabeled compression schemes of the joins of some families and conjecture that these give a larger gap between the VC-dimension and the size of the smallest unlabeled compression scheme for them.
Analyzing Federated Learning through an Adversarial Lens
Authors: Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, Seraphin Calo
Date: 2018-11-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Federated Learning, Model Poisoning, parameter estimation

Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence. We explore a number of strategies to carry out this attack, starting with simple boosting of the malicious agent's update to overcome the effects of other agents' updates. To increase attack stealth, we propose an alternating minimization strategy, which alternately optimizes for the training loss and the adversarial objective. We follow up by using parameter estimation for the benign agents' updates to improve on attack success. Finally, we use a suite of interpretability techniques to generate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable. Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.
Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity
Authors: Úlfar Erlingsson, Vitaly Feldman, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Abhradeep Thakurta
Date: 2018-11-29 00:00:00

Sensitive statistics are often collected across sets of users, with repeated collection of reports done over time. For example, trends in users' private preferences or software usage may be monitored via such reports. We study the collection of such statistics in the local differential privacy (LDP) model, and describe an algorithm whose privacy cost is polylogarithmic in the number of changes to a user's value. More fundamentally---by building on anonymity of the users' reports---we also demonstrate how the privacy cost of our LDP algorithm can actually be much lower when viewed in the central model of differential privacy. We show, via a new and general privacy amplification technique, that any permutation-invariant algorithm satisfying $\varepsilon$-local differential privacy will satisfy $(O(\varepsilon \sqrt{\log(1/\delta)/n}), \delta)$-central differential privacy. By this, we explain how the high noise and $\sqrt{n}$ overhead of LDP protocols is a consequence of them being significantly more private in the central model. As a practical corollary, our results imply that several LDP-based industrial deployments may have much lower privacy cost than their advertised $\varepsilon$ would indicate---at least if reports are anonymized.
Uncertainty propagation in neural networks for sparse coding
Authors: Danil Kuzin, Olga Isupova, Lyudmila Mihaylova
Date: 2018-11-29 00:00:00
Tasks: Bayesian Inference

A novel method to propagate uncertainty through the soft-thresholding
nonlinearity is proposed in this paper. At every layer the current distribution
of the target vector is represented as a spike and slab distribution, which
represents the probabilities of each variable being zero, or
Gaussian-distributed. Using the proposed method of uncertainty propagation, the
gradients of the logarithms of normalisation constants are derived, that can be
used to update a weight distribution. A novel Bayesian neural network for
sparse coding is designed utilising both the proposed method of uncertainty
propagation and Bayesian inference algorithm.
Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Models
Authors: Daniel Ritchie, Kai Wang, Yu-an Lin
Date: 2018-11-29 00:00:00
Proceeding: CVPR 2019 6
Tasks: Indoor Scene Synthesis

We present a new, fast and flexible pipeline for indoor scene synthesis that
is based on deep convolutional generative models. Our method operates on a
top-down image-based representation, and inserts objects iteratively into the
scene by predicting their category, location, orientation and size with
separate neural network modules. Our pipeline naturally supports automatic
completion of partial scenes, as well as synthesis of complete scenes. Our
method is significantly faster than the previous image-based method and
generates result that outperforms it and other state-of-the-art deep generative
scene models in terms of faithfulness to training data and perceived visual
quality.
Unifying Decision-Making: a Review on Evolutionary Theories on Rationality and Cognitive Biases
Authors: Catarina Moreira
Date: 2018-11-29 00:00:00
Tasks: Decision Making

In this paper, we make a review on the concepts of rationality across several
different fields, namely in economics, psychology and evolutionary biology and
behavioural ecology. We review how processes like natural selection can help us
understand the evolution of cognition and how cognitive biases might be a
consequence of this natural selection. In the end we argue that humans are not
irrational, but rather rationally bounded and we complement the discussion on
how quantum cognitive models can contribute for the modelling and prediction of
human paradoxical decisions.
A rule-based system proposal to aid in the evaluation and decision-making in external beam radiation treatment planning
Authors: R. C. Fernandes, T. M. Machado, H. J. Onisto, A. D. Muñoz, R. O. Silva, L. R. Domingues, G. C. Fonseca, J. E. Bertuzzo, M. T. Pereira, B. Biazotto, E. T. Costa
Date: 2018-11-29 00:00:00
Tasks: Decision Making

As part of a plan launched by the Ministry of Health of Brazil to increase
the availability of linear accelerators for radiotherapy treatment for the
whole country, for which Varian Medical Systems company has won the bidding, a
technical cooperation agreement was signed inviting Brazilian Scientific and
Technological Institutions to participate in a technology transfer program. As
a result, jointly, the Eldorado Research Institute and the Center for
Biomedical Engineering of the University of Campinas presents in this work, the
concepts behind of a proposed rule engine to aid in the evaluation and
decision-making in radiotherapy treatment planning. Normally, the determination
of the radiation dose for a given patient is a complex and intensive procedure,
which requires a lot of domain knowledge and subjective experience from the
oncologists' team. In order to help them in this complex task, and
additionally, provide an auxiliary tool for less experienced oncologists, it is
presented a project conception of a software system that will make use of a
hybrid data-oriented approach. The proposed rule engine will apply both
inference mechanism and expression evaluation to verify and accredit the
quality of an external beam radiation treatment plan by considering, at first,
the 3D-conformal radiotherapy (3DCRT) technique.
Flow Shape Design for Microfluidic Devices Using Deep Reinforcement Learning
Authors: Xian Yeow Lee, Aditya Balu, Daniel Stoecklein, Baskar Ganapathysubramanian, Soumik Sarkar
Date: 2018-11-29 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Microfluidic devices are utilized to control and direct flow behavior in a
wide variety of applications, particularly in medical diagnostics. A
particularly popular form of microfluidics -- called inertial microfluidic flow
sculpting -- involves placing a sequence of pillars to controllably deform an
initial flow field into a desired one. Inertial flow sculpting can be formally
defined as an inverse problem, where one identifies a sequence of pillars
(chosen, with replacement, from a finite set of pillars, each of which produce
a specific transformation) whose composite transformation results in a
user-defined desired transformation. Endemic to most such problems in
engineering, inverse problems are usually quite computationally intractable,
with most traditional approaches based on search and optimization strategies.
In this paper, we pose this inverse problem as a Reinforcement Learning (RL)
problem. We train a DoubleDQN agent to learn from this environment. The results
suggest that learning is possible using a DoubleDQN model with the success
frequency reaching 90% in 200,000 episodes and the rewards converging. While
most of the results are obtained by fixing a particular target flow shape to
simplify the learning problem, we later demonstrate how to transfer the
learning of an agent based on one target shape to another, i.e. from one design
to another and thus be useful for a generic design of a flow shape.
Freeform Diffractive Metagrating Design Based on Generative Adversarial Networks
Authors: Jiaqi Jiang, David Sell, Stephan Hoyer, Jason Hickey, Jianji Yang, Jonathan A. Fan
Date: 2018-11-29 00:00:00

A key challenge in metasurface design is the development of algorithms that can effectively and efficiently produce high performance devices. Design methods based on iterative optimization can push the performance limits of metasurfaces, but they require extensive computational resources that limit their implementation to small numbers of microscale devices. We show that generative neural networks can train from images of periodic, topology-optimized metagratings to produce high-efficiency, topologically complex devices operating over a broad range of deflection angles and wavelengths. Further iterative optimization of these designs yields devices with enhanced robustness and efficiencies, and these devices can be utilized as additional training data for network refinement. In this manner, generative networks can be trained, with a onetime computation cost, and used as a design tool to facilitate the production of near-optimal, topologically-complex device designs. We envision that such data-driven design methodologies can apply to other physical sciences domains that require the design of functional elements operating across a wide parameter space.
AdaFrame: Adaptive Frame Selection for Fast Video Recognition
Authors: Zuxuan Wu, Caiming Xiong, Chih-Yao Ma, Richard Socher, Larry S. Davis
Date: 2018-11-29 00:00:00
Proceeding: CVPR 2019 6
Tasks: Policy Gradient Methods, Video Recognition

We present AdaFrame, a framework that adaptively selects relevant frames on a
per-input basis for fast video recognition. AdaFrame contains a Long Short-Term
Memory network augmented with a global memory that provides context information
for searching which frames to use over time. Trained with policy gradient
methods, AdaFrame generates a prediction, determines which frame to observe
next, and computes the utility, i.e., expected future rewards, of seeing more
frames at each time step. At testing time, AdaFrame exploits predicted
utilities to achieve adaptive lookahead inference such that the overall
computational costs are reduced without incurring a decrease in accuracy.
Extensive experiments are conducted on two large-scale video benchmarks, FCVID
and ActivityNet. AdaFrame matches the performance of using all frames with only
8.21 and 8.65 frames on FCVID and ActivityNet, respectively. We further
qualitatively demonstrate learned frame usage can indicate the difficulty of
making classification decisions; easier samples need fewer frames while harder
ones require more, both at instance-level within the same class and at
class-level among different categories.
From Context to Concept: Exploring Semantic Relationships in Music with Word2Vec
Authors: Ching-Hua Chuan, Kat Agres, Dorien Herremans
Date: 2018-11-29 00:00:00
Tasks: Music Generation

We explore the potential of a popular distributional semantics vector space
model, word2vec, for capturing meaningful relationships in ecological (complex
polyphonic) music. More precisely, the skip-gram version of word2vec is used to
model slices of music from a large corpus spanning eight musical genres. In
this newly learned vector space, a metric based on cosine distance is able to
distinguish between functional chord relationships, as well as harmonic
associations in the music. Evidence, based on cosine distance between
chord-pair vectors, suggests that an implicit circle-of-fifths exists in the
vector space. In addition, a comparison between pieces in different keys
reveals that key relationships are represented in word2vec space. These results
suggest that the newly learned embedded vector representation does in fact
capture tonal and harmonic characteristics of music, without receiving explicit
information about the musical content of the constituent slices. In order to
investigate whether proximity in the discovered space of embeddings is
indicative of `semantically-related' slices, we explore a music generation
task, by automatically replacing existing slices from a given piece of music
with new slices. We propose an algorithm to find substitute slices based on
spatial proximity and the pitch class distribution inferred in the chosen
subspace. The results indicate that the size of the subspace used has a
significant effect on whether slices belonging to the same key are selected. In
sum, the proposed word2vec model is able to learn music-vector embeddings that
capture meaningful tonal and harmonic relationships in music, thereby providing
a useful tool for exploring musical properties and comparisons across pieces,
as a potential input representation for deep learning models, and as a music
generation device.
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models
Authors: Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, Artur Kadurin, Simon Johansson, Hongming Chen, Sergey Nikolenko, Alan Aspuru-Guzik, Alex Zhavoronkov
Date: 2018-11-29 00:00:00
Tasks: Benchmarking, Diversity, Drug Discovery, Molecular Graph Generation

Generative models are becoming a tool of choice for exploring the molecular space. These models learn on a large training dataset and produce novel molecular structures with similar properties. Generated structures can be utilized for virtual screening or training semi-supervised predictive models in the downstream tasks. While there are plenty of generative models, it is unclear how to compare and rank them. In this work, we introduce a benchmarking platform called Molecular Sets (MOSES) to standardize training and comparison of molecular generative models. MOSES provides a training and testing datasets, and a set of metrics to evaluate the quality and diversity of generated structures. We have implemented and compared several molecular generation models and suggest to use our results as reference points for further advancements in generative chemistry research. The platform and source code are available at https://github.com/molecularsets/moses.
Naive Dictionary On Musical Corpora: From Knowledge Representation To Pattern Recognition
Authors: Qiuyi Wu, Ernest Fokoue
Date: 2018-11-29 00:00:00

In this paper, we propose and develop the novel idea of treating musical
sheets as literary documents in the traditional text analytics parlance, to
fully benefit from the vast amount of research already existing in statistical
text mining and topic modelling. We specifically introduce the idea of
representing any given piece of music as a collection of "musical words" that
we codenamed "muselets", which are essentially musical words of various
lengths. Given the novelty and therefore the extremely difficulty of properly
forming a complete version of a dictionary of muselets, the present paper
focuses on a simpler albeit naive version of the ultimate dictionary, which we
refer to as a Naive Dictionary because of the fact that all the words are of
the same length. We specifically herein construct a naive dictionary featuring
a corpus made up of African American, Chinese, Japanese and Arabic music, on
which we perform both topic modelling and pattern recognition. Although some of
the results based on the Naive Dictionary are reasonably good, we anticipate
phenomenal predictive performances once we get around to actually building a
full scale complete version of our intended dictionary of muselets.
Customer Lifetime Value in Video Games Using Deep Learning and Parametric Models
Authors: Pei Pei Chen, Anna Guitart, Ana Fernández del Río, África Periáñez
Date: 2018-11-28 00:00:00
Tasks: Feature Engineering

Nowadays, video game developers record every virtual action performed by
their players. As each player can remain in the game for years, this results in
an exceptionally rich dataset that can be used to understand and predict player
behavior. In particular, this information may serve to identify the most
valuable players and foresee the amount of money they will spend in in-app
purchases during their lifetime. This is crucial in free-to-play games, where
up to 50% of the revenue is generated by just around 2% of the players, the
so-called whales.
  To address this challenge, we explore how deep neural networks can be used to
predict customer lifetime value in video games, and compare their performance
to parametric models such as Pareto/NBD. Our results suggest that convolutional
neural network structures are the most efficient in predicting the economic
value of individual players. They not only perform better in terms of accuracy,
but also scale to big data and significantly reduce computational time, as they
can work directly with raw sequential data and thus do not require any feature
engineering process. This becomes important when datasets are very large, as is
often the case with video game logs.
  Moreover, convolutional neural networks are particularly well suited to
identify potential whales. Such an early identification is of paramount
importance for business purposes, as it would allow developers to implement
in-game actions aimed at retaining big spenders and maximizing their lifetime,
which would ultimately translate into increased revenue.