pwc-archive/papers-with-abstracts huggingface dataset:
Collapse of Deep and Narrow Neural Nets
Authors: Lu Lu, Yanhui Su, George Em. Karniadakis
Date: 2018-08-15 00:00:00
Proceeding: ICLR 2019 5

Recent theoretical work has demonstrated that deep neural networks have
superior performance over shallow networks, but their training is more
difficult, e.g., they suffer from the vanishing gradient problem. This problem
can be typically resolved by the rectified linear unit (ReLU) activation.
However, here we show that even for such activation, deep and narrow neural
networks (NNs) will converge to erroneous mean or median states of the target
function depending on the loss with high probability. Deep and narrow NNs are
encountered in solving partial differential equations with high-order
derivatives. We demonstrate this collapse of such NNs both numerically and
theoretically, and provide estimates of the probability of collapse. We also
construct a diagram of a safe region for designing NNs that avoid the collapse
to erroneous states. Finally, we examine different ways of initialization and
normalization that may avoid the collapse problem. Asymmetric initializations
may reduce the probability of collapse but do not totally eliminate it.
Automatic Derivation Of Formulas Using Reforcement Learning
Authors: MinZhong Luo, Li Liu
Date: 2018-08-15 00:00:00
Tasks: Q-Learning

This paper presents an artificial intelligence algorithm that can be used to
derive formulas from various scientific disciplines called automatic derivation
machine. First, the formula is abstractly expressed as a multiway tree model,
and then each step of the formula derivation transformation is abstracted as a
mapping of multiway trees. Derivation steps similar can be expressed as a
reusable formula template by a multiway tree map. After that, the formula
multiway tree is eigen-encoded to feature vectors construct the feature space
of formulas, the Q-learning model using in this feature space can achieve the
derivation by making training data from derivation process. Finally, an
automatic formula derivation machine is made to choose the next derivation step
based on the current state and object. We also make an example about the
nuclear reactor physics problem to show how the automatic derivation machine
works.
Folksonomication: Predicting Tags for Movies from Plot Synopses Using Emotion Flow Encoded Neural Network
Authors: Sudipta Kar, Suraj Maharjan, Thamar Solorio
Date: 2018-08-15 00:00:00
Proceeding: COLING 2018 8
Tasks: Retrieval

Folksonomy of movies covers a wide range of heterogeneous information about
movies, like the genre, plot structure, visual experiences, soundtracks,
metadata, and emotional experiences from watching a movie. Being able to
automatically generate or predict tags for movies can help recommendation
engines improve retrieval of similar movies, and help viewers know what to
expect from a movie in advance. In this work, we explore the problem of
creating tags for movies from plot synopses. We propose a novel neural network
model that merges information from synopses and emotion flows throughout the
plots to predict a set of tags for movies. We compare our system with multiple
baselines and found that the addition of emotion flows boosts the performance
of the network by learning ~18\% more tags than a traditional machine learning
system.
Holographic Visualisation of Radiology Data and Automated Machine Learning-based Medical Image Segmentation
Authors: Lucian Trestioreanu
Date: 2018-08-15 00:00:00
Tasks: BIG-bench Machine Learning, Computed Tomography (CT), Image Segmentation, Liver Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation, Tumor Segmentation

Within this thesis we propose a platform for combining Augmented Reality (AR)
hardware with machine learning in a user-oriented pipeline, offering to the
medical staff an intuitive 3D visualization of volumetric Computed Tomography
(CT) and Magnetic Resonance Imaging (MRI) medical image segmentations inside
the AR headset, that does not need human intervention for loading, processing
and segmentation of medical images. The AR visualization, based on Microsoft
HoloLens, employs a modular and thus scalable frontend-backend architecture for
real-time visualizations on multiple AR headsets. As Convolutional Neural
Networks (CNNs) have lastly demonstrated superior performance for the machine
learning task of image semantic segmentation, the pipeline also includes a
fully automated CNN algorithm for the segmentation of the liver from CT scans.
The model is based on the Deep Retinal Image Understanding (DRIU) model which
is a Fully Convolutional Network with side outputs from feature maps with
different resolution, extracted at different stages of the network. The
algorithm is 2.5D which means that the input is a set of consecutive scan
slices. The experiments have been performed on the Liver Tumor Segmentation
Challenge (LiTS) dataset for liver segmentation and demonstrated good results
and flexibility. While multiple approaches exist in the domain, only few of
them have focused on overcoming the practical aspects which still largely hold
this technology away from the operating rooms. In line with this, we also are
next planning an evaluation from medical doctors and radiologists in a
real-world environment.
Deep EHR: Chronic Disease Prediction Using Medical Notes
Authors: Jingshu Liu, Zachariah Zhang, Narges Razavian
Date: 2018-08-15 00:00:00
Proceeding: Machine Learning for Health Care conference 2018 9
Tasks: Disease Prediction, Management, Prediction

Early detection of preventable diseases is important for better disease
management, improved inter-ventions, and more efficient health-care resource
allocation. Various machine learning approacheshave been developed to utilize
information in Electronic Health Record (EHR) for this task. Majorityof
previous attempts, however, focus on structured fields and lose the vast amount
of information inthe unstructured notes. In this work we propose a general
multi-task framework for disease onsetprediction that combines both free-text
medical notes and structured information. We compareperformance of different
deep learning architectures including CNN, LSTM and hierarchical models.In
contrast to traditional text-based prediction models, our approach does not
require disease specificfeature engineering, and can handle negations and
numerical values that exist in the text. Ourresults on a cohort of about 1
million patients show that models using text outperform modelsusing just
structured data, and that models capable of using numerical values and
negations in thetext, in addition to the raw text, further improve performance.
Additionally, we compare differentvisualization methods for medical
professionals to interpret model predictions.
How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
Authors: Divyansh Kaushik, Zachary C. Lipton
Date: 2018-08-14 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Reading Comprehension, Sentence

Many recent papers address reading comprehension, where examples consist of
(question, passage, answer) tuples. Presumably, a model must combine
information from both questions and passages to predict corresponding answers.
However, despite intense interest in the topic, with hundreds of published
papers vying for leaderboard dominance, basic questions about the difficulty of
many popular benchmarks remain unanswered. In this paper, we establish sensible
baselines for the bAbI, SQuAD, CBT, CNN, and Who-did-What datasets, finding
that question- and passage-only models often perform surprisingly well. On $14$
out of $20$ bAbI tasks, passage-only models achieve greater than $50\%$
accuracy, sometimes matching the full model. Interestingly, while CBT provides
$20$-sentence stories only the last is needed for comparably accurate
prediction. By comparison, SQuAD and CNN appear better-constructed.
Cross-Lingual Cross-Platform Rumor Verification Pivoting on Multimedia Content
Authors: Weiming Wen, Songwen Su, Zhou Yu
Date: 2018-08-14 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Semantic Similarity, Semantic Textual Similarity

With the increasing popularity of smart devices, rumors with multimedia
content become more and more common on social networks. The multimedia
information usually makes rumors look more convincing. Therefore, finding an
automatic approach to verify rumors with multimedia content is a pressing task.
Previous rumor verification research only utilizes multimedia as input
features. We propose not to use the multimedia content but to find external
information in other news platforms pivoting on it. We introduce a new features
set, cross-lingual cross-platform features that leverage the semantic
similarity between the rumors and the external information. When implemented,
machine learning methods utilizing such features achieved the state-of-the-art
rumor verification results.
Vendor-independent soft tissue lesion detection using weakly supervised and unsupervised adversarial domain adaptation
Authors: Joris van Vugt, Elena Marchiori, Ritse Mann, Albert Gubern-Mérida, Nikita Moriakov, Jonas Teuwen
Date: 2018-08-14 00:00:00
Tasks: Domain Adaptation, Lesion Detection, Sensitivity, Transfer Learning

Computer-aided detection aims to improve breast cancer screening programs by
helping radiologists to evaluate digital mammography (DM) exams. DM exams are
generated by devices from different vendors, with diverse characteristics
between and even within vendors. Physical properties of these devices and
postprocessing of the images can greatly influence the resulting mammogram.
This results in the fact that a deep learning model trained on data from one
vendor cannot readily be applied to data from another vendor. This paper
investigates the use of tailored transfer learning methods based on adversarial
learning to tackle this problem. We consider a database of DM exams (mostly
bilateral and two views) generated by Hologic and Siemens vendors. We analyze
two transfer learning settings: 1) unsupervised transfer, where Hologic data
with soft lesion annotation at pixel level and Siemens unlabelled data are used
to annotate images in the latter data; 2) weak supervised transfer, where exam
level labels for images from the Siemens mammograph are available. We propose
tailored variants of recent state-of-the-art methods for transfer learning
which take into account the class imbalance and incorporate knowledge provided
by the annotations at exam level. Results of experiments indicate the
beneficial effect of transfer learning in both transfer settings. Notably, at
0.02 false positives per image, we achieve a sensitivity of 0.37, compared to
0.30 of a baseline with no transfer. Results indicate that using exam level
annotations gives an additional increase in sensitivity.
Embedding Grammars
Authors: David Wingate, William Myers, Nancy Fulda, Tyler Etchart
Date: 2018-08-14 00:00:00
Tasks: Intent Detection, Word Embeddings

Classic grammars and regular expressions can be used for a variety of
purposes, including parsing, intent detection, and matching. However, the
comparisons are performed at a structural level, with constituent elements
(words or characters) matched exactly. Recent advances in word embeddings show
that semantically related words share common features in a vector-space
representation, suggesting the possibility of a hybrid grammar and word
embedding. In this paper, we blend the structure of standard context-free
grammars with the semantic generalization capabilities of word embeddings to
create hybrid semantic grammars. These semantic grammars generalize the
specific terminals used by the programmer to other words and phrases with
related meanings, allowing the construction of compact grammars that match an
entire region of the vector space rather than matching specific elements.
Skill Rating for Generative Models
Authors: Catherine Olsson, Surya Bhupatiraju, Tom Brown, Augustus Odena, Ian Goodfellow
Date: 2018-08-14 00:00:00

We explore a new way to evaluate generative models using insights from
evaluation of competitive games between human players. We show experimentally
that tournaments between generators and discriminators provide an effective way
to evaluate generative models. We introduce two methods for summarizing
tournament outcomes: tournament win rate and skill rating. Evaluations are
useful in different contexts, including monitoring the progress of a single
model as it learns during the training process, and comparing the capabilities
of two different fully trained models. We show that a tournament consisting of
a single model playing against past and future versions of itself produces a
useful measure of training progress. A tournament containing multiple separate
models (using different seeds, hyperparameters, and architectures) provides a
useful relative comparison between different trained GANs. Tournament-based
rating methods are conceptually distinct from numerous previous categories of
approaches to evaluation of generative models, and have complementary
advantages and disadvantages.
A Precision Environment-Wide Association Study of Hypertension via Supervised Cadre Models
Authors: Alexander New, Kristin P. Bennett
Date: 2018-08-14 00:00:00
Tasks: Binary Classification, Clustering

We consider the problem in precision health of grouping people into
subpopulations based on their degree of vulnerability to a risk factor. These
subpopulations cannot be discovered with traditional clustering techniques
because their quality is evaluated with a supervised metric: the ease of
modeling a response variable over observations within them. Instead, we apply
the supervised cadre model (SCM), which does use this metric. We extend the SCM
formalism so that it may be applied to multivariate regression and binary
classification problems. We also develop a way to use conditional entropy to
assess the confidence in the process by which a subject is assigned their
cadre. Using the SCM, we generalize the environment-wide association study
(EWAS) workflow to be able to model heterogeneity in population risk. In our
EWAS, we consider more than two hundred environmental exposure factors and find
their association with diastolic blood pressure, systolic blood pressure, and
hypertension. This requires adapting the SCM to be applicable to data generated
by a complex survey design. After correcting for false positives, we found 25
exposure variables that had a significant association with at least one of our
response variables. Eight of these were significant for a discovered
subpopulation but not for the overall population. Some of these associations
have been identified by previous researchers, while others appear to be novel.
We examine several discovered subpopulations in detail, and we find that they
are interpretable and that they suggest further research questions.
Latent Agents in Networks: Estimation and Targeting
Authors: Baris Ata, Alexandre Belloni, Ozan Candogan
Date: 2018-08-14 00:00:00

We consider a network of agents. Associated with each agent are her covariate and outcome. Agents influence each other's outcomes according to a certain connection/influence structure. A subset of the agents participate on a platform, and hence, are observable to it. The rest are not observable to the platform and are called the latent agents. The platform does not know the influence structure of the observable or the latent parts of the network. It only observes the data on past covariates and decisions of the observable agents. Observable agents influence each other both directly and indirectly through the influence they exert on the latent agents. We investigate how the platform can estimate the dependence of the observable agents' outcomes on their covariates, taking the latent agents into account. First, we show that this relationship can be succinctly captured by a matrix and provide an algorithm for estimating it under a suitable approximate sparsity condition using historical data of covariates and outcomes for the observable agents. We also obtain convergence rates for the proposed estimator despite the high dimensionality that allows more agents than observations. Second, we show that the approximate sparsity condition holds under the standard conditions used in the literature. Hence, our results apply to a large class of networks. Finally, we apply our results to two practical settings: targeted advertising and promotional pricing. We show that by using the available historical data with our estimator, it is possible to obtain asymptotically optimal advertising/pricing decisions, despite the presence of latent agents.
Multi-user Communication Networks: A Coordinated Multi-armed Bandit Approach
Authors: Orly Avner, Shie Mannor
Date: 2018-08-14 00:00:00

Communication networks shared by many users are a widespread challenge
nowadays. In this paper we address several aspects of this challenge
simultaneously: learning unknown stochastic network characteristics, sharing
resources with other users while keeping coordination overhead to a minimum.
The proposed solution combines Multi-Armed Bandit learning with a lightweight
signalling-based coordination scheme, and ensures convergence to a stable
allocation of resources. Our work considers single-user level algorithms for
two scenarios: an unknown fixed number of users, and a dynamic number of users.
Analytic performance guarantees, proving convergence to stable marriage
configurations, are presented for both setups. The algorithms are designed
based on a system-wide perspective, rather than focusing on single user
welfare. Thus, maximal resource utilization is ensured. An extensive
experimental analysis covers convergence to a stable configuration as well as
reward maximization. Experiments are carried out over a wide range of setups,
demonstrating the advantages of our approach over existing state-of-the-art
methods.
Generalization of Equilibrium Propagation to Vector Field Dynamics
Authors: Benjamin Scellier, Anirudh Goyal, Jonathan Binas, Thomas Mesnard, Yoshua Bengio
Date: 2018-08-14 00:00:00

The biological plausibility of the backpropagation algorithm has long been
doubted by neuroscientists. Two major reasons are that neurons would need to
send two different types of signal in the forward and backward phases, and that
pairs of neurons would need to communicate through symmetric bidirectional
connections. We present a simple two-phase learning procedure for fixed point
recurrent networks that addresses both these issues. In our model, neurons
perform leaky integration and synaptic weights are updated through a local
mechanism. Our learning method generalizes Equilibrium Propagation to vector
field dynamics, relaxing the requirement of an energy function. As a
consequence of this generalization, the algorithm does not compute the true
gradient of the objective function, but rather approximates it at a precision
which is proven to be directly related to the degree of symmetry of the
feedforward and feedback weights. We show experimentally that our algorithm
optimizes the objective function.
Mitigating Sybils in Federated Learning Poisoning
Authors: Clement Fung, Chris J. M. Yoon, Ivan Beschastnikh
Date: 2018-08-14 00:00:00
Tasks: Diversity, Federated Learning, Model Poisoning

Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susceptible to a variety of attacks, including model poisoning, which is made substantially worse in the presence of sybils. In this paper we first evaluate the vulnerability of federated learning to sybil-based poisoning attacks. We then describe \emph{FoolsGold}, a novel defense to this problem that identifies poisoning sybils based on the diversity of client updates in the distributed learning process. Unlike prior work, our system does not bound the expected number of attackers, requires no auxiliary information outside of the learning process, and makes fewer assumptions about clients and their data. In our evaluation we show that FoolsGold exceeds the capabilities of existing state of the art approaches to countering sybil-based label-flipping and backdoor poisoning attacks. Our results hold for different distributions of client data, varying poisoning targets, and various sybil strategies. Code can be found at: https://github.com/DistributedML/FoolsGold
Top-Down Tree Structured Text Generation
Authors: Qipeng Guo, Xipeng Qiu, xiangyang xue, Zheng Zhang
Date: 2018-08-14 00:00:00
Tasks: Sentence, Text Generation

Text generation is a fundamental building block in natural language
processing tasks. Existing sequential models performs autoregression directly
over the text sequence and have difficulty generating long sentences of complex
structures. This paper advocates a simple approach that treats sentence
generation as a tree-generation task. By explicitly modelling syntactic
structures in a constituent syntactic tree and performing top-down,
breadth-first tree generation, our model fixes dependencies appropriately and
performs implicit global planning. This is in contrast to transition-based
depth-first generation process, which has difficulty dealing with incomplete
texts when parsing and also does not incorporate future contexts in planning.
Our preliminary results on two generation tasks and one parsing task
demonstrate that this is an effective strategy.
GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
Authors: Hao Tang, Wei Wang, Dan Xu, Yan Yan, Nicu Sebe
Date: 2018-08-14 00:00:00
Tasks: Data Augmentation, Generative Adversarial Network, Gesture-to-Gesture Translation, Translation

Hand gesture-to-gesture translation in the wild is a challenging task since hand gestures can have arbitrary poses, sizes, locations and self-occlusions. Therefore, this task requires a high-level understanding of the mapping between the input source gesture and the output target gesture. To tackle this problem, we propose a novel hand Gesture Generative Adversarial Network (GestureGAN). GestureGAN consists of a single generator $G$ and a discriminator $D$, which takes as input a conditional hand image and a target hand skeleton image. GestureGAN utilizes the hand skeleton information explicitly, and learns the gesture-to-gesture mapping through two novel losses, the color loss and the cycle-consistency loss. The proposed color loss handles the issue of "channel pollution" while back-propagating the gradients. In addition, we present the Fr\'echet ResNet Distance (FRD) to evaluate the quality of generated images. Extensive experiments on two widely used benchmark datasets demonstrate that the proposed GestureGAN achieves state-of-the-art performance on the unconstrained hand gesture-to-gesture translation task. Meanwhile, the generated images are in high-quality and are photo-realistic, allowing them to be used as data augmentation to improve the performance of a hand gesture classifier. Our model and code are available at https://github.com/Ha0Tang/GestureGAN.
Two Local Models for Neural Constituent Parsing
Authors: Zhiyang Teng, Yue Zhang
Date: 2018-08-14 00:00:00
Proceeding: COLING 2018 8
Tasks: Vocal Bursts Valence Prediction

Non-local features have been exploited by syntactic parsers for capturing
dependencies between sub output structures. Such features have been a key to
the success of state-of-the-art statistical parsers. With the rise of deep
learning, however, it has been shown that local output decisions can give
highly competitive accuracies, thanks to the power of dense neural input
representations that embody global syntactic information. We investigate two
conceptually simple local neural models for constituent parsing, which make
local decisions to constituent spans and CFG rules, respectively. Consistent
with previous findings along the line, our best model gives highly competitive
results, achieving the labeled bracketing F1 scores of 92.4% on PTB and 87.3%
on CTB 5.1.