pwc-archive/papers-with-abstracts huggingface dataset:
Workflow-Driven Distributed Machine Learning in CHASE-CI: A Cognitive Hardware and Software Ecosystem Community Infrastructure
Authors: Ilkay Altintas, Kyle Marcus, Isaac Nealey, Scott L. Sellars, John Graham, Dima Mishin, Joel Polizzi, Daniel Crawl, Thomas DeFanti, Larry Smarr
Date: 2019-02-26 00:00:00
Tasks: BIG-bench Machine Learning, GPU, Management

The advances in data, computing and networking over the last two decades led
to a shift in many application domains that includes machine learning on big
data as a part of the scientific process, requiring new capabilities for
integrated and distributed hardware and software infrastructure. This paper
contributes a workflow-driven approach for dynamic data-driven application
development on top of a new kind of networked Cyberinfrastructure called
CHASE-CI. In particular, we present: 1) The architecture for CHASE-CI, a
network of distributed fast GPU appliances for machine learning and storage
managed through Kubernetes on the high-speed (10-100Gbps) Pacific Research
Platform (PRP); 2) A machine learning software containerization approach and
libraries required for turning such a network into a distributed computer for
big data analysis; 3) An atmospheric science case study that can only be made
scalable with an infrastructure like CHASE-CI; 4) Capabilities for virtual
cluster management for data communication and analysis in a dynamically
scalable fashion, and visualization across the network in specialized
visualization facilities in near real-time; and, 5) A step-by-step workflow and
performance measurement approach that enables taking advantage of the dynamic
architecture of the CHASE-CI network and container management infrastructure.
Learning Hierarchical Discourse-level Structure for Fake News Detection
Authors: Hamid Karimi, Jiliang Tang
Date: 2019-02-27 00:00:00
Proceeding: NAACL 2019 6
Tasks: Articles, Fake News Detection

On the one hand, nowadays, fake news articles are easily propagated through
various online media platforms and have become a grand threat to the
trustworthiness of information. On the other hand, our understanding of the
language of fake news is still minimal. Incorporating hierarchical
discourse-level structure of fake and real news articles is one crucial step
toward a better understanding of how these articles are structured.
Nevertheless, this has rarely been investigated in the fake news detection
domain and faces tremendous challenges. First, existing methods for capturing
discourse-level structure rely on annotated corpora which are not available for
fake news datasets. Second, how to extract out useful information from such
discovered structures is another challenge. To address these challenges, we
propose Hierarchical Discourse-level Structure for Fake news detection. HDSF
learns and constructs a discourse-level structure for fake/real news articles
in an automated and data-driven manner. Moreover, we identify insightful
structure-related properties, which can explain the discovered structures and
boost our understating of fake news. Conducted experiments show the
effectiveness of the proposed approach. Further structural analysis suggests
that real and fake news present substantial differences in the hierarchical
discourse-level structures.
Deep Text-to-Speech System with Seq2Seq Model
Authors: Gary Wang
Date: 2019-03-11 00:00:00
Tasks: model, Speech Synthesis, text-to-speech, Text to Speech

Recent trends in neural network based text-to-speech/speech synthesis
pipelines have employed recurrent Seq2seq architectures that can synthesize
realistic sounding speech directly from text characters. These systems however
have complex architectures and takes a substantial amount of time to train. We
introduce several modifications to these Seq2seq architectures that allow for
faster training time, and also allows us to reduce the complexity of the model
architecture at the same time. We show that our proposed model can achieve
attention alignment much faster than previous architectures and that good audio
quality can be achieved with a model that's much smaller in size. Sample audio
available at https://soundcloud.com/gary-wang-23/sets/tts-samples-for-cmpt-419.
Automatic Classification of Pathology Reports using TF-IDF Features
Authors: Shivam Kalra, Larry Li, Hamid. R. Tizhoosh
Date: 2019-03-05 00:00:00
Tasks: Classification, Diagnostic, General Classification, Optical Character Recognition (OCR)

A Pathology report is arguably one of the most important documents in
medicine containing interpretive information about the visual findings from the
patient's biopsy sample. Each pathology report has a retention period of up to
20 years after the treatment of a patient. Cancer registries process and encode
high volumes of free-text pathology reports for surveillance of cancer and
tumor diseases all across the world. In spite of their extremely valuable
information they hold, pathology reports are not used in any systematic way to
facilitate computational pathology. Therefore, in this study, we investigate
automated machine-learning techniques to identify/predict the primary diagnosis
(based on ICD-O code) from pathology reports. We performed experiments by
extracting the TF-IDF features from the reports and classifying them using
three different methods---SVM, XGBoost, and Logistic Regression. We constructed
a new dataset with 1,949 pathology reports arranged into 37 ICD-O categories,
collected from four different primary sites, namely lung, kidney, thymus, and
testis. The reports were manually transcribed into text format after collecting
them as PDF files from NCI Genomic Data Commons public dataset. We subsequently
pre-processed the reports by removing irrelevant textual artifacts produced by
OCR software. The highest classification accuracy we achieved was 92\% using
XGBoost classifier on TF-IDF feature vectors, the linear SVM scored 87\%
accuracy. Furthermore, the study shows that TF-IDF vectors are suitable for
highlighting the important keywords within a report which can be helpful for
the cancer research and diagnostic workflow. The results are encouraging in
demonstrating the potential of machine learning methods for classification and
encoding of pathology reports.
Evolutionary Deep Learning to Identify Galaxies in the Zone of Avoidance
Authors: David Jones, Anja Schroeder, Geoff Nitschke
Date: 2019-03-06 00:00:00
Tasks: Astronomy

The Zone of Avoidance makes it difficult for astronomers to catalogue
galaxies at low latitudes to our galactic plane due to high star densities and
extinction. However, having a complete sky map of galaxies is important in a
number of fields of research in astronomy. There are many unclassified sources
of light in the Zone of Avoidance and it is therefore important that there
exists an accurate automated system to identify and classify galaxies in this
region. This study aims to evaluate the efficiency and accuracy of using an
evolutionary algorithm to evolve the topology and configuration of
Convolutional Neural Network (CNNs) to automatically identify galaxies in the
Zone of Avoidance. A supervised learning method is used with data containing
near-infrared images. Input image resolution and number of near-infrared
passbands needed by the evolutionary algorithm is also analyzed while the
accuracy of the best evolved CNN is compared to other CNN variants.
Un duel probabiliste pour départager deux présidents (LIA @ DEFT'2005)
Authors: Marc El-Bèze, Juan-Manuel Torres-Moreno, Frédéric Béchet
Date: 2019-03-11 00:00:00
Tasks: Binary Classification

We present a set of probabilistic models applied to binary classification as
defined in the DEFT'05 challenge. The challenge consisted a mixture of two
differents problems in Natural Language Processing : identification of author
(a sequence of Fran\c{c}ois Mitterrand's sentences might have been inserted
into a speech of Jacques Chirac) and thematic break detection (the subjects
addressed by the two authors are supposed to be different). Markov chains,
Bayes models and an adaptative process have been used to identify the paternity
of these sequences. A probabilistic model of the internal coherence of speeches
which has been employed to identify thematic breaks. Adding this model has
shown to improve the quality results. A comparison with different approaches
demostrates the superiority of a strategy that combines learning, coherence and
adaptation. Applied to the DEFT'05 data test the results in terms of precision
(0.890), recall (0.955) and Fscore (0.925) measure are very promising.
Multiparametric Deep Learning Tissue Signatures for a Radiological Biomarker of Breast Cancer: Preliminary Results
Authors: Vishwa S. Parekh, Katarzyna J. Macura, Susan Harvey, Ihab Kamel, Riham EI-Khouli, David A. Bluemke, Michael A. Jacobs
Date: 2018-02-10 00:00:00
Tasks: Deep Learning, Sensitivity, Specificity

A new paradigm is beginning to emerge in Radiology with the advent of
increased computational capabilities and algorithms. This has led to the
ability of real time learning by computer systems of different lesion types to
help the radiologist in defining disease. For example, using a deep learning
network, we developed and tested a multiparametric deep learning (MPDL) network
for segmentation and classification using multiparametric magnetic resonance
imaging (mpMRI) radiological images. The MPDL network was constructed from
stacked sparse autoencoders with inputs from mpMRI. Evaluation of MPDL
consisted of cross-validation, sensitivity, and specificity. Dice similarity
between MPDL and post-DCE lesions were evaluated. We demonstrate high
sensitivity and specificity for differentiation of malignant from benign
lesions of 90% and 85% respectively with an AUC of 0.93. The Integrated MPDL
method accurately segmented and classified different breast tissue from
multiparametric breast MRI using deep leaning tissue signatures.
Sampling and Reconstruction of Signals on Product Graphs
Authors: Guillermo Ortiz-Jiménez, Mario Coutino, Sundeep Prabhakar Chepuri, Geert Leus
Date: 2018-06-30 00:00:00
Tasks: Active Learning, Recommendation Systems, Time Series, Time Series Analysis

In this paper, we consider the problem of subsampling and reconstruction of
signals that reside on the vertices of a product graph, such as sensor network
time series, genomic signals, or product ratings in a social network.
Specifically, we leverage the product structure of the underlying domain and
sample nodes from the graph factors. The proposed scheme is particularly useful
for processing signals on large-scale product graphs. The sampling sets are
designed using a low-complexity greedy algorithm and can be proven to be
near-optimal. To illustrate the developed theory, numerical experiments based
on real datasets are provided for sampling 3D dynamic point clouds and for
active learning in recommender systems.
Kernel-based Translations of Convolutional Networks
Authors: Corinne Jones, Vincent Roulet, Zaid Harchaoui
Date: 2019-03-19 00:00:00
Tasks: Translation

Convolutional Neural Networks, as most artificial neural networks, are
commonly viewed as methods different in essence from kernel-based methods. We
provide a systematic translation of Convolutional Neural Networks (ConvNets)
into their kernel-based counterparts, Convolutional Kernel Networks (CKNs), and
demonstrate that this perception is unfounded both formally and empirically. We
show that, given a Convolutional Neural Network, we can design a corresponding
Convolutional Kernel Network, easily trainable using a new stochastic gradient
algorithm based on an accurate gradient computation, that performs on par with
its Convolutional Neural Network counterpart. We present experimental results
supporting our claims on landmark ConvNet architectures comparing each ConvNet
to its CKN counterpart over several parameter settings.
Corners for Layout: End-to-End Layout Recovery from 360 Images
Authors: Clara Fernandez-Labrador, Jose M. Facil, Alejandro Perez-Yus, Cédric Demonceaux, Javier Civera, Jose J. Guerrero
Date: 2019-03-19 00:00:00
Tasks: 3D Room Layouts From A Single RGB Panorama, Robot Navigation

The problem of 3D layout recovery in indoor scenes has been a core research
topic for over a decade. However, there are still several major challenges that
remain unsolved. Among the most relevant ones, a major part of the
state-of-the-art methods make implicit or explicit assumptions on the scenes --
e.g. box-shaped or Manhattan layouts. Also, current methods are computationally
expensive and not suitable for real-time applications like robot navigation and
AR/VR. In this work we present CFL (Corners for Layout), the first end-to-end
model for 3D layout recovery on 360 images. Our experimental results show that
we outperform the state of the art relaxing assumptions about the scene and at
a lower cost. We also show that our model generalizes better to camera position
variations than conventional approaches by using EquiConvs, a type of
convolution applied directly on the sphere projection and hence invariant to
the equirectangular distortions.
  CFL Webpage: https://cfernandezlab.github.io/CFL/
Hyper-Parameter Sweep on AlphaZero General
Authors: Hui Wang, Michael Emmerich, Mike Preuss, Aske Plaat
Date: 2019-03-19 00:00:00
Tasks: Game of Go

Since AlphaGo and AlphaGo Zero have achieved breakground successes in the
game of Go, the programs have been generalized to solve other tasks.
Subsequently, AlphaZero was developed to play Go, Chess and Shogi. In the
literature, the algorithms are explained well. However, AlphaZero contains many
parameters, and for neither AlphaGo, AlphaGo Zero nor AlphaZero, there is
sufficient discussion about how to set parameter values in these algorithms.
Therefore, in this paper, we choose 12 parameters in AlphaZero and evaluate how
these parameters contribute to training. We focus on three objectives~(training
loss, time cost and playing strength). For each parameter, we train 3 models
using 3 different values~(minimum value, default value, maximum value). We use
the game of play 6$\times$6 Othello, on the AlphaZeroGeneral open source
re-implementation of AlphaZero. Overall, experimental results show that
different values can lead to different training results, proving the importance
of such a parameter sweep. We categorize these 12 parameters into
time-sensitive parameters and time-friendly parameters. Moreover, through
multi-objective analysis, this paper provides an insightful basis for further
hyper-parameter optimization.
Exact Gaussian Processes on a Million Data Points
Authors: Ke Alexander Wang, Geoff Pleiss, Jacob R. Gardner, Stephen Tyree, Kilian Q. Weinberger, Andrew Gordon Wilson
Date: 2019-03-19 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Gaussian Processes, GPU

Gaussian processes (GPs) are flexible non-parametric models, with a capacity that grows with the available data. However, computational constraints with standard inference procedures have limited exact GPs to problems with fewer than about ten thousand training points, necessitating approximations for larger datasets. In this paper, we develop a scalable approach for exact GPs that leverages multi-GPU parallelization and methods like linear conjugate gradients, accessing the kernel matrix only through matrix multiplication. By partitioning and distributing kernel matrix multiplies, we demonstrate that an exact GP can be trained on over a million points, a task previously thought to be impossible with current computing hardware, in less than 2 hours. Moreover, our approach is generally applicable, without constraints to grid data or specific kernel classes. Enabled by this scalability, we perform the first-ever comparison of exact GPs against scalable GP approximations on datasets with $10^4 \!-\! 10^6$ data points, showing dramatic performance improvements.
Identifying Experts in Software Libraries and Frameworks among GitHub Users
Authors: Joao Eduardo Montandon, Luciana Lourdes Silva, Marco Tulio Valente
Date: 2019-03-19 00:00:00
Tasks: BIG-bench Machine Learning, Clustering

Software development increasingly depends on libraries and frameworks to
increase productivity and reduce time-to-market. Despite this fact, we still
lack techniques to assess developers expertise in widely popular libraries and
frameworks. In this paper, we evaluate the performance of unsupervised (based
on clustering) and supervised machine learning classifiers (Random Forest and
SVM) to identify experts in three popular JavaScript libraries: facebook/react,
mongodb/node-mongodb, and socketio/socket.io. First, we collect 13 features
about developers activity on GitHub projects, including commits on source code
files that depend on these libraries. We also build a ground truth including
the expertise of 575 developers on the studied libraries, as self-reported by
them in a survey. Based on our findings, we document the challenges of using
machine learning classifiers to predict expertise in software libraries, using
features extracted from GitHub. Then, we propose a method to identify library
experts based on clustering feature data from GitHub; by triangulating the
results of this method with information available on Linkedin profiles, we show
that it is able to recommend dozens of GitHub users with evidences of being
experts in the studied JavaScript libraries. We also provide a public dataset
with the expertise of 575 developers on the studied libraries.
Preconditioned P-ULA for Joint Deconvolution-Segmentation of Ultrasound Images -- Extended Version
Authors: Corbineau Marie-Caroline, Kouamé Denis, Chouzenoux Emilie, Tourneret Jean-Yves, Pesquet Jean-Christophe
Date: 2019-03-19 00:00:00
Tasks: Segmentation

Joint deconvolution and segmentation of ultrasound images is a challenging problem in medical imaging. By adopting a hierarchical Bayesian model, we propose an accelerated Markov chain Monte Carlo scheme where the tissue reflectivity function is sampled thanks to a recently introduced proximal unadjusted Langevin algorithm. This new approach is combined with a forward-backward step and a preconditioning strategy to accelerate the convergence, and with a method based on the majorization-minimization principle to solve the inner nonconvex minimization problems. As demonstrated in numerical experiments conducted on both simulated and in vivo ultrasound images, the proposed method provides high-quality restoration and segmentation results and is up to six times faster than an existing Hamiltonian Monte Carlo method.
Online Non-Convex Learning: Following the Perturbed Leader is Optimal
Authors: Arun Sai Suggala, Praneeth Netrapalli
Date: 2019-03-19 00:00:00

We study the problem of online learning with non-convex losses, where the learner has access to an offline optimization oracle. We show that the classical Follow the Perturbed Leader (FTPL) algorithm achieves optimal regret rate of $O(T^{-1/2})$ in this setting. This improves upon the previous best-known regret rate of $O(T^{-1/3})$ for FTPL. We further show that an optimistic variant of FTPL achieves better regret bounds when the sequence of losses encountered by the learner is `predictable'.
Residual Deep Convolutional Neural Network for EEG Signal Classification in Epilepsy
Authors: Diyuan Lu, Jochen Triesch
Date: 2019-03-19 00:00:00
Tasks: EEG, EEG Signal Classification, Electroencephalogram (EEG), General Classification

Epilepsy is the fourth most common neurological disorder, affecting about 1%
of the population at all ages. As many as 60% of people with epilepsy
experience focal seizures which originate in a certain brain area and are
limited to part of one cerebral hemisphere. In focal epilepsy patients, a
precise surgical removal of the seizure onset zone can lead to effective
seizure control or even a seizure-free outcome. Thus, correct identification of
the seizure onset zone is essential. For clinical evaluation purposes,
electroencephalography (EEG) recordings are commonly used. However, their
interpretation is usually done manually by physicians and is time-consuming and
error-prone. In this work, we propose an automated epileptic signal
classification method based on modern deep learning methods. In contrast to
previous approaches, the network is trained directly on the EEG recordings,
avoiding hand-crafted feature extraction and selection procedures. This
exploits the ability of deep neural networks to detect and extract relevant
features automatically, that may be too complex or subtle to be noticed by
humans. The proposed network structure is based on a convolutional neural
network with residual connections. We demonstrate that our network produces
state-of-the-art performance on two benchmark data sets, a data set from Bonn
University and the Bern-Barcelona data set. We conclude that modern deep
learning approaches can reach state-of-the-art performance on epileptic EEG
classification and automated seizure onset zone identification tasks when
trained on raw EEG data. This suggests that such approaches have potential for
improving clinical practice.
Natural Language Generation at Scale: A Case Study for Open Domain Question Answering
Authors: Alessandra Cervone, Chandra Khatri, Rahul Goel, Behnam Hedayatnia, Anu Venkatesh, Dilek Hakkani-Tur, Raefer Gabriel
Date: 2019-03-19 00:00:00
Proceeding: WS 2019 10
Tasks: Decoder, Knowledge Graphs, Multi-Task Learning, Open-Domain Question Answering, Question Answering, Response Generation, Text Generation

Current approaches to Natural Language Generation (NLG) for dialog mainly focus on domain-specific, task-oriented applications (e.g. restaurant booking) using limited ontologies (up to 20 slot types), usually without considering the previous conversation context. Furthermore, these approaches require large amounts of data for each domain, and do not benefit from examples that may be available for other domains. This work explores the feasibility of applying statistical NLG to scenarios requiring larger ontologies, such as multi-domain dialog applications or open-domain question answering (QA) based on knowledge graphs. We model NLG through an Encoder-Decoder framework using a large dataset of interactions between real-world users and a conversational agent for open-domain QA. First, we investigate the impact of increasing the number of slot types on the generation quality and experiment with different partitions of the QA data with progressively larger ontologies (up to 369 slot types). Second, we perform multi-task learning experiments between open-domain QA and task-oriented dialog, and benchmark our model on a popular NLG dataset. Moreover, we experiment with using the conversational context as an additional input to improve response generation quality. Our experiments show the feasibility of learning statistical NLG models for open-domain QA with larger ontologies.
Learning Reciprocity in Complex Sequential Social Dilemmas
Authors: Tom Eccles, Edward Hughes, János Kramár, Steven Wheelwright, Joel Z. Leibo
Date: 2019-03-19 00:00:00
Tasks: Reinforcement Learning

Reciprocity is an important feature of human social interaction and underpins
our cooperative nature. What is more, simple forms of reciprocity have proved
remarkably resilient in matrix game social dilemmas. Most famously, the
tit-for-tat strategy performs very well in tournaments of Prisoner's Dilemma.
Unfortunately this strategy is not readily applicable to the real world, in
which options to cooperate or defect are temporally and spatially extended.
Here, we present a general online reinforcement learning algorithm that
displays reciprocal behavior towards its co-players. We show that it can induce
pro-social outcomes for the wider group when learning alongside selfish agents,
both in a $2$-player Markov game, and in $5$-player intertemporal social
dilemmas. We analyse the resulting policies to show that the reciprocating
agents are strongly influenced by their co-players' behavior.