pwc-archive/papers-with-abstracts huggingface dataset:
Type-Driven Automated Learning with Lale
Authors: Martin Hirzel, Kiran Kate, Avraham Shinnar, Subhrajit Roy, Parikshit Ram
Date: 2019-05-24 00:00:00
Tasks: Time Series, Time Series Analysis, Vocal Bursts Type Prediction

Machine-learning automation tools, ranging from humble grid-search to hyperopt, auto-sklearn, and TPOT, help explore large search spaces of possible pipelines. Unfortunately, each of these tools has a different syntax for specifying its search space, leading to lack of portability, missed relevant points, and spurious points that are inconsistent with error checks and documentation of the searchable base components. This paper proposes using types (such as enum, float, or dictionary) both for checking the correctness of, and for automatically searching over, hyperparameters and pipeline configurations. Using types for both of these purposes guarantees consistency. We present Lale, an embedded language that resembles scikit learn but provides better automation, correctness checks, and portability. Lale extends the reach of existing automation tools across data modalities (tables, text, images, time-series) and programming languages (Python, Java, R). Thus, data scientists can leverage automation while remaining in control of their work.
Predicting Global Variations in Outdoor PM2.5 Concentrations using Satellite Images and Deep Convolutional Neural Networks
Authors: Kris Y. Hong, Pedro O. Pinheiro, Scott Weichenthal
Date: 2019-06-01 00:00:00
Tasks: Model Selection

Here we present a new method of estimating global variations in outdoor PM$_{2.5}$ concentrations using satellite images combined with ground-level measurements and deep convolutional neural networks. Specifically, new deep learning models were trained over the global PM$_{2.5}$ concentration range ($<$1-436 $\mu$g/m$^3$) using a large database of satellite images paired with ground level PM$_{2.5}$ measurements available from the World Health Organization. Final model selection was based on a systematic evaluation of well-known architectures for the convolutional base including InceptionV3, Xception, and VGG16. The Xception architecture performed best and the final global model had a root mean square error (RMSE) value of 13.01 $\mu$g/m$^3$ (R$^2$=0.75) in the disjoint test set. The predictive performance of our new global model (called IMAGE-PM$_{2.5}$) is similar to the current state-of-the-art model used in the Global Burden of Disease study but relies only on satellite images as input. As a result, the IMAGE-PM$_{2.5}$ model offers a fast, cost-effective means of estimating global variations in long-term average PM$_{2.5}$ concentrations and may be particularly useful for regions without ground monitoring data or detailed emissions inventories. The IMAGE-PM$_{2.5}$ model can be used as a stand-alone method of global exposure estimation or incorporated into more complex hierarchical model structures.
Collage Inference: Achieving low tail latency during distributed image classification using coded redundancy models
Authors: Krishna Narra, Zhifeng Lin, Ganesh Ananthanarayanan, Salman Avestimehr, Murali Annavaram
Date: 2019-06-05 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, image-classification, Image Classification

Reducing the latency variance in machine learning inference is a key requirement in many applications. Variance is harder to control in a cloud deployment in the presence of stragglers. In spite of this challenge, inference is increasingly being done in the cloud, due to the advent of affordable machine learning as a service (MLaaS) platforms. Existing approaches to reduce variance rely on replication which is expensive and partially negates the affordability of MLaaS. In this work, we argue that MLaaS platforms also provide unique opportunities to cut the cost of redundancy. In MLaaS platforms, multiple inference requests are concurrently received by a load balancer which can then create a more cost-efficient redundancy coding across a larger collection of images. We propose a novel convolutional neural network model, Collage-CNN, to provide a low-cost redundancy framework. A Collage-CNN model takes a collage formed by combining multiple images and performs multi-image classification in one shot, albeit at slightly lower accuracy. We then augment a collection of traditional single image classifiers with a single Collage-CNN classifier which acts as a low-cost redundant backup. Collage-CNN then provides backup classification results if a single image classification straggles. Deploying the Collage-CNN models in the cloud, we demonstrate that the 99th percentile tail latency of inference can be reduced by 1.47X compared to replication based approaches while providing high accuracy. Also, variation in inference latency can be reduced by 9X with a slight increase in average inference latency.
Cormorant: Covariant Molecular Neural Networks
Authors: Brandon Anderson, Truong-Son Hy, Risi Kondor
Date: 2019-06-06 00:00:00
Proceeding: NeurIPS 2019 12

We propose Cormorant, a rotationally covariant neural network architecture for learning the behavior and properties of complex many-body physical systems. We apply these networks to molecular systems with two goals: learning atomic potential energy surfaces for use in Molecular Dynamics simulations, and learning ground state properties of molecules calculated by Density Functional Theory. Some of the key features of our network are that (a) each neuron explicitly corresponds to a subset of atoms; (b) the activation of each neuron is covariant to rotations, ensuring that overall the network is fully rotationally invariant. Furthermore, the non-linearity in our network is based upon tensor products and the Clebsch-Gordan decomposition, allowing the network to operate entirely in Fourier space. Cormorant significantly outperforms competing algorithms in learning molecular Potential Energy Surfaces from conformational geometries in the MD-17 dataset, and is competitive with other methods at learning geometric, energetic, electronic, and thermodynamic properties of molecules on the GDB-9 dataset.
Acceleration of Radiation Transport Solves Using Artificial Neural Networks
Authors: Mauricio Tano, Jean Ragusa
Date: 2019-06-06 00:00:00

Discontinuous Finite Element Methods (DFEM) have been widely used for solving $S_n$ radiation transport problems in participative and non-participative media. In the DFEM $S_n$ methodology, the transport equation is discretized into a set of algebraic equations that have to be solved for each spatial cell and angular direction, strictly preserving the following of radiation in the system. At the core of a DFEM solver a small matrix-vector system (of 8 independent equations for tri-linear DFEM in 3D hexehdral cells) has to be assembled and solved for each cell, angle, energy group, and time step. These systems are generally solved by direct Gaussian Elimination. The computational cost of the Gaussian Elimination, repeated for each phase-space cell, amounts to a large fraction to the total compute time. Here, we have designed a Machine Learning algorithm based in a shallow Artificial Neural Networks (ANNs) to replace that Gaussian Elimination step, enabling a sizeable speed up in the solution process. The key idea is to train an ANN with a large set of solutions of random one-cell transport problems and then to use the trained ANN to replace Gaussian Elimination large scale transport solvers. It has been observed that ANNs decrease the solution times by at least a factor of 4, while introducing mean absolute errors between 1-3 \% in large scale transport solutions.
PHiSeg: Capturing Uncertainty in Medical Image Segmentation
Authors: Christian F. Baumgartner, Kerem C. Tezcan, Krishna Chaitanya, Andreas M. Hötker, Urs J. Muehlematter, Khoschy Schawkat, Anton S. Becker, Olivio Donati, Ender Konukoglu
Date: 2019-06-07 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Segmentation of anatomical structures and pathologies is inherently ambiguous. For instance, structure borders may not be clearly visible or different experts may have different styles of annotating. The majority of current state-of-the-art methods do not account for such ambiguities but rather learn a single mapping from image to segmentation. In this work, we propose a novel method to model the conditional probability distribution of the segmentations given an input image. We derive a hierarchical probabilistic model, in which separate latent variables are responsible for modelling the segmentation at different resolutions. Inference in this model can be efficiently performed using the variational autoencoder framework. We show that our proposed method can be used to generate significantly more realistic and diverse segmentation samples compared to recent related work, both, when trained with annotations from a single or multiple annotators.
When and Why Metaheuristics Researchers Can Ignore "No Free Lunch" Theorems
Authors: James McDermott
Date: 2019-06-07 00:00:00

The No Free Lunch (NFL) theorem for search and optimisation states that averaged across all possible objective functions on a fixed search space, all search algorithms perform equally well. Several refined versions of the theorem find a similar outcome when averaging across smaller sets of functions. This paper argues that NFL results continue to be misunderstood by many researchers, and addresses this issue in several ways. Existing arguments against real-world implications of NFL results are collected and re-stated for accessibility, and new ones are added. Specific misunderstandings extant in the literature are identified, with speculation as to how they may have arisen. This paper presents an argument against a common paraphrase of NFL findings -- that algorithms must be specialised to problem domains in order to do well -- after problematising the usually undefined term "domain". It provides novel concrete counter-examples illustrating cases where NFL theorems do not apply. In conclusion it offers a novel view of the real meaning of NFL, incorporating the anthropic principle and justifying the position that in many common situations researchers can ignore NFL.
Enhanced Optimization with Composite Objectives and Novelty Pulsation
Authors: Hormoz Shahrzad, Babak Hodjat, Camille Dollé, Andrei Denissov, Simon Lau, Donn Goodhew, Justin Dyer, Risto Miikkulainen
Date: 2019-06-07 00:00:00
Tasks: Diversity

An important benefit of multi-objective search is that it maintains a diverse population of candidates, which helps in deceptive problems in particular. Not all diversity is useful, however: candidates that optimize only one objective while ignoring others are rarely helpful. A recent solution is to replace the original objectives by their linear combinations, thus focusing the search on the most useful trade-offs between objectives. To compensate for the loss of diversity, this transformation is accompanied by a selection mechanism that favors novelty. This paper improves this approach further by introducing novelty pulsation, i.e. a systematic method to alternate between novelty selection and local optimization. In the highly deceptive problem of discovering minimal sorting networks, it finds state-of-the-art solutions significantly faster than before. In fact, our method so far has established a new world record for the 20-lines sorting network with 91 comparators. In the real-world problem of stock trading, it discovers solutions that generalize significantly better on unseen data. Composite Novelty Pulsation is therefore a promising approach to solving deceptive real-world problems through multi-objective optimization.
Secrets of the Brain: An Introduction to the Brain Anatomical Structure and Biological Function
Authors: Jiawei Zhang
Date: 2019-05-31 00:00:00
Tasks: Articles

In this paper, we will provide an introduction to the brain structure and function. Brain is an astonishing living organ inside our heads, weighing about 1.5kg, consisting of billions of tiny cells. The brain enables us to sense the world around us (to touch, to smell, to see and to hear, etc.), to think and to respond to the world as well. The main obstacles that prevent us from creating a machine which can behavior like real-world creatures are due to our limited knowledge about the brain in both its structure and its function. In this paper, we will focus introducing the brain anatomical structure and biological function, as well as its surrounding sensory systems. Many of the materials used in this paper are from wikipedia and several other neuroscience introductory articles, which will be properly cited in this article. This is the first of the three tutorial articles about the brain (the other two are [26] and [27]). In the follow-up two articles, we will further introduce the low-level composition basis structures (e.g., neuron, synapse and action potential) and the high-level cognitive functions (e.g., consciousness, attention, learning and memory) of the brain, respectively.
Kernel Mean Embedding Based Hypothesis Tests for Comparing Spatial Point Patterns
Authors: Raif M. Rustamov, James T. Klosowski
Date: 2019-05-31 00:00:00
Tasks: Point Processes, Two-sample testing

This paper introduces an approach for detecting differences in the first-order structures of spatial point patterns. The proposed approach leverages the kernel mean embedding in a novel way by introducing its approximate version tailored to spatial point processes. While the original embedding is infinite-dimensional and implicit, our approximate embedding is finite-dimensional and comes with explicit closed-form formulas. With its help we reduce the pattern comparison problem to the comparison of means in the Euclidean space. Hypothesis testing is based on conducting t-tests on each dimension of the embedding and combining the resulting p-values using one of the recently introduced p-value combination techniques. If desired, corresponding Bayes factors can be computed and averaged over all tests to quantify the evidence against the null. The main advantages of the proposed approach are that it can be applied to both single and replicated pattern comparisons and that neither bootstrap nor permutation procedures are needed to obtain or calibrate the p-values. Our experiments show that the resulting tests are powerful and the p-values are well-calibrated; two applications to real world data are presented.
Optimal Convergence for Stochastic Optimization with Multiple Expectation Constraints
Authors: Kinjal Basu, Preetam Nandy
Date: 2019-06-08 00:00:00
Tasks: Stochastic Optimization

In this paper, we focus on the problem of stochastic optimization where the objective function can be written as an expectation function over a closed convex set. We also consider multiple expectation constraints which restrict the domain of the problem. We extend the cooperative stochastic approximation algorithm from Lan and Zhou [2016] to solve the particular problem. We close the gaps in the previous analysis and provide a novel proof technique to show that our algorithm attains the optimal rate of convergence for both optimality gap and constraint violation when the functions are generally convex. We also compare our algorithm empirically to the state-of-the-art and show improved convergence in many situations.
Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
Authors: Shaowu Pan, Karthik Duraisamy
Date: 2019-06-09 00:00:00
Tasks: Variational Inference

The Koopman operator has emerged as a powerful tool for the analysis of nonlinear dynamical systems as it provides coordinate transformations to globally linearize the dynamics. While recent deep learning approaches have been useful in extracting the Koopman operator from a data-driven perspective, several challenges remain. In this work, we formalize the problem of learning the continuous-time Koopman operator with deep neural networks in a measure-theoretic framework. Our approach induces two types of models: differential and recurrent form, the choice of which depends on the availability of the governing equations and data. We then enforce a structural parameterization that renders the realization of the Koopman operator provably stable. A new autoencoder architecture is constructed, such that only the residual of the dynamic mode decomposition is learned. Finally, we employ mean-field variational inference (MFVI) on the aforementioned framework in a hierarchical Bayesian setting to quantify uncertainties in the characterization and prediction of the dynamics of observables. The framework is evaluated on a simple polynomial system, the Duffing oscillator, and an unstable cylinder wake flow with noisy measurements.
Dissecting Content and Context in Argumentative Relation Analysis
Authors: Juri Opitz, Anette Frank
Date: 2019-06-07 00:00:00
Proceeding: WS 2019 8
Tasks: Relation

When assessing relations between argumentative units (e.g., support or attack), computational systems often exploit disclosing indicators or markers that are not part of elementary argumentative units (EAUs) themselves, but are gained from their context (position in paragraph, preceding tokens, etc.). We show that this dependency is much stronger than previously assumed. In fact, we show that by completely masking the EAU text spans and only feeding information from their context, a competitive system may function even better. We argue that an argument analysis system that relies more on discourse context than the argument's content is unsafe, since it can easily be tricked. To alleviate this issue, we separate argumentative units from their context such that the system is forced to model and rely on an EAU's content. We show that the resulting classification system is more robust, and argue that such models are better suited for predicting argumentative relations across documents.
Classifying the reported ability in clinical mobility descriptions
Authors: Denis Newman-Griffis, Ayah Zirikly, Guy Divita, Bart Desmet
Date: 2019-06-07 00:00:00
Proceeding: WS 2019 8
Tasks: Natural Language Inference, Negation

Assessing how individuals perform different activities is key information for modeling health states of individuals and populations. Descriptions of activity performance in clinical free text are complex, including syntactic negation and similarities to textual entailment tasks. We explore a variety of methods for the novel task of classifying four types of assertions about activity performance: Able, Unable, Unclear, and None (no information). We find that ensembling an SVM trained with lexical features and a CNN achieves 77.9% macro F1 score on our task, and yields nearly 80% recall on the rare Unclear and Unable samples. Finally, we highlight several challenges in classifying performance assertions, including capturing information about sources of assistance, incorporating syntactic structure and negation scope, and handling new modalities at test time. Our findings establish a strong baseline for this novel task, and identify intriguing areas for further research.
Deep Contextualized Biomedical Abbreviation Expansion
Authors: Qiao Jin, Jinling Liu, Xinghua Lu
Date: 2019-06-08 00:00:00
Proceeding: WS 2019 8
Tasks: Information Retrieval, Question Answering, Retrieval

Automatic identification and expansion of ambiguous abbreviations are essential for biomedical natural language processing applications, such as information retrieval and question answering systems. In this paper, we present DEep Contextualized Biomedical. Abbreviation Expansion (DECBAE) model. DECBAE automatically collects substantial and relatively clean annotated contexts for 950 ambiguous abbreviations from PubMed abstracts using a simple heuristic. Then it utilizes BioELMo to extract the contextualized features of words, and feed those features to abbreviation-specific bidirectional LSTMs, where the hidden states of the ambiguous abbreviations are used to assign the exact definitions. Our DECBAE model outperforms other baselines by large margins, achieving average accuracy of 0.961 and macro-F1 of 0.917 on the dataset. It also surpasses human performance for expanding a sample abbreviation, and remains robust in imbalanced, low-resources and clinical settings.
Clinical Concept Extraction for Document-Level Coding
Authors: Sarah Wiegreffe, Edward Choi, Sherry Yan, Jimeng Sun, Jacob Eisenstein
Date: 2019-06-08 00:00:00
Proceeding: WS 2019 8
Tasks: Clinical Concept Extraction

The text of clinical notes can be a valuable source of patient information and clinical assessments. Historically, the primary approach for exploiting clinical notes has been information extraction: linking spans of text to concepts in a detailed domain ontology. However, recent work has demonstrated the potential of supervised machine learning to extract document-level codes directly from the raw text of clinical notes. We propose to bridge the gap between the two approaches with two novel syntheses: (1) treating extracted concepts as features, which are used to supplement or replace the text of the note; (2) treating extracted concepts as labels, which are used to learn a better representation of the text. Unfortunately, the resulting concepts do not yield performance gains on the document-level clinical coding task. We explore possible explanations and future research directions.
Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations
Authors: Rui Zhang, Caitlin Westerfield, Sungrok Shim, Garrett Bingham, Alexander Fabbri, Neha Verma, William Hu, Dragomir Radev
Date: 2019-06-08 00:00:00
Proceeding: ACL 2019 7
Tasks: Cross-Lingual Information Retrieval, Cross-Lingual Word Embeddings, Information Retrieval, Reranking, Retrieval, Translation, Word Embeddings

In this paper, we propose to boost low-resource cross-lingual document retrieval performance with deep bilingual query-document representations. We match queries and documents in both source and target languages with four components, each of which is implemented as a term interaction-based deep neural network with cross-lingual word embeddings as input. By including query likelihood scores as extra features, our model effectively learns to rerank the retrieved documents by using a small number of relevance labels for low-resource language pairs. Due to the shared cross-lingual word embedding space, the model can also be directly applied to another language pair without any training label. Experimental results on the MATERIAL dataset show that our model outperforms the competitive translation-based baselines on English-Swahili, English-Tagalog, and English-Somali cross-lingual information retrieval tasks.
Making Asynchronous Stochastic Gradient Descent Work for Transformers
Authors: Alham Fikri Aji, Kenneth Heafield
Date: 2019-06-08 00:00:00
Proceeding: WS 2019 11
Tasks: GPU, Machine Translation, Translation

Asynchronous stochastic gradient descent (SGD) is attractive from a speed perspective because workers do not wait for synchronization. However, the Transformer model converges poorly with asynchronous SGD, resulting in substantially lower quality compared to synchronous SGD. To investigate why this is the case, we isolate differences between asynchronous and synchronous methods to investigate batch size and staleness effects. We find that summing several asynchronous updates, rather than applying them immediately, restores convergence behavior. With this hybrid method, Transformer training for neural machine translation task reaches a near-convergence level 1.36x faster in single-node multi-GPU training with no impact on model quality.