pwc-archive/papers-with-abstracts huggingface dataset:
EPIC30M: An Epidemics Corpus Of Over 30 Million Relevant Tweets
Authors: Junhua Liu, Trisha Singhal, Lucienne T. M. Blessing, Kristin L. Wood, Kwan Hui Lim
Date: 2020-06-09 00:00:00
Tasks: Natural Language Understanding

Since the start of COVID-19, several relevant corpora from various sources are presented in the literature that contain millions of data points. While these corpora are valuable in supporting many analyses on this specific pandemic, researchers require additional benchmark corpora that contain other epidemics to facilitate cross-epidemic pattern recognition and trend analysis tasks. During our other efforts on COVID-19 related work, we discover very little disease related corpora in the literature that are sizable and rich enough to support such cross-epidemic analysis tasks. In this paper, we present EPIC30M, a large-scale epidemic corpus that contains 30 millions micro-blog posts, i.e., tweets crawled from Twitter, from year 2006 to 2020. EPIC30M contains a subset of 26.2 millions tweets related to three general diseases, namely Ebola, Cholera and Swine Flu, and another subset of 4.7 millions tweets of six global epidemic outbreaks, including 2009 H1N1 Swine Flu, 2010 Haiti Cholera, 2012 Middle-East Respiratory Syndrome (MERS), 2013 West African Ebola, 2016 Yemen Cholera and 2018 Kivu Ebola. Furthermore, we explore and discuss the properties of the corpus with statistics of key terms and hashtags and trends analysis for each subset. Finally, we demonstrate the value and impact that EPIC30M could create through a discussion of multiple use cases of cross-epidemic research topics that attract growing interest in recent years. These use cases span multiple research areas, such as epidemiological modeling, pattern recognition, natural language understanding and economical modeling.
Exact and heuristic methods for the discrete parallel machine scheduling location problem
Authors: Raphael Kramer, Arthur Kramer
Date: 2020-06-09 00:00:00
Tasks: ARC, Combinatorial Optimization, Scheduling

The discrete parallel machine makespan scheduling location (ScheLoc) problem is an integrated combinatorial optimization problem that combines facility location and job scheduling. The problem consists in choosing the locations of $p$ machines among a finite set of candidates and scheduling a set of jobs on these machines, aiming to minimize the makespan. Depending on the machine location, the jobs may have different release dates, and thus the location decisions have a direct impact on the scheduling decisions. To solve the problem, it is proposed a new arc-flow formulation, a column generation and three heuristic procedures that are evaluated through extensive computational experiments. By embedding the proposed procedures into a framework algorithm, we are able to find proven optimal solutions for all benchmark instances from the related literature and to obtain small percentage gaps for a new set of challenging instances.
An Algorithm to find Superior Fitness on NK Landscapes under High Complexity: Muddling Through
Authors: Sasanka Sekhar Chanda, Sai Yayavaram
Date: 2020-06-06 00:00:00

Under high complexity - given by pervasive interdependence between constituent elements of a decision in an NK landscape - our algorithm obtains fitness superior to that reported in extant research. We distribute the decision elements comprising a decision into clusters. When a change in value of a decision element is considered, a forward move is made if the aggregate fitness of the cluster members residing alongside the decision element is higher. The decision configuration with the highest fitness in the path is selected. Increasing the number of clusters obtains even higher fitness. Further, implementing moves comprising of up to two changes in a cluster also obtains higher fitness. Our algorithm obtains superior outcomes by enabling more extensive search, allowing inspection of more distant configurations. We name this algorithm the muddling through algorithm, in memory of Charles Lindblom who spotted the efficacy of the process long before sophisticated computer simulations came into being.
Automated Diagram Generation to Build Understanding and Usability
Authors: William Schoenberg
Date: 2020-05-27 00:00:00

Causal loop and stock and flow diagrams are broadly used in System Dynamics because they help organize relationships and convey meaning. Using the analytical work of Schoenberg (2019) to select what to include in a compressed model, this paper demonstrates how that information can be clearly presented in an automatically generated causal loop diagram. The diagrams are generated using tools developed by people working in graph theory and the generated diagrams are clear and aesthetically pleasing. This approach can also be built upon to generate stock and flow diagrams. Automated stock and flow diagram generation opens the door to representing models developed using only equations, regardless or origin, in a clear and easy to understand way. Because models can be large, the application of grouping techniques, again developed for graph theory, can help structure the resulting diagrams in the most usable form. This paper describes the algorithms developed for automated diagram generation and shows a number of examples of their uses in large models. The application of these techniques to existing, but inaccessible, equation-based models can help broaden the knowledge base for System Dynamics modeling. The techniques can also be used to improve layout in all, or part, of existing models with diagrammatic informtion.
Application of Fuzzy Rule based System for Highway Research Board Classification of Soils
Authors: Sujatha A, L Govindaraju, N Shivakumar
Date: 2020-05-10 00:00:00
Tasks: Classification, General Classification

Fuzzy rule-based model is a powerful tool for imitating the human way of thinking and solving uncertainty-related problems as it allows for understandable and interpretable rule bases. The objective of this paper is to study the applicability of fuzzy rule-based modelling to quantify soil classification for engineering purposes by qualitatively considering soil index properties. The classification system of the Highway Research Board is considered to illustrate a fuzzy rule-based model. The soil's index properties are fuzzified using triangular functions, and the fuzzy membership values are calculated. Fuzzy arithmetical operators are then applied to the membership values obtained for classification. Fuzzy decision tree classification algorithm is used to derive fuzzy if-then rules to quantify qualitative soil classification. The proposed system is implemented in MATLAB. The results obtained are checked and the implementation of the proposed model is measured against the outcomes of the laboratory tests.
Finding the Loops that Matter
Authors: Robert Eberlein, William Schoenberg
Date: 2020-05-27 00:00:00

The Loops that Matter method (Schoenberg et. al, 2019) for understanding model behavior provides metrics showing the contribution of the feedback loops in a model to behavior at each point in time. To provide these metrics, it is necessary find the set of loops on which to compute them. We show in this paper the necessity of including loops that are important at different points in the simulation. These important loops may not be independent of one another and cannot be determined from static analysis of the model structure. We then describe an algorithm that can be used to discover the most important loops in models that are too feedback rich for exhaustive loop discovery. We demonstrate the use of this algorithm in terms of its ability to find the most explanatory loops, and its computational performance for large models. By using this approach, the Loops that Matter method can be applied to models of any size or complexity.
Probing Neural Dialog Models for Conversational Understanding
Authors: Abdelrhman Saleh, Tovly Deutsch, Stephen Casper, Yonatan Belinkov, Stuart Shieber
Date: 2020-06-07 00:00:00
Proceeding: WS 2020 7
Tasks: Open-Domain Dialog

The predominant approach to open-domain dialog generation relies on end-to-end training of neural models on chat datasets. However, this approach provides little insight as to what these models learn (or do not learn) about engaging in dialog. In this study, we analyze the internal representations learned by neural open-domain dialog systems and evaluate the quality of these representations for learning basic conversational skills. Our results suggest that standard open-domain dialog systems struggle with answering questions, inferring contradiction, and determining the topic of conversation, among other tasks. We also find that the dyadic, turn-taking nature of dialog is not fully leveraged by these models. By exploring these limitations, we highlight the need for additional research into architectures and training methods that can better capture high-level information about dialog.
A Dataset and Benchmarks for Multimedia Social Analysis
Authors: Bofan Xue, David Chan, John Canny
Date: 2020-06-05 00:00:00
Tasks: Image Captioning, image-classification, Image Classification, Language Modeling, Language Modelling, Sentiment Analysis

We present a new publicly available dataset with the goal of advancing multi-modality learning by offering vision and language data within the same context. This is achieved by obtaining data from a social media website with posts containing multiple paired images/videos and text, along with comment trees containing images/videos and/or text. With a total of 677k posts, 2.9 million post images, 488k post videos, 1.4 million comment images, 4.6 million comment videos, and 96.9 million comments, data from different modalities can be jointly used to improve performances for a variety of tasks such as image captioning, image classification, next frame prediction, sentiment analysis, and language modeling. We present a wide range of statistics for our dataset. Finally, we provide baseline performance analysis for one of the regression tasks using pre-trained models and several fully connected networks.
Deep learning mediated single time-point image-based prediction of embryo developmental outcome at the cleavage stage
Authors: Manoj Kumar Kanakasabapathy, Prudhvi Thirumalaraju, Charles L Bormann, Raghav Gupta, Rohan Pooniwala, Hemanth Kandula, Irene Souter, Irene Dimitriadis, Hadi Shafiee
Date: 2020-05-21 00:00:00
Tasks: Prognosis

In conventional clinical in-vitro fertilization practices embryos are transferred either at the cleavage or blastocyst stages of development. Cleavage stage transfers, particularly, are beneficial for patients with relatively poor prognosis and at fertility centers in resource-limited settings where there is a higher chance of developmental failure in embryos in-vitro. However, one of the major limitations of embryo selections at the cleavage stage is the availability of very low number of manually discernable features to predict developmental outcomes. Although, time-lapse imaging systems have been proposed as possible solutions, they are cost-prohibitive and require bulky and expensive hardware, and labor-intensive. Advances in convolutional neural networks (CNNs) have been utilized to provide accurate classifications across many medical and non-medical object categories. Here, we report an automated system for classification and selection of human embryos at the cleavage stage using a trained CNN combined with a genetic algorithm. The system selected the cleavage stage embryo at 70 hours post insemination (hpi) that ultimately developed into top-quality blastocyst at 70 hpi with 64% accuracy, outperforming the abilities of embryologists in identifying embryos with the highest developmental potential. Such systems can have a significant impact on IVF procedures by empowering embryologists for accurate and consistent embryo assessment in both resource-poor and resource-rich settings.
Bayesian Neural Network via Stochastic Gradient Descent
Authors: Abhinav Sagar
Date: 2020-06-04 00:00:00
Tasks: regression, Variational Inference

The goal of bayesian approach used in variational inference is to minimize the KL divergence between variational distribution and unknown posterior distribution. This is done by maximizing the Evidence Lower Bound (ELBO). A neural network is used to parametrize these distributions using Stochastic Gradient Descent. This work extends the work done by others by deriving the variational inference models. We show how SGD can be applied on bayesian neural networks by gradient estimation techniques. For validation, we have tested our model on 5 UCI datasets and the metrics chosen for evaluation are Root Mean Square Error (RMSE) error and negative log likelihood. Our work considerably beats the previous state of the art approaches for regression using bayesian neural networks.
Affective Conditioning on Hierarchical Networks applied to Depression Detection from Transcribed Clinical Interviews
Authors: D. Xezonaki, G. Paraskevopoulos, A. Potamianos, S. Narayanan
Date: 2020-06-04 00:00:00
Tasks: Depression Detection

In this work we propose a machine learning model for depression detection from transcribed clinical interviews. Depression is a mental disorder that impacts not only the subject's mood but also the use of language. To this end we use a Hierarchical Attention Network to classify interviews of depressed subjects. We augment the attention layer of our model with a conditioning mechanism on linguistic features, extracted from affective lexica. Our analysis shows that individuals diagnosed with depression use affective language to a greater extent than not-depressed. Our experiments show that external affective information improves the performance of the proposed architecture in the General Psychotherapy Corpus and the DAIC-WoZ 2017 depression datasets, achieving state-of-the-art 71.6 and 68.6 F1 scores respectively.
DeepVar: An End-to-End Deep Learning Approach for Genomic Variant Recognition in Biomedical Literature
Authors: Chaoran Cheng, Fei Tan, Zhi Wei
Date: 2020-06-05 00:00:00
Tasks: Feature Engineering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

We consider the problem of Named Entity Recognition (NER) on biomedical scientific literature, and more specifically the genomic variants recognition in this work. Significant success has been achieved for NER on canonical tasks in recent years where large data sets are generally available. However, it remains a challenging problem on many domain-specific areas, especially the domains where only small gold annotations can be obtained. In addition, genomic variant entities exhibit diverse linguistic heterogeneity, differing much from those that have been characterized in existing canonical NER tasks. The state-of-the-art machine learning approaches in such tasks heavily rely on arduous feature engineering to characterize those unique patterns. In this work, we present the first successful end-to-end deep learning approach to bridge the gap between generic NER algorithms and low-resource applications through genomic variants recognition. Our proposed model can result in promising performance without any hand-crafted features or post-processing rules. Our extensive experiments and results may shed light on other similar low-resource NER applications.
Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
Authors: Tim Z. Xiao, Aidan N. Gomez, Yarin Gal
Date: 2020-06-08 00:00:00
Tasks: Machine Translation, Sentence, Translation

We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure of uncertainty designed specifically for long sequences of discrete random variables -- i.e. words in the output sentence. Our new measure of uncertainty solves a major intractability in the naive application of existing approaches on long sentences. We use our new measure on a Transformer model trained with dropout approximate inference. On the task of German-English translation using WMT13 and Europarl, we show that with dropout uncertainty our measure is able to identify when Dutch source sentences, sentences which use the same word types as German, are given to the model instead of German.
StackOverflow vs Kaggle: A Study of Developer Discussions About Data Science
Authors: David Hin
Date: 2020-06-06 00:00:00

Software developers are increasingly required to understand fundamental Data science (DS) concepts. Recently, the presence of machine learning (ML) and deep learning (DL) has dramatically increased in the development of user applications, whether they are leveraged through frameworks or implemented from scratch. These topics attract much discussion on online platforms. This paper conducts large-scale qualitative and quantitative experiments to study the characteristics of 197836 posts from StackOverflow and Kaggle. Latent Dirichlet Allocation topic modelling is used to extract twenty-four DS discussion topics. The main findings include that TensorFlow-related topics were most prevalent in StackOverflow, while meta discussion topics were the prevalent ones on Kaggle. StackOverflow tends to include lower-level troubleshooting, while Kaggle focuses on practicality and optimising leaderboard performance. In addition, across both communities, DS discussion is increasing at a dramatic rate. While TensorFlow discussion on StackOverflow is slowing, interest in Keras is rising. Finally, ensemble algorithms are the most mentioned ML/DL algorithms in Kaggle but are rarely discussed on StackOverflow. These findings can help educators and researchers to more effectively tailor and prioritise efforts in researching and communicating DS concepts towards different developer communities.
Open-Domain Question Answering with Pre-Constructed Question Spaces
Authors: Jinfeng Xiao, Lidan Wang, Franck Dernoncourt, Trung Bui, Tong Sun, Jiawei Han
Date: 2020-06-02 00:00:00
Proceeding: NAACL 2021 4
Tasks: Information Retrieval, Knowledge Graphs, Open-Domain Question Answering, Question Answering, Retrieval

Open-domain question answering aims at solving the task of locating the answers to user-generated questions in massive collections of documents. There are two families of solutions available: retriever-readers, and knowledge-graph-based approaches. A retriever-reader usually first uses information retrieval methods like TF-IDF to locate some documents or paragraphs that are likely to be relevant to the question, and then feeds the retrieved text to a neural network reader to extract the answer. Alternatively, knowledge graphs can be constructed from the corpus and be queried against to answer user questions. We propose a novel algorithm with a reader-retriever structure that differs from both families. Our reader-retriever first uses an offline reader to read the corpus and generate collections of all answerable questions associated with their answers, and then uses an online retriever to respond to user queries by searching the pre-constructed question spaces for answers that are most likely to be asked in the given way. We further combine retriever-reader and reader-retriever results into one single answer by examining the consistency between the two components. We claim that our algorithm solves some bottlenecks in existing work, and demonstrate that it achieves superior accuracy on real-world datasets.
Graph-Stega: Semantic Controllable Steganographic Text Generation Guided by Knowledge Graph
Authors: Zhongliang Yang, Baitao Gong, Yamin Li, Jinshuai Yang, Zhiwen Hu, Yongfeng Huang
Date: 2020-06-02 00:00:00
Tasks: Text Generation

Most of the existing text generative steganographic methods are based on coding the conditional probability distribution of each word during the generation process, and then selecting specific words according to the secret information, so as to achieve information hiding. Such methods have their limitations which may bring potential security risks. Firstly, with the increase of embedding rate, these models will choose words with lower conditional probability, which will reduce the quality of the generated steganographic texts; secondly, they can not control the semantic expression of the final generated steganographic text. This paper proposes a new text generative steganography method which is quietly different from the existing models. We use a Knowledge Graph (KG) to guide the generation of steganographic sentences. On the one hand, we hide the secret information by coding the path in the knowledge graph, but not the conditional probability of each generated word; on the other hand, we can control the semantic expression of the generated steganographic text to a certain extent. The experimental results show that the proposed model can guarantee both the quality of the generated text and its semantic expression, which is a supplement and improvement to the current text generation steganography.
Subjective Question Answering: Deciphering the inner workings of Transformers in the realm of subjectivity
Authors: Lukas Muttenthaler
Date: 2020-06-02 00:00:00
Tasks: Opinion Mining, Question Answering

Understanding subjectivity demands reasoning skills beyond the realm of common knowledge. It requires a machine learning model to process sentiment and to perform opinion mining. In this work, I've exploited a recently released dataset for span-selection Question Answering, namely SubjQA. SubjQA is the first QA dataset that contains questions that ask for subjective opinions corresponding to review paragraphs from six different domains. Hence, to answer these subjective questions, a learner must extract opinions and process sentiment for various domains, and additionally, align the knowledge extracted from a paragraph with the natural language utterances in the corresponding question, which together enhance the difficulty of a QA task. The primary goal of this thesis was to investigate the inner workings (i.e., latent representations) of a Transformer-based architecture to contribute to a better understanding of these not yet well understood "black-box" models. Transformer's hidden representations, concerning the true answer span, are clustered more closely in vector space than those representations corresponding to erroneous predictions. This observation holds across the top three Transformer layers for both objective and subjective questions and generally increases as a function of layer dimensions. Moreover, the probability to achieve a high cosine similarity among hidden representations in latent space concerning the true answer span tokens is significantly higher for correct compared to incorrect answer span predictions. These results have decisive implications for down-stream applications, where it is crucial to know about why a neural network made mistakes, and in which point, in space and time the mistake has happened (e.g., to automatically predict correctness of an answer span prediction without the necessity of labeled data).
Representation Learning for Information Extraction from Form-like Documents
Authors: Bodhisattwa Majumder, Navneet Potti, Sandeep Tata, James B. Wendt, Qi Zhao, Marc Najork
Date: 2020-06-15 00:00:00
Proceeding: ACL 2020 6
Tasks: Form, Representation Learning

We propose a novel approach using representation learning for tackling the problem of extracting structured information from form-like
document images. We propose an extraction
system that uses knowledge of the types of the
target fields to generate extraction candidates,
and a neural network architecture that learns a
dense representation of each candidate based
on neighboring words in the document. These
learned representations are not only useful in
solving the extraction task for unseen document templates from two different domains,
but are also interpretable, as we show using
loss cases.