pwc-archive/papers-with-abstracts huggingface dataset:
Neural representation and generation for RNA secondary structures
Authors: Zichao Yan, William L. Hamilton, Mathieu Blanchette
Date: 2021-02-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Diversity, Drug Discovery, Inductive Bias, Navigate

Our work is concerned with the generation and targeted design of RNA, a type of genetic macromolecule that can adopt complex structures which influence their cellular activities and functions. The design of large scale and complex biological structures spurs dedicated graph-based deep generative modeling techniques, which represents a key but underappreciated aspect of computational drug discovery. In this work, we investigate the principles behind representing and generating different RNA structural modalities, and propose a flexible framework to jointly embed and generate these molecular structures along with their sequence in a meaningful latent space. Equipped with a deep understanding of RNA molecular structures, our most sophisticated encoding and decoding methods operate on the molecular graph as well as the junction tree hierarchy, integrating strong inductive bias about RNA structural regularity and folding mechanism such that high structural validity, stability and diversity of generated RNAs are achieved. Also, we seek to adequately organize the latent space of RNA molecular embeddings with regard to the interaction with proteins, and targeted optimization is used to navigate in this latent space to search for desired novel RNA molecules.
Revisiting the Prepositional-Phrase Attachment Problem Using Explicit Commonsense Knowledge
Authors: Yida Xin, Henry Lieberman, Peter Chin
Date: 2021-02-01 00:00:00
Tasks: Prepositional Phrase Attachment

We revisit the challenging problem of resolving prepositional-phrase (PP) attachment ambiguity. To date, proposed solutions are either rule-based, where explicit grammar rules direct how to resolve ambiguities; or statistical, where the decision is learned from a corpus of labeled examples. We argue that explicit commonsense knowledge bases can provide an essential ingredient for making good attachment decisions. We implemented a module, named Patch-Comm, that can be used by a variety of conventional parsers, to make attachment decisions. Where the commonsense KB does not provide direct answers, we fall back on a more general system that infers "out-of-knowledge-base" assertions in a manner similar to the way some NLP systems handle out-of-vocabulary words. Our results suggest that the commonsense knowledge-based approach can provide the best of both worlds, integrating rule-based and statistical techniques. As the field is increasingly coming to recognize the importance of explainability in AI, a commonsense approach can enable NLP developers to better understand the behavior of systems, and facilitate natural dialogues with end users.
Forensicability of Deep Neural Network Inference Pipelines
Authors: Alexander Schlögl, Tobias Kupek, Rainer Böhme
Date: 2021-02-01 00:00:00
Tasks: BIG-bench Machine Learning

We propose methods to infer properties of the execution environment of machine learning pipelines by tracing characteristic numerical deviations in observable outputs. Results from a series of proof-of-concept experiments obtained on local and cloud-hosted machines give rise to possible forensic applications, such as the identification of the hardware platform used to produce deep neural network predictions. Finally, we introduce boundary samples that amplify the numerical deviations in order to distinguish machines by their predicted label only.
Counting Protests in News Articles: A Dataset and Semi-Automated Data Collection Pipeline
Authors: Tommy Leung, L. Nathan Perkins
Date: 2021-02-01 00:00:00
Tasks: Articles, coreference-resolution, Coreference Resolution, Decision Making, Descriptive, slot-filling, Slot Filling

Between January 2017 and January 2021, thousands of local news sources in the United States reported on over 42,000 protests about topics such as civil rights, immigration, guns, and the environment. Given the vast number of local journalists that report on protests daily, extracting these events as structured data to understand temporal and geographic trends can empower civic decision-making. However, the task of extracting events from news articles presents well known challenges to the NLP community in the fields of domain detection, slot filling, and coreference resolution. To help improve the resources available for extracting structured data from news stories, our contribution is three-fold. We 1) release a manually labeled dataset of news article URLs, dates, locations, crowd size estimates, and 494 discrete descriptive tags corresponding to 42,347 reported protest events in the United States between January 2017 and January 2021; 2) describe the semi-automated data collection pipeline used to discover, sort, and review the 144,568 English articles that comprise the dataset; and 3) benchmark a long-short term memory (LSTM) low dimensional classifier that demonstrates the utility of processing news articles based on syntactic structures, such as paragraphs and sentences, to count the number of reported protest events.
Text-to-hashtag Generation using Seq2seq Learning
Authors: Augusto Camargo, Wesley Carvalho, Felipe Peressim, Alan Barzilay, Marcelo Finger
Date: 2021-02-01 00:00:00

In this paper, we studied whether models based on BiLSTM and BERT can predict hashtags in Brazilian Portuguese for Ecommerce websites. Hashtags have a sizable financial impact on Ecommerce. We processed a corpus of Ecommerce reviews as inputs, and predicted hashtags as outputs. We evaluated the results using four quantitative metrics: NIST, BLEU, METEOR and a crowdsourced score. A word cloud was used as a qualitative metric. While all computer-generated metrics (NIST, BLEU and METEOR) indicated bad results, the crowdsourced results produced amazing scores. We concluded that the texts predicted by the neural networks are very promising for use as hashtags for products on Ecommerce websites. The code for this work is available at https://github.com/augustocamargo/text-to-hashtag.
Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring
Authors: Michael Weiss, Paolo Tonella
Date: 2021-02-01 00:00:00

Modern software systems rely on Deep Neural Networks (DNN) when processing complex, unstructured inputs, such as images, videos, natural language texts or audio signals. Provided the intractably large size of such input spaces, the intrinsic limitations of learning algorithms, and the ambiguity about the expected predictions for some of the inputs, not only there is no guarantee that DNN's predictions are always correct, but rather developers must safely assume a low, though not negligible, error probability. A fail-safe Deep Learning based System (DLS) is one equipped to handle DNN faults by means of a supervisor, capable of recognizing predictions that should not be trusted and that should activate a healing procedure bringing the DLS to a safe state. In this paper, we propose an approach to use DNN uncertainty estimators to implement such a supervisor. We first discuss the advantages and disadvantages of existing approaches to measure uncertainty for DNNs and propose novel metrics for the empirical assessment of the supervisor that rely on such approaches. We then describe our publicly available tool UNCERTAINTY-WIZARD, which allows transparent estimation of uncertainty for regular tf.keras DNNs. Lastly, we discuss a large-scale study conducted on four different subjects to empirically validate the approach, reporting the lessons-learned as guidance for software engineers who intend to monitor uncertainty for fail-safe execution of DLS.
DRLDO: A novel DRL based De-ObfuscationSystem for Defense against Metamorphic Malware
Authors: Mohit Sewak, Sanjay K. Sahay, Hemant Rathore
Date: 2021-02-01 00:00:00
Tasks: Deep Reinforcement Learning, Intrusion Detection

In this paper, we propose a novel mechanism to normalize metamorphic and obfuscated malware down at the opcode level and hence create an advanced metamorphic malware de-obfuscation and defense system. We name this system DRLDO, for Deep Reinforcement Learning based De-Obfuscator. With the inclusion of the DRLDO as a sub-component, an existing Intrusion Detection System could be augmented with defensive capabilities against 'zero-day' attacks from obfuscated and metamorphic variants of existing malware. This gains importance, not only because there exists no system to date that uses advanced DRL to intelligently and automatically normalize obfuscation down even to the opcode level, but also because the DRLDO system does not mandate any changes to the existing IDS. The DRLDO system does not even mandate the IDS' classifier to be retrained with any new dataset containing obfuscated samples. Hence DRLDO could be easily retrofitted into any existing IDS deployment. We designed, developed, and conducted experiments on the system to evaluate the same against multiple-simultaneous attacks from obfuscations generated from malware samples from a standardized dataset that contains multiple generations of malware. Experimental results prove that DRLDO was able to successfully make the otherwise un-detectable obfuscated variants of the malware detectable by an existing pre-trained malware classifier. The detection probability was raised well above the cut-off mark to 0.6 for the classifier to detect the obfuscated malware unambiguously. Further, the de-obfuscated variants generated by DRLDO achieved a very high correlation (of 0.99) with the base malware. This observation validates that the DRLDO system is actually learning to de-obfuscate and not exploiting a trivial trick.
Metric-Type Identification for Multi-Level Header Numerical Tables in Scientific Papers
Authors: Lya Hulliyyatus Suadaa, Hidetaka Kamigaito, Manabu Okumura, Hiroya Takamura
Date: 2021-02-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Metric-Type Identification, Vocal Bursts Type Prediction

Numerical tables are widely used to present experimental results in scientific papers. For table understanding, a metric-type is essential to discriminate numbers in the tables. We introduce a new information extraction task, metric-type identification from multi-level header numerical tables, and provide a dataset extracted from scientific papers consisting of header tables, captions, and metric-types. We then propose two joint-learning neural classification and generation schemes featuring pointer-generator-based and BERT-based models. Our results show that the joint models can handle both in-header and out-of-header metric-type identification problems.
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
Authors: Nora Kassner, Philipp Dufter, Hinrich Schütze
Date: 2021-02-01 00:00:00
Proceeding: EACL 2021 2
Tasks: Diversity

Recently, it has been found that monolingual English language models can be used as knowledge bases. Instead of structural knowledge base queries, masked sentences such as "Paris is the capital of [MASK]" are used as probes. We translate the established benchmarks TREx and GoogleRE into 53 languages. Working with mBERT, we investigate three questions. (i) Can mBERT be used as a multilingual knowledge base? Most prior work only considers English. Extending research to multiple languages is important for diversity and accessibility. (ii) Is mBERT's performance as knowledge base language-independent or does it vary from language to language? (iii) A multilingual model is trained on more text, e.g., mBERT is trained on 104 Wikipedias. Can mBERT leverage this for better performance? We find that using mBERT as a knowledge base yields varying performance across languages and pooling predictions across languages improves performance. Conversely, mBERT exhibits a language bias; e.g., when queried in Italian, it tends to predict Italy as the country of origin.
Semi-Supervised Disentanglement of Class-Related and Class-Independent Factors in VAE
Authors: Sina Hajimiri, Aryo Lotfi, Mahdieh Soleymani Baghshah
Date: 2021-02-01 00:00:00
Tasks: Disentanglement

In recent years, extending variational autoencoder's framework to learn disentangled representations has received much attention. We address this problem by proposing a framework capable of disentangling class-related and class-independent factors of variation in data. Our framework employs an attention mechanism in its latent space in order to improve the process of extracting class-related factors from data. We also deal with the multimodality of data distribution by utilizing mixture models as learnable prior distributions, as well as incorporating the Bhattacharyya coefficient in the objective function to prevent highly overlapping mixtures. Our model's encoder is further trained in a semi-supervised manner, with a small fraction of labeled data, to improve representations' interpretability. Experiments show that our framework disentangles class-related and class-independent factors of variation and learns interpretable features. Moreover, we demonstrate our model's performance with quantitative and qualitative results on various datasets.
Fixation probabilities in evolutionary dynamics under weak selection
Authors: Alex McAvoy, Benjamin Allen
Date: 2019-08-10 00:00:00

In evolutionary dynamics, a key measure of a mutant trait's success is the probability that it takes over the population given some initial mutant-appearance distribution. This "fixation probability" is difficult to compute in general, as it depends on the mutation's effect on the organism as well as the population's spatial structure, mating patterns, and other factors. In this study, we consider weak selection, which means that the mutation's effect on the organism is small. We obtain a weak-selection perturbation expansion of a mutant's fixation probability, from an arbitrary initial configuration of mutant and resident types. Our results apply to a broad class of stochastic evolutionary models, in which the size and spatial structure are arbitrary (but fixed). The problem of whether selection favors a given trait is thereby reduced from exponential to polynomial complexity in the population size, when selection is weak. We conclude by applying these methods to obtain new results for evolutionary dynamics on graphs.
Gamified Crowdsourcing for Idiom Corpora Construction
Authors: Gülşen Eryiğit, Ali Şentaş, Johanna Monti
Date: 2021-02-01 00:00:00
Tasks: Machine Translation

Learning idiomatic expressions is seen as one of the most challenging stages in second language learning because of their unpredictable meaning. A similar situation holds for their identification within natural language processing applications such as machine translation and parsing. The lack of high-quality usage samples exacerbates this challenge not only for humans but also for artificial intelligence systems. This article introduces a gamified crowdsourcing approach for collecting language learning materials for idiomatic expressions; a messaging bot is designed as an asynchronous multiplayer game for native speakers who compete with each other while providing idiomatic and nonidiomatic usage examples and rating other players' entries. As opposed to classical crowdprocessing annotation efforts in the field, for the first time in the literature, a crowdcreating & crowdrating approach is implemented and tested for idiom corpora construction. The approach is language independent and evaluated on two languages in comparison to traditional data preparation techniques in the field. The reaction of the crowd is monitored under different motivational means (namely, gamification affordances and monetary rewards). The results reveal that the proposed approach is powerful in collecting the targeted materials, and although being an explicit crowdsourcing approach, it is found entertaining and useful by the crowd. The approach has been shown to have the potential to speed up the construction of idiom corpora for different natural languages to be used as second language learning material, training data for supervised idiom identification systems, or samples for lexicographic studies.
Decentralized Federated Learning Preserves Model and Data Privacy
Authors: Thorsten Wittkopp, Alexander Acker
Date: 2021-02-01 00:00:00
Tasks: Anomaly Detection, Federated Learning

The increasing complexity of IT systems requires solutions, that support operations in case of failure. Therefore, Artificial Intelligence for System Operations (AIOps) is a field of research that is becoming increasingly focused, both in academia and industry. One of the major issues of this area is the lack of access to adequately labeled data, which is majorly due to legal protection regulations or industrial confidentiality. Methods to mitigate this stir from the area of federated learning, whereby no direct access to training data is required. Original approaches utilize a central instance to perform the model synchronization by periodical aggregation of all model parameters. However, there are many scenarios where trained models cannot be published since its either confidential knowledge or training data could be reconstructed from them. Furthermore the central instance needs to be trusted and is a single point of failure. As a solution, we propose a fully decentralized approach, which allows to share knowledge between trained models. Neither original training data nor model parameters need to be transmitted. The concept relies on teacher and student roles that are assigned to the models, whereby students are trained on the output of their teachers via synthetically generated input data. We conduct a case study on log anomaly detection. The results show that an untrained student model, trained on the teachers output reaches comparable F1-scores as the teacher. In addition, we demonstrate that our method allows the synchronization of several models trained on different distinct training data subsets.
Evolutionary computational platform for the automatic discovery of nanocarriers for cancer treatment
Authors: Namid Stillman, Igor Balaz, Antisthenis Tsompanas, Marina Kovacevic, Sepinoud Azimi, Sebastien Lafond, Andrew Adamatzky, Sabine Hauert
Date: 2021-02-01 00:00:00
Tasks: BIG-bench Machine Learning

We present the EVONANO platform for the evolution of nanomedicines with application to anti-cancer treatments. EVONANO includes a simulator to grow tumours, extract representative scenarios, and then simulate nanoparticle transport through these scenarios to predict nanoparticle distribution. The nanoparticle designs are optimised using machine learning to efficiently find the most effective anti-cancer treatments. We demonstrate our platform with two examples optimising the properties of nanoparticles and treatment to selectively kill cancer cells over a range of tumour environments.
A probabilistic Taylor expansion with Gaussian processes
Authors: Toni Karvonen, Jon Cockayne, Filip Tronarp, Simo Särkkä
Date: 2021-02-01 00:00:00
Tasks: Gaussian Processes, regression

We study a class of Gaussian processes for which the posterior mean, for a particular choice of data, replicates a truncated Taylor expansion of any order. The data consist of derivative evaluations at the expansion point and the prior covariance kernel belongs to the class of Taylor kernels, which can be written in a certain power series form. We discuss and prove some results on maximum likelihood estimation of parameters of Taylor kernels. The proposed framework is a special case of Gaussian process regression based on data that is orthogonal in the reproducing kernel Hilbert space of the covariance kernel.
The Complexity of Learning Linear Temporal Formulas from Examples
Authors: Nathanaël Fijalkow, Guillaume Lagarde
Date: 2021-02-01 00:00:00

In this paper we initiate the study of the computational complexity of learning linear temporal logic (LTL) formulas from examples. We construct approximation algorithms for fragments of LTL and prove hardness results; in particular we obtain tight bounds for approximation of the fragment containing only the next operator and conjunctions, and prove NP-completeness results for many fragments.
Scaling Federated Learning for Fine-tuning of Large Language Models
Authors: Agrin Hilmkil, Sebastian Callh, Matteo Barbieri, Leon René Sütfeld, Edvin Listo Zec, Olof Mogren
Date: 2021-02-01 00:00:00
Tasks: Federated Learning, Sentiment Analysis, text-classification, Text Classification

Federated learning (FL) is a promising approach to distributed compute, as well as distributed data, and provides a level of privacy and compliance to legal frameworks. This makes FL attractive for both consumer and healthcare applications. While the area is actively being explored, few studies have examined FL in the context of larger language models and there is a lack of comprehensive reviews of robustness across tasks, architectures, numbers of clients, and other relevant factors. In this paper, we explore the fine-tuning of Transformer-based language models in a federated learning setting. We evaluate three popular BERT-variants of different sizes (BERT, ALBERT, and DistilBERT) on a number of text classification tasks such as sentiment analysis and author identification. We perform an extensive sweep over the number of clients, ranging up to 32, to evaluate the impact of distributed compute on task performance in the federated averaging setting. While our findings suggest that the large sizes of the evaluated models are not generally prohibitive to federated training, we found that the different models handle federated averaging to a varying degree. Most notably, DistilBERT converges significantly slower with larger numbers of clients, and under some circumstances, even collapses to chance level performance. Investigating this issue presents an interesting perspective for future research.
Impact of Congestion Charge and Minimum Wage on TNCs: A Case Study for San Francisco
Authors: Sen Li, Kameshwar Poolla, Pravin Varaiya
Date: 2020-03-05 00:00:00

This paper describes the impact on transportation network companies (TNCs) of the imposition of a congestion charge and a driver minimum wage. The impact is assessed using a market equilibrium model to calculate the changes in the number of passenger trips and trip fare, number of drivers employed, the TNC platform profit, the number of TNC vehicles, and city revenue. Two charges are considered: (a) a charge per TNC trip similar to an excise tax, and (b) a charge per vehicle operating hour (whether or not it has a passenger) similar to a road tax. Both charges reduce the number of TNC trips, but this reduction is limited by the wage floor, and the number of TNC vehicles reduced is not significant. The time-based charge is preferable to the trip-based charge since, by penalizing idle vehicle time, the former increases vehicle occupancy. In a case study for San Francisco, the time-based charge is found to be Pareto superior to the trip-based charge as it yields higher passenger surplus, higher platform profits, and higher tax revenue for the city.
Self-Supervised Equivariant Scene Synthesis from Video
Authors: Cinjon Resnick, Or Litany, Cosmas Heiß, Hugo Larochelle, Joan Bruna, Kyunghyun Cho
Date: 2021-02-01 00:00:00

We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method capitalizes on moving characters being equivariant with respect to their transformation across frames and the background being constant with respect to that same transformation. After training, we can manipulate image encodings in real time to create unseen combinations of the delineated components. As far as we know, we are the first method to perform unsupervised extraction and synthesis of interpretable background, character, and animation. We demonstrate results on three datasets: Moving MNIST with backgrounds, 2D video game sprites, and Fashion Modeling.