pwc-archive/papers-with-abstracts huggingface dataset:
"The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition
Authors: Yichao Zhou, Jyun-Yu Jiang, Jieyu Zhao, Kai-Wei Chang, Wei Wang
Date: 2020-04-29 00:00:00
Tasks: Sentence

Humor plays an important role in human languages and it is essential to model humor when building intelligence systems. Among different forms of humor, puns perform wordplay for humorous effects by employing words with double entendre and high phonetic similarity. However, identifying and modeling puns are challenging as puns usually involved implicit semantic or phonological tricks. In this paper, we propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor, detect if a sentence contains puns and locate them in the sentence. PCPR derives contextualized representation for each word in a sentence by capturing the association between the surrounding context and its corresponding phonetic symbols. Extensive experiments are conducted on two benchmark datasets. Results demonstrate that the proposed approach significantly outperforms the state-of-the-art methods in pun detection and location tasks. In-depth analyses verify the effectiveness and robustness of PCPR.
SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification
Authors: Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri, Preslav Nakov
Date: 2020-04-29 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: Language Identification

The widespread use of offensive content in social media has led to an abundance of research in detecting language such as hate speech, cyberbullying, and cyber-aggression. Recent work presented the OLID dataset, which follows a taxonomy for offensive language identification that provides meaningful information for understanding the type and the target of offensive messages. However, it is limited in size and it might be biased towards offensive language as it was collected using keywords. In this work, we present SOLID, an expanded dataset, where the tweets were collected in a more principled manner. SOLID contains over nine million English tweets labeled in a semi-supervised fashion. We demonstrate that using SOLID along with OLID yields sizable performance gains on the OLID test set for two different models, especially for the lower levels of the taxonomy.
Pragmatic Issue-Sensitive Image Captioning
Authors: Allen Nie, Reuben Cohn-Gordon, Christopher Potts
Date: 2020-04-29 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Descriptive, Image Captioning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Image captioning systems have recently improved dramatically, but they still tend to produce captions that are insensitive to the communicative goals that captions should meet. To address this, we propose Issue-Sensitive Image Captioning (ISIC). In ISIC, a captioning system is given a target image and an issue, which is a set of images partitioned in a way that specifies what information is relevant. The goal of the captioner is to produce a caption that resolves this issue. To model this task, we use an extension of the Rational Speech Acts model of pragmatic language use. Our extension is built on top of state-of-the-art pretrained neural image captioners and explicitly reasons about issues in our sense. We establish experimentally that these models generate captions that are both highly descriptive and issue-sensitive, and we show how ISIC can complement and enrich the related task of Visual Question Answering.
What Happens To BERT Embeddings During Fine-tuning?
Authors: Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, Ian Tenney
Date: 2020-04-29 00:00:00
Proceeding: EMNLP (BlackboxNLP) 2020 11
Tasks: Dependency Parsing, Sentence

While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks. Using a suite of analysis techniques (probing classifiers, Representational Similarity Analysis, and model ablations), we investigate how fine-tuning affects the representations of the BERT model. We find that while fine-tuning necessarily makes significant changes, it does not lead to catastrophic forgetting of linguistic phenomena. We instead find that fine-tuning primarily affects the top layers of BERT, but with noteworthy variation across tasks. In particular, dependency parsing reconfigures most of the model, whereas SQuAD and MNLI appear to involve much shallower processing. Finally, we also find that fine-tuning has a weaker effect on representations of out-of-domain sentences, suggesting room for improvement in model generalization.
The Effect of Natural Distribution Shift on Question Answering Models
Authors: John Miller, Karl Krauth, Benjamin Recht, Ludwig Schmidt
Date: 2020-04-29 00:00:00
Proceeding: ICML 2020 1
Tasks: Articles, Question Answering

We build four new test sets for the Stanford Question Answering Dataset (SQuAD) and evaluate the ability of question-answering systems to generalize to new data. Our first test set is from the original Wikipedia domain and measures the extent to which existing systems overfit the original test set. Despite several years of heavy test set re-use, we find no evidence of adaptive overfitting. The remaining three test sets are constructed from New York Times articles, Reddit posts, and Amazon product reviews and measure robustness to natural distribution shifts. Across a broad range of models, we observe average performance drops of 3.8, 14.0, and 17.4 F1 points, respectively. In contrast, a strong human baseline matches or exceeds the performance of SQuAD models on the original domain and exhibits little to no drop in new domains. Taken together, our results confirm the surprising resilience of the holdout method and emphasize the need to move towards evaluation metrics that incorporate robustness to natural distribution shifts.
Distantly-Supervised Neural Relation Extraction with Side Information using BERT
Authors: Johny Moreira, Chaina Oliveira, David Macêdo, Cleber Zanchettin, Luciano Barbosa
Date: 2020-04-29 00:00:00
Tasks: Relation, Relation Extraction, Sentence

Relation extraction (RE) consists in categorizing the relationship between entities in a sentence. A recent paradigm to develop relation extractors is Distant Supervision (DS), which allows the automatic creation of new datasets by taking an alignment between a text corpus and a Knowledge Base (KB). KBs can sometimes also provide additional information to the RE task. One of the methods that adopt this strategy is the RESIDE model, which proposes a distantly-supervised neural relation extraction using side information from KBs. Considering that this method outperformed state-of-the-art baselines, in this paper, we propose a related approach to RESIDE also using additional side information, but simplifying the sentence encoding with BERT embeddings. Through experiments, we show the effectiveness of the proposed method in Google Distant Supervision and Riedel datasets concerning the BGWA and RESIDE baseline methods. Although Area Under the Curve is decreased because of unbalanced datasets, P@N results have shown that the use of BERT as sentence encoding allows superior performance to baseline methods.
Learning Bayesian Networks from Incomplete Data with the Node-Average Likelihood
Authors: Tjebbe Bodewes, Marco Scutari
Date: 2020-04-29 00:00:00

Bayesian network (BN) structure learning from complete data has been extensively studied in the literature. However, fewer theoretical results are available for incomplete data, and most are related to the Expectation-Maximisation (EM) algorithm. Balov (2013) proposed an alternative approach called Node-Average Likelihood (NAL) that is competitive with EM but computationally more efficient; and he proved its consistency and model identifiability for discrete BNs. In this paper, we give general sufficient conditions for the consistency of NAL; and we prove consistency and identifiability for conditional Gaussian BNs, which include discrete and Gaussian BNs as special cases. Furthermore, we confirm our results and the results in Balov (2013) with an independent simulation study. Hence we show that NAL has a much wider applicability than originally implied in Balov (2013), and that it is competitive with EM for conditional Gaussian BNs as well.
Whittle index based Q-learning for restless bandits with average reward
Authors: Konstantin E. Avrachenkov, Vivek S. Borkar
Date: 2020-04-29 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A novel reinforcement learning algorithm is introduced for multiarmed restless bandits with average reward, using the paradigms of Q-learning and Whittle index. Specifically, we leverage the structure of the Whittle index policy to reduce the search space of Q-learning, resulting in major computational gains. Rigorous convergence analysis is provided, supported by numerical experiments. The numerical experiments show excellent empirical performance of the proposed scheme.
A Benchmark Dataset of Check-worthy Factual Claims
Authors: Fatma Arslan, Naeemul Hassan, Chengkai Li, Mark Tremayne
Date: 2020-04-29 00:00:00
Tasks: Fact Checking

In this paper we present the ClaimBuster dataset of 23,533 statements extracted from all U.S. general election presidential debates and annotated by human coders. The ClaimBuster dataset can be leveraged in building computational methods to identify claims that are worth fact-checking from the myriad of sources of digital or traditional media. The ClaimBuster dataset is publicly available to the research community, and it can be found at http://doi.org/10.5281/zenodo.3609356.
UAV and Machine Learning Based Refinement of a Satellite-Driven Vegetation Index for Precision Agriculture
Authors: Vittorio Mazzia, Lorenzo Comba, Aleem Khaliq, Marcello Chiaberge, Paolo Gay
Date: 2020-04-29 00:00:00
Tasks: BIG-bench Machine Learning, Management

Precision agriculture is considered to be a fundamental approach in pursuing a low-input, high-efficiency, and sustainable kind of agriculture when performing site-specific management practices. To achieve this objective, a reliable and updated description of the local status of crops is required. Remote sensing, and in particular satellite-based imagery, proved to be a valuable tool in crop mapping, monitoring, and diseases assessment. However, freely available satellite imagery with low or moderate resolutions showed some limits in specific agricultural applications, e.g., where crops are grown by rows. Indeed, in this framework, the satellite's output could be biased by intra-row covering, giving inaccurate information about crop status. This paper presents a novel satellite imagery refinement framework, based on a deep learning technique which exploits information properly derived from high resolution images acquired by unmanned aerial vehicle (UAV) airborne multispectral sensors. To train the convolutional neural network, only a single UAV-driven dataset is required, making the proposed approach simple and cost-effective. A vineyard in Serralunga d'Alba (Northern Italy) was chosen as a case study for validation purposes. Refined satellite-driven normalized difference vegetation index (NDVI) maps, acquired in four different periods during the vine growing season, were shown to better describe crop status with respect to raw datasets by correlation analysis and ANOVA. In addition, using a K-means based classifier, 3-class vineyard vigor maps were profitably derived from the NDVI maps, which are a valuable tool for growers.
To Reduce Gross NPA and Classify Defaulters Using Shannon Entropy
Authors: Ambarish Moharil, Nikhil Sonavane, Chirag Kedia, Mansimran Singh Anand
Date: 2020-04-29 00:00:00

Non Performing Asset(NPA) has been in a serious attention by banks over the past few years. NPA cause a huge loss to the banks hence it becomes an extremely critical step in deciding which loans have the capabilities to become an NPA and thereby deciding which loans to grant and which ones to reject. In this paper which focuses on the exact crux of the matter we have proposed an algorithm which is designed to handle the financial data very meticulously to predict with a very high accuracy whether a particular loan would be classified as a NPA in future or not. Instead of the conventional less accurate classifiers used to decide which loans can turn to be NPA we build our own classifier model using Entropy as the base. We have created an entropy based classifier using Shannon Entropy. The classifier model categorizes our data points in two categories accepted or rejected. We make use of local entropy and global entropy to help us determine the output. The entropy classifier model is then compared with existing classifiers used to predict NPAs thereby giving us an idea about the performance.
Revealing the Phase Diagram of Kitaev Materials by Machine Learning: Cooperation and Competition between Spin Liquids
Authors: Ke Liu, Nicolas Sadoune, Nihal Rao, Jonas Greitemann, Lode Pollet
Date: 2020-04-29 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

Kitaev materials are promising materials for hosting quantum spin liquids and investigating the interplay of topological and symmetry-breaking phases. We use an unsupervised and interpretable machine-learning method, the tensorial-kernel support vector machine, to study the honeycomb Kitaev-$\Gamma$ model in a magnetic field. Our machine learns the global classical phase diagram and the associated analytical order parameters, including several distinct spin liquids, two exotic $S_3$ magnets, and two modulated $S_3 \times Z_3$ magnets. We find that the extension of Kitaev spin liquids and a field-induced suppression of magnetic order already occur in the large-$S$ limit, implying that critical parts of the physics of Kitaev materials can be understood at the classical level. Moreover, the two $S_3 \times Z_3$ orders are induced by competition between Kitaev and $\Gamma$ spin liquids and feature a different type of spin-lattice entangled modulation, which requires a matrix description instead of scalar phase factors. Our work provides a direct instance of a machine detecting new phases and paves the way towards the development of automated tools to explore unsolved problems in many-body physics.
Classification of Equation of State in Relativistic Heavy-Ion Collisions Using Deep Learning
Authors: Yu. Kvasiuk, E. Zabrodin, L. Bravina, I. Didur, M. Frolov
Date: 2020-04-29 00:00:00
Tasks: Deep Learning, General Classification

Convolutional Neural Nets, which is a powerful method of Deep Learning, is applied to classify equation of state of heavy-ion collision event generated within the UrQMD model. Event-by-event transverse momentum and azimuthal angle distributions of protons are used to train a classifier. An overall accuracy of classification of 98\% is reached for Au+Au events at $\sqrt{s_{NN}} = 11$ GeV. Performance of classifiers, trained on events at different colliding energies, is investigated. Obtained results indicate extensive possibilities of application of Deep Learning methods to other problems in physics of heavy-ion collisions.
Learning from Sparse Datasets: Predicting Concrete's Strength by Machine Learning
Authors: Boya Ouyang, Yuhai Li, Yu Song, Feishu Wu, Huizi Yu, Yongzhe Wang, Mathieu Bauchy, Gaurav Sant
Date: 2020-04-29 00:00:00
Tasks: BIG-bench Machine Learning

Despite enormous efforts over the last decades to establish the relationship between concrete proportioning and strength, a robust knowledge-based model for accurate concrete strength predictions is still lacking. As an alternative to physical or chemical-based models, data-driven machine learning (ML) methods offer a new solution to this problem. Although this approach is promising for handling the complex, non-linear, non-additive relationship between concrete mixture proportions and strength, a major limitation of ML lies in the fact that large datasets are needed for model training. This is a concern as reliable, consistent strength data is rather limited, especially for realistic industrial concretes. Here, based on the analysis of a large dataset (>10,000 observations) of measured compressive strengths from industrially-produced concretes, we compare the ability of select ML algorithms to "learn" how to reliably predict concrete strength as a function of the size of the dataset. Based on these results, we discuss the competition between how accurate a given model can eventually be (when trained on a large dataset) and how much data is actually required to train this model.
Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks
Authors: Gerrit Schoettler, Ashvin Nair, Juan Aparicio Ojea, Sergey Levine, Eugen Solowjow
Date: 2020-04-29 00:00:00
Tasks: Friction, Meta Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Robotic insertion tasks are characterized by contact and friction mechanics, making them challenging for conventional feedback control methods due to unmodeled physical effects. Reinforcement learning (RL) is a promising approach for learning control policies in such settings. However, RL can be unsafe during exploration and might require a large amount of real-world training data, which is expensive to collect. In this paper, we study how to use meta-reinforcement learning to solve the bulk of the problem in simulation by solving a family of simulated industrial insertion tasks and then adapt policies quickly in the real world. We demonstrate our approach by training an agent to successfully perform challenging real-world insertion tasks using less than 20 trials of real-world experience. Videos and other material are available at https://pearl-insertion.github.io/
Fast Gradient Descent Algorithm for Image Classification with Neural Networks
Authors: Abdelkrim El Mouatasim
Date: 2020-04-18 00:00:00
Proceeding: Signal Image and Video Processing 2020 4
Tasks: Classification, General Classification, image-classification, Image Classification

Any optimization of gradient descent methods involves selecting a learning rate. Tuning the learning rate can quickly become repetitive with deeper models of image classification, does not necessarily lead to optimal convergence. We proposed in this paper, a modification of the gradient descent algorithm in which the Nestrove step is added, and the learning rate is update in each epoch. Instead, we learn learning rate itself, either by Armijo rule, or by control step. Our algorithm called fast gradient descent (FGD) for solving image classification with neural networks problems, the quadratic convergence rate o(k^2) of FGD algorithm are proved. FGD algorithm are applicate to a MNIST dataset. The numerical experiment, show that our approach FGD algorithm is faster than gradient descent algorithms.
Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability
Authors: Nathan Inkawhich, Kevin J Liang, Binghui Wang, Matthew Inkawhich, Lawrence Carin, Yiran Chen
Date: 2020-04-29 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Adversarial Attack

We consider the blackbox transfer-based targeted adversarial attack threat model in the realm of deep neural network (DNN) image classifiers. Rather than focusing on crossing decision boundaries at the output layer of the source model, our method perturbs representations throughout the extracted feature hierarchy to resemble other classes. We design a flexible attack framework that allows for multi-layer perturbations and demonstrates state-of-the-art targeted transfer performance between ImageNet DNNs. We also show the superiority of our feature space methods under a relaxation of the common assumption that the source and target models are trained on the same dataset and label space, in some instances achieving a $10\times$ increase in targeted success rate relative to other blackbox transfer methods. Finally, we analyze why the proposed methods outperform existing attack strategies and show an extension of the method in the case when limited queries to the blackbox model are allowed.
Hedging and machine learning driven crude oil data analysis using a refined Barndorff-Nielsen and Shephard model
Authors: Humayra Shoshi, Indranil SenGupta
Date: 2020-04-29 00:00:00

In this paper, a refined Barndorff-Nielsen and Shephard (BN-S) model is implemented to find an optimal hedging strategy for commodity markets. The refinement of the BN-S model is obtained with various machine and deep learning algorithms. The refinement leads to the extraction of a deterministic parameter from the empirical data set. The problem is transformed to an appropriate classification problem with a couple of different approaches: the volatility approach and the duration approach. The analysis is implemented to the Bakken crude oil data and the aforementioned deterministic parameter is obtained for a wide range of data sets. With the implementation of this parameter in the refined model, the resulting model performs much better than the classical BN-S model.
Transfer Learning for Thermal Comfort Prediction in Multiple Cities
Authors: Nan Gao, Wei Shao, Mohammad Saiedur Rahaman, Jun Zhai, Klaus David, Flora D. Salim
Date: 2020-04-29 00:00:00
Tasks: Transfer Learning

HVAC (Heating, Ventilation and Air Conditioning) system is an important part of a building, which constitutes up to 40% of building energy usage. The main purpose of HVAC, maintaining appropriate thermal comfort, is crucial for the best utilisation of energy usage. Besides, thermal comfort is also crucial for well-being, health, and work productivity. Recently, data-driven thermal comfort models have got better performance than traditional knowledge-based methods (e.g. Predicted Mean Vote Model). An accurate thermal comfort model requires a large amount of self-reported thermal comfort data from indoor occupants which undoubtedly remains a challenge for researchers. In this research, we aim to tackle this data-shortage problem and boost the performance of thermal comfort prediction. We utilise sensor data from multiple cities in the same climate zone to learn thermal comfort patterns. We present a transfer learning based multilayer perceptron model from the same climate zone (TL-MLP-C*) for accurate thermal comfort prediction. Extensive experimental results on ASHRAE RP-884, the Scales Project and Medium US Office datasets show that the performance of the proposed TL-MLP-C* exceeds the state-of-the-art methods in accuracy, precision and F1-score.