pwc-archive/papers-with-abstracts huggingface dataset:
Anomaly detection with superexperts under delayed feedback
Authors: Raisa Dzhamtyrova, Carsten Maple
Date: 2020-10-08 00:00:00
Tasks: Anomaly Detection, Unsupervised Anomaly Detection

The increasing connectivity of data and cyber-physical systems has resulted in a growing number of cyber-attacks. Real-time detection of such attacks, through the identification of anomalous activity, is required so that mitigation and contingent actions can be effectively and rapidly deployed. We propose a new approach for aggregating unsupervised anomaly detection algorithms and incorporating feedback when it becomes available. We apply this approach to open-source real datasets and show that both aggregating models, which we call experts, and incorporating feedback significantly improve the performance. An important property of the proposed approaches is their theoretical guarantees that they perform close to the best superexpert, which can switch between the best performing experts, in terms of the cumulative average losses.
Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders
Authors: Jue Wang, Wei Lu
Date: 2020-10-08 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Joint Entity and Relation Extraction, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation, Relation Extraction, Representation Learning, Zero-shot Relation Triplet Extraction

Named entity recognition and relation extraction are two important fundamental problems. Joint learning algorithms have been proposed to solve both tasks simultaneously, and many of them cast the joint task as a table-filling problem. However, they typically focused on learning a single encoder (usually learning representation in the form of a table) to capture information required for both tasks within the same space. We argue that it can be beneficial to design two distinct encoders to capture such two different types of information in the learning process. In this work, we propose the novel {\em table-sequence encoders} where two different encoders -- a table encoder and a sequence encoder are designed to help each other in the representation learning process. Our experiments confirm the advantages of having {\em two} encoders over {\em one} encoder. On several standard datasets, our model shows significant improvements over existing approaches.
Guided Curriculum Learning for Walking Over Complex Terrain
Authors: Brendan Tidd, Nicolas Hudson, Akansel Cosgun
Date: 2020-10-08 00:00:00
Tasks: Deep Reinforcement Learning

Reliable bipedal walking over complex terrain is a challenging problem, using a curriculum can help learning. Curriculum learning is the idea of starting with an achievable version of a task and increasing the difficulty as a success criteria is met. We propose a 3-stage curriculum to train Deep Reinforcement Learning policies for bipedal walking over various challenging terrains. In the first stage, the agent starts on an easy terrain and the terrain difficulty is gradually increased, while forces derived from a target policy are applied to the robot joints and the base. In the second stage, the guiding forces are gradually reduced to zero. Finally, in the third stage, random perturbations with increasing magnitude are applied to the robot base, so the robustness of the policies are improved. In simulation experiments, we show that our approach is effective in learning walking policies, separate from each other, for five terrain types: flat, hurdles, gaps, stairs, and steps. Moreover, we demonstrate that in the absence of human demonstrations, a simple hand designed walking trajectory is a sufficient prior to learn to traverse complex terrain types. In ablation studies, we show that taking out any one of the three stages of the curriculum degrades the learning performance.
Improve Adversarial Robustness via Weight Penalization on Classification Layer
Authors: Cong Xu, Dan Li, Min Yang
Date: 2020-10-08 00:00:00
Tasks: Adversarial Robustness, Classification, General Classification

It is well-known that deep neural networks are vulnerable to adversarial attacks. Recent studies show that well-designed classification parts can lead to better robustness. However, there is still much space for improvement along this line. In this paper, we first prove that, from a geometric point of view, the robustness of a neural network is equivalent to some angular margin condition of the classifier weights. We then explain why ReLU type function is not a good choice for activation under this framework. These findings reveal the limitations of the existing approaches and lead us to develop a novel light-weight-penalized defensive method, which is simple and has a good scalability. Empirical results on multiple benchmark datasets demonstrate that our method can effectively improve the robustness of the network without requiring too much additional computation, while maintaining a high classification precision for clean data.
Association rules over time
Authors: Iztok Fister Jr., Iztok Fister
Date: 2020-10-08 00:00:00

Decisions made nowadays by Artificial Intelligence powered systems are usually hard for users to understand. One of the more important issues faced by developers is exposed as how to create more explainable Machine Learning models. In line with this, more explainable techniques need to be developed, where visual explanation also plays a more important role. This technique could also be applied successfully for explaining the results of Association Rule Mining.This Chapter focuses on two issues: (1) How to discover the relevant association rules, and (2) How to express relations between more attributes visually. For the solution of the first issue, the proposed method uses Differential Evolution, while Sankey diagrams are adopted to solve the second one. This method was applied to a transaction database containing data generated by an amateur cyclist in past seasons, using a mobile device worn during the realization of training sessions that is divided into four time periods. The results of visualization showed that a trend in improving performance of an athlete can be indicated by changing the attributes appearing in the selected association rules in different time periods.
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers
Authors: Tom Hope, Aida Amini, David Wadden, Madeleine van Zuylen, Sravanthi Parasa, Eric Horvitz, Daniel Weld, Roy Schwartz, Hannaneh Hajishirzi
Date: 2020-10-08 00:00:00
Proceeding: NAACL 2021 4
Tasks: Navigate

The COVID-19 pandemic has spawned a diverse body of scientific literature that is challenging to navigate, stimulating interest in automated tools to help find useful knowledge. We pursue the construction of a knowledge base (KB) of mechanisms -- a fundamental concept across the sciences encompassing activities, functions and causal relations, ranging from cellular processes to economic impacts. We extract this information from the natural language of scientific papers by developing a broad, unified schema that strikes a balance between relevance and breadth. We annotate a dataset of mechanisms with our schema and train a model to extract mechanism relations from papers. Our experiments demonstrate the utility of our KB in supporting interdisciplinary scientific search over COVID-19 literature, outperforming the prominent PubMed search in a study with clinical experts.
Clustering Analysis of Interactive Learning Activities Based on Improved BIRCH Algorithm
Authors: Xiaona Xia
Date: 2020-10-08 00:00:00
Tasks: Clustering, Decision Making, Retrieval

Group tendency is a research branch of computer assisted learning. The construction of good learning behavior is of great significance to learners' learning process and learning effect, and is the key basis of data-driven education decision-making. Clustering analysis is an effective method for the study of group tendency. Therefore, it is necessary to obtain the online learning behavior big data set of multi period and multi course, and describe the learning behavior as multi-dimensional learning interaction activities. First of all, on the basis of data initialization and standardization, we locate the classification conditions of data, realize the differentiation and integration of learning behavior, and form multiple subsets of data to be clustered; secondly, according to the topological relevance and dependence between learning interaction activities, we design an improved algorithm of BIRCH clustering based on random walking strategy, which realizes the retrieval evaluation and data of key learning interaction activities; Thirdly, through the calculation and comparison of several performance indexes, the improved algorithm has obvious advantages in learning interactive activity clustering, and the clustering process and results are feasible and reliable. The conclusion of this study can be used for reference and can be popularized. It has practical significance for the research of education big data and the practical application of learning analytics.
A Comparative Study on Effects of Original and Pseudo Labels for Weakly Supervised Learning for Car Localization Problem
Authors: Cenk Bircanoglu
Date: 2020-10-08 00:00:00
Tasks: Edge Detection, Weakly-supervised Learning

In this study, the effects of different class labels created as a result of multiple conceptual meanings on localization using Weakly Supervised Learning presented on Car Dataset. In addition, the generated labels are included in the comparison, and the solution turned into Unsupervised Learning. This paper investigates multiple setups for car localization in the images with other approaches rather than Supervised Learning. To predict localization labels, Class Activation Mapping (CAM) is implemented and from the results, the bounding boxes are extracted by using morphological edge detection. Besides the original class labels, generated class labels also employed to train CAM on which turn to a solution to Unsupervised Learning example. In the experiments, we first analyze the effects of class labels in Weakly Supervised localization on the Compcars dataset. We then show that the proposed Unsupervised approach outperforms the Weakly Supervised method in this particular dataset by approximately %6.
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling
Authors: Parker Riley, Noah Constant, Mandy Guo, Girish Kumar, David Uthus, Zarana Parekh
Date: 2020-10-08 00:00:00
Proceeding: ACL 2021 5
Tasks: Decoder, Style Transfer, Text Style Transfer

We present a novel approach to the problem of text style transfer. Unlike previous approaches requiring style-labeled training data, our method makes use of readily-available unlabeled text by relying on the implicit connection in style between adjacent sentences, and uses labeled data only at inference time. We adapt T5 (Raffel et al., 2020), a strong pretrained text-to-text model, to extract a style vector from text and use it to condition the decoder to perform style transfer. As our label-free training results in a style vector space encoding many facets of style, we recast transfers as "targeted restyling" vector operations that adjust specific attributes of the input while preserving others. We demonstrate that training on unlabeled Amazon reviews data results in a model that is competitive on sentiment transfer, even compared to models trained fully on labeled data. Furthermore, applying our novel method to a diverse corpus of unlabeled web text results in a single model capable of transferring along multiple dimensions of style (dialect, emotiveness, formality, politeness, sentiment) despite no additional training and using only a handful of exemplars at inference time.
Learning the Linear Quadratic Regulator from Nonlinear Observations
Authors: Zakaria Mhammedi, Dylan J. Foster, Max Simchowitz, Dipendra Misra, Wen Sun, Akshay Krishnamurthy, Alexander Rakhlin, John Langford
Date: 2020-10-08 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: continuous-control, Continuous Control, Decoder

We introduce a new problem setting for continuous control called the LQR with Rich Observations, or RichLQR. In our setting, the environment is summarized by a low-dimensional continuous latent state with linear dynamics and quadratic costs, but the agent operates on high-dimensional, nonlinear observations such as images from a camera. To enable sample-efficient learning, we assume that the learner has access to a class of decoder functions (e.g., neural networks) that is flexible enough to capture the mapping from observations to latent states. We introduce a new algorithm, RichID, which learns a near-optimal policy for the RichLQR with sample complexity scaling only with the dimension of the latent state space and the capacity of the decoder function class. RichID is oracle-efficient and accesses the decoder class only through calls to a least-squares regression oracle. Our results constitute the first provable sample complexity guarantee for continuous control with an unknown nonlinearity in the system model and general function approximation.
Mapping of Real World Problems to Nature Inspired Algorithm using Goal based Classification and TRIZ
Authors: Palak Sukharamwala, Manojkumar Parmar
Date: 2020-10-08 00:00:00

The technologies and algorithms are growing at an exponential rate. The technologies are capable enough to solve technically challenging and complex problems which seemed impossible task. However, the trending methods and approaches are facing multiple challenges on various fronts of data, algorithms, software, computational complexities, and energy efficiencies. Nature also faces similar challenges. Nature has solved those challenges and formulation of those are available as Nature Inspired Algorithms (NIA), which are derived based on the study of nature. A novel method based on TRIZ to map the real-world problems to nature problems is explained here.TRIZ is a Theory of inventive problem solving. Using the proposed framework, best NIA can be identified to solve the real-world problems. For this framework to work, a novel classification of NIA based on the end goal that nature is trying to achieve is devised. The application of the this framework along with examples is also discussed.
The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive Experiments and a Paradox Concerning Logging Policy
Authors: Masahiro Kato, Shota Yasui, Kenichiro McAlinn
Date: 2020-10-08 00:00:00
Tasks: Causal Inference, Time Series Analysis

The doubly robust (DR) estimator, which consists of two nuisance parameters, the conditional mean outcome and the logging policy (the probability of choosing an action), is crucial in causal inference. This paper proposes a DR estimator for dependent samples obtained from adaptive experiments. To obtain an asymptotically normal semiparametric estimator from dependent samples with non-Donsker nuisance estimators, we propose adaptive-fitting as a variant of sample-splitting. We also report an empirical paradox that our proposed DR estimator tends to show better performances compared to other estimators utilizing the true logging policy. While a similar phenomenon is known for estimators with i.i.d. samples, traditional explanations based on asymptotic efficiency cannot elucidate our case with dependent samples. We confirm this hypothesis through simulation studies.
Age and Gender Prediction From Face Images Using Attentional Convolutional Network
Authors: Amirali Abdolrashidi, Mehdi Minaei, Elham Azimi, Shervin Minaee
Date: 2020-10-08 00:00:00
Tasks: Gender Prediction, Multi-Task Learning

Automatic prediction of age and gender from face images has drawn a lot of attention recently, due it is wide applications in various facial analysis problems. However, due to the large intra-class variation of face images (such as variation in lighting, pose, scale, occlusion), the existing models are still behind the desired accuracy level, which is necessary for the use of these models in real-world applications. In this work, we propose a deep learning framework, based on the ensemble of attentional and residual convolutional networks, to predict gender and age group of facial images with high accuracy rate. Using attention mechanism enables our model to focus on the important and informative parts of the face, which can help it to make a more accurate prediction. We train our model in a multi-task learning fashion, and augment the feature embedding of the age classifier, with the predicted gender, and show that doing so can further increase the accuracy of age prediction. Our model is trained on a popular face age and gender dataset, and achieved promising results. Through visualization of the attention maps of the train model, we show that our model has learned to become sensitive to the right regions of the face.
Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines
Authors: Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Pushkar Shukla, Sadhana Kumaravel, Gerald Tesauro, Kartik Talamadupula, Mrinmaya Sachan, Murray Campbell
Date: 2020-10-08 00:00:00
Tasks: Common Sense Reasoning, Commonsense Reasoning for RL, Decision Making, Reinforcement Learning (RL), Sequential Decision Making, text-based games

Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making. In this paper, we examine the problem of infusing RL agents with commonsense knowledge. Such knowledge would allow agents to efficiently act in the world by pruning out implausible actions, and to perform look-ahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We also introduce several baseline RL agents which track the sequential context and dynamically retrieve the relevant commonsense knowledge from ConceptNet. We show that agents which incorporate commonsense knowledge in TWC perform better, while acting more efficiently. We conduct user-studies to estimate human performance on TWC and show that there is ample room for future improvement.
An Empirical Study on Model-agnostic Debiasing Strategies for Robust Natural Language Inference
Authors: Tianyu Liu, Xin Zheng, Xiaoan Ding, Baobao Chang, Zhifang Sui
Date: 2020-10-08 00:00:00
Proceeding: CONLL 2020
Tasks: Data Augmentation, Mixture-of-Experts, Natural Language Inference

The prior work on natural language inference (NLI) debiasing mainly targets at one or few known biases while not necessarily making the models more robust. In this paper, we focus on the model-agnostic debiasing strategies and explore how to (or is it possible to) make the NLI models robust to multiple distinct adversarial attacks while keeping or even strengthening the models' generalization power. We firstly benchmark prevailing neural NLI models including pretrained ones on various adversarial datasets. We then try to combat distinct known biases by modifying a mixture of experts (MoE) ensemble method and show that it's nontrivial to mitigate multiple NLI biases at the same time, and that model-level ensemble method outperforms MoE ensemble method. We also perform data augmentation including text swap, word substitution and paraphrase and prove its efficiency in combating various (though not all) adversarial attacks at the same time. Finally, we investigate several methods to merge heterogeneous training data (1.35M) and perform model ensembling, which are straightforward but effective to strengthen NLI models.
Detect All Abuse! Toward Universal Abusive Language Detection Models
Authors: Kunze Wang, Dong Lu, Soyeon Caren Han, Siqu Long, Josiah Poon
Date: 2020-10-08 00:00:00
Proceeding: COLING 2020 8
Tasks: Abusive Language, All, Graph Embedding

Online abusive language detection (ALD) has become a societal issue of increasing importance in recent years. Several previous works in online ALD focused on solving a single abusive language problem in a single domain, like Twitter, and have not been successfully transferable to the general ALD task or domain. In this paper, we introduce a new generic ALD framework, MACAS, which is capable of addressing several types of ALD tasks across different domains. Our generic framework covers multi-aspect abusive language embeddings that represent the target and content aspects of abusive language and applies a textual graph embedding that analyses the user's linguistic behaviour. Then, we propose and use the cross-attention gate flow mechanism to embrace multiple aspects of abusive language. Quantitative and qualitative evaluation results show that our ALD algorithm rivals or exceeds the six state-of-the-art ALD algorithms across seven ALD datasets covering multiple aspects of abusive language and different online community domains.
Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention
Authors: Yang Li, Tao Shen, Guodong Long, Jing Jiang, Tianyi Zhou, Chengqi Zhang
Date: 2020-10-08 00:00:00
Proceeding: COLING 2020 8
Tasks: Relation, Relation Extraction, Sentence

Wrong labeling problem and long-tail relations are two main challenges caused by distant supervision in relation extraction. Recent works alleviate the wrong labeling by selective attention via multi-instance learning, but cannot well handle long-tail relations even if hierarchies of the relations are introduced to share knowledge. In this work, we propose a novel neural network, Collaborating Relation-augmented Attention (CoRA), to handle both the wrong labeling and long-tail relations. Particularly, we first propose relation-augmented attention network as base model. It operates on sentence bag with a sentence-to-relation attention to minimize the effect of wrong labeling. Then, facilitated by the proposed base model, we introduce collaborating relation features shared among relations in the hierarchies to promote the relation-augmenting process and balance the training data for long-tail relations. Besides the main training objective to predict the relation of a sentence bag, an auxiliary objective is utilized to guide the relation-augmenting process for a more accurate bag-level representation. In the experiments on the popular benchmark dataset NYT, the proposed CoRA improves the prior state-of-the-art performance by a large margin in terms of Precision@N, AUC and Hits@K. Further analyses verify its superior capability in handling long-tail relations in contrast to the competitors.
DBLFace: Domain-Based Labels for NIR-VIS Heterogeneous Face Recognition
Authors: Ha Le, Ioannis A. Kakadiaris
Date: 2020-10-08 00:00:00
Tasks: Face Recognition, Heterogeneous Face Recognition

Deep learning-based domain-invariant feature learning methods are advancing in near-infrared and visible (NIR-VIS) heterogeneous face recognition. However, these methods are prone to overfitting due to the large intra-class variation and the lack of NIR images for training. In this paper, we introduce Domain-Based Label Face (DBLFace), a learning approach based on the assumption that a subject is not represented by a single label but by a set of labels. Each label represents images of a specific domain. In particular, a set of two labels per subject, one for the NIR images and one for the VIS images, are used for training a NIR-VIS face recognition model. The classification of images into different domains reduces the intra-class variation and lessens the negative impact of data imbalance in training. To train a network with sets of labels, we introduce a domain-based angular margin loss and a maximum angular loss to maintain the inter-class discrepancy and to enforce the close relationship of labels in a set. Quantitative experiments confirm that DBLFace significantly improves the rank-1 identification rate by 6.7% on the EDGE20 dataset and achieves state-of-the-art performance on the CASIA NIR-VIS 2.0 dataset.