pwc-archive/papers-with-abstracts huggingface dataset:
VANiLLa : Verbalized Answers in Natural Language at Large Scale
Authors: Debanjali Biswas, Mohnish Dubey, Md Rashad Al Hasan Rony, Jens Lehmann
Date: 2021-05-24 00:00:00
Tasks: BIG-bench Machine Learning, Knowledge Graphs, Question Answering, Sentence, Text Generation

In the last years, there have been significant developments in the area of Question Answering over Knowledge Graphs (KGQA). Despite all the notable advancements, current KGQA datasets only provide the answers as the direct output result of the formal query, rather than full sentences incorporating question context. For achieving coherent answers sentence with the question's vocabulary, template-based verbalization so are usually employed for a better representation of answers, which in turn require extensive expert intervention. Thus, making way for machine learning approaches; however, there is a scarcity of datasets that empower machine learning models in this area. Hence, we provide the VANiLLa dataset which aims at reducing this gap by offering answers in natural language sentences. The answer sentences in this dataset are syntactically and semantically closer to the question than to the triple fact. Our dataset consists of over 100k simple questions adapted from the CSQA and SimpleQuestionsWikidata datasets and generated using a semi-automatic framework. We also present results of training our dataset on multiple baseline models adapted from current state-of-the-art Natural Language Generation (NLG) architectures. We believe that this dataset will allow researchers to focus on finding suitable methodologies and architectures for answer verbalization.
IITP at AILA 2019: System Report for Artificial Intelligence for Legal Assistance Shared Task
Authors: Baban Gain, Dibyanayan Bandyopadhyay, Arkadipta De, Tanik Saikh, Asif Ekbal
Date: 2021-05-24 00:00:00
Tasks: Information Retrieval, Retrieval, Task 2

In this article, we present a description of our systems as a part of our participation in the shared task namely Artificial Intelligence for Legal Assistance (AILA 2019). This is an integral event of Forum for Information Retrieval Evaluation-2019. The outcomes of this track would be helpful for the automation of the working process of the Indian Judiciary System. The manual working procedures and documentation at any level (from lower to higher court) of the judiciary system are very complex in nature. The systems produced as a part of this track would assist the law practitioners. It would be helpful for common men too. This kind of track also opens the path of research of Natural Language Processing (NLP) in the judicial domain. This track defined two problems such as Task 1: Identifying relevant prior cases for a given situation and Task 2: Identifying the most relevant statutes for a given situation. We tackled both of them. Our proposed approaches are based on BM25 and Doc2Vec. As per the results declared by the task organizers, we are in 3rd and a modest position in Task 1 and Task 2 respectively.
RobeCzech: Czech RoBERTa, a monolingual contextualized language representation model
Authors: Milan Straka, Jakub Náplava, Jana Straková, David Samuel
Date: 2021-05-24 00:00:00
Tasks: Semantic Parsing

We present RobeCzech, a monolingual RoBERTa language representation model trained on Czech data. RoBERTa is a robustly optimized Transformer-based pretraining approach. We show that RobeCzech considerably outperforms equally-sized multilingual and Czech-trained contextualized language representation models, surpasses current state of the art in all five evaluated NLP tasks and reaches state-of-the-art results in four of them. The RobeCzech model is released publicly at https://hdl.handle.net/11234/1-3691 and https://huggingface.co/ufal/robeczech-base.
DaN+: Danish Nested Named Entities and Lexical Normalization
Authors: Barbara Plank, Kristian Nørgaard Jensen, Rob van der Goot
Date: 2021-05-24 00:00:00
Proceeding: COLING 2020 8
Tasks: Cross-Lingual Transfer, Lexical Normalization, Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

This paper introduces DaN+, a new multi-domain corpus and annotation guidelines for Danish nested named entities (NEs) and lexical normalization to support research on cross-lingual cross-domain learning for a less-resourced language. We empirically assess three strategies to model the two-layer Named Entity Recognition (NER) task. We compare transfer capabilities from German versus in-language annotation from scratch. We examine language-specific versus multilingual BERT, and study the effect of lexical normalization on NER. Our results show that 1) the most robust strategy is multi-task learning which is rivaled by multi-label decoding, 2) BERT-based NER models are sensitive to domain shifts, and 3) in-language BERT and lexical normalization are the most beneficial on the least canonical data. Our results also show that an out-of-domain setup remains challenging, while performance on news plateaus quickly. This highlights the importance of cross-domain evaluation of cross-lingual transfer.
Introducing the Talk Markup Language (TalkML):Adding a little social intelligence to industrial speech interfaces
Authors: Peter Wallis
Date: 2021-05-24 00:00:00
Tasks: Management, Natural Language Understanding

Virtual Personal Assistants like Siri have great potential but such developments hit the fundamental problem of how to make computational devices that understand human speech. Natural language understanding is one of the more disappointing failures of AI research and it seems there is something we computer scientists don't get about the nature of language. Of course philosophers and linguists think quite differently about language and this paper describes how we have taken ideas from other disciplines and implemented them. The background to the work is to take seriously the notion of language as action and look at what people actually do with language using the techniques of Conversation Analysis. The observation has been that human communication is (behind the scenes) about the management of social relations as well as the (foregrounded) passing of information. To claim this is one thing but to implement it requires a mechanism. The mechanism described here is based on the notion of language being intentional - we think intentionally, talk about them and recognise them in others - and cooperative in that we are compelled to help out. The way we are compelled points to a solution to the ever present problem of keeping the human on topic. The approach has led to a recent success in which we significantly improve user satisfaction independent of task completion. Talk Markup Language (TalkML) is a draft alternative to VoiceXML that, we propose, greatly simplifies the scripting of interaction by providing default behaviours for no input and not recognised speech events.
Assessing perceived organizational leadership styles through twitter text mining
Authors: A. La Bella, A. Fronzetti Colladon, E. Battistoni, S. Castellan, M. Francucci
Date: 2021-05-24 00:00:00
Tasks: text-classification, Text Classification

We propose a text classification tool based on support vector machines for the assessment of organizational leadership styles, as appearing to Twitter users. We collected Twitter data over 51 days, related to the first 30 Italian organizations in the 2015 ranking of Forbes Global 2000-out of which we selected the five with the most relevant volumes of tweets. We analyzed the communication of the company leaders, together with the dialogue among the stakeholders of each company, to understand the association with perceived leadership styles and dimensions. To assess leadership profiles, we referred to the 10-factor model developed by Barchiesi and La Bella in 2007. We maintain the distinctiveness of the approach we propose, as it allows a rapid assessment of the perceived leadership capabilities of an enterprise, as they emerge from its social media interactions. It can also be used to show how companies respond and manage their communication when specific events take place, and to assess their stakeholder's reactions.
Editorial introduction: The power of words and networks
Authors: A. Fronzetti Colladon, P. Gloor, D. F. Iezzi
Date: 2021-05-24 00:00:00
Tasks: Articles, Management

According to Freud "words were originally magic and to this day words have retained much of their ancient magical power". By words, behaviors are transformed and problems are solved. The way we use words reveals our intentions, goals and values. Novel tools for text analysis help understand the magical power of words. This power is multiplied, if it is combined with the study of social networks, i.e. with the analysis of relationships among social units. This special issue of the International Journal of Information Management, entitled "Combining Social Network Analysis and Text Mining: from Theory to Practice", includes heterogeneous and innovative research at the nexus of text mining and social network analysis. It aims to enrich work at the intersection of these fields, which still lags behind in theoretical, empirical, and methodological foundations. The nine articles accepted for inclusion in this special issue all present methods and tools that have business applications. They are summarized in this editorial introduction.
PTR: Prompt Tuning with Rules for Text Classification
Authors: Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, Maosong Sun
Date: 2021-05-24 00:00:00
Tasks: Classification, Natural Language Inference, Relation Classification, Sentiment Analysis, Sentiment Classification, text-classification, Text Classification

Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Prompt tuning has achieved promising results on some few-class classification tasks such as sentiment classification and natural language inference. However, manually designing lots of language prompts is cumbersome and fallible. For those auto-generated prompts, it is also expensive and time-consuming to verify their effectiveness in non-few-shot scenarios. Hence, it is still challenging for prompt tuning to address many-class classification tasks. To this end, we propose prompt tuning with rules (PTR) for many-class text classification and apply logic rules to construct prompts with several sub-prompts. In this way, PTR is able to encode prior knowledge of each class into prompt tuning. We conduct experiments on relation classification, a typical and complicated many-class classification task, and the results show that PTR can significantly and consistently outperform existing state-of-the-art baselines. This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.
Cross-lingual Text Classification with Heterogeneous Graph Neural Network
Authors: ZiYun Wang, Xuan Liu, Peiji Yang, Shixing Liu, Zhisheng Wang
Date: 2021-05-24 00:00:00
Proceeding: ACL 2021 5
Tasks: Classification, Graph Neural Network, Semantic Similarity, Semantic Textual Similarity, text-classification, Text Classification

Cross-lingual text classification aims at training a classifier on the source language and transferring the knowledge to target languages, which is very useful for low-resource languages. Recent multilingual pretrained language models (mPLM) achieve impressive results in cross-lingual classification tasks, but rarely consider factors beyond semantic similarity, causing performance degradation between some language pairs. In this paper we propose a simple yet effective method to incorporate heterogeneous information within and across languages for cross-lingual text classification using graph convolutional networks (GCN). In particular, we construct a heterogeneous graph by treating documents and words as nodes, and linking nodes with different relations, which include part-of-speech roles, semantic similarity, and document translations. Extensive experiments show that our graph-based method significantly outperforms state-of-the-art models on all tasks, and also achieves consistent performance gain over baselines in low-resource settings where external tools like translators are unavailable.
Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs
Authors: Tianming Liang, Yang Liu, Xiaoyan Liu, Hao Zhang, Gaurav Sharma, Maozu Guo
Date: 2021-05-24 00:00:00
Tasks: Denoising, Relation, Relation Extraction, Relationship Extraction (Distant Supervised), Representation Learning

Label noise and long-tailed distributions are two major challenges in distantly supervised relation extraction. Recent studies have shown great progress on denoising, but paid little attention to the problem of long-tailed relations. In this paper, we introduce a constraint graph to model the dependencies between relation labels. On top of that, we further propose a novel constraint graph-based relation extraction framework(CGRE) to handle the two challenges simultaneously. CGRE employs graph convolution networks to propagate information from data-rich relation nodes to data-poor relation nodes, and thus boosts the representation learning of long-tailed relations. To further improve the noise immunity, a constraint-aware attention module is designed in CGRE to integrate the constraint information. Extensive experimental results indicate that CGRE achieves significant improvements over the previous methods for both denoising and long-tailed relation extraction. The pre-processed datasets and source code are publicly available at https://github.com/tmliang/CGRE.
De-identification of Privacy-related Entities in Job Postings
Authors: Kristian Nørgaard Jensen, Mike Zhang, Barbara Plank
Date: 2021-05-24 00:00:00
Proceeding: NoDaLiDa 2021 5
Tasks: De-identification, Multi-Task Learning, Privacy Preserving

De-identification is the task of detecting privacy-related entities in text, such as person names, emails and contact data. It has been well-studied within the medical domain. The need for de-identification technology is increasing, as privacy-preserving data handling is in high demand in many domains. In this paper, we focus on job postings. We present JobStack, a new corpus for de-identification of personal data in job vacancies on Stackoverflow. We introduce baselines, comparing Long-Short Term Memory (LSTM) and Transformer models. To improve upon these baselines, we experiment with contextualized embeddings and distantly related auxiliary data via multi-task learning. Our results show that auxiliary data improves de-identification performance. Surprisingly, vanilla BERT turned out to be more effective than a BERT model trained on other portions of Stackoverflow.
StructuralLM: Structural Pre-training for Form Understanding
Authors: Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si
Date: 2021-05-24 00:00:00
Proceeding: ACL 2021 5
Tasks: document-image-classification, Document Image Classification, Form, image-classification, Image Classification, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, while neglecting cell-level layout information that is important for form image understanding. In this paper, we propose a new pre-training approach, StructuralLM, to jointly leverage cell and layout information from scanned documents. Specifically, we pre-train StructuralLM with two new designs to make the most of the interactions of cell and layout information: 1) each cell as a semantic unit; 2) classification of cell positions. The pre-trained StructuralLM achieves new state-of-the-art results in different types of downstream tasks, including form understanding (from 78.95 to 85.14), document visual question answering (from 72.59 to 83.94) and document image classification (from 94.43 to 96.08).
Towards Standard Criteria for human evaluation of Chatbots: A Survey
Authors: Hongru Liang, Huaqing Li
Date: 2021-05-24 00:00:00
Tasks: Diversity, Survey

Human evaluation is becoming a necessity to test the performance of Chatbots. However, off-the-shelf settings suffer the severe reliability and replication issues partly because of the extremely high diversity of criteria. It is high time to come up with standard criteria and exact definitions. To this end, we conduct a through investigation of 105 papers involving human evaluation for Chatbots. Deriving from this, we propose five standard criteria along with precise definitions.
Context-Preserving Text Simplification
Authors: Christina Niklaus, Matthias Cetto, André Freitas, Siegfried Handschuh
Date: 2021-05-24 00:00:00
Tasks: Sentence, Text Simplification

We present a context-preserving text simplification (TS) approach that recursively splits and rephrases complex English sentences into a semantic hierarchy of simplified sentences. Using a set of linguistically principled transformation patterns, input sentences are converted into a hierarchical representation in the form of core sentences and accompanying contexts that are linked via rhetorical relations. Hence, as opposed to previously proposed sentence splitting approaches, which commonly do not take into account discourse-level aspects, our TS approach preserves the semantic relationship of the decomposed constituents in the output. A comparative analysis with the annotations contained in the RST-DT shows that we are able to capture the contextual hierarchy between the split sentences with a precision of 89% and reach an average precision of 69% for the classification of the rhetorical relations that hold between them.
Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models
Authors: Jieyu Lin, Jiajie Zou, Nai Ding
Date: 2021-05-24 00:00:00
Proceeding: ACL 2021 5
Tasks: Machine Reading Comprehension, Reading Comprehension

Pre-trained language models have achieved human-level performance on many Machine Reading Comprehension (MRC) tasks, but it remains unclear whether these models truly understand language or answer questions by exploiting statistical biases in datasets. Here, we demonstrate a simple yet effective method to attack MRC models and reveal the statistical biases in these models. We apply the method to the RACE dataset, for which the answer to each MRC question is selected from 4 options. It is found that several pre-trained language models, including BERT, ALBERT, and RoBERTa, show consistent preference to some options, even when these options are irrelevant to the question. When interfered by these irrelevant options, the performance of MRC models can be reduced from human-level performance to the chance-level performance. Human readers, however, are not clearly affected by these irrelevant options. Finally, we propose an augmented training method that can greatly reduce models' statistical biases.
Unsupervised Speech Recognition
Authors: Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, Michael Auli
Date: 2021-05-24 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: speech-recognition, Speech Recognition, Unsupervised Speech Recognition

Despite rapid progress in the recent past, current speech recognition systems still require labeled training data which limits this technology to a small fraction of the languages spoken around the globe. This paper describes wav2vec-U, short for wav2vec Unsupervised, a method to train speech recognition models without any labeled data. We leverage self-supervised speech representations to segment unlabeled audio and learn a mapping from these representations to phonemes via adversarial training. The right representations are key to the success of our method. Compared to the best previous unsupervised work, wav2vec-U reduces the phoneme error rate on the TIMIT benchmark from 26.1 to 11.3. On the larger English Librispeech benchmark, wav2vec-U achieves a word error rate of 5.9 on test-other, rivaling some of the best published systems trained on 960 hours of labeled data from only two years ago. We also experiment on nine other languages, including low-resource languages such as Kyrgyz, Swahili and Tatar.
Automatic Product Ontology Extraction from Textual Reviews
Authors: Joel Oksanen, Oana Cocarascu, Francesca Toni
Date: 2021-05-23 00:00:00

Ontologies have proven beneficial in different settings that make use of textual reviews. However, manually constructing ontologies is a laborious and time-consuming process in need of automation. We propose a novel methodology for automatically extracting ontologies, in the form of meronomies, from product reviews, using a very limited amount of hand-annotated training data. We show that the ontologies generated by our method outperform hand-crafted ontologies (WordNet) and ontologies extracted by existing methods (Text2Onto and COMET) in several, diverse settings. Specifically, our generated ontologies outperform the others when evaluated by human annotators as well as on an existing Q&A dataset from Amazon. Moreover, our method is better able to generalise, in capturing knowledge about unseen products. Finally, we consider a real-world setting, showing that our method is better able to determine recommended products based on their reviews, in alternative to using Amazon's standard score aggregations.
Photonic neural field on a silicon chip: large-scale, high-speed neuro-inspired computing and sensing
Authors: Satoshi Sunada, Atsushi Uchida
Date: 2021-05-22 00:00:00
Tasks: Time Series Analysis, Time Series Prediction

Photonic neural networks have significant potential for high-speed neural processing with low latency and ultralow energy consumption. However, the on-chip implementation of a large-scale neural network is still challenging owing to its low scalability. Herein, we propose the concept of a photonic neural field and implement it experimentally on a silicon chip to realize highly scalable neuro-inspired computing. In contrast to existing photonic neural networks, the photonic neural field is a spatially continuous field that nonlinearly responds to optical inputs, and its high spatial degrees of freedom allow for large-scale and high-density neural processing on a millimeter-scale chip. In this study, we use the on-chip photonic neural field as a reservoir of information and demonstrate a high-speed chaotic time-series prediction with low errors using a training approach similar to reservoir computing. We discuss that the photonic neural field is potentially capable of executing more than one peta multiply-accumulate operations per second for a single input wavelength on a footprint as small as a few square millimeters. In addition to processing, the photonic neural field can be used for rapidly sensing the temporal variation of an optical phase, facilitated by its high sensitivity to optical inputs. The merging of optical processing with optical sensing paves the way for an end-to-end data-driven optical sensing scheme.