pwc-archive/papers-with-abstracts huggingface dataset:
Controllable Sentence Simplification with a Unified Text-to-Text Transfer Transformer
Authors: Kim Cheng SHEANG, Horacio Saggion
Date: 2021-08-01 00:00:00
Proceeding: INLG (ACL) 2021 8
Tasks: Language Modeling, Language Modelling, Sentence, Text Simplification

Recently, a large pre-trained language model called T5 (A Unified Text-to-Text Transfer Transformer) has achieved state-of-the-art performance in many NLP tasks. However, no study has been found using this pre-trained model on Text Simplification. Therefore in this paper, we explore the use of T5 fine-tuning on Text Simplification combining with a controllable mechanism to regulate the system outputs that can help generate adapted text for different target audiences. Our experiments show that our model achieves remarkable results with gains of between +0.69 and +1.41 over the current state-of-the-art (BART+ACCESS). We argue that using a pre-trained model such as T5, trained on several tasks with large amounts of data, can help improve Text Simplification.
SEPRG: Sentiment aware Emotion controlled Personalized Response Generation
Authors: Mauajama Firdaus, Umang Jain, Asif Ekbal, Pushpak Bhattacharyya
Date: 2021-08-01 00:00:00
Proceeding: INLG (ACL) 2021 8
Tasks: Chatbot, Dialogue Generation, Response Generation

Social chatbots have gained immense popularity, and their appeal lies not just in their capacity to respond to the diverse requests from users, but also in the ability to develop an emotional connection with users. To further develop and promote social chatbots, we need to concentrate on increasing user interaction and take into account both the intellectual and emotional quotient in the conversational agents. Therefore, in this work, we propose the task of sentiment aware emotion controlled personalized dialogue generation giving the machine the capability to respond emotionally and in accordance with the persona of the user. As sentiment and emotions are highly co-related, we use the sentiment knowledge of the previous utterance to generate the correct emotional response in accordance with the user persona. We design a Transformer based Dialogue Generation framework, that generates responses that are sensitive to the emotion of the user and corresponds to the persona and sentiment as well. Moreover, the persona information is encoded by a different Transformer encoder, along with the dialogue history, is fed to the decoder for generating responses. We annotate the PersonaChat dataset with sentiment information to improve the response quality. Experimental results on the PersonaChat dataset show that the proposed framework significantly outperforms the existing baselines, thereby generating personalized emotional responses in accordance with the sentiment that provides better emotional connection and user satisfaction as desired in a social chatbot.
Decoding, Fast and Slow: A Case Study on Balancing Trade-Offs in Incremental, Character-level Pragmatic Reasoning
Authors: Sina Zarrieß, Hendrik Buschmeier, Ting Han, Simeon Schüz
Date: 2021-08-01 00:00:00
Proceeding: INLG (ACL) 2021 8
Tasks: Image Captioning, Informativeness, Language Modeling, Language Modelling, Rolling Shutter Correction

Recent work has adopted models of pragmatic reasoning for the generation of informative language in, e.g., image captioning. We propose a simple but highly effective relaxation of fully rational decoding, based on an existing incremental and character-level approach to pragmatically informative neural image captioning. We implement a mixed, ‘fast’ and ‘slow’, speaker that applies pragmatic reasoning occasionally (only word-initially), while unrolling the language model. In our evaluation, we find that increased informativeness through pragmatic decoding generally lowers quality and, somewhat counter-intuitively, increases repetitiveness in captions. Our mixed speaker, however, achieves a good balance between quality and informativeness.
Exploring Structural Encoding for Data-to-Text Generation
Authors: Joy Mahapatra, Utpal Garain
Date: 2021-08-01 00:00:00
Proceeding: INLG (ACL) 2021 8
Tasks: Data-to-Text Generation, Decoder, Text Generation

Due to efficient end-to-end training and fluency in generated texts, several encoder-decoder framework-based models are recently proposed for data-to-text generations. Appropriate encoding of input data is a crucial part of such encoder-decoder models. However, only a few research works have concentrated on proper encoding methods. This paper presents a novel encoder-decoder based data-to-text generation model where the proposed encoder carefully encodes input data according to underlying structure of the data. The effectiveness of the proposed encoder is evaluated both extrinsically and intrinsically by shuffling input data without changing meaning of that data. For selecting appropriate content information in encoded data from encoder, the proposed model incorporates attention gates in the decoder. With extensive experiments on WikiBio and E2E dataset, we show that our model outperforms the state-of-the models and several standard baseline systems. Analysis of the model through component ablation tests and human evaluation endorse the proposed model as a well-grounded system.
A Framework to Assist Chat Operators of Mental Healthcare Services
Authors: Thiago Madeira, Heder Bernardino, Jairo Francisco De Souza, Henrique Gomide, Nathália Munck Machado, Bruno Marcos Pinheiro da Silva, Alexandre Vieira Pereira Pacelli
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11

Conversational agents can be used to make diagnoses, classify mental states, promote health education, and provide emotional support. The benefits of adopting conversational agents include widespread access, increased treatment engagement, and improved patient relationships with the intervention. We propose here a framework to assist chat operators of mental healthcare services, instead of a fully automated conversational agent. This design eases to avoid the adverse effects of applying chatbots in mental healthcare. The proposed framework is capable of improving the quality and reducing the time of interactions via chat between a user and a chat operator. We also present a case study in the context of health promotion on reducing tobacco use. The proposed framework uses artificial intelligence, specifically natural language processing (NLP) techniques, to classify messages from chat users. A list of suggestions is offered to the chat operator, with topics to be discussed in the session. These suggestions were created based on service protocols and the classification of previous chat sessions. The operator can also edit the suggested messages. Data collected can be used in the future to improve the quality of the suggestions offered.
ARBML: Democritizing Arabic Natural Language Processing Tools
Authors: Zaid Alyafeai, Maged Al-Shaibani
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: Natural Language Understanding

Automating natural language understanding is a lifelong quest addressed for decades. With the help of advances in machine learning and particularly, deep learning, we are able to produce state of the art models that can imitate human interactions with languages. Unfortunately, these advances are controlled by the availability of language resources. Arabic advances in this field , although it has a great potential, are still limited. This is apparent in both research and development. In this paper, we showcase some NLP models we trained for Arabic. We also present our methodology and pipeline to build such models from data collection, data preprocessing, tokenization and model deployment. These tools help in the advancement of the field and provide a systematic approach for extending NLP tools to many languages.
End-to-end NLP Pipelines in Rust
Authors: Guillaume Becquin
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11

The recent progress in natural language processing research has been supported by the development of a rich open source ecosystem in Python. Libraries allowing NLP practitioners but also non-specialists to leverage state-of-the-art models have been instrumental in the democratization of this technology. The maturity of the open-source NLP ecosystem however varies between languages. This work proposes a new open-source library aimed at bringing state-of-the-art NLP to Rust. Rust is a systems programming language for which the foundations required to build machine learning applications are available but still lacks ready-to-use, end-to-end NLP libraries. The proposed library, rust-bert, implements modern language models and ready-to-use pipelines (for example translation or summarization). This allows further development by the Rust community from both NLP experts and non-specialists. It is hoped that this library will accelerate the development of the NLP ecosystem in Rust. The library is under active development and available at https://github.com/guillaume-be/rust-bert.
Going Beyond T-SNE: Exposing whatlies in Text Embeddings
Authors: Vincent Warmerdam, Thomas Kober, Rachael Tatman
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: Dimensionality Reduction, Sentence, Sentence Embeddings, Word Embeddings

We introduce whatlies, an open source toolkit for visually inspecting word and sentence embeddings. The project offers a unified and extensible API with current support for a range of popular embedding backends including spaCy, tfhub, huggingface transformers, gensim, fastText and BytePair embeddings. The package combines a domain specific language for vector arithmetic with visualisation tools that make exploring word embeddings more intuitive and concise. It offers support for many popular dimensionality reduction techniques as well as many interactive visualisations that can either be statically exported or shared via Jupyter notebooks. The project documentation is available from https://rasahq.github.io/whatlies/.
KLPT – Kurdish Language Processing Toolkit
Authors: Sina Ahmadi
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: Diversity, Lemmatization, Transliteration

Despite the recent advances in applying language-independent approaches to various natural language processing tasks thanks to artificial intelligence, some language-specific tools are still essential to process a language in a viable manner. Kurdish language is a less-resourced language with a remarkable diversity in dialects and scripts and lacks basic language processing tools. To address this issue, we introduce a language processing toolkit to handle such a diversity in an efficient way. Our toolkit is composed of fundamental components such as text preprocessing, stemming, tokenization, lemmatization and transliteration and is able to get further extended by future developers. The project is publicly available.
Pimlico: A toolkit for corpus-processing pipelines and reproducible experiments
Authors: Mark Granroth-Wilding
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11

We present Pimlico, an open source toolkit for building pipelines for processing large corpora. It is especially focused on processing linguistic corpora and provides wrappers around existing, widely used NLP tools. A particular goal is to ease distribution of reproducible and extensible experiments by making it easy to document and re-run all steps involved, including data loading, pre-processing, model training and evaluation. Once a pipeline is released, it is easy to adapt, for example, to run on a new dataset, or to re-run an experiment with different parameters. The toolkit takes care of many common challenges in writing and distributing corpus-processing code, such as managing data between the steps of a pipeline, installing required software and combining existing toolkits with new, task-specific code.
iobes: Library for Span Level Processing
Authors: Brian Lester
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), slot-filling, Slot Filling

Many tasks in natural language processing, such as named entity recognition and slot-filling, involve identifying and labeling specific spans of text. In order to leverage common models, these tasks are often recast as sequence labeling tasks. Each token is given a label and these labels are prefixed with special tokens such as B- or I-. After a model assigns labels to each token, these prefixes are used to group the tokens into spans. Properly parsing these annotations is critical for producing fair and comparable metrics; however, despite its importance, there is not an easy-to-use, standardized, programmatically integratable library to help work with span labeling. To remedy this, we introduce our open-source library, iobes. iobes is used for parsing, converting, and processing spans represented as token level decisions.
TextAttack: Lessons learned in designing Python frameworks for NLP
Authors: John Morris, Jin Yong Yoo, Yanjun Qi
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: Adversarial Attack, Data Augmentation

TextAttack is an open-source Python toolkit for adversarial attacks, adversarial training, and data augmentation in NLP. TextAttack unites 15+ papers from the NLP adversarial attack literature into a single framework, with many components reused across attacks. This framework allows both researchers and developers to test and study the weaknesses of their NLP models. To build such an open-source NLP toolkit requires solving some common problems: How do we enable users to supply models from different deep learning frameworks? How can we build tools to support as many different datasets as possible? We share our insights into developing a well-written, well-documented NLP Python framework in hope that they can aid future development of similar packages.
TOMODAPI: A Topic Modeling API to Train, Use and Compare Topic Models
Authors: Pasquale Lisena, Ismail Harrando, Oussama Kandakji, Raphael Troncy
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: Topic Models

From LDA to neural models, different topic modeling approaches have been proposed in the literature. However, their suitability and performance is not easy to compare, particularly when the algorithms are being used in the wild on heterogeneous datasets. In this paper, we introduce ToModAPI (TOpic MOdeling API), a wrapper library to easily train, evaluate and infer using different topic modeling algorithms through a unified interface. The library is extensible and can be used in Python environments or through a Web API.
User-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool
Authors: Nitin Madnani, Anastassia Loukina
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11

For the last 5 years, we have developed and maintained RSMTool – an open-source tool for evaluating NLP systems that automatically score written and spoken responses. RSMTool is designed to be cross-disciplinary, borrowing heavily from NLP, machine learning, and educational measurement. Its cross-disciplinary nature has required us to learn a user-centered development approach in terms of both design and implementation. We share some of these lessons in this paper.
WAFFLE: A Graph for WordNet Applied to FreeForm Linguistic Exploration
Authors: Berk Ekmekci, Blake Howald
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11

The WordNet database of English (Fellbaum, 1998) is a key source of semantic information for research and development of natural language processing applications. As the sophistication of these applications increases with the use of large datasets, deep learning, and graph-based methods, so should the use of WordNet. To this end, we introduce WAFFLE: WordNet Applied to FreeForm Linguistic Exploration which makes WordNet available in an open source graph data structure. The WAFFLE graph relies on platform agnostic formats for robust interrogation and flexibility. Where existing implementations of WordNet offer dictionary-like lookup, single degree neighborhood operations, and path based similarity-scoring, the WAFFLE graph makes all nodes (semantic relation sets) and relationships queryable at scale, enabling local and global analysis of all relationships without the need for custom code. We demonstrate WAFFLE’s ease of use, visualization capabilities, and scalable efficiency with common queries, operations, and interactions. WAFFLE is available at github.com/TRSS-NLP/WAFFLE.
Comparison between NMT and PBSMT Performance for Translating Noisy User-Generated Content
Authors: José Carlos Rosales Núñez, Djamé Seddah, Guillaume Wisniewski
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9
Tasks: Machine Translation, NMT, Translation

This work compares the performances achieved by Phrase-Based Statistical Machine Translation systems (PB-SMT) and attention-based Neuronal Machine Translation systems (NMT) when translating User Generated Content (UGC), as encountered in social medias, from French to English. We show that, contrary to what could be expected, PBSMT outperforms NMT when translating non-canonical inputs. Our error analysis uncovers the specificities of UGC that are problematic for sequential NMT architectures and suggests new avenue for improving NMT models.
Bootstrapping UD treebanks for Delexicalized Parsing
Authors: Prasanth Kolachina, Aarne Ranta
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9

Standard approaches to treebanking traditionally employ a waterfall model (Sommerville, 2010), where annotation guidelines guide the annotation process and insights from the annotation process in turn lead to subsequent changes in the annotation guidelines. This process remains a very expensive step in creating linguistic resources for a target language, necessitates both linguistic expertise and manual effort to develop the annotations and is subject to inconsistencies in the annotation due to human errors. In this paper, we propose an alternative approach to treebanking—one that requires writing grammars. This approach is motivated specifically in the context of Universal Dependencies, an effort to develop uniform and cross-lingually consistent treebanks across multiple languages. We show here that a bootstrapping approach to treebanking via interlingual grammars is plausible and useful in a process where grammar engineering and treebanking are jointly pursued when creating resources for the target language. We demonstrate the usefulness of synthetic treebanks in the task of delexicalized parsing. Our experiments reveal that simple models for treebank generation are cheaper than human annotated treebanks, especially in the lower ends of the learning curves for delexicalized parsing, which is relevant in particular in the context of low-resource languages.
Lexical Resources for Low-Resource PoS Tagging in Neural Times
Authors: Barbara Plank, Sigrid Klerke
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9
Tasks: Cross-Lingual POS Tagging, POS, POS Tagging

More and more evidence is appearing that integrating symbolic lexical knowledge into neural models aids learning. This contrasts the widely-held belief that neural networks largely learn their own feature representations. For example, recent work has shows benefits of integrating lexicons to aid cross-lingual part-of-speech (PoS). However, little is known on how complementary such additional information is, and to what extent improvements depend on the coverage and quality of these external resources. This paper seeks to fill this gap by providing a thorough analysis on the contributions of lexical resources for cross-lingual PoS tagging in neural times.
Gender Bias in Pretrained Swedish Embeddings
Authors: Magnus Sahlgren, Fredrik Olsson
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9

This paper investigates the presence of gender bias in pretrained Swedish embeddings. We focus on a scenario where names are matched with occupations, and we demonstrate how a number of standard pretrained embeddings handle this task. Our experiments show some significant differences between the pretrained embeddings, with word-based methods showing the most bias and contextualized language models showing the least. We also demonstrate that the previously proposed debiasing method does not affect the performance of the various embeddings in this scenario.
A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics
Authors: Astrid van Aggelen, Antske Fokkens, Laura Hollink, Jacco van Ossenbruggen
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9

Determining how words have changed their meaning is an important topic in Natural Language Processing. However, evaluations of methods to characterise such change have been limited to small, handcrafted resources. We introduce an English evaluation set which is larger, more varied, and more realistic than seen to date, with terms derived from a historical thesaurus. Moreover, the dataset is unique in that it represents change as a shift from the term of interest to a WordNet synset. Using the synset lemmas, we can use this set to evaluate (standard) methods that detect change between word pairs, as well as (adapted) methods that detect the change between a term and a sense overall. We show that performance on the new data set is much lower than earlier reported findings, setting a new standard.
Some steps towards the generation of diachronic WordNets
Authors: Yuri Bizzoni, Marius Mosbach, Dietrich Klakow, Stefania Degaetano-Ortlieb
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9

We apply hyperbolic embeddings to trace the dynamics of change of conceptual-semantic relationships in a large diachronic scientific corpus (200 years). Our focus is on emerging scientific fields and the increasingly specialized terminology establishing around them. Reproducing high-quality hierarchical structures such as WordNet on a diachronic scale is a very difficult task. Hyperbolic embeddings can map partial graphs into low dimensional, continuous hierarchical spaces, making more explicit the latent structure of the input. We show that starting from simple lists of word pairs (rather than a list of entities with directional links) it is possible to build diachronic hierarchical semantic spaces which allow us to model a process towards specialization for selected scientific fields.
An evaluation of Czech word embeddings
Authors: Karolína Hořeňovská
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9
Tasks: Word Embeddings

We present an evaluation of Czech low-dimensional distributed word representations, also known as word embeddings. We describe five different approaches to training the models and three different corpora used in training. We evaluate the resulting models on five different datasets, report the results and provide their further analysis.
Language Modeling with Syntactic and Semantic Representation for Sentence Acceptability Predictions
Authors: Adam Ek, Jean-Philippe Bernardy, Shalom Lappin
Date: 2019-09-01 00:00:00
Proceeding: WS (NoDaLiDa) 2019 9
Tasks: Language Modeling, Language Modelling, Sentence

In this paper, we investigate the effect of enhancing lexical embeddings in LSTM language models (LM) with syntactic and semantic representations. We evaluate the language models using perplexity, and we evaluate the performance of the models on the task of predicting human sentence acceptability judgments. We train LSTM language models on sentences automatically annotated with universal syntactic dependency roles (Nivre, 2016), dependency depth and universal semantic tags (Abzianidze et al., 2017) to predict sentence acceptability judgments. Our experiments indicate that syntactic tags lower perplexity, while semantic tags increase it. Our experiments also show that neither syntactic nor semantic tags improve the performance of LSTM language models on the task of predicting sentence acceptability judgments.