pwc-archive/papers-with-abstracts huggingface dataset:
TRSM-RS: A Movie Recommender System Based on Users' Gender and New Weighted Similarity Measure
Authors: Mostafa Khalaji
Date: 2020-11-10 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

With the growing data on the Internet, recommender systems have been able to predict users' preferences and offer related movies. Collaborative filtering is one of the most popular algorithms in these systems. The main purpose of collaborative filtering is to find the users or the same items using the rating matrix. By increasing the number of users and items, this algorithm suffers from the scalability problem. On the other hand, due to the unavailability of a large number of user preferences for different items, there is a cold start problem for a new user or item that has a significant impact on system performance. The purpose of this paper is to design a movie recommender system named TRSM-RS using users' demographic information (just users' gender) along with the new weighted similarity measure. By segmenting users based on their gender, the scalability problem is improved, and by considering the reliability of the users' similarity as the weight in the new similarity measure (Tanimoto Reliability Similarity Measure, TRSM), the effect of the cold-start problem is undermined and the performance of the system is improved. Experiments were performed on the MovieLens dataset and the system was evaluated using mean absolute error (MAE), Accuracy, Precision, and Recall metrics. The results of the experiments indicate improved performance (accuracy and precision) and system error rate compared to other research methods of the researchers. The maximum improved MAE rate of the system for men and women is 5.5% and 13.8%, respectively.
Boosting Search Performance Using Query Variations
Authors: Rodger Benham, Joel Mackenzie, Alistair Moffat, J. Shane Culpepper
Date: 2018-11-15 00:00:00
Tasks: Re-Ranking, Retrieval

Rank fusion is a powerful technique that allows multiple sources of information to be combined into a single result set. However, to date fusion has not been regarded as being cost-effective in cases where strict per-query efficiency guarantees are required, such as in web search. In this work we propose a novel solution to rank fusion by splitting the computation into two parts -- one phase that is carried out offline to generate pre-computed centroid answers for queries with broadly similar information needs, and then a second online phase that uses the corresponding topic centroid to compute a result page for each query. We explore efficiency improvements to classic fusion algorithms whose costs can be amortized as a pre-processing step, and can then be combined with re-ranking approaches to dramatically improve effectiveness in multi-stage retrieval systems with little efficiency overhead at query time. Experimental results using the ClueWeb12B collection and the UQV100 query variations demonstrate that centroid-based approaches allow improved retrieval effectiveness at little or no loss in query throughput or latency, and with reasonable pre-processing requirements. We additionally show that queries that do not match any of the pre-computed clusters can be accurately identified and efficiently processed in our proposed ranking pipeline.
CLAR: A Cross-Lingual Argument Regularizer for Semantic Role Labeling
Authors: Ishan Jindal, Yunyao Li, Siddhartha Brahma, Huaiyu Zhu
Date: 2020-11-09 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Semantic Role Labeling, Sentence

Semantic role labeling (SRL) identifies predicate-argument structure(s) in a given sentence. Although different languages have different argument annotations, polyglot training, the idea of training one model on multiple languages, has previously been shown to outperform monolingual baselines, especially for low resource languages. In fact, even a simple combination of data has been shown to be effective with polyglot training by representing the distant vocabularies in a shared representation space. Meanwhile, despite the dissimilarity in argument annotations between languages, certain argument labels do share common semantic meaning across languages (e.g. adjuncts have more or less similar semantic meaning across languages). To leverage such similarity in annotation space across languages, we propose a method called Cross-Lingual Argument Regularizer (CLAR). CLAR identifies such linguistic annotation similarity across languages and exploits this information to map the target language arguments using a transformation of the space on which source language arguments lie. By doing so, our experimental results show that CLAR consistently improves SRL performance on multiple languages over monolingual and polyglot baselines for low resource languages.
Adversarial Semantic Collisions
Authors: Congzheng Song, Alexander M. Rush, Vitaly Shmatikov
Date: 2020-11-09 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Extractive Summarization, Paraphrase Identification, Retrieval

We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts-- including paraphrase identification, document retrieval, response suggestion, and extractive summarization-- are vulnerable to semantic collisions. For example, given a target query, inserting a crafted collision into an irrelevant document can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/csong27/collision-bert.
EstBERT: A Pretrained Language-Specific BERT for Estonian
Authors: Hasan Tanvir, Claudia Kittask, Sandra Eiche, Kairit Sirts
Date: 2020-11-09 00:00:00
Proceeding: NoDaLiDa 2021 5
Tasks: Morphological Tagging, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), POS, text-classification, Text Classification

This paper presents EstBERT, a large pretrained transformer-based language-specific BERT model for Estonian. Recent work has evaluated multilingual BERT models on Estonian tasks and found them to outperform the baselines. Still, based on existing studies on other languages, a language-specific BERT model is expected to improve over the multilingual ones. We first describe the EstBERT pretraining process and then present the results of the models based on finetuned EstBERT for multiple NLP tasks, including POS and morphological tagging, named entity recognition and text classification. The evaluation results show that the models based on EstBERT outperform multilingual BERT models on five tasks out of six, providing further evidence towards a view that training language-specific BERT models are still useful, even when multilingual models are available.
Simultaneous Speech-to-Speech Translation System with Neural Incremental ASR, MT, and TTS
Authors: Katsuhito Sudoh, Takatomo Kano, Sashi Novitasari, Tomoya Yanagita, Sakriani Sakti, Satoshi Nakamura
Date: 2020-11-10 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, Simultaneous Speech-to-Speech Translation, speech-recognition, Speech Recognition, Speech Synthesis, Speech-to-Speech Translation, text-to-speech, Text to Speech, Text-To-Speech Synthesis, Translation

This paper presents a newly developed, simultaneous neural speech-to-speech translation system and its evaluation. The system consists of three fully-incremental neural processing modules for automatic speech recognition (ASR), machine translation (MT), and text-to-speech synthesis (TTS). We investigated its overall latency in the system's Ear-Voice Span and speaking latency along with module-level performance.
Natural Language Inference in Context -- Investigating Contextual Reasoning over Long Texts
Authors: Hanmeng Liu, Leyang Cui, Jian Liu, Yue Zhang
Date: 2020-11-10 00:00:00
Tasks: Logical Reasoning, Natural Language Inference, Sentence

Natural language inference (NLI) is a fundamental NLP task, investigating the entailment relationship between two texts. Popular NLI datasets present the task at sentence-level. While adequate for testing semantic representations, they fall short for testing contextual reasoning over long texts, which is a natural part of the human inference process. We introduce ConTRoL, a new dataset for ConTextual Reasoning over Long texts. Consisting of 8,325 expert-designed "context-hypothesis" pairs with gold labels, ConTRoL is a passage-level NLI dataset with a focus on complex contextual reasoning types such as logical reasoning. It is derived from competitive selection and recruitment test (verbal reasoning test) for police recruitment, with expert level quality. Compared with previous NLI benchmarks, the materials in ConTRoL are much more challenging, involving a range of reasoning types. Empirical results show that state-of-the-art language models perform by far worse than educated humans. Our dataset can also serve as a testing-set for downstream tasks like Checking Factual Correctness of Summaries.
A Transfer Learning Approach for Dialogue Act Classification of GitHub Issue Comments
Authors: Ayesha Enayet, Gita Sukthankar
Date: 2020-11-10 00:00:00
Tasks: Dialogue Act Classification, General Classification, Sentence, Transfer Learning

Social coding platforms, such as GitHub, serve as laboratories for studying collaborative problem solving in open source software development; a key feature is their ability to support issue reporting which is used by teams to discuss tasks and ideas. Analyzing the dialogue between team members, as expressed in issue comments, can yield important insights about the performance of virtual teams. This paper presents a transfer learning approach for performing dialogue act classification on issue comments. Since no large labeled corpus of GitHub issue comments exists, employing transfer learning enables us to leverage standard dialogue act datasets in combination with our own GitHub comment dataset. We compare the performance of several word and sentence level encoding models including Global Vectors for Word Representations (GloVe), Universal Sentence Encoder (USE), and Bidirectional Encoder Representations from Transformers (BERT). Being able to map the issue comments to dialogue acts is a useful stepping stone towards understanding cognitive team processes.
On the Usefulness of Self-Attention for Automatic Speech Recognition with Transformers
Authors: Shucong Zhang, Erfan Loweimi, Peter Bell, Steve Renals
Date: 2020-11-08 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Self-attention models such as Transformers, which can capture temporal relationships without being limited by the distance between events, have given competitive speech recognition results. However, we note the range of the learned context increases from the lower to upper self-attention layers, whilst acoustic events often happen within short time spans in a left-to-right order. This leads to a question: for speech recognition, is a global view of the entire sequence useful for the upper self-attention encoder layers in Transformers? To investigate this, we train models with lower self-attention/upper feed-forward layers encoders on Wall Street Journal and Switchboard. Compared to baseline Transformers, no performance drop but minor gains are observed. We further developed a novel metric of the diagonality of attention matrices and found the learned diagonality indeed increases from the lower to upper encoder self-attention layers. We conclude the global view is unnecessary in training upper encoder layers.
When Do You Need Billions of Words of Pretraining Data?
Authors: Yian Zhang, Alex Warstadt, Haau-Sing Li, Samuel R. Bowman
Date: 2020-11-10 00:00:00
Proceeding: ACL 2021 5

NLP is currently dominated by general-purpose pretrained language models like RoBERTa, which achieve strong performance on NLU tasks through pretraining on billions of words. But what exact knowledge or skills do Transformer LMs learn from large-scale pretraining that they cannot learn from less data? We adopt four probing methods---classifier probing, information-theoretic probing, unsupervised relative acceptability judgment, and fine-tuning on NLU tasks---and draw learning curves that track the growth of these different measures of linguistic ability with respect to pretraining data volume using the MiniBERTas, a group of RoBERTa models pretrained on 1M, 10M, 100M and 1B words. We find that LMs require only about 10M or 100M words to learn representations that reliably encode most syntactic and semantic features we test. A much larger quantity of data is needed in order to acquire enough commonsense knowledge and other skills required to master typical downstream NLU tasks. The results suggest that, while the ability to encode linguistic features is almost certainly necessary for language understanding, it is likely that other forms of knowledge are the major drivers of recent improvements in language understanding among large pretrained models.
Does Social Support Expressed in Post Titles Elicit Comments in Online Substance Use Recovery Forums?
Authors: Anietie Andy, Sharath Guntuku
Date: 2020-11-10 00:00:00

Individuals recovering from substance use often seek social support (emotional and informational) on online recovery forums, where they can both write and comment on posts, expressing their struggles and successes. A common challenge in these forums is that certain posts (some of which may be support seeking) receive no comments. In this work, we use data from two Reddit substance recovery forums:/r/Leaves and/r/OpiatesRecovery, to determine the relationship between the social supports expressed in the titles of posts and the number of comments they receive. We show that the types of social support expressed in post titles that elicit comments vary from one substance use recovery forum to the other.
Multi-Task Sequence Prediction For Tunisian Arabizi Multi-Level Annotation
Authors: Elisa Gugliotta, Marco Dinarelli, Olivier Kraif
Date: 2020-11-10 00:00:00
Proceeding: COLING (WANLP) 2020 12
Tasks: POS, POS Tagging, text-classification, Text Classification

In this paper we propose a multi-task sequence prediction system, based on recurrent neural networks and used to annotate on multiple levels an Arabizi Tunisian corpus. The annotation performed are text classification, tokenization, PoS tagging and encoding of Tunisian Arabizi into CODA* Arabic orthography. The system is learned to predict all the annotation levels in cascade, starting from Arabizi input. We evaluate the system on the TIGER German corpus, suitably converting data to have a multi-task problem, in order to show the effectiveness of our neural architecture. We show also how we used the system in order to annotate a Tunisian Arabizi corpus, which has been afterwards manually corrected and used to further evaluate sequence models on Tunisian data. Our system is developed for the Fairseq framework, which allows for a fast and easy use for any other sequence prediction problem.
On the State of Social Media Data for Mental Health Research
Authors: Keith Harrigian, Carlos Aguirre, Mark Dredze
Date: 2020-11-10 00:00:00
Proceeding: NAACL (CLPsych) 2021 6

Data-driven methods for mental health treatment and surveillance have become a major focus in computational science research in the last decade. However, progress in the domain, in terms of both medical understanding and system performance, remains bounded by the availability of adequate data. Prior systematic reviews have not necessarily made it possible to measure the degree to which data-related challenges have affected research progress. In this paper, we offer an analysis specifically on the state of social media data that exists for conducting mental health research. We do so by introducing an open-source directory of mental health datasets, annotated using a standardized schema to facilitate meta-analysis.
Towards Preemptive Detection of Depression and Anxiety in Twitter
Authors: David Owen, Jose Camacho Collados, Luis Espinosa-Anke
Date: 2020-11-10 00:00:00
Proceeding: SMM4H (COLING) 2020 12
Tasks: Anxiety Detection

Depression and anxiety are psychiatric disorders that are observed in many areas of everyday life. For example, these disorders manifest themselves somewhat frequently in texts written by nondiagnosed users in social media. However, detecting users with these conditions is not a straightforward task as they may not explicitly talk about their mental state, and if they do, contextual cues such as immediacy must be taken into account. When available, linguistic flags pointing to probable anxiety or depression could be used by medical experts to write better guidelines and treatments. In this paper, we develop a dataset designed to foster research in depression and anxiety detection in Twitter, framing the detection task as a binary tweet classification problem. We then apply state-of-the-art classification models to this dataset, providing a competitive set of baselines alongside qualitative error analysis. Our results show that language models perform reasonably well, and better than more traditional baselines. Nonetheless, there is clear room for improvement, particularly with unbalanced training sets and in cases where seemingly obvious linguistic cues (keywords) are used counter-intuitively.
Medical Knowledge-enriched Textual Entailment Framework
Authors: Shweta Yadav, Vishal Pallagani, Amit Sheth
Date: 2020-11-10 00:00:00
Proceeding: COLING 2020 8
Tasks: Data Augmentation, Medical Question Answering, Natural Language Inference, Question Answering, Sentence

One of the cardinal tasks in achieving robust medical question answering systems is textual entailment. The existing approaches make use of an ensemble of pre-trained language models or data augmentation, often to clock higher numbers on the validation metrics. However, two major shortcomings impede higher success in identifying entailment: (1) understanding the focus/intent of the question and (2) ability to utilize the real-world background knowledge to capture the context beyond the sentence. In this paper, we present a novel Medical Knowledge-Enriched Textual Entailment framework that allows the model to acquire a semantic and global representation of the input medical text with the help of a relevant domain-specific knowledge graph. We evaluate our framework on the benchmark MEDIQA-RQE dataset and manifest that the use of knowledge enriched dual-encoding mechanism help in achieving an absolute improvement of 8.27% over SOTA language models. We have made the source code available here.
Neural Machine Translation for Extremely Low-Resource African Languages: A Case Study on Bambara
Authors: Allahsera Auguste Tapo, Bakary Coulibaly, Sébastien Diarra, Christopher Homan, Julia Kreutzer, Sarah Luger, Arthur Nagashima, Marcos Zampieri, Michael Leventhal
Date: 2020-11-10 00:00:00
Proceeding: loresmt (AACL) 2020 12
Tasks: Machine Translation, Translation

Low-resource languages present unique challenges to (neural) machine translation. We discuss the case of Bambara, a Mande language for which training data is scarce and requires significant amounts of pre-processing. More than the linguistic situation of Bambara itself, the socio-cultural context within which Bambara speakers live poses challenges for automated processing of this language. In this paper, we present the first parallel data set for machine translation of Bambara into and from English and French and the first benchmark results on machine translation to and from Bambara. We discuss challenges in working with low-resource languages and propose strategies to cope with data scarcity in low-resource machine translation (MT).
DoLFIn: Distributions over Latent Features for Interpretability
Authors: Phong Le, Willem Zuidema
Date: 2020-11-10 00:00:00
Proceeding: COLING 2020 8
Tasks: text-classification, Text Classification

Interpreting the inner workings of neural models is a key step in ensuring the robustness and trustworthiness of the models, but work on neural network interpretability typically faces a trade-off: either the models are too constrained to be very useful, or the solutions found by the models are too complex to interpret. We propose a novel strategy for achieving interpretability that -- in our experiments -- avoids this trade-off. Our approach builds on the success of using probability as the central quantity, such as for instance within the attention mechanism. In our architecture, DoLFIn (Distributions over Latent Features for Interpretability), we do no determine beforehand what each feature represents, and features go altogether into an unordered set. Each feature has an associated probability ranging from 0 to 1, weighing its importance for further processing. We show that, unlike attention and saliency map approaches, this set-up makes it straight-forward to compute the probability with which an input component supports the decision the neural model makes. To demonstrate the usefulness of the approach, we apply DoLFIn to text classification, and show that DoLFIn not only provides interpretable solutions, but even slightly outperforms the classical CNN and BiLSTM text classifiers on the SST2 and AG-news datasets.
Speaker De-identification System using Autoencodersand Adversarial Training
Authors: Fernando M. Espinoza-Cuadros, Juan M. Perero-Codosero, Javier Antón-Martín, Luis A. Hernández-Gómez
Date: 2020-11-09 00:00:00
Tasks: De-identification, Speaker Verification

The fast increase of web services and mobile apps, which collect personal data from users, increases the risk that their privacy may be severely compromised. In particular, the increasing variety of spoken language interfaces and voice assistants empowered by the vertiginous breakthroughs in Deep Learning are prompting important concerns in the European Union to preserve speech data privacy. For instance, an attacker can record speech from users and impersonate them to get access to systems requiring voice identification. Hacking speaker profiles from users is also possible by means of existing technology to extract speaker, linguistic (e.g., dialect) and paralinguistic features (e.g., age) from the speech signal. In order to mitigate these weaknesses, in this paper, we propose a speaker de-identification system based on adversarial training and autoencoders in order to suppress speaker, gender, and accent information from speech. Experimental results show that combining adversarial learning and autoencoders increase the equal error rate of a speaker verification system while preserving the intelligibility of the anonymized spoken content.
Pretraining Strategies, Waveform Model Choice, and Acoustic Configurations for Multi-Speaker End-to-End Speech Synthesis
Authors: Erica Cooper, Xin Wang, Yi Zhao, Yusuke Yasuda, Junichi Yamagishi
Date: 2020-11-10 00:00:00
Tasks: Speech Synthesis

We explore pretraining strategies including choice of base corpus with the aim of choosing the best strategy for zero-shot multi-speaker end-to-end synthesis. We also examine choice of neural vocoder for waveform synthesis, as well as acoustic configurations used for mel spectrograms and final audio output. We find that fine-tuning a multi-speaker model from found audiobook data that has passed a simple quality threshold can improve naturalness and similarity to unseen target speakers of synthetic speech. Additionally, we find that listeners can discern between a 16kHz and 24kHz sampling rate, and that WaveRNN produces output waveforms of a comparable quality to WaveNet, with a faster inference time.