Comparative Analysis of Transformers to Support Fine-Grained Emotion Detection in Short-Text Data

RH Frye, DC Wilson - The International FLAIRS Conference Proceedings, 2022

… A clean derivation of the CommonCrawl Corpus was used in pretraining XLM-R, with one English version and twelve versions inclusive of other languages. Conneau et al. reported 95.0% accuracy on the SST binary sentiment classification task. Clark …


Comparative Evaluation of Transformer-Based Nepali Language Models

… OSCAR (Open Super-large Crawled Aggregated coRpus) is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the Ungoliant architecture, a high-performance pipeline that provides …


Comparative Study of Classification Algorithms for Website Phishing Detection on Multiple Datasets

W Sarasjati, S Rustad, HA Santoso, A Syukur… - 2022 International Seminar …, 2022

… The legitimate sites are collected from Alexa and Common Crawl, whereas the phishing sites are collected from PhishTank and OpenPhish. Mendeley dataset is different from UCI dataset because it has binary labels for classes. For example, 0 …


Comparative Study of Marathi Text Classification Using Monolingual and Multilingual Embeddings

F Eranpurwala - … Network Technologies and Intelligent Computing: First …, 2022

… It was trained on Common Crawl and Wikipedia dataset using the CBOW model with position-weighted vocabulary of size 99882. IndicBERT was trained on ALBERT [18] using standard masked language model with vocabulary 200K and …


Comparative study of text representation and learning for Persian named entity recognition

MM Abdollah Pour, S Momtazi - ETRI Journal, 2022

… XLM-R [6] is a multilingual language model trained using a large corpus of CommonCrawl data from 100 languages. XLM-R has been shown to significantly outperform previous multilingual models, for example, the mBERT and XLM models. In addition …


Comparing Different Approaches for Detecting Hate Speech in Online Portuguese Comments

BC Matos, RB Santos, P Carvalho, R Ribeiro, F Batista - 11th Symposium on …, 2022

Online Hate Speech (OHS) has been growing dramatically on social media, which has motivated researchers to develop a diversity of methods for its automated detection. However, the detection of OHS in Portuguese is still little studied. To fill …


Comparing Emotion and Sentiment Analysis Tools on Italian anti-vaccination for COVID-19 posts

E Bellodi, A Bertagnon, M Gavanelli - Proceedings of the Sixth Workshop on Natural …, 2022

Since the beginning of the vaccination campaign against Covid-19 in our country, resistance to vaccination has emerged on the part of a not negligible portion of the Italian population. Emotions (such as sadness, fear, etc.) and the polarity (positive/negative) …


Comparison of Different Modeling Techniques for Flemish Twitter Sentiment Analysis

M Reusens, M Reusens, M Callens, S vanden Broucke… - Analytics, 2022

… The FastText approach, on the other hand, used Wikipedia and Common Crawl as training data. Common Crawl is a nonprofit company that crawls the web and then makes the obtained data publicly available [34]. It is possible that because our …


Comparison of fuzzy sets based on the concept of imprecision

J Miliauskaitė, D Kalibatienė - DAMSS 2022: 13th conference on data analysis …, 2022

DAMSS-2022 is the 13th International Conference on Data Analysis Methods for Software Systems, held in Druskininkai, Lithuania. Every year at the same place and time. The exception was in 2020, when the world was gripped by the Covid-19 …


Comparison of Named Entity Recognition Methods on Real-World and Highly Imbalanced Business Document Datasets

JC Vásquez-Correa, JR Orozco-Arroyave - Applied Computer Sciences in …, 2022

SA Moreno-Acevedo, D Escobar-Grisales… - Workshop on Engineering …, 2022

… RoBERTa was pre-trained with 2.5 TB of data from the cleaned Common Crawl corpus [18], which includes text data in 100 different languages and allows for computing text representations in multiple languages. Embeddings from RoBERTa …


Comparison of Semantic Similarity Models on Constrained Scenarios

R Teixeira, M Antunes, D Gomes, RL Aguiar - Information Systems Frontiers, 2022

The technological world has grown by incorporating billions of small sensing devices, collecting and sharing large amounts of diversified data over the new generation of wireless and mobile networks. We can use semantic similarity models to help …


Comparison of Text Mining Models for Food and Dietary Constituent Named-Entity Recognition

N Perera, TTL Nguyen, M Dehmer, F Emmert-Streib - Machine Learning and …, 2022

Biomedical Named-Entity Recognition (BioNER) has become an essential part of text mining due to the continuously increasing digital archives of biological and medical articles. While there are many well-performing BioNER tools for entities …


Comparison of text preprocessing methods

CP Chai - Natural Language Engineering, 2022

Text preprocessing is not only an essential step to prepare the corpus for modeling but also a key area that directly affects the natural language processing (NLP) application results. For instance, precise tokenization increases the accuracy of part-of-speech …


Compilation and evaluation of the Spanish SatiCorpus 2021 for satire identification using linguistic features and transformers

JA García-Díaz, R Valencia-García - Complex & Intelligent Systems, 2022

Satirical content on social media is hard to distinguish from real news, misinformation, hoaxes or propaganda when there are no clues as to which medium these news were originally written in. It is important, therefore, to provide Information Retrieval …


Complex Word Identification for Language Learners

A Srivastava, C Biemann, Ö Alaçam, X Wang - 2022

… In my experiment, I used the pre-trained English word vectors from FastText, which contains 2 million word vectors with subword information on Common Crawl (600B tokens). Table 6.3 displays the results achieved by training the sequential model …


Comprehensive Survey: Approaches to Emerging Technologies Detection within Scientific Publications

A Yelenov, AA Pak, AA A Ziyaden, I Akhmetov… - Computación y Sistemas, 2022

The identification of breakthrough topics and emerging technologies has been of interest to the governments of many countries and the scientific community since the last century. This study presents the status and trend of the research field through a …


Compressed FastText Models for Czech Tagger

Z Nevěřilová - RASLAN 2022 Recent Advances in Slavonic Natural …, 2022

… Instead of CommonCrawl, we decided to use the SYN corpus, version 9. The SYN corpus [9] … The SYN corpus is therefore about half of the size of the 2017 CommonCrawl used in the … This is surprising since we supposed the training data …


COMPS: Conceptual Minimal Pair Sentences for testing Property Knowledge and Inheritance in Pre-trained Language Models

K Misra, JT Rayz, A Ettinger - arXiv preprint arXiv:2210.01963, 2022

A characteristic feature of human semantic memory is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of properties (can breathe) from superordinate concepts (animal) to …


COMPUTATIONAL INFERENCE OF TRUSTWORTHINESS IN SOCIAL FIGURES THROUGH ANALYSIS OF SPEECH ACOUSTIC, TEXTUAL, AND VISUAL SIGNALS

This chapter provides a general introduction to the field of Social Signal Processing (SSP) and its application areas. It commences by providing a brief description of motivation for this research and use of SSP for behavioural analysis, presenting the research …


Computer Vision based Automated Spare Part Finder App

G Jugroop, P Seegolum, A Chiniah, BB Humaïra - 2022 International Conference on …, 2022

The “Automated car spare part finder app” aims at building a spare part detection mobile application using image recognition technique that will assist users in finding desired spare parts and provide a list of websites where those spare parts can be …


Conceptual framework of data science for good squad

D Krikščiūnienė, V Sakalauskas, G Romeika - DAMSS 2022: 13th conference on data …, 2022


Conditional Generation with a Question-Answering Blueprint

S Narayan, J Maynez, RK Amplayo, K Ganchev… - arXiv preprint arXiv …, 2022

The ability to convey relevant and faithful information is critical for many tasks in conditional generation and yet remains elusive for neural seq-to-seq models whose outputs often reveal hallucinations and fail to correctly cover important details. In this …


Conditional Supervised Contrastive Learning for Fair Text Classification

J Chi, W Shand, Y Yu, KW Chang, H Zhao, Y Tian - arXiv preprint arXiv:2205.11485, 2022

… The data consist of nearly 400,000 online biographies collected from the Common Crawl corpus. These biographies are annotated with one of the 28 professions to which their subject belongs. The data are mapped to a binary gender …


Considerations for Multilingual Wikipedia Research

I Johnson, E Lescak - arXiv preprint arXiv:2204.02483, 2022

English Wikipedia has long been an important data source for much research and natural language machine learning modeling. The growth of non-English language editions of Wikipedia, greater computational resources, and calls for equity in the …


Considerations for the social impact of natural language processing

Natural language processing (NLP) technologies have transformed how people access information and communicate with one another. It has thus become critical to take stock of the social impact of natural language processing technologies. In this …


Consolidation of Crowd-Sourced Geo-Tagged Data for Parameterized Travel Recommendations

The general subject area of this dissertation is automated creation of recommendations for tourism, with the focus on sights and visit-worthy locations. This stands in contrast to the recommendations provided by mainstream travel …


Constructing and analyzing domain-specific language model for financial text mining

M Suzuki, H Sakaji, M Hirano, K Izumi - Information Processing & Management, 2023

The application of natural language processing (NLP) to financial fields is advancing with an increase in the number of available financial documents. Transformer-based models such as Bidirectional Encoder Representations from Transformers (BERT) …


Constructing novel datasets for intent detection and ner in a korean healthcare advice system: guidelines and empirical results

YM Kim, TH Lee, SO Na - Applied Intelligence, 2022

The demand for intelligent dialogue systems has increased rapidly in recent years. However, building such systems involves numerous complicated processes, including training data construction for language understanding. Although training …


Construction and Evaluation of Sentiment Datasets for Low-Resource Languages: The Case of Uzbek

C Gómez-Rodrıguez - … Technology. Challenges for Computer Science and …

To our knowledge, the majority of human language processing technologies for low-resource languages don’t have well-established linguistic resources for the development of sentiment analysis applications. Therefore, it is in dire need of such tools and …


CONSTRUCTION OF A TRANSLATION TOOL FROM FRENCH TO FALC

We started our internship on 06/06/2021 and ended on the 16/09/2022, which was an opportunity got after a sequence of offers sent by Mr Yasser ALMEHIO amongst which was found the internship at ISEP-école d'ingénieurs du numérique. For this …


Context-aware IoT Service Recommendation: A Deep Collaborative Filtering-based Approach

Z Wang, CA Sun, M Aiello - 2022 IEEE International Conference on Web Services …, 2022

The advantages of Service-Oriented Architecture (SOA) combined with the emerging and diffusion of the Internet of Things (IoT) instances have given birth to a new paradigm for IoT components integration, ie, Service-Oriented IoT. Microservices …


Contextual Answer Validation

… Among these is the Common Crawl dataset[2]. It is acquired via scraping web pages, skipping over any HTML markup. Each month, it generates about 20TB of data via scraping. However, a significant quantity of Common Crawl contains …


Contextualization and Generalization in Entity and Relation Extraction

B Taillé - arXiv preprint arXiv:2206.07558, 2022

During the past decade, neural networks have become prominent in Natural Language Processing (NLP), notably for their capacity to learn relevant word representations from large unlabeled corpora. These word embeddings can then be …


Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias

R Wolfe, Y Yang, B Howe, A Caliskan - arXiv preprint arXiv:2212.11261, 2022

Nine language-vision AI models trained on web scrapes with the Contrastive Language-Image Pretraining (CLIP) objective are evaluated for evidence of a bias studied by psychologists: the sexual objectification of girls and women, which occurs …


Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents

M Khalifa, Y Vyas, S Wang, G Horwood, S Mallya… - arXiv preprint arXiv …, 2022

… B Pre-training data from Common Crawl We build our pre-training corpus by first extracting all documents from CommonCrawl with a ‘.pdf‘ extension. We then remove duplicate documents based on the MD5 hash using fdupes.. The resulting …


Controlling Formality in Low-Resource NMT with Domain Adaptation and Re-Ranking: SLT-CDT-UoS at IWSLT2022

ST Vincent, L Barrault, C Scarton - arXiv preprint arXiv:2205.05990, 2022

This paper describes the SLT-CDT-UoS group's submission to the first Special Task on Formality Control for Spoken Language Translation, part of the IWSLT 2022 Evaluation Campaign. Our efforts were split between two fronts: data engineering …


Conversational Humor Analysis: Developing Data, Annotation Schema and Models

Conversational humor (CH) is a sub-domain of humor where the participants (speakers or listeners) engage in different types of humor such as retorts or teasing for various purposes. There are shared complexities between this phenomenon and the larger …


Core words in semantic representation

A Wang, S De Deyne, M McKague, A Perfors - Proceedings of the Annual Meeting of …, 2022

A central question in cognitive science is how semantic information is mentally represented. Two dominant theories of semantic representation are language-based distributional semantic models (which suggest that word meaning is based on which …


Correcting diacritics and typos with ByT5 transformer model

L Stankevičius, M Lukoševičius, J Kapočiūtė-Dzikienė… - arXiv preprint arXiv …, 2022

… Good examples of the first side are the Common Crawl dataset of more than 20TB of data and its version OSCAR [107] filtered by language. Such huge datasets are now one of the main building blocks of popular transformer model pre-training but …


CoSearcher: studying the effectiveness of conversational search refinement and clarification through user simulation

A Salle, S Malmasi, O Rokhlenko, E Agichtein - Information Retrieval Journal, 2022

A key application of conversational search is refining a user’s search intent by asking a series of clarification questions, aiming to improve the relevance of search results. Training and evaluating such conversational systems currently requires …


Cost-Effective Updating of Distributed Reordered Indexes

Index reordering techniques allow document collections to be renumbered, with the goal of developing a permutation of the initial document ordinal identifiers that places documents that are (somehow) like each other into positions near each other …


COVID vaccine stigma: detecting stigma across social media platforms with computational model based on deep learning

The study presents the first computational model of COVID vaccine stigma that can identify stigmatised sentiment with a high level of accuracy and generalises well across a number of social media platforms. The aim of the study is to understand the …


COVID-19 and cyberbullying: deep ensemble model to identify cyberbullying from code-switched languages during the pandemic

S Paul, S Saha, JP Singh - Multimedia Tools and Applications, 2022

… We have experimented with two different language specific fasttext Footnote 6 word embeddings, ie, English and Hindi as our code-switched corpus contains aforementioned two languages which are trained on Common Crawl and Wikipedia …


COVID-19 Tweets Classification Based on a Hybrid Word Embedding Method. Big Data Cogn. Comput. 2022, 6, 58

… pre-trained Word2Vec, FastText, and GloVe embedding trained on Common Crawl and Wikipedia with 300-D vectors, for word embedding. … vectors freely available on corpora, which is the combination of Gigaword5 and Wikipedia2014 …


Cramming: Training a Language Model on a Single GPU in One Day

J Geiping, T Goldstein - arXiv preprint arXiv:2212.14034, 2022

Recent trends in language modeling have focused on increasing performance through scaling, and have resulted in an environment where training language models is out of reach for most researchers and practitioners. While most in the …


Crawling Under-Resourced Languages–A Portal for Community-Contributed Corpus Collection

E Körner, F Helfer, C Schröder, T Eckart, D Goldhahn

… The full texts might also be contained and accessible in larger web crawls and collections such as archive.org or commoncrawl.org. To alleviate the likelihood of copyright and license infringement, the largest related text segments published are …


Creating a Large Language Model of a Philosopher

E Schwitzgebel, D Schwitzgebel, A Strasser

Can large language models be trained to produce philosophical texts that are difficult to distinguish from texts produced by human philosophers? To address this question, we fine-tuned OpenAI’s GPT-3 with two sets of training data: the blog posts …


CRIM's Speech Recognition system description for OpenASR21 evaluation

CRIM participated in all the 15 low resource languages and the three languages with case sensitive scoring in the OpenASR21 Challenge for the constrained condition. For acoustic modeling, we developed both hybrid DNN-HMM systems and …


CRIM's Speech Recognition System for OpenASR21 Evaluation with Conformer and Voice Activity Detector Embeddings

V Gupta, G Boulianne - International Conference on Speech and Computer, 2022

CRIM participated in all the 15 low resource languages and the three languages with case sensitive scoring in OpenASR21 for the constrained condition. For acoustic modeling, we developed both hybrid DNN-HMM systems and a conformer …


Cross-Language Learning for Product Matching

… As multilingual models, we use multi-lingual BERT (’bert-base-multilingual-uncased’), which is trained on the top 100 largest Wikipedias, as well as XLMRoBERTa (’xlm-roberta-base’), which is trained on a CommonCrawl corpus consisting of 100 different languages …


Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure

Y Chai, Y Liang, N Duan - arXiv preprint arXiv:2203.08430, 2022

Multilingual pre-trained language models, such as mBERT and XLM-R, have shown impressive cross-lingual ability. Surprisingly, both of them use multilingual masked language model (MLM) without any cross-lingual supervision or aligned data …


Cross-Lingual and Cross-Domain Crisis Classification for Low-Resource Scenarios

C Sánchez, H Sarmiento, J Pérez, A Abeliuk, B Poblete - arXiv preprint arXiv …, 2022

Social media data has emerged as a useful source of timely information about real-world crisis events. One of the main tasks related to the use of social media for disaster management is the automatic identification of crisis-related messages. Most of the …


Cross-Lingual Approaches to Identifying Argument Components and Relations in Norwegian Reviews

Argument mining is the process of automatic extraction of certain argumentation structures from data. Argument mining consists of several stages such as argument component detection, argument component classification, and argumentative …


Cross-lingual Coreference Resolution and Neural Machine Translation

Like many other Natural Language Processing (NLP) tasks, coreference resolution has benefited massively from new developments in recent years. Deeper and more complex architectures enabled coreference resolution systems that work end-to-end …


Cross-lingual transfer of knowledge in distributional language models: Experiments in Hungarian

A Novák, B Novák - Acta Linguistica Academica, 2022

In this paper, we argue that the very convincing performance of recent deep-neural-model-based NLP applications has demonstrated that the distributionalist approach to language description has proven to be more successful than the earlier subtle rule-based …


Cross-Modal Recurrent Semantic Comprehension for Referring Image Segmentation

C Shang, H Li, H Qiu, Q Wu, F Meng, T Zhao, KN Ngan - IEEE Transactions on …, 2022

… For language encoder, we adopt a vanilla GRU [20] and initialize the word embeddings by GloVe [65] pretrained on Common Crawl 840B. The input images are resized to 416×416. The maximum length of language expression is set to 15 in …


Crosslingual Sharing for Low-Resource Natural Language Processing

… Common Crawl, a repository of content scraped from the web which is frequently used as a source of text data, includes text from hundreds of languages—but more than 45% of the documents crawled are in English, just ten languages together …


CrossQE: HW-TSC 2022 Submission for the Quality Estimation Shared Task

S Tao, C Su, M Ma, H Yang, M Zhang, X Geng…

Quality estimation (QE) is a crucial method to investigate automatic methods for estimating the quality of machine translation results without reference translations. This paper presents Huawei Translation Services Center’s (HWTSC’s) work called …


Current Research in Ecological and Social Psychology

M Atari, NK Reimer, J Graham, J Hoover, B Kennedy…

Infectious diseases have been an impending threat to the survival of individuals and groups throughout our evolutionary history. As a result, humans have developed psychological pathogen-avoidance mechanisms and groups have developed …


Customer Support Chat Intent Classification using Weak Supervision and Data Augmentation

S Prabhu, AK Brahma, H Misra - 5th Joint International Conference on Data Science …, 2022

Understanding the actual intent of customers is an essential step in automating the conversational experience on a chat platform. Typically, chatbots are powered by machine learning algorithms that rely on the acquisition of a large amount of high …


Cyber-Phishing Website Detection Using Fuzzy Rule Interpolation

M Almseidin, M Alkasassbeh, M Alzubi, J Al-Sawwa - Cryptography, 2022

This paper introduces a novel detection method for phishing website attacks while avoiding the issues associated with the deficiencies of the knowledge-based representation and the binary decision. The suggested detection method was …


Data Analytics and Modeling in IoT-Fog Environment for Resourceconstrained IoT-Applications: A Review

O Farooq, P Singh - Recent Advances in Computer Science and …, 2022

Objective: The emergence of the concepts like Big Data, Data Science, Machine Learning (ML), and the Internet of Things (IoT) in recent years has added the potential of research in today's world. The continuous use of IoT devices, sensors, etc …


Data Augmentation for Automated Essay Scoring using Transformer Models

Automated essay scoring is one of the most important problem in Natural Language Processing. It has been explored for a number of years, and it remains partially solved. In addition to its economic and educational usefulness, it presents research …


Data Augmentation for Inline Tag-Aware Neural Machine Translation

Despite the wide use of inline formatting, not much has been studied on translating sentences with inline formatted tags. The detag-andproject approach using word alignments is one solution to translating a tagged sentence. However, the method …


Data Augmentation for Intent Classification of German Conversational Agents in the Finance Domain

S Rentschler, M Riedl, C Stab, M Rückert - Proceedings of the 18th Conference on …, 2022

In this paper, we focus on improving the intent recognition for a conversational agent. For languages other than English, labeled data needed for training is often limited. Limitations rise even more when moving to specific domains. Here, our goal is to …


Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation

A Fernando, S Ranathunga - arXiv preprint arXiv:2205.08722, 2022

Out-of-Vocabulary (OOV) is a problem for Neural Machine Translation (NMT). OOV refers to words with a low occurrence in the training data, or to those that are absent from the training data. To alleviate this, word or phrase-based Data Augmentation (DA) …


Data Bootstrapping Approaches to Improve Low Resource Abusive Language Detection for Indic Languages

M Das, S Banerjee, A Mukherjee - arXiv preprint arXiv:2204.12543, 2022

Abusive language is a growing concern in many social media platforms. Repeated exposure to abusive speech has created physiological effects on the target users. Thus, the problem of abusive language should be addressed in all forms for online …


Data Efficient Multilingual Natural Language Processing

… ; the top five languages in CommonCrawl, a popular source of web-crawled data for the development of NLP models, account for about 70% of the total data, while the majority of languages in the world have much less data or no data at all in …


Data Governance in the Age of Large-Scale Data-Driven Language Technology

Y JERNITE, HUU NGUYEN, S BIDERMAN, A ROGERS… - 2022

New families of algorithms relying on deep learning have made it possible to extract ever more complex language statistics from growing numbers of text and speech records to drastically improve the performance and applicability of data-driven …


Data integration and ethical quality: fundamental steps of the data analysis pipeline

… and the corpus on which it was trained is Common Crawl2. Regarding GloVe we use the model glove.840B.300d [106] and again each word is represented as a 300-dimensional vector and the training corpus was still Common Crawl. Once the single words are …


Data paper

… The dataset was extracted from Common Crawl, a web archive, which has periodically crawled … We used the entire Common Crawl text archive of April 2019 for processing and conducting … We subsequently used the entire Common Crawl …


Data Selection Curriculum for Neural Machine Translation

T Mohiuddin, P Koehn, V Chaudhary, J Cross… - arXiv preprint arXiv …, 2022

… In our experiments, we use the newscrawl data as in-domain and commoncrawl data combined with newscrawl as general-domain for training the LMs. After scoring each parallel sentence pair (xi,yi) ∈ Dd by any of the above methods, we rank Dd …


Data-Centric Debugging: mitigating model failures via targeted data collection

S Singla, AM Chegini, M Moayeri, S Feiz - arXiv preprint arXiv:2211.09859, 2022

Deep neural networks can be unreliable in the real world when the training set does not adequately cover all the settings where they are deployed. Focusing on image classification, we consider the setting where we have an error distribution $\mathcal{E} …


Data-Driven Natural Language Inference

Natural Language Inference (NLI) research involves the development of models that can mimic human inference processes based on natural language and classify the inference relation between sentences. For example, given the premise that “In 2019 …


Database Systems for Advanced Applications: 27th International Conference, DASFAA 2022, Virtual Event, April 11–14, 2022, Proceedings, Part III

A Bhattacharya, JLM Li, D Agrawal, PK Reddy… - 2022

The three-volume set LNCS 13245, 13246 and 13247 constitutes the proceedings of the 26th International Conference on Database Systems for Advanced Applications, DASFAA 2022, held online, in April 2021. The total of 72 full papers, along with 76 …


Datasheet for the Pile

S Biderman, K Bicheno, L Gao - arXiv preprint arXiv:2201.07311, 2022

This datasheet describes the Pile, a 825 GiB dataset of human-authored text compiled by EleutherAI for use in large-scale language modeling. The Pile is comprised of 22 different text sources, ranging from original scrapes done for this …


DBkWik++--Multi Source Matching of Knowledge Graphs

S Hertling, H Paulheim - arXiv preprint arXiv:2210.02864, 2022

Large knowledge graphs like DBpedia and YAGO are always based on the same source, ie, Wikipedia. But there are more wikis that contain information about long-tail entities such as wiki hosting platforms like Fandom. In this paper, we present the …


Dead or Murdered? Predicting Responsibility Perception in Femicide News Reports

G Minnema, S Gemelli, C Zanchi, T Caselli, M Nissim - arXiv preprint arXiv …, 2022

Different linguistic expressions can conceptualize the same event from different viewpoints by emphasizing certain participants over others. Here, we investigate a case where this has social consequences: how do linguistic expressions of gender-based …


Debiasing isn't enough!--On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks

M Kaneko, D Bollegala, N Okazaki - arXiv preprint arXiv:2210.02938, 2022

We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures …


Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features

C Du, K Fu, J Li, H He - arXiv preprint arXiv:2210.06756, 2022

… several studies have addressed the idea of decoding naturalistic visual experiences from brain activity using purely linguistic features [15], [16], they merely use standard word vectors of class names that are automatically extracted from large …


Deep cascaded multitask framework for detection of temporal orientation, sentiment and emotion from suicide notes

S Ghosh, A Ekbal, P Bhattacharyya - Scientific Reports, 2022

… To capture the syntactic and semantic information of the words in a phrase, we use the efficacy of a 300-dimensional pre-trained GloVe word embedding 51 that is trained on the Common Crawl (42 billion tokens) corpus. …


Deep Learning Approach for Aspect-Based Sentiment Analysis of Restaurants Reviews in Spanish

BC Martínez-Seis, O Pichardo-Lagunas, S Miranda… - Computación y Sistemas, 2022

Online reviews of products and services have become important for customers and enterprises. Recent research focuses on analyzing and managing those kinds of reviews using natural language processing. This paper focuses on aspect-based …


Deep learning based semantic textual similarity for applications in translation technology

Semantic Textual Similarity (STS) measures the equivalence of meanings between two textual segments. It is a fundamental task for many natural language processing applications. In this study, we focus on employing STS in the context of translation …


Deep Learning in Sentiment Analysis: A Survey of Recent Architectures

T Abdullah, A Ahmet - ACM Computing Surveys (CSUR), 2022

Humans are increasingly integrated with devices that enable the collection of vast unstructured opinionated data. Accurately analysing subjective information from this data is the task of sentiment analysis (an actively researched area in NLP). Deep …


DEEP LEARNING-BASED DEPENDENCY PARSING FOR TURKISH

… While XLM-R is pretrained on 2.5TB of unlabeled text data across 100 languages extracted from CommonCrawl Corpus [48], the multilingual version of BERT is pretrained on a much smaller Wikipedia Corpus for 104 languages. Another …


Deep Lexical Hypothesis: Identifying personality structure in natural language

Recent advances in natural language processing (NLP) have produced general models that can perform complex tasks such as summarizing long passages and translating across languages. Here, we introduce a method to extract adjective …


Deepfake Text Detection: Limitations and Opportunities

J Pu, Z Sarwar, SM Abdullah, A Rehman, Y Kim…

… This generator is trained on the RealNews dataset [8], a large corpus (120GB) of news articles from Common Crawl [35]. The GPT-2 model is … [35] “Common crawl: A nonprofit 501 organization that crawls the web and freely provides its archives and …


Delivering Fairness in Human Resources AI: Mutual Information to the Rescue

L Hemamou, W Coleman - Proceedings of the 2nd Conference of the Asia-Pacific …, 2022

Automatic language processing is used frequently in the Human Resources (HR) sector for automated candidate sourcing and evaluation of resumes. These models often use pre-trained language models where it is difficult to know if possible biases …


Demographic-Aware Language Model Fine-tuning as a Bias Mitigation Technique

BERT-like language models (LMs), when exposed to large unstructured datasets, are known to learn and sometimes even amplify the biases present in such data. These biases generally reflect social stereotypes with respect to gender, race, age …


Demonstrator of Automatic Minuting

M Singh, R Kumar, T Ghosal, O Bojar, CC PV, AS PV… - 2022

… framework to suit a variety of challenges in the domain of natural language understanding T5 also explained the advantage of scaling up the model size (to 11B parameters) and pre-training corpus, by introducing C4 (a massive text corpus which …


Demystifying Prompts in Language Models via Perplexity Estimation

H Gonen, S Iyer, T Blevins, NA Smith, L Zettlemoyer - arXiv preprint arXiv:2212.04037, 2022

Language models can be prompted to perform a wide variety of zeroand few-shot learning problems. However, performance varies significantly with the choice of prompt, and we do not yet understand why this happens or how to pick the best …


Dense-to-Sparse Gate for Mixture-of-Experts

X Nie, S Cao, X Miao, L Ma, J Xue, Y Miao, Z Yang… - arXiv preprint arXiv …, 2021

… Wikipedia is a single domain of text, which contains about 2,500 words, and WebText is a large and diverse dataset in various domains, which is collected using 45 million links from Common Crawl. We apply GPT2-BPE (vocabulary size of 50257) …


DESIGN AND IMPLEMENTATION OF A REALWORLD SEARCH ENGINE BASED ON OKAPI BM25 AND SENTENCEBERT

The work conducted in this thesis aims to present an hybrid model for a real world application search engine. The project presented was part of an intern ship work carried out in a startup which deals with Knowledge Management and Artificial …


Design of an GIS-based Investment Heatmap System using Topic Classification and NER

TT Van, KV Sy, TT Anh, TL Quang, PH Xuan, HL Viet… - 2021 13th International …, 2021

… A multilingual fastText model [7], pre-trained on Common Crawl and Wikipedia corpus, is used to train on our collected corpus using transfer learning approach. The model was trained using CBOW with position-weights, in dimension 300, with …


Design of Oral English Intelligent Evaluation System Based on DTW Algorithm

Y Fang - Mobile Networks and Applications, 2022

The accuracy of the existing spoken English intelligent evaluation system is not high, and the spoken English evaluation effect is poor. To improve the accuracy and speed of spoken English evaluation, this paper puts forward the design and …


Designing Machine Learning Systems

Machine learning systems are both complex and unique. Complex because they consist of many different components and involve many different stakeholders. Unique because they're data dependent, with data varying wildly from one use case …


Detecting ambiguity in conversational systems

The question of detection of user search queries has been explored by many authors. With the advent of speech based search interfaces, narrowing down the scope of search based on user intent becomes even more important. A prominent part of …


Detecting autism from picture book narratives using deep neural utterance embeddings

A Wawer, I Chojnicka - International Journal of Language & Communication …, 2022

… Pre-training of ELMoForManyLangs was performed on a 20 million-word data set sampled from Wikipedia and Common Crawl. For each input utterance, the representation obtained from ELMo was a vector of 1024 elements. The vectors …


Detecting Backdoors in Deep Text Classifiers

Y Guo, J Wang, T Cohn - arXiv preprint arXiv:2210.11264, 2022

Deep neural networks are vulnerable to adversarial attacks, such as backdoor attacks in which a malicious adversary compromises a model during training such that specific behaviour can be triggered at test time by attaching a specific word or …


Detecting explicit lyrics: a case study in Italian music

M Rospocher - Language Resources and Evaluation, 2022

Preventing the reproduction of songs whose textual content is offensive or inappropriate for kids is an important issue in the music industry. In this paper, we investigate the problem of assessing whether music lyrics contain content unsuitable …


Detecting Psychological Phenomena in Dialogues

Instant messaging (IM) is a type of online communication that allows for the synchronous exchange of text, images, voice, and videos between two or more people [1] using applications and platforms, such as Messenger or WhatsApp. This …


Detection and Performance Evaluation of Online-Fraud Using Deep Learning Algorithms

A Khan, M Patil - Intelligent Computing and Networking, 2022

Online News Portals are currently one of the primary sources used by people, though its credibility is under serious question. Because the problem associated with this is Click-bait. Click-baiting being the growing phenomenon on internet has the …


Detection and Prevention of Sophisticated Cyberattacks

Despite extensive research by the security community, cyberattacks such as phishing and Internet of Things (IoT) attacks remain profitable to criminals and continue to cause substantial damage not only to the victim users that they target …


Detection of Bangla Hate Comments and Cyberbullying in Social Media Using NLP and Transformer Models

M Emon, I Haque, KN Iqbal, M Mehedi, H Kabir… - International Conference on …, 2022

Hate speech and cyberbullying detection on social media is one of the most trending natural language processing tasks in the current scenario. However, due to the lack of resources, a few work has been done in detection of cyberbullying in social media …


Detection of dangerous events on social media: a critical review

ML Jamil, S Pais, J Cordeiro - Social Network Analysis and Mining, 2022

… The additional data includes the CommonCrawl News dataset (63 million articles, 76 GB), Web text corpus (38 GB), and Stories from Common Crawl (31 GB). This, combined with a massive 1024 V100 Tesla GPU’s running for a day, resulting in pre-training …


Detection of prosodic boundaries in speech using Wav2Vec 2.0

M Kunešová, M Řezáčková - International Conference on Text, Speech, and …, 2022

Prosodic boundaries in speech are of great relevance to both speech synthesis and audio annotation. In this paper, we apply the wav2vec 2.0 framework to the task of detecting these boundaries in speech signal, using only acoustic information. We …


Developing a battery of measures for unobtrusive indicators of organisational culture: a research

… Then, using word embeddings based Common Crawl (2.6 unannotated billion web pages; 840 billion occurrences of 2.2 million unique tokens or words) and OntoNotes 5 (2.9 million annotated words), lists of similar words were generated for …


Developing a general-purpose clinical language inference model from a large corpus of clinical notes

M Sushil, D Ludwig, AJ Butte, VA Rudrapatna - arXiv preprint arXiv:2210.06566, 2022

Several biomedical language models have already been developed for clinical language inference. However, these models typically utilize general vocabularies and are trained on relatively small clinical corpora. We sought to evaluate the impact …


Developing a news classifier for Greek using BERT

G Gkolfopoulos, I Varlamis - 2022 7th South-East Europe Design Automation …, 2022

Text categorization is a significant task in the re-search field of text mining, which has recently benefited from deep neural network algorithms and advanced learning techniques that extract language models from large textual corpora. These Pre-Trained …


Developing disinformation detection models in low-resource contexts: Czech news article dataset for source-level credibility

… To first obtain URLs of sites with individual articles from those domains, we used the Commoncrawl API 1. We specified for the API to include only articles discovered between January 2019 and September 2021. Once these 1Commoncrawl library …


Developing Technology Tools to Combat Fake Science

C Impey, A Danehy - Future of Information and Communication Conference, 2022

… In the next phase of the project, this technology will be scaled to large samples of content drawn from CommonCrawl, and it will be applied across more domains of science. Then it will be deployed as a web browser extension that presents the …


Development of lumbar spine MRI referrals vetting models using machine learning and deep learning algorithms: Comparison models vs healthcare professionals

AH Alanazi, A Cradock, L Rainford - Radiography, 2022

Introduction Referrals vetting is a necessary daily task to ensure the appropriateness of radiology referrals. Vetting requires extensive clinical knowledge and may challenge those responsible. This study aims to develop AI models to automate the …


Diachronic Neural Network Predictor of Word Animacy

V Bochkarev, A Achkeev, A Shevlyakova… - … International Conference on …, 2022

The paper considers the problem of automatic recognition of animacy in the Russian language. We propose a recognizer that is based on the analysis of co-occurrence with the most frequent words and is trained on data from the Russian subcorpus of …


Dialog Inpainting: Turning Documents into Dialogs

Z Dai, AT Chaganty, V Zhao, A Amini, QM Rashid… - arXiv preprint arXiv …, 2022

Many important questions (eg "How to eat healthier?") require conversation to establish context and explore in depth. However, conversational question answering (ConvQA) systems have long been stymied by scarce training data that is expensive …


Dictionary with the Evaluation of Positivity/Negativity Degree of the Russian Words

V Solovyev, M Islamov, V Bayrasheva - International Conference on Speech and …, 2022

… It was a feed-forward neural network, with the fastText vectors of the Common Crawl model fed to its input. Significant improvement of results has been achieved: 0.870. Scores extrapolation was carried out for some other languages. …


Did I Miss Anything? A Study on Ranking Fusion and Manual Query Rewriting in Consumer Health Search

GM Di Nunzio, F Vezzani - International Conference of the Cross-Language …, 2022

In this paper, we describe the methodology and experimental analysis of a twofold strategy for the retrieval of medical relevant information: a ranking fusion and a query reformulation approach. In particular, the query reformulation approach is …


DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings

YS Chuang, R Dangovski, H Luo, Y Zhang, S Chang… - arXiv preprint arXiv …, 2022

… However, they use 1TB of the training data from Common Crawl dumps while our model only use 115MB of the Wikipedia data for pretraining. We put their scores in Table 2 for reference. In SimCSE, the authors propose to use MLM as an auxiliary …


Differentially Private Decoding in Large Language Models

J Majmudar, C Dupuy, C Peris, S Smaili, R Gupta… - arXiv preprint arXiv …, 2022

Recent large-scale natural language processing (NLP) systems use a pre-trained Large Language Model (LLM) on massive and diverse corpora as a headstart. In practice, the pre-trained model is adapted to a wide array of tasks via fine-tuning on …


Digital artifacts reveal development and diffusion of climate research

B Carneiro, G Resce, TB Sapkota - Scientific Reports, 2022

… While public web archives exist—eg WayBack Machine 48 or Common Crawl 49 —they might miss a good number of web pages, resulting in limited outcomes 50 . Building a dedicated search engine could solve the coverage issue but would …


Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration

B Csanády, A Lukács - arXiv preprint arXiv:2201.06757, 2022

… In the case of HunWeb2, we used the "2017-2018" part of the Common Crawl subcorpus. We decided not to split up the documents to multiple chunks or sentences, as we have found that there can be a lot of repeated sentences within one document …


DiscoSense: Commonsense Reasoning with Discourse Connectives

P Bhargava, V Ng - arXiv preprint arXiv:2210.12478, 2022

We present DiscoSense, a benchmark for commonsense reasoning via understanding a wide variety of discourse connectives. We generate compelling distractors in DiscoSense using Conditional Adversarial Filtering, an extension of …


Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation

C Wang, Y Liu, B Chen, J Zhang, W Luo, Z Huang… - arXiv preprint arXiv …, 2022

End-to-end Speech Translation (ST) aims at translating the source language speech into target language text without generating the intermediate transcriptions. However, the training of end-to-end methods relies on parallel ST data, which are difficult and …


Distillation or loss of information?: The effects of distillation on model redundancy

… The XLM-Roberta could be comparable to mBERT, as the model can process 100 languages, it was trained on 2.1 terabytes of data extracted from CommonCrawl. It is important to note, that although lacking the next sentence prediction functionality …


Distributional evidence for derivational paradigms

O Bonami, MG Naranjo - The semantics of derivational morphology. Theory …

This paper provides evidence from distributional semantics on the importance of paradigmatic relations in word formation systems. We argue that, if paradigmatic relations play a systemic role, that role should be measurable by examining whether …


Do children texts hold the key to commonsense knowledge?

J Romero, S Razniewski - arXiv preprint arXiv:2210.04530, 2022

Compiling comprehensive repositories of commonsense knowledge is a long-standing problem in AI. Many concerns revolve around the issue of reporting bias, ie, that frequency in text sources is not a good proxy for relevance or truth. This paper …


Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023?

S Liu, A Ritter - arXiv preprint arXiv:2212.09747, 2022

… Our dataset follows this distribution to collect Reuters news articles published between December 5th and 7th, 2020, collected from the Common Crawl Foundation3. We tokenized the data with the same tokenizer used for the CoNLL-2003 corpus …


Do Images really do the Talking?

SU Hegde, A Hande, R Priyadharshini, S Thavareesan…

… The text encoder was XLM-Roberta [58], which was trained on the 100 languages Common Crawl dataset, and the image feature extraction was ResNet [59] with 152 layers, followed by linear transformation to match the word embedding size of the …


Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?

JA Michaelov, BK Bergen - arXiv preprint arXiv:2208.14554, 2022

Some languages allow arguments to be omitted in certain contexts. Yet human language comprehenders reliably infer the intended referents of these zero pronouns, in part because they construct expectations about which referents are …


Do We Need a Specific Corpus and Multiple High-Performance GPUs for Training the BERT Model? An Experiment on COVID-19 Dataset

N Nuntachit, P Sugunnasil - Machine Learning and Knowledge Extraction, 2022

The COVID-19 pandemic has impacted daily lives around the globe. Since 2019, the amount of literature focusing on COVID-19 has risen exponentially. However, it is almost impossible for humans to read all of the studies and classify them. This article …


Document Classification with Hierarchical Graph Neural Networks

Various neural architectures have been explored for document classification, such as convolutional and recurrent networks or as of late, Transformers. In parallel, graph neural networks have vastly improved over the recent years. In this paper, we …


Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources

A McMillan-Major, Z Alyafeai, S Biderman, K Chen… - arXiv preprint arXiv …, 2022

… Typically, this data is collected from online sources, ranging from highly edited and structured text such as Wikipedia to the myriad text and audiovisual components of web pages collected by Common Crawl1. Several issues concerning their …


Does Corpus Quality Really Matter for Low-Resource Languages?

M Artetxe, I Aldabe, R Agerri, O Perez-de-Viñaspre… - arXiv preprint arXiv …, 2022

… 2020), respectively, and were built by filtering CommonCrawl. Wikipedia has been a … Instead of filtering CommonCrawl, we do tailored crawling on 33 websites with high-quality content in … First of all, we find that, even if CommonCrawl derived …


Domain Adaptation for Multi-Modal Foundation Models

The multi-modal foundation model CLIP extracts representations of texts and images that can be used to achieve unprecedented performance in various tasks. However, this model was only trained on public data from the internet. Thus it lacks knowledge …


Domain Adaptation for Sparse-Data Settings: What Do We Gain by Not Using Bert?

M Sedinkina, M Schmitt, H Schütze - arXiv preprint arXiv:2203.16926, 2022

The practical success of much of NLP depends on the availability of training data. However, in real-world scenarios, training data is often scarce, not least because many application domains are restricted and specific. In this work, we compare …


Domain Adaptation of Machine Translation with Crowdworkers

M Morishita, J Suzuki, M Nagata - arXiv preprint arXiv:2210.15861, 2022

… However, since CommonCrawl only covers a small portion of the web, it may overlook websites that contain valuable resources. Thus, the … In contrast, our work does not rely on CommonCrawl but on crowdworkers who can search the whole …


Domain Adaptation of Transformer-Based Models using Unlabeled Data for Relevance and Polarity Classification of German Customer Feedback

A Idrissi-Yaghir, H Schäfer, N Bauer, CM Friedrich - arXiv preprint arXiv:2212.05764, 2022

Understanding customer feedback is becoming a necessity for companies to identify problems and improve their products and services. Text classification and sentiment analysis can play a major role in analyzing this data by using a variety of machine …


Domain Mismatch Doesn't Always Prevent Cross-Lingual Transfer Learning

Cross-lingual transfer learning without labeled target language data or parallel text has been surprisingly effective in zero-shot cross-lingual classification, question answering, unsupervised machine translation, etc. However, some recent …


Domain over size: Clinical ELECTRA surpasses general BERT for bleeding site classification in the free text of electronic health records

JS Pedersen, MS Laursen, C Soguero-Ruiz… - 2022 IEEE-EMBS …, 2022

Bleeding can be a life-threatening condition which occurs for 3.2% of medical patients. Information about previous bleeding and bleeding site is used to predict the risk of future bleeding and guide anticoagulant treatment. However, obtaining this …


Domain Parking: Largely Present, Rarely Considered!

J Zirngibl, S Deusch, P Sattler, J Aulbach, G Carle…

… Common Crawl While visual identification allowed us to validate the inferences to a reasonable extent, we wanted to upscale validation. Therefore, we consider Common Crawl (CC) data [21] and calculate the similarity of pages. Common Crawl …


Domain-aware Multi-modality Fusion Network for Generalized Zero-shot Learning

J Wang, X Wang, H Zhang - Neurocomputing, 2022

… Specifically, we investigate two word embedding methods including Word2vec[30] and Glove [36], which are trained on 1.4M Google News 2 and 2.2M Common Crawl 3 . We also consider hierarchical embeddings where similarity between objects can …


Domain-specific language models for multi-label classification of medical text

Recent advancements in machine learning-based medical text multi-label classifications can be used to enhance the understanding of the human body and aid the need for patient care. This research considers predicting medical codes from …


Domain-Specific Machine Learning-A Human Learning Perspective

This dissertation focuses on Domain-Specific Machine Learning (DSML) with applications in the semiconductor industry. We first illustrate characteristics of DSML in view of semiconductor design and test processes, where a task is usually carried …


Double Retrieval and Ranking for Accurate Question Answering

Z Zhang, T Vu, A Moschitti - arXiv preprint arXiv:2201.05981, 2022

… Then, we used it to build a large index having up to 130MM passages extracted from 54MM documents of CommonCrawl2. We selected English Web documents of 5,000 most popular domains, including Wikipedia, from the recent releases of …


DS4DH at TREC Health Misinformation 2021: Multi-Dimensional Ranking Models with Transfer Learning and Rank Fusion

B Zhang, N Naderi, F Jaume-Santero, D Teodoro - arXiv preprint arXiv:2202.06771, 2022

This paper describes the work of the Data Science for Digital Health (DS4DH) group at the TREC Health Misinformation Track 2021. The TREC Health Misinformation track focused on the development of retrieval methods that provide relevant, correct …


Dutch SQuAD and Ensemble Learning for Question Answering from Labour Agreements

NJ Rouws, S Vakulenko, S Katrenko - Benelux Conference on Artificial Intelligence, 2021

The Dutch Ministry of Social Affairs and Employment has to regularly explore the content of labour agreements. Studies on topics such as diversity and work flexibility are conducted on the regular basis by means of specialised questionnaires. We …


Early depression detection in social media based on deep learning and underlying emotions

JSL Figuerêdo, ALLM Maia, RT Calumby - Online Social Networks and Media, 2022

Depression is a challenge to public health, frequently related to disability and one of the reasons that lead to suicide. Many of the ones who suffer depression use social media to obtain information or even to talk about their problem. Some studies have …


Early risk detection of mental illnesses using various types of textual features

R Ferreira, A Trifan, JL Oliveira - Working Notes of CLEF, 2022

This paper documents the participation of our team, BioInfo@ UAVR, in the first and second task of the 2022 edition of CLEF eRisk. With the goal of achieving an early detection of subjects at risk of specific mental illnesses, pathological gambling and …


EDIN: An End-to-end Benchmark and Pipeline for Unknown Entity Discovery and Indexing

N Kassner, F Petroni, M Plekhanov, S Riedel… - arXiv preprint arXiv …, 2022

Existing work on Entity Linking mostly assumes that the reference knowledge base is complete, and therefore all mentions can be linked. In practice this is hardly ever the case, as knowledge bases are incomplete and because novel concepts arise …


Effective Approaches to Neural Query Language Identification

X Ren, B Yang, D Liu, H Zhang, X Lv, L Yao, J Xie - Computational Linguistics, 2022

… Commoncow: Massively huge web corpora from commoncrawl data and a method to distribute them freely under restrictive EU copyright laws. In Proceedings of the Tenth International Conference on Language Resources and Evaluation …


Effective Transfer Learning for Low-Resource Natural Language Understanding

Z Liu - arXiv preprint arXiv:2208.09180, 2022

Natural language understanding (NLU) is the task of semantic decoding of human languages by machines. NLU models rely heavily on large training data to ensure good performance. However, substantial languages and domains have very few …


Effectiveness of French Language Models on Abstractive Dialogue Summarization Task

Y Zhou, F Portet, F Ringeval - arXiv preprint arXiv:2207.08305, 2022

Pre-trained language models have established the state-of-the-art on various natural language processing tasks, including dialogue summarization, which allows the reader to quickly access key information from long conversations in meetings …


Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks

E Villatoro-Tello, S Madikeri, J Zuluaga-Gomez… - arXiv preprint arXiv …, 2022

In this paper, we perform an exhaustive evaluation of different representations to address the intent classification problem in a Spoken Language Understanding (SLU) setup. We benchmark three types of systems to perform the SLU intent detection task …


Effects of Physiological Signals in Different Types of Multimodal Sentiment Estimation

S Katada, S Okada, K Komatani - IEEE Transactions on Affective Computing, 2022

… FastText word vectors FastText is a recently developed language model for training word vectors with subword information on Wikipedia and the Common Crawl corpus [41]. This language model is not restricted to English and is available …


Effects of Similarity Score Functions in Attention Mechanisms on the Performance of Neural Question Answering Systems

Y Shen, EMK Lai, M Mohaghegh - Neural Processing Letters, 2022

Attention mechanisms have been incorporated into many neural network-based natural language processing (NLP) models. They enhance the ability of these models to learn and reason with long input texts. A critical part of such mechanisms …


Efficient and Accurate Closed-Domain and Open-Domain Long-Form Question Answering

RS Butler, VD Duggirala, F Banaei-Kashani - International Conference on Computer …, 2022

We present an efficient and accurate long-form question answering platform, dubbed iLFQA (ie, short for intelligent Long-Form Question Answering). iLFQA was created as a continuation of iTA (ie, short for intelligent Teaching Assistant). iTA was …


Efficient Language Modeling with Sparse all-MLP

P Yu, M Artetxe, M Ott, S Shleifer, H Gong, V Stoyanov… - arXiv preprint arXiv …, 2022

All-MLP architectures have attracted increasing interest as an alternative to attention-based models. In NLP, recent work like gMLP shows that all-MLPs can match Transformers in language modeling, but still lag behind in downstream tasks. In this work, we …


Efficient Multilingual Multi-modal Pre-training through Triple Contrastive Loss

Y Lee, KT Lim, W Baek, B Roh, S Kim - … of the 29th International Conference on …, 2022

… Image-text pairs We follow the data collection process of ALIGN to create our in-house 1B imagetext pairs dataset from Common crawl 2. The raw descriptions are gathered from the Alt-text HTML attribute associated with web images. We only apply …


Effortless and beneficial processing of natural languages using transformers

K Amrutha, P Prabu - Journal of Discrete Mathematical Sciences and …, 2022

Natural Language Processing plays a vital role in our day-to-day life. Deep learning models for NLP help make human life easier as computers can think, talk, and interact like humans. Applications of the NLP models can be seen in many domains …


ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trolls

Online trolls increase social costs and cause psychological damage to individuals. With the proliferation of automated accounts making use of bots for trolling, it is difficult for targeted individual users to handle the situation both quantitatively and …


Elite Environmental Aesthetics: Placing Nature in a Changing Climate

A McCumber, A Davis - American Journal of Cultural Sociology, 2022

What does it mean to escape to nature in a changing climate? We investigate the properties that make a place resonant with ideas of “nature,” particularly for liberal elite vacationers consuming place through travel. By analyzing over 10,000 articles …


ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models

J Li, T Tang, Z Gong, L Yang, Z Yu, Z Chen, J Wang… - arXiv preprint arXiv …, 2022

Nowadays, pretrained language models (PLMs) have dominated the majority of NLP tasks. While, little research has been conducted on systematically evaluating the language abilities of PLMs. In this paper, we present a large-scale empirical study …


Emerging trends: General fine-tuning (gft)

KW Church, X Cai, Y Ying, Z Chen, G Xun, Y Bian - Natural Language Engineering, 2022

This paper describes gft (general fine-tuning), a little language for deep nets, introduced at an ACL-2022 tutorial. gft makes deep nets accessible to a broad audience including non-programmers. It is standard practice in many fields to use …


Emerging Trends: SOTA-Chasing

KW Church, V Kordoni - Natural Language Engineering, 2022

Many papers are chasing state-of-the-art (SOTA) numbers, and more will do so in the future. SOTA-chasing comes with many costs. SOTA-chasing squeezes out more promising opportunities such as coopetition and interdisciplinary collaboration. In …


Emotion Classification of MIDI Lyrics Using Transformers

… It uses the common crawl web extracted text [34] as a dataset for pre-training, where after some data cleaning and filtering, it ends up with 750GB of uncompressed English text, way more than RoBERTa, which was already an …


Emotion Detection using Deep Learning

Y Shaaban, H Korashy, W Medhat - 2021 16th International Conference on Computer …, 2021

Emotion detection is one of the challenging tasks in Natural Language Processing. The applications of emotion detection can be utilized in different fields like data mining, psychology, and human-computer interaction. The paper aims to train …


Emotion-Aware Multimodal Pre-training for Image-Grounded Emotional Response Generation

Z Tian, Z Wen, Z Wu, Y Song, J Tang, D Li, NL Zhang - International Conference on …, 2022

Face-to-face communication leads to better interactions between speakers than text-to-text conversations since the speakers can capture both textual and visual signals. Image-grounded emotional response generation (IgERG) tasks requires chatbots to generate a …


Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey

K Nozawa, I Sato - arXiv preprint arXiv:2204.08226, 2022

Representation learning enables us to automatically extract generic feature representations from a dataset to solve another machine learning task. Recently, extracted feature representations by a representation learning algorithm and a …


Employing a Multilingual Transformer Model for Segmenting Unpunctuated Arabic Text

AM Alshanqiti, S Albouq, AB Alkhodre, A Namoun… - 2022

Long unpunctuated texts containing complex linguistic sentences are a stumbling block to processing any low-resource languages. Thus, approaches that attempt to segment lengthy texts with no proper punctuation into simple candidate sentences …


En-SeqGAN: An Efficient Sequence Generation Model for Deceiving URL Classifiers

TD Pham, TTT Pham, VC Ta - Asian Conference on Intelligent Information and …, 2022

… The URLs in the data are crawled from Common Crawl website [19] and Phishtank [20] It resulted in 6 million legitimate URLs from Common Crawl and 1 million phish URLs from Phishtank. We follow the preprocessing procedure from [21] …


Energy-Efficient Data Transfer Optimization via Decision-Tree Based Uncertainty Reduction

H Jamil, L Rodolph, J Goldverg, T Kosar - arXiv preprint arXiv:2204.07601, 2022

… These include: (1) the small file size dataset consisting of 20,000 HTML files derived from the common crawl project [24]; (2) the medium file size dataset consisting of 5,000 image files derived from Flickr [25]; (3) the large file size dataset …


Enhanced Distance-aware Self-attention and Multi-level Match for Sentence Semantic Matching

Y Deng, X Li, M Zhang, X Lu, X Sun - Neurocomputing, 2022

Sentence semantic matching is a core research area in natural language processing, which is widely used in various natural language tasks. In recent years, attention mechanism has shown good performance in deep neural networks for sentence …


Enhancing Cross-lingual Prompting with Mask Token Augmentation

M Zhou, X Li, Y Jiang, L Bing - arXiv preprint arXiv:2202.07255, 2022

… Model Details XLM-R base model, containing 270M parameters, is pretrained on 2.5TB of filtered CommonCrawl on 100 languages. It contains 12 Transformer layers with hidden space dimensions of 768 and 12 attention heads in each layer. …


Enhancing Indic Handwritten Text Recognition Using Global Semantic Information

A Mondal, CV Jawahar - International Conference on Frontiers in Handwriting …, 2022

Handwritten Text Recognition (htr) is more interesting and challenging than printed text due to uneven variations in the handwriting style of the writers, content, and time. htr becomes more challenging for the Indic languages because of (i) multiple …


Enhancing Mobile App Bug Reporting via Real-time Understanding of Reproduction Steps

M Fazzini, K Moran, CB Cardenas, T Wendland, A Orso… - arXiv preprint arXiv …, 2022

… The module uses the FastText model trained on the Common Crawl dataset [39] to identify the semantic similarity between S2R descriptions and GUI element properties. The web application also provides access to bug reports saved in a …


Enhancing the Performance of Text Mining

The amount of text data produced in science, finance, social media, and medicine is growing at an unprecedented pace. The raw text data typically introduces major computational and analytical obstacles (eg, extremely high dimensionality) to data …


Enhancing Vietnamese Question Generation with Reinforcement Learning

N Vu, K Van Nguyen - Asian Conference on Intelligent Information and …, 2022

Along with automatic question answering and machine reading comprehension, question generation has become a popular yet challenging task of natural language understanding in recent years. However, as far as we are concerned, there was no …


Enriching Grammatical Error Correction Resources for Modern Greek

… T5 was pre-trained on 750GB of Englishlanguage text that was sourced from the public Common Crawl web.MT5 was pre-trained on data from all of the 71 monthly web scrapes released so far by Common Crawl, which is more than the source data …


Ensembling of Various Transformer Based Models for the Fake News Detection Task in the Urdu Language

S Kalraa, P Vermaa, Y Sharmaa, GS Chauhanb

The spread of misinformation has become a severe issue affecting society. Inaccurate information has enormous potential to cause real-world impacts. Developing algorithms to detect fake news automatically will be very useful in …


Entity-Assisted Language Models for Identifying Check-worthy Sentences

T Su, C Macdonald, I Ounis - arXiv preprint arXiv:2211.10678, 2022

We propose a new uniform framework for text classification and ranking that can automate the process of identifying check-worthy sentences in political debates and speech transcripts. Our framework combines the semantic analysis of the sentences …


Entry and Acquisitions in Software Markets

How do acquisitions of young, innovative, venture capital funded firms (startups) affect entry incentives of new firms? I create a new dataset by merging web-scraped product-level data on enterprise software with data on firms’ entry and acquisition …


Epidemic Question Answering: question generation and entailment for Answer Nugget discovery

MA Weinzierl, SM Harabagiu - Journal of the American Medical Informatics …, 2022

Objective The rapidly growing body of communications during the COVID-19 pandemic posed a challenge to information seekers, who struggled to find answers to their specific and changing information needs. We designed a Question …


Equivocal URLs: Understanding the Fragmented Space of URL Parser Implementations

J Reynolds, A Bates, M Bailey - European Symposium on Research in Computer …, 2022

Uniform Resource Locators (URLs) are integral to the Web and have existed for nearly three decades. Yet URL parsing differs subtly among parser implementations, leading to ambiguity that can be abused by attackers. We measure agreement …


ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages

Y Chai, S Wang, C Pang, Y Sun, H Tian, H Wu - arXiv preprint arXiv:2212.06742, 2022

… 25 languages from CommonCrawl. It learns to reconstruct the full NL texts from corrupted ones with an arbitrary noising function. mT5 (Xue et al., 2021) is a multilingual-NL encoder-decoder model adapted from T5. It is trained on 101 NLs …


esCorpius: A Massive Spanish Crawling Corpus

A Gutiérrez-Fandiño, D Pérez-Fernández… - arXiv e-prints, 2022

… In this paper, we introduce esCorpius, a Spanish crawling corpus obtained from near 1 Pb of Common Crawl data. It is the most extensive corpus in Spanish with this level of quality in the extraction, purification and deduplication of web textual content …


Estimating Confidence of Predictions of Individual Classifiers and Their Ensembles for the Genre Classification Task

… It has a similar architecture but uses a bigger and more genre-diverse corpus based on Common Crawl (instead of Wikipedia for the multilingual BERT). Therefore, we choose XLM-RoBERTa as the classifier for the experiments in our research. One …


Estimating Intelligence Quotient Using Stylometry and Machine Learning Techniques: A Review

GO Adebayo, RV Yampolskiy - Big Data Mining and Analytics, 2022

… the curve of the collegiate word ratio (CWR) — the ratio of the total count of collegiate words (words SAT consider a part of strong vocabulary usage) used in a written text to the total count of words in the text, of sample texts with more than 100 …


Estimating time-series changes in social sentiment@ Twitter in US metropolises during the COVID-19 pandemic

R Saito, S Haruyama - Journal of Computational Social Science, 2022

Since early 2020, the global coronavirus pandemic has strained economic activities and traditional lifestyles. For such emergencies, our paper proposes a social sentiment estimation model that changes in response to infection conditions and …


Estimation on the Importance of Semantic Web Integration for Art and Culture Related Online Media Outlets

A Giannakoulopoulos, M Pergantis, N Konstantinou… - Future Internet, 2022

… Such a list was acquired through Common Crawl a “non-profit organization dedicated to … This process, which was developed in PHP, used Common Crawl’s API to receive a list of domains … The language of a website was identified based …


Estonian Language Understanding: a Case Study on the COPA Task

The lack of Estonian NLU datasets severely affects advancing Estonian-specific NLP research. With this paper we aim to relieve the issue by publishing a new Estonian NLU dataset EstCOPA. We benchmark the task on several Estonian and multilingual …


Ethnic Bias in Judicial Decision-making: Evidence from Criminal Appeals in Kenya

… ), we used the Common Crawl GLoVe model that was trained using a . million word vocabulary. Rodriguez and Spirling (Nd) further nd that pretrained word vectors perform well against both locally trained vectors and human coders. …


eTranslation's Submissions to the WMT22 General Machine Translation Task

C Oravecz, K Bontcheva, D Kolovratník, B Kovachev…

The paper describes the 3 NMT models submitted by the eTranslation team to the WMT22 general machine translation shared task. In the WMT news task last year, multilingual systems with deep and complex architectures utilizing immense …


EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention

C Tang, C Lin, H Huang, F Guerin, Z Zhang - arXiv preprint arXiv:2210.12463, 2022

One of the key challenges of automatic story generation is how to generate a long narrative that can maintain fluency, relevance, and coherence. Despite recent progress, current story generation systems still face the challenge of how to …


European Language Equality-Report on the French Language

G Adda, A Braffort, I Vasilescu, F Yvon - 2022

This report presents a survey of the current state of technologies for the automatic processing of the French language as well as French Sign Language (FSL). Similar reports have been prepared independently for all languages of the European Union …


Evaluate and Visualize Legal Embeddings for Explanation Purpose

NH Thanh, DT Binh, BM Quan, N Le Minh - 2021 13th International Conference on …, 2021

Deep learning models have recently demonstrated promising results on legal-related tasks. However, the performance of the models is not enough for us to have a clear understanding of their behavior and the reason for such performance. This paper is …


Evaluate Similarity of Requirements with Multilingual Natural Language Processing

U Bisang, J Brünnhäußer, P Lünnemann, L Kirsch… - Proceedings of the Design …, 2022

Finding redundant requirements or semantically similar ones in previous projects is a very time-consuming task in engineering design, especially with multilingual data. Due to modern NLP it is possible to automate such tasks. In this paper we compared …


Evaluating Computational Models of Similarity against a Human Rated Dataset

… The English distributional models were trained on Wikipedia and on the Common Crawl corpus containing English text. The semantic … a language identifier from Common Crawl and Wikipedia. The model is trained with a vector size of 100 …


Evaluating Pre-Trained Transformers on Italian Administrative Texts

S Auriemma, M Miliani, A Bondielli, LC Passaro… - 2022

In recent years, Transformer-based models have been widely used in NLP for various downstream tasks and in different domains. However, a language model explicitly built for the Italian administrative language is still lacking. Therefore, in this …


Evaluating Pre-training Objectives for Low-Resource Translation into Morphologically Rich Languages

The scarcity of parallel data is a major limitation for Neural Machine Translation (NMT) systems, in particular for translation into morphologically rich languages (MRLs). An important way to overcome the lack of parallel data is to leverage target monolingual …


Evaluating the Construct Validity of Text Embeddings with Application to Survey Questions

Q Fang, D Nguyen, DL Oberski - arXiv preprint arXiv:2202.09166, 2022

Text embedding models from Natural Language Processing can map text data (eg words, sentences, documents) to supposedly meaningful numerical representations (aka text embeddings). While such models are increasingly applied in social science …


Evaluating the Impact of OCR Quality on Short Texts Classification Task

O Vitman, Y Kostiuk, P Plachinda, A Zhila, G Sidorov… - … International Conference on …, 2022

… As those embeddings have been trained on different kinds of text corpora ranging from Wikipedia to CommonCrawl Corpus Footnote 7 , they … On the other hand, Common Crawl Corpus is a huge corpus of a large chunk of the Internet, and the …


Evaluating Topic Modeling Pre-processing Pipelines for Portuguese Texts

APDS Júnior, P Cecilio, F Viegas, W Cunha… - Brazilian Symposium on …, 2022

… De acordo com [31], os autores utilizam os vetores pré-treinados do FastText – modelo pré-treinado a partir do corpus Common Crawl. Dessa forma, adaptamos as CluWords para representação de textos em português utilizando os vetores …


Evaluating transfer learning approach for detecting Arabic anti-refugee/migrant speech on social media

D Mohdeb, M Laifa, F Zerargui, O Benzaoui - Aslib Journal of Information …, 2022

Purpose The present study was designed to investigate eight research questions that are related to the analysis and the detection of dialectal Arabic hate speech that targeted African refugees and illegal migrants on the YouTube Algerian space …


Evaluation and Analysis of the NLP Model Zoo for Ukrainian Text Classification

D Panchenko, D Maksymenko, O Turuta, A Yerokhin… - International Conference on …, 2022

One of the crucial problems of natural language processing for languages such as Ukrainian is lack of datasets both unlabeled (for pretraining of word embeddings or large deep learning models) and labeled (for benchmarking existing approaches). In …


Evaluation Methods for Representation Learning: A Survey

K Nozawa, I Sato - IJCAI-ECAI Survey Track, 2022

Abstract Representation learning enables us to automatically extract generic feature representations from a dataset to solve another machine learning task. Recently, extracted feature representations by a representation learning algorithm and a …


Evaluation of an automated Presidio anonymisation model for unstructured radiation oncology electronic medical records in an Australian setting

DP Kotevsk, RI Smee, M Field, YN Nemes, K Broadley… - International Journal of …, 2022

Background Electronic medical records (EMRs) contain valuable information for clinical research, however, the presence of personally identifying information (PII) restricts their use. Anonymisation of PII from EMRs enables clinical information to be …


Evaluation of Transfer Learning for Polish with a Text-to-Text Model

A Chrabrowa, Ł Dragan, K Grzegorczyk, D Kajtoch… - arXiv preprint arXiv …, 2022

We introduce a new benchmark for assessing the quality of text-to-text models for Polish. The benchmark consists of diverse tasks and datasets: KLEJ benchmark adapted for text-to-text, en-pl translation, summarization, and question answering. In …


Evaluation of Word Embedding Models in Latvian NLP Tasks Based on Publicly Available Corpora

R Laucis, G Jēkabsons - Applied Computer Systems, 2021

Nowadays, natural language processing (NLP) is increasingly relaying on pre-trained word embeddings for use in various tasks. However, there is little research devoted to Latvian–a language that is much more morphologically complex than English. In …


Event prediction from news text using subgraph embedding and graph sequence mining

RF Cekinel, P Karagoz - World Wide Web, 2022

Event detection from textual content by using text mining concepts is a well-researched field in the literature. On the other hand, graph modeling and graph embedding techniques in recent years provide an opportunity to represent textual contents as …


Expanding Knowledge Graphs with Humans in the Loop

E Manzoor, J Tong, S Vijayaraghavan, R Li - arXiv preprint arXiv:2212.05189, 2022

… The PinText embeddings are trained on millions of tokens from the Pinterest platform8, whereas the FastText embeddings are trained on 6.3 billion tokens from the Common Crawl website archive. Thus, the two embedding approaches have …


Experimental Comparison of Ansätze for Quantum Natural Language Processing

The DisCoCat model of natural language is a framework which serves as the foundation for several contemporary experiments in Quantum Natural Language Processing (QNLP). In this model, sentences are represented as string diagrams …


Experimental Study on Backwash of Tsunami Acting on Caisson-Type Quay Wall

Documents in HENRY are made available under the Creative Commons License CC BY 4.0, if no other license is applicable. Under CC BY 4.0 commercial use and sharing, remixing, transforming, and building upon the material of the work is …


Explainable detection of adverse drug reaction with imbalanced data distribution

J Wang, LC Yu, X Zhang - PLOS Computational Biology, 2022

… For BiLSTM, the word vectors were pre-trained using GloVe on the 840B Common Crawl corpus [28]. The dimensionality of the word vectors is 300. Words that don’t appear in GloVe were initialized with a uniform distribution U(−0.25, 0.25). …


Exploiting Document-based Features for Clarification in Conversational Search

I Sekulić, M Aliannejadi, F Crestani - European Conference on Information Retrieval, 2022

… For that purpose, we utilise Chatnoir [7] – a freely accessible Elasticsearch-based search engine with indexes of ClueWeb and CommonCrawl … Elastic ChatNoir: search engine for the ClueWeb and the common crawl. In: Pasi, G., Piwowarski, B …


Exploiting Ensemble of Transformer Models for Detecting Informative Tweets

AN Chowdhury, S Guha, N Amin, SI Khan - 2021 IEEE 18th India Council …, 2021

Microblogging platforms especially Twitter is considered as one of the prominent medium of getting user-generated information. Millions of tweets were posted daily during COVID-19 pandemic days and the rate increases gradually. Tweets include a …


Exploiting Linguistic Information from Nepali Transcripts for Early Detection of Alzheimer's Disease using the State-of-the-art Techniques of Machine Learning and …

S Adhikari, S Thapa, U Naseem, P Singh, A Huo…

… Similarly, for the pre-trained fastText embeddings, the pre-trained word vectors trained on Common Crawl and Wikipedia using fastText were used [39]. The model was trained by using CBOW with position-weights, in dimension 300, with character …


Exploration of text features representation on applications to the insurance domain

The recent popularization of a more data-centric paradigm in machine learning has turned the study of the quality of feature representation into a very relevant topic. In this document an exploration of different text features representation applied to a …


Exploring Automatic Text Simplification of Requirements

… The mined paraphrases are sequences of words with the same meaning extracted from a large collection of texts from Common Crawl7, an open-source collection of web crawl data. The system simplifies text by using the mined …


Exploring Capabilities of Monolingual Audio Transformers using Large Datasets in Automatic Speech Recognition of Czech

J Lehečka, J Švec, A Pražák, JV Psutka - arXiv preprint arXiv:2206.07627, 2022

… is a huge collection of cleaned and deduplicated web pages from Common Crawl project4. Since this text corpus contains almost 13 billion … • adding LM into the beam search CTC decoder; we found the Common Crawl project to be a useful …


Exploring Dimensionality Reduction Techniques in Multilingual Transformers

Á Huertas-García, A Martín, J Huertas-Tato… - arXiv preprint arXiv …, 2022

Both in scientific literature and in industry,, Semantic and context-aware Natural Language Processing-based solutions have been gaining importance in recent years. The possibilities and performance shown by these models when dealing with …


Exploring Models for Automatic Keyword Labelling of Scientific Documents

Automatic keyword labelling methods generate a set of short phrases for a given document providing a short and good description of its content. Those labels are critical in tasks such as exploratory search and for improving the information …


Exploring Swedish & English fastText Embeddings

In this paper, we show that embeddings from relatively smaller corpora sometimes outperform those from larger corpora and we introduce a new Swedish analogy test set and make it publicly available. To achieve good performance in Natural …


Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping

J He, X Li, D Yu, H Zhang, J Kulkarni, YT Lee… - arXiv preprint arXiv …, 2022

Differentially private deep learning has recently witnessed advances in computational efficiency and privacy-utility trade-off. We explore whether further improvements along the two axes are possible and provide affirmative answers …


Exploring Transformers models for Emotion Recognition: a comparision of BERT, DistilBERT, RoBERTa, XLNET and ELECTRA

D Cortiz - 2022 3rd International Conference on Control …, 2022

This paper investigates the performance of different Transformers Models for Emotion Recognition, a specific NLU (Natural Language Understanding) task in Affective Computing. We implemented and finetuned different transformers …


Exposing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders

I Vulić, G Glavaš, F Liu, N Collier, EM Ponti… - arXiv preprint arXiv …, 2022

Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; eg, LaBSE, xMPNET) via additional fine-tuning or model distillation on parallel data. However, it remains uncertain how to best …


Exposing the many biases in machine learning

S Richardson - Business Information Review, 2022

In recent years, there have been numerous articles highlighting issues with bias in machine learning algorithms underpinning the use of AI in decision making. Specifically, algorithms trained on historical real-world observations. However, less …


Exposure and Emergence in Usage-Based Grammar: Computational Experiments in 35 Languages

J Dunn - arXiv preprint arXiv:2211.14160, 2022

This paper uses computational experiments to explore the role of exposure in the emergence of construction grammars. While usage-based grammars are hypothesized to depend on a learner's exposure to actual language use, the …


Extended Overview of HIPE-2022: Named Entity Recognition and Linking in Multilingual Historical Documents

M Ehrmann, M Romanello, S Najem-Meyer, A Doucet… - Working Notes of CLEF, 2022

This paper presents an overview of the second edition of HIPE (Identifying Historical People, Places and other Entities), a shared task on named entity recognition and linking in multilingual historical documents. Following the success of the first CLEF-HIPE-2020 …


Extracting Cultural Commonsense Knowledge at Scale

TP Nguyen, S Razniewski, A Varde, G Weikum - arXiv preprint arXiv:2210.07763, 2022

Structured knowledge is important for many AI applications. Commonsense knowledge, which is crucial for robust human-centric AI, is covered by a small number of structured knowledge projects. However, they lack knowledge about …


Extracting Impact Model Narratives from Social Services' Text

Named entity recognition (NER) is an important task in narration extraction. Narration, as a system of stories, provides insights into how events and characters in the stories develop over time. This paper proposes an architecture for NER on a corpus …


Extractive and Abstractive Summarization Methods for Financial Narrative Summarization in English, Spanish and Greek

A Vaca, A Segurado, D Betancur, A Barbero - Proceedings of the 4th Financial …, 2022

This paper describes the three summarization systems submitted to the Financial Narrative Summarization Shared Task (FNS-2022). We developed a task-specific extractive summarization method for the reports in English. It was based on a …


Extractive Question Answering on Queries in Hindi and Tamil

A Thirumala, E Ferracane - arXiv preprint arXiv:2210.06356, 2022

… This is achieved by training the model on a large corpus of webcrawl data called the Common Crawl. The Common Crawl is a dataset that contains over 7 years of webcrawl data in over 40 languages. The question answering head applied to XLM- …


Extreme Multilabel Text Classification on Indonesian Tax Court Ruling using Single Channel CNN and IndoBERT Embedding

IN Khasanah, AA Krisnadhi - 2021 6th International Workshop on Big Data and …, 2021

Manual searching for legal basis such as paragraphs, articles, and laws when preparing for a tax court hearing is time-consuming. In this paper, we use extreme multilabel text classification approach to predict paragraphs, articles, and laws …


Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videos

A Waibel, M Behr, FI Eyiokur, D Yaman, TN Nguyen… - arXiv preprint arXiv …, 2022

In this paper, we propose a neural end-to-end system for voice preserving, lip-synchronous translation of videos. The system is designed to combine multiple component models and produces a video of the original speaker speaking in the target …


Fact-checking Multidimensional Statistic Claims in French

O Balalau, S Ebel, T Galizzi, A Deiana, E Gautreau… - TTO 2022-Truth and Trust …, 2022

To strengthen public trust and counter disinformation, computational fact-checking, leveraging digital data sources, attracts interest from the journalists and the computer science community. A particular class of interesting data sources …


FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization

D Wan, M Bansal - arXiv preprint arXiv:2205.07830, 2022

We present FactPEGASUS, an abstractive summarization model that addresses the problem of factuality during pre-training and fine-tuning: (1) We augment the sentence selection strategy of PEGASUS's (Zhang et al., 2020) pre-training objective …


Factual Consistency of Multilingual Pretrained Language Models

C Fierro, A Søgaard - arXiv preprint arXiv:2203.11552, 2022

Pretrained language models can be queried for factual knowledge, with potential applications in knowledge base acquisition and tasks that require inference. However, for that, we need to know how reliable this knowledge is, and recent work …


Fair and Optimal Classification via Transports to Wasserstein-Barycenter

R Xian, L Yin, H Zhao - arXiv preprint arXiv:2211.01528, 2022

… The task is to determine the occupation (out of 28) of female and male individuals by their biography in text; the biographies, occupations and genders are mined from the Common Crawl corpus. Gender is the sensitive attribute, and to which certain …


Fake news detection: When complex problems demand complex solutions

C Oliva, IP Marín, LF Lago-Fernández… - 2021

Fake news detection is one of the most challenging problems in today's information and communication systems. In this article we address the challenge of detecting the generation and spreading of misleading information in the specific scenario of …


Fake or Genuine? Contextualised Text Representation for Fake Review Detection

R Mohawesh, S Xu, M Springer, M Al-Hawawreh… - arXiv preprint arXiv …, 2021

… Along with the English Wikipedia and book corpora, RoBERTa is pre-trained on the Common Crawl News dataset, comprising 63 million English-language news stories. RoBERTa base architecture consists of 768 hidden layers, 12 layers, 125 …


FastClass: A Time-Efficient Approach to Weakly-Supervised Text Classification

T Xia, Y Wang, Y Tian, Y Chang - arXiv preprint arXiv:2212.05506, 2022

Weakly-supervised text classification aims to train a classifier using only class descriptions and unlabeled data. Recent research shows that keyword-driven methods can achieve state-of-the-art performance on various tasks. However, these …


FCM: Forgetful Causal Masking Makes Causal Language Models Better Zero-Shot Learners

H Liu, X Geng, L Lee, I Mordatch, S Levine, S Narang… - arXiv preprint arXiv …, 2022

Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing impressive zero-shot and few-shot capabilities across a wide …


FD-GATDR: A Federated-Decentralized-Learning Graph Attention Network for Doctor Recommendation Using EHR

L Bi, Y Wang, F Zhang, Z Liu, Y Cai, E Zhao - arXiv preprint arXiv:2207.05750, 2022

… The GloVe model is loaded with the pretrained parameters on Common Crawl which has 840B tokens and 2.2M vocab [11]. Each word is converted to a 300-dimension vector. Then the model is fine-tuned with LSTM for the classification. …


Feasibility study of long-form question answering

Long-form question answering (LFQA) is a challenging natural language processing task as it asks to produce paragraph-length answers to questions. Meaningful progress in LFQA would significantly speed up how fast people can seek information …


Feature-based Question Routing in Community Question Answering Platforms

S Sorkhani, R Etemadi, A Bigdeli, M Zihayat, E Bagheri - Information Sciences, 2022

… To do so, we use the Fasttext model with the pre-trained embeddings on the Common Crawl dataset 4 to obtain embedding vectors for question’s tags and tags of the questions answered by the user. Subsequently, we compute this feature for …


Few shot learning with fine-tuned language model for suicidal text detection

S Varma, S Shivam, B Ray, A Banerjee - 2022

… The corpus used for pre-training the language model has been obtained from Wikipedia and Common crawl web scrapes which are vast and prone to have fairly low quality. Therefore, cleaning methods like deduplication, removal of incomplete …


Few-shot learning with language models: Learning from instructions and contexts

Pretraining deep neural networks to perform language modeling–that is, to reconstruct missing words from incomplete pieces of text–has brought large improvements throughout natural language processing (NLP). However, even …


Few-shot Learning with Retrieval Augmented Language Models

G Izacard, P Lewis, M Lomeli, L Hosseini, F Petroni… - arXiv preprint arXiv …, 2022

… Another explanation might be that Wikipedia is a higher-quality and denser source of knowledge than common crawl. Second, when using a common crawl index, we observe that pre-training on Wikipedia data leads to lower performance …


Few-Shot Natural Language Processing by Meta-Learning Without Labeled Data

Humans show a remarkable capability to accurately solve a wide range of problems efficiently--utilizing a limited amount of computation and experience. Deep learning models, by stark contrast, can be trained to be highly accurate on a narrow task …


Fight Against COVID-19 Misinformation via Clustering-Based Subset Selection Fusion Methods

Y Huang, Q Xu, S Wu, C Nugent, A Moore - 2022

The worldwide COVID-19 pandemic has brought about a lot of changes in people’s life. It also emerges as a new challenge to information search services. This is because up to now our understanding about the virus is still limited, and there is a lot …


Finding and Fixing Undesirable Behaviors in Pretrained Language Models

In machine learning, we often do not know how to specify the desired objective for our learning algorithms. For example, we typically train models to answer questions correctly by using a proxy objective: training models to answer questions as people …


Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks

V Raunak, A Menezes - arXiv preprint arXiv:2210.12929, 2022

Memorization presents a challenge for several constrained Natural Language Generation (NLG) tasks such as Neural Machine Translation (NMT), wherein the proclivity of neural models to memorize noisy and atypical samples reacts adversely …


Findings of the 2022 Conference on Machine Translation (WMT22)

T Kocmi, R Bawden, O Bojar, A Dvorkovich…

This paper presents the results of the General Machine Translation Task organised as part of the Conference on Machine Translation (WMT) 2022. In the general MT task, participants were asked to build machine translation systems for any of 11 …


Findings of the WMT 2022 Biomedical Translation Shared Task: Monolingual Clinical Case Reports

M Neves, AJYA Siu, R Roller, P Thomas, MV Navarro…

In the seventh edition of the WMT Biomedical Task, we addressed a total of seven language pairs, namely English/German, English/French, English/Spanish, English/Portuguese, English/Chinese, English/Russian, English/Italian. This year’s test sets covered …


Findings of the WMT 2022 shared task on large-scale machine translation evaluation for african languages

DI Adelani, MMI Alam, A Anastasopoulos, A Bhagia… - Proceedings of the Seventh …, 2022

… We then performed bitext mining against 21.5 billion English sentences from Common Crawl, and 3.3 million sentences in French, respectively. These resources as well as the sentence encoders were made available to the participants of this …


Fine-grained Czech News Article Dataset: An Interdisciplinary Approach to Trustworthiness Analysis

M Boháček, M Bravanský, F Trhlík, V Moravec - arXiv preprint arXiv:2212.08550, 2022

We present the Verifee Dataset: a novel dataset of news articles with fine-grained trustworthiness annotations. We develop a detailed methodology that assesses the texts based on their parameters encompassing editorial transparency, journalist …


Fine-grained semantic type discovery for heterogeneous sources using clustering

F Piai, P Atzeni, P Merialdo, D Srivastava - The VLDB Journal, 2022

We focus on the key task of semantic type discovery over a set of heterogeneous sources, an important data preparation task. We consider the challenging setting of multiple Web data sources in a vertical domain, which present sparsity of data and a …


Fine-tuning and multilingual pre-training for abstractive summarization task for the Arabic language

M Kahla, A Novák, ZG Yang - Annales Mathematicae et Informaticae

… We chose to download Arabic Deutsche Welle resources from Common Crawl4, as this solution does not interfere with the site. A very positive aspect of this resource is that the items downloaded from the DW news website address a wide range of …


Fine-tuning Pre-trained Language Models with Noise Stability Regularization

H Hua, X Li, D Dou, CZ Xu, J Luo - arXiv preprint arXiv:2206.05658, 2022

The advent of large-scale pre-trained language models has contributed greatly to the recent progress in natural language processing. Many state-of-the-art language models are first trained on a large text corpus and then fine-tuned on downstream …


FINE-TUNING TRANSFORMERS: ASSESSMENTS OF

… in the development of pre-trained systems such as BERT [4](Bidirectional Encoder Representations from Transformers) and GPT [23](Generative Pre-trained Transformer), which was trained on large language datasets such as the Wikipedia …


FitLayout: An RDF-Based Framework and Toolkit for Web Page Content Analysis

Despite the ongoing development of technologies that allow the publication of structured information within web pages, there still exist many web sources that publish useful data solely in the form of plain HTML documents. The data published …


FLightNER: A Federated Learning Approach to Lightweight Named-Entity Recognition

We introduce FLightNER, a Federated Learning model that extends a state-of-the-art Named-Entity Recognition model using prompt-tuning known as LightNER. FLightNER allows the aggregation of only the trainable parameters of LightNER …


Forecasting Future World Events with Neural Networks

A Zou, T Xiao, R Jia, J Kwon, M Mazeika, R Li, D Song… - arXiv preprint arXiv …, 2022

… To this end, we provide a corpus of news articles scraped from CommonCrawl news (Nagel, 2016; Hamborg et al., 2017) that is organized by publish date. The articles were generated between 2016 to mid-2022 and total more than 200GB of …


Formal Mathematics Statement Curriculum Learning

S Polu, JM Han, K Zheng, M Baksys, I Babuschkin…

We explore the use of expert iteration in the context of language modeling applied to formal mathematics. We show that at same compute budget, expert iteration, by which we mean proof search interleaved with learning, dramatically outperforms …