Meta-Learning for Few-Shot NMT Adaptation

A Sharaf, H Hassan, H Daumé III - arXiv preprint arXiv:2004.02745, 2020

… In all cases, the baseline machine translation system is a neural English to German (En-De) transformer model (Vaswani et al., 2017), initially trained on 5.2M sentences filtered from the the standard parallel data …


Method and apparatus for improved automatic subtitle segmentation using an artificial neural network model

P WILKEN, E Matusov - US Patent App. 16/876,780, 2020

… These data included all other publicly available training data, including ParaCrawl, CommonCrawl, EUbookshop, JRCAcquis, EMEA, and other corpora from the OPUS collection … This may be done to avoid oversampling …


Method and system for interactive keyword optimization for opaque search engines

R Puzis, A ELYASHAR, M REUBEN - US Patent App. 16/840,538, 2020

… 2018]. The model was trained on Common Crawl (http://commoncrawl.org/) and Wikipedia (https://www.wikipedia.org/) using fastText library (https://fasttext.cc/). For the distance measure, the simple Euclidean distance was used …


G Gottlob, E SALLINGER, R FAYZRAKHMANOV… - US Patent App. 16/630,485, 2020

US20200167393A1 - Method for automatically generating a wrapper for extracting web data, and a computer system - Google Patents. Method for automatically generating a wrapper for extracting web data, and a computer system. Download PDF Info …


Methods for morphology learning in low (er)-resource scenarios

Page 1. This thesis has been submitted in fulfilment of the requirements for a postgraduate degree (eg PhD, MPhil, DClinPsychol) at the University of Edinburgh. Please note the following terms and conditions of use: This work …


Page 1. FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO Metrics and tools for exploring toxicity in social media Pedro Silva Mestrado Integrado em Engenharia Informática e Computação Supervisor: Sérgio …


Mis-shapes, Mistakes, Misfits: An Analysis of Domain Classification Services

P Vallina, V Le Pochat, Á Feal, M Paraschiv, J Gamba…

… service. Their popularity is further reflected by the fact that 47% of the 4.4M domains are indexed in the Chrome User Experience Report [71] and 0.5% by Common Crawl [72], both generated between August and October 2019 …


Mitigating Bias in Deep Nets with Knowledge Bases: the Case of Natural Language Understanding for Robots

M Mensio, E Bastianelli, I Tiddi, G Rizzo

… The connections in green represent highway connections be- tween the first and the third layer. over the Common Crawl resource3 … For this reason, it can intrinsically provide an ex- planation for the model behavior, as it summarizes a much 3http://commoncrawl.org …


Mitigating Gender Bias in Machine Learning Data Sets

S Leavy, G Meaney, K Wade, D Greene - arXiv preprint arXiv:2005.06898, 2020

… evaluation of system accuracy and learned associations in machine learning technologies that underlie many search and recommendation systems [9]. Implicit Association Tests (IATs) were found to be effective in uncovering …


Page 1. Modeling Recurring Concepts in Single-label and Multi-label Streams A thesis submitted for the degree of DN at the Department of Physics, Mathematics and Computer Science at the Johannes …


Modeling remotely collected speech data: Applications for psychiatry

… To base the analysis on a corpus with a wide variety of animal-word sources, we used a set of pretrained word vectors calculated from approximately 42 billion tokens from the entire internet, courtesy of the Common Crawl project (Pennington et al., 2014) …


Modeling the Music Genre Perception across Language-Bound Cultures

EV Epure, G Salha, M Moussallam, R Hennequin - arXiv preprint arXiv:2010.06325, 2020

… representations as described next. Multilingual Static Word Embeddings. The classical word embeddings we study are the multilingual fastText word vectors trained on Wikipedia and Common Crawl (Grave et al., 2018). The model is an …


Modeling Word Formation in English–German Neural Machine Translation

M Weller-Di Marco, A Fraser - Proceedings of the 58th Annual Meeting of the …, 2020

… We compare four training settings: small (248,730 sentences: newscommentary), large2M (1,956,444 sentences: Europarl + news-commentary), large4M (4,116,215 sentences: Europarl + news-commentary + …


Moral Concerns are Differentially Observable in Language

B Kennedy, M Atari, AM Davani, J Hoover, A Omrani… - 2020

… lexical semantic relatedness. We compute text representations by averaging GloVe word embedding vectors (Pennington et al., 2014) which were trained on text from the Common Crawl3. In 2 https://github.com/lda-project/lda …


Moral Framing and Ideological Bias of News

K Lerman - … : 12th International Conference, SocInfo 2020, Pisa …

N Mokhberian, A Abeliuk, P Cummings, K Lerman - arXiv preprint arXiv:2009.12979, 2020

… then and V the m− denote semantic to axis set corresponding to this MF dimension is: Am= mean (V m+)− mean (V m−)(1) For the computations of this part, the embeddings of words are obtained from the pretrained GloVe …


Morphological and pseudomorphological effects in English visual word processing: How much can we attribute the statistical structure of the language?

… Real-valued 300dimensional semantic vectors generated from the Common Crawl internet text corpus were converted to 200-dimensional binary vectors using a binary multidimensional scaling algorithm (Rohde, 2002) …


Morphological Skip-Gram: Using morphological knowledge to improve word representation

F Santos, H Macedo, T Bispo, C Zanchetting - arXiv preprint arXiv:2007.10055, 2020

… Keeping the quality of word embeddings and decreasing training time is very important because usually, a corpus to training embeddings is composed of 1B tokens. For example, the Common Crawl corpora contain 820B tokens …


mT5: A massively multilingual pre-trained text-to-text transformer

A Roberts, A Barua, A Siddhant, C Raffel, L Xue… - 2021

L Xue, N Constant, A Roberts, M Kale, R Al-Rfou… - arXiv preprint arXiv …, 2020

… In this paper, we in- troduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages … mC4 comprises natural text in 101 languages drawn from the public Common Crawl web scrape …


Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance

SF Yilmaz, EB Kaynak, A Koç, H Dibeklioğlu, SS Kozat - arXiv preprint arXiv …, 2020

… 2, we use XLM-RoBERTa pretrained tokenizer and pretrained model [17]. XLM-RoBERTa is pretrained on CommonCrawl corpora of 100 different languages. We first tokenize the input sentence si into subword units via …


Multi-model transfer and optimization for cloze task

J Tang, L Ling, C Ma, H Zhang, J Huang - … on Artificial Intelligence and Robotics 2020, 2020

… Transformer Enc. PLM ≈ BERT WikiEn+BookCorpus+Giga5 +ClueWeb+Common Crawl RoBERTa Transformer Enc. MLM 355M BookCorpus+CCNews +OpenWebText+STORIES XLM-R Transformer Enc. MLM 550M CommonCrawl ALBERT Transformer Enc …


I Vulić, S Baker, EM Ponti, U Petti, I Leviant, K Wing… - arXiv preprint arXiv …, 2020

Page 1. Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity https://multisimlex.com/ Ivan Vulic ∗♠ LTL, University of Cambridge Simon Baker ∗♠ LTL, University of Cambridge …


Multilingual AMR-to-Text Generation

A Fan, C Gardent - Proceedings of the 2020 Conference on Empirical …, 2020

… 4.1 Data Pretraining For encoder pretraining on silver AMR, we take thirty million sentences from the English portion of CCNET 2 (Wenzek et al., 2019), a cleaned version of Common Crawl (an open source version of the web) …


Multilingual Denoising Pre-training for Neural Machine Translation

Y Liu, J Gu, N Goyal, X Li, S Edunov, M Ghazvininejad… - arXiv preprint arXiv …, 2020

… 2 Multilingual Denoising Pre-training We use a large-scale common crawl (CC) corpus (§2.1) to pre-train BART models (§2.2). Our ex- periments in the later sections involve finetuning a range of models pre-trained on different subsets of the CC languages §2.3) …


Multilingual Dependency Parsing from Universal Dependencies to Sesame Street

J Nivre - International Conference on Text, Speech, and …, 2020

… pre-trained models provided by Che et al. [3], who train ELMo on 20 million words randomly sampled from raw WikiDump and Common Crawl datasets for 44 languages. For BERT, we employ the pretrained multilingual cased …


Z Jiang, A Anastasopoulos, J Araki, H Ding, G Neubig - arXiv preprint arXiv …, 2020

Page 1. X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models Zhengbao Jiang†, Antonios Anastasopoulos♣,∗, Jun Araki‡, Haibo Ding‡, Graham Neubig† †Languages Technologies Institute …


Multilingual Legal Information Retrieval System for Mapping Recitals and Normative Provisions

AK JOHN - Legal Knowledge and Information Systems: JURIX …, 2020

… cc/docs/en/crawl-vectors. html): pre-trained on Common Crawl and Wikipedia. We used the word-average method, which divides the sum of the word embeddings in a legal norm by the norm length. The embedding di- mension size was set to 128 …


G Gül Şahin, C Vania, I Kuznetsov, I Gurevych - Computational Linguistics

Page 1. Computational Linguistics Just Accepted MS. https://doi.org/10.1162/ COLI_a_00376 © Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license LINSPECTOR Multilingual Probing Tasks for Word Representations …


Multilingual Stance Detection in Tweets: The Catalonia Independence Corpus

E Zotova, R Agerri, M Nuñez, G Rigau - … of The 12th Language Resources and …, 2020

… Initial experimentation showed that the Common Crawl5 models performed better for our particular task. The Common Crawl models are trained using a Continuous Bag-of-Words (CBOW) architecture with position-weights and …


Multilingual Stance Detection: The Catalonia Independence Corpus

E Zotova, R Agerri, M Nuñez, G Rigau - arXiv preprint arXiv:2004.00050, 2020

… task. The Common Crawl models are trained using a Continuous Bag-of-Words (CBOW) architecture with position-weights and 300 di- mensions on a vocabulary of 2M words … information. 5http://commoncrawl.org/ Page 4. 3.5 …


Multilingual Translation with Extensible Multilingual Pretraining and Finetuning

Y Tang, C Tran, X Li, PJ Chen, N Goyal, V Chaudhary… - arXiv preprint arXiv …, 2020

… challenging to train from scratch. In contrast, monolingual data exists even for low resource languages, particularly in resources such as Wikipedia or Commoncrawl, a version of the web. Thus, leveraging this monolingual …


Multilingual Unsupervised Sentence Simplification

L Martin, A Fan, É de la Clergerie, A Bordes, B Sagot - arXiv preprint arXiv …, 2020

… We mine a large quantity of paraphrases from the Common Crawl using libraries such as LASER (Artetxe et al., 2018) and faiss (Johnson et al … CCNET is an extraction of Common Crawl,2 an open source snapshot of the web …


T Kim, S Lee - arXiv preprint arXiv:2004.13805, 2020

Page 1. Multilingual Zero-shot Constituency Parsing Taeuk Kim and Sang-goo Lee Department of Computer Science and Engineering Seoul National University, Seoul, Korea {taeuk,sglee}@europa.snu.ac.kr Abstract Zero-shot …


MultiMix: A Robust Data Augmentation Strategy for Cross-Lingual NLP

MS Bari, MT Mohiuddin, S Joty - arXiv preprint arXiv:2004.13240, 2020

… samples around each selected sample. XLM-R is a multilingual LM that is trained on massive multilingual corpora (2.5 TB of refined Common-Crawl data in 100 languages) with a masked LM (MLM) objective. We chose XLM-R …


Multiple Knowledge GraphDB (MKGDB)

S Faralli, P Velardi, F Yusifli - Proceedings of The 12th Language Resources and …, 2020

… Crawl11 Web corpus. 7https://www.objectivity.com/products/ thingspan/ thingspanfeatures/ 8https://titan.thinkaurelius.com/ 9https://neo4j.com/ 10the Neo4j platform provides an interface for the development and inclusion of …


Multiscale System for Alzheimer's Dementia Recognition through Spontaneous Speech

E Edwards, C Dognin, B Bollepalli, M Singh…

… Deep Random Forest Setting: We extract features using three pre-trained embeddings: Word2Vec (CBOW) with subword information [29] (pre-trained on Common Crawl), GloVe [30] pre-trained on Common Crawl and Sent2Vec …


MuSe 2020 Challenge and Workshop: Multimodal Sentiment Analysis, Emotion-target Engagement and Trustworthiness Detection in Real-life Media: Emotional Car …

L Stappen, A Baird, G Rizos, P Tzirakis, X Du, F Hafner… - Proceedings of the 1st …, 2020

… 4.3 Language FastText [5] is a library for efficient learning of word embeddings. It is based on the skipgram model where a vector representation is associated to each character n-gram. The model is trained on the English Common Crawl corpus (600B tokens) …


MuSe 2020--The First International Multimodal Sentiment Analysis in Real-life Media Challenge and Workshop

L Stappen, A Baird, G Rizos, P Tzirakis, X Du, F Hafner… - arXiv preprint arXiv …, 2020


Page 1. MWPD2020: Semantic Web Challenge on Mining the Web of HTML-embedded Product Data Ziqi Zhang1[0000−0002−8587−8618], Christian Bizer2[0000−0003−2367−0237], Ralph Peeters2[0000−0003 …


Névszói kötőhangzók variabilitásának korpuszalapú vizsgálata Corpus-based analysis of the variability of linking vowels in nouns and adjectives

… 3.1 Corpus The corpus on which we conducted our measurements is the prepublished version of the Webcorpus 2 (Nemeskey, 2020). It is based on the Common Crawl webcorpus, which is a collection of pages …


Named Entity Recognition for Code-Mixed Indian Corpus using Meta Embedding

R Priyadharshini, BR Chakravarthi, M Vegupatti… - 2020 6th International …, 2020

… IV. EXPERIMENT We use FastText word embedding trained from Common Crawl and Wikipedia [30] for English and Hindi-Devanagari script (native script for Hindi). We also add the English Twitter GloVe word embeddings since the NER data is from Twitter …


Naming unrelated words reliably predicts creativity

JA Olson, J Nahas, D Chmoulevitch, ME Webb - PsyArXiv. December, 2020

… We trained the GloVe model with the Common Crawl corpus, which contains the text of billions of web pages … We chose the GloVe algorithm and the Common Crawl corpus; this combination correlates best with …


Narrative Origin Classification of Israeli-Palestinian Conflict Texts

J Wei, E Santos Jr - The Thirty-Third International Flairs Conference, 2020

… For training and testing, we converted text inputs into nu- merical representations using 300-dimensional distributed embeddings pre-trained on the Common Crawl database with the GloVe method (Pennington, Socher, and Manning 2014) …


Natural Language Correction-Thesis Proposal

… considered. This table displays top 15 languages with worst results on dictionary baseline. a new pipeline utilizing both clean data from Wikipedia and also not that clean data from general web (utilizing CommonCrawl corpus). The …


Natural Language Generation using Transformer Network in an Open-domain Setting

D Varshney, A Ekbal, GP Nagaraja, M Tiwari… - International Conference on …, 2020

… The embeddings used in our model are trained on Common Crawl dataset with 840B tokens and 2.2M vocab. We use 300-dimensional sized vectors. 3.3 Baseline Models We formulate our task of response generation as a machine translation problem …


Natural Language Processing (NLP) and Text Analytics

JM Patel - Getting Structured Data from the Internet, 2020

In the preceding chapters, we have solely relied on the structure of the HTML documents themselves to scrape information from them, and that is a powerful method to extract information.


Natural Language Processing Model for Managing Maintenance Requests in Buildings

Y Bouabdallaoui, Z Lafhaj, P Yim, L Ducoulombier… - Buildings, 2020

… In order to overcome the limited corpus of vocabulary in our dataset, a pre-trained word embedding was used. It is based on the FastText model [44] and trained on a large corpus of French vocabulary from Wikipedia and Common Crawl [45] …


Natural Language Transfer Learning for Physiological Textual Similarity

V Awatramani, P Gupta - 2020 10th International Conference on Cloud …, 2020

… Moreover, RoBERTa is trained over 160 GB of text that includes English Wikipedia and BooksCorporus used earlier in BERT and additionally, CommonCrawl News (CC-NEWS) dataset consisting of 63 million articles …


Naver Labs Europe's Participation in the Robustness, Chat, and Biomedical Tasks at WMT 2020

A Bérard, V Nikoulina, I Calapodescu, J Philip - … of the Fifth Conference on Machine …, 2020

… Corpus Sents Docs Paracrawl 33.9M – Rapid2019 965k 48.3k Europarl 1.75M 6.7k Commoncrawl 1.97M – Wikimatrix 5.68M – Wikititles … of large monolingual English and German datasets (100M lines in total per language …


Nearest Neighbor Machine Translation

U Khandelwal, A Fan, D Jurafsky, L Zettlemoyer… - arXiv preprint arXiv …, 2020

… The parallel sentences are mined from cleaned monolingual commoncrawl data created using the ccNet pipeline (Wenzek et al., 2019) … with only the kNN distribution (λ = 1) with beam size 1, retrieving k = 8 neighbors from …


NestMSA: a new multiple sequence alignment algorithm

M Kayed, AA Elngar - The Journal of Supercomputing, 2020

Multiple sequence alignment (MSA) is a core problem in many applications. Various optimization algorithms such as genetic algorithm and particle swarm opti.


Neural Aspect-based Text Generation

Page 1. November 25, 2020 DRAFT Thesis Proposal Neural Aspect-based Text Generation Hiroaki Hayashi November 25, 2020 Language Technologies Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15123 Thesis Committee …


Neural Databases

J Thorne, M Yazdani, M Saeidi, F Silvestri, S Riedel… - arXiv preprint arXiv …, 2020

Page 1. Neural Databases James Thorne University of Cambridge Facebook AI jt719@cam.ac.uk Majid Yazdani Facebook AI myazdani@fb.com Marzieh Saeidi Facebook AI marzieh@fb.com Fabrizio Silvestri Facebook AI fsilvestri@fb.com …


NEURAL MACHINE TRANSLATION WITH UNIVERSAL VISUAL REPRESENTATION

… We used newsdev2016 as the dev set and newstest2016 as the test set. 2) For the EN-DE translation task, 4.43M bilingual sentence pairs of the WMT14 dataset were used as training data, including Common Crawl, News Commentary, and Europarl v7 …


P Wilken, T Alkhouli, E Matusov, P Golik - arXiv preprint arXiv:2005.14489, 2020

Page 1. arXiv:2005.14489v1 [cs.CL] 29 May 2020 Neural Simultaneous Speech Translation Using Alignment-Based Chunking Patrick Wilken, Tamer Alkhouli, Evgeny Matusov, Pavel Golik Applications Technology (AppTek), Aachen …


Neural Text Segmentation and Its Application to Sentiment Analysis

J Li, B Chiu, S Shang, L Shao - IEEE Transactions on Knowledge and Data …, 2020

Page 1. 1041-4347 (c) 2020 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


Neural Word Embeddings for Sentiment Analysis

Page 1. 1 Page 2. NEURAL WORD EMBEDDINGS FOR SENTIMENT ANALYSIS DUYGU ANAL˙IZ˙I˙IC¸˙IN S˙IN˙IRSEL S¨OZC¨UK¨OZ YERLES¸˙IKLER˙I BEHZAD NADERALVOJOUD PROF. DR. EBRU SEZER Supervisor Submitted to …


NewB: 200,000+ Sentences for Political Bias Detection

J Wei - arXiv preprint arXiv:2006.03051, 2020

… When inputting sentences into the model, we tokenize each sentence at the word-level and convert each word into a vector using 300dimensional distributed embeddings trained on the Common Crawl database with the GloVe method (Pennington et al., 2014) …


News topic classification as a first step towards diverse news recommendation

O De Clercq, L De Bruyne, V Hoste - Computational Linguistics in the Netherlands …, 2020

… The RobBERT model was trained on the Dutch part of the 39GB OSCAR corpus, a part of the Common Crawl corpus (Suárez et al. 2019). As sub-word token input, BERTje uses WordPiece, whereas RobBERT uses byte-level Byte Pair Encoding (BPE) …


NLNDE at CANTEMIST: Neural Sequence Labeling and Parsing Approaches for Clinical Concept Extraction

L Lange, X Dai, H Adel, J Strötgen - arXiv preprint arXiv:2010.12322, 2020

… In particular, we use pre-trained fastText embeddings [22] that were trained on articles from Wikipedia and the Common Crawl, as well as domain-speci c fastText embeddings [23] that were pretrained on articles of the Spanish …


NLP North at WNUT-2020 Task 2: Pre-training versus Ensembling for Detection of Informative COVID-19 English Tweets

AG Møller, R van der Goot, B Plank - Proceedings of the 6th Workshop on Noisy …, 2020

… twitter. ArXiv, abs/2005.07503. Sebastian Nagel. 2016. https://commoncrawl org/2016/10/news-dataset-available/. Dat Quoc Nguyen, Thanh Vu, Afshin Rahimi, Mai Hoang Dao, Linh The Nguyen, and Long Doan. 2020. WNUT …


No computation without representation: Avoiding data and algorithm biases through diversity

C Kuhlman, L Jackson, R Chunara - arXiv preprint arXiv:2002.11836, 2020

… Page 3. Dataset Description Sensitive Attribute race gender age other Adult: US Census income data [36]. [43, 57, 72, 77] [2, 3, 25, 27, 43, 57, 64, 77] [57] [2, 57] Common Crawl: Occupation biographies [42]. [34] Comm …


Noise Pollution in Hospital Readmission Prediction: Long Document Classification with Reinforcement Learning

L Xu, J Hogan, RE Patzer, JD Choi - arXiv preprint arXiv:2005.01259, 2020

… Averaged Word Embedding For the averaged word embedding encoder (AWE; Section 4.2), em- beddings generated by FastText trained on the Common Crawl and the English Wikipedia with the 300 dimension is …


Not All Swear Words Are Used Equal: Attention over Word n-grams for Abusive Language Identification

HJ Jarquín-Vásquez, M Montes-y-Gómez… - Mexican Conference on …, 2020

L Villasenor-Pineda - Pattern Recognition: 12th Mexican Conference, MCPR …

… On the other hand, for word representation we used pre-trained fastText embeddings [3], trained with subword information on Common Crawl. Table 2. Proposed attention-based deep neural network hyperparameters. Layer …


NOTICE TO BORROWERS

Page 1. DISTRIBUTION AGREEMENT In presenting this thesis as a partial fulfillment of the requirements for an advanced degree from Emory University, I agree that the Library of the University shall make it available for inspection …


Novel Entity Discovery from Web Tables

S Zhang, E Meij, K Balog, R Reinanda - arXiv preprint arXiv:2002.00206, 2020

Page 1. Novel Entity Discovery from Web Tables Shuo Zhang Bloomberg London, United Kingdom szhang611@bloomberg.net Edgar Meij Bloomberg London, United Kingdom emeij@bloomberg.net Krisztian Balog University …


Novel Opinion mining System for Movie Reviews

AH AbdulHafiz - International Journal of Intelligent Systems and …, 2020

… We have adopted the Word2Vec feature representation, the CSG in particular, in our work. It has a pre-trained word vector for the English language trained on 1 million common crawl and Wikipedia documents. Word2vec is a two-layer neural net …


NRC Systems for the 2020 Inuktitut–English News Translation Task

R Knowles, D Stewart, S Larkin, P Littell - Proceedings of the Fifth Conference on …, 2020

… org/wmt20/translation-task.html Page 3. 157 Wiki Titles or Common Crawl Inuktitut data.9 We incorporated the news portion of the development data in training our models to alleviate the domain mismatch issue (Section 5.1). 4 Preprocessing and Postprocessing …


NUIG-DSI at the WebNLG+ challenge: Leveraging Transfer Learning for RDF-to-text generation

N Pasricha, M Arcan, P Buitelaar - Proceedings of the 3rd WebNLG Workshop on …, 2020

… middle) and reference lexicalisation (bottom). (Vaswani et al., 2017) and is pre-trained using un- supervised learning on a large corpus of unlabeled data obtained from the Web using the Common Crawl project. It is trained using …


ODArchive–Creating an Archive for Structured Data from Open Data Portals

… We deem this project particularly useful as a resource for experiments on real-world structured data: to name an example, while large corpora of tabular data from Web tables have been made available via CommonCrawl [6], the …


On Finding Similar Verses from the Holy Quran using Word Embeddings

S Saeed, S Haider, Q Rajput - 2020 International Conference on Emerging Trends in …, 2020

… It is an English multitask Convolution Neural Network (CNN)[4] trained on OntoNotes[14], with GloVe[11] vectors trained on Common Crawl[3]. It contains 685,000 keys, 20,000 unique words and each word …


On Multilingual Word Embeddings & their applications in machine translation

… crosslingual signals. The hard/challenging dataset comprise of English-Italian, English-German, English-Finnish and English-Spanish pairs. These embeddings are trained on Wacky crawling corpora/common crawl corpora. We notice …


On revealing shared conceptualization among open datasets

M Bogdanović, N Veljković, MF Gligorijević, D Puflović… - Journal of Web Semantics, 2020

JavaScript is disabled on your browser. Please enable JavaScript to use all the features on this page. Skip to main content Skip to article …


On the comparability of Pre-trained Language Models

M Aßenmacher, C Heumann - arXiv preprint arXiv:2001.00781, 2020

… Wikipedia data sets are available in the Tensorflow Datasets-module3. CommonCrawl Among other resources, Yang et al. (2019) used data from CommonCrawl … CommonCrawl https://commoncrawl.org/ Unclear Fully available XLNet ClueWeb 2012-B Callan et al …


On the diminishing return of labeling clinical reports

JB Lamare, T Olatunji, L Yao - arXiv preprint arXiv:2010.14587, 2020

… linear classifiers. Recent NLP ad- vance pushes the envelop much further by leveraging web-scale data – for instance, the Common Crawl project 1 that produces 20TB of textual data from the Internet each month. To cope …


On the Effectiveness of Behavior-Based Ransomware Detection

J Han, Z Lin, DE Porter - International Conference on Security and Privacy in …, 2020

… To measure whether partial encryption is effective at withholding user data, we collected 200 different PDF documents from the web using Common Crawl Document Download [4]. We choose PDF documents with a minimum of 10 pages …


On the evaluation of retrofitting for supervised short-text classification

… we considered the 300-dimensional word vectors: (i) Paragram [16], learned from the text content in the paraphrase database PPDB, (ii) Glove [22] learned from Wikipedia and Common Crawl data, (iii) MUSE, a fastText embedding …


On the impact of publicly available news and information transfer to financial markets

M Jazbec, B Pásztor, F Faltings, N Antulov-Fantulin… - arXiv preprint arXiv …, 2020

… To address ihttps://commoncrawl.org iiDetailed statistics about the Common Crawl can found here: https://commoncrawl.github.io/cc-crawl-statistics iiiWe omitted the domain www.nbonews.com. While the most frequently occurring …


On the importance of pre-training data volume for compact language models

V Micheli, M D'Hoffschmidt, F Fleuret - arXiv preprint arXiv:2010.03813, 2020

… OSCAR 2 (Ortiz Suárez et al., 2019) is a large-scale multilingual open source collection of corpora ob- tained by language classification and filtering of the Common Crawl corpus 3. The whole French part amounts to 138 GB …


On the Language Neutrality of Pre-trained Multilingual Representations

J Libovický, R Rosa, A Fraser - arXiv preprint arXiv:2004.05160, 2020

… XLM-RoBERTa. Conneau et al. (2019) claim that the original mBERT is under-trained and train a similar model on a larger dataset that consists of two terabytes of plain text extracted from CommonCrawl (Wenzek et al., 2019) …


On the Persistence of Persistent Identifiers of the Scholarly Web

M Klein, L Balakireva - arXiv preprint arXiv:2004.03011, 2020

… These findings were confirmed in a large scale study by Thompson and Jian [16] based on two samples of the web taken from Common Crawl6 datasets … Thompson, HS, Tong, J.: Can common crawl reliably track persistent identifier (PID) use over time …


On the synthesis of metadata tags for HTML files

P Jiménez, JC Roldán, FO Gallego, R Corchuelo - Software: Practice and Experience

… Recently, an analysis of the 32.04 million domains in the November 2019 Common Crawl has revealed that only 11.92 million domains provide metadata tags,1 which clearly argues for a method that helps software agents deal with the documents provided by the remaining …


On using Product-Specific Schema. org from Web Data Commons: An Empirical Set of Best Practices

R Kiran Selvam, M Kejriwal - arXiv e-prints, 2020

RK Selvam, M Kejriwal - arXiv preprint arXiv:2007.13829, 2020

… on e-commerce websites. The Web Data Commons (WDC) project has extracted schema.org data at scale from webpages in the Common Crawl and made it available as an RDF 'knowledge graph' at scale. The portion of this …


On-The-Fly Information Retrieval Augmentation for Language Models

H Wang, D McAllester - Proceedings of the First Joint Workshop on Narrative …, 2020

… News etc. For language modelling we use the NY Times portion because it is written by native English speakers. Since GPT 2.0 is trained on Common Crawl which contains news collections started from 2008. To avoid testing …


One Belt, One Road, One Sentiment? A Hybrid Approach to Gauging Public Opinions on the New Silk Road Initiative

… ABSA. We used the Common Crawl GloVe version [44], a pre-trained 300-dimension vector representation database of 840 billion tokens and 2.2 million vocabulary, to convert our preprocessed tweets into word embeddings …


Open Information Extraction as Additional Source for Kazakh Ontology Generation

N Khairova, S Petrasova, O Mamyrbayev, K Mukhsina - Asian Conference on …, 2020

… also for many others. For example, an experiment was conducted in [19] for assessing the adequacy of measuring the factual density of 50 randomly selected Spanish documents in the CommonCrawl corpus. In a recent study …


Open Intent Extraction from Natural Language Interactions

N Vedula, N Lipka, P Maneriker, S Parthasarathy - Proceedings of The Web …, 2020

… 2A commercial Customer Relationship Management (CRM) software. Implementation: We use the 300-dimensional GloVe embeddings [50] pre-trained on the Common Crawl dataset3, and character embeddings as per Ma et al [42] …


Open science-based framework to reveal open data publishing: an experience from using Common Crawl

A Correa, I Fernandes - ELPUB 24rd edition of the International Conference on …, 2020

The publishing of open data is considered a key element for civic participation paving the way to the 'public value', a term which underpins the social contribution. A result of that can be seen through the popularity of data portals published all around …


Open source speech recognition on edge devices

R Peinl, B Rizk, R Szabad - 2020 10th International Conference on Advanced …, 2020

… To make the comparison as fair as possible we used the KenLM 6-gram language model for all ASR models except DS2, which came with its own word-level language model trained on CommonCrawl (en-00) from the Common Crawl Corpus7 …


Open-Domain Question Answering Goes Conversational via Question Rewriting

R Anantha, S Vakulenko, Z Tu, S Longpre, S Pulman… - arXiv preprint arXiv …, 2020

… relevant pages with randomly sampled web pages that constitute 1% of the Common Crawl dataset identified … Wayback Machine and 9.9M random web pages from the Common Crawl dataset … 3 https://commoncrawl.org/2019 …


Optimal Subarchitecture Extraction For BERT

A de Wynter, DJ Perry - arXiv preprint arXiv:2010.10499, 2020

… In order to have a sufficiently diverse dataset to pre-train Bort, we combined corpora obtained from Wikipedia7, Wiktionary8, OpenWebText (Gokaslan and Cohen, 2019), UrbanDictionary9, One Billion Words (Chelba et al., 2014) …


Page 1. Optimizing Distributed Computing Systems via Machine Learning by Hao Wang A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Electrical …


OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings

S Dev, T Li, JM Phillips, V Srikumar - arXiv preprint arXiv:2007.00049, 2020

… Our code for reproducing experiments will be released upon publication. Debiasing contextualized embeddings. The operations above are described for a noncontextualized embedding; we use one of the largest such …


Out-of-the-Box and into the Ditch? Multilingual Evaluation of Generic Text Extraction Tools

A Barbaresi, G Lejeune - Proceedings of the 12th Web as Corpus Workshop, 2020

… Recently, approaches using the CommonCrawl1 have flourished as they allow for faster download and processing by skipping (or more precisely outsourcing) the crawling phase (Habernal et al., 2016; Schäfer, 2016) … 1https://commoncrawl.org …


Overview of the CLEF eHealth 2020 task 2: consumer health search with ad hoc and spoken queries

L Goeuriot, Z Liu, G Pasi, GG Saez, M Viviani, C Xu - … Notes of Conference and Labs of …, 2020

… 2.1 Documents The 2018 CLEF eHealth Consumer Health Search document collection was used in this year's IR challenge. As detailed in [17], this collection consists of web pages acquired from the CommonCrawl. An …


Overview of the CLEF eHealth Evaluation Lab 2020

C Xu - … IR Meets Multilinguality, Multimodality, and Interaction …

… Task 2. The 2018 CLEF eHealth Consumer Health Search document collection was used in this year's IR challenge. As detailed in [14], this collection consists of web pages acquired from the CommonCrawl. An initial list of websites was identified for acquisition …


Overview of the seventh Dialog System Technology Challenge: DSTC7

LF D'Haro, K Yoshino, C Hori, TK Marks… - Computer Speech & …, 2020

JavaScript is disabled on your browser. Please enable JavaScript to use all the features on this page. Skip to main content Skip to article …


Overview of the Transformer-based Models for NLP Tasks

A Gillioz, J Casas, E Mugellini, O Abou Khaled

… they contain. The tokenization is done with SentencePiece [15]. In a few cases, for example, in [16], the authors only used a subset of those datasets (eg Stories [17] is a subset of CommonCrawl dataset). IV. BENCHMARKS …


Overview of Touché 2020: Argument Retrieval

A Bondarenko, M Fröbe, M Beloucif, L Gienapp… - Working Notes Papers of the …, 2020

H Wachsmuth, M Potthast, M Hagen - … IR Meets Multilinguality, Multimodality, and Interaction …

Y Ajjour, A Panchenko, C Biemann, B Stein… - Experimental IR Meets …

… Stab et al.[26] retrieve documents from the Common Crawl 3 and then use a topic-dependent neural network to extract arguments from the retrieved documents … This method is shown to outperform several 3 http://commoncrawl. org. Page 398 …


ParaCrawl: Web-Scale Acquisition of Parallel Corpora

M Banón, P Chen, B Haddow, K Heafield, H Hoang…

… In an ex- ploratory study, only 5% of a collection of web pages with useful content were found in CommonCrawl. This may have improved with recent more extensive crawls by CommonCrawl but there is still a strong argument for targeted crawling. 4 Crawling …


Parallelograms revisited: Exploring the limitations of vector space models for simple analogies

JC Peterson, D Chen, TL Griffiths - Cognition, 2020


Pedro Javier Ortiz Suárez1, 3 [0000− 0003− 0343− 8852], Yoann Dupont2

… For the fixed word embeddings we used the Common Crawl-based FastText embeddings [10] originally trained by Facebook as opposed to the embeddings provided by the HIPE shared task, as we obtained better dev …


Persistent metadata catalog

GS Mcpherson, Y Mikhaylyuta, TD Baker, RJ Cole - US Patent 10,853,356, 2020

… sources. In one embodiment metadata catalog service 120 may host metadata for a collection of data sets that are available in the public domain, such as the 1000 genome, NASA NEX, and the Common Crawl Corpus data sets …


Phishing Detection Using Machine Learning Technique

J Rashid, T Mahmood, MW Nisar, T Nazir - 2020 First International Conference of …, 2020

… and June 2017. In particular, we selected 5000 phishing web pages, and all web pages are more stable, especially based on URLs. The fish tank is based entirely on the Alexa URL and Common Crawl archives. B. Step 2 Vocabulary …


PhishingLine: Hybrid Phishing Classifier with Logo Detection

… 29 5.5 Web Capture . . . . . 30 5.5.1 WARC . . . . . 30 5.5.2 Common Crawl . . . . . 30 5.6 Client Server Architecture . . . . . 30 5.6.1 TCP …


Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID Twitter BERT and Bagging Ensemble Technique based on Plurality Voting

A Wadhawan - arXiv preprint arXiv:2010.00294, 2020

… Table 2. 4.2 System Settings For training the CNN, LSTM and BiLSTMs, word vectors for english language pre-trained on Common Crawl2 and Wikipedia3 are downloaded4 and used using FastText5 library. These word vectors …


Photo Stream Question Answer

W Zhang, S Tang, Y Cao, J Xiao, S Pu, F Wu, Y Zhuang - Proceedings of the 28th …, 2020

Page 1. Photo Stream Question Answer Wenqiao Zhang Zhejiang University wenqiaozhang@zju.edu.cn Siliang Tang* Zhejiang Universety siliang@zju.edu.cn Yanpeng Cao,Jun Xiao Zhejiang University caoyp,junx@zju.edu.cn …


PMap: Ensemble Pre-training Models for Product Matching

… In this section, we explain the pre-train models and how to fine-tune them. 5 http://webdatacommons.org/largescaleproductcorpus/v2/index. html 6 http://webdatacommons.org/structureddata/ 7 https://commoncrawl …


Page 1. PoKED: A Semi-Supervised System for Word Sense Disambiguation Feng Wei 1 Abstract In this paper, we propose a semi-supervised neural system, named Position-wise Orthogonal Knowledge-Enhanced Disambiguator …


Page 1. Practical Data Science for Information Professionals Page 2. Every purchase of a Facet book helps to fund CILIP's advocacy, awareness and accreditation programmes for information professionals. Page 3. Practical Data …


Pragmatic Aspects of Discourse Production for the Automatic Identification of Alzheimer's Disease

A Pompili, A Abad, DM de Matos, IP Martins - IEEE Journal of Selected Topics in …, 2020

… For this purpose, we rely on a pre-trained model of word vector representations containing 2 million word vectors, in 300 dimensions, trained with fastText on Common Crawl [42]. In the process of converting a sentence into its vector …


Pre-indexing Pruning Strategies

S Altin, R Baeza-Yates, BB Cambazoglu - International Symposium on String …, 2020

… 4 Experimental Setup. 4.1 Document Collection. As web document collection, we mostly use the open source web collection provided by Common Crawl, CC, in November 2017 … 5 Experimental Results. 5.1 Common Crawl and BM25 …


X Qiu, T Sun, Y Xu, Y Shao, N Dai, X Huang - arXiv preprint arXiv:2003.08271, 2020

Page 1. .Invited Review . Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu*, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai & Xuanjing Huang School of Computer Science, Fudan University, Shanghai …


Pre-training Polish Transformer-based Language Models at Scale

S Dadas, M Perełkiewicz, R Poświata - arXiv preprint arXiv:2006.04229, 2020

… the size of the training corpus. 2) We proposed a method for collecting and pre-processing the data from the Common Crawl database to obtain clean, high-quality text corpora. 3) We conducted a comprehensive evaluation …


Pre-training via Leveraging Assisting Languages and Data Selection for Neural Machine Translation

H Song, R Dabre, Z Mao, F Cheng, S Kurohashi… - arXiv preprint arXiv …, 2020

… Additionally, we used Common Crawl4 monolingual corpora for pre-training … We filled the CurrentDistribution line by line in Common Crawl file if the ratio of the length of current line had been less than the ratio of this length in target length distribution …


Pre-training via Leveraging Assisting Languages for Neural Machine Translation

H Song, R Dabre, Z Mao, F Cheng, S Kurohashi…

… We used Common Crawl3 monolingual corpora for pre-training … We created a shared sub-word vocabulary us- ing Japanese and English data from ASPEC mixing with Japanese, English, Chinese and French data from Common Crawl …


Predicting Consumers' Brand Sentiment Using Text Analysis on Reddit

… with a F-score of 86.25 for NER tasks. The en_core_web_md model is an English multi-task CNN trained on OntoNotes, with GloVe vectors trained on Common Crawl For NER tasks, it assigns various labels to identified entities, such as “CARDINAL,” “ORG,” …


Predicting Themes within Complex Unstructured Texts: A Case Study on Safeguarding Reports

A Edwards, D Rogers, J Camacho-Collados… - arXiv preprint arXiv …, 2020

… Pre-trained word embeddings We leverage two pre-trained 300-dimensional word embedding models: Word2vec (Mikolov et al., 2013) trained on Google news dataset and fastText (Bojanowski et al., 2017) trained with subword information on Common Crawl …


Predicting Twitter Engagement With Deep Language Models

M Volkovs, Z Cheng, M Ravaut, H Yang, K Shen…

… text comprehension tasks. The majority of published language models are pre-trained on large text corpora such as Wikipedia or CommonCrawl, that typically contain longer and properly worded pieces of text. Tweets on the …


Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Z Tian, H Zhao, M Shu, Z Yang, R Li, J Jia - IEEE Annals of the History of Computing, 2020

… Therefore the prior is not,applicable and we only verify FEM on the baseline with,VGG-16 backbone in the zero-shot setting.,Structural Change,Embeddings of Word2Vec [24] and,FastText [22] are trained …


Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies

M Srinath, S Wilson, CL Giles - arXiv preprint arXiv:2004.11131, 2020

… As a consequence, 2https://commoncrawl.org/ Page 3 … Thus, we selected those URLs which had the word “privacy” or the words “data” and “protection” from the Common Crawl URL archive. We were able to extract 3.9 million URLs that fit this selection criterion …


Privacy Policies over Time: Curation andAnalysis of a Million-Document Dataset

R Amos, G Acar, E Lucherini, M Kshirsagar… - arXiv preprint arXiv …, 2020

… app privacy policy URLs [23]. Concurrent to this work, Srinath et al. contribute PrivaSeer, a dataset of over 1 million English privacy policies extracted from May 2019 Common Crawl data [14]. Our work advances this area of …


Privacy-Preserving Passive DNS

P Papadopoulos, N Pitropakis, WJ Buchanan, O Lo… - Computers, 2020

… These sources include but are not limited to Public Blacklists, the Alexa ranking, the Common Crawl project, and various Top Level Domain (TLD) zone files. This system's output is a refined dataset that can be …


Privacy-Preserving Visual Content Tagging using Graph Transformer Networks

XS Vu, DT Le, C Edlund, L Jiang, HD Nguyen

… that local knowledge can be derived from data observations including label semantics or multimedia content se- mantics (eg, optical character recognition); whereas, global knowledge can be drawn from publicly available corpora …


Probing Task-Oriented Dialogue Representation from Language Models

CS Wu, C Xiong - arXiv preprint arXiv:2010.13912, 2020

… is to maximize left-to-right generation likelihood. To ensure diverse and nearly unlimited text sources, they use Common Crawl to obtain 8M documents as its training data. Budzianowski and Vulic (2019) trained GPT2 on task …


Page 1. Bachelor's thesis presented to the Faculty of Arts and Social Sciences of the University of Zurich for the degree of Bachelor of Arts UZH Probing Tasks for Noised Back-Translation Author: Nicolas Spring Student …


Probing Text Models for Common Ground with Visual Representations

G Ilharco, R Zellers, A Farhadi, H Hajishirzi - arXiv preprint arXiv:2005.00619, 2020

… GloVe embeddings (Pennington et al., 2014). For such, we use embeddings trained on 840 billion tokens of web data from Common Crawl, with dL = 300 and a vocabulary size of 2.2 million2. Models trained on text and images …


Programming in Natural Language with fuSE: Synthesizing Methods from Spoken Utterances Using Deep Natural Language Understanding

S Weigelt, V Steurer, T Hey, WF Tichy - Proceedings of the 58th Annual Meeting of …, 2020

… on the Common Crawl dataset4 by Facebook Research (Mikolov et al., 3Note that we do not discuss the influence of varying epoch numbers, since we used early stopping, ie the training stops when the validation loss stops …


Projecting Heterogeneous Annotations for Named Entity Recognition

R Agerri, G Rigau - Proceedings of the Iberian Languages Evaluation …, 2020

… Page 3. of Common Crawl text … The biggest update that XLM-Roberta offers is a significantly increased amount of training data, 2.5TB of Common Crawl clean data [6]. As for BERT, in this paper we use the base version of XLM-RoBERTa …


Projecting named entity recognizers from resource-rich to resource-poor languages without annotated or parallel corpora

Page 1. Projecting named entity recognizers from resource-rich to resourcepoor languages without annotated or parallel corpora Hou, Jue Helsinki October 20, 2019 UNIVERSITY OF HELSINKI Department of Computer …


PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data

D Carmo, M Piau, I Campiotti, R Nogueira, R Lotufo - arXiv preprint arXiv:2008.09144, 2020

… The original T5 vocabulary uses the SentencePiece library [7] using English, German, French, and Romanian web pages from Common Crawl.1 We use a similar procedure to create our Portuguese vocabulary: we train …


PublishInCovid19 at the FinSBD-2 Task: Sentence and List Extraction in Noisy PDF Text Using a Hybrid Deep Learning and Rule-Based Approach

J Singh - Proceedings of the Second Workshop on Financial …, 2020

… For the pretrained word embeddings, we use GloVE 6 which are trained on large Common Crawl dataset and can effectively represent 84 billion cased tokens. To train the BERT for our task, we fine-tune a pre-trained model, namely bert-base-cased …


Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?

Ł Augustyniak, P Szymanski, M Morzy, P Zelasko… - arXiv preprint arXiv …, 2020

… In this work, we use pre-trained GloVe embeddings trained on the Common Crawl dataset consisting of 2.6 billion textual documents … In our case, the punctuation in conversational transcripts is substantially different from the …


PyChain: A Fully Parallelized PyTorch Implementation of LF-MMI for End-to-End ASR

Y Shao, Y Wang, D Povey, S Khudanpur - arXiv preprint arXiv:2005.09824, 2020

… We hope that our experience with PYCHAIN will inspire other efforts to build next-generation hybrid ASR tools. 812k hours AM train set and common crawl LM. 9Data augmentation and pre-trained on LibriSpeech …


Quality and Relevance Metrics for Selection of Multimodal Pretraining Data

R Rao, S Rao, E Nouri, D Dey, A Celikyilmaz, B Dolan - Proceedings of the IEEE/CVF …, 2020

… The GloVe vectors used are pretrained on 840 billion tokens from Common Crawl. Let o ∈ Oi be the set of objects de- tected by the RCNN for a given image i, w ∈ d be the set 0 100 200 300 400 500 ConceptualCaptions Ngram …


Quality Estimation for Machine Translation with Multi-granularity Interaction⋆

… (7) 4 Experiments 4.1 Dataset The bilingual parallel corpus that we use for pre-trained multilingual BERT is officially released by the WMT17 Shared Task: Machine Translation of News1, in- cluding Europarl v7, Common Crawl …


Quality Evaluation

JM Gomez-Perez, R Denaux, A Garcia-Silva - A Practical Guide to Hybrid Natural …, 2020

… Besides the embeddings trained by us, we also include, as part of our study, several pre-trained embeddings, notably the GloVe embeddings for CommonCrawl— code glove_840B provided by Stanford11 —fastText …


Query focused abstractive summarization using BERTSUM model

… Conneau et al. (2019) have introduced a multilingual masked language model from Facebook AI. This model has been trained on 2.5 TB of newly created clean CommonCrawl (Wenzek et al., 2019) data in 100 languages. The model has shown state-of-the-art results …


Question Answering for Comparative Questions with GPT-2

… https://www.microsoft.com/en-us/research/blog/ turing-nlg-a-17-billion-parameter-languagemodel-by-microsoft/, accessed: 6.3.2020 3. Bevendorff, J., Stein, B., Hagen, M., Potthast, M.: Elastic chatnoir: search engine for the clueweb and the common crawl …


Question Answering When Knowledge Bases are Incomplete

C Pradel, D Sileo, Á Rodrigo, A Peñas, E Agirre - International Conference of the …, 2020

E Agirre - … IR Meets Multilinguality, Multimodality, and Interaction …

… with bag of word embeddings. We use FastText CommonCrawl word embeddings [10] 4 and a max pooling to produce the continuous bag of word representations of table columns and the question text. The column bag of words …


Question Type Classification Methods Comparison

T Seidakhmetov - arXiv preprint arXiv:2001.00571, 2020

… The GLoVe vectors were pre-trained using 840 billion tokens from Common Crawl, and each token is mapped into a 300-dimensional vector [3]. Xembeddings = GloveEmbedding( Xword) ∈ RNxDword where Dword is a number of dimensions of a word vector …


Questioning the Use of Bilingual Lexicon Induction as an Evaluation Task for Bilingual Word Embeddings

… gual word embeddings. In fact, this corpus was significantly smaller than the Wikipedia corpora for all the other languages, and than the Finnish Common Crawl corpus used to train Finnish Vecmap-emb. Another finding is …


REALTOXICITYPROMPTS: Evaluating Neural Toxic Degeneration in Language Models

S Gehman, S Gururangan, M Sap, Y Choi, NA Smith - arXiv preprint arXiv …, 2020

… GPT-2 (specifically, GPT-2-small; Radford et al., 2019), is a similarly sized model pretrained on OPENAI-WT, which contains 40GB of English web text and is described in §6.7 GPT-3 (Brown et al., 2020) is pretrained on a mix …


M Naghshnejad, T Joshi, VN Nair - arXiv preprint arXiv:2009.02358, 2020

Page 1. 1 Recent Trends in the Use of Deep Learning Models for Grammar Error Handling Mina Naghshnejad1, Tarun Joshi, and Vijayan N. Nair Corporate Model Risk, Wells Fargo2 Abstract Grammar error handling (GEH) is …


AC Stickland, X Li, M Ghazvininejad - arXiv preprint arXiv:2004.14911, 2020

Page 1. Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation Asa Cooper Stickland♣ Xian Li♠ ♣ University of Edinburgh, ♠ Facebook AI a.cooper.stickland@ed.ac.uk, {xianl,ghazvini}@fb.com Marjan Ghazvininejad♠ Abstract …


ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning

W Yu, Z Jiang, Y Dong, J Feng - arXiv preprint arXiv:2002.04326, 2020

Page 1. Published as a conference paper at ICLR 2020 RECLOR: AREADING COMPREHENSION DATASET REQUIRING LOGICAL REASONING Weihao Yu∗, Zihang Jiang∗, Yanfei Dong & Jiashi Feng National University …


Recognai's Working Notes for CANTEMIST-NER Track

DC Fidalgo, D Vila-Suero, FA Montes - … of the Iberian Languages Evaluation Forum …, 2020

… light-weight solution regarding the pretrained component. For this reason we chose the pretrained Spanish word vectors provided by FastText[4]. These vectors encompass 2 million words that were trained on Common Crawl4 …


G KV, A Mittal - arXiv preprint arXiv:2007.06198, 2020

Page 1. Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encoder Gouthaman KV and Anurag Mittal Indian Institute of Technology Madras, India {gkv,amittal}@cse.iitm.ac.in Abstract …


S Huang, T Hui, S Liu, G Li, Y Wei, J Han, L Liu, B Li - … of the IEEE/CVF Conference on …, 2020

Page 1. Referring Image Segmentation via Cross-Modal Progressive Comprehension Shaofei Huang1,2∗ Tianrui Hui1,2∗ Si Liu3† Guanbin Li4 Yunchao Wei5 Jizhong Han1,2 Luoqi Liu6 Bo Li3 1 Institute of Information Engineering …


Refinement of Unsupervised Cross-Lingual Word Embeddings

M Biesialska, MR Costa-jussà - arXiv preprint arXiv:2002.09213, 2020

… Finnish. Monolingual embeddings of 300 di- mensions were created using Word2Vec3 [18] and were trained on WMT News Crawl (Spanish), WacKy crawling corpora (English, German), and Common Crawl (Finnish). To evaluate …


ReINTEL: A Multimodal Data Challenge for Responsible Information Identification on Social Network Sites

DT Le, XS Vu, ND To, HQ Nguyen, TT Nguyen, L Le… - arXiv preprint arXiv …, 2020

… Word2VecVN (Vu, 2016) x Trained on 7GB texts of Vietnamese news FastText (Vietnamese version) (Joulin et al., 2016) x Trained on Vietnamese texts of the CommonCrawl corpus ETNLP (Vu et al., 2019) x Trained on 1GB texts of Vietnamese Wikipedia …


Related Tasks can Share! A Multi-task Framework for Affective language

KS Deep, MS Akhtar, A Ekbal, P Bhattacharyya - arXiv preprint arXiv:2002.02154, 2020

… 3. Character-level embeddings2: Character-level embeddings are trained over common crawl glove corpus providing 300 dimensional vectors for each character (used in case if word is not present in other two embeddings) …


Relational and Fine-Grained Argument Mining

D Trautmann, M Fromm, V Tresp, T Seidl, H Schütze - Datenbank-Spektrum, 2020

… Crowdworkers had the task of selecting argumentative spans for a given set of topics and topic related sentences. The sentences were from textual data extracted from Common Crawl Footnote 6 for a predefined list of eight …


Relational Databases and SQL Language

JM Patel - Getting Structured Data from the Internet, 2020

… itself. We are mainly discussing PostgreSQL here so that you can scale up to 64 TB if you decide to index large portions of common crawl datasets for creating a backlinks and news database in Chapters 6 and 7, respectively …


D Li, J Zhang, P Li - US Patent App. 16/691,554, 2020

US20200184339A1 - Representation learning for input classification via topic sparse autoencoder and entity embedding - Google Patents. Representation learning for input classification via topic sparse autoencoder and entity embedding. Download PDF Info …


Reproducible Extraction of Cross-lingual Topics (rectr)

CH Chan, J Zeng, H Wessler, M Jungblut, K Welbers… - … Methods and Measures, 2020

… 17 Researchers at Facebook Research created these aligned word embeddings by first training fastText word embeddings (Bojanowski et al., 2017) using Wikipedia and Common Crawl articles of each individual …


Research Challenges in Designing Differentially Private Text Generation Mechanisms

O Feyisetan, A Aggarwal, Z Xu, N Teissier - arXiv preprint arXiv:2012.05403, 2020

… A natural way to “bias” an exponential mechanism without changing its privacy properties is to modulate it with a public “prior” µ(z). For example, such a prior can be constructed over a publicly available corpus such as Wikipedia or Common Crawl …


Residual Energy-Based Models for Text Generation

Y Deng, A Bakhtin, M Ott, A Szlam, MA Ranzato - arXiv preprint arXiv:2004.11714, 2020

… Page 6. Published as a conference paper at ICLR 2020 genres, totaling about half a billion words. The latter is a de-duplicated subset of the English portion of the CommonCrawl news dataset (Nagel, 2016), which totals around 16 Billion words …


Rethinking embedding coupling in pre-trained language models

HW Chung, T Févry, H Tsai, M Johnson, S Ruder - arXiv preprint arXiv:2010.12821, 2020

Page 1. Preprint. Under review. RETHINKING EMBEDDING COUPLING IN PRE-TRAINED LANGUAGE MODELS Hyung Won Chung∗† Google Research hwchung@google.com Thibault Févry∗† thibaultfevry@gmail …


Rethinking Evaluation in ASR: Are Our Models Robust Enough?

T Likhomanenko, Q Xu, V Pratap, P Tomasello, J Kahn… - arXiv preprint arXiv …, 2020

… only; for TL – both train transcriptions and provided LM data. We also train a 4-gram LM on Common Crawl (CC) data with 200k top words and pruning of all 3,4-grams appearing once. Perplexity of all LMs is shown in Table 2 …


Retrieving Comparative Arguments using Deep Pre-trained Language Models and NLU

… ChatNoir is an Elasticsearch-based5 engine providing access to nearly 3 billion web pages from ClueWeb and Common Crawl corpora … ACM. 2. J. Bevendorff, B. Stein, M. Hagen, and M. Potthast. Elastic ChatNoir: Search …


Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

A Chronopoulou, D Stojanovski, A Fraser - arXiv preprint arXiv:2009.07610, 2020

… We use 68M En sentences from NewsCrawl, 2.4M Mk and 4M Sq, both from CommonCrawl and Wikipedia … Second, in En- De, we use high-quality corpora for both languages (NewsCrawl), whereas Mk and Sq are trained on low-quality CommonCrawl data …


Review of the Recent Techniques for Learning Commonsense Knowledge applied to the Winograd Schema Challenge

… 4 A Combined Approach Prakash et al. [2], to the best of our knowledge, are the first ones to combine methods from KRR with methods from ML such as language models. They propose a framework in which four different …


Review rating prediction framework using deep learning

BH Ahmed, AS Ghabayen - Journal of Ambient Intelligence and Humanized …, 2020

… word representation) embedding. There are several Glove embeddings from different sources, such as Twitter, Wikipedia or the common crawl. We utilized we utilize the Glove embedding trained by (Pennington et al. 2014) on …


J Moon, H Cho, EL Park - arXiv preprint arXiv:2004.13937, 2020

Page 1. Revisiting Round-Trip Translation for Quality Estimation Jihyung Moon Naver Papago Hyunchang Cho Naver Papago {jihyung.moon, hyunchang.cho, lucypark}@navercorp.com Eunjeong L. Park Naver Papago Abstract …


RobBERT: a Dutch RoBERTa-based Language Model

P Delobelle, T Winters, B Berendt - arXiv preprint arXiv:2001.06286, 2020

… 3.1 Data We pre-trained our model on the Dutch section of the OSCAR corpus, a large multilingual corpus which was obtained by language classification in the Common Crawl corpus (Ortiz Suárez et al., 2019). This Dutch corpus …


Robust Cross-lingual Embeddings from Parallel Sentences

A Sabet, P Gupta, JB Cordonnier, R West, M Jaggi - arXiv preprint arXiv:1912.12481, 2019

… MUSE 0.38 0.30 0.74 0.64 RCSLS 0.38 0.30 0.74 0.64 FASTTEXTCommon Crawl 0.49 0.32 0.75 0.57 BIVEC 0.40 0.36 0.70 0.60 … We also include FASTTEXT monolingual vectors trained on CommonCrawl data (Grave et …


Robust Prediction of Punctuation and Truecasing for Medical ASR

M Sunkara, S Ronanki, K Dixit, S Bodapati, K Kirchhoff - Proceedings of the First …, 2020

MSSRK Dixit, SBK Kirchhoff - ACL 2020, 2020

… But just like any other model, these Language Models are biased by their training data. In particular, they are typically trained on data that is easily available in large quantities on the internet eg Wikipedia, CommonCrawl etc …


Robust Prediction of Punctuation and Truecasingfor Medical ASR

M Sunkara, S Ronanki, K Dixit, S Bodapati, K Kirchhoff - arXiv preprint arXiv …, 2020


Russian-English Bidirectional Machine Translation System

… For the monolingual data we use English and Russian Newscrawl as well as a filtered part of Commoncrawl in Russian … Russian is relatively smaller than that of English, we have to augment the Newscrawl data for Russian …


Samsung R&D Institute Poland submission to WMT20 News Translation Task

M Krubinski, M Chochowski, B Boczek, M Koszowski… - Proceedings of the Fifth …, 2020

… Pretraining a complete encoder-decoder model allows for later direct fine-tuning on the translation ob- jective, with parallel corpora. In our experiment, we sampled 250M sentences from CommonCrawl for Czech, English and Polish (ie 750M in total) …


SandiDoc at CLEF 2020-Consumer Health Search: AdHoc IR Task

S Seneviratne, E Daskalaki, M Zakir - Conference and Labs of the Evaluation (CLEF) …, 2020

… respectively. 2 Resources 2.1 Dataset The document collection used in the document retrieval task was acquired by the common crawl dump of 2018-19. This included web pages of the formats such as HTML, XHTML, XML. The …


SardiStance@ EVALITA2020: Overview of the Task on Stance Detection in Italian Tweets

AT Cignarella, M Lai, C Bosco, V Patti, P Rosso - … of the 7th Evaluation Campaign of …, 2020

… In particular, they trained three classifiers based respectively on SENTIPOLC 2016 (Barbieri et al., 2016) for sentiment analysis classification, on HaSpeeDe 2018 (Bosco et al., 2018) 4https://huggingface.co/Musixmatch/ umberto-commoncrawl-cased-v1 …


SberQuAD–Russian Reading Comprehension Dataset: Description and Analysis

P Braslavski - … IR Meets Multilinguality, Multimodality, and Interaction …

… We tokenized text using spaCy. 12 To initialize the embedding layer for BiDAF, DocQA, DrQA, and R-Net we use Russian casesensitive fastText embeddings trained on Common Crawl and Wikipedia. 13 This initialization is used for both questions and paragraphs …


Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation

K Webster, E Pitler - arXiv preprint arXiv:2006.08881, 2020

… glish was most recently contested at WMT'132 (Bojaretal., 2013), which offered participants 14,980,513 sentence pairs from Eu- roparl3 (Habash et al., 2017), Common Crawl (Smith et al., 2013), the United Nations cor …


GM Slota, C Root, K Devine, K Madduri… - IEEE Transactions on …, 2020

Page 1. Scalable, Multi-Constraint, Complex-Objective Graph Partitioning GeorgeM.Slota ,CameronRoot,KarenDevine ,KameshMadduri , and Sivasankaran Rajamanickam Abstract—We introduce XTRAPULP, a distributed-memory …


Scaling Laws for Neural Language Models

J Kaplan, S McCandlish, T Henighan, TB Brown… - arXiv preprint arXiv …, 2020

01/23/20 - We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with mode...


Scientific Question Answering with AAN

… have easy to understand language as well as little background knowledge required. They then used web data provided by Common Crawl to find supporting web documents from which to draw information and construct …


Scones: Towards Conversational Authoring of Sketches

… detail in Section 4.1. For each text token t, we use a 300-dimensional GLoVe vector trained on 42B tokens from the Common Crawl dataset [22] to semantically represent these words in the instructions. To train the Transformer …


Scoring Dimension-Level Job Performance From Narrative Comments: Validity and Generalizability When Using Natural Language Processing

AB Speer - Organizational Research Methods, 2020

Performance appraisal narratives are qualitative descriptions of employee job performance. This data source has seen increased research attention due to the ability to efficiently derive insights u...


SDRS: A new lossless dimensionality reduction for text corpora

IV de Mendizabal, V Basto-Fernandes, E Ezpeleta… - Information Processing & …, 2020

… Clueweb 12, Web (html) pages, English, unknown, 870M. Common Crawl Data, Web (html) pages, multilingual, 100% spam, 9 Billion in 2014 and increasing. YouTube Comments Dataset, Youtube comments, multilingual, 7% spam, 6M …


Searching the Web for Cross-lingual Parallel Data

A El-Kishky, P Koehn, H Schwenk - Proceedings of the 43rd International ACM SIGIR …, 2020

… and Tokenization – CommonCrawl Preprocessing Code • Open-source Code for Generating Cross-lingual Datasets – Code for Generating High-quality Monolingual Data from CommonCrawl – Code for Generating …


SEDAR: a Large Scale French-English Financial Domain Parallel Corpus

A Ghaddar, P Langlais - Proceedings of The 12th Language Resources and …, 2020

… It contains 40.8M sentence pairs extracted from five datasets that cover various domains: EUROPARL V7 (Koehn, 2005), UNITED NATIONS CORPUS (Eisele and Chen, 2010), COMMON CRAWL CORPUS, NEWS COMMENTARY, and 109 FRENCH-ENGLISH corpus …


Seeking Meaning: Examining a Cross-situational Solution to Learn Action Verbs Using Human Simulation Paradigm

… of words in a given corpus. We used the GloVe model pretrained on 840B tokens of Common Crawl text to create semantic distance measures (Pennington, Socher & Manning, 2014). We discovered that both semantic knowledge …


Self-training Improves Pre-training for Natural Language Understanding

J Du, E Grave, B Gunel, V Chaudhary, O Celebi, M Auli… - arXiv preprint arXiv …, 2020

… As a large-scale external bank of unannotated sentences, we extract and filter text from CommonCrawl 1 (Wenzek et al., 2019) … CommonCrawl data contains a wide variety of domains and text styles which makes it a good general-purpose corpus …


Semantic image retrieval

T Berg, PN Belhumeur - US Patent 10,769,502, 2020

T Berg, PN Belhumeur - US Patent App. 16/999,616, 2020

… GloVe. Common Crawl is a public repository of web crawl data (eg, blogs, news, and comments) available on the internet in the commoncrawl.org domain, the entire contents of which is hereby incorporated by reference. GloVe …


Semantic Matching: Dynamic Composition of Matcher Ensembles for Ontology Alignment

Page 1. ISBN 978-82-326-4842-9 (printed ver.) ISBN 978-82-326-4843- 6 (electronic ver.) ISSN 1503-8181 Doctoral theses at NTNU, 2020:247 Audun Vennesland Semantic Matching Dynamic Composition of Matcher …


Semantic Networks for Engineering Design: A Survey

J Han, S Sarica, F Shi, J Luo - arXiv preprint arXiv:2012.07060, 2020

… relations No Pre-trained word2vec (Mikolov et al., 2013) Unsupervised Google News Cosine similarity No Pre-trained GloVe (Pennington et al., 2014) Unsupervised Wikipedia, Gigaword, Common Crawl Cosine similarity No B-Link …


Page 1. DRAFT Running head: SEMANTIC NORM EXTRAPOLATION 1 Semantic Norm Extrapolation is a Missing Data Problem Bryor Snefjella & Idan Blank University of California, Los Angeles Department of Psychology Author Note …


Semantic Recommendations of Books Using Recurrent Neural Networks

M Nitu, S Ruseti, M Dascalu, S Tomescu - Ludic, Co-design and Tools Supporting Smart …

… We conducted the experiments using pre-trained FastText embeddings for Romanian language. The embedding model consists of 2 million word vectors trained on Common Crawl and Wikipedia (approx. 600 billion tokens). Page 239. 240 M. Nitu et al. Fig …


Semantic-Based Algorithm for Scoring Alternative Uses Tests (AUT)

… Page 6. Each response was mapped to a word vector in 300 dimensions using a fastText pretrained model for Dutch. This model was trained on Wikipedia and Common Crawl data, using CBOW model with character …


Semantical Search Term Clustering for Performance Prediction

Page 1. Eindhoven University of Technology MASTER Semantical search term clustering for performance prediction Coenders, R. Award date: 2019 Link to publication Disclaimer This document contains a student thesis (bachelor's …


Semi-autonomous methodology to validate and update customer needs database through text data analytics

AM Bigorra, O Isaksson, M Karlberg - International Journal of Information …, 2020

… Two different pre-trained word vectors based on 1 and 2 million English words from Wikipedia and Common Crawl are considered and they are referred along the rest of the presented paper as emb1 and emb2, respectively 3 …


SemSeq: A Regime for Training Widely-Applicable Word-Sequence Encoders

H Tsuyuki, TY Ogawa, HTB Kobayashi - … : 16th International Conference of the Pacific …, 2020


Sense Inventories for Arabic Texts

… E. Fasttext pre-trained embeddings Arabic Fasttext embeddings are provided by Grave et al. [16]. These embeddings are resulted from training on Wikipedia and Common Crawl corpus. They have used an extension of the Fasttext model with subword information …


Sentence Matching with Deep Self-attention and Co-attention Features

Z Wang, D Yan - … Conference on Knowledge Science, Engineering and …, 2021

… 4.1 Implementation Details. In our experiments, word embedding vectors are initialized with 300d GloVe vectors pre-trained from the 840B Common Crawl corpus. Embeddings of out of the vocabulary of GloVe is initialized …


Sentence-Embedding and Similarity via Hybrid Bidirectional-LSTM and CNN Utilizing Weighted-Pooling Attention

D HUANG, A AHMED, SY ARAFAT, KI RASHID… - IEICE TRANSACTIONS on …, 2020

… bag-of-words architecture. • A GloVe is a 300-dimensional word embedding model learned on aggregated global word co-occurrence statistics from Common Crawl (840 billion to- kens) [32]. 4.2 Datasets The datasets are concisely …


Sentiment Analysis Approach Based on Combination of Word Embedding Techniques

I Kaibi, H Satori - Embedded Systems and Artificial Intelligence, 2020

… The fastText pre-trained word vectors is a high-quality word representation for 157 languages, two sources of data are used to train fastText pre-trained models: the free online encyclopedia Wikipedia and data from the common crawl project …


C Zuheros, E Martínez-Cámara, E Herrera-Viedma… - arXiv preprint arXiv …, 2020

Page 1. arXiv:2008.00032v1 [cs.CL] 31 Jul 2020 Sentiment Analysis based Multi-person Multi-criteria Decision Making Methodology: Using Natural Language Processing and Deep Learning for Decision Aid Cristina Zuheros1 …


Sentiment analysis for customer relationship management: an incremental learning approach

N Capuano, L Greco, P Ritrovato, M Vento - Applied Intelligence, 2020

… text corpora. In particular, Universal Dependencies and WikiNER corpora [31] were used for Italian, while OntoNotes [32] and Common Crawl Footnote 3 corpora were used for English. The classification model. Once the WEs …


Sentiment Analysis for Hinglish Code-mixed Tweets by means of Cross-lingual Word Embeddings

P Singh, E Lefever - LREC 2020–4th Workshop on Computational …, 2020

… This can probably be attributed to the quality of the monolingual embeddings, since the English embeddings were trained on the vast Common Crawl data while the Code-Mixed embeddings were trained on a little more than 100,000 scraped tweets …


Sentiment Aware Word Embeddings Using Refinement and Senti-Contextualized Learning Approach

B Naderalvojoud, EA Sezer - Neurocomputing, 2020

JavaScript is disabled on your browser. Please enable JavaScript to use all the features on this page. Skip to main content Skip to article …


Sentiment detection with FedMD: Federated Learning via Model Distillation

… value due to the nature of tweets - they are short, have little context and contain misspelled and shortened words, all of which stands in general contrast to the GloVe training data and logic, which was based on structured …


Seq2Seq Models for Recommending Short Text Conversations

J Torres, C Vaca, L Terán, CL Abad - Expert Systems with Applications, 2020

… to a lower-dimensional representation ( w ∈ R d ). For the initialization of the word embeddings, we use the pre-trained vectors provided by Mikolov, Grave, Bojanowski, Puhrsch, and Joulin (2018), which consist of 2 …


Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue

B Kim, J Ahn, G Kim - arXiv preprint arXiv:2002.07510, 2020

Page 1. Published as a conference paper at ICLR 2020 SEQUENTIAL LATENT KNOWLEDGE SELECTION FOR KNOWLEDGE-GROUNDED DIALOGUE Byeongchang Kim Jaewoo Ahn Gunhee Kim Department of Computer …


Sequential Neural Networks for Noetic End-to-End Response Selection

Q Chen, W Wang - Computer Speech & Language, 2020


Sequential Transfer Learning for Event Detection and Key Sentence Extraction

… Training is performed on the base version trained on cased text. XLNet [22] 12-layer, 768-hidden, 12-heads 110M parameters Pre-trained models are based on English texts from Wikipedia, BooksCorpus, Giga5, ClueWeb, and Common Crawl …


Shopify in Germany: An analysis of a Canadian e-commerce platform's marketing strategy and activities in an international market.

Page 1. Available at: http://hdl.handle.net/2078.1/thesis:25661 [Downloaded 2020/09/25 at 09:14:22 ] "Shopify in Germany: An analysis of a Canadian e-commerce platform's marketing strategy and activities in an international …


Sidecar: Augmenting Word Embedding Models with Expert Knowledge

M Lemay, D Shapiro, MK MacPherson, K Yee… - Future of Information and …, 2020

… spaCy's en_core_web_lg GloVe model 8 , trained on Common Crawl 9. Facebook's crawl-300d-2M fastText model 10 , also trained on Common Crawl. All three models produce vectors of size 300. To generate vectors for the …


Sights, titles and tags: mining a worldwide photo database for sightseeing

… Explosion AI spaCy pretrained model [7]. We use a medium sized web code model. The model is based on Common Crawl and OntoNotes 5 [22] sources … Facebook fastText pretrained model [8]. The model is pretrained on Common Crawl and Wikipedia data …


SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings

MJ Sabet, P Dufter, H Schütze - arXiv preprint arXiv:2004.08728, 2020

… In addition, we use XLM-RoBERTa base (Conneau et al., 2019), which is pretrained on 100 languages on CommonCrawl data. We denote alignments obtained using the embeddings from the i-th layer by XLM-R[i] …


Similarity judgment within and across categories: A comprehensive model comparison

… Google News 100B 300 None Magnitude Librarya fastText 600B Common Crawl FastText with Continuous Bag of Words (CBOW) … GloVe 840B Common Crawl GloVe Common Crawl 840B 300 None Magnitude Library Glove 840B Common Crawl, Paragram …


Simulation Induces Durable, Extensive Changes to Self-knowledge

J Rubin-McGregor, Z Zhao, D Tamir - PsyArXiv. December, 2020

Page 1. SIMULATION CHANGES SELF-KNOWLEDGE 1 1 2 3 4 5 6 Simulation Induces Durable, Extensive Changes to Self-Knowledge 7 Jordan Rubin-McGregora, Zidong Zhaoa, and Diana Tamira 8 aDepartment of …


SINAI at eHealth-KD Challenge 2020: Combining Word Embeddings for Named Entity Recognition in Spanish Medical Records

… we have used two specific pre-trained word embeddings: BETO [28], which follows a BERT model trained on a big Spanish corpus, and XLM-RoBERTa [29], which were generated by using a large multilingual language model …


Siva at WNUT-2020 Task 2: Fine-tuning Transformer Neural Networks for Identification of Informative Covid-19 Tweets

S Sai - Proceedings of the Sixth Workshop on Noisy User …, 2020

… The base version of RoBERTa has 125M parameters, and the large version has 355M parameters. XLM-RoBERTa XLM-RoBERTa(Conneau et al., 2019) is a multilingual model trained on 2.5 TB data from CommonCrawl. This …


SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task

Z Li, H Zhao, R Wang, K Chen, M Utiyama, E Sumita - arXiv preprint arXiv …, 2020

… In the supervised PL→EN translation direction, we based on the XLM framework to pre-train a Polish language model using common crawl and news crawl monolingual data, and proposed the XLM enhanced NMT model …


SMAN: Stacked Multi-Modal Attention Network for Cross-Modal Image-Text Retrieval

Page 1. warwick.ac.uk/lib-publications Manuscript version: Author's Accepted Manuscript The version presented in WRAP is the author's accepted manuscript and may differ from the published version or Version of Record. Persistent …


SML: Semantic Meta-learning for Few-shot Semantic Segmentation

AK Pambala, T Dutta, S Biswas - arXiv preprint arXiv:2009.06680, 2020

… Word2vec (Mikolov et al. 2013) is trained on Google News dataset (Wang, Ye, and Gupta 2018) which contains 3-million words; (2) FastText (Joulin et al. 2016) is trained on Common-Crawl dataset (Mikolov et al. 2018). We use these …


SNK@ DANKMEMES: Leveraging Pretrained Embeddings for Multimodal Meme Detection

S Fiorucci - Proceedings of Seventh Evaluation Campaign of …, 2020

… et al., 2018). Word embeddings are trained on Common Crawl and Wikipedia, using CBOW with positionweights, in dimension 300, with character n-grams of length 5, a window of size 5 and 10 negatives. We calculated the …


Social biases in word embeddings and their relation to human cognition

… state-of-the-art word embeddings is the vast amount of training data available. GloVe is trained on 840 billion tokens and more than 2 million unique words of Common Crawl data which is a crawl of the entire world wide web. Similarly, Word2vec is trained on a …


Social Media Attributions in the Context of Water Crisis

R Sarkar, H Sarkar, S Mahinder, AR KhudaBukhsh - arXiv preprint arXiv:2001.01697, 2020

… used embedding in this preprocessing step. We used the 300 dimensional GloVe model trained on 840 billion tokens of the CommonCrawl corpus, having a vocabulary size of 2.2 million. While calculating the embedding of …


Sociolinguistic Properties of Word Embeddings

A Arseniev-Koehler, JG Foster - SocArXiv. August, 2020

… These studies use large, commonly available pre-trained embeddings or their training corpora, such as Google News, web data (Common Crawl), and Google Books … They replicated results using a pretrained model on Common Crawl data …


Software for creating and analyzing semantic representations

FÅ Nielsen, LK Hansen - Statistical Semantics, 2020

… This package provides models for the tagger, parser, named-entity recognizer and distributional semantic vectors trained on OntoNotes Release 5 and the Common Crawl dataset … 10 K–50 K. 300. 29 languages. GloVe. Common …


Spoken words as biomarkers: using machine learning to gain insight into communication as a predictor of anxiety

G Demiris, KL Corey Magan, D Parker Oliver… - Journal of the American …, 2020

… The validity of using cosine distance in an embedding space to measure text similarity depends largely on how well the embedding space represents the semantic concepts present in the text. In our case, the word embeddings …


G Nicolas Ferreira - 2020
GN Ferreira - 2020

Page 1. SPONTANEOUS STEREOTYPE CONTENT: MEASUREMENT AIMING TOWARD THEORETICAL INTEGRATION AND DISCOVERY GANDALF NICOLAS FERREIRA A DISSERTATION PRESENTED TO THE FACULTY OF PRINCETON UNIVERSITY IN …


Stanza: A Python Natural Language Processing Toolkit for Many Human Languages

P Qi, Y Zhang, Y Zhang, J Bolton, CD Manning - arXiv preprint arXiv:2003.07082, 2020

… For the character-level language models in the NER component, we pretrained them on a mix of the Common Crawl and Wikipedia dumps, and the news corpora released by the WMT19 Shared Task (Barrault et al., 2019), with …


STIL--Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++

JGM FitzGerald - arXiv preprint arXiv:2010.00760, 2020

… The mBART.cc25 model was trained on 25 languages for 500k steps using a 1.4 TB corpus of scraped website data taken from Common Crawl (Wenzek et al., 2019). The model was trained to reconstruct masked tokens and to rearrange scrambled sentences …


STILTool: A Semantic Table Interpretation evaLuation Tool

E Jimenez-Ruiz, A Maurino - The Semantic Web: ESWC 2020 Satellite Events …

M Cremaschi, A Siano, R Avogadro, E Jimenez-Ruiz…

… In order to size the spread of tabular data, 2.5 M tables have been identified within the Common Crawl repository1 [3]. The current snapshot of Wikipedia contains more than 3.23 M tables from more than 520k Wikipedia articles …


Structured deep neural network with low complexity

Page 1. STRUCTURED DEEP NEURAL NETWORK WITH LOW COMPLEXITY By SIYU LIAO A dissertation submitted to the School of Graduate Studies Rutgers, The State University of New Jersey in partial fulfillment of the …


Study and Creation of Datasets for Comparative Questions Classification

… The data used by the system is a preprocessed version of the Common Crawl Text Corpus8, which crawled from the world wide web … Which one is better suited for me, Xbox One or PS4? 8https://commoncrawl.org/ 4 Page 11. CHAPTER 1. INTRODUCTION …


Studying the Evolution of Greek Words via Word Embeddings

V Barzokas, E Papagiannopoulou, G Tsoumakas - 11th Hellenic Conference on …, 2020

… Despite the limited size of the Greek corpus compared to Common Crawl and Wikipedia used for the pre-trained fastText embeddings, we didn't detect any notable difference in the quality of our models in comparison with the pre-trained one …


A Hegel, S Rao, A Celikyilmaz, B Dolan - arXiv preprint arXiv:2010.08618, 2020

Page 1. Substance over Style: Document-Level Targeted Content Transfer Allison Hegel1∗ Sudha Rao2 Asli Celikyilmaz2 Bill Dolan2 1Lexion, Seattle, WA, USA 2Microsoft Research, Redmond, WA, USA allison@lexion.ai {sudhra,aslicel,billdol}@microsoft.com Abstract …


Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translation

A Rios, M Müller, R Sennrich - arXiv preprint arXiv:2011.01703, 2020

AR Gonzales, M Müller, R Sennrich - Proceedings of the Fifth Conference on …, 2020

… Page 3. 530 corpora training dev test Language Pairs with English: de↔en Commoncrawl, Europarl-v9, Wikititles-v1 5M 250 2000 cs↔en Europarl-v9, CzEng1.7 5M 250 2000 fr↔en Commoncrawl, Europarl-v7 …


Suggesting Citations for Wikidata Claims based on Wikipedia's External References

… Offline: Given that some Wikidata items do not have an associated Wikipedia article, that many Wikipedia articles have few references, etc., it would be interesting to develop a broader corpus with more documents from the Web, perhaps from the Common Crawl …


Supervised Understanding of Word Embeddings

… In our experiments, we have used scikit-learn linear logistic regression model with a positive class weight of 2 to enhance the effect of positive words. We have used top 250k words of Fasttext Common Crawl word …


Surface pattern-enhanced relation extraction with global constraints

H Jiang, JT Liu, S Zhang, D Yang, Y Xiao, W Wang - Knowledge and Information …, 2020

Relation extraction is one of the most important tasks in information extraction. The traditional works either use sentences or surface patterns (ie, the.


Survey on RNN and CRF models for de-identification of medical free text

JL Leevy, TM Khoshgoftaar, F Villanustre - Journal of Big Data, 2020

The increasing reliance on electronic health record (EHR) in areas such as medical research should be addressed by using ample safeguards for patient privacy. These records often tend to be big data, and given that a significant …


SYMPTOM EXTRACTION FROM ATRIAL FIBRILLATION PATIENT CLINICAL NOTES USING DEEP LEARNING

Page 1. Eindhoven University of Technology MASTER Symptom extraction from atrial fibrillation patient clinical notes using deep learning van Putten, TE Award date: 2020 Link to publication Disclaimer This document contains …


Z Li, H Zhao, S He, J Cai - arXiv preprint arXiv:2009.05737, 2020

Page 1. Syntax Role for Neural Semantic Role Labeling Zuchao Li Shanghai Jiao Tong University Department of Computer Science and Engineering charlee@sjtu. edu.cn Hai Zhao∗ Shanghai Jiao Tong University Department …


System and method for model derivation for entity prediction

FI Wyss, A Ganapathiraju, P Buduguppa - US Patent App. 16/677,989, 2020

… The dense representation can be used to capture the contextual information. For an NER system, the information can be encoded in the form of “world knowledge' by using a corpus such as the Wikipedia corpus or Google's common crawl data …


Systematic Mapping on Embedded Semantic Markup Validated with Data Mining Techniques

R Navarrete, C Montenegro, L Recalde - … Conference on Applied Human Factors and …, 2020

… Markup format: microdata, rdfa, jsonld. Approach of the Research: ads, commerce, commoncrawl, crawl, deploy, education, egovernment, entity, error, extract, extraction, fix, government, learning, lod, mistake, owl, pld, plds, rdf, video, wdc, webdatacommons …


SYSTEMS AND METHODS FOR LEARNING USER REPRESENTATIONS FOR OPEN VOCABULARY DATA SETS

T Durand, G Mori - US Patent App. 16/826,215, 2020

Systems and methods adapted for training a machine learning model to predict data labels are described. The approach includes receiving a first data set comprising first data objects and associated fi.


TabEAno: Table to Knowledge Graph Entity Annotation

P Nguyen, N Kertkeidkachorn, R Ichise, H Takeda - arXiv preprint arXiv:2010.01829, 2020

… Note that, tables in this study refer to relational vertical tables. A 3 Open Data Vision: https://opendatabarometer.org 4 Common Crawl: http://commoncrawl.org/ arXiv:2010.01829v1 [cs.AI] 5 Oct 2020 Page 2. 2 Phuc Nguyen et al …


TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

P Yin, G Neubig, W Yih, S Riedel - arXiv preprint arXiv:2005.08314, 2020

… Page 5. interesting avenue for future work. Specifically, we collect tables and their surrounding NL text from English Wikipedia and the WDC WebTable Corpus (Lehmberg et al., 2016), a large-scale table collection from CommonCrawl …


Tagging Reading Comprehension Materials with Document Extraction Attention Networks

B Sun, Y Zhu, R Xiao, Y Xiao, YG Wei - IEEE Transactions on Learning …, 2020

Page 1. 1939-1382 (c) 2020 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


Tagging with Weak Labels. Paper presented at AAAI Conference on

… For FAMULUS, we use 300-dimensional German fastText embeddings (Grave et al. 2018), and for NER and PICO we use 300-dimensional English GloVe 3 embeddings trained on 840 billion tokens from Common Crawl. To …


Tailored retrieval of health information from the web for facilitating communication and empowerment of elderly people

M Alfano, B Lenzitti, D Taibi, M Helfert - 2020

… contain all Microformat, Microdata and RDFa (Resource Description Framework in Attributes) data extracted from the open repository of Web crawl data named Common Crawl (CC)9 … 8 http://webdatacommons.org/ 9 …


Target driven visual navigation exploiting object relationships

Y Qiu, A Pal, HI Christensen - arXiv preprint arXiv:2003.06749, 2020

… For the word embeddings, we used the 300-D GloVe vectors pretrained on 840 billion tokens of Common Crawl [49]. The A3C model is based on [50], and the model hyperparameters used were: learning rate …


Targeted Poisoning Attacks on Black-Box Neural Machine Translation

C Xu, J Wang, Y Tang, F Guzman, BIP Rubinstein… - arXiv preprint arXiv …, 2020

… 3We find that the crawling services commonly used for parallel data collection, eg, Common Crawl (commoncrawl.org), are also fetching news articles from self-publishing sources like blogs (eg, with a subdomain of blogspot.com) …


Tell and guess: cooperative learning for natural image caption generation with hierarchical refined attention

W Zhang, S Tang, J Su, J Xiao, Y Zhuang - Multimedia Tools and Applications, 2020

… Implementation details. We use Tensorflow to implement our model and its variants. Given a textual caption, we employ the word2vec model (ie, GloVe word embedding [22] ) which is pre-trained on the Common Crawl dataset [25] …


Tell Me Why You Feel That Way: Processing Compositional Dependency for Tree-LSTM Aspect Sentiment Triplet Extraction (TASTE)

A Sutherland, S Bensch, T Hellström, S Magg… - International Conference on …, 2020

… aligned}$$. (6). $$\begin{aligned}&{h}_{j} = o_j \odot tanh(c_j). \end{aligned}$$. (7). Words in a sentence are represented as Word Embeddings from the pre-trained Common-Crawl 840 B data 2 before they are fed to the DTLSTM. To …


Tencent AI Lab machine translation systems for the WMT20 chat translation task

L Wang, Z Tu, X Wang, L Ding, L Ding, S Shi - Proceedings of the Fifth Conference on …, 2020

… Out-of-domain Parallel Data The participants are allowed to use all the training data in the News shared task.4 Thus, we combine six corpora including Euporal, ParaCrawl, CommonCrawl, TildeRapid, NewsCommentary and WikiMatrix …


Testing pre-trained Transformer models for Lithuanian news clustering

L Stankevičius, M Lukoševičius - arXiv preprint arXiv:2004.03461, 2020

… Specifically, we will use well known baselines – multilingual BERT and recently published XLM-R, trained on more than two terabytes of filtered CommonCrawl data. We chose clustering task to also try to advance the field of data mining …


Text as data: a machine learning-based approach to measuring uncertainty

R Nyman, P Ormerod - arXiv preprint arXiv:2006.06457, 2020

… The authors assemble a very large corpus of words from various sources. We use the one described on the GloVe website as Common Crawl (glove.42B.300d.zip). A co-occurrence matrix is constructed, which describes …


Page 1. Text Classification: Exploiting the Social Network Sakhar Alkhereyf Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy under the Executive Committee of the Graduate School of …


Text mining-based construction site accident classification using hybrid supervised machine learning

MY Cheng, D Kusoemo, RA Gosno - Automation in Construction, 2020

… There are various pre-trained databases in the GloVe website that is open for public, such as Wikipedia database consists of 6 billion words and 100 dimension, Common Crawl database consists of 42 billion words and 300 …


Text-based classification of interviews for mental health--juxtaposing the state of the art

JV Wouts - arXiv preprint arXiv:2008.01543, 2020

… Model name Pretrain corpus Tokenizer type Acc Sentiment analysis belabBERT Common Crawl Dutch (non-shuffled) BytePairEncoding 95.92∗ % RobBERT Common Crawl Dutch (shuffled) BytePairEncoding 94.42 …


TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling

P Riley, N Constant, M Guo, G Kumar, D Uthus… - arXiv preprint arXiv …, 2020

… Furthermore, we demonstrate that a single model trained on unlabeled Common Crawl data is capable of transferring along multiple dimensions including dialect, emotiveness, formality, politeness, and sentiment. 1 INTRODUCTION …


TF-CR: Weighting Embeddings for Text Classification

A Zubiaga - arXiv preprint arXiv:2012.06606, 2020

… Page 6. • cglove: GloVe embeddings trained from Common Crawl. • wglove: GloVe embeddings trained from Wikipedia.6 We use two different classifiers for these experiments, SVM and Logistic Regression, which are known …


The 2019 BBN Cross-lingual Information Retrieval System

DK Le Zhang, W Hartmann, M Srivastava, L Tarlin… - LREC 2020 Language Resources …

L Zhang, D Karakos, W Hartmann, M Srivastava… - Proceedings of the …, 2020

… 4.1. Training Data The primary data source for constructing MT models is parallel data from the build language pack, augmented with a variety of web data, such as CommonCrawl2 and open parallel corpus (Tiedemann …


The 2020 bilingual, bi-directional webnlg+ shared task overview and evaluation results (webnlg+ 2020)

TC Ferreira, C Gardent, C van der Lee, N Ilinykh… - Proceedings of the 3rd …, 2020

… 3.3 Mono-task, Bilingual Approaches cuni-ufal. The mBART model (Liu et al., 2020) is pre-trained for multilingual denoising on the large-scale multilingual CC25 corpus extracted from Common Crawl, which contains …


THE ABILITY OF WORD EMBEDDINGS TO CAPTURE WORD SIMILARITIES

M Toshevska, F Stojanovska, J Kalajdjieski

… architectures [25]. In our experiments, we have used pre-trained models both trained with subword information on Wikipedia 2017 (16B tokens) and trained with subword information on Common Crawl (600B tokens)4. 2https …


The ADAPT Centre's neural MT systems for the WAT 2020 document-level translation task

… Finally, source-language monolingual data with n-grams similar to that of the documents in the test set was mined from the Common Crawl Corpus6 to be used as a source-side original synthetic corpus (SOSC) for fine-tuning the NMT model parameters …


The afrl wmt20 news-translation systems

J Gwinnup, T Anderson - Proceedings of the Fifth Conference on Machine …, 2020

… Page 2. 207 corpus unfiltered lines filtered lines percent remain commoncrawl 723,256 655,069 90.57% newscommentaryv15 319,242 286,947 89.88% yandex 1,000,000 901,318 90.13 … 2013. Dirt cheap webscale parallel text from the common crawl …


The Art of Reproducible Machine Learning

V Novotný - RASLAN 2020 Recent Advances in Slavonic Natural …, 2020

… We then use the initializations to reproduce 14 the results of Mikolov et al.(2018)[19, Table 2] us- ing the subword cbow model of Bojanowski et al.(2017)[2] and the 2017 English Wikipedia 15 training corpus (4% of the …


The birth of Romanian BERT

SD Dumitrescu, AM Avram, S Pyysalo - arXiv preprint arXiv:2009.08712, 2020

… In total, the OPUS corpus contains around 4GB of Romanian text. OSCAR OSCAR (Ortiz Suárez et al., 2019), or Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language …


The Case For Alternative Web Archival Formats To Expedite The Data-To-Insight Cycle

X Wang, Z Xie - arXiv preprint arXiv:2003.14046, 2020

… Large-scale, comprehensive web archiving initiatives include the Internet Archive [8], the Common Crawl [16], and many programs at national libraries and archives. These … 5.2 Data We chose to use Common Crawl's web …


The Challenge of Diacritics in Yoruba Embeddings

TP Adewumi, F Liwicki, M Liwicki - arXiv preprint arXiv:2011.07605, 2020

… (2018) are tabulated: Wiki, U_Wiki, C3 & CC, representing embeddings from the cleaned Wikipedia dump, its undiacritized (normalized) version, the diacritized data from Alabi et al. (2020) and the Common Crawl embedding by Grave et al. (2018), respectively …


The Danish Gigaword Project

L Strømberg-Derczynski, R Baglini, MH Christiansen… - arXiv preprint arXiv …, 2020

… Similarly, other huge monolithic datasets such as the Common Crawl Danish data suffer from large amounts of non-Danish content, possibly due to the pervasive confusion between Danish and Norwegian Bokmål … Common …


The ELTE. DH Pilot Corpus–Creating a Handcrafted Gigaword Web Corpus with Metadata

B Indig, Á Knap, Z Sárközi-Lindner, M Timári, G Palkó - … of the 12th Web as Corpus …, 2020

… Nowadays, large corpora are utilising the Common Crawl archive like the OSCAR corpus (Ortiz Suárez et al., 2019) with 5.16 billion (2.33 … All of these corpora – except the ones based on Common Crawl – have the same …


The Emergence, Advancement and Future of Textual Answer Triggering

KN Acheampong, W Tian, EB Sifah… - Science and Information …, 2020

… A similar observation is realized (\(A_1 \approx 0.8986\); \(A_2 \approx 0.7352\); difference in margin \(\approx 0.1634 \)) when the model the 300-dimensional word vectors trained on Common Crawl with GloVe from spaCy is used …


The Geometry of Distributed Representations for Better Alignment, Attenuated Bias, and Improved Interpretability

S Dev - arXiv preprint arXiv:2011.12465, 2020

… 39 5 RMSE variation with word frequency in (a) GloVe Wiki to GloVe Common Crawl and (b) word2vec to GloVe evaluated for Wiki dataset. All words were used for tests in lower case as listed in the table. . . . . 40 …


The NiuTrans Machine Translation Systems for WMT20

Y Zhang, Z Wang, R Cao, B Wei, W Shan, S Zhou… - Proceedings of the Fifth …, 2020

… mentary, Common Crawl , TED Talks 4 Japanese monolingual data corpus about 1.7 billion. After the data filter, 12 million parallel data was left and 11 million selected by the neural language model was used as training data …


The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings

B Mathew, S Sikdar, F Lemmerich, M Strohmaier - arXiv preprint arXiv:2001.09876, 2020

… (2) GloVe embeddings [27]3 trained on Web data from Common Crawl … 3We used the Common Crawl embeddings with 42B tokens: https://nlp. stanford.edu/ projects/glove/ 4The datasets are available here: https://github …


The POLUSA Dataset: 0.9 M Political News Articles Balanced by Time and Outlet Popularity

L Gebhard, F Hamborg - arXiv preprint arXiv:2005.14024, 2020

… RoBERTa: A Robustly Optimized BERT Pretraining Ap- proach. arXiv: 1907.11692 [cs] [6] Sebastian Nagel. 2016. Common Crawl – News Dataset Available. Retrieved May 8, 2020 from https://commoncrawl.org/2016/10 …


The presence of occupational structure in online texts based on word embedding NLP models

Z Kmetty, J Koltai, T Rudas - arXiv preprint arXiv:2005.08612, 2020

… We used three pre-trained vector spaces in the analysis. The first vector model we used was trained on the English language texts of the Common Crawl (CC) corpus1, a huge web archive … 2016) 1 http://commoncrawl.org 2 …


The role of affective meaning, semantic associates, and orthographic neighbours in modulating the N400 in single words

F Blomberg, M Roll, J Frid, M Lindgren, M Horne - The Mental Lexicon, 2020

… of fastText compared to other popular implementations (such as Word2Vec and Glove) is that it already has a model for Swedish trained on millions of words taken from the Swedish version of the free online encyclopedia …


The Two-Pass Softmax Algorithm

M Dukhan, A Ablavatski - arXiv preprint arXiv:2001.04438, 2020

Page 1. The Two-Pass Softmax Algorithm Marat Dukhan ∗1,2 and Artsiom Ablavatski1 1Google Research 2Georgia Institute of Technology Abstract The softmax (also called softargmax) function is widely used in machine learning …


The University of Edinburgh's English-Tamil and English-Inuktitut Submissions to the WMT20 News Translation Task

R Bawden, A Birch, R Dobreva, A Oncevay… - 5th Conference on Machine …, 2020

… The only additional monolingual Inuktitut data was 163k sentences of common-crawl data, which we backtranslated for the English→Inuktitut system … Synthetic (from en Europarl) en-iu 650k Synthetic (from en News …


The University of Edinburgh's submission to the German-to-English and English-to-German Tracks in the WMT 2020 News Translation and Zero-shot Translation …

… High-quality parallel data Europarl ca. 1.79 M Rapid ca. 1.45 M News Commentary ca. 0.35 M Crawled parallel data ParaCrawl 5.1 ca. 34.37 M CommonCrawl ca. 2.40 M WikiMatrix ca. 6.22 M WikiTitles ca. 1.38 M Monolingual crawled news data German ca …


The University of Helsinki and Aalto University submissions to the WMT 2020 news and lowresource translation tasks

Y Scherrer, SA Grönroos, S Virpioja - the Fifth Conference on Machine Translation, 2020

… NewsDiscuss 2019 2 000 000 1 000 000 CommonCrawl 80 244 80 244 … In terms of monolingual Inuktitut data, besides the unaligned NH data, the organizers only provided a CommonCrawl dump. This corpus was again backtranslated to English and filtered …


The University of Helsinki submission to the IWSLT2020 Offline Speech Translation Task

R Vázquez, M Aulamo, U Sulubacak, J Tiedemann - The 17th International …, 2020

… filter out noisy translations. OpenSubtitles2018, which consists of subtitle translations, and corpora gathered by crawling the internet, Common Crawl and ParaCrawl, are especially likely to contain noisy data. For filtering the …


The Unreasonable Effectiveness of Machine Learning in Moldavian versus Romanian Dialect Identification

M Găman, RT Ionescu - arXiv preprint arXiv:2007.15700, 2020

… We note that these representations are learned from Romanian corpora, such as the corpus for contemporary Romanian language (CoRoLa) (Mititelu, Tufis, and Irimia 2018; Pais and Tufis 2018), Common Crawl (CC) and Wikipedia (Grave et al …


The Voice and Speech Processing within Language Technology Applications: Perspective of the Russian Data Protection Law

… of collecting, systematizing and annotating language data are various language datasets such as Open Subtitles43, the Common Crawl da- taset44, the … 43 Available at: https://www.opensubtitles.org/ru (accessed: 18.05.2020) …