Learning Multimodal Word Representation via Dynamic Fusion Methods

S Wang, J Zhang, C Zong - arXiv preprint arXiv:1801.00532, 2018

… Experimental Setup Datasets We use 300-dimensional GloVe vectors3 which are trained on the Common Crawl corpus consisting of 840B tokens and a vocabulary of 2.2M words. Our source of visual vectors are collected from ImageNet (Russakovsky et al …


Learning Neural Emotion Analysis from 100 Observations: The Surprising Effectiveness of Pre-Trained Word Representations

S Buechel, J Sedoc, HA Schwartz, L Ungar - arXiv preprint arXiv:1810.10949, 2018

… For ANET, we rely on the FastText embeddings trained on Common Crawl (Mikolov et al., 2018). For ANPST and MAS, we use the FastText embeddings by Grave et al. (2018) trained on the respective Wikipedias … ANET English …


Learning Patent Speak: Investigating Domain-Specific Word Embeddings

… used to train word embeddings. It contains more than twice the number of tokens of the English Wikipedia (16 billion) and is only exceeded by the Common Crawl dataset, which consists of 600 billion tokens. We assume that the …


Learning representations for sentiment classification using Multi-task framework

H Meisheri, H Khadilkar - Proceedings of the 9th Workshop on Computational …, 2018

… First set of embeddings are generated by using three different pre-trained embeddings. • Pre-trained embeddings which are generated from common crawl corpus have 6 Billion to- kens which help in a better encoding of the …


H Pham, PP Liang, T Manzini, LP Morency, B Póczos

Page 1. Learning Robust Joint Representations for Multimodal Sentiment Analysis Hai Pham*, Paul Pu Liang*, Thomas Manzini, Louis-Philippe Morency, Barnabás Póczos School of Computer Science, Carnegie Mellon University …


Learning semantic similarity in a continuous space

M Deudon - Advances in Neural Information Processing Systems, 2018

… that connect those parts. Word2Vec [5] and GloVe [6] are semantic embeddings of words based on their context and frequency of co-occurence in a text corpus such as Wikipedia, Google News or Common-Crawl. The key idea …


Learning to Rank Query Graphs for Complex Question Answering over Knowledge Graphs

G Maheshwari, P Trivedi, D Lukovnikov, N Chakraborty… - arXiv preprint arXiv …, 2018

… All our models pe- formed poorly on QALD-7, with the best being DAM dot. 4in keeping with the train-test splits suggested by the authors. 5Trained over the common Crawl corpus. 300 dimensions, and with 1.9M tokens in the …


Learning Word Embeddings for Data Sparse and Sentiment Rich Data Sets

PK Sarma - Proceedings of the 2018 Conference of the North …, 2018

… a very different distribution from generic corpora, so pre-trained generic word embeddings such as those trained on Common Crawl or Wikipedia … d2 be the matrix of generic word embeddings (obtained by, eg, running the GloVe …


Learning Word Meta-Embeddings by Autoencoding

… (2014). The released version we used contains 1,917,494 word embeddings with dimensionality of 300 that is trained on Common Crawl dataset. The intersection of two vocabularies contains 154,076 words for which we create meta-embeddings …


Learning Word Vectors for 157 Languages

E Grave, P Bojanowski, P Gupta, A Joulin, T Mikolov - arXiv preprint arXiv …, 2018

… In this work, we contribute word vectors trained on Wikipedia and the Common Crawl, as well as three new analogy datasets to evaluate these … We also show that using the common crawl data, while being noisy, can lead to models with larger coverage, and better models for …


Lecture Video Search Engine Using Hadoop MapReduce

… While doing so, they dont need to bother. about crawling the data. Gil Elbaz, former Google employee funded Common CrawlCommonCrawl is implemented using Hadoop MapReduce Technology. Their strategy is to crawl. broadly and frequently, to cover all the web pages …


Legal Deposit Web Archives and the Digital Humanities: A Universe of Lost Opportunity?

… Restricted deposit library ac- cess requires researchers to look elsewhere for portable web data: by undertaking their own web crawls, or by utilising datasets from Common Crawl (http://commoncrawl. org/) and the Internet Archive (https://archive.org) …


Lessons from Natural Language Inference in the Clinical Domain

A Romanov, C Shivade - arXiv preprint arXiv:1808.06752, 2018

… fastText[Wiki]: fastText embeddings (Bo- janowski et al., 2017), trained on Wikipedia. 7http://commoncrawl.org/ Page 7. • fastText[CC]: fastText embeddings, trained on Common Crawl. Furthermore, we trained fastText …


Lessons Learned and Research Agenda for Big Data Integration of Product Specifications (Discussion Paper)

L Barbosa, V Crescenzi, XL Dong, P Merialdo, F Piai…

… About 68% of the sources discovered by our approach were not present in Common Crawl … these sources were product pages: on a sample set of 12 websites where Common Crawl presented … DEXTER) is an extension …


Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora

A Hazem, E Morin - Proceedings of the 27th International Conference on …, 2018

… Common crawl corpus (CC) is an open repository of data collected over 7 years of web crawling sets of raw web page data and text extracts7. 4 www.sciencedirect.com/ 5 www.ttc-project.eu/index.php/releases-publications …


Leveraging textual properties of bug reports to localize relevant source files

R Gharibi, AH Rasekh, MH Sadreddini… - Information Processing & …, 2018


Lifelong Domain Word Embedding via Meta-Learning

H Xu, B Liu, L Shu, PS Yu - arXiv preprint arXiv:1805.09991, 2018

… Page 6. GloVe.6B: This is the lower-cased embeddings pre-trained from Wikipedia and Gigaword 5, which has 6 billion tokens. GloVe.840B: This is the cased embeddings pre-trained from Common Crawl corpus, which has 840 billion tokens …


Linguistic Features of Helpfulness in Automated Support for Creative Writing

… tween the context and suggestion in terms of their Jaccard similarity (proportion of overlapping words; Metric 8), GloVe word embeddings7 trained on the Common Crawl corpus (Metric 9), and sentence (skip-thought) …


S Kim, T Lee, S Hwang, S Elnikety - Proceedings of the VLDB Endowment

Page 1. List Intersection for Web Search: Algorithms, Cost Models, and Optimizations Sunghwan Kim POSTECH sunghwan08@gmail.com Taesung Lee IBM Research AI Taesung.Lee@ibm.com Seung-won …


LMU Munich's Neural Machine Translation Systems at WMT 2018

M Huck, D Stojanovski, V Hangya, A Fraser - Proceedings of the Third Conference on …, 2018

… BLEU (Pa- pineni et al., 2002)) of different system setups after several development stages is presented in the top section of Table 1. WMT_parallel de- notes the Europarl, News Commentary, and Common Crawl parallel …


Local Homology of Word Embeddings

T Temčinas - arXiv preprint arXiv:1810.10136, 2018

Page 1. Local Homology of Word Embeddings Tadas Temcinas 1 Abstract Topological data analysis (TDA) has been widely used to make progress on a number of problems. However, it seems that TDA application in natural …


D Ustalov, A Panchenko, C Biemann, SP Ponzetto - arXiv preprint arXiv:1808.06696, 2018

Page 1. Local-Global Graph Clustering with Applications in Sense and Frame Induction Dmitry Ustalov∗ University of Mannheim Alexander Panchenko∗∗ University of Hamburg Chris Biemann University of Hamburg Simone Paolo Ponzetto University of Mannheim …


Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion

A Joulin, P Bojanowski, T Mikolov, H Jégou, E Grave - Proceedings of the 2018 …, 2018

… word vectors of Mikolov et al. (2018) to Spanish. It leads to an improvement of 1% both for vectors trained on Common Crawl (85% to 86%) and Wikipedia + News (87% to 88%). 5 Conclusion This paper shows that minimizing …


M1. 2–Corpora for the Machine Translation Engines

C Espana-Bonet, J Stiller, S Henning - 2018

… Table 5: Size of the parallel corpora obtained from the different sources described in Section 2.1: United Nations (UN), Europarl V7 (EP), Common Crawl (ComCrawl), EMEA and Scielo. Figures of the PubPsych corpus are …


MAC: Mining Activity Concepts for Language-based Temporal Localization

R Ge, J Gao, K Chen, R Nevatia - arXiv preprint arXiv:1811.08925, 2018

… We only keep the twoword tuple as our VO. ie open door, wash hands. After lemmatizing every word, we use the GloVe [22] word en- coder which is pre-trained on the Common Crawl data to get each word a 300-dimensional word embedding …


Machine Learning with and for Semantic Web Knowledge Graphs

H Paulheim - Reasoning Web International Summer School, 2018

… To that end, it scans the Common Crawl Web corpus 20 for so-called Hearst patterns, eg X such as Y, to infer relations like X skos:broader Y. The graph comes with very detailed provenance data, including the patterns used …


Machine Reading Comprehension on SQuAD

… The end-to-end neural architecture consists of the following components. Page 3. Input vectors. For each word in context (C) and question (Q), we use the 300-dim GloVe embeddings pretrained on the 840B Common Crawl corpus [5] as input vectors …


MAJE Submission to the WMT2018 Shared Task on Parallel Corpus Filtering

M Fomicheva, J González-Rubio - Proceedings of the Third Conference on Machine …, 2018

… We also conducted some initial experiments us- ing the Common Crawl corpus, under the rationale that it would be closer to the domain of the noisy data from the Paracrawl corpus. However, Common Crawl data …


Making Caches Work for Graph Analytics

Y Zhang, V Kiriansky, C Mendis, S Amarasinghe…

… The system has 128GB of DDR3-1600 memory. The machine runs with Transparent Huge Pages (THP) enabled. Data Sets: We used the social networks, including LiveJournal [10] and Twitter [8], the SD web graph from 2012 common crawl [11], and the RMAT graphs …


Manifold Scholarship: Hybrid Publishing in a Print/Digital Era

MK Gold, J Karlin, Z Davis, P Gooding, M Terras… - Digital Humanities 2018 …, 2018

… Restricted deposit library ac- cess requires researchers to look elsewhere for portable web data: by undertaking their own web crawls, or by utilising datasets from Common Crawl (http://commoncrawl. org/) and the Internet Archive (https://archive. org) …


Manual vs Automatic Bitext Extraction

A Makazhanov, B Myrzakhmetov, Z Assylbekov - Proceedings of the Eleventh …, 2018

… Biometrika, 52(3/4):591–611. Smith, JR, Saint-Amand, H., Plamada, M., Koehn, P., Callison-Burch, C., and Lopez, A. (2013). Dirt cheap web-scale parallel text from the common crawl. In ACL (1), pages 1374–1383. Tiedemann, J. (2007) …


M Artetxe, H Schwenk - arXiv preprint arXiv:1811.01136, 2018

Page 1. Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings Mikel Artetxe University of the Basque Country (UPV/EHU)∗ mikel.artetxe@ehu.eus Holger Schwenk Facebook AI Research schwenk@fb.com Abstract …


MEADE: Towards a Malicious Email Attachment Detection Engine

EM Rudd, R Harang, J Saxe - arXiv preprint arXiv:1804.08162, 2018

… To this end, we collected a dataset of over 5 million malicious/benign Microsoft Office documents from VirusTotal for evaluation as well as a dataset of benign Microsoft Office documents from the Common Crawl …


Measuring Semantic Coherence of a Conversation

S Vakulenko, M de Rijke, M Cochez, V Savenkov… - arXiv preprint arXiv …, 2018

… We utilise two publicly available word embedding models: GloVe embeddings pretrained on the Common Crawl corpus (2.2M words, 300 dimensions)14 and Word2Vec model trained on the Google News corpus …


Measuring the Effects of Data Parallelism on Neural Network Training

CJ Shallue, J Lee, J Antognini, J Sohl-Dickstein… - arXiv preprint arXiv …, 2018

… For LM1B, we chose the Transformer model and an LSTM model. For Common Crawl, we chose the Transformer model … Common Crawl Text Language modeling ∼25.8 billion sentences Cross entropy error Table …


Meet the Data

… Size refers to compressed data ahttps://lemurproject.org/clueweb09/ bhttps://lemurproject.org/clueweb12/ chttp://commoncrawl.org/2017/06/may … Common Crawl Common Crawl5 is a nonprofit organization that regularly …


Memory Fusion Network for Multi-view Sequential Learning

A Zadeh, PP Liang, N Mazumder, S Poria, E Cambria… - arXiv preprint arXiv …, 2018

… The Glove embeddings used are 300 dimensional word embeddings trained on 840 billion tokens from the common crawl dataset, resulting in a sequence of dimension T × dxtext = T × 300 after alignment. The timing of word utterances is extracted using P2FA forced aligner …


Merging Search Results Generated by Multiple Query Variants Using Data Fusion

N Motlogelwa, T Leburu-Dingalo, E Thuma - CEUR Workshop Proceedings: Working …, 2018

… In Section 3.2 we describe the queries used for retrieval. 3.1 Document Collection “The document collection used in CLEF 2018 consists of web pages acquired from the CommonCrawl. An initial list of websites was identified for acquisition …


METHODS AND SYSTEMS FOR QUERY SEGMENTATION

AG Kale, T Taula, A Srivastava, S Hewavitharana - US Patent App. 15/681,663, 2018

… This process may be repeated but using pre-trained GloVe vectors on common crawl and facebook fasttext pretrained model over a Wikipedia corpus with 2.5 M word vocabulary. Table 2 shows the experimental results of this analysis …


microNER: A Micro-Service for German Named Entity Recognition based on BiLSTM-CRF

G Wiedemann, R Jindal, C Biemann - arXiv preprint arXiv:1811.02902, 2018

… It is rather determined by the fact that it was trained on several billion tokens of the Common Crawl corpus.2 To evaluate the contribution of different German word … 2http://commoncrawl org 3For fastText, we use the publicly available German model from Bojanowski et al …


Mining Training Data for Language Modeling Across the World's Languages

… 64 Page 5. [16] Common crawlCommon Crawl Foundation. Ac- cessed: June 4, 2018. [Online]. Available: https://www.commoncrawl.org [17] UL Abteilung Automatische Sprachverarbeitung. Corpora collection. Deutscher …


Miracl at Clef 2018: Consumer Health Search Task

S ZAYANI, N KSENTINI, M TMAR, F GARGOURI - CEUR Workshop Proceedings …, 2018

… concepts. Page 6. 4 Resources employed 4.1 Datasets The document collection used in CLEFeHealth 2018 is composed of web pages acquired from the CommonCrawl An initial list of websites was identified for acquisition. The …


MIsA: Multilingual" IsA" Extraction from Corpora

S Faralli, E Lefever, S Paolo Ponzetto - the Eleventh International Conference on …, 2018

… 2016). Our MIsA is an extension of the WebIsADb framework (Seitner et al., 2016) - a publicly available database with more than 400 million English hypernymy relations ex- tracted from the CommonCrawl web corpus - where: 1 …


Mitigating Bottlenecks in Wide Area Data Analytics via Machine Learning

H Wang, B Li - IEEE Transactions on Network Science and …, 2018

Page 1. 2327-4697 (c) 2018 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework

S Kiyono, J Suzuki, K Inui - arXiv preprint arXiv:1810.05788, 2018

… in DNNs has been gaining the performance. For example, several studies have utilized unlabeled 1http://commoncrawl.org arXiv:1810.05788v1 [cs.CL] 13 Oct 2018 Page 2. data as additional training data, which essentially …


Modeling Empathy and Distress in Reaction to News Stories

S Buechel, A Buffone, B Slaff, L Ungar, J Sedoc - arXiv preprint arXiv:1808.10399, 2018

… problems. The input to our models is based on word embeddings, namely the publicly available FastText embeddings which were trained on Common Crawl (≈600B tokens) (Bojanowski et al., 2017; Mikolov et al., 2018). Ridge …


Modeling voice calls to improve an outcome of a call between a representative and a customer

R Raanani, R Levy, MY Breakstone - US Patent App. 16/017,646, 2018

R Raanani, R Levy, MY Breakstone - US Patent App. 16/689,688, 2020

… At the same time, natural language processing (NLP) approaches to both topic modeling and world-knowledge modeling, have become much more efficient due to the availability of large, freely accessible natural language corpora (eg, CommonCrawl), as well as freely …


Multi-task Projected Embedding for Igbo

I Ezeani, M Hepple, I Onyenwe, C Enemuo - International Workshop on Temporal …, 2018

… igSwproj from same as igWkproj but with subword information. igCrlproj from fastText Common Crawl dataset. Table 3 shows the vocabulary lengths (\(Vocabs^L\)), and the dimensions (Dimension) of each of the models used in our experiments …


Multi-turn QA: A RNN Contextual Approach to Intent Classification for Goal-oriented Systems

M Mensio, G Rizzo, M Morisio - Companion of the The Web Conference 2018 on The …, 2018

… 8. Christian Buck, Kenneth Heafield, and Bas van Ooyen. 2014. N-gram Counts and Language Models from the Common Crawl Proceedings of the Language Resources and Evaluation Conference (LREC), Vol. Vol. 2. Citeseer, Reykjavik, Iceland, 4 …


Multilingual word embeddings and their utility in cross-lingual learning

… language corpus. When said corpus is large enough (eg Wikipedia, Common Crawl, or the concatenation of the two), the resulting DSM can be assumed to represent the distributional semantics of an entire language. The algorithms …


PP Liang, Z Liu, A Zadeh, LP Morency - arXiv preprint arXiv:1808.03920, 2018

Page 1. Multimodal Language Analysis with Recurrent Multistage Fusion Paul Pu Liang1, Ziyin Liu2, Amir Zadeh2, Louis-Philippe Morency2 1Machine Learning Department, 2Language Technologies Institute Carnegie Mellon …


Multimodal Language Analysis with Recurrent Multistage Fusion: Supplementary Material

… 1.1 Multimodal Features Here we present extra details on feature extraction for the language, visual and acoustic modalities. Language: We used 300 dimensional Glove word embeddings trained on 840 billion tokens from …


Natural language processing using a neural network

B McCann, C Xiong, R Socher - US Patent App. 16/000,638, 2018

… in the second language. In some examples, training of an MT-LSTM of the encoder 310 uses fixed 300-dimensional word vectors, such as the CommonCrawl-840B GloVe model for English word vectors. These word vectors …


Navigating Online Semantic Resources for Entity Set Expansion

WT Adrian, M Manna - International Symposium on Practical Aspects of …, 2018

… the given entry (synset) in BabelNet. WebIsADatabase [25] is a publicly available database containing more than 400 million hypernymy relations extracted from the CommonCrawl web corpus. The tuples of the database are …


Near Human-Level Performance in Grammatical Error Correction with Hybrid Machine Translation

R Grundkiewicz, M Junczys-Dowmunt - arXiv preprint arXiv:1804.05945, 2018

… ops). All systems use a 5-gram Language Model (LM) and OSM (Durrani et al., 2011) both estimated from the target side of the training data, and a 5-gram LM and 9-gram WCLM trained on Common Crawl data (Buck et al., 2014) …


Neural Adaptation Layers for Cross-domain Named Entity Recognition

BY Lin, W Lu - arXiv preprint arXiv:1810.06368, 2018

… ter stream grab4); 3) general emb, which is pretrained on CommonCrawl containing both formal and user-generated content.5 We obtain the intersection of the top 5K words from source and target vocabularies sorted by frequency to build P1 …


J Xie, Z Yang, G Neubig, NA Smith, J Carbonell - arXiv preprint arXiv:1808.09861, 2018

Page 1. Neural Cross-Lingual Named Entity Recognition with Minimal Resources Jiateng Xie,1 Zhilin Yang,1 Graham Neubig,1 Noah A. Smith,2,3 and Jaime Carbonell1 1Language Technologies Institute, Carnegie Mellon University …


Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss

P Xu, D Barbosa - arXiv preprint arXiv:1803.03378, 2018

… appearing in the training set. For this purpose, we used the freely available 300-dimensional cased word embedding trained on 840 billion tokens from the Common Crawl supplied by Pennington et al. (2014). For both datasets, we …


Neural Learning for Question Answering in Italian

D Croce, A Zelenanska, R Basili - International Conference of the Italian Association …, 2018

… In [5], both the paragraph and the question encodings use \(d=128\) hidden units. The best performing system reported in [5] uses GloVe embeddings [14] trained on Common Crawl of dimension \(n=300\) with more than 2 millions of tokens …


Neural Machine Translation with Decoding History Enhanced Attention

M Wang, J Xie, Z Tan, J Su, D Xiong, C Bian - … of the 27th International Conference on …, 2018

… translation are presented in Table 2. We compare our NMT systems with various other systems including the winning system in WMT14 (Buck et al., 2014), a phrase-based system whose language models were trained on …


Neural models of factuality

R Rudinger, AS White, B Van Durme - arXiv preprint arXiv:1804.02472, 2018

Page 1. Neural models of factuality Rachel Rudinger Johns Hopkins University Aaron Steven White University of Rochester Benjamin Van Durme Johns Hopkins University Abstract We present two neural models for event fac …


Page 1. NEURAL NETWORKS FOR NARRATIVE CONTINUATION by Melissa Roemmele A Ph.D. Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA …


Neural Networks for Semantic Textual Similarity

… tors. This specific set of word vectors have 300 dimensions and were pre-trained on 840 billion tokens taken from Common Crawl3. Different pre-trained word vectors may be used in place of this specific pre-trained set. After …


Neural Quality Estimation of Grammatical Error Correction

S Chollampatt, HT Ng - Proceedings of the 2018 Conference on Empirical …, 2018

Page 1. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2528–2539 Brussels, Belgium, October 31 - November 4, 2018. c 2018 Association for Computational Linguistics 2528 …


Next Utterance Ranking Based On Context Response Similarity

BEA Boussaha, N Hernandez, C Jacquin, E Morin

… E. Parameter tuning Word embeddings were initialized with Glove [21] pretrained on Common Crawl Corpus4 then fine tuned during training5 … 4http://commoncrawl.org/the-data/ 5Note that we trained word embeddings …


NICT's Neural and Statistical Machine Translation Systems for the WMT18 News Translation Task

B Marie, R Wang, A Fujita, M Utiyama, E Sumita - arXiv preprint arXiv:1809.07037, 2018

… As English monolingual data, we used all the available data except the “Common Crawl” and “News Discussions” corpora.2 For all other languages, we used all the available monolingual corpora, except for Turkish for which we …


NL-FIIT at IEST-2018: Emotion Recognition utilizing Neural Networks and Multi-level Preprocessing

S Pecar, M Farkaš, M Simko, P Lacko, M Bielikova - Proceedings of the 9th Workshop …, 2018

… input words, we used various pre-trained word embeddings available online like GloVe em- beddings1. With GloVe embeddings, we experimented with domain specific embeddings trained on Twitter data but also with embeddings …


Noising and Denoising Natural Language: Diverse Backtranslation for Grammar Correction

Z Xie, G Genthial, S Xie, A Ng, D Jurafsky - Proceedings of the 2018 Conference of …, 2018

… h) (Wu et al., 2016). The final scoring function also incorporates a 5-gram language model trained on a subset of Common Crawl, estimated with Kneser-Ney smoothing using KenLM (Heafield, 2011). We in- corporate the …


Not just about size-A Study on the Role of Distributed Word Representations in the Analysis of Scientific Publications

A Garcia, JM Gomez-Perez - arXiv preprint arXiv:1804.01772, 2018

… our evaluation task, embeddings from a scientific publication corpus consistently generate classifiers with a top performance that is only matched by classifiers learned from embeddings from very large document corpora …


Novelty Goes Deep. A Deep Neural Solution To Document Level Novelty Detection

T Ghosal, V Edithal, A Ekbal, P Bhattacharyya…

… Training is done with a mini-batch size of 64. GloVe vectors trained on Common Crawl 840B (https://nlp.stanford.edu/projects/glove/) with 300 dimensions are used as fixed word embeddings. 7SNLI has 3 classes of …


NTT's Neural Machine Translation Systems for WMT 2018

M Morishita, J Suzuki, M Nagata - Proceedings of the Third Conference on Machine …, 2018

… This year's submission includes the following features: • Noisy data filtering for Common Crawl and ParaCrawl corpora (Section 3.1) … This year, ParaCrawl and Common Crawl corpora, which were created by crawling parallel websites, were provided for training …


NTU NLP Lab System at SemEval-2018 Task 10: Verifying Semantic Differences by Integrating Distributional Information and Expert Knowledge

YT Shiue, HH Huang, HH Chen - Proceedings of The 12th International Workshop on …, 2018

… al., 2017). We use the pre-trained embeddings of English concepts3. 4. GloVe(Common Crawl): The GloVe model (Pennington et al., 2014) obtains word representation according to global co-occurrence statistics. We use …


On Link Stability Detection for Online Social Networks

J Zhang, X Tao, L Tan, JCW Lin, H Li, L Chang - International Conference on …, 2018

… Since the social network we obtain from the repositories of common crawl contains missing links and partial information, stochastic estimations are used to measure the accuracy and reliability of our experimental MVVA results [12] …


On NDN and (“lack of”) Measurement

… URLs Dataset • Web Crawling of 7 main organizations – Amazon, Ask, Stackoverflow, BBC, CNN, Google, Yahoo – Common Crawl Data Set repository • 1.73B URLs -> 7M for each organization /(Organization) …


On the Compressed Sensing Properties of Word Embeddings

… word2vec embeddings trained on Google News and GloVe vectors trained on Common Crawl were obtained from public repositories [20, 23] while Amazon and Wikipedia embeddings were trained for 100 iterations using …


On the Design and Tuning of Machine Learning Models for Language Toxicity Classification in Online Platforms

M Rybinski, W Miller, J Del Ser, MN Bilbao… - International Symposium on …, 2018

… For (B) we have used vectors pre-trained on Common Crawl corpus with GloVe algorithm [9]. A more complete description of the text representations involved in our experiments is given below … For pre-trained embeddings …


Ontology Augmentation Through Matching with Web Tables

… 3 http://commoncrawl.org … The used PageRank values are obtained from the publicly available Common Crawl WWW Ranking.4 For each partition of columns, we use the maximum PageRank of all source web pages and …


Ontology Driven Extraction of Research Processes

V Pertsas, P Constantopoulos, I Androutsopoulos

… Our experiments with other general-purpose, publicly available embeddings, such as those trained on the Common Crawl corpus using GloVe9, or those trained on Wikipedia articles with word2vec, showed inferior performance …


Open Bibliometrics and Undiscovered Public Knowledge

D Stuart - Online Information Review, 2018

… Whether altmetrics is really any more open than traditional citation analysis is a matter of debate, although services such as Common Crawl (http://commoncrawl.org), an open repository of web crawl data, provides …


OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models

O Kuchaiev, B Ginsburg, I Gitman, V Lavrukhin, C Case… - arXiv preprint arXiv …, 2018

… training). In our experiments, we used WMT 2016 English→German data set obtained by combining the Europarlv7, News Commentary v10, and Common Crawl corpora and resulting in roughly 4.5 million sentence pairs …


Optimizing Automatic Evaluation of Machine Translation with the ListMLE Approach

M Li, M Wang - ACM Transactions on Asian and Low-Resource …, 2018

… Bilingual parallel data comprising Europarl v7, Common Crawl corpus, and News Commentary v10, released by the WMT'2015 Machine Translation Shared Task [39] were employed to train the bidirectional lexical translation probability …


Out-of-Distribution Detection using Multiple Semantic Label Representations

G Shalev, Y Adi, J Keshet - arXiv preprint arXiv:1808.06664, 2018

… respectively. The third and forth representations were based on GloVe [36], where the third one was trained using both Wikipedia corpus and Gigawords [34] dataset, the fourth was trained using Common Crawl dataset. The …


Pangloss: Fast Entity Linking in Noisy Text Environments

M Conover, M Hayes, S Blackburn, P Skomoroch… - arXiv preprint arXiv …, 2018

… For each surface form Pangloss calls a RocksDB key-value store to retrieve candidate entries (represented by circles) based on associations between hyperlink anchor text and Wikipedia URLs in Wikipedia and Common Crawl (Section 3.5) … 3.3.4 Common Crawl …


ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations

J Wieting, K Gimpel - Proceedings of the 56th Annual Meeting of the …, 2018

… Dataset Avg. Length Avg. IDF Avg. Para. Score Vocab. Entropy Parse Entropy Total Size Common Crawl 24.0±34.7 7.7±1.1 0.83±0.16 7.2 3.5 0.16M … (2017). Its training data includes four sources: Common Crawl, CzEng 1.6 (Bojar …


Page 1. Periodizing Web Archiving: Biographical, Event-Based, National and Autobiographical Traditions Richard Rogers INTRODUCTION: HISTORIOGRAPHIES BUILT INTO WEB ARCHIVES The purpose of this chapter is …


Phrase-Level Metaphor Identification using Distributed Representations of Word Meaning

O Zayed, JP McCrae, P Buitelaar - NAACL HLT 2018, 2018

… 10. –GloVe Common Crawl5: We used a pretrained model on the Common Crawl dataset containing 840 billion tokens of web data (about 2 million words). The vectors are 300dimensional using 100 training iteration. For …


Phrase-level Self-Attention Networks for Universal Sentence Encoding

W Wu, H Wang, T Liu, S Ma - Proceedings of the 2018 Conference on Empirical …, 2018

… and sentence textual similarity. 3.1 Model Configuration 300-dimensional GloVe (Pennington et al., 2014) word embeddings (Common Crawl, uncased) are used to represent words. Following Parikh et al. (2016), out-of-vocabulary …


Platypus–A Multilingual Question Answering Platform for Wikidata

TP Tanon, MD de Assunçao, E Caron, FM Suchanek

… The template analyzer is implemented using RasaNLU [34]. We used the Glove [35] word vectors trained on Common Crawl provided by Spacy and the RasaNLU entity extractor based on the CRFsuite library [36]. Our system can be accessed in three ways …


Page 1. Pointer-CNN for Visual Question Answering Jakob Svidt Aalborg University jsvidt13@student.aau.dk Jens Søholm Jepsen Aalborg University jjepse12@student. aau June 14, 2018 Abstract Visual Question Answering …


Pooling Word Vector Representations Across Models

AC Graesser, V Rus - … Linguistics and Intelligent Text Processing: 18th …

… The model was trained on non-zero elements in a global word co-occurrence matrix. We used the pre-trained model GloVe-42B which was trained on 42 billion words of Common Crawl corpus and it contains about 1.9 million unique tokens …


Predicting and Generating Discussion Inspiring

… response time before being presented to the human. • An LSTM model [2] [4] using pretrained 300 dimensional GloVe word embeddings [5] on Common Crawl to embed the comments. A self-attention layer was added on top …


Predicting Company Ratings through Glassdoor

… This dataset has 300-dimnesional vectors for 3 million words and phrases. The GloVe pre-trained embeddings come from a model trained on 42 billion tokens encountered by Common Crawl, a program designed to crawl the web and extract text …


Predictive Embeddings for Hate Speech Detection on Twitter

R Kshirsagar, T Cukuvac, K McKeown, S McGregor - arXiv preprint arXiv:1809.10644, 2018

… 5 Experimental Setup We tokenize the data using Spacy (HonnibalandJohnson, 2015). We use 300 Dimensional Glove Common Crawl Embeddings (840B Token) (Pennington et al., 2014) and fine tune them for the task. We …


Preferred Answer Selection in Stack Overflow: Better Text Representations... and Metadata, Metadata, Metadata

X Xu, A Bennett, D Hoogeveen, JH Lau, T Baldwin

… Word embeddings were set to 150 di- mensions. The co-occurrence weighting function's maximum value xmax was kept at the default of 10. For SEMEVAL, we used pretrained Common Crawl cased embeddings with 840G tokens …


Preserved Structure Across Vector Space Representations

A Amatuni, E He, E Bergelson - arXiv preprint arXiv:1802.00840, 2018

… We use the set of vectors pretrained by the GloVe authors on the Common Crawl corpus with 42 billion tokens, resulting in 300 dimensional vectors for 1.9 million unique words1. Such vectors have shown promise in modeling early semantic networks (Amatuni & Bergelson …


Probabilistic German Morphosyntax

Page 1. Probabilistic German Morphosyntax HABILITATIONSSCHRIFT zur Erlangung der Lehrbefähigung für das Fach Germanistische und Allgemeine Sprachwissenschaft vorgelegt der Philosophischen Fakultät II der …


PROMT Systems for WMT 2018 Shared Translation Task

A Molchanov - Proceedings of the Third Conference on Machine …, 2018

… The CommonCrawl and (especially) ParaCrawl corpora were heavily filtered and normalized using the PROMT tools and algorithms (including language recognition, removal of meaningless sentences, in-house tools for parallel …


Pseudo Descriptions for Meta-Data Retrieval

T Gollub, E Genc, N Lipka, B Stein - Proceedings of the 2018 ACM SIGIR International …, 2018

… As reference collections, the TREC collections themselves as well as Wikipedia and the CommonCrawl are used … 2Not considered were lists, disambiguations, and short (#words < 250) articles. 3http://commoncrawl …


QED: A fact verification system for the FEVER shared task

J Luken, N Jiang, MC de Marneffe - Proceedings of the First Workshop on Fact …, 2018

… described below. 4.2 Embedding We used GloVe word embeddings (Pennington et al., 2014) with 300 dimensions pre-trained us- ing CommonCrawl to get a vector representation of the evidence sentence. We also experimented …


Quantifying macroeconomic expectations in stock markets using Google Trends

J Bock - arXiv preprint arXiv:1805.00268, 2018

… trends.google.com/trends/, March 5, 2018. 3 Common Crawl (42B tokens) GloVe word embeddings, retrieved from Stanford University, https://nlp.stanford.edu/projects/ glove/, March 4, 2018. 4 GloVe word embeddings are vector …


Quantitative Web History Methods

A Cocciolo - The SAGE Handbook of Web History, 2018


Quantum-like Generalization of Complex Word Embedding: a lightweight approach for textual classification

… (B) - crawl-300d-2M-vec ‡ 2 Million 600 Billion (C) - GloVe.Common Crawl.840B.300d † 2.2 Million 840 Billion Table 1. The pre-trained word embedding models selected for this experiment, where †= GloVe algorithm embeddings, ‡= Fasttext algorithm embeddings …


Quester: A Speech-Based Question Answering Support System for Oral Presentations

R Asadi, H Trinh, HJ Fell, TW Bickmore - … of the 23rd International Conference on …, 2018

… ( , ) = √ 2 =0 (2) ( ) is the word vector representation of keyword k. We used a pre-trained GloVe [14] vector representation with 1.9 million uncased words and vectors with 300 elements, trained using 42 billion tokens of web data from Common Crawl …


Question Answering on SQuAD Dataset

… We use the 300-dimensional case-insensitive Common Crawl GloVe word embeddings [7], and do not retrain the embeddings during training … We believe this is mainly because the Common Crawl version has a much larger …


Question Answering System with Question Type Modelling

… and bug fixing. It might be also useful to consider additional training data sets with more variety of question types present and using a larger pretrained word vectors set, such as CommonCrawl.840B.300d. Acknowledgments I …


Ranking Documents by Answer-Passage Quality

E Yulianti, RC Chen, F Scholer, WB Croft, M Sanderson - 2018

… We use the same set of word embeddings learned from the Y!A data (as with EmbYA), but the effectiveness is roughly comparable to a pre-trained model learned on the Common Crawl data [9]. For all methods tested in …


Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study

T Ge, F Wei, M Zhou - arXiv preprint arXiv:1807.01270, 2018

Page 1. Microsoft Research Technical Report REACHING HUMAN-LEVEL PERFORMANCE IN AUTOMATIC GRAMMATICAL ERROR CORRECTION: AN EMPIRICAL STUDY Tao Ge, Furu Wei, Ming Zhou Natural Language …


Reasoning with Sarcasm by Reading In-between

Y Tay, LA Tuan, SC Hui, J Su - arXiv preprint arXiv:1805.02856, 2018

… We use the GloVe model trained on 2B Tweets for the Tweets and Reddit dataset. The Glove model trained on Common Crawl is used for the Debates corpus. The size of the word em- beddings is fixed at d = 100 and are fine-tuned during training …


Recommendation System Developer

E Nandini, J Neal, T Olson, C Prater-Lee

… This matrix is mapped to a vector space; GloVe uses a least squares regression model to minimize the dot product between word vectors. spaCy trains GloVe on Common Crawl corpus by default, which offers free web page data …


Refining Word Embeddings Using Intensity Scores for Sentiment Analysis

Page 1. 2329-9290 (c) 2017 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation

H Khayrallah, B Thompson, K Duh, P Koehn - Proceedings of the 2nd Workshop on …, 2018

… large, out-of-domain corpus we utilize bi- text from WMT2017 (Bojar et al., 2017),4 which contains data from several sources: Europarl parliamentary proceedings (Koehn, 2005),5 News Commentary (political and economic …


Report on the Third Quality Translation Shared Task

P Williams, P Koehn, O Bojar, T Kocmi, L Specia… - 2018

Page 1. This document is part of the Research and Innovation Action “Quality Translation 21 (QT21)”. This project has received funding from the European Union's Horizon 2020 program for ICT under grant agreement no. 645452 …


Representativeness of Latent Dirichlet Allocation Topics Estimated from Data Samples with Application to Common Crawl

Y Du, A Herzog, A Luckow, R Nerella, C Gropp, A Apon

Abstract—Common Crawl is a massive multi-petabyte dataset hosted by Amazon. It contains archived HTML web page data from 2008 to date. Common Crawl has been widely used for text mining purposes. Using data extracted from Common Crawl has several advantages


Reproducible Web Corpora: Interactive Archiving with Automatic Quality Assessment

J KIESEL, F KNEIST, M ALSHOMARY, B STEIN… - 2018

… since disappeared. The Common Crawl [36] is also missing many of such resources. Other … the web. As a population of web pages to draw a sample from, we resort to the recent billion-page Common Crawl 2017-04 [36]. From …


Research Data Management

… the web as graph data - 886 m. nodes (07/2018) - 5.4 bn. edges (07/2018) THE COMMON CRAWL ARCHIVE, http://commoncrawl.org Some research questions - prevalence of Web advertising - etymologies of words …


Page 1. WORKSHOP REPORT Research Frontiers in Information Retrieval Report from the Third Strategic Workshop on Information Retrieval in Lorne (SWIRL 2018) Editors J. Shane Culpepper, Fernando Diaz, and Mark D. Smucker …


RI-Match: Integrating Both Representations and Interactions for Deep Semantic Matching

L Chen, Y Lan, L Pang, J Guo, J Xu, X Cheng - Asia Information Retrieval Symposium, 2018

… First, we introduce our experimental settings, including parameter setting, and evaluation metrics. Parameter Settings. We initialize word embeddings in the word embedding layer with 300-dimensional Glove word vectors pre-trained in the 840B Common Crawl corpus …


VL Pochat, T Van Goethem, W Joosen - arXiv preprint arXiv:1806.01156, 2018

Page 1. Rigging Research Results by Manipulating Top Websites Rankings Victor Le Pochat, Tom Van Goethem and Wouter Joosen imec-DistriNet, KU Leuven 3001 Leuven, Belgium Email: firstname.lastname@cs.kuleuven.be …


Risk Analysis of Information-Leakage Through Interest Packets in NDN

D Kondo, T Silverston, H Tode, T Asami, O Perrin

… We collected URLs from the data repository provided by Common Crawl and we evaluate the performances of our per-packet filters … All the URLs in our data set provided by Common Crawl did not necessarily have <fragment> part as defined in RFC 1808 …


Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots

Y Yoon, WR Ko, M Jang, J Lee, J Kim, G Lee - arXiv preprint arXiv:1810.12541, 2018

… In the space of word embedding, words of similar meaning have similar representations, so understanding natural language is easier. We used the pretrained word embedding model GloVe, trained on the Common Crawl corpus [21] …


S3BD: Secure Semantic Search over Encrypted Big Data in the Cloud

J Woodworth, MA Salehi - arXiv preprint arXiv:1809.07927, 2018

Page 1. CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 0000; 00:1–22 Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/cpe …


Sandpiper: Scaling Probabilistic Inferencing to Large Scale Graphical Models

A Ulanov, M Marwah, M Kim, R Dathathri, C Zubieta…

… It is the hyperlinked graph obtained from a web crawl conducted by Common Crawl in August 2012 [32] … We used a real, large-scale web graph for this use case. We derived it from the web graph available at Web Data Commons [30], based on the Common Crawl data [32] …


E Kiperwasser, M Ballesteros - arXiv preprint arXiv:1804.08915, 2018

Page 1. Scheduled Multi-Task Learning: From Syntax to Translation Eliyahu Kiperwasser∗ Computer Science Department Bar-Ilan University Ramat-Gan, Israel elikip@gmail.com Miguel Ballesteros IBM Research 1101 …


SDC: structured data collection by yourself

T Ohshima, M Toyama - Proceedings of the 8th International Conference on …, 2018

… PVLDB 1, 1 (2008), 538--549. http://www.vldb.org/pvldb/171453916.pdf. 2. Common Crawl [nd]. Common Crawl. http://commoncrawl.org/. ([nd]). 3. DataHub - Frictionless Data [nd]. DataHub - Frictionless Data. http://datahub.io/. ([nd]) …


Searching Arguments in German with ArgumenText

… for arguments on a topic such as “nuclear energy”, it first retrieves relevant documents via Elasticsearch from a large collection of documents, such as Common Crawl2. In the … 1A demonstrator is publicly available at …


Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings

M Tkachenko, CC Chia, H Lauw - Proceedings of the 56th Annual Meeting of the …, 2018

… embeddings. The previous preoccupation centers around corpus size, ie, a larger corpus is perceived to be richer in statistical information. For instance, popular corpora include Wikipedia, Common Crawl, and Google News. We …


J Li, A Sun, S Joty - IJCAI. Under Review, 2018

Page 1. SEGBOT: A Generic Neural Text Segmentation Model with Pointer Network Jing Li, Aixin Sun and Shafiq Joty School of Computer Science and Engineering, Nanyang Technological University, Singapore …


Semantic Term" Blurring" and Stochastic" Barcoding" for Improved Unsupervised Text Classification

RF Martorano III - arXiv preprint arXiv:1811.02456, 2018

… trained on millions of documents from Google News. In the case of later models, they have trained on common crawl, a dataset of billions of web pages. The high level intuition with these models, is that terms used in similar contexts, likely have similar semantics …


Semi-Supervised Neural System for Tagging, Parsing and Lemmatization

P Rybak, A Wróblewska - CoNLL 2018, 2018

… For Uyghur language only 3M words are available. The provided data sets come either from Wikipedia or Commom Crawl. Where it is possible we choose the sentences from Common Crawl, due to longer (on average) sentence sizes …


Sentence Classification for Investment Rules Detection

Y Mansar, S Ferradans - Proceedings of the First Workshop on Economics and …, 2018

… embedding. This is justified by the fact that some words used in prospectuses are uncommon in the general use of language and thus are not included in available word vectors pre-trained on Wikipedia or common crawl alone …


Sentence Encoding with Tree-constrained Relation Networks

L Yu, CM d'Autume, C Dyer, P Blunsom, L Kong… - arXiv preprint arXiv …, 2018

Page 1. SENTENCE ENCODING WITH TREE-CONSTRAINED RE- LATION NETWORKS Lei Yu Cyprien de Masson d'Autume Chris Dyer Phil Blunsom Lingpeng Kong Wang Ling DeepMind {leiyu, cyprien, cdyer, pblunsom, lingpenk, lingwang}@google.com ABSTRACT …


Sentence Modeling via Multiple Word Embeddings and Multi-level Comparison for Semantic Textual Similarity

HN Tien, MN Le, Y Tomohiro, I Tatsuya - arXiv preprint arXiv:1805.07882, 2018

… The em- bedding representations in fastText are 300dimensional vectors. • GloVe is a 300-dimensional word embedding model learned on aggregated global wordword co-occurrence statistics from Common Crawl (840 billion tokens) …


Sentence Selection and Weighting for Neural Machine Translation Domain Adaptation

R Wang, M Utiyama, A Finch, L Liu, K Chen, E Sumita - IEEE/ACM Transactions on …, 2018

Page 1. 2329-9290 (c) 2018 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


Sentence Similarity Learning Method based on Attention Hybrid Model

Y Wang, X Di, J Li, H Yang, L Bi - Journal of Physics: Conference Series, 2018

… with our method. 4.3. Experimental Setup We initialize word representation in the word embedding layer with the 300-dimensional GloVe word vectors pre-trained from the Common Crawl Corpus [18]. Embeddings for words …


Sentence Simplification with Memory-Augmented Neural Networks

T Vu, B Hu, T Munkhdalai, H Yu - arXiv preprint arXiv:1804.07445, 2018

… We used the same hyperparameters across all datasets. Word embeddings were initialized either randomly or with Glove vectors (Pennington et al., 2014) pre-trained on Common Crawl data (840B tokens), and fine-tuned during training …


SentEval: An Evaluation Toolkit for Universal Sentence Representations

A Conneau, D Kiela - arXiv preprint arXiv:1803.05449, 2018

… Continuous bag-of-words embeddings (average of word vectors). We consider the most commonly used pretrained word vectors available, namely the fastText (Mikolov et al., 2017) and the GloVe (Pennington et al., 2014) vectors trained on CommonCrawl …


Sentiment Bias in Predictive Text Recommendations Results in Biased Writing

… lending, or law enforcement—if the data sets used to train the algorithms are bi- ased [2]. Such biased data sets are more common than initially suspected: Recent work demonstrated that two popular text corpora, the …


Sentiment Expression Boundaries in Sentiment Polarity Classification

R Kaljahi, J Foster - Proceedings of the 9th Workshop on Computational …, 2018

… The input layer for these systems is the concatenation of an embedding layer, which uses pre-trained GloVe (Pennington et al., 2014) word embeddings 2 (1.9M vocabulary Common Crawl), concatenated with a one-hot vector …


Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples

M Cheng, J Yi, H Zhang, PY Chen, CJ Hsieh - arXiv preprint arXiv:1803.01128, 2018

… one is trained from scratch. For the machine translation task, we train our model using 453k pairs from the Europal corpus of German-English WMT 157, common crawl and news-commentary. We use the hyper-parameters suggested …


Sequence-to-sequence Models for Cache Transition Systems

… Hidden state sizes for both encoder and decoder are set to 100. The word embeddings are initialized from Glove pretrained word embeddings (Pennington et al., 2014) on Common Crawl, and are not updated during training …


Shortcutting Label Propagation for Distributed Connected Components

S Stergiou, D Rughwani, K Tsioutsiouliklis - … Conference on Web Search and Data …, 2018

… Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references. 1. 2016. Common Crawl. (2016). http://commoncrawl.org/. 2. 2016. Twitter Graph. (2016) …


Simple Algorithms For Sentiment Analysis On Sentiment Rich, Data Poor Domains.

PK Sarma, W Sethares - Proceedings of the 27th International Conference on …, 2018

… Furthermore, in some of these domains, representing words from off-the-shelf word embeddings such as ones obtained from training word2vec, GloVe on Wikipedia or common-crawl may not be efficient. This is because the …


SLIND: Identifying Stable Links in Online Social Networks

J Zhang, L Tan, X Tao, X Zheng, Y Luo, JCW Lin - International Conference on …, 2018

… The dataset chosen for this study, as well as for the demo, was crawled from Facebook and obtained from the repositories of the Common Crawl (August 2016) 1 . It is de-anonymized to reveal the following relational …


Smart Focused Web Crawler for Hidden Web

S Kaur, G Geetha - Information and Communication Technology for …, 2019

… The number of partitions will depend on the number of URLs in site database. Tel-8 and common crawl datasets will be used. MapReduce function will be called, and the input will split into 64 MB plus copies of this on other clusters …


Y Nechaev, F Corcoglioniti, C Giuliano - Progress in Artificial Intelligence, 2018

Page 1. Progress in Artificial Intelligence https://doi.org/10.1007/s13748-018-0160-x REGULAR PAPER SocialLink: exploiting graph embeddings to link DBpedia entities to Twitter profiles Yaroslav Nechaev1,2 · Francesco Corcoglioniti1 · Claudio Giuliano1 …


Software Requirements Classification Using Word Embeddings and Convolutional Neural Networks

Page 1. SOFTWARE REQUIREMENTS CLASSIFICATION USING WORD EMBEDDINGS AND CONVOLUTIONAL NEURAL NETWORKS A Thesis presented to the Faculty of California Polytechnic State University, San Luis Obispo In Partial Fulfillment …


SOLVENT: A Mixed Initiative System for Finding Analogies between Research Papers

J CHAN, JC CHANG, TOM HOPE, D SHAHAF… - 2018

… We originally used pre-trained GloVe [35] vectors trained on the Common Crawl dataset 5. However, baseline performance was very poor … Finally, we used an initial prototype GloVe model (with Common Crawl) to suggest new matches we might have missed …


Speech-Based Real-Time Presentation Tracking Using Semantic Matching. Abstract. Oral presentations are an essential yet challenging aspect of academic and professional life. To date, many commercial and research products …


Stop Illegal Comments: A Multi-Task Deep Learning Approach

A Elnaggar, B Waltl, I Glaser, J Landthaler… - arXiv preprint arXiv …, 2018

… 12] and our Glove). The Fast Text is 2 million word vectors trained on Common Crawl with dimension 300, while Glove is 2.2 million word vectors trained on Common Crawl with dimension 300. Furthermore, we trained a custom …


Studio Ousia's Quiz Bowl Question Answering System at NIPS HCQA 2017

… Moreover, we use the GloVe word embeddings [10] trained on the 840 billion Common Crawl corpus to initialize the word representations. We randomly select 10% questions from the dataset as a validation set and use the remaining questions to train the model …


Studio Ousia's Quiz Bowl Question Answering System

I Yamada, R Tamaki, H Shindo, Y Takefuji

… 1,000. We use filter window sizes of 2, 3, 4, and 5, and 1,000 feature maps for each filter. We use the GloVe word embeddings [12] trained on the 840 billion Common Crawl corpus to initialize the word representations. As in …


Studying the Difference Between Natural and Programming Language Corpora

C Casalnuovo, K Sagae, P Devanbu - arXiv preprint arXiv:1806.02437, 2018

… The German and Spanish corpora were selected from a sample of files from the unlabeled datasets from the ConLL 2017 Shared Task (Ginter et al, 2017), which consist of web text obtained from CommonCrawl.8 Like the 1 billion …


Style Transfer Through Back-Translation

S Prabhumoye, Y Tsvetkov, R Salakhutdinov, AW Black - arXiv preprint arXiv …, 2018

… We used data from Workshop in Statistical Machine Translation 2015 (WMT15) (Bojar et al., 2015) to train our translation models. We used the French– English data from the Europarl v7 corpus, the news commentary …


SumeCzech: Large Czech News-Based Summarization Dataset

M Straka, N Mediankin, T Kocmi, Z Žabokrtský… - Proceedings of the Eleventh …, 2018

… The raw data for the dataset was collected from the Common Crawl project2 using the Common Crawl API. Initially, five Czech news websites were selected to create the dataset: novinky.cz, lidovky.cz, denik.cz, idnes.cz, and ihned.cz …


Supplementary Material for “Improved Fusion of Visual and Language Representations by Dense Symmetric Co-Attention for Visual Question Answering”

… All the questions were tokenized using Python Natural Language Toolkit (nltk) [2]. We used the vocabulary provided by the CommonCrawl-840B Glove model for English word vectors [11], and set out-of-vocabulary words to unk …


Survey of Simple Neural Networks in Semantic Textual Similarity Analysis

… vectors. This specific set of word vectors have 300 dimensions and were pre-trained on 840 billion tokens taken from Common Crawl3. Different pretrained word vectors may be used in-place of this specific pretrained set …


SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

R Zellers, Y Bisk, R Schwartz, Y Choi - arXiv preprint arXiv:1808.05326, 2018

… We consider three different types of word representations: 300d GloVe vectors from Common Crawl (Pennington et al., 2014), 300d Numberbatch vectors retrofitted using ConceptNet relations (Speer et al., 2017), and …


Systems and methods for improved user interface

Z Wei, T Nguyen, I Chan, KM Liou, H Wang, H Lu - US Patent App. 15/621,647, 2018

Aspects of the present disclosure relate to systems and methods for a voice-centric virtual or soft keyboard (or keypad). Unlike other keyboards, embodiments of the present disclosure prioritize the voice keyboard, meanwhile providing users with a quick and uniform navigation to …


T2S: An Encoder-Decoder Model for Topic-Based Natural Language Generation

W Ou, C Chen, J Ren - International Conference on Applications of Natural …, 2018

… We initialize our word embeddings with publicly available 300-dimensional Glove vectors [13], which is trained on 840 billion tokens of Common Crawl data 2 . Words that do not exist in the pretrained Glove vectors are replaced by “<unk>” token …


TabVec: Table Vectors for Classification of Web Tables

M Ghasemi-Gol, P Szekely - arXiv preprint arXiv:1802.06290, 2018

… They evaluated their system on the common crawl dataset, and reported signi cant improvement compared to previous feature based methods … Three of these datasets are from unusual domains, and one is a sample from Common Crawl …


TCS Research at SemEval-2018 Task 1: Learning Robust Representations using Multi-Attention Architecture

H Meisheri, L Dey - Proceedings of The 12th International Workshop on …, 2018

… corpus which results in parallel attention mechanism - one set from the twitter space and another from a common crawl corpus … 2014) trained over common crawl corpus with 300 dimension vector, Character1 level embeddings trained …


Temporal Modular Networks for Retrieving Complex Compositional Activities in Videos

… 4.2. The Stanford Parser [27] is used to obtain the initial parse trees for the compositional structure. For word vectors as part of the base module input, we use the 300-dimensional GloVe [36] vectors pretrained on Common Crawl (42 billion tokens) …


Ten Years of WebTables

M Cafarella, A Halevy, H Lee, J Madhavan, C Yu… - Proceedings of the VLDB …, 2018

… Several researchers produced web tables from the public Common Crawl [1, 24, 15], thereby making them available to a broad audience outside the large Web companies. Wang, et al. [36] improved ex- traction quality by leveraging curated knowledge bases …


Text Embeddings for Retrieval From a Large Knowledge Base

T Cakaloglu, C Szegedy, X Xu - arXiv preprint arXiv:1810.10176, 2018

… We specially utilized the ”glove-840B-300d” pre-trained word vectors where it was trained on using the common crawl within 840B tokens, 2.2M vocab, cased, 300d vectors. We created the GloVe representation of our corpus …


Text-based Sentiment Analysis and Music Emotion Recognition

… 38 3.4 Confusion matrix of lexicon-generated song labels . . . . . 42 5.1 Listofwordembeddingcorpora . . . . . 65 5.2 Google News compared with Common Crawl . . . . . 69 5.3 Propertiesofself_w2vmodels . . . . . 70 …


Text-Driven Head Motion Synthesis Using Neural Networks

… We also compared embeddings trained on Common Crawl (a large collection of websites), and on Wikipedia and the Gigaword corpus of news articles. The model trained was a baseline RNN with the architecture specified in Table 4.2 …


TEXTBUGGER: Generating Adversarial Text Against Real-world Applications

… This is because we observe that the stop-words also have impact on the prediction results. In particular, our experiments utilize the 300-dimension GloVe embeddings7 trained on 840 billion tokens of Common Crawl. Words …


The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task

A Poncelas, A Way, K Sarasola - International Workshop on Spoken Language …, 2018

… pair (see Table 4)[12]. In particular, we use the CommonCrawl, Europarl V7, NewsCommentary V12 and UN datasets for training, 5 the NewsTest 2008-2012 corpora for validation and NewsTest 2013 for testing. We did not use …


The AFRL WMT18 Systems: Ensembling, Continuation and Combination

J Gwinnup, T Anderson, G Erdmann, K Young - … of the Third Conference on Machine …, 2018

… We took the Russian and English monolingual CommonCrawl (Smith et al., 2013) data provided by the organizers and applied tokenization and BPE with our common, joint model … 2013. Dirt cheap web-scale parallel text from the common crawl …


AC Kozlowski, M Taddy, JA Evans - arXiv preprint arXiv:1803.09288, 2018

Page 1. The Geometry of Culture: Analyzing Meaning through Word Embeddings Austin C. Kozlowski1 Matt Taddy2,3 James A. Evans1 1 University of Chicago, Department of Sociology 2 University of Chicago, Booth School of Business 3 Amazon …


The Importance of Subword Embeddings in Sentence Pair Modeling

… GloVe word vectors (Pennington et al.), trained on 27 billion words from Twitter (vocabulary size of 1.2 milion words) for social media datasets, and 300-dimensional GloVe vectors, trained on 840 billion words (vocabulary …


The Knowledge and Language Gap in Medical Information Seeking. Abstract. Interest in medical information retrieval has risen significantly in the last few years. The Internet has become a primary source for consumers looking …


The MLLP-UPV German-English Machine Translation System for WMT18

J Iranzo-Sánchez, P Baquero-Arnal, GVG Dıaz-Munıo…

… 422 Page 2. Table 1: Size by corpus of the WMT18 parallel dataset Corpus Sentences (M) News Commentary v13 0.3 Rapid (press releases) 1.3 Common Crawl 1.9 Europarl v7 2.4 ParaCrawl 36.4 WMT18 total 42.3 the rest of the WMT corpora …


B McCann, NS Keskar, C Xiong, R Socher - arXiv preprint arXiv:1806.08730, 2018

Page 1. The Natural Language Decathlon: Multitask Learning as Question Answering Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, Richard Socher Salesforce Research {bmccann,nkeskar,cxiong,rsocher}@salesforce.com Abstract …


The RWTH Aachen Machine Translation Systems for IWSLT 2017

P Bahar, J Rosendahl, N Rossenbach, H Ney

… The majority of removed sentence pairs are part of the Common Crawl (300k sentences ie 14% of Common Crawl) and the OpenSubtitles corpora (1000k sentences ie 8% of OpenSubtitles) … Common Crawl Europarl UN News Comment OpenSub QED TED Wiki Total …


The RWTH Aachen University Filtering System for the WMT 2018 Parallel Corpus Filtering Task

N Rossenbach, J Rosendahl, Y Kim, M Graça… - Proceedings of the Third …, 2018

… We train IBM1 models for both directions (s2t and t2s) using the bilingual data from the WMT 2018 German↔English task namely the Europarl, CommonCrawl, NewsCommentary and Rapid corpus. 4.3 Neural Network Language Model …


The RWTH Aachen University Supervised Machine Translation Systems for WMT 2018

J Schamper, J Rosendahl, P Bahar, Y Kim, A Nix… - Proceedings of the Third …, 2018

… 1.4% BLEU. The Transformer model was trained using the standard parallel WMT 2018 data sets (namely Europarl, CommonCrawl, NewsCommentary and Rapid, in total 5.9M sentence pairs) as well as the 4.2M sen3http://www …


The Speechmatics Parallel Corpus Filtering System for WMT18

T Ash, R Francis, W Williams - Proceedings of the Third Conference on Machine …, 2018

… This data comprises the data for the WMT 2018 news translation task data for German-English without the Paracrawl parallel corpus. This data is approximately 130M words, drawn from Europarl, Common Crawl, News …


Page 1. The study of keyword search in open source search engines and digital forensics tools with respect to the needs of cyber crime investigations Joachim Hansen Master in Information Security Supervisor: Katrin Franke …


The University of Cambridge's Machine Translation Systems for WMT18

F Stahlberg, A de Gispert, B Byrne - arXiv preprint arXiv:1808.09465, 2018

… Page 3. Corpus Over-sampling #Sentences Common Crawl 2x 4.43M Europarl v7 2x 3.76M News Commentary v13 2x 0.57M Rapid 2016 2x 2.27M ParaCrawl 1x 11.16M Synthetic (news-2017) 1x 20.00M Total 42.19M Table …


The University of Edinburgh's Submissions to the WMT18 News Translation Task

B Haddow, N Bogoychev, D Emelin, U Germann… - Proceedings of the Third …, 2018

… closely Corpus % Back translations1 50% CommonCrawl 5% Europarl 15% News-commentary 10% ParaCrawl 10% Rapid 10% Table 1: Blend of data for training the DE↔EN en- semble models (40M sentence pairs total) …


The USTC-NEL Speech Translation system at IWSLT 2018

D Liu, J Liu, W Guo, S Xiong, Z Ma, R Song, C Wu… - arXiv preprint arXiv …, 2018

… Page 2. Table 2: text training data. Corpus raw filtered commoncrawl 2.39M 1.80M rapid 1.32M 1.00M europal 1.92M 1.81M commentary 0.284M 0.233M paracrawl 36.35M 12.35M opensubtitles 22.51M 14.24M WIT3(in domain) 0.209M 0.207M …


Topic coherence analysis for the classification of Alzheimer's disease}}

A Pompili, A Abad, DM de Matos, IP Martins - Proc. IberSPEECH 2018, 2018

… regularities among sentences. To this purpose, we rely on a pre-trained model of word vector representations containing 2 million word vectors, in 300 dimensions, trained with fastText on Common Crawl [27]. In the process …


Topic Modeling for Analyzing Open-Ended Survey Responses

… Word2Vec [32] and the second one on Common Crawl web data via Global Vectors for Word Representation (GloVe) [33] … The set is trained on 42 billion tokens of Common Crawl web data and contains 300-dimensional vectors …


Toward better reasoning from natural language

Page 1. Toward Better Reasoning from Natural Language by Adam Lee Purtee Submitted in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Supervised by Professor Lenhart Schubert and …


Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detection

L Konstantinovskiy, O Price, M Babakar, A Zubiaga - arXiv preprint arXiv:1809.08193, 2018

… The method provided by InferSent involves words be- ing converted to their common crawl GloVe implementations before being passed through a bidirectional long-short-term memory (BiLSTM) network (Hochreiter & Schmidhuber, 1997) …


Towards Knowledge Graph Construction from Entity Co-occurrence

… patterns. 2 https://www.mturk.com/ 3 Pages starting with List of in http://downloads. dbpedia.org/2016-10/corei18n/en/labels en.ttl.bz2 4 http://commoncrawl.org/ 5 http://webdatacommons.org/structureddata/#results-2017-1 Page 7 …


Towards Linear Time Neural Machine Translation with Capsule Networks

M Wang, J Xie, Z Tan, J Su - arXiv preprint arXiv:1811.00287, 2018

… French translation are presented in Table 1. We compare CAPSNMT with various other systems including the winning system in WMT'14 (Buck et al., 2014), a phrase-based system whose language models were trained on …


Page 1. Towards Personalized Learning using Counterfactual Inference for Randomized Controlled Trials by Siyuan Zhao A Dissertation Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE …


Towards Two-Dimensional Sequence to Sequence Model in Neural Machine Translation

P Bahar, C Brix, H Ney - arXiv preprint arXiv:1810.03975, 2018

… 5 Experiments We have done the experiments on the WMT 2017 German→English and English→German news tasks consisting of 4.6M training samples collected from the well-known data sets Europarl-v7, News-Commentary-v10 and Common-Crawl …


Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data

MA Hedderich, D Klakow - arXiv preprint arXiv:1807.00745, 2018

… Page 4. tries other than Britain until the scientific” where ”Britain” is the target word with label y = LOC. Sentence boundaries are padded. We encode the words using the 300-dimensional GloVe vectors trained on cased text …


Training Tips for the Transformer Model

M Popel, O Bojar - arXiv preprint arXiv:1804.00247, 2018

… commoncrawl 161 k 3.3 M 2.9 M … Most of our training data comes from the CzEng parallel treebank, version 1.7 (57M sentence pairs), and the rest (1M sentence pairs) comes from three smaller sources (Europarl, News …


Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter

G Wiedemann, E Ruppert, R Jindal, C Biemann - Austrian Academy of Sciences …, 2018

… classification labels per task. have not been seen during training the embedding model. We use a model pre-trained with German language data from Wikipedia and Common Crawl provided by Mikolov et al.(2018). First, we unify all …


Transferred Embeddings for Igbo Similarity, Analogy and Diacritic Restoration Tasks

IEMHI Onyenwe, C Enemuo - COLING 2018, 2018

… org news dataset. • igWkSbwd from same as igWkNews but with subword information. • igWkCrl from fastText Common Crawl dataset Table 1 shows the vocabulary lengths (vocabs), and the dimensions (vectors) of each of the models used in our experiments …


Transferred Embeddings for Igbo Similarity, Analogy, and Diacritic Restoration Tasks

I Ezeani, I Onyenwe, M Hepple - Proceedings of the Third Workshop on Semantic …, 2018

… igWkSbwd from same as igWkNews but with subword information. • igWkCrl from fastText Common Crawl dataset Table 1 shows the vocabulary lengths (vocabs), and the dimensions (vectors) of each of the models used in our experiments. 3 Model Evaluation …


Translation of Biomedical Documents with Focus on Spanish-English

MS Duma, W Menzel - Proceedings of the Third Conference on Machine …, 2018

… 2http://commoncrawl.org/ 3https://paracrawl.eu/index.html … Track / Corpora EN-ES EN-PT EN-RO Commoncrawl 1.8M - - Paracrawl - 2.1M 2.4M Wikipedia 1.6M 1.6M - EMEA 678K 1.08M 994K Scielo-gma 2016 166K 613K - Table 1: Corpora used for DSTF 3.2 Tools …


Two-Step Multi-factor Attention Neural Network for Answer Selection

P Zhang, Y Hou, Z Su, Y Su - Pacific Rim International Conference on Artificial …, 2018

… 3.3 Experimental Settings. We initialize word embeddings with 300-dimensional GloVe word vectors pre-trained from the 840B Common Crawl corpus [20]. For out of vocabulary (OOV) words, their embeddings are initialized randomly …


UBC-NLP at IEST 2018: Learning Implicit Emotion With an Ensemble of Language Models

… morphology like English. Additionally, fastText partially solves issues with out-of-vocabulary words since it exploits character sequences. FastText is trained on the Common Crawl dataset, consisting of 600B tokens. For this and the …


Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation

M Artetxe, G Labaka, I Lopez-Gazpio, E Agirre - arXiv preprint arXiv:1809.02094, 2018

… analogies. We use the largest pre-trained model published by the authors9, which was trained on 840 billion words of the Common Crawl corpus and contains 300dimensional vectors for 2.2 million words. Fasttext (Bojanowski …


Understanding Back-Translation at Scale

S Edunov, M Ott, M Auli, D Grangier - arXiv preprint arXiv:1808.09381, 2018

… It also allows us to estimate the value of BT data for domain adaptation since the newscrawl corpus (BT-news) is pure news whereas the bitext is a mixture of eu- roparl and commoncrawl with only a small newscommentary portion …


Understanding Search Queries in Natural Language

Z Neverilová, M Kvaššay - RASLAN 2018 Recent Advances in Slavonic Natural …, 2018

… stop-words are removed. Tokens are mapped to 300-dimensional word embeddings using publicly available vocabulary of FastText [2] vectors trained on CommonCrawl dataset. Missing words are ignored. Vectors are then …


Page 1. NoPConj : PConj AAnter : Ant TTAnt : Temp whoever_NP : NP NoVoc : Voc PPos : Pol is_right_VP : VP PhrUtt : Phr TPast : Tense PredVP : Cl UseCl : S UttS : Utt Unsupervised Disambiguation of Abstract Syntax A Language …


Unsupervised Domain Adaptation by Adversarial Learning for Robust Speech Recognition

P Denisov, NT Vu, MF Font - arXiv preprint arXiv:1807.11284, 2018

… Summary of the used corpora is given in Tab. 1. In addition to that, 197 millions words of Italian Deduplicated CommonCrawl Text are used to build Italian language model. Italian dictionary ILE with pronunciations for 588k words is used as a lexicon. 3.2 Baseline …


Unsupervised Mining of Analogical Frames by Constraint Satisfaction

L De Vine, S Geva, P Bruza - Australasian Language Technology Association …

… 2 a3, 3 Figure 4: Determining an analogy completion from a larger frame We conducted experiments with embeddings constructed by ourselves as well as with publicly accessible embeddings from the fastText web site2 trained …


Unsupervised Neural Machine Translation Initialized by Unsupervised Statistical Machine Translation

B Marie, A Fujita - arXiv preprint arXiv:1810.12703, 2018

… These methods usually exploit existing accurate translation models and have shown to be useful especially when targeting 1See for instance the Common Crawl project: http:// commoncrawl.org/ low-resource language pairs and domains …


Unsupervised Post-processing of Word Vectors via Conceptor Negation

T Liu, L Ungar, J Sedoc - arXiv preprint arXiv:1811.11001, 2018

… We use the publicly available pre-trained Google News Word2Vec (Mikolov et al. 2013)5 and Common Crawl GloVe6 (Pennington, Socher, and Manning 2014) to perform lexical-level experiments. For CN, we fix α = 2 for …


Unsupervised semantic frame induction using triclustering

D Ustalov, A Panchenko, A Kutuzov, C Biemann… - arXiv preprint arXiv …, 2018

… In our evaluation, we use triple frequencies from the DepCC dataset (Panchenko et al., 2018) , which is a dependency-parsed version of the Common Crawl corpus, and the standard 300-dimensional word embeddings …


Unsupervised Sense-Aware Hypernymy Extraction

D Ustalov, A Panchenko, C Biemann, SP Ponzetto - arXiv preprint arXiv:1809.06223, 2018

… Recent approaches to hypernym extraction went into three directions: (1) unsupervised methods based on such huge corpora as CommonCrawl1 to ensure extraction coverage using Hearst (1992) patterns (Seitner et al …


User-Centric Ontology Population

K Clarkson, AL Gentile, D Gruhl, P Ristoski, J Terdiman…

… Ristoski et al. [29] use standard word embeddings and graph embeddings to align instances extracted from the Common Crawl4 to the DBpedia ontology. The use of deep learning models has also been explored for this task. Dong et al …


Using a Stacked Residual LSTM Model for Sentiment Intensity Prediction

J Wang, B Peng, X Zhang - Neurocomputing, 2018

… To enhance performance of LSTM layers, we also introduce a bi-directional strategy [34]. The word embeddings used in this experiment was respectively pre-trained on Common Crawl 840B 2 (English) and wiki dumps 3 (Chinese) by GloVe [55] …


Using context to identify the language of face-saving

… For all our Neural Network models, we initialized our word representations using the publicly available GloVe pre-trained word embeddings (Pennington et al., 2014)8 (300-dimensional vectors trained on Common Crawl data) …


Using Deep Learning For Title-Based Semantic Subject Indexing To Reach Competitive Performance to Full-Text

F Mai, L Galke, A Scherp - arXiv preprint arXiv:1801.06717, 2018

… We adopt the preprocessing and tokenization scheme of Galke et. al [5]. For the LSTM and CNN, we use 300-dimensional pretrained word embeddings obtained from training GloVe [28] on Common Crawl with 840 billion tokens7. Out-of-vocabulary words are discarded …


Using Machine Learning to Detect Malicious Websites

… Benign Data Benign data was obtained from the Common Crawl6. The Common Crawl is a massive, continuously updated collection of crawled websites available for download … PhishTank 36,485 65,000 VirusTotal 5,036 0 6 http://commoncrawl.org/ Page 21. 12 …


Using Monolingual Data in Neural Machine Translation: a Systematic Study

F Burlot, F Yvon - Proceedings of the Third Conference on Machine …, 2018

… For German, we use samples from News-Commentary-11, Rapid, Common-Crawl (WMT 2017) and Multi- UN (see table 1). Bilingual BPE units (Sennrich et al., 2016b) are learned with 50k merge operations, yielding …


Using Wikipedia Edits in Low Resource Grammatical Error Correction

… 2.4 Language Model For reranking, we train a language model on the first one billion lines (~12 billion tokens) of the deduplicated German Common Crawl corpus (Buck et al., 2014). 3 Method … 2014. N-gram counts and language models from the Common Crawl …


Using Word Embeddings for Information Retrieval: How Collection and Term Normalization Choices Affect Performance

D Roy, D Ganguly, S Bhatia, S Bedathur, M Mitra - 2018

… In future, we plan to solidify these observations to offer general best practices for a range of different neural IR methods (eg DRRM[7]) as well as experiment using large datasets (eg Common Crawl). REFERENCES [1] Qingyao Ai …


M Trotzek, S Koitka, CM Friedrich - arXiv preprint arXiv:1804.07000, 2018

Page 1. SUBMITTED FOR PUBLICATION TO THE IEEE, 2018 1 Utilizing Neural Networks and Linguistic Metadata for Early Detection of Depression Indications in Text Sequences Marcel Trotzek, Sven Koitka, and Christoph M. Friedrich, Member, IEEE …


UWB at SemEval-2018 Task 10: Capturing Discriminative Attributes from Word Distributions

T Brychcín, T Hercig, J Steinberger, M Konkol - … of The 12th International Workshop on …, 2018

… SS-GloVe 6B, Wikipedia + Gigaword 5 n = 300 62.0% 62.5% SS-GloVe 42B, Common Crawl n = 300 62.6% 62.7% SS-GloVe 840B, Common Crawl n = 300 62.1% 62.6 … SS-LDA 1-5B, Wikipedia n = 200 60.5% 63.1 …


Vecsigrafo: Corpus-based Word-Concept Embeddings

… To compare our embeddings to those trained on a very large corpus, we use pre-calculated GloVe embeddings that were trained on CommonCrawl7. Besides the text corpora, the tested embeddings con …


Visual and affective grounding in language and mind

S De Deyne, DJ Navarro, G Collell, A Perfors

… We also included an extremely large corpus consisting of 840 billion words from the Common Crawl project.5 As before, the language vectors were combined in a multimodal visual or affective model and the correlations were optimized by fitting values of β …


Visual Concept Selection with Textual Knowledge for Understanding Activities of Daily Living and Life Moment Retrieval

TH Tang12, MH Fu, HH Huang, KT Chen, HH Chen13

… Page 8. GloVe [9] trained on Common Crawl with 840B tokens and ConceptNet Numberbatch [8]. The comparison in percentage dissimilarity [1] is shown in Table 1, where (G) and (N) denote GloVe and ConceptNet Numberbatch word vectors, respectively …


Visual Question Answering using Explicit Visual Attention

V Lioutas, N Passalis, A Tefas - Circuits and Systems (ISCAS), 2018 IEEE …, 2018

… For extracting textual representations we used pre-trained GloVe embedding vectors (Common Crawl (42B tokens), 300d) [1]. Note that the GloVe embeddings were used only for initialization and then they were optimized during the training …


Visual Relationship Detection Based on Guided Proposals and Semantic Knowledge Distillation

F Plesse, A Ginsca, B Delezoide, F Prêteux - arXiv preprint arXiv:1805.10802, 2018

… iterations. The word embeddings used by the semantic knowledge introduced in Section 2.1 were obtained from the publicly available Glove model [19] trained on the Common Crawl corpus, consisting of 42B tokens. 4.2. Results …


Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs

S Kumar, Y Tsvetkov - arXiv preprint arXiv:1812.04616, 2018

… target word embeddings for English and French on corpora constructed using WMT'16 (Bojar et al., 2016) monolingual datasets containing data from Europarl, News Commentary, News Crawl from 2007 to 2015 and News …


WBI at CLEF eHealth 2018 Task 1: Language-independent ICD-10 coding using multi-lingual embeddings and recurrent neural networks

… Each token is represented using pre-trained fastText5 word embeddings [4]. We utilize fastText embedding models for French, Italian and Hungarian trained on Common Crawl and Wikipedia articles6. Independently from …


Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading

M Raison, PE Mazaré, R Das, A Bordes - arXiv preprint arXiv:1804.10490, 2018

… Unless otherwise noted, we use 300dimensional FastText word embeddings trained on Common Crawl (Mikolov et al., 2017) and keep them fixed during training. Out-of-vocabulary words are represented with a fixed randomly initialized vector …


Web archives and Knowledge organisation

… Internet Archive, established in 1996, and Common Crawl (commoncrawl.org) established in 2007.12 Since 2006 the Internet Archive also provide a subscriptionbased archive service, Archive-it (archive-it.org) allowing anybody …


What can we learn from Semantic Tagging?

M Abdou, A Kulmizev, V Ravishankar, L Abzianidze… - arXiv preprint arXiv …, 2018

… sets of experiments: we optimized using Adam with a learning rate of 0.00005; we weight the auxiliary semantic tagging loss with λ = 0.1; the pre-trained word embeddings we use are GloVe embeddings of dimension 300 trained …


What's Cached is Prologue: Reviewing Recent Web Archives Research Towards Supporting Scholarly Use

… Internet Archive. Samar et al. (2016) analyze coverage of trending topics for the Netherlands in 2014 by comparing the National Library of the Netherlands' web archive to the Common Crawl dataset. Milligan et al. (2016) use …


When data permutations are pathological: the case of neural natural language inference

N Schluter, D Varab - Proceedings of the 2018 Conference on Empirical …, 2018

… via an LSTM. Other hyperparameters. We use 300 dimensional GloVe embeddings trained on the Common Crawl 840B tokens dataset (Pennington et al., 2014), which remain fixed during training. Out of vocabulary (OOV …


Who gets held accountable when a facial recognition algorithm fails?

… In that experiment, the machine learning tool was trained on what's called a “common crawl” corpus: a list of 840 billion words in material published on the Web. Training AI on historical data can freeze our society in its current setting, or even turn it back …


WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse Supplementary Material

… 3.3 Experimental Setting We use FastText (Mikolov et al., 2018; Grave et al., 2018)3 word vectors of length 300, originally trained on more than 600 billion word to- kens each from Common Crawl corpus for each language …


Wikipedia Text Reuse: Within and Without

M Alshomary, M Völske, T Licht, H Wachsmuth, B Stein… - arXiv preprint arXiv …, 2018

… Abstract. We study text reuse related to Wikipedia at scale by compiling the first corpus of text reuse cases within Wikipedia as well as without (ie, reuse of Wikipedia text in a sample of the Common Crawl) … 3 …


WikiRef: Wikilinks as a route to recommending appropriate references for scientific Wikipedia pages

A Jana, P Kanojiya, A Mukherjee, P Goyal - arXiv preprint arXiv:1806.04092, 2018

… proposed by Conneau et al. (2017). Note that, for applying this architecture we use the GloVe vectors trained on Common Crawl data (840B tokens)7as seeds for representing words in a document. We name these variants of …


Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining

E Shnarch, C Alzate, L Dankin, M Gleize, Y Hou… - Proceedings of the 56th …, 2018

… maximum global norm of 1.0. Words are represented using the 300 dimensional GloVe embeddings learned on 840B Common Crawl tokens and are left untouched during training (Pennington et al., 2014). We note that even …


Word embedding for French natural language in healthcare: a comparative study

… [30] compared the three word embedding methods but the three models were trained on different datasets (Word2Vec on news data, while FastText and GloVe trained on more definitional data, Wikipedia and Common Crawl respectively) …


Page 1. Word embeddings for monolingual and cross-lingual domain-specific information retrieval CHAYA WIGDER Master in Computer Science Date: June 4, 2018 Supervisor: Johan Boye Examiner: Viggo Kann Swedish title …


Word Emotion Induction for Multiple Languages as a Deep Multi-Task Learning Problem

… experiments, we rely on the following widely used, publicly available embedding models trained on very large corpora (summarized in Table 3): the SGNS model trained on the Google News corpus2 (GOOGLE), the …


Word2Bits-Quantized Word Vectors

M Lam - arXiv preprint arXiv:1803.05651, 2018

… complete picture of the relative performance of the two. We would also like to train quantized word vectors on much larger corpuses of data such as Common Crawl or Google News. Another task is to validate that overfitting occurs …


XNLI: Evaluating Cross-lingual Sentence Representations

A Conneau, G Lample, R Rinott, A Williams… - arXiv preprint arXiv …, 2018

… on the word translation task. In this paper, we pretrain our embeddings using the common-crawl word embeddings (Grave et al., 2018) aligned with the MUSE library of Conneau et al. (2018b). 4.2.2 Universal Multilingual Sentence …


T Li, L Lin, M Choi, K Fu, S Gong, J Wang - arXiv preprint arXiv:1807.11227, 2018

Page 1. YouTube AV 50K: an Annotated Corpus for Comments in Autonomous Vehicles Tao Li Department of Computer Science Purdue University West Lafayette, IN 47907 Email: taoli@purdue.edu Kaiming Fu Weldon School …


Zero-Shot Object Detection by Hybrid Region Embedding

B Demirel, RG Cinbis, N Ikizler-Cinbis - arXiv preprint arXiv:1805.06157, 2018

… 4.2 Class Embeddings For the Fashion-ZSD dataset, we generate 300-dimensional GloVe word embedding vectors [31] for each class name, using Common Crawl Data1. For the class names that contain multiple words, we take the average of the word vectors …


Zewen at SemEval-2018 Task 1: An Ensemble Model for Affect Prediction in Tweets

Z Chi, H Huang, J Chen, H Wu, R Wei - … of The 12th International Workshop on …, 2018

… GloVe (Pennington et al., 2014) trained by Common Crawl … the same model hyperparameters which are listed in Table 1 and Table 2. Also, the four methods use the same word em- beddings, which is a pre-trained …