# English-Welsh Cross-Lingual Embeddings
## 3 Materials
### Text Corpus Creation  
We developed Welsh and English corpora to train our bilingual embeddings, drawing on a range of pre-existing data sets. The full Welsh-language data set extended to 144,976,542 words after tokenization. The names and corresponding number of words in each individual text corpus are displayed in Table 1. We now provide a brief description of each individual text corpus. **Welsh Wikipedia--**Wikipedia is a multilingual crowd-sourced online encyclopedia and one of the world's most popular websites. English Wikipedia was the first edition of Wikipedia and was founded in January 2001. As of 29 September 2019 (when these data were collected), there were 5,938,555 articles contained in this project. Given the large number of articles, English Wikipedia is a text corpus commonly used to train English language word embeddings. Welsh Wikipedia is the Welsh language edition of Wikipedia and was founded in July 2003. It is significantly smaller than English Wikipedia and as of 29 September 2019 it contains 106,128 articles. Web crawling of this was undertaken, specifically, using the Python library urllib and the Python library Beautiful Soup to extract all text within paragraph tags <p>. We subsequently removed all citations and mathematical equations. **National Assembly for Wales 1999-2006--**The National Assembly for Wales is the devolved parliament of Wales, which has many powers, including those to make legislation and set taxes. By performing a web crawling of the Assembly website ([[http://xixona.dlsi.ua.es/corpora/UAGT-PNAW/](http://xixona.dlsi.ua.es/corpora/UAGT-PNAW/)]([http://xixona.dlsi.ua.es/corpora/UAGT-PNAW/](http://xixona.dlsi.ua.es/corpora/UAGT-PNAW/)), accessed on 15 July 2021), [PERSON] et al. [16] created a bilingual aligned corpus of Welsh and English from the online version of the Proceedings of the Plenary Meetings of the Assembly between the years 1999 and 2006 inclusive. This is freely available as a plain text file. Only the Welsh part of this corpus was used for the purposed of the current project. **National Assembly for Wales 2007-2011--**[PERSON] [42] created the Kynulliad3 corpus, which is similar to the previous bilingual aligned corpus except that it covers the period between the years 2007 and 2011 inclusive. This corpus, which contains 350,000 aligned Welsh and English sentences, was extracted by querying an SQL database. Only the Welsh half of this corpus was used in the current project. **The Bible--**Beibl.net ([[http://www.beibl.net](http://www.beibl.net)]([http://www.beibl.net](http://www.beibl.net)), accessed on 15 July 2021) includes all books of the Bible in modern Welsh. Texts were scraped using urllib and Beautiful Soup in Python. **OPUS--**OPUS is a collection of technical texts on the web, mainly including software documentation, in a number of languages. We extracted a range of en-cy (English-Welsh) texts from this resource in plain text format. **Welsh Government translations memories--**The collection of translation memory files contains published bilingual documents and other materials from the Welsh Government (from August 2019 to May 2020). The data set comprises.tmx files, which were extracted using Python's translate toolkit package. **National Assembly for Wales 2016-2020--**Records of the proceedings of the Welsh Assembly, including plenary information from the start of the Fifth Assembly (May 2016) and