# English-Welsh Cross-Lingual Embeddings
## 6 Conclusions  
We have described the process of training bilingual English-Welsh embeddings. We start by discussing the corpora we used to train monolingual embeddings in both languages using both _word2 vec_ and _fastText_, and continue by explaining the curation of the supervision signal (the training bilingual dictionary), as well as the linear transformation method we use for mapping both monolingual spaces into a shared bilingual space. We have evaluated this shared space both intrinsically and extrinsically. The intrinsic evaluation was based on _dictionary induction_, which was used to measure the alignment of two monolingual spaces directly by translating between the two languages and measuring the distance within a monolingual space. The best alignment was achieved by training the monolingual spaces using fastText and aligning them using the the CSLS metric. The true value of aligning two vector spaces lies in the ability to facilitate NLP applications in minoritized languages by taking advantage of readily available resources in a language such as English. To evaluate the cross-lingual embeddings extrinsically, we measured the effects of supplementing Welsh-language data with data in English on the accuracy of sentiment analysis in Welsh. We were able to use an existing neural network architecture based on CNNs and LSTMs originally developed for sentiment analysis in English. By training this neural network on cross-lingual embeddings and data from both languages, we managed to obtain highly competitive results in Welsh without having to modify the original method in any way. In particular, we demonstrated that a relatively small data set of 2 K documents in the target language seems to suffice. This opens exciting avenues for future work, where cross-lingual embeddings can be combined with neural architectures and data augmentation techniques to develop Welsh language technology at a negligible cost. The Welsh language can be categorized, within the language resource landscape, as being a low-resource language, i.e., the availability of (raw and annotated) corpora, glossaries, thesauri, encyclopedias, etc. is limited when compared to other languages such as English, Chinese, Spanish, or Indo-Aryan languages. This study allows one to automatically compare the meaning of words not only within the Welsh language but also across the two languages, thus facilitating applications such as the creation of bilingual language resources, as well as the development of NLP systems for Welsh with limited Welsh training data, as we successfully demonstrated with sentiment analysis. Cross-lingual embedding we generated therefore unlocks access to a plethora of open-source NLP solutions developed originally for English. This in turn opens a possibility of supporting a wide range of applications, such as computer-assisted translation, cross-lingual information retrieval, and conversational artificial intelligence. These applications encourage the use of Welsh in activities of daily life, which contributes to maintaining and improving Welsh language skills. Figure 1: Accuracy results for the cross-lingual sentiment analysis experiment. **Author Contributions:** Conceptualization, D.K. ; methodology, L.E.-A., G.P., P.C., M.F., and I.S. ; software, L.E.-A., G.P., M.F., and I.S. ; validation, I.S. and D.K. ; formal analysis, L.E.-A., G.P., and I.S. ; investigation, L.E.-A., G.P., and I.S. ; resources, L.E.-A., G.P., P.C., I.S., and D.K. ; data curation, L.E.-A., G.P., P.C., I.S., and D.K. ; writing--L.E.-A. ; writing--review and editing, L.E.-A., G.P., P.C., I.S., and D.K. ; visualization, M.F. ; supervision, L.E.-A., I.S., and D.K. ; project administration, P.C., I.S., and D.K. ; funding acquisition, I.S. and D.K. All authors have read and agreed to the published version of the manuscript. **Funding:** This research was funded by the Welsh Government, under the Grant \"Learning English-Welsh bilingual embeddings and applications in text categorisation\". **Institutional Review Board Statement:** Not applicable.