# Fully Attentional Network for Low-Resource Academic Machine Translation and Post Editing
## 1 Introduction  
Machine Translation (MT) is based on sequence-to-sequence Machine Learning (ML) that translates from one language to another without human intervention [1]. MT is crucial not only for academics but also for large technology companies (Google Translate, Bing Translator, Yandex Translate, DeepL, etc.). The performance of machine translation heavily relies on the quality of the parallel corpus. The quality and quantity of the corpus directly affect the quality of translation. Translation models (English-French, English-German) with a large number of resources have achieved the quality of human translation [2]. However, the translation quality has not reached that level for the Turkish-English language pair yet due to the lack of adequate studies and the lack of high-quality parallel data. Google Translate (GT) and DeepL translation systems are currently the most widely used translation systems from Turkish to English. Both translation systems use transformer architecture and create very successful translations. However, academic terminology translations are different from everyday spoken language translations. In this respect, the translation systems sometimes mistranslate academic terms (see Table 1).  
In this study, firstly, a parallel corpus was created with the use of the abstracts of graduate theses in Turkey. Researchers in Turkey are supposed to prepare the abstracts in Turkish and English and researchers' dissertations are uploaded to the Council of Higher Education's website \"CoHE's Thesis System\" [[https://tez.yok.gov.tr/](https://tez.yok.gov.tr/)]([https://tez.yok.gov.tr/](https://tez.yok.gov.tr/)) (Accessed on 3 November 2022). The method we use while creating the corpus consists of three stages. First, the abstracts of the theses in \"CoHE's Thesis System\" were recorded in text files. Secondly, text files were separated into sentences with text pre-processing steps. Finally, sentences were aligned via the developed sentence alignment algorithms to find the English sentence corresponding to each Turkish sentence. 1,217,300 parallel sentences were obtained after sentence alignment algorithms. Secondly, a neural machine translation system was created using the obtained parallel corpus. There are two main transformer architectures in this proposed model. The first architecture is the encoder-decoder network, and the second architecture is a pre-trained language model that has achieved high success in Natural Language Generation tasks. The shallow fusion method was employed to benefit most from these two architectures. In the shallow fusion method, the last decoder network layer output is introduced as the input of the language model. Then, the language model output and the decoder network output are combined via an additional neural network. In the known shallow fusion method Feed-Forward Network (FFN) is used for this network. On the other hand, a Fully Attentional Network (FAN) is used in the proposed model for this network. The proposed architecture is explained in detail in the method section. The corpus of the proposed model was also utilized to develop new models with the current neural machine translation architectures (LSTM, Convolutional, Base Transformer). The success of the proposed model was compared to the performance of the models with current neural machine translation architectures. In addition to the translation systems, the test set was translated sentence by sentence via GT. Metrics BLEU, METEOR, TER, and Perplexity were used to evaluate the models' success. FAN architecture used model scored 45.1 BLEU and 73.2 METEOR score and achieved higher success than FFN architecture.  
In addition, the proposed model achieved 20.12 and 20.56 BLEU scores respectively on the World Machine Translation (WMT) (2017-2018) test data sets via the zero-shot translation model.