# Empirical Analysis of Parallel Corpora and In-Depth Analysis Using LIWC
## 1 Introduction  
In the past few years, the demand for machine translation (MT) systems has been continuously increasing and its importance is growing, especially for the industrial services [1; 2] Companies, such as Google, Facebook, Microsoft, Amazon, and Unbabel continue to conduct research and formulate plans to commercialize applications related to MT. From the late 1950s, numerous MT-related projects were proceeded by mainly focusing on rule-based and statistical-based approaches before the advent of deep learning technology. As deep learning based neural machine translation (NMT) was proposed and adopted to several research works, it has been gradually figured out that more superior performance can be derived through NMT approach [3; 4; 5; 6]. Followed by the adoption of deep learning based technique, the improvements of computing power (e.g., _GPU_) and corresponding enhancement of parallel processing accelerated the advancement of NMT. Recently, release of open source frameworks, such as Pytorch [7], and lowered accessibility to the big data further facilitated vigorous and diverse research. However, several issues considering the enhancement of the NMT system remain still. Representatively, limitations in ensuring the quality of data is an unresolved issue. As have previously been studied, the quality of the training data is deeply related to the NMT performance [8; 9]. The major problem is that the process of building a high-qualityparallel corpus is time-consuming and expensive, and it is significantly difficult for low-resource languages, such as Korean. Although data-augmentation techniques, such as back translation [10] and copied translation [11] have been introduced, as the human supervision is generally minimized or excluded in the data generation process, the quality of such pseudo-generated parallel corpus cannot be guaranteed [12; 13]. This restricted the usage of pseudo-generated parallel to complements of human-labeled gold parallel corpus, rather than its substitutes [14]. For the alleviation of above limitations, numerous studies on the collection of high-quality training data have been conducted, such as parallel corpus filtering (PCF) research and Data Dam project. PCF refers to a research field that aims to filter out low-quality noisy data (i.e., sentence pairs) residing in the parallel corpus, and improve the overall quality of the corpus. PCF is currently being applied to various NMT studies and contributed to the advancement of the NMT systems [15; 16]. While the amount of training data caused significant impact on the statistical-based MT approaches, the quality of data is treated as more important than the amount of data in general deep learning-based MT approaches [17; 18]. Moreover, Data Dam ([[http://www.data-alliance.kr/default/](http://www.data-alliance.kr/default/)]([http://www.data-alliance.kr/default/](http://www.data-alliance.kr/default/)), accessed on 25 May 2022) projects for building high-quality parallel corpora nationally are in progress. In the Republic of Korea, a large number of parallel corpora is open to the public through AI Hub ([[http://aihub.or.kr/](http://aihub.or.kr/)]([http://aihub.or.kr/](http://aihub.or.kr/)), accessed on 25 May 2022), which is organized by the National Information Society Agency (NIA) [19]. Following these research trends, where the quality is treated more importantly than the quantity in the data construction process, we analyzed the above Korean-English parallel corpus distributed by AI Hub. Despite its sufficient amount of data, the quality of corresponding corpus has not been confirmed clearly. This may restrict the unconstrained utilization of such corpus in adoption to the NMT model, as low quality data may degrade the overall performance. In this study, we conducted several quality verification experiments including Linguistic Inquiry and Word Count (LIWC) [20; 21], and clarified the quality and characteristics of such corpus.