## 3 The Architecture of the NMT System
### B.4. B.4. B.5.
## 4 Corpus

Parallel corpora of 259,623 sentences were developed to train the NMT model. The MWEs were extracted from this parallel corpus by [PERSON] [9; 30]. [PERSON] also collected the named entities dataset for the Punjabi-English language pair during his research work. The name entities dataset was also used in the development of the NMT model. Table 5 shows the number of Punjabi-English parallel sentences used to train the model. ### Division of Dataset into Different Sets

To train and test the different NMT models, the dataset was split into different sets. The training set contained 85% of the corpus, the validation set had 5% of the corpus, and the testing set contained 10% of the parallel corpus. The division of the parallel corpus into various sets is displayed in Table 6.

### Pre-Processing of Corpus

The first stage in the production of the NMT method is pre-processing. The pre-processing method involves a variety of phases. * Tokenization of the corpus

Tokenization splits a sentence on a word-by-word basis. To segment English sentences into word levels, the [PERSON] tokenization script was used [31]. A contemporary tokenizer for Punjabi was developed using Python to break down sentences into phrases. * Cleaning of the long sentences from the corpus

The training is often influenced by the length of the sentence. Parallel sentences with lengths of more than 50 characters were cleaned using the [PERSON] cleaning script [32]. * Lowercasing the English corpus

Both lower-case and upper-case letters are used in English sentences. The true casing would aid in the proposed system's increased precision. This project used the [PERSON] casing script. * Replacement of contractions in English corpus \begin{table}
\begin{tabular}{c c} \hline \hline
**Waala Morpheme Construct** & **Translation in English** \\ \hline Training set & 85\% \\ Validation set & 5\% \\ Test set & 10\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Different sets of the parallel corpus. \begin{table}
\begin{tabular}{c c c} \hline \hline



A short term used in the writing of words or syllables is contraction. Cannot, we are, and other widely used contractions are only a few examples. Contractions were tested across the entire corpus. To substitute the contractions with the proper words, a Python script was written. * Byte pair encoding to the dataset BPE was used in the training, validation, and testing files to reduce the size of the vocabulary. Table 7 compares the scale of Punjabi and English vocabulary before and after using the 35 K merge operation to apply BPE. ## 5 Experiments

There are many toolkits available to train the NMT models including Nematus [33], OpenNMT [34], Neural Monkey [35], CytonMT [36], etc. Out of all these toolkits, the OpenNMT toolkit was chosen to create the NMT models. Training the NMT model also requires a high hardware configuration. Therefore, a graphical processing unit (GPU) with 4 GB memory was used to train the different NMT models. ### NMT Model Details

The OpenNMT toolkit was used to build the different NMT templates. In all models, some training parameters are fixed. All models were conditioned in batches of 64 for a total of 30 training cycles. In all versions, for the encoder, BiLSTM was used, and LSTM was used as a decoder. The beam size was set to seven during the decoding process of NMT [33]. The optimization function was the stochastic gradient descent (SGD). The attention was also used to improve the accuracy of the MT system [37; 38]. The baseline NMT model 1 was trained by using the parallel corpus of 259,623 parallel sentences. Table 8 shows the configurations of different NMT models. NMT model 2 was trained by using the baseline model and WE to handle the OOV words of the Punjabi language. NMT model 3 was trained by including the MWE corpus in the baseline model. NMT model 4 contained the MWE corpus and name entities dataset corpus to the baseline model, and WE to tackle the OOV and MWEs of the source text. The overall architecture of NMT model 4 is shown in Figure 3.