# Developing a Hybrid Morphological Analyzer for Low-Resource Languages
## 3 Methodology
### Dataset Creation
#### 3.1.3 Paradigm Generation

Table 4: Statistics of data distribution across train, validation, and test sets for morphological analysis generation. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyper Parameters** & **Values** \\ \hline Word vector size & 512 \\ Encoding layers & 6 \\ Decoding layers & 6 \\ Attention Heads & 8 \\ Learning rate & 2 \\ Batch size & 2048 \\ Train steps & 10,000 \\ Encoder type & Transformer \\ Label smoothing & 0.1 \\ Optimizer & Adam \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Hyperparameters and their values used for training the Transformer. ```
Input: Training dataset \(\mathcal{D}_{\text{train}}\), Validation dataset \(\mathcal{D}_{\text{val}}\) Output: Trained Transformer model \(\mathcal{M}\)
1 Initialize:
2 Set hyperparameters: \(N,d_{\text{model}},h,d_{\text{tf}},p,V,L,E,B,\eta\)
3 Initialize model \(\mathcal{M}\leftarrow\text{Transformer}(N,d_{\text{model}},h,d_{\text{tf}},V,L,p)\)
4 Initialize optimizer: Adam\((\mathcal{M},\eta)\)
5 Define loss function: CrossEntropyLoss (ignore padding)
6 forepoch\(\gets 1\)to\(E\)do
7 Set model to training mode
8 foreachbatch\((x,y)\) in \(\mathcal{D}_{\text{train}}\)do
9\(y_{\text{in}}\gets y[:,:-1]\)//Decoder input
10\(y_{\text{out}}\gets y[:,1:]\)//Expected output
11\(\hat{y}\leftarrow\mathcal{M}(x,y_{\text{in}})\)//Forward pass
12\(\mathcal{L}\leftarrow\text{CrossEntropyLoss}(\hat{y},y_{\text{out}})\)
13 Backpropagate: \(\mathcal{L}.\text{backward}()\)
14 optimizer.step()
15 optimizer.zero_grad()
16
17 end for
18 Set model to evaluation mode
19\(L_{\text{val}}\gets 0\)
20 foreachbatch\((x,y)\) in \(\mathcal{D}_{\text{val}}\)do
21\(y_{\text{in}}\gets y[:,:-1]\)
22\(y_{\text{out}}\gets y[:,1:]\)
23\(\hat{y}\leftarrow\mathcal{M}(x,y_{\text{in}})\)
24\(L_{\text{val}}\gets L_{\text{val}}+\text{CrossEntropyLoss}(\hat{y},y_{ \text{out}})\)
25 end for
26 Print epoch, training loss, validation loss
27
28 end for
```

**Algorithm 2**Training a Transformer Model

The validation and testing data are distinct from the training data, which is essential for achieving accurate results. Considering morphological analysis as a sequence of generated words, we train a Transformer architecture to obtain the morphological analysis. The generated output is written onto a file for further evaluation using evaluation metrics like Precision, Recall, and F1 score. \[Precision\quad P=\frac{\#correct}{\#output\_sentence\_length} \tag{1}\]

\[Recall\quad R=\frac{\#correct}{\#reference\_sentence\_length} \tag{2}\]

\[F1\quad Score=\frac{2 PR}{(P+R)} \tag{3}\]

Here, \(\#correct\) is the number of features correctly predicted in the generated analysis with the length \(\#output\_sentence\_length\). \(\#reference\_sentence\_length\) is the expected output, which is used for calculating the scores. ## 4 Implementation Details

The details of the tools and languages used for implementing the morphological analyzer are described in this section. ### Dataset

The initial prototype was created for nouns and pronouns using the Kannada Tree bank dataset developed by IIIT Hyderabad [36]. Using the 2617 stems, a dictionary of nouns was built. After analyzing the results, the paradigms were extended to a larger dictionary. Table 6 depicts the paradigm's label and count of words that follow the specified paradigm. The dataset used here is the dictionary _Sankshipta Adhunika Kannada Nightatu_, Saujanya Prasaranga Kannada Vishwa Vidhyalaya Hampi [37], with 9879 words obtained after pre-processing. Table 7 furnishes the overall classification of the number of paradigms based on POSs with their corresponding inflection count.