# Evaluating Finnish Dialect Normalization in GPT
## Models with and without Reasoning

Eiaki Morooka, Yuto Omae, Hirotaka Takahashi, Mika Hämäläinen

► To cite this version:

Eiaki Morooka, Yuto Omae, Hirotaka Takahashi, Mika Hämäläinen. Evaluating Finnish Dialect Normalization in GPT Models with and without Reasoning. Lightning Proceedings of the 10th International Workshop on Computational Linguistics for Uralic Languages, Dec 2025, Joensuu, Finland. pp.35-39. hal-05406660

HAL Id: hal-05406660

https://hal.science/hal-05406660v1

Submitted on 9 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Evaluating Finnish Dialect Normalization in GPT Models with and without Reasoning

Eiaki V. Morooka<fnref n="1" />, Yuto Omae<fnref n="2" />, Hirotaka

Takahashi³ and Mika Hämäläinen<fnref n="1" />

## Introduction

Normalization improves the compatibility of non-standard language—such as dialectal, colloquial, or transcribed spoken Finnish—with tools designed for the standard variety. Building on earlier work showing that normalization by Partanen et al. [1], our study evaluates whether modern GPT-based models can further advance dialect-to-standard Finnish normalization. We first fine-tune a non-reasoning model to establish a baseline, and then fine-tune a reasoning-enabled model to test whether its inference capabilities lead to higher normalization accuracy.

<figure><img src="image_2.png" /></figure>

<footnote n="1">¹Metropolia University of Applied Sciences, Finland</footnote>

<footnote n="2">²Nihon University, Japan ³Tokyo City University, Japan</footnote>
# Experiment 1 (Non-Reasoning)

<table><caption>Table 1: Model Performance Comparison 1</caption><thead><tr><th>Model</th><th>WER</th><th>Word Match</th><th>Std Dev (Word Match)</th><th>Exact Match</th></tr></thead><tbody><tr><td>DeepSeek-R1:1.5B (Distill Qwen)</td><td>18.55 %</td><td>81.98 %</td><td>13.37 %</td><td>18.78 %</td></tr><tr><td>DeepSeek-R1:7B (Distill Qwen)</td><td>11.17 %</td><td>89.08 %</td><td>10.86 %</td><td>31.79 %</td></tr><tr><td>Gemma3:1B</td><td>12.79 %</td><td>87.63 %</td><td>12.15 %</td><td>29.5 %</td></tr><tr><td>Gemma3:270M</td><td>18.29 %</td><td>82.59 %</td><td>15.26 %</td><td>21.27 %</td></tr><tr><td>Poro-2-8B (Llama)</td><td>5.55 %</td><td>94.68 %</td><td>8.31 %</td><td>56.55 %</td></tr><tr><td>Viking-7B</td><td>7.76 %</td><td>92.32 %</td><td>9.96 %</td><td>47.77 %</td></tr><tr><td>Viking-13B</td><td>6.81 %</td><td>93.31 %</td><td>9.51 %</td><td>51.07 %</td></tr></tbody></table>

In our first experiment, we fine-tuned several open models—DeepSeek-R1 1.5B and 7B (Distill Qwen) [2], Gemma 270M and 1B [3], Poro-2 8B (Llama) [4], and Viking 7B and 13B [5]—on 34,822-sample training set from the Institute for the Languages of Finland [6] to produce normalized Finnish text. All models were trained under identical settings using LoRA ($r = 16, \alpha = 16$) targeting the $q\_proj$, $k\_proj$, $v\_proj$, and $o\_proj$ modules, with no hyperparameter tuning, a learning rate of 5e-5, and three epochs. Evaluation on the 7,687-sample test set (Table 1) shows that model size had some effect, but the extent of Finnish pretraining was the primary factor. The Finnish-centric Poro-2-8B outperformed all others, slightly surpassing the normalization quality reported in Dialect Text Normalization to Normative [1] (WER 5.73%). These results demonstrate that GPT-style models can achieve higher accuracy, though at the cost of additional computational resources.

# Experiment 2 (Reasoning)

<table><caption>Table 2: Deepseek-R1:7B Performance Comparison 2 (100 test samples)</caption><thead><tr><th>Model</th><th>WER</th><th>Word Match</th><th>Std Dev (Word Match)</th><th>Exact Match</th></tr></thead><tbody><tr><td>Reasoning (Experiment 2)</td><td>35.85 %</td><td>66.09 %</td><td>16.5 %</td><td>7.0 %</td></tr><tr><td>Non-Reasoning (Experiment 1)</td><td>12.4 %</td><td>87.64 %</td><td>10.63 %</td><td>29.00 %</td></tr></tbody></table>
In the second experiment, we attempted to create a reasoning-capable normalization model by fine-tuning DeepSeek 7B with chain-of-thought style supervision. Using DeepSeek's official full reasoning model, we generated 3k training examples by providing each dialectal input and its normalized target and prompting the model to explain WHY the normalization was appropriate while “acting” as if it did not know the answer. Each explanation was wrapped in `<think>...</think>` tags, followed by the correct normalized output, and this sequence formed the fine-tuning sample. The LoRA configuration was identical to Experiment 1 (r = 16, α = 16, same target modules, learning rate, and epoch count). The resulting reasoning-augmented model was evaluated on the same test set as Experiment 1, but using a 100-sample subset to allow close inspection of CoT behavior.

The results (Table 2) showed that adding chain-of-thought supervision substantially degraded normalization performance: the CoT-trained DeepSeek 7B reached a WER of 38.85%, compared with 11.17% for the same model in Experiment 1. This drop may stem from the much smaller CoT training set (an order of magnitude less data), the possibility that explicit reasoning traces introduce noise rather than useful structure for this task, or the need for reinforcement-learning-based optimization instead of pure supervised fine-tuning.
Conclusion and Future Work

Across the non-reasoning models, our experiments show that model size provides modest gains, while a model's underlying Finnish proficiency is the dominant factor for successful normalization. Finnish-centric models such as Poro-2 8B clearly outperform larger but less linguistically aligned LLMs. Future work includes investigating the number of training samples required for convergence, exploring hyperparameter optimization (e.g., LoRA configurations, learning rate, and target modules), and extending the task in reverse by generating dialectal variants from normalized Finnish.

In contrast, the reasoning-focused approach in Experiment 2 did not improve performance, indicating that naively injecting chain-of-thought might be insufficient for dialect normalization with 3k training samples using supervised fine-tuning. Future work could start with a Finnish-proficient model and fine-tune it for reasoning, expand the volume and diversity of CoT training data, or apply reinforcement-learning methods such as PPO or GRPO with tailored reward functions to better align reasoning traces with accurate normalization.

References

[1] Partanen, N, Hämäläinen, M & Alnajjar, K 2019, Dialect
Text Normalization to Normative
Standard Finnish. in W Xu, A Ritter, T Baldwin & A Rahimi
(eds), The Fifth Workshop on Noisy User-generated Text
(W-NUT 2019) : Proceedings of the Workshop.

[2] DeepSeek-Al, Guo, D, Yang, D, Zhang, H, Song, J, Zhang, R, Xu, R, Zhu, Q, Ma, S, Wang, P et al. 2025, *DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning*. arXiv:2501.12948 [cs.CL]. https://arxiv.org/abs/2501.12948.

[3] Gemma Team, Kamath, A, Ferret, J, Pathak, S, Vieillard, N, Merhej, R, Perrin, S, Matejovicova, T, Ramé, A, Rivière, M et al. 2025, GEMMA 3 TECHNICAL REPORT. arXiv:2503.19786 [cs.CL]. https://arxiv.org/abs/2503.19786.

[4] Luukkonen, R, Burdge, J, Zosa, E, Talman, A, Komulainen, V, Hatanpää, V, Sarlin, P & Pyysalo, S 2024, PORO 34B AND THE BLESSING OF MULTILINGUALITY. arXiv:2404.01856 [cs.CL]. https://arxiv.org/abs/2404.01856.

[5] Silo-Al: Viking 7B-13B-33B: Sailing the Nordic seas of multilinguality. https://www.silo.ai/blog/viking-7b-13b-33b-sailing-the-nordic-seas-of-multilinguality (2021)

[6] Institute for the Languages of Finland. 2014. Suomen kielen n"aytteit"a - Samples of Spo-ken Finnish [online-corpus], version 1.0. http://urn.fi/urn:nbn:fi:lb-201407141.