Fietje: An open, efficient LLM for Dutch
Authors: name: Bram Vanroy
Published: 2024-12-19 23:06:01

Abstract: This paper introduces Fietje, a family of small language models (SLMs)
specifically designed for the Dutch language. The model is based on Phi 2, an
English-centric model of 2.7 billion parameters. Fietje demonstrated
competitive results with larger language models upon its release. A core
emphasis of this work is transparency and reproducibility: Fietje is fully
open-source, with model weights, datasets, training, and evaluation code all
publicly accessible.
  The paper discusses the performance of Fietje and many other models on an
extensive evaluation suite of benchmarks on reasoning, sentiment analysis,
world knowledge, linguistic acceptability and word sense disambiguation.
Evaluation results illustrate the rapid progress in the field of LLMs, where
recent small models outperform older, larger models that were fine-tuned for
Dutch. This trend signals an exciting future for Dutch language processing,
suggesting that even compact LLMs are becoming increasingly capable.
Furthermore, ongoing and future efforts to adapt LLMs to Dutch are poised to
enhance these models even further, broadening their applicability and
accessibility. Fietje is only an intermediate step in improving accessibility
to language technology for users of the Dutch language.