sharpsy/slovene-morphological-segmentation-sloberta

LLMSegm SloBERTa: Slovene Morphological Segmentation

Model Description

SloBERTa-based model for morphological segmentation of Slovene words, fine-tuned using the LLMSegm binary classification approach. The model predicts morpheme boundaries at each character position, decomposing words into their smallest meaningful morphological units. This model focuses on inflectional morphology - it identifies all morphemes including inflectional endings that encode grammatical information like case, gender, number, and aspect.

Training Data

Fine-tuned on a manually annotated dataset of 1,935 Slovene words with multidimensional morphological annotations:

• Morphological segments: All morphemes including inflectional endings (e.g., nepozidan → ne-po-zid-a-n-Ø)
• Word formation segments: Derivational formants used to create new words (e.g., nepozidan → ne-po-zida-n)
• Simplex annotations: Base words that cannot be further divided
• Zero-morpheme (Ø): Grammatical markers without phonetic form Dataset: Slovene Morphological and Word Formation Segmentation Dataset (CLARIN.SI repository) The dataset achieves inter-annotator agreement of 86.80% Krippendorff's Alpha for morphological segmentation

Performance

Metric	Score
BPR F1 (boundary precision-recall)	87.78%
Accuracy (exact word match)	53.61%
Evaluated using a holdout test set (10% split) with systematic hyperparameter optimization.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
# Load model
tokenizer = AutoTokenizer.from_pretrained("sharpsy/slovene-morphological-segmentation-sloberta")
model = AutoModelForSequenceClassification.from_pretrained("sharpsy/slovene-morphological-segmentation-sloberta")
model.eval()
# Segment a word
word = "nepozidan"  # "not built-up"
candidates = [f"{word} <__word-separator> {word[:i]}<__morph-boundary>{word[i:]}" 
              for i in range(1, len(word))]
inputs = tokenizer(candidates, return_tensors="pt", padding=True)
predictions = model(**inputs).logits.argmax(1).tolist()
# Reconstruct segmentation
segmented = word[0]
for i, boundary in enumerate(predictions):
    segmented += ("-" if boundary else "") + word[i+1]
print(segmented)  # Output: ne-po-zid-a-n

Model Architecture

• Base model: SloBERTa (Slovenian BERT)
• Method: LLMSegm binary classification
• Task: For each character position, predict whether a morpheme boundary exists
• Training: Binary cross-entropy loss with class weighting for imbalanced data

Hyperparameters

• Batch size: 128
• Learning rate: 4.36 × 10⁻⁵
• Weight decay: 3.80 × 10⁻⁴
• Label smoothing: 0.023
• Optimizer: AdamW

Morphological vs Word Formation Segmentation

This model predicts only morphological boundaries between segments such as roots and stems, prefixes (e.g., ne- for negation), suffixes (e.g., -ost for abstract nouns), inflectional endings (e.g., -a for nominative feminine).

For derivational analysis (how words are formed from base words), use our companion model for word-formation segmentation.

Limitations

• Training dataset is relatively small (1,935 words)
• Model may struggle with rare morphological patterns
• Trained on standard Slovene; may not handle dialectal or archaic forms well
• Assumes a single valid segmentation per word during training
• Does not predict zero-morphemes (Ø)
• Does not predict type of a morpheme (root, prefix, suffix, ...)

Citation

@inproceedings{pranjic-etal-2026-dataset,
  author    = {Pranjić, Marko and Kern, Boris and Voršič, Ines and Pollak, Senja},
  title     = {Slovene Morphological and Word Formation Segmentation: A Novel Dataset and Evaluation},
  booktitle = {Proceedings of the 15th Language Resources and Evaluation Conference (LREC 2026)},
  month     = {May},
  year      = {2026},
  address   = {Palma de Mallorca, Spain},
  publisher = {European Language Resources Association (ELRA)},
}

Acknowledgments

This work was supported by the Slovenian Research and Innovation Agency (ARIS) through:

• Core research program Knowledge Technologies (P2-0103)
• Project Formant combinatorics in Slovenian (J6-3131)