Sentence similarity model based on SlovakBERT

This is a sentence similarity model based on SlovakBERT. The model was fine-tuned using STSbenchmark [Cer et al 2017] translated to Slovak using M2M100. The model can be used as an universal sentence encoder for Slovak sentences.


The model was evaluated in our paper [Pikuliak et al 2021, Section 4.3]. It achieves 0.7910.791 Spearman correlation on STSbenchmark test set.


Using this model becomes easy when you have sentence-transformers installed:

pip install -U sentence-transformers

Then you can use the model like this:

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('kinit/slovakbert-sts-stsb')
embeddings = model.encode(sentences)


  author    = {Mat{\'{u}}s Pikuliak and
               Stefan Grivalsky and
               Martin Konopka and
               Miroslav Blst{\'{a}}k and
               Martin Tamajka and
               Viktor Bachrat{\'{y}} and
               Mari{\'{a}}n Simko and
               Pavol Bal{\'{a}}zik and
               Michal Trnka and
               Filip Uhl{\'{a}}rik},
  title     = {SlovakBERT: Slovak Masked Language Model},
  journal   = {CoRR},
  volume    = {abs/2109.15254},
  year      = {2021},
  url       = {},
  eprinttype = {arXiv},
  eprint    = {2109.15254},
Downloads last month
Hosted inference API
Sentence Similarity
This model can be loaded on the Inference API on-demand.

Dataset used to train kinit/slovakbert-sts-stsb