use-stochastic-no-pt

This model is a fine-tuned version of paraphrase-multilingual-mpnet-base-v2, trained on the Ukrainian text corpus UberText 2.0 with markov-based stochastic augmentation without pool targets. It is part of the Ukrainian Sentence Embeddings collection, which explores the effect of different training strategies on sentence embedding quality for Ukrainian.

Model Description

The model was fine-tuned using a contrastive objective on UberText 2.0, markov-based stochastic augmentation.

Collection Overview

Model	Description
use-natural-no-pt	Raw UberText 2.0, no augmentation, no pool targets
use-natural-pt	Raw UberText 2.0, no augmentation, pool targets
use-generated-no-pt	Generated augmentation, no pool targets
use-generated-pt	Generated augmentation, pool targets
use-translation-no-pt	Back-translation augmentation, no pool targets
use-translation-pt	Back-translation augmentation, pool targets
use-mask-no-pt	Masking augmentation, no pool targets
use-mask-pt	Masking augmentation, pool targets
use-dropout-no-pt	Dropout augmentation, no pool targets
use-dropout-pt	Dropout augmentation, pool targets
use-token-shuffle-no-pt	Token-shuffling augmentation, no pool targets
use-token-shuffle-pt	Token-shuffling augmentation, pool targets
use-combined-no-pt	Combined augmentation strategies, no pool targets
use-combined-pt	Combined augmentation strategies, pool targets
use-stochastic-no-pt	Markov-based stochastic augmentation, no pool targets
use-stochastic-pt	Markov-based stochastic augmentation, pool targets

Usage

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("victormuryn/use-stochastic-no-pt")
sentences = [
    "Проводжає сина мати захищати рідний край",
    "Хоч би малесеньку хатину він мріяв мати над Дніпром",
]
embeddings = model.encode(sentences)
print(embeddings.shape)

Training Details

• Base model: paraphrase-multilingual-mpnet-base-v2
• Training corpus: UberText 2.0
• Augmentation: Markov-based stochastic
• Pool targets: No

Citation

To be added

License

Apache 2.0