Isolde

Isolde is an experimental, gated research model released under the ORIS Research License.

It is designed primarily for private, high-throughput local data processing, rather than as a general-purpose encoder. The model is intentionally optimized around a specific production stack and should currently be treated as a research component rather than a drop-in replacement for substantially larger Polish encoders.

Architecturally, Isolde is a close sibling of ORIS Bert Small C. It retains the same general encoder philosophy and most of the Transformer backbone, while introducing a smaller factorized embedding path, a new tokenizer, and a different pre-training data stack.

Isolde is evaluated together with the newer OFlow 0.2 runtime and its more aggressive OFlow 0.2-MAX execution profile. The model, data pipeline and runtime are treated as one system: the objective is not only encoder quality in isolation, but practical quality at very high local processing throughput.

This remains an experimental research release. Runtime results are workload- and hardware-specific.

---

Isolde vs. ORIS Small C

Isolde keeps the main Transformer geometry of ORIS Small C largely unchanged. The largest architectural reduction is in the factorized token-embedding path, while the training data and tokenizer stack were also revised.

Property	Isolde	ORIS Bert Small C
Parameters	~16.87M	25.41M
Token embedding size	64	128
Factorized embeddings	64 → 384	128 → 384
Vocabulary / tokenizer	128K, new tokenizer	128K ORIS Polish BPE
Pre-training data stack	V1 + AUX; data previously cleaned through the ORIS pipeline	Original ORIS data stack
Pre-training tokens	~6.2B	8.00B

Isolde is not intended as a full redesign of the ORIS encoder core. The goal was to retain most of the useful 6 × 384 encoder capacity while reducing parameter cost around the embedding path and moving to a newer data/tokenizer stack.

The ~6.2B pre-training-token figure is the current release estimate and should be replaced by the exact consumed-token counter before final publication if the release checkpoint changes.

---

Polish downstream benchmarks

Benchmark datasets were downloaded directly from the Hugging Face Hub with the datasets library:

from datasets import load_dataset

dataset = load_dataset("<dataset_id>")

No local clone of the historical KLEJ benchmark repository was required. The exact public datasets and evaluation metrics used were:

Task	Hugging Face dataset	Metric
PolEmo2.0-IN	allegro/klej-polemo2-in	Accuracy ↑
PolEmo2.0-OUT	allegro/klej-polemo2-out	Accuracy ↑
DYK	allegro/klej-dyk	Macro-F1 ↑
PSC	allegro/klej-psc	Macro-F1 ↑
CBD	allegro/klej-cbd	Macro-F1 ↑
CDSC-E	allegro/klej-cdsc-e	Accuracy ↑
CDSC-R	allegro/klej-cdsc-r	Spearman correlation ↑
NKJP-NER	allegro/klej-nkjp-ner	Accuracy ↑
Allegro Reviews	allegro/klej-allegro-reviews	AR score ↑

All compared encoders used the same local fine-tuning and evaluation protocol: the same optimizer family, seed, masked mean pooling, task-head structure and early-stopping logic. Pair tasks were tokenized as tokenizer(text_a, text_b). When a dataset did not expose a validation split, 10% of the training set was deterministically held out using the same fixed seed.

For Allegro Reviews, the reported score is:

1 - mean(class-wise normalized MAE)

Ratings are normalized from 1..5 to 0..1, MAE is computed separately for every true rating class, and the final value is one minus the macro-average class-wise error. Higher is better for every metric reported below.

Results

Task	Metric	Isolde (~16.87M)	ORIS Small C (25.41M)	Polish RoBERTa-v2 (~124.4M)
PolEmo2.0-IN	Accuracy ↑	84.63	86.57	88.92
PolEmo2.0-OUT	Accuracy ↑	71.26	68.83	73.08
DYK	Macro-F1 ↑	62.82	65.54	75.63
PSC	Macro-F1 ↑	64.77	54.62	97.35
CBD	Macro-F1 ↑	69.14	67.74	71.19
CDSC-E	Accuracy ↑	90.90	91.20	93.80
CDSC-R	Spearman ↑	89.32	88.07	92.98
NKJP-NER	Accuracy ↑	81.10	82.90	88.44
Allegro Reviews	AR score ↑	85.25	85.61	85.81

Bold values in the Isolde/ORIS columns indicate the better result between the two compact ORIS-family encoders only.

The larger reference used here is sdadas/polish-roberta-base-v2, not the older PolBERTa baseline used in the original ORIS Small C model card.

At approximately 124.4M parameters, Polish RoBERTa-v2 is about 7.4× larger than Isolde and about 4.9× larger than ORIS Small C. It is included as a strong modern reference rather than as a parameter-matched competitor and, as expected, is stronger on most of the evaluated tasks.

Despite reducing the encoder from 25.41M to approximately 16.87M parameters, Isolde remains close to ORIS Small C across much of the suite and exceeds it on PolEmo2.0-OUT, PSC, CBD and CDSC-R.

The objective was not to consistently outperform an encoder more than seven times larger. The goal was to retain as much practical downstream quality as possible while moving toward a substantially smaller encoder designed for high-throughput production use.

---

Pipeline and OFlow runtime

The production filtering stack retains the full semantic path:

read/decompress → JSON decode → surface/template analysis → tokenization → document encoder → paragraph encoder → semantic analysis → scoring → KEEP/CLEAN/SPLIT/DROP → serialization → gzip output

OFlow does not remove these stages or change the filtering semantics. It reorganizes how work is batched, packed, transferred and scheduled between CPU and GPU.

OFlow evolution

Runtime	Main optimization	Controlled throughput	Relative to reference
Reference	Conventional sequential execution	154.74 docs/s	1.00×
OFlow 0.1	Length-aware batching, persistent model residency, pinned memory, asynchronous H2D and initial CPU/GPU overlap	316.55 docs/s	2.05×
OFlow 0.2	Deeper document/paragraph/semantic scheduling and model-specific execution geometry	516.71 docs/s	3.34×
OFlow 0.2-MAX	Exact token-budget packing, validated maximum batch geometry and aggressive hot-path execution profile	520.56 docs/s	3.36×

These controlled measurements isolate the main filtering compute path; sample preparation, long-running corpus I/O and persistent output/state handling are not included.

All optimized candidates are checked against the reference execution path. Configurations that fail the accepted numerical-equivalence check are rejected.

What changed

Area	OFlow 0.1	OFlow 0.2 / 0.2-MAX
Model residency	Persistent	Persistent
Batching	Length-aware batches	Exact token-budget batching
Padding	Reduced through bucketing	Shape-aware packing and alignment
Host → GPU	Pinned memory + async copy	Integrated with deeper scheduling
Tokenization	Background execution	Prefetched tokenization + cheaper packing
Document pass	Batched GPU execution	Model-specific batch geometry
Paragraph pass	Batched execution	Independent token budget + overlap
Semantic stage	Partial overlap	Bounded in-flight overlap with GPU work
Scheduling	Basic pipeline overlap	Producer / inference / semantic / writer overlap
Validation	Output equivalence	Equivalence + rejection of invalid or slower candidates

OFlow 0.2-MAX denotes the most aggressive validated full-stack profile, not simply the locally fastest microbenchmark configuration. A candidate is only useful when it improves the complete runtime while preserving the accepted outputs.

---

Sustained production throughput

Controlled OFlow benchmarks measure the optimized filtering compute path. Production throughput additionally includes corpus I/O, preprocessing, JSON handling, serialization, compression and persistent state management.

Production stack	Sustained throughput	Relative to early OFlow production
ORIS + OFlow 0.1	~126 docs/s	1.00×
ORIS + newer OFlow runtime	225.91 docs/s	1.79×
Isolde + OFlow 0.2-MAX	257.54 docs/s	2.04×

The current Isolde production run processed 100,000 documents in 388.284 seconds.

Metric	Isolde + OFlow 0.2-MAX
Documents / second	257.54
Documents / minute	~15.45k
Documents / hour	~927k
Approx. tokens / second	166.96k
Approx. tokens / minute	~10.02M
Approx. tokens / hour	~601M

Compared with the earlier sustained ORIS + OFlow production result of approximately 126 docs/s, the current stack provides about 2.04× throughput. Compared with the newer ORIS runtime measurement of 225.91 docs/s, Isolde + OFlow 0.2-MAX is approximately 14% faster on this production workload.

Current production profile

The 100k-document run remained primarily inference-bound rather than writer-bound:

Signal	Result
OFlow exposed wall	378.59 s / 97.5% of total wall
Document pipeline wall	251.41 s
└ document model CUDA	216.69 s
└ document tokenizer	30.47 s
└ document packing	8.63 s
Paragraph pipeline wall	99.52 s
└ paragraph model CUDA	71.14 s
└ paragraph tokenizer	25.57 s
└ paragraph packing	4.60 s
Semantic task	118.25 s
└ semantic exposed wait	23.43 s
Semantic work hidden	80.2%
Paragraph-build work hidden	~100%
Writer backpressure	0 s

The dominant remaining cost is the document encoder GPU path, followed by the paragraph encoder and tokenizer work. Producer and writer stages retain headroom at the current throughput.

Hardware context. The reported production measurements were collected on a local mid-range workstation:

• GPU: NVIDIA GeForce RTX 5060 Ti 16GB, BF16
• CPU: AMD Ryzen 7 5700X, 8 cores / 16 threads
• Storage: PCIe 4.0 x4 NVMe SSD, approximately 7.0 GB/s sequential read and 3.9 GB/s sequential write, with up to roughly 450k / 900k 4K read/write IOPS

At the current stage, the Isolde + OFlow 0.2-MAX stack is no longer primarily limited by output writing, Python orchestration, or basic preprocessing. Most of the remaining exposed runtime is concentrated in GPU encoder execution.

This indicates that further gains on the tested hardware are increasingly likely to require model-, kernel-, or GPU-level optimization, rather than additional producer/writer scheduling improvements. A faster CPU, a higher-memory-bandwidth GPU, or multi-GPU execution could shift the bottleneck and expose a different performance profile.

What happens with a stronger CPU and dual GPUs? Who knows... :p

Runtime results are specific to the tested hardware, input-length distribution, tokenizer, storage configuration, and filtering workload. OFlow speedups should not be interpreted as universal speedup factors for other models or pipelines.

---

Limitations

• Isolde is an experimental encoder optimized primarily for Polish high-throughput data-processing workloads.
• Raw sentence representations are not intended as a ready-to-use retrieval or sentence-embedding system without additional fine-tuning.
• Performance remains task dependent; substantially larger Polish encoders remain stronger on several downstream benchmarks.
• The model was trained primarily for Polish.
• OFlow performance depends on hardware, input-length distribution, tokenizer behavior and the surrounding CPU / I/O pipeline.
• Reported production throughput reflects the combined Isolde + OFlow stack and should not be interpreted as a model-only speedup.

---

Citation

@misc{OgrodzkiIsolde2026,
  author       = {Aleksander Ogrodzki},
  title        = {Isolde},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/OrisTeam/Isolde},
  note         = {Model architecture, training data pipeline, tokenizer pipeline, OFlow runtime development, and model training by the author}
}