zeromodels/all-distilroberta-v1

Run sentence embeddings with Keras 3

zeromodels/all-distilroberta-v1

Pure-Keras 3 conversion of sentence-transformers/all-distilroberta-v1 for ZeroModels. It encodes text into 768-dimensional vectors using a roberta backbone. Its pooling is mean, its embeddings are L2-normalized, and the intended scoring function is cosine similarity. The conversion preserves the source model's embedding recipe; it does not introduce a new training run.

For details about training and intended use, see the original model card.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # set before importing Keras / ZeroModels

from keras import ops
from zeromodels.models.sentence_transformer import (
    SentenceTransformerModel,
    SentenceTransformerTokenizer,
)

repo = "zeromodels/all-distilroberta-v1"
model = SentenceTransformerModel.from_weights(repo)
tokenizer = SentenceTransformerTokenizer.from_weights(repo)

texts = ["A person is reading a book.", "Someone reads a novel."]
embeddings = model.encode(texts, tokenizer=tokenizer)
scores = ops.convert_to_numpy(model.similarity(embeddings, embeddings))
print(embeddings.shape)  # (2, 768)
print(scores)

model.similarity() uses cosine similarity, matching the source model. If you need unit-length vectors for a checkpoint that is not normalized, pass normalize=True to model.encode(...); this changes the original embedding recipe and may change retrieval scores.

Model details

Property	Value
Backbone	roberta
Pooling	mean
Dense projection	none
Embedding dimension	768
L2 normalization	yes
Scoring	cosine similarity
Maximum input length	512 tokens
Original checkpoint	sentence-transformers/all-distilroberta-v1
Original revision	842eaed40bee4d61673a81c92d5689a8fed7a09f

All 13 converted variants use from_weights("zeromodels/<variant>"):

Variant	Encoder and recipe
all-mpnet-base-v2	MPNet · mean + normalize
multi-qa-mpnet-base-dot-v1	MPNet · CLS + dot product
all-distilroberta-v1	RoBERTa · mean + normalize
all-MiniLM-L12-v2	BERT · mean + normalize
multi-qa-distilbert-cos-v1	DistilBERT · mean + normalize
all-MiniLM-L6-v2	BERT · mean + normalize
multi-qa-MiniLM-L6-cos-v1	BERT · mean + normalize
paraphrase-multilingual-mpnet-base-v2	XLM-R · mean
paraphrase-albert-small-v2	ALBERT · mean
paraphrase-multilingual-MiniLM-L12-v2	BERT · mean
paraphrase-MiniLM-L3-v2	BERT · mean
distiluse-base-multilingual-cased-v1	DistilBERT · mean + Dense
distiluse-base-multilingual-cased-v2	DistilBERT · mean + Dense

Tips

• Set KERAS_BACKEND before importing Keras or ZeroModels. Embedding parity was checked on the Torch backend; full TensorFlow/JAX parity is not claimed.
• Use the tokenizer from the same repo as the model. It preserves the source truncation length of 512 tokens for this checkpoint.
• For exact source loading without these converted Keras weights, use SentenceTransformerModel.from_weights("hf:sentence-transformers/all-distilroberta-v1").
• The repository contains model.weights.h5, zm_config.json, and tokenizer.json. The pinned source commit is listed above.

Verification and attribution

The conversion was compared with Sentence Transformers at the pinned source revision. Maximum embedding absolute difference: 8.19564e-08; minimum embedding cosine: 1.00000000; maximum similarity-score absolute difference: 2.98023e-07.

Special thanks to the Sentence Transformers team for the original checkpoint. The upstream card declares apache-2.0.

ZeroModels documentation · Source model · This checkpoint