naver

/

v-splade-efficient

like 7

Follow

NAVER 231

Hello!

This is the companion PR to https://huggingface.co/naver/v-splade-quality/discussions/2, except for this model instead. See that PR description for more details. I also want to point out that the existing README scores were computed with the quality model, despite the efficient model listed in the code block above it. My new README scores were computed with this model, so the scores look like they differ now (when they actually match).

You can try it like this:

pip install -U sentence-transformers[image]

from sentence_transformers import SparseEncoder

model = SparseEncoder("naver/v-splade-efficient", trust_remote_code=True, revision="refs/pr/1")

queries = ["send signed forms", "records office"]
documents = ["https://raw.githubusercontent.com/naver/v-splade/main/examples/sample_page.png"]

query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# torch.Size([2, 50368]) torch.Size([1, 50368])

similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7757],
#         [0.4524]], device='cuda:0')

# Inspect the top activated tokens of the page image
decoded = model.decode(document_embeddings[0], top_k=5)
print([(token, round(weight, 3)) for token, weight in decoded])
# [('Ġdog', 1.664), ('ĠRecords', 1.5), ('Ġpuppy', 1.469), ('ĠBennett', 1.414), ('Ġdogs', 1.398)]

Note that the massive PR diff is because of the ~3MB tokenizer.json being duplicated for the custom Li-LSR module.

• Tom Aarsen