michaelfeil/zembed-1-infinity

zembed-1 for Infinity

Infinity configuration adaptation of ZeroEntropy's zembed-1-embedding, pinned to revision cf13c81f3274394053d166740294f7eea4586f7a. Model weights, tokenizer, pooling, and learned projection weights are unchanged. The original model card is retained in UPSTREAM_README.md, and the original license is retained in LICENSE.

modules.json replaces modeling_zembed.ZembedTransformer with the standard Sentence Transformers Transformer module so Infinity v3's LibTorch loader accepts the checkpoint. The original wrapper appends <|im_end|>\n to each input. Clients must still append this suffix; this copy does not make raw-text requests automatically equivalent to the original model.

Run

docker run --rm --gpus all -p 7997:80 -v infinity-hf-cache:/data \
  ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.0-alpha.3 \
  --model-id michaelfeil/zembed-1-infinity \
  --backend libtorch --torch-device cuda --device-id 0 \
  --dtype bfloat16 --pooling last-token \
  --default-prompt-name document --max-batch-tokens 32768

The saved query and document prompts provide the prefix. Supply the suffix exactly once, after the unformatted input text:

curl http://localhost:7997/embed \
  -H 'Content-Type: application/json' \
  -d '{"model":"michaelfeil/zembed-1-infinity","inputs":["What is backpropagation?<|im_end|>\n"],"prompt_name":"query","normalize":true,"truncate":false}'

Use prompt_name: "document" for documents. infinity_client.py adds the suffix automatically:

from infinity_client import embed

queries = embed(["What is backpropagation?"], kind="query")
documents = embed(["Backpropagation computes gradients."], kind="document")

This setup returns normalized, full-width 2560-dimensional embeddings with last-token pooling. The original learned dimensionality projections are preserved, but are not applied by this Infinity setup; simple truncation is not equivalent to those projections. Inputs above 32768 tokens should be chunked. Avoid truncating away the terminal suffix.

This repository is a configuration adaptation, not a new training run. Weight hashes are recorded in infinity-preparation.json. It has not yet undergone an end-to-end embedding parity run against the original Sentence Transformers model.