ChrisGVE/unixcoder-base-Q8_0-GGUF

UniXcoder base — GGUF Q8_0

microsoft/unixcoder-base converted to GGUF at Q8_0 for llama.cpp's llama-server --embeddings.

File	unixcoder-base-Q8_0.gguf (138 MB)
SHA-256	81a5a20e748a1c97e63aaacf81febb82560e61734e572ca619f202719c0319c1
Source revision	558aa506226b13a7e83f66cb38c53e61b9603eac (safetensors)
Context	1024 tokens
Pooling	mean (stored in the file, no --pooling flag needed)
Dimensions	768

Prefix every input with <encoder-only></s>

UniXcoder reads a mode token. Its own encoder mode feeds <s> <encoder-only> </s> text </s>; llama-server adds <s> and the closing </s>, so send the rest as text:

llama-server -m unixcoder-base-Q8_0.gguf --embeddings --port 8080
curl -s localhost:8080/v1/embeddings -H 'Content-Type: application/json' \
  -d '{"input":["<encoder-only></s>def f(): pass"]}'

The three mode tokens (<encoder-decoder>, <encoder-only>, <decoder-only>, ids 5–7) are marked special in this file so the text form maps to the single id; in the upstream tokenizer they are ordinary vocabulary entries that UniXcoder's code inserts by id.

How it was converted

convert_hf_to_gguf.py (llama.cpp, 2026-10-01) with --outtype q8_0, after:

• writing a tokenizer.json with RobertaTokenizerFast.save_pretrained (upstream ships only vocab.json + merges.txt; without it the converter falls back to a WordPiece vocabulary), with the three mode tokens registered as special added tokens (ids unchanged);
• adding a sentence-transformers modules.json + 1_Pooling/config.json declaring mean pooling;
• mapping this tokenizer's pre-tokenizer hash (f956de60…) to llama.cpp's roberta-bpe. It is a RobertaTokenizer with an extended vocabulary, so the GPT-2 byte-level split applies; the converter's hash table did not list it.

Verification

40 random code/console/output blocks (first 600 characters), embedded by this file on llama-server (CPU) with the prefix above and by the original model in PyTorch fp32 ([CLS] <encoder-only> [SEP] text [SEP], masked mean over the last hidden state). Token ids are identical; cosine similarity min 0.9994, median 0.99978.

The weights and their licence (Apache-2.0) are Microsoft's; see the UniXcoder paper.