harrier-gst-embed-v1

Domain-supervised retrieval embedding for Indian GST (Goods and Services Tax) law — a LoRA fine-tune of microsoft/harrier-oss-v1-0.6b trained to bind practitioner queries (including statute-cited queries like "section 138 of the CGST Act compounding of offences") to the correct provision in a 189k-chunk Central GST legal corpus: bare acts, rules, CBIC notifications/circulars, tribunal & high-court judgments, and GSTN/NIC advisories.

Key property: Matryoshka (1024/512/256) is trained in — the 0.6B base has no native MRL, and this fine-tune installs it, so client-side 1024→256 truncation + re-normalization stays coherent (256-d retains ~89% of the 1024-d margin).

Results (internal dev set, 500 synthetic pairs, mined hard negatives)

Metric	Base (harrier-oss-v1-0.6b)	Fine-tuned	Δ
Margin @1024d	0.0392	0.1784	4.6×
Margin @512d	0.0288	0.1684	5.8×
Margin @256d	0.0191	0.1597	8.4×

By query style @256d (failure rate = hard negative outranked the true positive):

Style	Base fail-rate	FT fail-rate
statute-qualified	28%	5%
term-of-art	39%	17%
plain practitioner	42%	22%
scenario/fact-pattern	41%	22%

Q8_0 GGUF round-trip vs merged HF fp32: cosine ≥ 0.9988 at 1024d and 256d.

Usage

sentence-transformers (queries must carry the instruction; passages get none):

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("chanderbalaji/harrier-gst-embed-v1")
instruction = "Retrieve passages from the Central GST legal corpus that answer the query."
query = f"Instruct: {instruction}\nQuery: what is the time limit for issuing a tax invoice in continuous supply"
passages = ["Section 31 of the CGST Act provides ...", ...]  # raw text, no prefix

q = model.encode(query)                 # 1024-d, L2-normalized
d = model.encode(passages)
scores = q @ d.T

Matryoshka truncation (deployed configuration):

import numpy as np
emb_256 = emb[:, :256]
emb_256 = emb_256 / np.linalg.norm(emb_256, axis=1, keepdims=True)

llama.cpp (production serving configuration, verified):

llama-server -m harrier-gst-embed-v1-q8_0.gguf --embedding --pooling last -ngl 99 -c 8192

--pooling last is required — the model uses last-token pooling; default pooling silently produces garbage.

Training details

• Base: microsoft/harrier-oss-v1-0.6b (Qwen3 architecture, 1024-d, last-token pooling, MIT)
• Adapter: LoRA r=32, α=64, dropout 0.05, all-linear targets (~20.2M params)
• Loss: MultipleNegativesRankingLoss (in-batch + 5 hard negatives/row) wrapped in MatryoshkaLoss (dims 1024/512/256, weights 1.0/0.5/0.25)
• Data: 23,997 train / 500 dev synthetic query↔passage pairs; queries generated with a local Qwen3.8-27B server over a 10k-chunk stratified seed bundle (legislation 16%, notifications 35%, judgments 20%, circulars 15%, rules 10%, advisories/forms 4%); 16% statute-qualified queries; ~5% Hinglish; hard negatives mined per chunk via BM25 plus deliberate same-section-number-wrong-statute pairs (the "section 138 collides across statutes" defect)
• Filters: statute-grounding guard (every section/rule/form/act reference in a query must appear in its passage), length/syntax guards, global exact + 8-gram near-duplicate dedupe
• Schedule: 2 epochs, batch 16, lr 1e-4, warmup 5%, bf16, gradient checkpointing, 3,000 steps
• Provenance: full sha256 manifest, per-pair provenance (query hash → source chunk ids → negative chunk ids), training loss/dev-margin curves retained

Limitations

• Evaluated on internal synthetic dev margins, not public MTEB; treat general-domain retrieval quality as inherited from the base, not improved.
• Queries are English with a small Hinglish slice; passages are English only.
• Trained for GST-domain retrieval against a specific hybrid (lexical+vector+graph) stack; statute-collision resistance was optimized for Indian-statute collisions specifically.

License

MIT — same as the base model. Copyright (c) 2026 Shikhar Pant. See LICENSE for the full text. The training dataset itself is not distributed with this model; queries were generated from only publicly available Indian GST legal text such as statutes, notifications, finance acts, circulars, instructions, limited landmark public court orders from High Courts and the Supreme Court, and GSTN advisories. The model is a retrieval aid over legal text, not legal advice; verify outputs against primary sources.