reign-base-l3_gn-gte-large_val-selected

REIGN base-l3 cross-chunk encoder trained on GoodWiki-Long-Synthetic over a frozen GTE-large guidance network — a long-document bi-encoder that reads a sequence of cached chunk embeddings instead of tokens.

Configuration

REIGN encoder	base-l3 — 3 layers, d = 768, 12 heads, FFN 3072, 22.45M trainable parameters
Guidance network (frozen)	thenlper/gte-large — GTE-large, 335M
Combined stack	357M parameters (paper Table 2)
Chunk size K	512 (the guidance network's context window)
Training stride S	512 — released-recipe default S = K; neither the name nor config.json records a stride for this run
Chunk-position signal	none — the encoder is a permutation-equivariant set function
Pooling	mean over the chunk sequence
Training data	devrim/goodwiki_long_synthetic_ir
Training recipe	released cosine recipe (below)
Checkpoint selection	best validation nDCG@10 on the val qrels split

Reported results

Every number below is the value the paper reports for this exact checkpoint; nothing is re-derived here.

Benchmark	Metric	Eval stride	Value	Paper
GoodWiki-Long test	nDCG@10	best stride	66.73	Table 2 / Table 7
LoCo (macro-avg, zero-shot)	nDCG@10	s384	70.77	Tables 3, 8
LoCo (macro-avg, zero-shot)	nDCG@10	s512	70.49	Table 8
DAPFAM test (zero-shot)	nDCG@100	s384	33.06	Tables 4, 8
DAPFAM test (zero-shot)	nDCG@100	s512	33.10	Tables 4, 5, 8

Measured inference cost (paper Table 11, Appendix G)

Setting	ms / query	Peak GPU (GB)
Bare guidance network, chunked mean-pool	121.4	—
REIGN, cold cache (guidance network re-run)	118.2	1.73
REIGN, cached chunk embeddings	0.5	1.45

Cached, that is a 229.4x lower per-query latency than re-running the guidance network. Uncached, REIGN is at parity with its own chunked guidance-network baseline, not faster. Protocol: 500 corpus documents, 100 queries, batch 8, one warm-up plus three timed repeats on a single 24 GB consumer GPU.

This is the checkpoint of the paper's paired significance test (Table 5). At stride 512 on DAPFAM it is statistically indistinguishable from Jina-Embeddings-v3 (+0.12, p = 0.86) and Stella-en-1.5B-v5 (+0.09, p = 0.92) under Holm-corrected paired tests, while beating BM25 by +7.48 and TF-IDF by +3.03 (both p < 0.001).

Usage

The checkpoint holds only the REIGN cross-chunk encoder. The guidance network is loaded separately and stays frozen, so both must be named at construction time.

pip install git+https://github.com/devrimcavusoglu/reign.git

import numpy as np
from huggingface_hub import snapshot_download
from reign.encoders.reign import ReignBaselineEncoder

checkpoint_path = snapshot_download("devrim/reign-base-l3_gn-gte-large_val-selected")

encoder = ReignBaselineEncoder(
    checkpoint_path=checkpoint_path,
    gn_model="thenlper/gte-large",
    chunk_size=512,
    stride=512,
)
docs = [open("doc_a.txt").read(), open("doc_b.txt").read()]
emb = encoder.encode(docs, batch_size=8)   # (2, hidden_size), L2-normalised
print(float(np.dot(emb[0], emb[1])))       # cosine similarity

ReignBaselineEncoder returns L2-normalised vectors, so the cosine is a dot product. chunk_size is the guidance network's sliding-window size — 512 for every released checkpoint, matching its context window — and stride controls the overlap, with stride == chunk_size giving non-overlapping chunking. The evaluation-time stride is a runtime argument, and the paper's headline tables report the best-performing stride per guidance network.

For the lower-level surface, ReignModel (a PreTrainedModel consuming inputs_embeds) and ReignFeatureExtractor (the guidance-network wrapper, with the on-disk embedding cache) are importable from reign and reign.feature_extractor.

Operating regime

REIGN targets multi-chunk inputs and primarily document-to-document retrieval. Inputs shorter than the chunk size collapse to a single chunk embedding, leaving the cross-chunk encoder nothing to aggregate — that regime is served by the guidance network alone, and this checkpoint should not be used for it.

Training recipe

The released checkpoints use a three-way cosine embedding loss over document pairs carrying graded targets s ∈ {1, 0, −1} — positive, partial, negative — with partial weight λ = 0.5.

Setting	Value
Objective	three-way cosine embedding loss, λ = 0.5
Batch construction	18 anchors × (1 positive + 2 partials + 17 in-batch negatives) = 360 pairs / step
Optimiser	AdamW, lr 1e-5, weight decay 1e-4, cosine annealing
Epochs	50, validating every 4
Selection	best validation nDCG@10
Precision	16-mixed
Seed	42
Guidance-network embeddings	precomputed and cached
Hardware	single 24 GB consumer GPU

A second, separate protocol (InfoNCE at τ = 0.07, batch 48, 20 epochs) is used only for the paper's positional-encoding and training-objective ablations. Those arms sit below this released operating point by construction and are not comparable to it. See docs/TRAINING.md in the code repository.

Because 16-mixed training is not bit-reproducible even at a fixed seed, a retrained checkpoint will not match these weights bit-for-bit; compare metrics, not weights.

Files

• config.json — ReignModel configuration
• model.safetensors — encoder weights (float32)

Links

• Code: https://github.com/devrimcavusoglu/reign
• Project page: https://devrimcavusoglu.github.io/reign
• Dataset: https://huggingface.co/datasets/devrim/goodwiki_long_synthetic_ir
• Paper: REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling, Findings of the Association for Computational Linguistics: EMNLP 2026 (to appear).

Citation

@inproceedings{cavusoglu2026reign,
  title     = {{REIGN}: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling},
  author    = {{\c{C}}avu{\c{s}}o{\u{g}}lu, Devrim and Akba{\c{s}}, Emre},
  booktitle = {Findings of the Association for Computational Linguistics: {EMNLP} 2026},
  year      = {2026},
  publisher = {Association for Computational Linguistics},
  note      = {To appear}
}

License

Apache License 2.0. The devrim/goodwiki_long_synthetic_ir dataset is released under CC BY-SA 4.0, preserving the share-alike licensing and attribution of GoodWiki and the underlying Wikipedia text.