SC-Embedding-4B

LoRA adapter for SC-Embedding: Retrieval Augmented Selective Contextualization for Long-Context Retrieval.

Code and training · 中文说明 · Base model

SC-Embedding combines selective history retrieval with contextual passage encoding. A single adapter supports query encoding, semantic history retrieval, and contextual passage representations. Outputs are normalized dense vectors for similarity search.

Usage

Install the SC-Embedding package and download this adapter and its base model. Base weights are required separately.

hf download linlp22/SC-Embedding-4B --local-dir checkpoints/SC-Embedding-4B
hf download Qwen/Qwen3-4B-Instruct-2507 --local-dir models/Qwen3-4B-Instruct-2507

import torch
import torch_npu
from sc_embedding.model import SCEmbedding

torch.npu.set_device(0)
model = SCEmbedding.from_pretrained(
    "/path/to/Qwen3-4B-Instruct-2507",
    adapter="/path/to/SC-Embedding-4B",
    device="npu:0",
)
documents = model.encode_passages([
    "Alice moved to Paris.",
    "She now works at the city hospital.",
])
queries = model.encode_queries(["Where does Alice work?"])
scores = queries @ documents.T

Each passage list is one ordered document or conversation. Queries are encoded independently. Use contextual=False for passages without history. For CUDA, use device="cuda:0" and omit the Ascend import and device selection.

Encoding configuration

• Backbone: Qwen/Qwen3-4B-Instruct-2507.
• History: semantic similarity + BM25 with reciprocal rank fusion (constant 60).
• Context: up to four strictly preceding passages, ordered chronologically, within a 2,048-token history budget.
• Representation: normalized final-token hidden state, with explicit history/passage boundaries.

sc_config.json supplies the inference configuration. This repository contains the LoRA adapter, not the full base model, optimizer state, or a separate teacher.

Checkpoint provenance

This selected existing adapter retains its original fixed-initial-teacher training provenance. The companion repository's training entry point implements online cross-view self-distillation. The exported tensors are unchanged and are not relabeled as outputs of that entry point. See provenance.json for the source policy and checksums.

Adapter SHA-256: ad64cbe0fedaf2ece963a08a249e7be8b77e765570782c6af20df98ffc8232eb.

License and acknowledgment

The adapter is distributed under Apache-2.0. Its base model and training data retain their respective licenses. Training builds on EvoEmbedding / EvoTrain-180K, Qwen, Transformers, and PEFT.