EphAsad/SatellaRAG-0.8B

Satella-RAG-0.8B

Satella-RAG-0.8B is a compact retrieval-grounded reasoning model fine-tuned from Qwen/Qwen3.5-0.8B.

It is designed to act as the generation component of a Retrieval-Augmented Generation (RAG) pipeline: given retrieved documents and a question, Satella is trained to prioritize the supplied evidence, ignore irrelevant context, preserve useful citations, and state when the evidence is insufficient rather than inventing an answer.

Satella is a text-focused fine-tune. The vision-side parameters of the Qwen3.5 base model were frozen during training and vision behavior was not evaluated.

What Satella is designed to do

Satella is trained to:

• answer questions from supplied retrieved evidence;
• prioritize retrieved evidence over unsupported prior knowledge;
• distinguish relevant evidence from distractor documents;
• identify when supplied evidence is insufficient;
• preserve document citations when citation identifiers are available;
• identify conflicts between supplied documents rather than silently resolving them;
• support grounded-only and mixed evidence/knowledge response modes;
• retain useful general reasoning ability when no retrieval context is supplied.

Satella is not a retriever or embedding model. It expects relevant context to be supplied by an external retrieval pipeline.

Quick start

Run locally with llama.cpp

For most users, the GGUF releases are the easiest way to run Satella locally.

Q5_K_M is a good starting point if you want a practical balance between model size and quality.

First, install the Hugging Face CLI and download the GGUF:

pip install -U huggingface_hub

hf download EphAsad/Satella-RAG-0.8B \
  gguf/Satella-RAG-0.8B-Q5_K_M.gguf \
  --local-dir .

Then run it with a recent build of llama.cpp:

./llama-cli \
  -m gguf/Satella-RAG-0.8B-Q5_K_M.gguf \
  -c 8192 \
  -cnv \
  --reasoning on

Reasoning mode is strongly recommended for this release. Release validation showed a large degradation in grounded RAG performance when reasoning was disabled.

Try a grounded RAG prompt

Once Satella is running, try:

### Retrieved evidence

Document:0
Text: Reagent Alpha is prepared by adding 5 mL of concentrate to 95 mL of sterile water.

Document:1
Text: Reagent Beta is prepared using 10 mL of concentrate and 90 mL of sterile water.

### Response mode

GROUNDED

Use only information supported by the supplied evidence.
Ignore irrelevant retrieved material.
If the evidence does not contain the answer, state that clearly.

### Question

How much concentrate is used to prepare Reagent Alpha?

Expected answer:

5 mL

Run Satella as a local API

llama-server can expose the GGUF through an OpenAI-compatible chat-completions endpoint:

./llama-server \
  -m gguf/Satella-RAG-0.8B-Q5_K_M.gguf \
  -c 8192 \
  --reasoning on \
  --host 127.0.0.1 \
  --port 8080

The API will then be available locally at:

http://127.0.0.1:8080/v1

A minimal Python RAG request:

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8080/v1",
    api_key="local",
)

system_prompt = """You are Satella, a compact retrieval-grounded reasoning model.

When retrieved evidence is supplied:
- Treat the supplied evidence as the primary source of truth.
- Ignore irrelevant retrieved material.
- Do not invent unsupported claims.
- If the evidence is insufficient, state this clearly.
"""

retrieved_context = """
Document:0
Text: Reagent Alpha is prepared by adding 5 mL of concentrate
to 95 mL of sterile water.

Document:1
Text: Reagent Beta uses 10 mL of concentrate.
"""

user_prompt = f"""### Retrieved evidence

{retrieved_context}

### Response mode

GROUNDED

Use only information supported by the supplied evidence.

### Question

How much concentrate is used to prepare Reagent Alpha?
"""

response = client.chat.completions.create(
    model="Satella-RAG-0.8B",
    messages=[
        {
            "role": "system",
            "content": system_prompt,
        },
        {
            "role": "user",
            "content": user_prompt,
        },
    ],
    temperature=1.0,
    top_p=0.95,
    presence_penalty=1.5,
    max_tokens=2048,
    extra_body={
        "top_k": 20,
        "min_p": 0.0,
    },
)

print(response.choices[0].message.content)

The retrieval step is intentionally external: replace retrieved_context with documents returned by your own vector database, keyword search, BM25 retriever, reranker, or other RAG pipeline.

Other GGUF sizes

If you want a different size/quality trade-off, replace the filename with another included release:

Satella-RAG-0.8B-Q8_0.gguf
Satella-RAG-0.8B-Q6_K.gguf
Satella-RAG-0.8B-Q5_K_M.gguf
Satella-RAG-0.8B-Q4_K_M.gguf

Q8_0 retains the most precision among the quantized builds, while Q4_K_M is the smallest included release.

Recommended inference mode

Thinking/reasoning mode is strongly recommended.

During release validation, Satella showed a substantial difference between thinking and non-thinking inference. In a small controlled diagnostic using grounded prompts with distractor evidence, thinking mode answered all tested runs correctly, while non-thinking mode succeeded on only 2/10 runs.

This diagnostic is small and should not be treated as a general benchmark, but the difference was large and consistent enough that the current release should be considered reasoning-first.

For llama.cpp, do not disable reasoning for the default Satella configuration.

Example:

./llama-cli \
  -m gguf/Satella-RAG-0.8B-Q5_K_M.gguf \
  -c 8192 \
  -cnv \
  --reasoning on

For other runtimes, preserve the repository's Qwen3.5 chat template and keep thinking/reasoning enabled where supported.

Recommended RAG prompt structure

The fine-tuning pipeline normalized grounded examples into a structure similar to:

### Retrieved evidence

Document:0
Text: <retrieved text>

Document:1
Text: <retrieved text>

### Response mode

GROUNDED

Use only information supported by the supplied evidence.

### Question

<question>

For tasks where outside knowledge is intentionally allowed, a mixed mode can instead instruct the model to use retrieved evidence as the primary source while permitting relevant prior knowledge.

Canonical Satella system prompt

You are Satella, a compact retrieval-grounded reasoning model.

Your primary purpose is to reason over supplied evidence and produce accurate, concise, traceable answers.

When retrieved evidence is supplied:
- Treat the supplied evidence as the primary source of truth.
- Use only claims that are supported by the evidence unless the requested response mode explicitly permits relevant prior knowledge.
- Ignore irrelevant or distracting retrieved material.
- Cite supplied documents when citation identifiers are available.
- Never fabricate a citation or claim that evidence says something it does not.
- If the supplied evidence is insufficient to answer the question, state this clearly rather than inventing an answer.
- If supplied documents conflict, identify the conflict rather than silently choosing or inventing a resolution.

When no retrieved evidence is supplied:
- Answer normally using your reasoning and existing knowledge.
- Prefer clear, direct and concise responses.

You are Satella.

Training

Satella was trained with a single BF16 LoRA supervised fine-tuning run.

Setting	Value
Base model	Qwen/Qwen3.5-0.8B
Training examples	22,805
Training tokens	27,673,370
Maximum rendered sequence	8,192 tokens
Maximum assistant response used for SFT	2,048 tokens
Epochs	1
Per-device batch size	8
Gradient accumulation	4
Effective batch size	32
Learning rate	8e-5
Scheduler	cosine
Warmup ratio	0.03
Weight decay	0.01
LoRA rank	32
LoRA alpha	32
LoRA dropout	0
Trainable parameters	~21.6M
Trainable fraction	~2.47%
Packing	disabled
Loss	assistant-response tokens only

The LoRA targets covered standard attention/MLP projections as well as Qwen3.5 DeltaNet projection modules.

Training examples were rendered through the Qwen chat template before length filtering. Examples exceeding the 8,192-token context limit were discarded rather than truncated.

Training data

The SFT mixture used the following public datasets:

• glaiveai/RAG-v1
• neural-bridge/rag-dataset-12000
• neural-bridge/rag-hallucination-dataset-1000
• OpenDataArena/Spark-234K

The mixture was balanced by rendered training tokens, not raw row counts.

Source	Rows	Tokens	Approx. token share
Glaive RAG-v1	13,803	15,210,966	54.97%
Neural Bridge RAG	5,047	5,531,985	19.99%
Spark-234K	3,125	6,085,734	21.99%
Neural Bridge hallucination	766	829,829	3.00%
Satella identity examples	64	14,856	0.05%

The Satella identity examples are small synthetic examples used only to establish the model name and retrieval-grounded role.

Split hygiene

The pipeline:

1. created held-out validation/test splits before training mixture sampling;
2. normalized all sources into a common schema;
3. applied exact prompt deduplication;
4. removed training prompts that collided with held-out evaluation prompts;
5. rejected overlength examples rather than truncating them.

This reduces exact train/evaluation leakage, but does not guarantee semantic independence between related examples originating from the same source datasets.

Initial held-out comparison

The initial project-specific held-out comparison produced:

Metric	Qwen3.5-0.8B	Satella-RAG-0.8B
Overall reference F1	0.3270	0.7624
Glaive reference F1	0.4812	0.5893
Glaive citation F1	0.0250	0.9100
Neural RAG reference F1	0.4819	0.6978
Unanswerable detection	0.0750	1.0000

These figures are not external benchmark results and are not claims of general model superiority.

Important limitations of this comparison:

• the evaluation examples came from held-out splits of dataset families also used during training;
• exact prompt decontamination was performed, but semantic overlap may still exist;
• reference word F1 is sensitive to wording and does not fully measure semantic correctness;
• the initial 120-example comparison did not contain a valid Spark subset;
• the unanswerable subset contains relatively standardized answer patterns and should not be interpreted as proof of perfect hallucination resistance.

The results are best viewed as evidence that the SFT materially changed the base model toward the intended RAG behavior, not as a comprehensive evaluation.

Release formats

The repository may include:

• the merged Hugging Face checkpoint;
• the original LoRA adapter;
• the canonical Satella system prompt;
• GGUF language-model exports:
  • F16
  • Q8_0
  • Q6_K
  • Q5_K_M
  • Q4_K_M

Approximate GGUF sizes from the release build:

Quantization	Approx. size
F16	1.45 GB
Q8_0	0.78 GB
Q6_K	0.60 GB
Q5_K_M	0.55 GB
Q4_K_M	0.50 GB

All five language-model GGUF variants passed basic llama.cpp smoke validation during the release build.

A multimodal projector may also be present because the Qwen3.5 base architecture supports multimodality. It is not a standalone model, and Satella's fine-tuning/evaluation in this release was text-only.

ONNX is not included as a validated release format.

Choosing a GGUF

The quantized releases are intended to make Satella practical on modest hardware.

As a general starting point:

• Q8_0 — highest-precision quantized option;
• Q6_K — high-quality middle ground;
• Q5_K_M — practical balance of size and quality;
• Q4_K_M — smallest included build.

Quantization quality should be evaluated on the target workload. The project does not currently claim that one quantization is universally best.

Intended use

Good fits include:

• retrieval-grounded assistants;
• document question answering;
• local knowledge-base chat;
• citation-aware RAG generation;
• multi-document evidence synthesis;
• answering with explicit abstention when evidence is missing;
• compact CPU-oriented RAG applications using GGUF runtimes.

Limitations

Satella can still:

• hallucinate;
• misread or over-weight retrieved evidence;
• fail to abstain when it should;
• abstain when sufficient evidence is available;
• produce incorrect or malformed citations;
• be affected by contradictory, misleading, or adversarial retrieval context;
• behave differently across quantizations and inference runtimes.

The model is particularly sensitive to disabling its reasoning mode in the current release.

Retrieval quality remains part of the overall system: Satella cannot ground an answer in evidence that was never retrieved.

Safety and high-stakes use

Satella should not be treated as an autonomous authority for medical, laboratory, legal, financial, safety-critical, or regulated decisions.

For operational or regulated workflows, deterministic software, validated procedures, source documents, and qualified human review should remain authoritative.

Data and privacy

The SFT run described in this model card used the public datasets listed above plus a small set of synthetic Satella identity examples.

No private workplace data, proprietary documents, personal records, or private user conversations were used as training data for this release.

The training dataset itself is not distributed with the public model release.

Users building RAG applications with Satella are responsible for the privacy, access controls, retention rules, and licensing of the documents they retrieve into the model's context.

Reproducibility notes

The release pipeline was designed to:

• normalize heterogeneous RAG datasets into a common prompt schema;
• split evaluation data before training mixture construction;
• deduplicate/decontaminate exact prompts;
• measure sequence length after full chat-template rendering;
• reject rather than truncate overlength samples;
• train one supervised LoRA stage;
• compare the resulting model against the untouched base model;
• merge the adapter;
• export multiple GGUF quantizations;
• validate language-model GGUFs with llama.cpp.

Future work

Potential follow-up work includes:

• stronger independent RAG evaluation;
• multi-seed quantization comparisons;
• conflict-resolution benchmarks;
• citation precision/recall under adversarial retrieval;
• more difficult unanswerable-question evaluation;
• long-context retrieval stress tests;
• reasoning-mode latency/quality evaluation;
• a separate fast/non-thinking distillation variant;
• optional preference/RL optimization for groundedness and abstention calibration.

Acknowledgements

Satella-RAG-0.8B is fine-tuned from Qwen/Qwen3.5-0.8B.

Training data was sourced from the public datasets listed in the Training data section. Please consult the original model and dataset repositories for their respective documentation, licenses, and usage terms.