Instructions to use sascha-frank-ai-research/tsft-rag-gemma-3-12b-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sascha-frank-ai-research/tsft-rag-gemma-3-12b-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sascha-frank-ai-research/tsft-rag-gemma-3-12b-it")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("sascha-frank-ai-research/tsft-rag-gemma-3-12b-it")
model = AutoModelForMultimodalLM.from_pretrained("sascha-frank-ai-research/tsft-rag-gemma-3-12b-it", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sascha-frank-ai-research/tsft-rag-gemma-3-12b-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sascha-frank-ai-research/tsft-rag-gemma-3-12b-it

SGLang

How to use sascha-frank-ai-research/tsft-rag-gemma-3-12b-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sascha-frank-ai-research/tsft-rag-gemma-3-12b-it with Docker Model Runner:
```
docker model run hf.co/sascha-frank-ai-research/tsft-rag-gemma-3-12b-it
```

TSFT-RAG Gemma-3-12B-IT

Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation

Part of the TSFT-RAG model family for Retrieval-Augmented Generation across Gemma, Qwen and Llama architectures.

This repository contains TSFT-RAG Gemma-3-12B-IT, a full-parameter fine-tuned derivative of Google's Gemma-3-12B-IT developed within the TSFT-RAG (Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation) research project.

The objective is to investigate how supervised full fine-tuning changes the behaviour of modern language models when used as Retrieval-Augmented Generation (RAG) systems.

TSFT-RAG Model Series

Model	Parameters	Status
Gemma-3-1B-IT	1B	Released
Gemma-3-4B-IT	4B	Released
Gemma-3-12B-IT	12B	Released

Abstract

General-purpose instruction models are optimized for broad conversational abilities but not specifically for Retrieval-Augmented Generation.

The TSFT-RAG project investigates whether full supervised fine-tuning can specialize foundation models for:

grounded question answering
unsupported question detection
structured information extraction
citation generation
reliable JSON generation

without changing the underlying transformer architecture.

The 12B model represents the largest Gemma model currently released within the TSFT-RAG project.

Research Background

Reliable Retrieval-Augmented Generation requires considerably more than fluent language generation.

An effective RAG model should

answer only from retrieved evidence,
reject unsupported questions,
preserve factual grounding,
generate machine-readable structured outputs,
remain deterministic across repeated executions.

The TSFT-RAG project evaluates these properties using identical datasets, prompts and evaluation procedures across Gemma, Qwen and Llama model families.

Model Details

Property	Value
Base model	google/gemma-3-12b-it
Training implementation	unsloth/gemma-3-12b-it
Architecture	Gemma3ForConditionalGeneration
Parameters	~12 Billion
Hidden Size	3840
Transformer Layers	48
Attention Heads	16
Context Window	131,072 tokens
Training Length	1,024 tokens
Precision	bfloat16
Fine-tuning	Full Parameter

Training Dataset

The TSFT-RAG benchmark contains supervised examples covering

Grounded Question Answering
Unsupported Question Detection
Topic Extraction
Key Message Extraction
Argument Extraction
Structured JSON Generation

The identical benchmark is used for every TSFT-RAG model to enable direct scientific comparison.

Evaluation Results

Metric	Base	TSFT-RAG	Improvement
Aggregate Score	0.450	0.686	+0.236
Hard Negative QA	0.162	0.829	+0.667
Grounded QA	0.613	0.594	-0.019
Topic Extraction	0.651	0.632	-0.019
Key Message	0.605	0.580	-0.026
Main Arguments	0.589	0.614	+0.025
JSON Validity	0.872	0.987	+0.115
Citation Precision	0.000	0.950	+0.950
Citation Recall	0.000	0.950	+0.950

Discussion

Among the currently released Gemma models, the 12B variant achieves the strongest overall performance.

The most notable improvements are observed for

unsupported-question detection,
structured JSON generation,
citation accuracy,
argument extraction.

Unlike many task-specific fine-tuning approaches, the model preserves strong grounded reasoning while substantially improving rejection behaviour and structured output quality.

These results demonstrate that full-parameter fine-tuning can effectively specialize large language models for Retrieval-Augmented Generation without architectural modifications.

Intended Use

Recommended for

Retrieval-Augmented Generation
Enterprise Knowledge Assistants
Scientific Information Systems
University Information Systems
Document Question Answering
Structured Information Extraction

Not intended as a general-purpose conversational assistant.

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
    "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it"
)

model = AutoModelForCausalLM.from_pretrained(
    "sascha-frank-ai-research/tsft-rag-gemma-3-12b-it"
)

Reproducibility

Training was performed using

Hugging Face Transformers
TRL
Unsloth

All relevant configuration files, tokenizer files, evaluation summaries and licensing information are included in this repository.

Limitations

Optimized specifically for Retrieval-Augmented Generation.
General open-domain conversation was not the primary optimization objective.
Evaluated primarily on German-language benchmark datasets.
No multimodal benchmark has yet been conducted.

Ethical Considerations

The model should always be used together with appropriate source attribution and human oversight.

It should not replace expert judgement in legal, medical or other safety-critical domains.

License

This repository contains a modified derivative of Google's Gemma model.

Please consult

LICENSE
NOTICE
MODIFICATIONS.md

for detailed licensing information.

Associated Publication

This model was developed and evaluated as part of the following research:

Frank, S., & Singh, R. (2026). Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation: A Multi-Family Evaluation Across Open-Weight Language Models.

Zenodo. DOI: 10.5281/zenodo.21638352

If you use this model in academic work, please cite the publication above.

@misc{Frank2026TSFTRAG,
  author       = {Frank, Sascha and Singh, Rawel},
  title        = {Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation:
                  A Multi-Family Evaluation Across Open-Weight Language Models},
  year         = {2026},
  month        = jul,
  version      = {v1},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.21638352},
  url          = {https://doi.org/10.5281/zenodo.21638352},
  note         = {Preprint}
}