Instructions to use sascha-frank-ai-research/tsft-rag-gemma-3-4b-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sascha-frank-ai-research/tsft-rag-gemma-3-4b-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sascha-frank-ai-research/tsft-rag-gemma-3-4b-it")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("sascha-frank-ai-research/tsft-rag-gemma-3-4b-it")
model = AutoModelForMultimodalLM.from_pretrained("sascha-frank-ai-research/tsft-rag-gemma-3-4b-it", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sascha-frank-ai-research/tsft-rag-gemma-3-4b-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sascha-frank-ai-research/tsft-rag-gemma-3-4b-it

SGLang

How to use sascha-frank-ai-research/tsft-rag-gemma-3-4b-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sascha-frank-ai-research/tsft-rag-gemma-3-4b-it with Docker Model Runner:
```
docker model run hf.co/sascha-frank-ai-research/tsft-rag-gemma-3-4b-it
```

TSFT-RAG Gemma-3-4B-IT

Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation

Part of the TSFT-RAG model family for Retrieval-Augmented Generation across Gemma, Qwen and Llama architectures.

This repository contains TSFT-RAG Gemma-3-4B-IT, a full-parameter fine-tuned derivative of Google's Gemma-3-4B-IT developed within the TSFT-RAG research project. The objective is to investigate how task-specific full fine-tuning improves Retrieval-Augmented Generation (RAG), grounded question answering, unsupported-question detection and structured information extraction.

TSFT-RAG Model Series

Model	Parameters	Status
Gemma-3-1B-IT	1B	Released
Gemma-3-4B-IT	4B	Released
Gemma-3-12B-IT	12B	Released

Abstract

Large instruction-tuned language models perform well in general dialogue but are not specifically optimized for Retrieval-Augmented Generation. TSFT-RAG investigates whether full supervised fine-tuning can specialize foundation models for RAG-oriented tasks without modifying the underlying architecture. This model represents the 4B-parameter member of the TSFT-RAG Gemma series.

Research Background

Reliable RAG systems require more than fluent text generation. They must:

answer strictly from retrieved evidence,
reject unsupported questions,
produce structured outputs,
remain deterministic and reproducible.

The TSFT-RAG project evaluates these capabilities across Gemma, Qwen and Llama model families using identical datasets and evaluation procedures.

Model Details

Property	Value
Base model	google/gemma-3-4b-it
Training implementation	unsloth/gemma-3-4b-it
Architecture	Gemma3ForConditionalGeneration
Parameters	~4 Billion
Hidden size	2560
Transformer layers	34
Context window	131,072 tokens
Training length	1,024 tokens
Precision	bfloat16
Fine-tuning	Full parameter

Training Dataset

The TSFT-RAG dataset contains supervised examples for:

Grounded Question Answering
Unsupported Question Detection
Topic Extraction
Key Message Extraction
Argument Extraction
Structured JSON Generation

The same dataset and benchmark are used across all TSFT-RAG models to enable reproducible comparisons.

Training Procedure

Hyperparameter	Value
Epochs	3
Learning Rate	5e-7
Batch Size	4
Gradient Accumulation	4
Optimizer	AdamW
Scheduler	Cosine
Warmup Ratio	0.08
Weight Decay	0.01
Max Gradient Norm	1.0
Sequence Length	1024
BF16	Yes
Seed	42

Training runtime: 61.8 minutes

Final training loss: 1.3081

Evaluation Results

Metric	Base	TSFT-RAG	Improvement
Aggregate Score	0.412	0.610	+0.198
Hard Negative QA	0.127	0.694	+0.566
Grounded QA	0.567	0.532	-0.035
Topic Extraction	0.624	0.615	-0.009
Key Message	0.599	0.584	-0.016
Argument Extraction	0.504	0.518	+0.014
JSON Validity	0.856	0.959	+0.102
Citation Precision	0.000	0.637	+0.637
Citation Recall	0.000	0.637	+0.637

Discussion

Compared with the 1B model, the 4B variant achieves a noticeably better balance between reliable abstention, grounded reasoning and structured generation. The most significant improvements are observed for unsupported-question detection, JSON validity and citation behaviour, while maintaining nearly the same performance on grounded question answering and analytical extraction tasks.

Rather than maximizing a single benchmark metric, TSFT-RAG aims to optimize the overall behaviour of language models within Retrieval-Augmented Generation pipelines.

Intended Use

Recommended for:

Retrieval-Augmented Generation
Enterprise knowledge assistants
University information systems
Research assistants
Structured information extraction

Not intended as a general-purpose conversational model.

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
    "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it"
)

model = AutoModelForCausalLM.from_pretrained(
    "sascha-frank-ai-research/tsft-rag-gemma-3-4b-it"
)

Reproducibility

Training was performed using the Hugging Face Transformers ecosystem together with the Unsloth implementation for full-parameter supervised fine-tuning. All relevant hyperparameters, tokenizer configuration and evaluation summaries are included in this repository.

Limitations

Optimized specifically for RAG workflows.
General conversational performance was not the primary optimization objective.
Evaluated primarily on German-language benchmark datasets.
No multimodal benchmark has been conducted.

Ethical Considerations

This model should be used together with appropriate source attribution and human oversight in high-impact applications. It should not replace expert judgement in legal, medical or safety-critical domains.

License

This repository contains a modified derivative of Google's Gemma model.

Please refer to the accompanying LICENSE, NOTICE and MODIFICATIONS.md files.

Associated Publication

This model was developed and evaluated as part of the following research:

Frank, S., & Singh, R. (2026). Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation: A Multi-Family Evaluation Across Open-Weight Language Models.

Zenodo. DOI: 10.5281/zenodo.21638352

If you use this model in academic work, please cite the publication above.

@misc{Frank2026TSFTRAG,
  author       = {Frank, Sascha and Singh, Rawel},
  title        = {Task-Specific Full Fine-Tuning for Retrieval-Augmented Generation:
                  A Multi-Family Evaluation Across Open-Weight Language Models},
  year         = {2026},
  month        = jul,
  version      = {v1},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.21638352},
  url          = {https://doi.org/10.5281/zenodo.21638352},
  note         = {Preprint}
}