Instructions to use brunogonzaga/pinhao-gemma4-rag-adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use brunogonzaga/pinhao-gemma4-rag-adapter with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("google/gemma-4-E4B-it")
model = PeftModel.from_pretrained(base_model, "brunogonzaga/pinhao-gemma4-rag-adapter")

Transformers

How to use brunogonzaga/pinhao-gemma4-rag-adapter with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="brunogonzaga/pinhao-gemma4-rag-adapter")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("brunogonzaga/pinhao-gemma4-rag-adapter", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use brunogonzaga/pinhao-gemma4-rag-adapter with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "brunogonzaga/pinhao-gemma4-rag-adapter"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brunogonzaga/pinhao-gemma4-rag-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/brunogonzaga/pinhao-gemma4-rag-adapter

SGLang

How to use brunogonzaga/pinhao-gemma4-rag-adapter with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "brunogonzaga/pinhao-gemma4-rag-adapter" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brunogonzaga/pinhao-gemma4-rag-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "brunogonzaga/pinhao-gemma4-rag-adapter" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brunogonzaga/pinhao-gemma4-rag-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use brunogonzaga/pinhao-gemma4-rag-adapter with Docker Model Runner:
```
docker model run hf.co/brunogonzaga/pinhao-gemma4-rag-adapter
```

Pinhao Gemma 4 RAG Adapter

Public LoRA/PEFT adapter for google/gemma-4-E4B-it.

This is a learning experiment for answer behavior in a local Obsidian RAG workflow. It is not a memory store and does not contain private vault notes.

Intended Use

Use this adapter with local RAG context to encourage:

short PT-BR answers;
citations in [S#] format;
exact refusal when context is insufficient;
fewer invented citations;
cleaner answer format for a second-brain assistant.

The knowledge source is still the local RAG system. The adapter only shapes behavior and output format.

Base Model

Base: google/gemma-4-E4B-it
Adapter format: LoRA / PEFT
Task style: chat/text generation
Serving target: OpenAI-compatible endpoint with output cleanup

Use of this adapter still requires access to, and compliance with, the base Gemma model terms.

Training Data

Training used only synthetic data generated for the lab:

data/gemma4-rag-contract-train.jsonl: 160 synthetic chat rows
data/gemma4-rag-contract-eval.jsonl: 40 synthetic holdout rows

The dataset contains fake project names, fake source paths, fake decisions, and fake next steps. It intentionally excludes real Obsidian notes, SQLite indexes, embeddings, personal data, secrets, receipts, CPF, email, phone numbers, and private vault content.

Training Run

Date: 2026-06-16
Runner: RunPod PyTorch pod
GPU: A40 48 GB
Runtime after setup/cache: 118.7s
Steps: 20
Final reported train loss: 1.341
Stack: torch 2.11.0+cu128, transformers 5.12.1, TRL 1.6.0, PEFT 0.19.1

LoRA targets were limited to model.language_model.* modules to avoid multimodal wrapper incompatibilities.

Evaluation

Holdout eval: 40 synthetic RAG contract rows.

Evaluation Snapshot

Scores are counts over 40 synthetic holdout rows. The adapter changes response behavior, not vault memory.

Variant	Answer contract	Raw strict	Served strict after cleanup	Exact refusal	Expected citations	No invented citations
base	18/40	14/40	18/40	33/40	27/40	40/40
adapter	37/40	20/40	37/40	40/40	37/40	40/40

Interpretation:

the adapter improved answer-contract behavior from 45% to 92.5%;
exact refusal improved to 40/40;
no invented citations were observed in the holdout;
raw generation can still leak markers such as model, thought, or trailing refusal fragments;
serving must keep cleanup and token caps in the response path.

Serving Smoke

The adapter was served in an interactive RunPod pod with:

GET /health
POST /v1/chat/completions
local SSH tunnel from 127.0.0.1:18000 to pod 127.0.0.1:8000

End-to-end smoke:

local rag_ask.py -> local retrieval -> SSH tunnel -> RunPod adapter server

Final smoke result:

local retrieval stayed local;
only selected prompt context crossed the tunnel;
response was PT-BR with [S1] and [S2] citations;
server was stopped after the smoke;
GPU returned idle.

Example

Question:

o que aconteceu no serving smoke do adapter Pinhao?

Answer:

O serving smoke do adapter Pinhao validou um servidor HTTP mínimo [S1].

*   O servidor respondeu `ok` ao `/health` e teve uma resposta limpa no POST único [S1, S2].
*   O mini-eval HTTP obteve 5/5 no contrato estrito e 5/5 no formato limpo [S1, S2].
*   Não houve envio de vault, SQLite, embeddings ou nota real ao pod [S1].
*   O servidor foi parado após o smoke, e a GPU voltou ao estado idle [S1, S2].

Limitations

This adapter does not store private knowledge.
It should not be used without retrieval context for factual answers.
It still needs serving cleanup and a conservative token cap.
It was evaluated on a small synthetic holdout, not on broad real-world tasks.
It is not production-ready without additional evals and monitoring.

Minimal Loading Sketch

from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForMultimodalLM, BitsAndBytesConfig
import torch

base_id = "google/gemma-4-E4B-it"
adapter_id = "brunogonzaga/pinhao-gemma4-rag-adapter"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, use_fast=False)
quantization = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
base = AutoModelForMultimodalLM.from_pretrained(
    base_id,
    device_map="auto",
    quantization_config=quantization,
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

Privacy

This repository is public and contains adapter artifacts plus documentation only. It was trained on synthetic data and does not include Obsidian vault notes, SQLite indexes, embeddings, secrets, or real personal data. The intended production workflow should still keep the actual vault and retrieval index local/private.

Downloads last month: 31

Model tree for brunogonzaga/pinhao-gemma4-rag-adapter

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Adapter

(133)

this model