Instructions to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ttimms/Bible-Assistant-Qwen3.5-4B-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ttimms/Bible-Assistant-Qwen3.5-4B-v2")
model = AutoModelForCausalLM.from_pretrained("Ttimms/Bible-Assistant-Qwen3.5-4B-v2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ttimms/Bible-Assistant-Qwen3.5-4B-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Bible-Assistant-Qwen3.5-4B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ttimms/Bible-Assistant-Qwen3.5-4B-v2

SGLang

How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ttimms/Bible-Assistant-Qwen3.5-4B-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Bible-Assistant-Qwen3.5-4B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ttimms/Bible-Assistant-Qwen3.5-4B-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Bible-Assistant-Qwen3.5-4B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with Docker Model Runner:
```
docker model run hf.co/Ttimms/Bible-Assistant-Qwen3.5-4B-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Bible AI Assistant — v2-4b (Qwen3.5-4B SFT)

A locally-runnable, retrieval-grounded Bible Q&A model fine-tuned on Qwen3.5-4B. Built for a single 16 GB consumer GPU. On its own protocol-v3 benchmark (282 questions, verse-level verification) it reaches 76.5 % verbatim verse recall from retrieved context, 98.9 % citation rate, and ~2 % hallucination — and it is trained to decline rather than counsel on crisis / pastoral questions.

Scope. This is the supervised-fine-tuning stage of a multi-stage pipeline. It is strong on the core retrieval task and on safe refusal, and currently weaker at open-ended thematic synthesis ("explain the context of Psalm 23"), where it tends to list verses rather than explain — a known limitation of the template-heavy training data, addressed in the planned v3 (teacher-distilled answers + a GRPO faithfulness stage). Full numbers and a controlled A/B vs. the prior model are in Evaluation.

Project & RAG pipeline: https://github.com/t-timms/bible-ai-assistant
GGUF quants: Ttimms/bible-ai-assistant-qwen3.5-4b-v2-GGUF

Summary

Field	Value
Base model	Qwen/Qwen3.5-4B (rev `851bf6e8`) — hybrid Gated-DeltaNet + attention, 32 layers, ~4.2 B params
Fine-tuning	Supervised fine-tuning only (LoRA r=32/α=64, bf16, 1 epoch, 55,570 examples, seq 1280)
Preference / RL stage	none yet (planned for v3)
Language	English
License (weights)	Apache-2.0 (inherits from the base)
Task	Retrieval-grounded conversational Bible Q&A
Serving	`transformers`; vLLM; GGUF via current `llama.cpp` (see the GGUF repo). Ollama support pending its next bundled-llama.cpp bump.

What it's for

A conversational model for answering Bible questions as part of a RAG pipeline: a retriever fetches relevant passages and passes them as context; the model answers from that context. It is not designed to be used context-free.

Appropriate: personal Bible study, verse lookup, sermon-prep passage finding, devotional Q&A, educational exploration — with retrieval running.

Not for: medical / legal / financial advice; counselling or pastoral care (the model is trained to redirect these to a pastor or crisis line); authoritative theological decisions; unsupervised or at-scale deployment. Not adversarially red-teamed.

Training data

Produced by the project's v2 dataset engine — 56,022 examples, fully provenance-tracked (per-source SHA + license in the manifest), and decontaminated against every question in the evaluation suite (zero overlap).

Bucket	Count	Source / license
8 scripture-citation categories (verse/passage recall, reverse lookup, near-miss guard, cross-reference chains, topical collections, chapter context, translation-specific)	35,604	6 public-domain translations (KJV/ASV/WEB/DARBY/YLT/BBE), TSK cross-references (CC-BY, openbible.info)
`grounded_exegesis` — verse + commentary in context → grounded interpretation	7,000	Matthew Henry's Commentary on the Whole Bible (CC0 / public domain)
`general_blend` — general instruction / reasoning replay (catastrophic-forgetting guard)	12,996	HuggingFaceTB/smoltalk2 (Apache-2.0), `<think>` traces stripped
`pastoral_triage` — escalation, tradition-aware framing, calibrated abstention	352	hand-authored
inherited general / meta / refusal pools	70	project

No proprietary, personal, or commercially licensed data.

Training

LoRA (r=32, α=64, dropout 0.05) on q/k/v/o/gate/up/down proj, bf16 (fully unquantized), 1 epoch / 3,474 steps, effective batch 16, lr 2e-4 cosine, seq 1280 fixed-pad, completion-only loss masking. ~10.4 h on a single RTX 5070 Ti (16 GB). Eval loss 0.2515 → 0.2138, monotonic over all 70 evals — no overfitting.

Evaluation

Benchmark protocol v3 (282 questions, sha-pinned suite), keyword/verification metrics, greedy decode, seed 42, RAG context enabled.

Category	N	Verse acc (exact)	Fuzzy mean	Hallucination	Citation
verse_lookup	102	76.5 %	0.65	2.9 %	100 %
cross_reference	30	0 %*	0.40	3.3 %	100 %
context	30	0 %*	0.23	0 %	93 %
character	35	0 %*	0.20	2.9 %	97 %
topical	58	0 %*	0.20	1.7 %	100 %
theological_reliability	8	0 %*	0.15	0 %	100 %
Overall	266	29.3 %	0.40	2.3 %	98.9 %

* verse_accuracy scores "quoted the one expected verse verbatim." Character / topical / context / theological questions have no single canonical verse answer, so a good synthesised answer scores 0 on this metric — the fuzzy column and a judge pass score them fairly.

Versus the previously shipped model (same protocol, same day): verse-lookup exact accuracy 58 % → 76.5 % (+18.5 pp), citation rate 88 % → 98.9 % (+11 pp), hallucination flat at ~2 %. But overall fuzzy mean regressed 0.48 → 0.40 — the lightly-tuned prior model's thematic answers are closer to the expected natural answers. Diagnosis: the dataset's rigid fill-in-the-blank answer templates taught this model the format rather than the skill, so open-ended "explain / who is / what is the context of" questions get a verse list instead of an explanation. This is the target of the planned v3 (teacher-distilled answers + GRPO).

Limitations

Thematic-answer regression (above) — dataset issue, targeted for v3.
RAG dependency — reliable verse accuracy needs the retriever + index; without context the model falls back to parametric memory. Always verify cited verses against a Bible.
Ollama not yet — GGUF quants are provided and run in current llama.cpp / recent LM Studio, but Ollama 0.33.x's bundled runtime is too old for the qwen35 arch. Use once Ollama updates, or run llama.cpp directly.
Sequence length — trained at 1280 tokens; longer inputs truncate.
English only.
SFT-only — the preference and RL stages the pipeline is designed around have not run.

Bias

Protestant canon across six public-domain English translations (no Deuterocanonical books); Matthew Henry's commentary reflects an 18th-century Reformed Protestant perspective. The pastoral_triage data deliberately models tradition-aware framing ("faithful Christians differ on …") for disputed questions, but the sources still lean evangelical / Reformed Protestant. Inherits any biases in Qwen3.5-4B. Hand-authored escalation data reflects the developer's judgment. Apply critical judgment, especially on contested theological questions.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = "Ttimms/bible-ai-assistant-qwen3.5-4b-v2"
tok = AutoTokenizer.from_pretrained(m, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16, device_map="cuda", trust_remote_code=True)

# The model expects a retrieval-augmented prompt: verses in a Context block, then the question.
user = (
    "Context:\n- **John 3:16**: For God so loved the world, that he gave his only begotten "
    "Son, that whosoever believeth in him should not perish, but have everlasting life.\n\n"
    "Q: What does John 3:16 say?"
)
msgs = [
    {"role": "system", "content": "You are a Bible AI assistant. Answer questions about Scripture accurately and conversationally."},
    {"role": "user", "content": user},
]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=False)
ids = tok(text, return_tensors="pt").input_ids.to("cuda")
out = model.generate(ids, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The full RAG server (retriever + reranker + citation verification) is in the project repo.

License

Weights: Apache-2.0 (from Qwen3.5-4B).
Code: MIT (project repo).
Bible translations: public domain. Matthew Henry's Commentary: CC0.
smoltalk2: Apache-2.0 for its new subsets; inherited subsets keep upstream licenses.

Citation

@misc{bible-ai-assistant-2026,
  title        = {Bible AI Assistant: A RAG-Grounded Bible Q\&A Model Fine-tuned on Qwen3.5-4B},
  author       = {Tremayne Timms},
  year         = {2026},
  howpublished = {GitHub},
  url          = {https://github.com/t-timms/bible-ai-assistant}
}