buildrestart/Gitallama

Llama 3.1 8B — Bhagavad Gita Guide

A QLoRA fine-tune of Meta-Llama-3.1-8B-Instruct trained to answer questions about the Bhagavad Gita — its verses, philosophy, characters, and teachings.

Model Details

Property	Value
Base model	meta-llama/Meta-Llama-3.1-8B-Instruct
Fine-tune method	QLoRA (4-bit NF4 + LoRA) via Unsloth
LoRA rank / alpha	r=32, α=64 (RSLoRA)
LoRA target modules	q, k, v, o, gate, up, down projections
Adapter size	~321 MB
Max sequence length	2048
Precision	bfloat16 + TF32

Training Details

Property	Value
Epochs	3
Train examples	3,801
Validation examples	422
Effective batch size	512 (8 × grad_accum 64)
Learning rate	2e-4 (cosine, 5% warmup)
Optimizer	AdamW 8-bit
Train loss	1.277
Eval loss	0.924

Usage

Load the adapter (recommended — memory efficient)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"
adapter_id    = "buildrestart/Gitallama"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    load_in_4bit=True,
)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

Generate a response

SYSTEM_PROMPT = (
    "You are a knowledgeable guide on the Bhagavad Gita, the ancient Hindu scripture "
    "containing the dialogue between Prince Arjuna and Lord Krishna. Provide thoughtful, "
    "accurate explanations of its verses, teachings, and philosophical concepts."
)

messages = [
    {"role": "system",  "content": SYSTEM_PROMPT},
    {"role": "user",    "content": "What does the Bhagavad Gita teach about karma?"},
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=500,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )

response = tokenizer.decode(
    output[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(response)

Using Unsloth (2× faster inference)

import torch
from unsloth import FastLanguageModel
from peft import PeftModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/meta-llama-3.1-8b-instruct-bnb-4bit",
    max_seq_length=2048,
    dtype=torch.bfloat16,
    load_in_4bit=True,
)
model = PeftModel.from_pretrained(model, "buildrestart/Gitallama")
model = model.merge_and_unload()
FastLanguageModel.for_inference(model)

Example Outputs

Q: What is the central message of the Bhagavad Gita?

The central message of the Bhagavad Gita is the path of self-realization and liberation through the practice of yoga, which is often interpreted as a path of devotion, knowledge, and action. The Gita emphasizes the importance of fulfilling one's duties (dharma) without attachment to the outcomes, and it teaches the concept of non-attachment (vairagya) to worldly possessions and desires...

Q: Why was Arjuna reluctant to fight at Kurukshetra?

Arjuna was reluctant to fight because he was torn between his duty as a warrior (Kshatriya) and his personal feelings of compassion and non-violence. He was worried about killing his kin and friends, and he questioned the purpose of war. Krishna addressed Arjuna's doubts by explaining the concept of dharma (duty) and the inevitability of change...

Limitations

• Trained on ~4K examples; may give brief answers on less common topics.
• Knowledge is limited to the Bhagavad Gita — not a general Hindu scripture expert.
• As with all LLMs, outputs should be verified against authoritative translations for scholarly use.

License

This adapter inherits the Llama 3.1 Community License. The training data is sourced from public domain and permissively licensed HuggingFace datasets.