renatoschust/gama-agent-lora

GAMA Agent 35B — LoRA adapter

LoRA adapter (r=16, α=16, no dropout) for renatoschust/gama-agent, an agent / tool-calling fine-tune of Qwen/Qwen3.5-35B-A3B for Bosnian (and regional HR/SR/EN). Built by gamAI.

Use this repo to merge the adapter into the base weights, or to continue training. For ready-to-run inference, use the quantised GGUF in renatoschust/gama-agent.

Details

Base model	Qwen/Qwen3.5-35B-A3B
PEFT type	LoRA (fused-expert MoE LoRA)
r / alpha / dropout	16 / 16 / 0
Target	attention + MLP (incl. MoE expert projections), text/language stack
Task	CAUSAL_LM
Trained with	transformers + trl + peft + unsloth

Load / merge (transformers + peft)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3.5-35B-A3B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "renatoschust/gama-agent-lora")
model = model.merge_and_unload()
model.save_pretrained("gama-agent-merged")
tok.save_pretrained("gama-agent-merged")

Files

• adapter_model.safetensors — LoRA weights (~3.7 GB)
• adapter_config.json — PEFT config
• chat_template.jinja, tokenizer*, processor_config.json — prompt template & tokenizer

License

Apache-2.0 (inherited from the base model).

---

Built with ❤️ by gamAI · GitHub