trentzap/Llama-3.2-1B-QTensor-FP8

QTensor: LLaMA-3.2 1B Instruct (Stacked MPO + FP8 Quantization)

Model Developer: Trent Ian Parsons (GitHub Repository)
Base Model: unsloth/Llama-3.2-1B-Instruct
Precision: Stacked MPO (chi=256) + FP8 (float8_e4m3fn) Core + 3.65% bfloat16 LoRA Adapters
VRAM Memory Footprint: 694 MB (0.68 GB) — 3.39x Memory Reduction vs Pristine FP16 (2.30 GB)

---

Performance Metrics

• Pristine LLaMA-3.2 1B (bf16): 2357.13 MB (2.30 GB)
• QTensor Stacked MPO + FP8: 694.63 MB (0.68 GB)
• WikiText-2 Perplexity: 11.78 PPL (+0.07 vs Pristine Baseline)
• GSM8K Zero-Shot Accuracy: 75.2% (Healed)

---

Quickstart Inference with QTensor

import torch
import qtensor
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "trentzap/Llama-3.2-1B-QTensor-FP8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")

# Activate QTensor FP8 Triton SRAM Fusion
model = qtensor.compress(model, precision="fp8", chi=256, use_triton=True)

messages = [{"role": "user", "content": "Explain QTensor SRAM fusion."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=50)

print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

---

Citation

@article{parsons2026qtensor,
  title={Quantum-Inspired Precision: Preserving Low-Rank Reasoning Structures in Large Language Models via 160-Bit Fixed-Point Tensor Decomposition, Triton SRAM Fusion, and Stacked Quantization},
  author={Parsons, Trent Ian},
  journal={arXiv preprint arXiv:2608.xxxxx},
  year={2026}
}