ASVD-Bridge-Coder-1.5B (Experimental PoC)

This is an experimental Proof-of-Concept (PoC) model utilizing the ASVD-Bridge (Asymmetric Singular Value Decomposition with Subspace Stabilization) architecture, distilled from Qwen/Qwen2.5-Coder-1.5B.

⚠️ WARNING: EXPERIMENTAL PO-C ⚠️ This specific checkpoint was distilled for a short 625-step schedule. While the physical architectural footprint (VRAM/TPS) is perfectly maintained, the logical reasoning and syntax generation capabilities are heavily degraded in this specific release due to the short distillation cycle.

🚀 Extreme Hardware Telemetry

By mapping a dynamic rank SVD across the Attention blocks and a Fake-INT4 physical packing across the MLP blocks, this model achieves unprecedented compression and execution speeds:

• Peak VRAM Footprint: 1.03 GB (Easily fits on the smallest consumer GPUs or edge devices)
• Inference Throughput: 38.39 TPS (Tokens Per Second)
• Architecture: Zero-Overhead Folded (No custom QTensor wrappers needed at runtime, provided the custom topology loader is used)

🧠 ASVD-Bridge Topology

This model leverages a hybrid asymmetric topology:

• Attention Blocks ($q, k, v, o$): Dynamic Rank Singular Value Decomposition (SVD), achieving deep linear parameter reduction.
• MLP Blocks ($gate, up, down$): Activation-Aware Weight Quantization (AWQ) physical 4-bit simulation.
• Subspace Bridge ($\gamma$): A specialized healing scalar trained via Quantization-Aware Distillation (QAD) to absorb the massive numerical truncations, folded directly into the $o_proj$ weights at export to eliminate runtime overhead.

💻 Usage

To load this model, you cannot use a standard dense causal model auto-loader. You must initialize the SVD+INT4 QTensor blank topology before loading the .safetensors.

Dependencies: You must have the QTensor Engine codebase locally to access the topology mappings (qtensor_core.py).

import torch
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
from accelerate import load_checkpoint_in_model
from huggingface_hub import hf_hub_download

# Requires cloning: https://github.com/trentzap/qtensor-engine
from qtensor_core import apply_coder_compression_blank 

model_path = "trentzap/ASVD-Bridge-Coder-1.5B"

# 1. Download the dynamic rank topology config directly from this repo
config_path = hf_hub_download(repo_id=model_path, filename="asvd_coder_1.5b.yaml")

tokenizer = AutoTokenizer.from_pretrained(model_path)
config = AutoConfig.from_pretrained(model_path)
model = AutoModelForCausalLM.from_config(config, torch_dtype=torch.bfloat16)

# 2. Initialize ASVD-Bridge topological mapping
model = apply_coder_compression_blank(model, config_path)

# 3. Load safetensors precisely into the factored blocks
load_checkpoint_in_model(model, model_path)
model = model.to("cuda")

Authors

Trent Ian Parsons (QTensor)