vxltxrllc/vxtral-dustdevil-4.6b-text-llama

VXTRAL-DustDevil-4.6B Text-Llama

vxltxrllc/vxtral-dustdevil-4.6b-text-llama is a custom fork of the Voxtral family — a specialized, standalone text-only language model backbone from Mistral AI's Voxtral-Mini-3B.

This model has been specifically restructured, re-aligned, and finetuned for enhanced tool-calling consistency and instruction-following stability, transforming the architecture into a high-performance pure-text decoder.

By stripping away the speech encoder, this repository delivers a lightweight (~5B parameters) causal LLM mapped directly into standard Llama architecture (LlamaForCausalLM). This is still multimodal decoder, it has all same prior knowledge on acoustical embeddings while being more capable than base model.

Key Highlights

• Enhanced Tool-Calling Consistency: Specially re-aligned and tuned to ensure precise tool selection, strict adherence to expected argument schemas, and the complete elimination of malformed or hallucinated invocations.
• Instruction-Following Stability: Designed for high robustness against conflicting prompts, maintaining strict adherence to system constraints and formatting rules across multi-turn dialogues.
• Standard Llama Compatibility: Fully compatible out-of-the-box with vLLM, llama.cpp, Ollama, SGLang, TGI, and standard transformers pipelines without requiring custom model code, monkey-patching, or multimodal overrides.
• Optimized Footprint: Removal of all non-text weights results in faster cold starts, lower RAM/VRAM consumption, and clean, friction-free quantization pipelines (GGUF, AWQ, GPTQ).
• Core Architecture: Pure Causal LM based on the Transformer architecture utilizing Rotary Position Embeddings (RoPE), SwiGLU activations, and Grouped-Query Attention (GQA).

Model Details

• Developed by: Custom fork, vxltxr LLC // Original base weights by Mistral AI
• Model Type: Causal Language Model (LlamaForCausalLM)
• Parameter Count: ~4.6B
• Context Window: 8,192 tokens
• License: CC 4.0

Quickstart & Usage

1. Using Hugging Face transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "vxltxrllc/voxtral-mini-3b-text-llama"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

prompt = "[INST] Вызови инструмент проверки статуса системы для архитектуры 'arch_secure_01'. [/INST]"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=128,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Fast Serving with vLLM

Because the custom fork maps directly to LlamaForCausalLM, you can launch it with vLLM immediately for high-throughput production serving: Bash

vllm serve vxltxrllc/voxtral-mini-3b-text-llama
--dtype bfloat16
--max-model-len 8192
--port 8000

Custom Fork & Conversion Methodology

The engineering pipeline applied to create this release includes:

Audio-Path Isolation & Stripping: Purging audio encoder blocks (audio_encoder.*), projector layers, and cross-attention keys to isolate the pure text reasoning engine.

Key Mapping & Normalization: Remapping internal proprietary layer naming conventions to standard LlamaForCausalLM structures (model.layers.{i}.self_attn.*, model.embed_tokens, lm_head).

Re-Alignment & Synthesis: Generating an optimized config.json defining architectures as ["LlamaForCausalLM"] with native RoPE theta and GQA parameters matching the extracted shapes, coupled with re-alignment for structured instruction execution.

Intended Use & Limitations Intended Use

Structured multi-turn conversational agents and robust JSON tool-calling integrations.

Foundation for downstream domain-specific fine-tuning (LoRA, QLoRA, DPO).

Edge and local hardware deployment via llama.cpp, GGUF, or vLLM.

Limitations

No Audio Processing: This model is a pure-text decoder fork and cannot accept raw audio/speech inputs. For multimodal tasks, refer to the original mistralai/Voxtral-Mini-3B.

Inherited Nuances: Inherits general language understanding and foundational capabilities from the Mistral AI base lineage, augmented with targeted instruction stability.

    @misc{voxtral_mini_3b_text_llama_2026,
  author = {vxltxr LLC},
  title = {Voxtral-Mini-3B Text-Llama: Custom Fork and Re-Aligned Tool-Calling Decoder},
  year = {2026},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  howpublished = {\url{[https://huggingface.co/vxltxrllc/voxtral-mini-3b-text-llama](https://huggingface.co/vxltxrllc/voxtral-mini-3b-text-llama)}}
}