Rcscan/Qwen3.5-2.8B-V0.1

Qwen3.5-2.8B v0.1 — Pruned + QLoRA

Qwen3.5-2.8B v0.1 is an early experimental release of a compressed Qwen3.5 model.

The original model was approximately 4B parameters. Through structural pruning, the model was reduced to approximately 2.8B parameters, followed by QLoRA fine-tuning to recover capabilities lost during pruning.

The main goal of this project is to explore how much capability can be retained after significantly reducing the original model.

This v0.1 release is only the beginning. It should be considered a baseline for future iterations rather than a final model.

---

Highlights

• 🧠 Based on Qwen3.5
• ✂️ Reduced from 32 → 20 transformer layers
• 📉 Approximately 2.8B parameters
• 🔧 QLoRA fine-tuning
• 💾 4-bit quantized release
• 🇧🇷 Strong focus on Brazilian Portuguese
• 🇺🇸 English is intentionally retained in the training mix
• 💭 Reasoning/thinking data is kept in its original language
• 🖥️ Designed for local inference
• 🧪 Experimental v0.1 baseline

---

Why This Model Exists

Large language models contain a considerable amount of redundancy.

This project explores whether a model can be made substantially smaller by removing transformer layers, while still recovering useful capabilities through parameter-efficient fine-tuning.

The basic process is:

Qwen3.5 → structural pruning → ~2.8B model → QLoRA recovery → quantized local model

The objective is not simply to create another small language model.

The objective is to investigate how much of a larger model can be preserved after structural compression.

---

Version 0.1

This release should be considered the base of the project.

There has not been an attempt to aggressively optimize the model for a single benchmark, language, or specialized task.

Instead, v0.1 focuses on establishing a usable compressed model that can serve as a foundation for future training.

Future versions may experiment with:

• additional datasets
• broader multilingual interaction
• improved reasoning
• tool use
• improved recovery of capabilities lost during pruning
• different pruning strategies
• different LoRA configurations
• improved quantization

The training process is therefore expected to evolve considerably in later releases.

---

Language and Reasoning

The model has a strong focus on Brazilian Portuguese, but the training process deliberately does not force all interaction into Portuguese.

Both Portuguese and English are used during training and interaction.

This is intentional.

The goal is to avoid unnecessarily narrowing the linguistic behavior of the model and to preserve some of the diversity inherited from the original model.

Reasoning / Thinking

The model's reasoning data is not translated into Portuguese.

Thinking/reasoning examples are kept in their original language whenever possible, while the model is still encouraged to interact naturally in Portuguese or English.

This means that the language used for internal reasoning and the language used in the final response are not necessarily the same.

This behavior is intentional rather than an artifact of the training process.

---

Model Details

Property	Value
Model family	Qwen3.5
Base model	aldair166/qwen3.5-2.8b
Original size	~4B parameters
Pruned size	~2.8B parameters
Original layers	32
Pruned layers	20
Compression	~30% smaller
Fine-tuning	QLoRA
Quantization	4-bit
Primary language	Portuguese (Brazil)
Secondary language	English
Release	v0.1
License	Apache 2.0

---

Architecture

The compressed model retains the hybrid-attention architecture of Qwen3.5.

Property	Value
architectures	Qwen3_5ForCausalLM
hidden_size	2560
num_hidden_layers	20
num_attention_heads	16
num_key_value_heads	4
head_dim	256
intermediate_size	9216
hidden_act	SiLU
vocab_size	248320
max_position_embeddings	262144
rope_theta	10000000
partial_rotary_factor	0.25
tie_word_embeddings	true
dtype	bfloat16

The architecture mixes linear-attention and full-attention layers.

---

QLoRA

After pruning, the model loses a substantial amount of its original capability.

Instead of restoring the model by fully retraining all parameters, this project uses QLoRA to adapt the compressed model.

Adapter configuration

Parameter	Value
LoRA rank (r)	64
LoRA alpha	128
LoRA dropout	0.05
Trainable parameters	~90M
Base parameters	~2.8B

The adapter is intended to recover useful capabilities from the compressed model while keeping the training cost comparatively low.

---

Training Data

The training dataset is available at:

aldair166/train

The dataset is intentionally varied rather than being dedicated to a single narrow task.

The model is trained on conversational and instructional examples covering multiple types of interaction.

Portuguese is a major part of the dataset, while English examples are also included to maintain linguistic diversity.

Some examples contain explicit reasoning/thinking content. These examples are not translated simply for the sake of making everything Portuguese.

The intention is to allow the model to retain a more natural multilingual behavior.

---

Quantization

The released model is provided in 4-bit form for practical local inference.

A GGUF release can be used with compatible runtimes such as llama.cpp.

4-bit quantization significantly reduces the memory required to load the model compared with higher-precision weights.

Actual memory usage depends on the runtime, context length, KV-cache configuration, and hardware.

---

llama.cpp

Example:

llama-server \
  -m qwen3.5-2.8b.gguf \
  --lora qwen3.5-2.8b-lora.gguf \
  -c 64000 \
  -ngl 99 \
  -ctk q4_0 \
  -ctv q4_0 \
  -fa on \
  --host 0.0.0.0 \
  --port 8080

Adjust the configuration for your hardware and desired context length.

---

Transformers + QLoRA

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "aldair166/qwen3.5-2.8b"
adapter_id = "YOUR_ADAPTER_REPOSITORY"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(base_id)

model = AutoModelForCausalLM.from_pretrained(
    base_id,
    quantization_config=bnb_config,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    model,
    adapter_id,
)

prompt = "Explique a diferença entre atenção linear e atenção completa."

inputs = tokenizer(
    prompt,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Intended Use

This model is intended for:

• local inference
• experimentation with small language models
• research into model pruning
• QLoRA experiments
• quantization experiments
• Portuguese/English conversational AI
• experimentation on hardware with limited VRAM

It can also serve as a starting point for additional fine-tuning.

---

Limitations

This is an early experimental release.

The model should not be expected to match the capabilities of the original Qwen3.5 model.

The main limitations include:

• quality loss caused by structural pruning
• possible additional degradation from 4-bit quantization
• incomplete validation of very long contexts
• uneven performance across languages and tasks
• experimental training and dataset composition
• lack of production-oriented safety tuning

The 262144 token context limit is inherited from the architecture/configuration and should not be interpreted as proof that the compressed and quantized model performs equally well at that context length.

---

Project Direction

v0.1 establishes the initial compressed model.

The broader project is exploring the relationship between:

model size → pruning → capability loss → QLoRA recovery

The long-term goal is to determine how far a model can be compressed while remaining genuinely useful.

Rather than optimizing only for parameter count, future versions will focus on finding a practical balance between:

size, speed, memory usage, reasoning ability, language quality, and general usefulness.

---

Credits

This project is based on the Qwen3.5 model family.

Please credit the original Qwen authors when using this derivative model.

Base compressed model:

aldair166/qwen3.5-2.8b

Training dataset:

aldair166/train

---

Citation

@misc{aldair166_qwen35_28b_v01,
  title  = {Qwen3.5-2.8B v0.1},
  author = {aldair166},
  year   = {2026},
  url    = {https://huggingface.co/aldair166/qwen3.5-2.8b}
}

When using the underlying Qwen3.5 model, please also cite the original Qwen project.

---

Disclaimer

This is an independent experimental derivative model and is not an official release from the Qwen team.

This v0.1 release is provided for research, experimentation, and local inference.