Qwen3.5-SWE

Qwen3.5-SWE is a SWE focused fine-tune of Qwen/Qwen3.5-4B-Base, trained using supervised fine-tuning (SFT) on NVIDIA's Nemotron-SFT-SWE-v3.5 dataset.

The repository provides two model variants:

• Standard: Fine-tuned with the original tokenizer vocabulary.
• Extended-vocab: Fine-tuned with an expanded vocabulary intended to improve the representation of programming-language keywords, operators, and other code-oriented lexical units.

Both variants are available as merged model weights and include their associated tokenizer files.

Model variants

Property	Standard	Extended-vocab
Base model	Qwen3.5-4B-Base	Qwen3.5-4B-Base
Training method	Supervised fine-tuning	Supervised fine-tuning
Vocabulary configuration	248,320	248,360
Hidden size	2,560	2,560
Transformer layers	32	32
Weight dtype	bfloat16	bfloat16
Maximum positional capacity	262,144	262,144
Merged weights	Available	Available
Fine-tuning adapter	Available	Available

The vocabulary sizes above come from each merged model's config.json. Use the tokenizer shipped with the selected variant.

Evaluation

The following results were obtained by the model author using EvalPlus with vLLM and greedy decoding. Results are reported as pass@1.

Benchmark	Test suite	Standard	Extended-vocab
HumanEval	Base tests	45.7%	52.4%
HumanEval+	Base + extra tests	39.0%	47.0%
MBPP	Base tests	65.3%	64.6%
MBPP+	Base + extra tests	54.5%	53.7%

Repository structure

Qwen3.5-SWE/
├── README.md
├── standard/
│   ├── config.json
│   ├── generation_config.json
│   ├── model.safetensors.index.json
│   ├── model-00001-of-00003.safetensors
│   ├── model-00002-of-00003.safetensors
│   ├── model-00003-of-00003.safetensors
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   ├── chat_template.jinja
│   ├── final_adapter/
│   └── checkpoints/
└── extended-vocab/
    ├── config.json
    ├── generation_config.json
    ├── model.safetensors.index.json
    ├── model-00001-of-00003.safetensors
    ├── model-00002-of-00003.safetensors
    ├── model-00003-of-00003.safetensors
    ├── tokenizer.json
    ├── tokenizer_config.json
    ├── chat_template.jinja
    ├── final_adapter/
    ├── checkpoints/
    └── evaluation/

Training

Base model

Qwen/Qwen3.5-4B-Base

Training dataset

NVIDIA Nemotron-SFT-SWE-v3.5

Dataset license and terms are described in the official dataset card. The dataset card states that the dataset is for commercial uses and identifies CC BY 4.0 alongside additional licenses. Users should review the source dataset's terms and the base model's terms for their intended use.

Training method

Both variants were fine-tuned using supervised fine-tuning.

The standard variant retains the original vocabulary configuration. The extended-vocabulary variant uses a modified tokenizer and corresponding model weights to accommodate the vocabulary expansion.

The repository retains the final adapter artifacts and training checkpoints alongside the merged weights. Detailed per-run hyperparameters, training duration, hardware utilization, and random seeds are not specified in this model card.

Training Notebooks & Experiment Tracking

The notebooks and experiment runs used to fine-tune and evaluate Qwen3.5-SWE are linked below.

Fine-tuning notebook

This notebook contains the supervised fine-tuning (SFT) workflow used for both model variants:

• Standard: Fine-tuning with the original vocabulary.
• Extended-vocab: Fine-tuning with the expanded vocabulary and corresponding tokenizer.

Evaluation notebook

This notebook contains the EvalPlus evaluation workflow used to benchmark the model variants on HumanEval, HumanEval+, MBPP, and MBPP+.

Weights & Biases training runs

View all Qwen3.5-4B SWE training runs on Weights & Biases →

The experiment workspace contains the training runs for both the Standard and Extended-vocab fine-tuning experiments. Use it to explore the available training metrics and run details.

These resources provide the notebooks and experiment-tracking workspace associated with the reported fine-tuning and evaluation results.

Results summary

The extended-vocabulary variant scored higher on HumanEval and HumanEval+, with gains of 6.7 and 8.0 percentage points, respectively. The standard variant scored higher on MBPP and MBPP+, with differences of 0.7 and 0.8 percentage points.

The unweighted average of the four reported scores is 46.38% for Standard and 48.44% for Extended-vocab. This is a descriptive average across the reported metrics, not an official composite benchmark score.

Evaluation setup

• Evaluation framework: EvalPlus.
• Inference backend: vLLM.
• Decoding: Greedy.
• Prompt option: --force-base-prompt.
• Benchmarks: HumanEval, HumanEval+, MBPP, and MBPP+.

Quick start

Install the required libraries in an environment compatible with the model architecture:

pip install torch transformers accelerate

Load either merged variant directly from this repository:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "SamLegion888/Qwen3.5-SWE"
variant = "standard"  # Or "extended-vocab"

tokenizer = AutoTokenizer.from_pretrained(
    repo_id,
    subfolder=variant,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder=variant,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "user",
        "content": "Write a Python function that returns the factorial of a non-negative integer.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
    )

generated_tokens = outputs[0, inputs["input_ids"].shape[1]:]
response = tokenizer.decode(
    generated_tokens,
    skip_special_tokens=True,
)

print(response)

Use the tokenizer and chat template corresponding to the selected variant. The example uses bfloat16 inference and greedy decoding; available memory and supported kernels determine practical inference settings.

Licensing and attribution

The model is derived from Qwen3.5-4B-Base and fine-tuned using NVIDIA Nemotron-SFT-SWE-v3.5.

• Base model: Qwen/Qwen3.5-4B-Base
• Training dataset: nvidia/Nemotron-SFT-SWE-v3.5
• Software-engineering harness referenced by the dataset: OpenCode

Review the applicable base-model, dataset, and third-party component terms before redistribution or deployment. The repository does not currently declare a separate license for the fine-tuned model weights.

Citation

If you use this model in research or evaluation, cite the repository:

@misc{qwen3.5_swe,
  title        = {{Qwen3.5-SWE}},
  author       = {{SamLegion888}},
  month        = {October},
  year         = {2026},
  url          = {https://huggingface.co/SamLegion888/Qwen3.5-SWE}
}

Acknowledgements

Thanks to the Qwen team for the base model, NVIDIA for the Nemotron-SFT-SWE-v3.5 dataset, and the developers of Hugging Face Transformers, TRL, PEFT, vLLM, and EvalPlus.