Qwen3-4B-Computer-Science

Qwen3-4B-Computer-Science is a supervised fine-tuned language model based on Qwen/Qwen3-4B, designed for computer science and software engineering tasks.

This repository contains the merged BF16 checkpoint compatible with the Hugging Face Transformers ecosystem.


Model Summary

The model specializes in programming-oriented instruction following across multiple computer science domains, including software engineering, debugging, algorithms, testing, and technical reasoning.

Training was performed using parameter-efficient supervised fine-tuning (LoRA). The released checkpoint contains merged BF16 weights and can be used directly without PEFT adapters.


Motivation

General-purpose language models provide strong performance across many domains but are not specifically optimized for computer science workflows.

Qwen3-4B-Computer-Science aims to improve programming-oriented instruction following while preserving the capabilities of the original Qwen3-4B base model.


Model Details

Field Value
Model Name Qwen3-4B-Computer-Science
Base Model Qwen/Qwen3-4B
Model Type Causal Language Model
Architecture Decoder-only Transformer
Parameters 4 Billion
Fine-Tuning Supervised Fine-Tuning (SFT)
Fine-Tuning Method LoRA
Training Strategy Distributed Data Parallel (DDP)
Released Weights Merged BF16
Framework Hugging Face Transformers
Primary Language English

Training

Training was performed using supervised fine-tuning (SFT) with parameter-efficient fine-tuning (LoRA).

Optimization utilized Distributed Data Parallel (DDP). After training, the LoRA adapters were merged into the base model to produce the released BF16 checkpoint.

The published model does not require PEFT adapters during inference.


Training Data

The final training corpus contains 60,989 training examples and 512 evaluation examples.

Dataset Configuration License Train Eval
HuggingFaceTB/smoltalk smol-magpie-ultra Apache-2.0 49,584 416
agentica-org/DeepCoder-Preview-Dataset primeintellect MIT 11,405 96

Dataset Attribution

The model was fine-tuned using publicly available datasets released under their respective licenses.

Dataset Configuration License
HuggingFaceTB/smoltalk smol-magpie-ultra Apache-2.0
agentica-org/DeepCoder-Preview-Dataset primeintellect MIT

Credit for the datasets belongs to their respective authors.


Intended Use

Recommended applications include:

  • Software engineering
  • Programming assistance
  • Python development
  • Code generation
  • Code explanation
  • Debugging
  • Unit testing
  • Technical documentation
  • Computer science education

Capabilities

The model has been fine-tuned for:

  • Programming-oriented instruction following
  • Code generation
  • Code completion
  • Code explanation
  • Refactoring
  • Debugging
  • Algorithm implementation
  • Standard library usage
  • Technical reasoning

The model inherits the general instruction-following capabilities of Qwen3-4B.


Installation

pip install -U transformers accelerate torch

Usage

from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM

model_name = "Irfanuruchi/Qwen3-4B-Computer-Science"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

Example

messages = [
    {
        "role": "user",
        "content": "Implement binary search in Python."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware Requirements

This repository contains merged BF16 weights.

Memory requirements depend on the selected precision and inference backend.

Users with limited GPU memory are encouraged to use the GGUF release when available.


Limitations

Although specialized for computer science tasks, the model remains a probabilistic language model.

Outputs should be reviewed before use in production environments.

The model may:

  • generate incorrect code
  • hallucinate APIs or libraries
  • produce incomplete implementations
  • misunderstand project-specific context

License

This repository is released under the Apache License 2.0.

Base Model

This project is derived from Qwen/Qwen3-4B, which is distributed under the Apache License 2.0.

Training Data

The datasets retain their original licenses.

Dataset License
HuggingFaceTB/smoltalk Apache-2.0
agentica-org/DeepCoder-Preview-Dataset MIT

Acknowledgements

This project builds upon the work of:

  • Alibaba Qwen Team
  • Hugging Face
  • HuggingFaceTB
  • Agentica
  • Unsloth

The contributions of these open-source projects made this work possible.


Citation

@misc{uruci2026qwen3cs,
  title={Qwen3-4B-Computer-Science},
  author={Irfan Uruçi},
  year={2026},
  publisher={Hugging Face},
  howpublished={https://huggingface.co/Irfanuruchi/Qwen3-4B-Computer-Science}
}

Contact

Questions, bug reports, and suggestions are welcome through the Hugging Face repository discussions.

Downloads last month
32
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Irfanuruchi/Qwen3-4B-Computer-Science

Finetuned
Qwen/Qwen3-4B
Finetuned
(1030)
this model
Finetunes
2 models
Quantizations
4 models

Datasets used to train Irfanuruchi/Qwen3-4B-Computer-Science