Kodep/qwen3-4b-effect-codegen

Qwen3-4B — Effect TypeScript Code Generation

Fine-tuned Qwen3-4B model specialized in generating high-quality Effect-style TypeScript code using Reinforcement Learning (GRPO).

Model Details

• Developed by: Kodep
• Model type: Qwen3-4B with LoRA adapter (rank 64)
• Language: TypeScript (Effect framework)
• License: MIT
• Base model: unsloth/Qwen3-4B
• Repository: github.com/belarusian/training

What is this model?

This model generates Effect-style TypeScript code — the popular effect system for functional programming in TypeScript. It's been fine-tuned using GRPO (Group Relative Policy Optimization), a reinforcement learning algorithm that improves code quality through reward-based training.

The model handles:

• Effect imports and core patterns (Effect.succeed, Effect.flatMap, etc.)
• Effect Schema definitions (Schema, decodeSync, etc.)
• Effect service patterns
• Proper TypeScript exports and types

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("Kodep/qwen3-4b-effect-codegen")
model = AutoModelForCausalLM.from_pretrained(
    "Kodep/qwen3-4b-effect-codegen",
    torch_dtype=torch.float16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are an expert TypeScript developer specializing in the Effect framework."},
    {"role": "user", "content": "Generate an Effect service pattern for a user repository"},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=1024, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))

With Unsloth (faster inference)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Kodep/qwen3-4b-effect-codegen",
    max_seq_length=4096,
    load_in_4bit=True,
)

# Inference
messages = [
    {"role": "system", "content": "You are an expert TypeScript developer specializing in the Effect framework."},
    {"role": "user", "content": "Generate an Effect Effect pattern for a user repository"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

• 428 TypeScript code samples extracted from:
  • effect-smol — 185 samples
  • effect — 208 samples
  • opencode — 28 samples
  • effect-examples — 7 samples
• Sources: Real Effect.js library code, OpenCode LLM integrations, and Effect examples

Training Procedure

1. Data extraction: TypeScript files scraped from Effect repositories, filtered for Effect-specific imports
2. SFT pre-training (optional): 2 epochs, learning rate 2e-4 — teaches code format
3. GRPO training: 1 epoch, learning rate 2e-6, batch size 1, gradient accumulation 4x
4. Reward functions:
   • +1.0 Code has <CODE> tags
   • +0.5 Has Effect imports
   • +0.3 Uses Schema
   • +0.2 Has exports
   • -0.5 Response too short (<100 chars)

Hyperparameters

Parameter	Value
Base model	Qwen3-4B
LoRA rank	64
Max sequence	4096
SFT lr	2e-4
GRPO lr	2e-6
SFT epochs	2
GRPO epochs	1
Optimizer	adamw_8bit
Gradient accum.	4

Hardware

• GPU: NVIDIA GeForce RTX 4090 (24GB VRAM)
• CUDA: 13.0
• PyTorch: 2.10.0+cu130
• Unsloth: 2026.5.8
• vLLM: Used for faster inference during GRPO

Risks and Limitations

• Fine-tuned on a small dataset (428 samples) — may not cover all Effect patterns
• May generate syntactically valid but logically incorrect code
• Not suitable for production use without evaluation
• Training focused on code format and import patterns, not correctness verification

Citation

@misc{qwen3-4b-effect-codegen,
  author = {Kodep},
  title = {Qwen3-4B Effect TypeScript Code Generation},
  year = {2026},
  url = {https://huggingface.co/Kodep/qwen3-4b-effect-codegen}
}

Related

• Effect TypeScript Library
• GRPO Paper
• Unsloth
• Training Repository