qwen3-1.7b-rlvr-codegen

LoRA adapter for Qwen/Qwen3-1.7B trained for Python competitive-programming style code generation with RLVR/GRPO. The model is intended to generate complete Python 3 stdin/stdout solutions inside a single code block.

Model Details

• Base model: Qwen/Qwen3-1.7B
• Adapter type: LoRA / PEFT
• Task: Python code generation for verifiable coding problems
• Training method: SFT warmup + GRPO/RLVR using execution-based rewards
• Library: peft, transformers, trl
• Language: English prompts, Python outputs
• License: Inherits constraints from the base model and training data licenses

Intended Use

This adapter is intended for research and experimentation with reinforcement learning from verifiable rewards for code generation.

Expected output format:

# complete Python 3 program

The model is best used for programming problems where solutions read from stdin and write to stdout.

Out-of-Scope Use

Do not use this model for:

• security-critical code without review
• malicious code generation
• unsandboxed execution of generated code
• production programming without tests and human review

Generated code may be incorrect, inefficient, unsafe, or incomplete.

Training Data

Training used PrimeIntellect/verifiable-coding-problems.

Rows were filtered to examples with parseable Python-compatible verification test cases. Prompts were formatted with a chat template instructing the model to output only a concise complete Python 3 solution in one code block.

Training Procedure

The pipeline used:

• rejection-sampling SFT warmup
• LoRA fine-tuning
• GRPO/RLVR with execution-based rewards
• Python subprocess sandboxing for generated solutions

Reward shaping used:

• small reward for extractable code
• small reward for syntactically valid Python
• main reward from test-pass fraction

Recent best run used:

• bf16 model loading
• LoRA adapter training
• num_generations=4
• max_completion_length=512
• beta=0
• no vLLM for the best reported checkpoint

Evaluation

Evaluation was run on a held-out slice of PrimeIntellect/verifiable-coding-problems:

• Offset: 1000
• Limit: 100
• Metric: pass@1 and pass@10
• Execution: generated Python code executed against parsed test cases

Results for uploaded checkpoint candidate (grpo_model_bf16_next/final):

Metric	Score
pass@1	17.00%
pass@10	41.00%

These results are from a small 100-problem slice and should not be treated as a broad benchmark.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-1.7B"
adapter = "lovesahaj1225/qwen3-1.7b-rlvr-codegen"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

messages = [
    {
        "role": "system",
        "content": (
            "You are an expert competitive programmer. Output only one concise complete Python 3 solution. "
            "Read from stdin and write to stdout. Enclose the entire answer in one ```python code block. "
            "Do not include explanations, reasoning, tests, examples, comments, or extra text. "
            "Use a clear correct approach. Stop immediately after the code block."
        ),
    },
    {"role": "user", "content": "Read two integers and print their sum."},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.8,
    pad_token_id=tokenizer.eos_token_id,
)

print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Limitations

• The model can still produce incorrect or incomplete code.
• It may fail on problems requiring long or complex algorithms.
• It was optimized against parsed execution tests, so reward hacking or brittle solutions are possible.
• Evaluation used a limited held-out slice, not a full public benchmark.
• Generated code should be run only in a sandbox.

Compute

Training was run on a single NVIDIA L4 GPU on Google Cloud.

Framework Versions

• PEFT: 0.19.1
• Transformers: 5.0.0+
• TRL: 1.3.0+
• Accelerate: 1.13.0+