shohuu/pyroton-primefix-v3

🔥 Pyroton — PrimeFix v3

A targeted repair-finetuned adapter for Pyroton, fixing correctness bugs in Python code generation.

---

Overview

Pyroton-PrimeFix-v3 is a patched adapter built on top of shohuu/pyroton. It applies targeted repair finetuning to fix specific correctness bugs that appeared in the base Pyroton adapter.

For the full project details, training strategy, and documentation, see the main repo: 👉 shohuu/pyroton 👉 github.com/TunasTuna/pyroton

---

What Was Fixed

The base Pyroton adapter had recurring correctness bugs in generated Python code:

Bug	Description
Missing import math	Model used math.sqrt() without importing math
Wrong edge case handling	Incorrect results for negative numbers, 0, and 1
Unsafe n**0.5	Caused crashes for negative inputs

Fix approach

• Filtered 570 bad training samples that used math. without the required import
• Applied targeted repair finetuning on corrected examples
• Evaluated against execution-based harness

---

Evaluation Results

Tested on is_prime() with inputs: -1, 0, 1, 2, 3, 4, 6, 9, 17, 49

Mode	Result
Greedy decoding (do_sample=False)	✅ 5/5 passing
Sampled decoding (temperature=0.3)	⚠️ Improved but less stable

Use greedy decoding for best results.

---

Quick Start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-0.5B-Instruct",
    dtype=torch.bfloat16,
    device_map="auto",
)

model = PeftModel.from_pretrained(base, "shohuu/pyroton-primefix-v3")
tokenizer = AutoTokenizer.from_pretrained("shohuu/pyroton-primefix-v3")
tokenizer.pad_token = tokenizer.eos_token

prompt = "### Instruction:\nWrite a Python function to check if a number is prime\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=220,
    do_sample=False,
    repetition_penalty=1.1,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Example Output

Prompt:

Write a Python function to check if a number is prime

Output:

import math

def is_prime(n):
    """Check if the given integer n is prime."""
    if n <= 1:
        return False
    for i in range(2, int(math.sqrt(n)) + 1):
        if n % i == 0:
            return False
    return True

---

Recommended Inference Settings

Correctness mode (recommended)

outputs = model.generate(
    **inputs,
    max_new_tokens=220,
    do_sample=False,
    repetition_penalty=1.1,
)

Sampled mode (more variety, less stable)

outputs = model.generate(
    **inputs,
    max_new_tokens=220,
    do_sample=True,
    temperature=0.1,
    top_p=0.9,
    repetition_penalty=1.2,
)

---

Known Limitations

• Still a 0.5B model — can degrade on harder or more complex tasks
• Repair finetuning improved specific failure modes but is not a full benchmark suite
• Greedy decoding recommended for correctness-critical use cases

---

Related

• 🔥 Main model: shohuu/pyroton
• 🚀 Upgraded model: shohuu/pyroton-1.5b (coming soon)
• 💻 GitHub: github.com/TunasTuna/pyroton

---

License

Apache 2.0 — see LICENSE for details.

Base model (Qwen2.5-Coder) is also Apache 2.0. Attribution to Alibaba Cloud / Qwen Team.

---

Acknowledgements

• Qwen Team for the base model
• Hugging Face for the training ecosystem
• My friend Yumi for the name 🔥