Chaman1234/Fine-Tuned-Blender

Fine-Tuned Blender V10

A Qwen2.5-Coder-1.5B-Instruct LoRA adapter for concise Blender 5.1 Python (bpy) script generation. V10 is a 10-step continuation from the locally saved Blender Assistant V8 adapter. It was evaluated against V8 using the same 20 prompts and Blender 5.1 fixture checker.

Both V8 and V10 passed all 20 fixtures. V10's lower code NLL is an observed result on a small, task-related dataset; it does not establish broad generalization. The fixture benchmark is at ceiling, and no inference speedup was measured.

Model details

• Base model: Qwen2.5-Coder-1.5B-Instruct
• Adapter: LoRA, rank 4, alpha 8, dropout 0, targeting q_proj and v_proj
• Adapter parameters: 544,768
• Base-model weights: not included; load them from the upstream model
• License: Apache-2.0, carried forward from this repository's existing model release; check applicable upstream and dataset terms for your use

V10 training

V10 starts from the locally saved V8 adapter and performs 10 additional optimizer steps. The local V9 checkpoint was unavailable, so the V10 label is a release label and does not imply that a V9 adapter was used. The AdamW optimizer state was reinitialized; this was a warm start from adapter weights, not an exact optimizer-state resume.

Setting	Value
Starting adapter	Local Blender Assistant V8 adapter
Training steps in this continuation	10
Unique training examples used	10 of 38
Validation examples	2
Test examples	20
Base precision during training	4-bit NF4 with double quantization
LoRA	Rank 4, alpha 8, query and value projections
Optimizer	AdamW, learning rate 0.0002, weight decay 0
Sequence length limit	320 tokens
Seed	42
Objective	Completion-token loss plus 50,000 times mean squared effective adapter update

The training data contains Blender bpy workflow prompt and code pairs. The full training corpus is not included in this repository. The evaluation artifact includes the 20 workflow cases used for the reported Blender results.

Matched V8 and V10 evaluation

Both versions used greedy decoding (do_sample=False) with a 220-token maximum and were graded with the same Blender 5.1 checker on the same 20 workflow prompts.

Measure	V8	V10
Blender 5.1 fixtures passed	20 / 20	20 / 20
Mean generated tokens	77.20	77.20
Validation code NLL (2 examples, 50 code tokens)	1.328022	1.117962
Test code NLL (20 examples, 1,555 code tokens)	0.005022	0.002022

Lower NLL is better for the listed token-level prediction metric. The test prompts are from the same fixed Blender task family as the benchmark, and the validation set has only two examples. The 20/20 score is a ceiling result and does not show a V10 fixture-accuracy advantage. The NLL changes should not be interpreted as evidence of broad generalization.

Raw per-case completions, checker source and logs, training metrics, and the comparison record are in benchmark/v10/.

Blender benchmark coverage

The results above come from this release's local deterministic Blender 5.1 workflow fixture suite. It checks whether each generated script satisfies the supplied scene-state assertions. It is not a result on an external leaderboard, and it does not score visual quality or interactive editing. Both adapters reach 20/20, so this suite does not distinguish V8 from V10.

We reviewed the following public Blender-related evaluations when choosing how to describe the result:

Benchmark	What it evaluates	V10 status
BlenderGym	245 visual start-to-goal graphics-editing tasks across geometry, lighting, materials, blend shapes, and placement.	Not run. Its protocol is designed for vision-language systems and uses start/goal scene images; this adapter and recorded harness are text/code-only.
BlenderWorld	223 Blender creation tasks with project/export deliverables and text or multimodal inputs.	Not run. Its published evaluation environment specifies Blender 4.2.10, Node.js, ffmpeg/ffprobe, a CLI agent, and configured model/vision-judge endpoints. This run used Blender 5.1 and did not use that evaluator.
3DCodeBench	212 procedural object categories; the text-to-3D setting generates Blender 5.0 Python and provides executability and geometry/visual metrics.	Not run. This release does not include results from its pinned task runner or metrics; the local 20-task fixture suite is a separate evaluation.
3DHarnessBench	100 3D reconstruction targets with single-view, multi-view, active-visual, and full 3D interaction harnesses.	Not run. It evaluates visual perception and, in some settings, iterative Blender tool use; neither is represented by the local one-shot fixture checker.
3DCodeEdit	2,592 instruction-guided edits of 108 procedural Blender programs, judged from renders against checklists.	Not run. No verified benchmark package was installed or staged for this experiment, and its render-based checklist evaluation is outside the local checker.
BlenderBench direct runner	27 tasks using a sequential agent with Blender MCP tools and saved edit/render/inspection rounds.	Not run. The available harness generated standalone scripts and did not provide the specified MCP agent loop.

These evaluations are complementary protocols, not directly comparable scores. The 20/20 local result should not be presented as a BlenderGym, BlenderWorld, 3DCodeBench, 3DHarnessBench, 3DCodeEdit, or BlenderBench score. A future benchmark run should use each benchmark's pinned dataset, evaluator, environment, and required visual/tool inputs.

Intended use and limits

Use this adapter as an experimental coding assistant for Blender 5.1 Python scripting. Generated scripts should be reviewed and tested in a disposable Blender scene before use in important projects.

This is a small-data experiment. It may produce invalid Python, incorrect Blender API calls, or incomplete workflows outside the evaluated examples. It has not been tested broadly across Blender versions, add-ons, complex scenes, or production projects. The evaluation reports no latency, FLOPs, memory reduction, or sparse execution benefit.

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
adapter_id = "Chaman1234/Fine-Tuned-Blender"
quant = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id, device_map="auto", quantization_config=quant
)
model = PeftModel.from_pretrained(base, adapter_id)

request = "Add a bevel modifier named EdgeSoft with width 0.02 to Cube."
content = (
    "Write a Blender 5.1 Python script using bpy. " + request +
    "\nKeep it concise. Output code only, with no comments or Markdown.\n"
    "Return only the Python solution."
)
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": content}],
    tokenize=True, add_generation_prompt=True
)
inputs = torch.tensor([inputs], device=model.device)
output = model.generate(inputs, max_new_tokens=220, do_sample=False)
print(tokenizer.decode(output[0, inputs.shape[1]:], skip_special_tokens=True))

Run the resulting code inside Blender with an appropriate scene and review the changes before saving.

Reproducibility

The benchmark/v10/ folder contains V8 and V10 outputs, exact 20-case inputs, raw checker logs, both metric JSON files, and comparison.json. Training was performed locally with Python 3.10.11, PyTorch 2.1.2+cu118, Transformers 4.46.3, PEFT 0.13.2, bitsandbytes 0.45.5, and a GeForce GTX 1650. The full training source data and base checkpoint are not bundled.