Qwen3.5-4B-rocketdata-v5

Qwen/Qwen3.5-4B fine-tuned with Unsloth (16-bit LoRA) on new_results/sft_v5.jsonl from Phorceon/rocketdata (branch sft-v5-data).

Given flight requirements, the model does two things:

1. It explains its airframe, propulsion, stability and recovery choices in <thought>…</thought>.
2. It writes the complete design in <design>…</design>. The design is the OpenRocket XML tree as JSON: {"openrocket": {"attrs": {...}, "children": [{"rocket": {...}}]}}, where every node is {tag: {attrs?, value?, children?}}. Motors are real motor files named by motor_id (for example AeroTech-K550W).

This model replaces the design format of RocketData/Qwen3.5-4B-rocketdata, which used the older flight/components JSON. The two models aren't interchangeable.

Files

Path	What it is
(top level)	Merged 16-bit weights in Hugging Face format. Load with from_pretrained("RocketData/Qwen3.5-4B-rocketdata-v5").
gguf/Qwen3.5-4B-rocketdata-v5.Q8_0.gguf	The model for llama.cpp, LM Studio or Unsloth Studio.
gguf/Qwen3.5-4B-rocketdata-v5.BF16-mmproj.gguf	Image encoder. Studio and LM Studio want it next to the model.
lora/	The LoRA adapter alone; it needs Qwen/Qwen3.5-4B as the base. Start here for more training or RL.
design.gbnf	llama.cpp grammar that keeps every answer in the <thought>…</thought><design>{json}</design> format with valid JSON.
eval/results_val.json	Every scored answer behind the results below.
train_log.json	Loss history and the exact training settings.

Use the 8-bit GGUF. A 4-bit (Q4_K_M) export of the earlier model broke deeply nested JSON, and v5 designs nest deeper.

How to use it

• Thinking off. The reasoning is in the answer's <thought> block, not in Qwen's thinking slot.

• Sampling: temperature 0.7, top-p 0.95, top-k 20. Lower temperatures make it loop.

• No presence or repetition penalty. JSON repeats keys constantly, and any penalty breaks it.

• Room to finish. Answers run about 5,000 tokens and up to 15,000, so allow 16,384 output tokens and a context of at least 17k.

• Grammar: use design.gbnf where the app supports one (llama.cpp --grammar-file, or the grammar field of a llama-server request).

• System prompt (the one from the training data):

  You are an aerospace design assistant. Explain the airframe, propulsion, stability, and recovery choices inside tags, then output the complete OpenRocket design JSON inside tags. Distinguish stored reference-flight results from static calculations. Do not claim to have run tools or guarantee flight safety. State missing data and safety limits.

Results

These are on the 48 validation prompts, which cover 16 rockets that are not in the training set. The scorer is sft/eval_test.py with --motor-lookup new_results/sft_v4_motor_lookup.json --max-tokens 16384. Generation used the grammar, with thinking off, at temperature 0.7.

Check	This model	Reference answers
Finished (no loop or cut-off)	46/48	48/48
Valid design JSON	46/48	48/48
Well-formed OpenRocket tree	35/48	48/48
Parts in the right places (stage, nose, body, fins, motor in a mount)	32/48	48/48
Parachute or streamer	45/48	48/48
Only real motors (motor_id in the lookup)	8/48	48/48
Uses every motor the prompt asks for	12/15	15/15

Average answer length: 6,020 tokens.

There are two main failure types:

• simulations outside the root. In 8 answers, simulations is written as a second top-level key next to openrocket instead of inside it; the training data always has it inside. That fails the tree and parts checks, but counting only the openrocket part, the tree is well-formed in 45/48 answers and has every part in place in 41/48.
• Made-up motors. The training answers use only 124 of the 1,421 motor files, and 14 of the 17 motors the validation rockets use never appear in training. Of the 56 motors in these answers:
  • 13 are an exact real motor file.
  • 15 name a real motor with a wrong motor_id: 10 under the wrong manufacturer, 5 with a missing or wrong -v1/-v2 file suffix.
  • 28 name a motor that doesn't exist (for example, H1650R).

Both could be fixed at generation time, without retraining, by a grammar that allows only one openrocket root and only the 1,421 real motor_id values. That grammar hasn't been built yet.

The 63 test prompts haven't been run yet. They're held back until a second training seed has been compared on validation.

Known gaps

• Not the final pick. This is the first of two training seeds; a second seed may replace it.
• No physics check. No design has been flown in OpenRocket. The checks above are structural: valid tree, parts in the right place, real motors, the motor that was asked for. Flight numbers in the reasoning are the model's own.
• The split is by .ork file, not by kit. Some kits appear in both training and validation with a different motor mount (for example PML AMRAAM 4 with a 29 mm vs a 38 mm mount). 12 of the 48 validation prompts have such a sibling in training.
• Some reference designs pair a motor_id with a sibling designation (for example AeroTech-J90 written as J90W). The scorer accepts either one.

Training

• Recipe: 16-bit LoRA, rank 16, alpha 16, learning rate 2e-4, batch 1 × gradient accumulation 8.
• Length: 3 epochs (177 steps), seed 42, last epoch kept, max length 11,264 tokens.
• Data: 471 training rows (154 rockets). Validation loss was computed on 45 validation rows, skipping the 3 longest.
• Hardware: RTX 5080. The run was resumed from its epoch-2 checkpoint after the first process slowed down from GPU memory pressure.
• Validation loss by epoch: 0.184, 0.150, 0.145.
• Versions: Unsloth 2026.10.7, transformers 5.5.0, TRL 0.23.1.