IndustryForge-27B

IndustryForge-27B is a domain-enhanced multimodal foundation model for industrial CAD. It is built on top of Qwen/Qwen3.5-27B by multi-task SFT on six curated industrial-CAD sub-corpora, and is intended to serve as a common substrate for industrial-agent projects rather than as an end-to-end CAD generator.

It covers four shared CAD skills in a single 27B-parameter multimodal model:

1. CAD visual understanding — reading multi-view engineering drawings, isometric renders, and structured dimensional annotations (CAD-VQA);
2. Parametric CAD code generation — natural language / image → executable CadQuery Python (text2cadquery);
3. Industrial-software API (COM) code generation — SolidWorks / Inventor / AutoCAD COM scripts (com_2d, com_3d, com_assembly);
4. Assembly-level CAD code generation — multi-part CadQuery + assembly scripts (text2cadquery-assembly).

This repository hosts the v4.1 release (checkpoint-6400-merged, 2026-04-29), the configuration evaluated in the technical report.

Highlights

• Large gains on CAD-domain benchmarks. On four domain benchmarks, IndustryForge-27B lifts the base model by +33.65 pp on average and wins 4/4 against gpt-5.4.
• Retains general capability. Across 11 general-capability benchmarks it averages +1.56 pp over the base model, with no catastrophic forgetting.
• First non-zero usable assembly-level pass rate (15.38%) among the models compared in the report.

Domain benchmark results

Benchmark	Qwen3.5-27B	gpt-5.4	IndustryForge-27B	Δ vs base
CAD VQA (accuracy)	84.10%	87.12%	88.23%	+4.13
CadQuery (passed@1e-3)	7.82%	72.36%	77.88%	+70.06
COM CAD (Inventor/SolidWorks, pass rate)	28.33%	30.00%	73.33%	+45.00
CadQuery Assembly (passed@1e-3)	0.00%	3.85%	15.38%	+15.38
Mean	30.06%	48.33%	63.71%	+33.65

General-capability results (11 benchmarks, mean)

Model	Mean
Qwen3.5-27B	81.86%
gpt-5.4	78.90%
IndustryForge-27B	83.42%

Benchmarks: aime26, arc, chartqa, gsm8k, gsm8k_v, math_vision, mmlu, mmmu, ocr_bench, science_qa, trivia_qa.

Training data

Six industrial-CAD sub-corpora, ~52k multimodal samples in total, mixed in a flat proportion (no hand-tuned re-weighting):

Task family	Sub-corpora	Size
CAD visual understanding	cad_vqa	4 k
Parametric CAD code	text2cadquery	17 k
COM parts	com_2d (20 k) + com_3d (5 k)	25 k
Assembly	text2cadquery-assembly (1 k) + com_assembly (5 k)	6 k
Total		~52 k

All code samples are sandbox-executed before use; COM scripts were executed in containerised Windows VMs (SolidWorks / Inventor / AutoCAD) and only samples producing valid .stl / .step artefacts were kept. The corpora are decontaminated against every downstream benchmark at both UUID and AST-structural level.

Training recipe (v4.1)

Item	Value
Base model	Qwen3.5-27B (multimodal, qwen3_5)
Method	Multi-task SFT (full-precision merged LoRA adapter)
PEFT	LoRA rank 8, alpha 32, all-linear; ViT + aligner frozen
Epochs	1
Effective batch size	64 (per_device_train_batch_size=2, 8 GPUs)
Learning rate	5e-5, cosine schedule, warmup ratio 0.05
Sequence parallel	8
Sharding	DeepSpeed ZeRO-3
Packing	padding_free=True (no padding tokens in loss)
Max sequence length	262144
Precision	bfloat16, FlashAttention (flash_attn)
Hardware	1 node × 8× A100-80GB
Framework	ms-swift (swift sft)
Training steps	6400 (max 6568; ≈0.97 epoch of the packed corpus)

The released weights are the merged full model (no adapter needed at inference time), serialised in 12 safetensors shards in bfloat16.

Quickstart

Requires transformers >= 5.2.0.

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch
from PIL import Image

MODEL_ID = "Nianchen/IndustryForge-27B"

processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL_ID,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": Image.open("drawing.png")},
            {"type": "text", "text": "Write a CadQuery script that reproduces this part."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
)
inputs = {k: (v.to(model.device) if hasattr(v, "to") else v) for k, v in inputs.items()}

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

The model was trained with the Qwen3.5 chat template and expects normal user/assistant turns; image inputs should follow Qwen3.5-VL preprocessing (handled by AutoProcessor).

Intended use

• As a foundation substrate for downstream industrial agents that need to read CAD drawings, write CadQuery / COM code, or generate assembly scripts.
• Research on CAD code generation, industrial multimodal reasoning, and industrial-software automation.

Limitations

• Assembly-level capability is still limited (15.38% passed@1e-3); single-part data dominates the training distribution.
• Coverage is limited to the "Windows CAD trio" (SolidWorks, Inventor, AutoCAD); no Office / Adobe COM domains.
• Simulation and CAE are not modelled at the foundation layer.
• Generated code should always be sandbox-executed and geometry-checked before use in a real engineering pipeline.
• This release covers SFT only; closed-loop RL is not included.

Citation

@techreport{industryforge2026,
  title  = {IndustryForge-27B: A Domain-Enhanced Multimodal Foundation Model for Industrial CAD},
  author = {Deng, Nianchen and Ai, Jiaxin and Hu, Tao and Zou, Shu and Dong, Yurui
            and Li, Siqi and Cai, Xinyu and Yang, Xuemeng and Wen, Licheng
            and Zhou, Hongbin and Zhang, Hairong and Cai, Pinlong and Shi, Botian},
  year   = {2026},
  note   = {Technical report. Model available at https://huggingface.co/Nianchen/IndustryForge-27B}
}

License

Released under the Apache License 2.0, following the license of the Qwen3.5-27B base model. Users must additionally comply with the base model's terms.