thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora

RefControl FLUX.2 Klein 9B – Reference Pose LoRA

📝 Short description

A LoRA for FLUX.2 Klein 9B Base that fuses a reference image (identity) with a pose control map (skeleton / keypoints).
It preserves identity and style from the reference while following the body pose and structure from the pose map.

Trigger phrase: apply pose from image 1 with reference from image 2

---

📊 Examples

Each preview is a single combined image from ComfyUI: Pose → Reference → Result (left to right).

Pose → Reference → Result

---

📖 Extended description

This LoRA was primarily trained on humans, but it also works with stylized characters and some objects.
Its main purpose is to preserve identity — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the pose and skeleton structure defined by the control map.

FLUX.2 Klein 9B Base already handles reference + pose transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and better preserves character identity and pose fidelity than the base model without LoRA.

Part of the RefControl family: reference + control fusion for consistent, controllable generation on FLUX.2 Klein 9B Base.

---

⚙️ How to use

1. Use the first image as the pose control map (skeleton / keypoints).
2. Use the second image as the reference (character, person, or object).
3. Add the trigger phrase apply pose from image 1 with reference from image 2 in your prompt.
4. Adjust LoRA weight (recommended 0.8–1.0) depending on how strongly you want to preserve identity.

ComfyUI requirenments

Pose extraction in the included workflow uses DWPose (via comfyui_controlnet_aux):

https://github.com/Fannovel16/comfyui_controlnet_aux

Base model

Trained on and recommended with black-forest-labs/FLUX.2-klein-base-9B.
The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, guidance_scale ~4.0).

The LoRA also works with the 4-step distilled black-forest-labs/FLUX.2-klein-9B for faster inference (~4 steps, guidance_scale ~1.0), but quality may be slightly lower — especially for identity and pose fidelity — compared to the Base model.

✅ Example prompt

apply pose from image 1 with reference from image 2

---

🎯 What it does

• Preserves character or object identity across generations.
• Adapts the subject to a new pose or action defined by the skeleton map.
• Works best when the pose map has similar proportions and scale to the reference.

---

⚡ Tips

• Best results when the pose map is not drastically different in body scale or framing from the reference.
• Combine with text prompts to refine background, lighting, or mood.
• DWPose skeleton maps on a black background work well as control input.

---

📌 Use cases

• Character posing for illustrations, comics, or storyboards.
• Consistent character design across different poses.
• Re-posing stylized characters while keeping their identity.
• Creating animation keyframes from static references.

---

📦 Files

• Weights: refcontrol_v2_poses.safetensors
• ComfyUI workflow: refcontrol_pose_flux_klein_9b.json
• Repo: thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora