juliagartor/ControlNetXL_LaserCode

🧠 Model Card: ControlNet-SDXL Fine-Tuned for Industrial Laser Marking

This ControlNet model is a fine-tuned version of diffusers/controlnet-canny-sdxl-1.0, optimized for generating realistic synthetic images of laser-marked industrial packaging. It was trained to align edge-guided structure with high visual fidelity for Optical Character Verification (OCV) tasks in manufacturing and traceability pipelines.

Figure 1: Synthetic images generated with ControlNet

---

🧩 Model Details

Model Description

• Model Type: ControlNet SDXL (Canny version), diffusion-based conditional image generator
• Finetuned From: diffusers/controlnet-canny-sdxl-1.0
• Base Model: Stable Diffusion XL stabilityai/stable-diffusion-xl-base-1.0
• Pretrained VAE: madebyollin/sdxl-vae-fp16-fix
• Use Case: Synthetic image generation for defect simulation and dataset augmentation
• Developed by: Julia Garcia
• Supervised by: Dr. David Castells Rufas, Universitat Autònoma de Barcelona
• Project Type: Final Degree Project

Model Sources

• Repository: Synthetic-Data-Generation-for-Robust-Optical-Character-Verification-in-Industrial-Laser-Marking

---

✅ Direct Use

This model can be used with the Diffusers library to generate grayscale, structurally accurate synthetic images of codes printed on real-world surfaces using Canny edge maps as guidance. It is ideal for: - Optical Character Verification (OCV) dataset augmentation - Industrial defect simulation (e.g., misplacement, beam distortion) - Training robust recognition models for food/pharma packaging

---

Recommendations

• Use clear, well-preprocessed edges (800x800 recommended)
• For novel packaging types, consider retraining with few-shot LoRA/IP-Adapter modules
• Do not use in safety-critical real-time environments without human review

---

🚀 Getting Started

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
from diffusers.utils import load_image
import torch

# Load fine-tuned ControlNet
controlnet = ControlNetModel.from_pretrained("juliagartor/ControlNetXL_LaserCode", torch_dtype=torch.float16)

vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)

pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
      "stabilityai/stable-diffusion-xl-base-1.0",
      controlnet=controlnet,
      vae=vae,
      torch_dtype=torch.float16
).to("cuda")

# Example generation

generated_img = pipe(
    prompt, 
    image=control_img.convert("RGB").resize((800, 800)), 
    num_inference_steps=20,
    num_images_per_prompt=1,
    controlnet_conditioning_scale=1.2,
).images[0]

generated_img.save("generated.png")