pranamjain/my-text-image-model

Pokémon Text-to-Image LoRA

A LoRA adapter fine-tuned on Stable Diffusion v1.5 for Pokémon-themed text-to-image generation.

Model Details

Property	Value
Base model	Stable Diffusion v1.5
Fine-tuning	LoRA
LoRA rank	8
Trainable parameters	1,594,368
Training epochs	5
Learning rate	1e-4
Batch size	1
Training examples	833
Resolution	512 × 512
Training steps	4,165

Dataset

The model was trained using the pranamjain/pokemon-blip-captions dataset.

The dataset contains Pokémon images paired with text captions.

Training

The base VAE, text encoder, and original UNet parameters were frozen during training.

LoRA adapters were inserted into the following UNet attention layers:

• to_k
• to_q
• to_v
• to_out.0

Only the LoRA parameters were optimized.

The model completed 5 epochs over 833 training examples per epoch.

Usage

import torch
from diffusers import StableDiffusionPipeline

base_model = "stable-diffusion-v1-5/stable-diffusion-v1-5"
lora_model = "pranamjain/my-text-image-model"

pipe = StableDiffusionPipeline.from_pretrained(
    base_model,
    torch_dtype=torch.float16
).to("cuda")

pipe.load_lora_weights(lora_model)

prompt = "a cute Pokemon character standing in a magical forest"

image = pipe(
    prompt,
    num_inference_steps=30,
    guidance_scale=7.5
).images[0]

image.save("generated.png")