Aksh16/Text_to_img

LoRA Fine-Tuned Stable Diffusion v1.5 for Text-to-Image

This repository hosts a LoRA fine-tuned version of the runwayml/stable-diffusion-v1-5 model for text-to-image generation using the peft and diffusers libraries. The model has been fine-tuned on custom captioned image datasets, resized to 512x512 resolution, and optimized for generating high-quality images from textual prompts.

Model Details

• Base Model: Stable Diffusion v1.5 (runwayml/stable-diffusion-v1-5)
• Frameworks: 🤗 Diffusers, PEFT
• Task: Text-to-Image Generation
• Technique: LoRA (Low-Rank Adaptation)
• Image Size: 512x512
• Dataset: Custom images with text captions

How to Use the Model

Below is a sample code snippet showing how to load and use the model:

from diffusers import StableDiffusionPipeline
from peft import PeftModel
import torch

# Load base pipeline
pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    torch_dtype=torch.float16
).to("cuda")

# Load the LoRA fine-tuned adapter
pipe.unet = PeftModel.from_pretrained(
    pipe.unet,
    "Aksh16/Text_to_img"  # Replace with your actual Hugging Face repo path
).to("cuda")

# Generate image from prompt
prompt = "A magical landscape with glowing mushrooms and waterfalls"
image = pipe(prompt).images[0]

# Save the output
image.save("output.png")