File size: 3,064 Bytes
759b045 dcc6682 100fe25 dcc6682 100fe25 a088e4b 100fe25 a088e4b 0f9dfe9 a088e4b 100fe25 a088e4b 100fe25 1fa17fd b4f27f3 cf854e7 1fa17fd 553005f 100fe25 a088e4b 100fe25 a088e4b dcc6682 87fcf14 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
---
datasets:
- Eunju2834/img_captioning_oilcanvas_style
- Eunju2834/oil_impressionism_style
pipeline_tag: text-to-image
base_model: runwayml/stable-diffusion-v1-5
tags:
- LoRA
- Diffusion
- OilCanvas
- stable-diffusion
- text-to-image
license: creativeml-openrail-m
---
# 🎨 LoRA text2image fine-tuning - Oil Canvas Style
## 📄 Model Description
This model is a fine-tuned version of the Stable Diffusion v1.5 model using Low-Rank Adaptation (LoRA) techniques to generate images in an oil canvas painting style, specifically focusing on the Impressionism genre. The model is designed to produce vibrant, brushstroke-rich images with a joyful and community-focused theme.
## 🔍 Model Details
- **Base Model:** Stable Diffusion v1.5 (`runwayml/stable-diffusion-v1-5`)
- **Fine-Tuning Method:** LoRA (Low-Rank Adaptation)
- **Training Data:** Custom oil canvas style dataset collected from Kaggle, focused on Impressionism artworks (`Eunju2834/img_captioning_oilcanvas_style`,`Eunju2834/oil_impressionism_style`)
- **Captioning Method:** BLIP-2 model used to generate image captions for the dataset
- **Training Configuration:**
- Epochs: 20
- Batch Size: 1
- Learning Rate: 1e-4
- Scheduler: Cosine
- Seed: 2024
## 🚀 Usage
```python
from diffusers import StableDiffusionPipeline
import torch
model_path = 'Eunju2834/LoRA_oilcanvas_style'
pipe = StableDiffusionPipeline.from_pretrained('runwayml/stable-diffusion-v1-5', torch_dtype=torch.float16, use_auth_token=True)
pipe.unet.load_attn_procs(model_path)
pipe.to('cuda')
prompt = '''(Oil Painting: 1.1), (Impressionism: 1.2), (oil painting with brush strokes: 1.2),
Park stroll, joyful atmosphere, laughter-filled time, playful dogs, vibrant park scene,
cheerful interactions, happy pet owners, heartwarming moments, vibrant community vibes'''
neg_prompt = '''FastNegativeV2, (bad-artist:1.0), (worst quality, low quality:1.4),
(watermark), error, missing fingers, extra digit, cropped, normal quality, blurry'''
image = pipe(prompt, negative_prompt=neg_prompt, num_inference_steps=30, guidance_scale=7.5).images[0]
image.save('oil_impressionism_park_stroll.png')
```
## 🖼️ Example Results
The model generates images like the ones below, showcasing an oil painting style with vibrant colors and Impressionist influences.
<p style="display: flex; justify-content: center;">
<img src="https://cdn-uploads.huggingface.co/production/uploads/64bad345f671da974eeb1ba3/SfjQLeVKie31DZSB23jrq.png" width="250px" />
<img src="https://cdn-uploads.huggingface.co/production/uploads/64bad345f671da974eeb1ba3/sUTDFT20kGhrRUilSgVOe.png" width="250px" />
<img src="https://cdn-uploads.huggingface.co/production/uploads/64bad345f671da974eeb1ba3/MhGz7DPm_Eb_NWoIxQx28.png" width="250px" />
</p>
## ⚠️ Limitations and Biases
- The model is optimized for oil canvas style images and may not generalize well to other artistic styles.
- Potential biases may exist due to the specific nature of the training dataset (e.g., Impressionism artworks).
## 📜 License
CreativeML Open RAIL-M
|