|
--- |
|
license: other |
|
datasets: |
|
- Mitsua/vroid-image-dataset-lite |
|
pipeline_tag: text-to-image |
|
--- |
|
# Model Card for VRoid Diffusion |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
This is a latent text-to-image diffusion model to demonstrate how U-Net training affects the generated images. |
|
|
|
- Text Encoder is from [OpenCLIP ViT-H/14](https://github.com/mlfoundations/open_clip), MIT License, Training Data : LAION-2B |
|
- VAE is from [Mitsua Diffusion One](https://huggingface.co/Mitsua/mitsua-diffusion-one), Mitsua Open RAIL-M License, Training Data: Public Domain/CC0 + Licensed |
|
- U-Net is trained from scratch using full version of [VRoid Image Dataset Lite](https://huggingface.co/datasets/Mitsua/vroid-image-dataset-lite) with some modifications. |
|
- VRoid is a trademark or registered trademark of Pixiv inc. in Japan and other regions. |
|
|
|
## Model Details |
|
|
|
- `vroid_diffusion_test.safetensors` |
|
- base variant. |
|
- `vroid_diffusion_test_invert_red_blue.safetensors` |
|
- `red` and `blue` in the caption is swapped. |
|
- `pink` and `skyblue` in the caption is swapped. |
|
- `vroid_diffusion_test_monochrome.safetensors` |
|
- all training images are converted to grayscale. |
|
|
|
## Model Variant |
|
- [VRoid Diffusion Unconditional](https://huggingface.co/Mitsua/vroid-diffusion-test-unconditional) |
|
- This is unconditional image generator without CLIP. |
|
|
|
### Model Description |
|
|
|
- **Developed by:** Abstract Engine. |
|
- **License:** Mitsua Open RAIL-M License. |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
Text-to-Image generation for research and educational purposes. |
|
|
|
### Out-of-Scope Use |
|
|
|
Any deployed use case of the model. |
|
|
|
## Training Details |
|
|
|
- Trained resolution : 256x256 |
|
- Batch Size : 48 |
|
- Steps : 45k |
|
- LR : 1e-5 with warmup 1000 steps |
|
|
|
### Training Data |
|
|
|
We use full version of [VRoid Image Dataset Lite](https://huggingface.co/datasets/Mitsua/vroid-image-dataset-lite) with some modifications. |
|
|
