In Working Models
Collection
4 items
•
Updated
Abstract:
The textile industry in India boasts a rich tapestry of traditional designs, each reflecting unique cultural heritage and artistic expressions. Among these, patterns like Bandhani, Ajrakh, Ikat, Bagh, Dabu, Kalamkari, Batik, Saanganeri, Shibori, and Block printing hold significant historical and aesthetic value. Recognizing these intricate designs manually is laborious and time-consuming. Hence, leveraging machine learning techniques can significantly aid in automating this process.
Compute By :
'Pattern-Name' Textile Pattern
Indian Textile Pattern Sampling
Classical Pattern [ Woven ]
South Indian Textile Pattern Sampling
Kanchipuram Pattu [ Woven ]
Model Description
- Study by: LABS.ML (Org by PrithivLabs)
- Model type: Diffusion-based text-to-image generative model
- Model Description: This is a model that can be used to generate and modify images based on text prompts. It is a Latent Diffusion Model that uses two fixed, pretrained text encoders (OpenCLIP-ViT/G and CLIP-ViT/L).
- Resources for more information: SDXL report on arXiv.
Model Sources
For research purposes, we recommend our generative-models Github repository (https://github.com/Stability-AI/generative-models), which implements the most popular diffusion frameworks (both training and inference) and for which new functionalities like distillation will be added over time.
Clipdrop provides free SDXL inference.
Evaluation
The chart above evaluates user preference for SDXL (with refinement) over SDXL 0.9 and Stable Diffusion 1.5 and 2.1. The SDXL base model performs significantly better than the previous variants, and the model combined with the refinement module achieves the best overall performance.
Stable Diffusion Zone
Implementation of SD
🧨 Diffusers
Make sure to upgrade diffusers to >= 0.19.0:
pip install diffusers --upgrade
In addition make sure to install transformers, safetensors, accelerate as well as the invisible watermark:
pip install invisible_watermark transformers accelerate safetensors
To just use the base model, you can run:
from diffusers import DiffusionPipeline
import torch
pipe = DiffusionPipeline.from_pretrained("prithivMLmods/Textile-Pattern-Labs", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
pipe.to("cuda")
# if using torch < 2.0
# pipe.enable_xformers_memory_efficient_attention()
prompt = "Yali Textile Pattern"
images = pipe(prompt=prompt).images[0]
To use the whole base + refiner pipeline as an ensemble of experts you can run:
from diffusers import DiffusionPipeline
import torch
# load both base & refiner
base = DiffusionPipeline.from_pretrained(
"prithivMLmods/Textile-Pattern-Labs", torch_dtype=torch.float16, variant="fp16", use_safetensors=True
)
base.to("cuda")
refiner = DiffusionPipeline.from_pretrained(
"prithivMLmods/Textile-Pattern-Labs",
text_encoder_2=base.text_encoder_2,
vae=base.vae,
torch_dtype=torch.float16,
use_safetensors=True,
variant="fp16",
)
refiner.to("cuda")
# Define how many steps and what % of steps to be run on each experts (80/20) here
n_steps = 40
high_noise_frac = 0.8
prompt = "Ajrakh Textile Pattern"
# run both experts
image = base(
prompt=prompt,
num_inference_steps=n_steps,
denoising_end=high_noise_frac,
output_type="latent",
).images
image = refiner(
prompt=prompt,
num_inference_steps=n_steps,
denoising_start=high_noise_frac,
image=image,
).images[0]
When using torch >= 2.0, you can improve the inference speed by 20-30% with torch.compile. Simple wrap the unet with torch compile before running the pipeline:
pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)
If you are limited by GPU VRAM, you can enable cpu offloading by calling pipe.enable_model_cpu_offload
instead of .to("cuda"):
- pipe.to("cuda")
+ pipe.enable_model_cpu_offload()
For more information on how to use Stable Diffusion XL with diffusers, please have a look at the Stable Diffusion XL Docs.
Generative Adversarial Networks (GANs)
Generative Adversarial Networks (GANs) have emerged as a powerful tool in the field of computer vision for generating realistic images. In the domain of textile design, GANs offer the potential to revolutionize pattern generation by autonomously creating intricate and culturally significant designs. This study conducts a quantitative comparison of various GAN architectures for textile pattern generation, focusing on their ability to capture the complexity and diversity of traditional Indian textile patterns.
We curate a comprehensive dataset consisting of high-resolution images representing a diverse range of Indian textile patterns, including but not limited to Bandhani, Ajrakh, Ikat, Bagh, Dabu, Kalamkari, Batik, Saanganeri, Shibori, and Block printing. The dataset is meticulously annotated to ensure accurate representation of pattern attributes such as color, texture, and motif arrangement.
We implement and train multiple GAN architectures, including DCGAN, WGAN, CGAN, and StyleGAN, using the curated dataset. Each architecture is fine-tuned and optimized for textile pattern generation, leveraging techniques such as progressive training, spectral normalization, and conditional generation.
To quantitatively evaluate the performance of each GAN architecture, we employ established metrics such as Inception Score (IS), Fréchet Inception Distance (FID), and Perceptual Path Length (PPL). These metrics provide insights into the fidelity, diversity, and perceptual quality of the generated textile patterns.
Additionally, we conduct a user study to assess subjective preferences and perceived visual quality of the generated patterns. Participants are presented with pairs of real and generated textile patterns and are asked to provide feedback on visual similarity, aesthetic appeal, and cultural authenticity.
Our findings reveal significant variations in the performance of different GAN architectures for textile pattern generation. While some architectures excel in capturing fine details and texture, others exhibit superior diversity and color fidelity. Furthermore, subjective evaluations highlight the importance of cultural authenticity and aesthetic appeal in assessing the quality of generated patterns.
This study contributes to the ongoing research in the application of GANs for textile pattern generation and provides valuable insights into the strengths and limitations of different architectures. By elucidating the quantitative and qualitative aspects of pattern generation, this research aims to inform future developments in the field and facilitate the creation of culturally rich and visually compelling textile designs.
OpenVINO
OpenVINO toolkit is a free toolkit facilitating the optimization of a deep learning model from a framework and deployment using an inference engine onto Intel hardware.
To install Optimum with the dependencies required for OpenVINO :
pip install optimum[openvino]
To load an OpenVINO model and run inference with OpenVINO Runtime, you need to replace StableDiffusionXLPipeline with Optimum OVStableDiffusionXLPipeline. In case you want to load a PyTorch model and convert it to the OpenVINO format on-the-fly, you can set export=True.
- from diffusers import StableDiffusionXLPipeline
+ from optimum.intel import OVStableDiffusionXLPipeline
model_id = "prithivMLmods/Textile-Pattern-Labs"
- pipeline = StableDiffusionXLPipeline.from_pretrained(model_id)
+ pipeline = OVStableDiffusionXLPipeline.from_pretrained(model_id)
prompt = "Bandhani Textile Pattern"
image = pipeline(prompt).images[0]
You can find more examples (such as static reshaping and model compilation) in optimum documentation.
ONNX
To install Optimum with the dependencies required for ONNX Runtime inference :
pip install optimum[onnxruntime]
To load an ONNX model and run inference with ONNX Runtime, you need to replace StableDiffusionXLPipeline with Optimum ORTStableDiffusionXLPipeline. In case you want to load a PyTorch model and convert it to the ONNX format on-the-fly, you can set export=True.
- from diffusers import StableDiffusionXLPipeline
+ from optimum.onnxruntime import ORTStableDiffusionXLPipeline
model_id = "prithivMLmods/Textile-Pattern-Labs"
- pipeline = StableDiffusionXLPipeline.from_pretrained(model_id)
+ pipeline = ORTStableDiffusionXLPipeline.from_pretrained(model_id)
prompt = "Bandhani Textile Pattern"
image = pipeline(prompt).images[0]
You can find more examples in optimum documentation.
GPU T4 Engine Computations
To calculate the computational time for GPU-based machine learning tasks such as stable diffusion, you need to consider several factors including the complexity of your model, the size of your dataset, the hardware specifications of your GPU, and the efficiency of your implementation. Here's a general formula to estimate the computational time:
Computational Time = Number of iterations × Time per iteration / Number of GPUs
Where:
Number of iterations: This refers to the number of iterations or epochs your machine learning algorithm will run for.
Time per iteration: This is the time taken by your algorithm to complete one iteration on a single GPU.
Number of GPUs: The number of GPUs you're using for computation.
Uses
Direct Use
The model is intended for research purposes only. Possible research areas and tasks include
- Generation of artworks and use in design and other artistic processes.
- Applications in educational or creative tools.
- Research on generative models.
- Safe deployment of models which have the potential to generate harmful content.
- Probing and understanding the limitations and biases of generative models.
Excluded uses are described below.
Out-of-Scope Use
The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
Limitations and Bias
Limitations
- The model does not achieve perfect photorealism
- The model cannot render legible text
- The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
- Faces and people in general may not be generated properly.
- The autoencoding part of the model is lossy.
Bias
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
- Downloads last month
- 456