Spaces:
Running
on
A10G
Models with better SDXL support?
I use exclusively SDXL these days, because I can't think of any good reason to use other models except in rare use cases.
I notice that CLIP-Interrogator has a dropdown for "clip model", but the only option is "ViT-L (best for Stable Diffusion 1.*"
I presume this means SDXL support is planned for the future?
Any ideas on the timeline for that?
Or any way I can help with that? I have a background as a software developer.
The CLIP-Interrogator is a really awesome concept, but without better SDXL support it's of but very limited use for me.
It already has SDXL support, but it's kinda tucked away on their Github page. The simplest way to use it is to:
Clone this repository to your computer using
git clone https://huggingface.co/spaces/pharmapsychotic/CLIP-Interrogator.Create a virtual env for it and activate it.
Modify requirements.txt in the following way, so it uses the latest packages:
--extra-index-url https://download.pytorch.org/whl/cu117
torch
torchvision
fairscale
ftfy
huggingface-hub
Pillow
timm
transformers
open_clip_torch
requests
clip-interrogator
- Install requirements with
pip install -r requirements.txt - Modify the first 16 lines of app.py like so:
#!/usr/bin/env python3
import gradio as gr
from clip_interrogator import Config, Interrogator
from share_btn import community_icon_html, loading_icon_html, share_js
# MODELS = ['ViT-L (best for Stable Diffusion 1.*)']#, 'ViT-H (best for Stable Diffusion 2.*)']
MODELS = ['ViT-G (best for SDXL)']
# load BLIP and ViT-L https://huggingface.co/openai/clip-vit-large-patch14
# config = Config(clip_model_name="ViT-L-14/openai")
# ci_vitl = Interrogator(config)
# ci_vitl.clip_model = ci_vitl.clip_model.to("cpu")
config = Config(clip_model_name="ViT-g-14/laion2B-s34B-b88K")
ci_vitl = Interrogator(config)
After this, you can start up the app using python app.py and interrogate images with the SDXL clip model. If you want to run it on your CPU, add the following on the 17th line:
ci_vitl.clip_model = ci_vitl.clip_model.to("cpu")
@baruga :
Thanks a lot for the info!
Unfortunately my PC isn't powerful enouigh to run this locally.
And running on CPU sounds painfully slow.