I made a colab for FLUX dev but it's stuck on CLIP

#77
by QES - opened

Hi, I made a colab for FLUX dev but it's stuck on CLIP. I'm trying to force the pipe to use T5 with something like:

Load the T5 tokenizer and encoder

t5_tokenizer = T5Tokenizer.from_pretrained("t5-large")
t5_encoder = T5EncoderModel.from_pretrained("t5-large").to(device)

Load the FLUX pipeline

pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
torch_dtype=torch.float16,
use_safetensors=True,
)

Replace the pipeline's text encoder and tokenizer with T5

pipe.text_encoder = t5_encoder
pipe.tokenizer = t5_tokenizer

I tried to max to 512 like:

image = pipe(
prompt=processed_caption,
num_inference_steps=num_inference_steps,
guidance_scale=guidance_scale,
width=width1 if i == 0 else width2,
height=height1 if i == 0 else height2,
generator=generator,
max_sequence_length=512
).images[0]

But it keeps using CLIP and my prompts are truncated...

How can I do that?

THANKS!

Sign up or log in to comment