Diffusion Single File
comfyui

Is Qwen3VL8b the actual text encoder? Can I use the 3.59b i2i as the text encoder?

#4
by jickman125 - opened

Some confusion on this, can I use the 3.5 9b for text encoder in comfyui or is it just standalone addtl prompt enhancer

I'm rather curious as well. It would be very unusual for a model to be that permissive with different parameter sizes but who knows lol 🤷.

The default clip for the model is the Qwen3VL 8B from the files directory. All 3 of the official workflows (edit, t2i and bg removal) in the latest ComfyUI update use only that and 9b isn't mentioned in their workflow notes. None of them use an LLM driven prompt enhancer.

I'm not certain what the 9b is intend for or what 'PE' is short for but the i2i and t2i suffixes in the filename is interesting. I'm at work so I don't have time to test it at the moment but it's got me wondering.

PE meaning prompt enhancer. I read some of the other discussions and they made it seem like the 9b models were specifically for prompt enhancement. I’d like to see this used in a Workflow but it is such a New model

do they use just the same as Qwen 3VL 8 B? Like, just replace the text encoder
Or do we need a specific workflow for 3.5_PE_9b models?

替換文字編碼器 出來的圖片是亂碼,不知道PE 該如何使用

image
i am facing this issue when i generate an image

These two Qwen3.5 PE models are specially fine-tuning for the best quality, for the best use for QI2.1. The PE models only for PE.

Text Encoder to generate video use the 3VL model.

替換文字編碼器 出來的圖片是亂碼,不知道PE 該如何使用

用 Comfyui llm 工作流。 提示词有英文版本的。Qwen 官方GitHub 有提供。

image
i am facing this issue when i generate an image

That is for llm workflow, please check comfyui llm workflow. Not for generate image.

qwen3.5_9b is like a custom LLM for prompt enhancement, sometimes it outputs gibberish, can fix this by using different seed.
pe
example workflow for t2i.
without pe: wo
with pe: with

for edit just swap the pe to i2i. (for i2i it sometimes overly verbose, so can constraint the prompt generation by adding keywords like short, compact, etc.)
ex: img 1 and img 2 1v1 at basketcourt, make it cinematic, refer to both <img1> and <img2>'s clothes and facial details, 2.5d anime style no need to describe character detail, short:
i2i with

Sometimes it just outputs its thinking process and run out of tokens.

This comment has been hidden (marked as Resolved)

Note that the original code suggests sampling settings that are different from default in comfy Generate Text node:

text2image
Temperature: 1.0
Top_P: 0.95
Top_K: 20
Max New Tokens (Context size): 16,256
Presence Penalty: 1.5

edit
Temperature: 1.0
Top_P: 0.95
Top_K: 20
Max New Tokens (Context size): 24,000
Presence Penalty: 0.0

I'm not 100% sure this can be directly correlated to Comfyui backend implementation of text generation, as Qwen team uses diffusers library to run PE pipeline.

Note that the original code suggests sampling settings that are different from default in comfy Generate Text node:

text2image
Temperature: 1.0
Top_P: 0.95
Top_K: 20
Max New Tokens (Context size): 16,256
Presence Penalty: 1.5

edit
Temperature: 1.0
Top_P: 0.95
Top_K: 20
Max New Tokens (Context size): 24,000
Presence Penalty: 0.0

I'm not 100% sure this can be directly correlated to Comfyui backend implementation of text generation, as Qwen team uses diffusers library to run PE pipeline.

Yeah, different params like higher temp will help on diversity and creativity, and keywords like only final output or compact output can help reducing the output length. Also Regex above to filterout <think> ... </think> also works.

This comment has been hidden (marked as Resolved)

Note that there is actually a system prompt intended to be used with the PE models, they weren't trained to work by themselves without it: https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I/blob/main/system_prompt.txt

Comfy does not embed this in the .safetensors file or automatically add it to the TextGenerate node, so if you want to be faithful the training of the PE model you need to manually add it by prefixing it and manually chat templating it (as far as I am aware there is no chat templating node in Comfy):

<|im_start|>system
# Image Prompt Rewriting Expert

[...]<|im_end|>
<|im_start|>user
Your actual prompt<|im_end|>
<|im_start|>assistant
<think>

And then use the JsonExtractString node to get the actual enhanced prompt (rewritten_prompt key).

The leading <|im_start|> controls whether Comfy wraps in a default chat template internally, so it needs to be right at the start without leading whitespace.

EDIT: wow, look at that timing. System prompt support was just added in https://github.com/Comfy-Org/ComfyUI/pull/16442

the node is so slow
I get > 100tokens/sec on llama.cpp with a long context window. Here, I can't even get a steady 50 token/s

I've already been using Qwen 3.5 9b to make prompts for months on a variety of models. I use the gguff version from HauHauCS here on hf, although that format is starting to get deprecated. I load it using the basic LLM text processor node from KingManiya that's in the custom nodes manager.

From reading this thread, idk that I'll try the i2v/t2v versions in this model's repo.

Qwen 3.5 9b was already about the best choice for a fast and lightweight prompt writer so I'm not sure how much room for improvement it had other than the better format (int8 convrot > gguff) . I don't see an mmproj file either so I'm uncertain how the prompt writer can see the same input images as the model/clip when using these for editing tasks.

I'm not sure how much room for improvement it had}

The main thing is that Qwen Image 2.1 was trained on the output of these specific PE models (they are fine-tuned Qwen 3.5 9B models, not the base one), so you'll probably get better results using them than you would a generic prompt rewriter model.

Sign up or log in to comment