Flux TE Neutered
A text encoder designed to serve as a lightweight alternative to the T5 model. For use with the Flux Dev model. Trained in the shadow of a giant.
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00036.png)
- Prompt
- fantasy art depicts a redhead woman, red scales, evil face, black dragon horns, black lipstick
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00025.png)
- Prompt
- drunk man hugging a barmaid next to him at the table, man wearing a hat, next to him, a woman in maid dress, cheerful scene, medieval, daylight, indoors, pub, man holding a beer
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00021.png)
- Prompt
- short redhead hair, pixie cut, freckles, beautiful symmetric eyes, filigree details, sharp focus, highly detailed skin texture, subsurface scattering, reflections, empty background, lipgloss, mascara, eyeshadow, round face, pale skin, full shot, from below, looking at viewer, blue eyes, gown, bracelet, earrings
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00007.png)
- Prompt
- realistic, cyberpunk woman in the shadows, photo of a Korean woman, cinematic, colored inner hair, green black streaked hair, black leather stylish jacket, stars and sparkles, black background, collar on her neck, stylish punk leather jacket, futuristic scene
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00008.png)
- Prompt
- the devil girl within the chamber in the shadows, eyeshadow, sitting, Korean woman, colored inner hair, her reflections shown on the surface of the deep blue water, looks up to the sky, peaking at viewer, from above, black leather stylish jacket, stars and sparkles, black background, collar on her neck, stylish punk leather jacket, futuristic scene
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00017.png)
- Prompt
- a gothpunk witch with a gothpunk hairstyle with pink highlights, coherently detailed perfectly manicured hands, wearing a big Halloween themed witch hat, a long goth dress with lace and black and pink checkered pattern, a broom and bats. The background is a gradient
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00034.png)
- Prompt
- He is dressed in dark, distressed clothing such as a black leather jacket over a dark, graphic tee and ripped jeans. His hair is styled with an emo fringe and dyed in darker tones. The makeup includes dark eyeliner and a pale foundation, reflecting a brooding, melancholic mood. His expression is one of deep sadness and introspection, contrasting with the usual confident demeanor. The background is a moody, dimly lit
![](https://huggingface.co/twodgirl/flux-text-encoder-neutered/resolve/main/images/00003.png)
- Prompt
- In the middles of this fantastical landscape, a vintage silver structure is nestled in deep purple. With intricate floral patterns that sparkle against its surface and the glowing orb of fire swirl from branch to branch, beckoning all who gaze upon it with unforgiving expression before continuing to grow into something else.
While the model can still follow prompts to some extent, its performance in this area is noticeably worse than the original text encoder.
Download
Download the T5 variant, the optional preview decoder and the FP8 quant.
Lightweight requirements:
text_encoder.py
text_encoder_2.safetensors
Setup
pip install accelerate diffusers einops optimum-quanto protobuf sentencepiece transformers
Inference
from diffusers import FluxPipeline, FluxTransformer2DModel
from optimum.quanto.models import QuantizedDiffusersModel, QuantizedTransformersModel
import torch
from text_encoder import t5_config, T5EncoderModel, PretrainedTextEncoder
class Flux2DModel(QuantizedDiffusersModel):
base_class = FluxTransformer2DModel
if __name__ == '__main__':
t5 = PretrainedTextEncoder(t5_config, T5EncoderModel(t5_config)).to(dtype=torch.float16)
t5.load_model('text_encoder_2.safetensors')
# transformer = Flux2DModel.from_pretrained('./flux-fp8')._wrapped
pipe = FluxPipeline.from_pretrained('black-forest-labs/FLUX.1-dev',
text_encoder_2=t5,
# transformer=transformer
)
pipe.enable_model_cpu_offload()
image = pipe('a black cat wearing a Pikachu cosplay', num_inference_steps=10, output_type='pil').images[0]
image.save('cat.png')
The example saves a preview image.
from diffusers import FluxPipeline, FluxTransformer2DModel
from optimum.quanto.models import QuantizedDiffusersModel, QuantizedTransformersModel
import torch
from tea_model import TeaDecoder
from text_encoder import t5_config, T5EncoderModel, PretrainedTextEncoder
class Flux2DModel(QuantizedDiffusersModel):
base_class = FluxTransformer2DModel
def preview_image(pipe, latents):
latents = FluxPipeline._unpack_latents(latents,
pipe.default_sample_size * pipe.vae_scale_factor,
pipe.default_sample_size * pipe.vae_scale_factor,
pipe.vae_scale_factor)
tea = TeaDecoder(ch_in=16)
load_model(tea, './vae_decoder.safetensors')
tea = tea.to(device='cuda')
output = tea(latents.to(torch.float32)) / 2.0 + 0.5
preview = transforms.ToPILImage()(output[0].clamp(0, 1))
return preview
if __name__ == '__main__':
t5 = PretrainedTextEncoder(t5_config, T5EncoderModel(t5_config)).to(dtype=torch.float16)
t5.load_model('text_encoder_2.safetensors')
# transformer = Flux2DModel.from_pretrained('./flux-fp8')._wrapped
pipe = FluxPipeline.from_pretrained('black-forest-labs/FLUX.1-dev',
text_encoder_2=t5,
# transformer=transformer
)
pipe.enable_model_cpu_offload()
latents = pipe('cat playing piano', num_inference_steps=10, output_type='latent').images
preview = preview_image(pipe, latents)
preview.save('cat.png')
Disclaimer
Use of this code and the model requires citation and attribution to the author via a link to their Hugging Face profile in all resulting work.
Model tree for twodgirl/flux-text-encoder-neutered
Base model
black-forest-labs/FLUX.1-dev
Finetuned
this model