PotatoForge/moss-image-4b-encoder-adapter

MOSS-Image Smaller Encoder — 4B to 9B Adapter

An experimental, lightweight text-encoder replacement for MOSS-Image. This adapter maps features from Qwen3.5-4B into the conditioning space of MOSS-Image's original Qwen3.5-9B text encoder. The goal is to reduce memory requirements for prompt encoding while retaining useful image-generation quality.

GitHub — ComfyUI node, source code, and installation instructions: bakapotatolord/moss-image-smaller-encoder

Current adapter: fitted on 800 prompts, with 200 separate validation prompts and 200 held-out test prompts. This is a feature-projection adapter, not a LoRA or a standalone text-to-image model. It requires MOSS-Image, a compatible Qwen3.5-4B encoder, and the adapter node. Equivalent quality to the original 9B encoder is not guaranteed.

How it works

The 4B and 9B encoders remain frozen. The same prompts are encoded using MOSS's prompt template, and token-aligned, final normalized hidden states are collected from both encoders.

A standardized affine projection is fitted using token-weighted ridge regression to approximate the 9B features from the 4B features. Neither encoder nor the diffusion model is fine-tuned.

Component	Details
Student encoder	Qwen3.5-4B
Target encoder	MOSS-Image's original Qwen3.5-9B
Tested encoder checkpoints	INT8 ConvRot versions of both encoders
Projection	2,560 → 4,096 dimensions
Projection parameters	Approximately 10.5 million
Adapter size	Approximately 21 MB / 20 MiB
Stored tensors	FP16 projection matrix and FP32 normalization statistics
Fitting method	Standardized, token-weighted ridge regression
ComfyUI projection	CPU FP32 computation with FP16 output rounding

Only the projection adapter is distributed here. The Qwen encoder and MOSS-Image model weights must be downloaded separately.

Current results — 800 fitting prompts

The current adapter uses a 1,200-prompt dataset, divided into:

Split	SFW prompts	Adult/NSFW prompts	Total
Training	400	400	800
Validation	100	100	200
Held-out test	100	100	200

Only the 800 training prompts were used to fit the projection. Validation selected the ridge regularization strength, and the separate test set measured feature reconstruction.

The mixed corpus includes prompts from jtatman/stable-diffusion-prompts-stats-full-uncensored. SFW/adult categories are based on source labels and text screening; they are not independently verified semantic annotations. Equal prompt counts also do not imply equal token contributions to the fit.

Feature reconstruction

On the 200-prompt held-out test set:

Metric	Result
Content-token relative L2 error ↓	0.4141
Content-token cosine similarity ↑	0.9082

These metrics measure how closely projected 4B features approximate the original 9B features. They are not image-quality scores.

Image comparisons

Eight image pairs compare the current 800-prompt adapter against the original 9B encoder using four SFW prompts and two seeds. Each pair uses matching prompts, seeds, and generation settings:

• Resolution: 640 × 960
• Sampling: 8 steps, Euler / simple
• CFG: 1
• MOSS Image 8-step DMD LoRA strength: 0.8

The examples retain the main subjects and artistic styles, with differences in composition, typography, and incidental details. They do not establish quality parity with the original encoder.

Adult prompts were included in fitting and feature evaluation, but adult image-generation quality has not been visually evaluated. Quality with other encoder quantizations, longer prompts, image editing, higher CFG, and other generation settings also remains unestablished.

Models and usage

Original MOSS-Image models

• MOSS-Image-Base
• MOSS-Image-SFT
• ComfyUI-format MOSS-Image files

Smaller encoder

The tested checkpoint is Qwen3.5-4B INT8 ConvRot. Your ComfyUI installation must support its quantization format.

Use Qwen3.5-4B, not Qwen3-VL-4B. These are different model families.

Download the adapter

The current adapter filename is:

moss_qwen35_4b_to_9b_800_train.safetensors

You can download it from the GitHub release. If downloading from this Hugging Face repository, select the same filename and verify the checksum below.

You only need one copy of the adapter.

Adapter SHA-256 checksum

948ddeaaf051ced32314a704fa0a37e8fc2faf8440abe8e2c9a07090507ae00a

ComfyUI setup

1. Install moss-image-smaller-encoder in ComfyUI/custom_nodes/.
2. Place the Qwen3.5-4B checkpoint in ComfyUI/models/text_encoders/.
3. Place the adapter in ComfyUI/models/moss_encoder_adapters/.
4. Restart ComfyUI.
5. Select Qwen3.5-4B in Load CLIP, with type set to moss_image.
6. Add MOSS Smaller Encoder Adapter, select the adapter file, and connect its CLIP output to the normal CLIP Text Encode node.

Load CLIP → Load LoRA (optional) → MOSS Smaller Encoder Adapter → CLIP Text Encode → KSampler

Keep your usual MOSS diffusion-model and VAE connections. The GitHub repository includes a sample workflow and further compatibility details.

The adapter cannot be loaded through a LoRA loader, and it does not replace the complete text encoder.

Development approach

This project is vibe coded-ish: AI assists with research, implementation, and experiments, while I review the proposed steps, verify the work, and inspect the results. The evaluation limits are documented above.

Credits and references

• ComfyUI-ClipProj by nicolab28 — inspiration for replacing a larger text encoder with a smaller encoder and a learned feature projection. See its calibration methodology.
• MOSS-Image by the OpenMOSS team — the original image-generation model.
• Qwen3.5 by the Qwen team — the encoder family used in this experiment.
• jtatman/stable-diffusion-prompts-stats-full-uncensored — a source of prompts for the expanded calibration corpus.

This is an independent community experiment and is not affiliated with the OpenMOSS, Qwen, or ComfyUI-ClipProj developers. Refer to the licenses of the original models, datasets, and software separately.