OpenMOSS-Team

/

MOSS-Image-SFT

Like 523

Follow

OpenMOSS 1.55k

Hi MOSS team, thanks for releasing MOSS-Image.

Several users have noticed that SFT paints faint, evenly spaced vertical colour stripes across smooth areas such as skies, fog, walls and studio backdrops. Base doesn't do this. We've narrowed it down, and some of what we found might help pin down the cause in training.

Same prompt and seed (SFT int8, skip-layer guidance on layer 22). Middle row: the colour-only view from bandviz, with brightness removed and colour ×5. Plain SFT (left) has regular vertical stripes and a sharp spectrum spike at 21.4 px. With the 64 directions removed (centre and right) the spike is gone and the image is otherwise unchanged.

What the stripes look like

• They're a colour (chroma) shift. Brightness is barely affected.
• There are always 48 cycles across the image width: 21.3 px apart at 1024 wide, 18 px at 864 and 16 px at 768. The count across the width stays fixed, while the spacing in pixels changes with resolution. To us that points to the width axis of the positional encoding rather than the VAE.
• They form in the middle of sampling (roughly steps 5–14 of 25 with ER-SDE) and are already visible in the model's predictions in latent space.

Where they come from

• MATLOWAI's block-swap analysis puts them in layers 32–41.

• We broke the Base→SFT weight change into directions (SVD per layer). We then used gradients to score how much each direction grows the stripe frequency. The stripes are concentrated in a small number of directions in layers 24–41, mostly the largest parts of SFT's ffn_up_gate change, plus a few in ffn_down and the attention q, k, v and o projections.

• Removing just the 64 strongest directions clears the stripes. Using MATLOWAI's bandviz score on 12 held-out images at 1024 px:

  	bandviz median (lower is better)
  SFT	24.0
  SFT with the 64 directions removed	5.9
  Base	4.0

  The output stays more than 99% similar to SFT (SSIM 0.993).

• Applying the same directions with a negative sign makes the stripes much stronger, so this is cause and effect, not a side effect:

768×1024, ER-SDE, 25 steps, CFG 2.5. Top row: negative strengths, where the stripes get stronger (bandviz 26 → 571). Bottom row: positive strengths, where they're gone (about 11) and the scene doesn't change.

Workaround for users

A 1.9 MB LoRA, with full numbers and method: moss_sft_destripe_top64_bf16.safetensors

Questions for you

1. Did something in the SFT stage differ from Base in a way that could produce a fixed-frequency pattern along the width? For example, positional encoding scaling, resolution bucketing, or a data source with this pattern, such as upscaled or resized images.
2. Would a fixed SFT checkpoint be possible? We're happy to share the analysis scripts or the per-direction scores if useful.

Credit to MATLOWAI for the original investigation and the bandviz metric.