PixelModel-v4 Image Codec Experiments

This repository documents experiments conducted on PixelModel-v4 only. PixelModel-v5 was released after these experiments and uses a substantially different architecture. The observations reported here should not be assumed to transfer directly to v5.

This repository is a research archive for measuring how image codecs affect a PixelModel-v4 weight image and the model reconstructed from it. It is not the main PixelModel-v4 model repository and is not a replacement model card.

Scope/Provenance

The upstream model card and model assets are licensed under MIT by bench-labs/PixelModel-v4. The upstream model revision used by the experiment is 80878fdd6e130d229c2581424a3456b47b979d4b.

The archive records a fixed comparison: 100 prompts, 50 sampling steps, CFG 6, Apple MPS, and identical prompt, seed, initial latent, and settings across conditions. The archive is intended for auditability and re-running the experiment, not for making claims about codecs outside these recorded settings. The samples/ tree contains one artifact for every successful generation row (800 artifacts in the full 17-condition run), while failed rows retain null sample provenance.

Motivation

PixelModel-v4 stores trained weights as image data, which makes ordinary image storage and transport tools relevant to model serialization. Earlier work used PNG as the lossless weight container. This repository explores what happens when the same fp16 weight image is passed through modern lossless and lossy codecs, then reconstructed with several byte-level methods. This is exploratory research: the goal is to record the observed relationship between codec output, reconstructed weight values, and generation behavior without proposing a general-purpose compression result.

Experimental Setup

Model: PixelModel-v4 DiT with an fp16 weight image, using the upstream revision above.
Weight representation: each fp16 word is serialized across image bytes; the high byte contains the sign, exponent, and upper mantissa bits.
Prompts: 100 prompts.
Sampling: 50 steps, CFG 6.
Controls: identical prompt, seed, initial latent, and inference settings.
Device: Apple MPS for the archived full run.
Raw reconstruction is accepted only when the decoded weight tensor is finite. A NaN count by itself is not called “nonfinite”; the gate checks the complete tensor and reports NaN, +Inf, and -Inf separately.
FID is N/A for this archive. CLIP and confidence intervals are not included unless they are present in the archived CSV.

Codecs Evaluated

Codec	Fixed setting	Role
PNG	lossless	baseline
WebP	lossless	lossless control
JPEG	Q100	lossy codec
JPEG	Q80	lossy codec
WebP	Q80	lossy codec
AVIF	Q70	lossy codec
JPEG XL	distance 1 (`d1`)	lossy codec

The protected representation keeps the high byte as PNG data and stores the low byte through the lossy codec. The code also records the raw and finite-repair paths separately.

Reconstruction Methods

Method	Description
Raw	Decode the codec payload directly into the weight image representation.
Finite Repair Zero	Replace decoded NaN, +Inf, and -Inf values with zero while leaving finite values unchanged.
Protected High Byte	Preserve the high byte losslessly and decode only the low byte through the lossy codec.

Experiment Flow

flowchart TD
    A[PixelModel-v4 fp16 weights] --> B[Serialize weights into image bytes]
    B --> C[Encode with PNG, WebP, JPEG, AVIF, or JPEG XL]
    C --> D[Decode codec payload]
    D --> E{Reconstruction method}
    E --> F[Raw]
    E --> G[Finite Repair Zero]
    E --> H[Protected High Byte]
    F --> I[Validate reconstructed tensor]
    G --> I
    H --> I
    I --> J{Finite-weight gate passes?}
    J -- Yes --> K[Run inference]
    J -- No --> L[Record blocked condition and diagnostics]
    K --> M[Record generation, weight, storage, and quality metrics]
    L --> M

Representative Results

The following links are relative archive paths. A failed cell is rendered as an explicit “NO SAMPLE” panel; it is not replaced with a black image or an inferred metric.

Caption: JPEG Q100 table showing raw, finite-repair, and protected cells for representative prompts.

Caption: JPEG Q80 table. Raw and Finite Repair Zero cells have no generated sample; Protected High Byte cells show the archived outputs.

Caption: lossy WebP Q80 table with explicit failed cells and archived Protected High Byte outputs.

Caption: AVIF Q70 table with the same prompts, seeds, and inference settings as the controls.

Caption: JPEG XL distance 1 table. Failed conditions are marked NO SAMPLE; no replacement image is used.

Results table from available CSV only

The values below are transcribed from the archived CSV outputs for the full 100-prompt run. They are counts and payload measurements, not estimates.

Condition	Raw generation	Finite Repair Zero	Protected High Byte	Raw NaN / +Inf / -Inf	Reliable payload saving vs PNG
PNG	100/100	N/A	N/A	0 / 0 / 0	0.0000%
WebP lossless	100/100	N/A	N/A	0 / 0 / 0	24.2799%
JPEG Q100	0/100	100/100	100/100	1,257 / 0 / 0	-6.8825%
JPEG Q80	0/100	0/100	100/100	144,962 / 289 / 0	36.5568%
WebP Q80	0/100	0/100	100/100	1,097,350 / 0 / 0	37.6701%
AVIF Q70	0/100	0/100	100/100	445,041 / 59 / 0	39.2740%
JPEG XL d1	0/100	0/100	100/100	59,017 / 13 / 0	37.2180%

The lossless WebP payload divided by the PNG payload is 0.7572009918694315. The saving percentages use PNG baseline payload bytes as the denominator; Protected High Byte payloads include the high-byte PNG and the codec-specific low-byte payload. If an archive CSV is absent or differs, the CSV is authoritative and this table must not be treated as an additional measurement.

Main Observations

PNG and WebP lossless completed 100/100 generations.
All five lossy raw conditions completed 0/100 generations.
JPEG was not uniquely fragile in this matrix: every tested lossy Raw setting introduced non-finite fp16 values and was blocked before generation.
Finite Repair Zero completed 100/100 only for JPEG Q100; the other four lossy conditions recorded non-finite generation-stage failures.
Protected High Byte completed 100/100 for all five lossy codecs.
The protected rows recorded zero exponent mismatches.
The raw diagnostic counts differ substantially across codecs; NaN, +Inf, and -Inf are kept as separate fields.

These observations motivated future work on bit-level importance analysis rather than further broad codec comparisons.

Discussion

Facts are the counts, byte diagnostics, exponent-mismatch counts, and payload measurements recorded in the CSV files. They describe this run and do not by themselves establish a causal mechanism.

A hypothesis consistent with the observations is that the fp16 high byte—which contains the sign, exponent, and upper mantissa bits—is substantially more important to the recorded model behavior than the lower byte. Protected High Byte preserves that high byte while allowing low-byte compression to alter less significant bits. The successful protected runs demonstrate the outcome for this matrix, but they do not establish the proposed bit-importance mechanism as a general causal fact. Protected High Byte is also a hybrid representation, not a pure lossy weight artifact. Codec implementation choices, decoder behavior, tensor layout, and the MPS runtime may contribute.

Repository Contents

images/                         Archived figures and representative images
results/                        CSV results, reports, and archive_manifest.json
samples/                        800 successful sample artifacts for all successful generation rows
weights/                        Archived payloads and weights/manifest.csv
scripts/cross_codec_report.py   Report generation
scripts/render_visual_tables.py Portable fail-closed visual-table renderer
scripts/verify_archive.py       Hash, byte-count, containment, and symlink verifier
scripts/pixelmodel_robustness/  11 packaged robustness modules
scripts/dit.py                  Packaged DiT implementation
scripts/config/                 Experiment configuration and prompt manifests
tests/                          Focused codec, repair, report, renderer, and archive tests

Reproducibility

The full runner may require Apple MPS because that is the archived runtime. The following commands are generic and intentionally do not promise universal hardware reproducibility.

uv sync --locked
uv run --locked pytest
PYTHONPATH=scripts uv run --locked python scripts/verify_archive.py --root .
PYTHONPATH=scripts uv run --locked python scripts/render_visual_tables.py \
  --run-dir . \
  --prompt-ids-manifest results/representative_prompt_ids.json \
  --output scratch/rendered-tables

The complete archived experiment uses the packaged runner and the archived configuration. Start with a small smoke test before attempting the full run. The full implementation may require MPS and the source assets.

How to verify/re-run

Verify the archive before reading results:

PYTHONPATH=scripts uv run --locked python scripts/verify_archive.py --root .

Run the focused tests:
```
uv run --locked pytest
```

Inspect the packaged report and experiment command surfaces:

PYTHONPATH=scripts uv run --locked python scripts/cross_codec_report.py --help
PYTHONPATH=scripts uv run --locked python -m pixelmodel_robustness.cross_codec_experiment --help

For a fresh source checkout, obtain the upstream assets at the exact revision. This command is a prerequisite only; it does not alter this archive:
```
hf download bench-labs/PixelModel-v4 \
  --revision 80878fdd6e130d229c2581424a3456b47b979d4b
```
Use the packaged runner's --help output to select the input paths and output run directory, then run on a machine with the required model runtime. Preserve the runner manifest, encoded payloads, materialized weight images, samples, resource log, and raw rows together. samples/ contains all 800 successful sample artifacts, so raw sample provenance is complete even though the compact archive intentionally deduplicates payloads. Complete report regeneration should target a fresh canonical run directory with materialized weight images; the compact archive layout is not sufficient for a fully fresh report regeneration.

For AVIF, Pillow must have AVIF support. JPEG XL requires an ImageMagick build with the libjxl delegate. These codec prerequisites are recorded by the experiment where available.

Weights archive explanation

The weights/ directory is an archive of experiment inputs and codec payloads, not a new or fine-tuned model release. weights/manifest.csv records the path, byte count, and SHA-256 for each listed payload. Git LFS is configured for this directory. The archive verifier checks that paths stay inside the repository, that listed files exist with the recorded bytes and hashes, that paths are not duplicated, and that no symlink is used in the archived trees.

Limitations

The experiment covers one model version, PixelModel-v4 at the pinned revision.
Only the listed codec settings were evaluated; the setting matrix is limited.
No result here establishes that one codec is universally superior.
The full run uses one fixed prompt/seed/latent/settings protocol and one Apple MPS runtime.
The results do not establish codec performance for other models, tensor layouts, image dimensions, devices, Pillow versions, or libjxl versions.
Generation success is a gate outcome, not a quality ranking.
No FID result is reported; FID is N/A here.
Missing or failed samples are not imputed.
The archive does not claim that protected high-byte storage preserves every aspect of a model outside the recorded experiment.
These observations should not be generalized beyond this experiment without additional models, settings, runtimes, and independently specified evaluations.

Future Work

Perform bit-level importance analysis with pre-registered corruption masks.
Evaluate exponent-only corruption.
Evaluate mantissa-only corruption at controlled bit positions.
Test additional codecs and codec settings.
Investigate learned, weight-aware compression methods.
Repeat the protocol across additional model revisions, tensor layouts, prompt sets, devices, and decoder versions.

License and acknowledgment

This archive is released under the MIT License. It acknowledges bench-labs/PixelModel-v4 and its MIT-licensed upstream model card. This repository is an experimental archive built from the robustness-study source; it is not the upstream model repository.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support