dotimothy/sima-cyws

CYWS change detection, compiled for the SiMa.ai MLSoC Modalix

The Change You Want to See (CYWS; Sachdeva & Zisserman, WACV 2023) takes a before and an after photo of the same scene and returns boxes around what changed: objects that appeared, disappeared or changed. It is a Siamese ResNet-50 U-Net with co-attention between the two images and a CenterNet head on each branch.

These are its pretrained weights (pretrained-resnet50-3x-coam-scSE-affine) compiled for the Modalix MLA with SiMa.ai's Model Compiler. The whole network is one MLA stage, with nothing on the CPU, and one before/after pair of 256x256 windows takes about 35 ms (bf16).

file	precision	MLA latency per window	changes found	pixel F1	false-alarm pairs	same boxes as FP32	size
bf16/cyws_bf16_mpk.tar.gz	BF16	35.2 ms	33 / 47	0.587	2 / 15	1.000	119 MB
w8a16/cyws_w8a16_mpk.tar.gz	A_BF16_W_INT8	27.2 ms	31 / 47	0.581	3 / 15	0.966	72 MB

• Latency: Model.benchmark on a Modalix DevKit (NEAT 0.5.0), plugins included. The bf16 model runs 36 windows/s at 13.2 W.
• Changes found, pixel F1, false alarms: the app's balanced preset (a global window plus gated 256-input tiles) on testCYWS: 28 before/after pairs of 12 MP phone photos, with 47 annotated changes, and 15 pairs where nothing changed. The settings were tuned on the same pairs, so treat these numbers as optimistic for new scenes.
• Same boxes as FP32: box agreement with the ONNX FP32 model on 97 inputs.

Input and output

Two inputs, pushed together and in this order:

• in0_before and in1_after, each float32 256x256x3, HWC, RGB, pixel / 255, with no mean/std normalisation.

One fused output, float32 (10, 256, 256). Its channels are heat_b, heat_a, wh_b(w,h), wh_a(w,h), off_b(x,y), off_a(x,y), with the heat channels already passed through a sigmoid.

• Decoding: take 3x3 peaks of each heat map above about 0.29, then build the boxes from wh and off (CenterNet style).
• Labels: a box on the after map only is appeared, a box on the before map only is disappeared, and a box on both is changed.

Using them

On a Modalix board, use Neat Change Studio. Its web demo lists the models in this repository (models.json) and downloads one onto the board from Settings → Model. From a terminal:

webapp/run.sh --list-hub          # what is here and what is on the board
webapp/run.sh --fetch bf16        # or w8a16, or all: downloaded into models/ and checked

Without the app:

hf download dotimothy/sima-cyws bf16/cyws_bf16_mpk.tar.gz --local-dir .

import numpy as np, pyneat

opt = pyneat.ModelOptions()
opt.preprocess.kind = pyneat.InputKind.Tensor               # raw tensors in, no image preprocessing
model = pyneat.Model("bf16/cyws_bf16_mpk.tar.gz", opt)

def tensor(hwc):
    t = pyneat.Tensor.from_numpy(np.ascontiguousarray(hwc, np.float32), copy=True)
    t.layout = pyneat.TensorLayout.HWC
    return t

before = after = np.zeros((256, 256, 3), np.float32)        # your two crops: RGB / 255, HWC
run = model.build([tensor(before), tensor(after)])
run.push_tensors([tensor(before), tensor(after)])
sample = run.pull(5000)
t = sample.tensors[0] if sample.tensors else sample.tensor
out = np.array(t.to_numpy(), np.float32, copy=True).transpose(2, 0, 1)   # (10, 256, 256)
run.close()

Photos larger than 256x256 should be tiled: run one global window, then 256-input tiles where the global heat map saw a change, so small objects are seen at the scale CYWS was trained on.

These files only run on the MLA. For PyTorch or ONNX Runtime, use the original checkpoint.

License and attribution

Apache-2.0. CYWS is by Ragav Sachdeva and Andrew Zisserman (The Change You Want to See, WACV 2023, https://github.com/ragavsachdeva/The-Change-You-Want-to-See). The authors have confirmed to the maintainer (2026) that CYWS, its code and pretrained weights, is released under the Apache License 2.0. This supersedes the CC BY-NC default mentioned in upstream issue #11 (2024). The upstream repository does not include a LICENSE file yet.

These files are derived from those weights and are Apache-2.0 too:

• BatchNorm is folded into the convolutions;
• the network is rewritten to compile for the MLA, with the same function;
• it is compiled in bf16, or with int8 weights for w8a16.

Nothing was retrained. The code that does this is at dotimothy/neat-change-studio.

@inproceedings{sachdeva2023change,
  title     = {The Change You Want to See},
  author    = {Sachdeva, Ragav and Zisserman, Andrew},
  booktitle = {Winter Conference on Applications of Computer Vision (WACV)},
  year      = {2023}
}