hoyporti/granite-switchboard

Granite 3.0 1B-A400M Router Logits

Granite 3.0 1B-A400M Instruct cut down to what its routers see. It takes input_ids with shape [1, tokens] and returns router_logits with shape [24, tokens, 32], which is every layer's score for each of the 32 experts. Take the top 8 per row and softmax them to get the experts and weights the model would use.

It powers Figure 2 of switchboard, which routes whatever you type in the browser with ONNX Runtime Web on WebGPU.

The experts and attention are stored in 8 bits (MatMulNBits, blocks of 32) and the embedding table in fp16. The routers stay in fp32, since they decide everything shown here. On the sentences switchboard traces, it picks the same 8 experts as the full fp32 model at 94.8% of token and layer pairs. A 4-bit copy only managed 57%.

The model can't generate text. The expert outputs still run, because each layer's router depends on what the layers before it computed, but nothing past the last layer's routers is kept.

Rebuild it with python -m switchboard routes from the switchboard repo.