decoder: fix decoder index handling

redeem ancient FIXME request...

No longer use a 'primary' boolean, just use
the decoder index instead.