I think if you have boards with different architectures (e.g. native and arm) you'll end up with mixed binaries in `RIOT/bin` as they won't be rebuilt.