This is why we want to switch to memoryviews: this sort of indexing ninjitsu is hard to read and maintain.