Properly get one-byte bytelists from cache when indexing with byte

Fixes #1361.