Use ImmutableBytesPtr as key for in-memory map