Handle failure reading the cache better in rm_read_packet