Don't consider packages missing from the cache an error on remove

Fixes #2017.