Always fetch the package metadata for remove

This is needed to get some of the information needed by the printing in the
plan module.