Add support for multiple cached packages, clean up