Make package caching work per plugin