It's only detected upon version change because this is a brand new method to support this new feature.  But you're right in that we could possibly work plugin config update into discovery of existing resources in other situations, possibly on-demand or possibly for DOWN resources.