Plugin tweaks

This splits the plugin tweaks I originally made on my RFC #287. Putting the tweaks here means the other PR can care about the actual config work.