Unset nested "fields" for deeper nested configs too

In addition to #514