Handle too many config files more smoothly