Add setting to determine if we cache settings