Filter out bad servers and throttle the amount of times we try to refresh