Added progress reporting and flush every 500 tweets.