Allow the twitterbot through, since it's not crawling so much as getting meta data from cards.