final version of the twitter crawling and functional