Package: twint-fork
This is a fork from twint maintain by yihong0618For now we can only backup 3200 tweetsTWINT - Twitter Intelligence ToolNo authentication. No API. No limits.Twint is an advanced Twitter scraping tool written in Python that allows for scraping Tweets from Twitter profileswithoutusing Twitter's API.Twint utilizes Twitter's search operators to let you scrape Tweets from specific users, scrape Tweets relating to certain topics, hashtags & trends, or sort outsensitiveinformation from Tweets like e-mail and phone numbers. I find this very useful, and you can get really creative with it too.Twint also makes special queries to Twitter allowing you to also scrape a Twitter user's followers, Tweets a user has liked, and who they followwithoutany authentication, API, Selenium, or browser emulation.tl;dr BenefitsSome of the benefits of using Twint vs Twitter API:Can fetch almostallTweets (Twitter API limits to last 3200 Tweets only);Fast initial setup;Can be used anonymously and without Twitter sign up;No rate limitations.Limits imposed by TwitterTwitter limits scrolls while browsing the user timeline. This means that with.Profileor with.Favoritesyou will be able to get ~3200 tweets.RequirementsPython 3.6;aiohttp;aiodns;beautifulsoup4;cchardet;dataclasseselasticsearch;pysocks;pandas (>=0.23.0);aiohttp_socks;schedule;geopy;fake-useragent;py-googletransx.InstallingGit:gitclone--depth=1https://github.com/twintproject/twint.gitcdtwint
pip3install.-rrequirements.txtPip:pip3install.orpip3install--user--upgradegit+https://github.com/twintproject/twint.git@origin/master#egg=twintPipenv:pipenvinstallgit+https://github.com/twintproject/twint.git#egg=twintMarch 2, 2021 UpdateAdded: DockerfileNoticed a lot of people are having issues installing (including me). Please use the Dockerfile temporarily while I look into them.CLI Basic Examples and CombosA few simple examples to help you understand the basics:twint -u username- Scrape all the Tweets of auser(doesn't includeretweetsbut includesreplies).twint -u username --followers- Scrape a Twitter user's followers.twint -u username --following- Scrape who a Twitter user follows.twint -u username --following- Collect full user information a person followstwint -u username --resume resume_file.txt- Resume a search starting from the last saved scroll-id.More detail about the commands and options are located in thewikiModule ExampleTwint can now be used as a module and supports custom formatting.More details are located in thewikiimporttwint# Configurec=twint.Config()c.Username="realDonaldTrump"# Runtwint.run.Search(c)Output955511208597184512 2018-01-22 18:43:19 GMT <now> pineapples are the best fruitimporttwintc=twint.Config()c.Username="noneprivacy"c.Custom["tweet"]=["id"]c.Custom["user"]=["bio"]c.Limit=10c.Store_csv=Truec.Output="none"twint.run.Search(c)Storing OptionsWrite to file;CSV;JSON;SQLite;Elasticsearch.Elasticsearch SetupDetails on setting up Elasticsearch with Twint is located in thewiki.Graph VisualizationGraphdetails are also located in thewiki.We are developing a Twint Desktop App.FAQI tried scraping tweets from a user, I know that they exist but I'm not getting themTwitter can shadow-ban accounts, which means that their tweets will not be available via search. To solve this, pass--profile-fullif you are using Twint via CLI or, if are using Twint as module, addconfig.Profile_full = True. Please note that this process will be quite slow.More ExamplesFollowers/FollowingTo get only follower usernames/following usernamestwint -u username --followerstwint -u username --followingTo get user info of followers/following userstwint -u username --followers --user-fulltwint -u username --following --user-fulluserlistTo get only user info of usertwint -u username --user-fullTo get user info of users from a userlisttwint --userlist inputlist --user-fulltweet translation (experimental)To get 100 english tweets and translate them to italiantwint -u noneprivacy --csv --output none.csv --lang en --translate --translate-dest it --limit 100orimporttwintc=twint.Config()c.Username="noneprivacy"c.Limit=100c.Store_csv=Truec.Output="none.csv"c.Lang="en"c.Translate=Truec.TranslateDest="it"twint.run.Search(c)Notes:Google translate has some quotasFeatured Blog Posts:How to use Twint as an OSINT toolBasic tutorial made by Null ByteAnalyzing Tweets with NLP in minutes with Spark, Optimus and TwintLoading tweets into Kafka and Neo4jContactIf you have any question, want to join in discussions, or need extra help, you are welcome to join our Twint focused channel atOSINT team