Package: favorites-crawler
FavoritesCrawlerCrawl your personal favorite images, photo albums, comics from website.Warning!Not ready for production.Appropriately reduce the crawling speed in the future and provide options to set performance, but your account is still at risk of being disabled by the website.Plan to supportinstagram.comAlready supportpixiv.net (crawl your liked illust, must login), Thanks for projectPixivPy.yande.re (crawl your voted posts, require your username)lmmpic.com (crawl your favorite albums, must login)nhentai.net  (crawl your favorite comics, must login)twitter.com (crawl your liked posts, must login)RequirementsPython3.7+Installpip install -U favorites_crawlerConfig Proxy (Optional)# on Windowssethttps_proxy=http://localhost:8080# replace with your proxy server# on Liunx/macOSexporthttps_proxy=http://localhost:8080Loginfavors login [-h] {pixiv,yandere}Login PixivThanks for@ZipFile Pixiv OAuth Flowrun commandfavors login pixivinput your user_id (Access your pixiv personal page, copy from address bar), after press Enter, Pixiv login page will open in browser.Open dev console (F12) and switch to network tab.Enable persistent logging ("Preserve log").Type into the filter field: callback?Proceed with Pixiv login.After logging in you should see a blank page and request that looks like this:https://app-api.pixiv.net/web/v1/users/auth/pixiv/callback?state=...&code=....
Copy value of the code param into the prompt and hit the Enter key.Login Yandererun command:favors login yandereinput your username and hit the Enter key.Login LmmpicOpen lmmpic on browser and login.Use "Get cookies.txt" extension download cookie file.Copy cookie file to {user_home}/.favorites_crawler.Login NHentaiOpen nhentai on browser and login.Use "Get cookies.txt" extension download cookie file.Copy cookie file to {user_home}/.favorites_crawler.Login Twitterrun commandfavors login twitterinput your username, after press Enter, likes page will open in browser.Open dev console (F12) and switch to network tab.Enable persistent logging ("Preserve log").Type into the filter field: Likes?Refresh Page.Copy Authorization, X-Csrf-Token and RequestURL from request(Likes?variables...) input on terminal.Use "Get cookies.txt" extension download cookie file.Copy cookie file to {user_home}/.favorites_crawler.Crawlfavors crawl [-h] {lemon,nhentai,pixiv,yandere}Crawl PixivBefore run this command, make sure you are alreadylogin.favors crawl pixivCrawl YandereBefore run this command, make sure you are alreadylogin.favors crawl yandereCrawl LmmpicBefore run this command, make sure you are alreadylogin.favors crawl lemonCrawl NHentaiBefore run this command, make sure you are alreadylogin.favors crawl nhantaiCrawl TwitterBefore run this command, make sure you are alreadylogin.favors crawl twitterConfigConfig file locate on{your_home}/.favorites_crawler/config.yml.
You can set anyscrapy built-in settingsin this file.By default, file content likes this:pixiv:ACCESS_TOKEN:xxxxxxxxxxxxxxxxxxxxxxxxxxxxREFRESH_TOKEN:xxxxxxxxxxxxxxxxxxxxxxxxxxxxUSER_ID:xxxxyandere:USERNAME:xxxxDownload locationBy default, pictures will download to working directory.If you want to change download location, you can add FILES_STORE option to config.For example, if you want save pixiv files topictures/a, and want save yandere files topictures/b, you can modify config file like this:pixiv:ACCESS_TOKEN:xxxxxxxxxxxxxxxxxxxxxxxxxxxxREFRESH_TOKEN:xxxxxxxxxxxxxxxxxxxxxxxxxxxxUSER_ID:xxxxFILES_STORE:pictures/ayandere:USERNAME:xxxxFILES_STORE:pictures/bOrganize file by artistif you want to organize pixiv illust by user, add this line to your config:pixiv:# FAVORS_PIXIV_ENABLE_ORGANIZE_BY_USER: true  # (Deprecation)ENABLE_ORGANIZE_BY_ARTIST:true# add this line to your yandere configif you want to organize yandere post by artist, add this line to your config:yandere:ENABLE_ORGANIZE_BY_ARTIST:true# add this line to your yandere configStore tags to IPTC/Keywordsonly support pixiv and yandere.yandere:ENABLE_WRITE_IPTC_KEYWORDS:true# default: trueEXIF_TOOL_EXECUTABLE:'<Pathtoyourexiftoolexecutable>'# default Nonepixiv:ENABLE_WRITE_IPTC_KEYWORDS:true# default: trueEXIF_TOOL_EXECUTABLE:'<Pathtoyourexiftoolexecutable>'# default NoneRestore your favoritesVote yandere posts$favorsrestoreyandere-h
usage:favorsrestoreyandere[-h]-s{0,1,2,3}-tCSRF_TOKEN-cCOOKIEpath

positionalarguments:pathThelocationoftheposttovote.(Sub-foldersareignored)optionalarguments:-h,--helpshowthishelpmessageandexit-s{0,1,2,3},--score{0,1,2,3}Set1,2or3tovote,0tocancelvote.-tCSRF_TOKEN,--csrf-tokenCSRF_TOKENCSRFtoken.Togetit:1.OpenyourbrowserDevTools.2.Switchtonetworktab.3.Voteanypostonyandere.4.Copyx-csrf-tokenvaluefromrequestheaders.-cCOOKIE,--cookieCOOKIECookie.Togetit:1.OpenyourbrowserDevTools.2.Switchtonetworktab.3.Voteanypostonyandere.4.Copycookievaluefromrequestheaders.Example:favorsrestoreyandere-s3-t"xxxx"-c"xx=x; xx=x; xx=x".