Package: ak_requests
ak_requestsRequests package with QOL improvements and anti-bot detection measuresView Demo·Documentation·Report Bug·Request FeatureTable of Contents1. About the Project1.1. Features2. Getting Started2.1. Installation2.1.1. Production2.1.2. Development3. Usage4. Roadmap5. License6. Contact7. Acknowledgements1. About the Projectak_requestsis a Python package that provides an interface for automating requests tasks usingrequestspackage. It comes with quality of life improvements like retires, custom redirects, spacing out requests and shuffling requests to not trigger anti-bot measures1.1. FeaturesBulk requests handlingBuilt-in retries and timeoutsCan log processes to fileHandles downloads of files/videosImplemented default rate-limiting checks and processSession objects are serialized to be able to save/load sessions from fileCan choose to handle exceptions or skip it completely withRAISE_EXCEPTIONSattributeCan support both basic.basic_auth()and OAuth.oauth2_auth()authentications.2. Getting Started2.1. Installation2.1.1. ProductionDownload the repo and install with flitpipinstallflit
flitinstall--depsproductionAlternatively, you can use pippipinstallak_requests2.1.2. DevelopmentInstall with flitflitinstall--pth-file3. Usagefromak_requestsimportRequestsSession# Initialize sessionsession=RequestsSession(log=False,retries=5,log_level='error')## Can update session level variablessession.MIN_REQUEST_GAP=1.5# seconds, Change min time bet. requestssession.RAISE_ERRORS=False# raises RequestErrors, else returns None; defaults to True# Update custom headersession.update_header({'Connection':'keep-alive'})# set cookiessession.update_cookies([{'name':'has_recent_activity','value':'1'}])# Get requestsres=session.get('https://reqres.in/api/users?page=2',data={},proxies={})# Can accept any requests parameters# Make bulk requestsurls=['https://reqres.in/api/users?page=2','https://reqres.in/api/unknown']responses=session.bulk_get(urls)# use beautifulsoupfromak_requestsimportsoupifyres=session.get('https://reqres.in/api/users?page=2')soup=soupify(res)## orsoup,res=session.soup('https://reqres.in/api/users?page=2')## Also works for bulk requestssoups,ress=session.bulk_soup(urls)# Download files## Check if file is downloadblesession.downloadble('https://www.youtube.com/watch?v=9bZkp7q19f0')#Falsesession.downloadble('http://google.com/favicon.ico')#Truesession.download(url='http://google.com/favicon.ico',#URL to downloadfifopath='C:\\',#Can be folderpath, filename or filepath. If existing folder specified - will extract filename from url contentsconfirm_downloadble=False#Will return `None`, if url not downloadble)# Download videosfrompathlibimportPathvideo_info=session.video(url='https://www.youtube.com/watch?v=BaW_jenozKc',folderpath=Path('.'),audio_only=False)#Downloads the video to specified path and returns dict of video info# Save/Restore session to/from file## Save the session state to a filesession.save_session('session_state.pkl')## Later, you can load the session state backrestored_session=RequestsSession.load_session('session_state.pkl')# Authenticationsession.basic_auth(username="johndoe",password="12345678")## basic authsession.oauth2_auth(token='x0-xxxxxxxxxxxxxxxxxxxxxxxx')## OAuth authentication4. RoadmapProxyAsynchronous RequestsResponse CachingRequest Preprocessing and PostprocessingFile Upload Support5. LicenseSee LICENSE for more information.6. ContactArun Kishore -@rpakishoreProject Link:https://github.com/rpakishore/ak_requests7. AcknowledgementsAwesome README TemplateShields.io