Saravanan666/pypi-org huggingface dataset:
Package: osrm-py
A Python client forOSRM APIRunning the test suitepythonsetup.pytestRequiresrequestsaiohttpUsageWith usingrequestsimportosrmclient=osrm.Client(host='http://localhost:5000')response=client.route(coordinates=[[-74.0056,40.6197],[-74.0034,40.6333]],overview=osrm.overview.full)print(response)With usingaiohttpimportasyncioimportosrmloop=asyncio.get_event_loop()asyncdefrequest():client=osrm.AioHTTPClient(host='http://localhost:5000')response=awaitclient.route(coordinates=[[-74.0056,40.6197],[-74.0034,40.6333]],overview=osrm.overview.full)print(response)awaitclient.close()loop.run_until_complete(request())
Package: os-rotatefile
# os-rotatefile[![Build Status](https://www.travis-ci.org/cfhamlet/os-rotatefile.svg?branch=master)](https://www.travis-ci.org/cfhamlet/os-rotatefile)[![codecov](https://codecov.io/gh/cfhamlet/os-rotatefile/branch/master/graph/badge.svg)](https://codecov.io/gh/cfhamlet/os-rotatefile)[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/os-rotatefile.svg)](https://pypi.python.org/pypi/os-rotatefile)[![PyPI](https://img.shields.io/pypi/v/os-rotatefile.svg)](https://pypi.python.org/pypi/os-rotatefile)Read and write size rotate file.# Install`pip install os-rotatefile`# Usage* Write```from os_roatefile import open_filef = open_file('file', 'w', roll_size='1G')f.write('Your data')f.close()```* Read```from os_roatefile import open_filef = open_file('file', 'r', buffer_size='128K')f.readline()f.read(100)if not f.read():f.close()```# Unit Tests`$ tox`# LicenseMIT licensed.
Package: osrparse
osrparse, a python parser for osu! replaysThis is a parser for the.osrformat for osu! replay files, as described bythe wiki.InstallationTo install, simply:pipinstallosrparseDocumentationPlease see the full documentation for a comprehensive guide:https://kszlim.github.io/osu-replay-parser. A quickstart follows below for the impatient, but you should read the full documentation if you are at all confused.QuickstartfromosrparseimportReplay,parse_replay_data# parse from a pathreplay=Replay.from_path("path/to/osr.osr")# or from an opened file objectwithopen("path/to/osr.osr")asf:replay=Replay.from_file(f)# or from a stringwithopen("path/to/osr.osr")asf:replay_string=f.read()replay=Replay.from_string(replay_string)# a replay has various attributesr=replayprint(r.mode,r.game_version,r.beatmap_hash,r.username,r.replay_hash,r.count_300,r.count_100,r.count_50,r.count_geki,r.count_miss,r.score,r.max_combo,r.perfect,r.mods,r.life_bar_graph,r.timestamp,r.replay_data,r.replay_id,r.rng_seed)# parse the replay data from api v1's /get_replay endpointlzma_string=retrieve_from_api()replay_data=parse_replay_data(lzma_string)# replay_data is a list of ReplayEvents# write a replay back to a pathreplay.write_path("path/to/osr.osr")# or to an opened file objectwithopen("path/to/osr.osr")asf:replay.write_file(f)# or to a stringpacked=replay.pack()# edited attributes are savedreplay.username="fake username"replay.write_path("path/to/new_osr.osr")
Package: os-rq-scrapy
os-rq-scrapyA framework forScrapyworking withos-rq-podandos-rq-hubto build"broad crawls"system.As you know, Scrapy is a very popular python crawler framework. It is suit for "focused crawl", start from several URLs of specific sites, fetch html, extract and save "structured data" also with patternd links to crawl recursively. But for large scale, long time crawling especially "broad crawls", scrapy is incompetent. Basically, you have to decouple the whole crawling system into several sub-systems, high-performance full-featured distributed fetcher, task scheduler, html extractor, link database, data storage, proxy and a lot of auxiliary equipments. It will be more complex when your system is for multi-tenancy.The os-rq-scrapy andos-rq-podproject are basic components to build "broad crawls" system. The core conceptions are very simple, os-rq-pod is multi-sites request queue have http api to recieve requests. os-rq-scrapy is fetcher, getting requests from os-rq-pod and crawl multi-sites concurrently.os-rq-hubcan also be used to connect multi pod and scrapy instances to work simultaneously.RequirementsPython 3.6+ (pypy3.6+)Scrapy2.0extra requirements:ujson, for json performanceInstallpip install os-rq-scrapyUsageCommand linerq-scrapycommand enhance the basicscrapycommand. When RQ_API is configured, thecrawlsubcommand will run on rq mode, get requests from rq.Unit TeststoxLicenseMIT licensed.
Package: osrs
The projectThe goal is to make a wrapper around the varius runescape related api's.RunelitepricesRunelite prices api:https://prices.runescape.wiki/api/v1/osrs/you must provide some identification to tell who you are, preferably with a discord tag or emailfrom osrs import runelite

api = runelite.RunelitePrices(identification='extreme4all#6455')to get a mapping of item names, id, low & high alch values etcprint(api.items())to get latest prices averaging over an interval or from a specific timestampintervals = [
    '5m',
    '10m',
    '30m',
    '1h',
    '6h',
    '24h'
]
print(api.prices(interval='24h'))
print(api.prices(interval='24h', timestamp=1628380800))to get a timeseries of the 300 values averaged over interval by item id or item nameprint(api.timeseries(interval='5m', id=2))
print(api.timeseries(interval='5m', name='Cannonball'))to get the latest prices of itemsprint(api.latest())OSRSThe osrs endpoints, these endpoints are heavily rate limitedosrsPricesfrom osrs import osrs

api = osrs.OsrsPrices(identification='extreme4all#6456')OSRS has only one category, with all the items, here you get each alpha or letter and howmany items are in itprint(api.category())The items endpoint is paginated and will return 12 items for each pageprint(api.items(letter='a', page=0))You can get the itemDetails for a specific item, based on item_idprint(api.itemDetail(item_id=4151))You can get the item price as a timeseries based on item_idprint(api.timeseries(item_id=4151))hiscoresreturn the hiscore for a playerfrom osrs import osrs

api = osrs.Hiscores(identification='extreme4all#6456')
modes = [
    'hiscore_oldschool', 'hiscore_oldschool_ironman', 'hiscore_oldschool_hardcore_ironman',
    'hiscore_oldschool_ultimate','hiscore_oldschool_deadman','hiscore_oldschool_seasonal',
    'hiscore_oldschool_tournament'
]
    
print(api.player(player_name='extreme4all', mode='hiscore_oldschool'))
Package: osrs-python-bot
OSRS Python BotHow to Use OSRS Python BotSetup GuideSystem RequirementsGetting Everything ReadyActually Running ScriptsContributing to the ProjectSkill CategoriesCombat 🤺NMZ Prayer FlickingHerblore 🌿Making Guthix RestCleaning HerbsMagic 🧙‍♂️Casting High AlchemySetup GuideSystem RequirementsWindows PC 💻RuneLite⚔️git😺python&pip🐍Basic understanding of programming 🧑‍💻Ability to run commands from the terminal 👩‍💻Getting Everything Ready======pip install osrs-python-botDownloadrunner.pyActually Running Scripts======Each bot is programmed to click on specific pixels, therefore it requires precise instructions. These instructions may involve the character's orientation, the location of the item in the bank, and even the arrangement of the inventory. It is of utmost importance to follow each script's instructions accurately. These scripts are inteded for private servers.Note:press space bar to stop a scriptUsing these scripts is solely at your own discretion and risk. Employing them may lead to a ban or loss of your old school runescape account. I cannot be held accountable for any misuse or consequences arising from the utilization of this code.Contributing to the ProjectContributions are highly encouraged. I am eager to incorporate computer vision to make the scripts more dynamic. If there are any issues or interest for collaboration please open an issue.I appreciate the donations, it keeps the coffee flowing ☕😊Skill CategoriesCombat 🤺NMZ Prayer FlickingScript#nightmare_zone.pyInstructionsCursor is hovering quick prayers buttonQuick prayers set to rapid healUse Dhorak's with rock cakeFill up on absorptionsTurn on run and auto retaliateCommandpython nightmare_zone.pyHerblore 🌿Making Guthix RestInstructionsRuneLite client is as small as possibleRuneLite client is in top left of monitorAt castle wars bankchestFace directly East (right click the compass)The viewport is fully zoomed inCamera is positioned in the most top down position (hold up arrow key)Inventory pane is open and emptyBank menu opens to a tab with guthix rest suppliesThe bank menu is closed to startSupplies ordered according to the imageLarge inventory of supplies as depletion can break scriptCommandpython make_guthix_rest.pyImageshot cup of water / guam / marrentill / harralanderCleaning HerbsInstructionsRuneLite client is as small as possibleRuneLite client is in top left of monitorCharacter is at castle wars bankchestCharacter is facing directly East (right click the compass)The viewport is fully zoomed in (scroll)Camera is positioned in the most top down position (hold up arrow key)Inventory pane is open and emptyBank menu opens to a tab with suppliesThe bank menu is closed to startHerb in top right bank slotLarge inventory of supplies as depletion can break scriptCommandpython clean_herbs.pyMagic 🧙‍♂️Casting High AlchemyInstructionsMouse is hovering high alchemy spellHigh alchemy item is in the designated spotCommandpython high_alchemy.py
Package: os-ryu
======
os-ryuA component-based software defined networking framework in OpenStack.License: Apache License, Version 2.0Documentation:https://docs.openstack.org/os-ryu/latest/Source:https://git.openstack.org/cgit/openstack/os-ryuBugs:https://bugs.launchpad.net/os-ryuRelease Notes:https://docs.openstack.org/releasenotes/os-ryu
Package: oss2
README of Chinese <https://github.com/aliyun/aliyun-oss-python-sdk/blob/master/README-CN.rst>OverviewAlibaba Cloud Object Storage Python SDK 2.x. This version is not compatible with the previous version (Version 0.x). The package name isoss2to avoid conflict with previous versions.The SDK of this version is dependent on the third-party HTTP libraryrequestsandcrcmod. Install the SDK following the methods below.Note:This version does not contain theosscmdcommand line tool.Running environmentPython 2.6(not recommended)，2.7，3.3(not recommended)，3.4，3.5，3.6Note:Python 2.6 is not recommended because it is no longer supported by the Python core team.
Do not use Python 3.3.0 or 3.3.1. Refer toPython Issue 16658.InstallingInstall the official release version through PIP (taking Linux as an example):$pipinstalloss2You can also install the unzipped installer package directly:$sudopythonsetup.pyinstallGetting started# -*- coding: utf-8 -*-importoss2endpoint='http://oss-cn-hangzhou.aliyuncs.com'# Suppose that your bucket is in the Hangzhou region.auth=oss2.Auth('<Your AccessKeyID>','<Your AccessKeySecret>')bucket=oss2.Bucket(auth,endpoint,'<your bucket name>')# The object key in the bucket is story.txtkey='story.txt'# Uploadbucket.put_object(key,'Ali Baba is a happy youth.')# Downloadbucket.get_object(key).read()# Deletebucket.delete_object(key)# Traverse all objects in the bucketforobject_infoinoss2.ObjectIterator(bucket):print(object_info.key)For more examples, refer to the code under the “examples” directory.Handling errorsThe Python SDK interface will throw an exception in case of an error (see oss2.exceptions sub-module) unless otherwise specified. An example is provided below:try:result=bucket.get_object(key)print(result.read())exceptoss2.exceptions.NoSuchKeyase:print('{0}not found: http_status={1}, request_id={2}'.format(key,e.status,e.request_id))Setup LoggingThe following code can set the logging level of ‘oss2’.importlogginglogging.getLogger('oss2').setLevel(logging.WARNING)TestingFirst set the required AccessKeyId, AccessKeySecret, endpoint and bucket information for the test through environment variables (Do not use the bucket for the production environment).
Take the Linux system for example:$exportOSS_TEST_ACCESS_KEY_ID=<AccessKeyId>$exportOSS_TEST_ACCESS_KEY_SECRET=<AccessKeySecret>$exportOSS_TEST_ENDPOINT=<endpoint>$exportOSS_TEST_BUCKET=<bucket>$exportOSS_TEST_STS_ID=<AccessKeyIdfortestingSTS>$exportOSS_TEST_STS_KEY=<AccessKeySecretfortestingSTS>$exportOSS_TEST_STS_ARN=<RoleARNfortestingSTS>Run the test in the following method:$nosetests# First install noseYou can set environment variable to test auth v2:$exportOSS_TEST_AUTH_VERSION=v2More resourcesMore examples.Python SDK API documentation.Official Python SDK documentation.Contacting usAlibaba Cloud OSS official website.Alibaba Cloud OSS official forum.Alibaba Cloud OSS official documentation center.Alibaba Cloud official technical support:Submit a ticket.LicenseMIT.
Package: oss2r
概述阿里云对象存储Python SDK 2.x版本。该版本不和上一个版本（0.x版本）兼容，包的名称为oss2，以避免和先前的版本冲突。该版本的SDK依赖于第三方HTTP库requests以及crcmod。 请按照下述安装方法进行安装。Note该版本不包含osscmd命令行工具运行环境Python 2.6，2.7，3.3，3.4，3.5Note请不要使用Python 3.3.0、3.3.1，参考Python Issue 16658安装方法通过pip安装官方发布的版本（以Linux系统为例）：$pipinstalloss2也可以直接安装解压后的安装包：$sudopythonsetup.pyinstall快速使用# -*- coding: utf-8 -*-importoss2endpoint='http://oss-cn-hangzhou.aliyuncs.com'# 假设你的Bucket处于杭州区域auth=oss2.Auth('<你的AccessKeyId>','<你的AccessKeySecret>')bucket=oss2.Bucket(auth,endpoint,'<你的Bucket名>')# Bucket中的文件名（key）为story.txtkey='story.txt'# 上传bucket.put_object(key,'Ali Baba is a happy youth.')# 下载bucket.get_object(key).read()# 删除bucket.delete_object(key)# 遍历Bucket里所有文件forobject_infoinoss2.ObjectIterator(bucket):print(object_info.key)更多例子请参考examples目录下的代码。出错处理除非特别说明，一旦出错，Python SDK的接口就会抛出异常（见oss2.exceptions子模块）。参考下面的例子：try:result=bucket.get_object(key)print(result.read())exceptoss2.exceptions.NoSuchKeyase:print('{0}not found: http_status={1}, request_id={2}'.format(key,e.status,e.request_id))测试首先通过环境变量来设置测试所需的AccessKeyId、AccessKeySecret、Endpoint以及Bucket信息（请不要使用生产环境的Bucket）。
以Linux系统为例：$exportOSS_TEST_ACCESS_KEY_ID=<AccessKeyId>$exportOSS_TEST_ACCESS_KEY_SECRET=<AccessKeySecret>$exportOSS_TEST_ENDPOINT=<endpoint>$exportOSS_TEST_BUCKET=<bucket>$exportOSS_TEST_STS_ID=<AccessKeyIdfortestingSTS>$exportOSS_TEST_STS_KEY=<AccessKeySecretfortestingSTS>$exportOSS_TEST_STS_ARN=<RoleARNfortestingSTS>然后通过以下方式运行测试：$nosetests# 请先安装nose更多使用更多例子Python SDK API文档官网Python SDK文档联系我们阿里云OSS官方网站阿里云OSS官方论坛阿里云OSS官方文档中心阿里云官方技术支持：提交工单代码许可MIT许可证，参见LICENSE文件。
Package: oss2-test-1
OverviewAlibaba Cloud Object Storage Python SDK 2.x. This version is not compatible with the previous version (Version 0.x). The package name isoss2to avoid conflict with previous versions.The SDK of this version is dependent on the third-party HTTP libraryrequestsandcrcmod. Install the SDK following the methods below.Note:This version does not contain theosscmdcommand line tool.Running environmentPython 2.6，2.7，3.3，3.4，3.5Note:Do not use Python 3.3.0 or 3.3.1. Refer toPython Issue 16658.InstallingInstall the official release version through PIP (taking Linux as an example):$pipinstalloss2You can also install the unzipped installer package directly:$sudopythonsetup.pyinstallGetting started# -*- coding: utf-8 -*-importoss2endpoint='http://oss-cn-hangzhou.aliyuncs.com'# Suppose that your bucket is in the Hangzhou region.auth=oss2.Auth('<Your AccessKeyID>','<Your AccessKeySecret>')bucket=oss2.Bucket(auth,endpoint,'<your bucket name>')# The object key in the bucket is story.txtkey='story.txt'# Uploadbucket.put_object(key,'Ali Baba is a happy youth.')# Downloadbucket.get_object(key).read()# Deletebucket.delete_object(key)# Traverse all objects in the bucketforobject_infoinoss2.ObjectIterator(bucket):print(object_info.key)For more examples, refer to the code under the “examples” directory.Handling errorsThe Python SDK interface will throw an exception in case of an error (see oss2.exceptions sub-module) unless otherwise specified. An example is provided below:try:result=bucket.get_object(key)print(result.read())exceptoss2.exceptions.NoSuchKeyase:print('{0}not found: http_status={1}, request_id={2}'.format(key,e.status,e.request_id))Setup LoggingThe following code can set the logging level of ‘oss2’.importlogginglogging.getLogger('oss2').setLevel(logging.WARNING)TestingFirst set the required AccessKeyId, AccessKeySecret, endpoint and bucket information for the test through environment variables (Do not use the bucket for the production environment).
Take the Linux system for example:$exportOSS_TEST_ACCESS_KEY_ID=<AccessKeyId>$exportOSS_TEST_ACCESS_KEY_SECRET=<AccessKeySecret>$exportOSS_TEST_ENDPOINT=<endpoint>$exportOSS_TEST_BUCKET=<bucket>$exportOSS_TEST_STS_ID=<AccessKeyIdfortestingSTS>$exportOSS_TEST_STS_KEY=<AccessKeySecretfortestingSTS>$exportOSS_TEST_STS_ARN=<RoleARNfortestingSTS>Run the test in the following method:$nosetests# First install noseYou can set environment variable to test auth v2:$exportOSS_TEST_AUTH_VERSION=v2More resourcesMore examples.Python SDK API documentation.Official Python SDK documentation.Contacting usAlibaba Cloud OSS official website.Alibaba Cloud OSS official forum.Alibaba Cloud OSS official documentation center.Alibaba Cloud official technical support:Submit a ticket.LicenseMIT.
Package: ossapi
ossapi (documentation)ossapi is the definitive python wrapper for the osu! api. ossapi has complete coverage ofapi v2andapi v1, and provides both sync (Ossapi) and async (OssapiAsync) versions for api v2.If you need support or would like to contribute, feel free to ask in the#ossapichannel of thecircleguard discord.InstallationQuickstartAsyncLazerEndpointsBeatmap PacksBeatmapsBeatmapsetsChangelogChatCommentsEventsForumsFriendsHomeMatchesMeNewsOAuthRankingsRoomsScoresSeasonal BackgroundsSpotlightsUsersWikiAPI v1 UsageInstallationTo install:pipinstallossapi# or, if you want to use OssapiAsync:pipinstallossapi[async]To upgrade:pipinstall-UossapiTo get started, read the docs:https://tybug.github.io/ossapi/.QuickstartThe docshave anin depth quickstart, but here's a super short version for api v2:fromossapiimportOssapi# create a new client at https://osu.ppy.sh/home/account/edit#oauthapi=Ossapi(client_id,client_secret)# see docs for full list of endpointsprint(api.user("tybug").username)print(api.user(12092800,mode="osu").username)print(api.beatmap(221777).id)Asyncossapi provides an async variant,OssapiAsync, which has an identical interface toOssapi:importasynciofromossapiimportOssapiAsyncapi=OssapiAsync(client_id,client_secret)asyncdefmain():awaitapi.user("tybug")asyncio.run(main())Read more about OssapiAsync on the docs.LazerYou can retrieve lazer-specific data (scores, leaderboards, etc) with ossapi:fromossapiimportOssapiapi_lazer=Ossapi(client_id,client_secret,domain="lazer")# best score on the lazer server (lazer + osu scores combined)scores=api_lazer.user_scores(12092800,"best")print(scores[0].pp)Read more about domains on the docs.EndpointsAll endpoints for api v2.Beatmap Packsapi.beatmap_packapi.beatmap_packsBeatmapsapi.beatmapapi.beatmap_attributesapi.beatmap_scoresapi.beatmap_user_scoreapi.beatmap_user_scoresapi.beatmapsBeatmapsetsapi.beatmapsetapi.beatmapset_discussion_postsapi.beatmapset_discussion_votesapi.beatmapset_discussionsapi.beatmapset_eventsapi.search_beatmapsetsChangelogapi.changelog_buildapi.changelog_build_lookupapi.changelog_listingChatapi.send_announcementapi.send_pmCommentsapi.commentapi.commentsEventsapi.eventsForumsapi.forum_create_topicapi.forum_edit_postapi.forum_edit_topicapi.forum_replyapi.forum_topicFriendsapi.friendsHomeapi.searchMatchesapi.matchapi.matchesMeapi.get_meNewsapi.news_listingapi.news_postOAuthapi.revoke_tokenRankingsapi.rankingRoomsapi.multiplayer_scoresapi.roomapi.room_leaderboardapi.roomsScoresapi.download_scoreapi.scoreSeasonal Backgroundsapi.seasonal_backgroundsSpotlightsapi.spotlightsUsersapi.userapi.user_beatmapsapi.user_kudosuapi.user_recent_activityapi.user_scoresapi.usersWikiapi.wiki_pageAPI v1 UsageYou can get your api v1 key athttps://osu.ppy.sh/home/account/edit#legacy-api.Basic usage:fromossapiimportOssapiV1api=OssapiV1("key")print(api.get_beatmaps(user=53378)[0].submit_date)print(api.get_match(69063884).games[0].game_id)print(api.get_scores(221777)[0].username)print(len(api.get_replay(beatmap_id=221777,user=6974470)))print(api.get_user(12092800).playcount)print(api.get_user_best(12092800)[0].pp)print(api.get_user_recent(12092800)[0].beatmap_id)
Package: oss-api
OSS_APIAllows to script the creation of scenarios and uploading of layers to OSS for displaying in synapps.installationpip install git+https://IMDCdocker:#######@gitlab.com:imdc/packages/oss_apicommand line interfaceconfigure the project id:oss configure -p 42create a scenario:oss createScenario -n "new scenario"# scenario_id = e.g. 108Upload a *.tif / *.shp file:oss uploadLayer -i 108 -f D:\shapefile.shp -s shapefile.sld -n "new layer" -g Group1python interfacefromoss_apiimportconfigure,createScenario,uploadLayerimportosconfigure(project=42)scen=createScenario(name="Testing new scenario")layers=[r"D:\dev\testdata\subcatchments.shp",r"D:\dev\testdata\dem.tif"]foridx,layerinenumerate(layers):uploadLayer(scenario=scen,name=os.path.basename(layer),filename=layer,sld="polygon",group="Group+%d"%idx)
Package: oss-client
oss-client集成了阿里云OSS、腾讯云对象存储和七牛云对象存储，为不同的对象存储实现了统一的Client，方便使用Installpip install oss-clientTutorialfrom oss_client import AliyunClient

client = AliyunClient(
    access_key=ALY_ACCESS_KEY_ID,
    secret_key=ALY_ACCESS_KEY_SECRET,
    endpoint=ALY_OSS_ENDPOINT,
    bucket=ALY_OSS_BUCKET,
)
Package: osscore
易于使用，人性化的阿里云oss工具包安装pipinstallosscore使用OSSFileSystemaccess_key_id or define env OSS_ACCESS_KEY_IDaccess_key_secret or define env OSS_ACCESS_KEY_SECRETendpoint or define env OSS_ENDPOINTtoken上传文件fromosscoreimportOSSFileSystemOSSFileSystem().upload("bucket_name","local_path","key")下载文件local_path 不写会返回 temp filefromosscoreimportOSSFileSystemOSSFileSystem().download("bucket_name","key","local_path")
Package: os-scrapy
os-scrapyOzzy's Scrapy scaffold.RequirementsPython 3.6+ (pypy3.6+)Scrapy2.0+Installpip install os-scrapyUsageCommand lineThe command is same as scrapyos-scrapy -hCRAWLER_CLASSos-scrapy support configCRAWLER_CLASSreplace the defaultscrapy.crawler.CrawlerIt can be configured insettings.pyfile or command lineos-scrapy crawl -c <your_crawler_class> <spider_name>EnhancedstartprojectThanks toos-scrapy-cookiecutter,-poption can be used to create project as well as python packageos-scrapy startproject -p <project_name> [project_dir]SetTWISTED_REACTORon command line-roption can be used to setTWISTED_REACTORos-scrapy crawl -r asyncio <spider_name>Unit TeststoxLicenseMIT licensed.
Package: os-scrapy-aio-resolver
os-scrapy-aio-resolverThis project provide a async DNS resolver.Installpip install os-scrapy-aio-resolverYou can run example spider directly in the project root pathscrapy crawl exampleUsageSettingsenable DNS resolver in the project settings file:DNS_RESOLVER = "os_scrapy_aio_resolver.AsyncResolver"use asyncio reactorTWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"or you can use os-scrapy command line to start crawling with-r asyncioos-scrapy crawl example -r asyncioenable/disable DNS cache, set cache sizeDNSCACHE_ENABLED = True
DNSCACHE_SIZE = 10000Unit Testssh scripts/test.shLicenseMIT licensed.
Package: os-scrapy-httpproxy-middleware
os-scrapy-httpproxy-middlewareThis project provide middleware to enhance Scrapy built-inHttpProxyMiddleware. You can set proxy without sheme as the following format:[user:password@]proxy:portInstallpip install os-scrapy-httpproxy-middlewareYou can run example spider directly in the project root pathscrapy crawl exampleUsageSettingsenable middleware, it is better disable Scrapy built-in HttpProxyMiddlewareDOWNLOADER_MIDDLEWARES = {
    "scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": None,
    "os_scrapy_httpproxy_middleware.middlewares.HttpProxyMiddleware": 750,
}Unit Testssh scripts/test.shLicenseMIT licensed.