Saravanan666/pypi-org huggingface dataset:
Package: proxy-manager-g4
Proxy ManagerProxy manager to download a list of proxy servers from the Internet with the required parameters
(protocol and degree of anonymity) and work with loading servers (for example, getting a random
proxy servers from this list).InstallationPackage Installation from PyPi$pip3installproxy-manager-g4Package Installation from Source CodeThe source code is available onGitHub.Download and install the package.$gitclonehttps://github.com/Genzo4/proxy_manager
$cdproxy_manager
$pip3install.Basic usageImport:fromproxy_manager_g4importProxyManagerfromproxy_manager_g4.constsimportPROTOCOL_HTTPSWe create an instance of the proxy manager. At the same time, loading a list of proxy servers from the Internet
with the required parameters (protocol and degree of anonymity).proxy_manager=ProxyManager(protocol=PROTOCOL_HTTPS,anonymity=True)Get random proxy:proxy=proxy_manager.get_random()When getting a random proxy server multiple times, proxy manager will track the number of uses of each
proxy for uniform delivery.Using the received proxy server:proxy.ip# "1.2.3.4"proxy.port# 8080proxy.get_ip_port()# "1.2.3.4:8080"print(proxy)# "1.2.3.4:8080"Usage example in file minitest.pyChangelogThe list of proxy servers is loaded fromhttps://github.com/fate0/proxylist/https://github.com/clarketm/proxy-list/Proxy ManagerПрокси менеджер для загрузки из интернета списка прокси серверов с требуемыми параметрами
(протокол и степень анонимности) и работы с загруженными серверами (например, получение случайного
прокси-сервера из данного списка).УстановкаУстановка пакета с PyPi$pip3installproxy-manager-g4Установка пакета из исходного кодаИсходный код размещается наGitHub.Скачайте его и установите пакет:$gitclonehttps://github.com/Genzo4/proxy_manager
$cdproxy_manager
$pip3install.ИспользованиеПодключаем:fromproxy_manager_g4importProxyManagerfromproxy_manager_g4.constsimportPROTOCOL_HTTPSСоздаём экземпляр прокси менеджера. При этом происходит загрузка списка прокси-серверов из интернета
с требуемыми параметрами: протокол и степень анонимности.proxy_manager=ProxyManager(protocol=PROTOCOL_HTTPS,anonymity=True)Получение случайного прокси-сервера из списка:proxy=proxy_manager.get_random()При многократном получении случайного прокси-сервера, прокси менеджер будет отслеживать количество использований
каждого прокси для равномерной выдачи.Использование полученного прокси-сервера:proxy.ip# "1.2.3.4"proxy.port# 8080proxy.get_ip_port()# "1.2.3.4:8080"print(proxy)# "1.2.3.4:8080"Пример использования см. в файле minitest.pyChangelogЗагрузка списка прокси-серверов осуществляется сhttps://github.com/fate0/proxylist/https://github.com/clarketm/proxy-list/
Package: proxy-master
This module provide asynchronously scraping a bunch ofhttp,https,socks4,socks5proxies from websites such as:free-proxy-list.netgeonode.comhidemy.nameJust now proxy-master scraped23265 proxiesfrom websites above. Count of unique by ip:port is18383,
but some servers support different protocols.I add more websites to scrap from time to time,BUTlooking for help and advices
from people with experienceDependenciespython3.10and higher. Pattern matching using in projectWill be installed automatically withpipaiohttpbs4lxmlpycountryaiohttp_socksInstallationpip install proxy-masterUsageThis will createproxy_master.jsonin you home directory and return list of proxies.importproxy_masteraspmproxies=pm.get_proxies(protocol='http',do_prints=True)Then you can test proxies:working_proxies=pm.test_proxies(proxies,proxy_protocol:'http',website_protocol:'http')FeaturesScrap different type of proxies includehttps,socks4,socks5Recursivelyscraping. Use already collected proxies to scrap another websitetest_public_ip(...)socks4,socks5usingaiohttp-proxyIn plans[]https://freeproxylists.net[]https://spys.one/en/free-proxy-list/
Package: proxy-mgr
This module loads and uses a proxy for the convenience of using requests, aiohttp, etc.Forkhttps://pypi.org/project/proxy-manager/InstallationThis module is available via pip:$ pip install proxy-MGRBasic Usageproxies.txt:https://00.11.222.33:4444
https://55.66.77.88:9999@username:password
socks5://10.11.121.13:1414@username:password
15.16.171.18:1919
20.21.222.23:2424@username:password
25.26.272.28:2929:username:passwordtest.py:fromProxyMGRimportProxyMGRMGR=ProxyMGR("proxies.txt")# MGR = ProxyMGR(proxy_list=["00.11.222.33:4444@username:password"])# Random proxyrandom_proxy=MGR.random_proxy()print(random_proxy)# https://00.11.222.33:4444# Next proxyproxy=MGR.next_proxy()print(proxy)# https://00.11.222.33:4444proxy=MGR.next_proxy()print(proxy)# https://55.66.77.88:9999@username:passwordproxy=MGR.next_proxy()print(proxy)# socks5://10.11.121.13:1414@username:password# Proxy dictprint(random_proxy.get_dict())# {'http': 'https://00.11.222.33:4444', 'https': 'https://00.11.222.33:4444'}# Proxy infoprint(random_proxy.get_info())# {"ip":"81.222.188.37","success":true,"type":"IPv4","continent":"Europe" ...DocumentationProxy File FormatProxies in proxy files must have one of the following formats:ip:port
ip:port:username:password
ip:port@username:password
http://ip:port
...These can be combined and alternated.ProxyMGRProxyMGR(proxy_file_path)ProxyMGR(proxy_list)Parametersproxy_file_path(string) — File path to the proxy (Required, if proxy_list is not specified)proxy_list(list) — List proxy (Required, if proxy_file_path is not specified)default_type(string) — Default proxy type (exampleip:port->https://ip:port) (optional)ProxyMGR.random_proxy()Returns a random proxy of those loaded into the proxy managerProxyMGR.next_proxy()Parametersloop(bool) — Repeat proxies if they run outProxyMGR.get_dict()dict - Proxy as a dict in the form{'http':'http://...', 'https':'https://...'}for use with requestsProxyMGR.get_info()Website Requesthttps://ipwho.iswith proxy and return json
Package: proxy-middleware
Scrapy middlewares that reads proxy config from settings.ContentsInstallProxyFromSettingsMiddlewareProxyOnlyTorMiddlewareLicenseHistory0.2.0 (2017-05-29)0.1.1 (2017-04-07)Installpip install proxy-middlewareProxyFromSettingsMiddlewareA middleware that sets proxy from settings file.Usage: add it toDOWNLOADER_MIDDLEWARESin scrapy settings:DOWNLOADER_MIDDLEWARES = {
   'proxy_middleware.ProxyFromSettingsMiddleware': 10,
   ...Pass proxy config viaHTTP_PROXYandHTTPS_PROXYsettings
variables.HTTPPROXY_AUTH_ENCODINGis also respected:scrapy crawl my-spider -s HTTP_PROXY=http://localhost:8118ProxyOnlyTorMiddlewareA middleware that proxies only requests to TOR sites (domain ends with “.onion”).Usage: add it toDOWNLOADER_MIDDLEWARESin scrapy settings:DOWNLOADER_MIDDLEWARES = {
   'proxy_middleware.ProxyOnlyTorMiddleware': 10,
   ...Settings:HTTP_PROXYfor an http proxy andHTTPS_PROXYfor an https proxy.LicenseLicense is MIT.History0.2.0 (2017-05-29)ProxyOnlyTorMiddlewareadded.0.1.1 (2017-04-07)Initial release
Package: proxy-ninja
Proxy NinjaPython3 Module to get https or socks(4) proxies by scraping the web.InstallationBefore installing the module , please install the ChromiumDriver as following debian/kalisudoapt-getupdate&&sudoapt-getfull-upgrade-ysudoapt-getinstallchromium-driverUse the package managerpipto install the module.
if not installed, install it using the following command.sudoapt-getinstallpython3-pipIt is advised to install the python requirements in a virtual environment, for that install the venv package.python3-mpipinstallvenvpython3-mvenvenvsourceenv/bin/activateAfter that run the following commands:python3-mpipinstall<pkg>-UUsage/ExamplesUsage:==> Download the proxies.#!/usr/bin/python3fromproxy_ninjaimportfetch_proxiesfetch_proxies(PROXY_TYPE,OUTPUT_FILENAME,OUTPUT_FORMAT)PROXY_TYPE: https/socksOUTPUT_FILENAME: Enter the filenameOUTPUT_FORMAT: txt/json-- proxies gonna save in parent dir.Example:#!/usr/bin/python3fromproxy_ninjaimportfetch_proxiesfetch_proxies("socks","socks_proxy","json")==> Get proxies as json#!/usr/bin/python3fromproxy_ninjaimportproxies_jsonjson_list=proxies_json(PROXY_TYPE)print(json_list)PROXY_TYPE: https/socksThis will gonna return a list.Example:#!/usr/bin/python3fromproxy_ninjaimportproxies_jsonjson_list=proxies_json("https")print(json_list)FeaturesChromeDriver to scrape the site.Stealth Profle implmented.Save output in txt or json format.User Friendly. :DLicenseMITFeedbackIf you have any feedback, please reach out to us atakalucifr@protonmail.chChange Log0.1.0 (17/8/2022)initial release
Package: proxy-ninja-ng
ProxyNinja-ngPython3 Module to get https or socks(4/5) proxies by scraping the web.🔧 Technologies & Tools📚 RequirementsPython 3.9+pip3InstallationUse the package managerpipto install the module.
if not installed, install it using the following command.sudoapt-getinstallpython3-pipIt is advised to install the python requirements in a virtual environment, for that install the venv package.python3-mpipinstallvenvpython3-mvenvenvsourceenv/bin/activateAfter that run the following commands:python3-mpipinstall<pkg>-UORgitclonehttps://github.com/sc4rfurry/ProxyNinja-ng-PyPi.gitcdProxyNinja-ng-PyPipython3-mpipinstall.Usage/ExamplesUsage:==>Download the proxies#!/usr/bin/python3fromproxy_ninjaimportfetch_proxiesfetch_proxies(PROXY_TYPE,OUTPUT_FILENAME,OUTPUT_FORMAT)PROXY_TYPE: https/socks4/socks5OUTPUT_FILENAME: Enter the filenameOUTPUT_FORMAT: txt/jsonproxies gonna save in parent dir.Example:#!/usr/bin/python3fromproxy_ninja_ngimportfetch_proxiesfetch_proxies("socks","socks_proxy","json")==>Get proxies as json#!/usr/bin/python3fromproxy_ninja_ngimportfetch_proxies_jsonjson_list=fetch_proxies_json(PROXY_TYPE)print(json_list)PROXY_TYPE: https/socks4/socks5This will gonna return a json list.Example:#!/usr/bin/python3fromproxy_ninja_ngimportfetch_proxies_jsonjson_list=fetch_proxies_json("https")print(json_list)==>Get proxies List#!/usr/bin/python3fromproxy_ninja_ngimportfetch_proxies_list_list=fetch_proxies_list(PROXY_TYPE)print(_list)PROXY_TYPE: https/socks4/socks5This will gonna return a list.Example:#!/usr/bin/python3fromproxy_ninja_ngimportfetch_proxies_listlst=fetch_proxies_list("https")print(lst)FeaturesFetch proxies from different url's and api's.Fast, using request lib.save output in txt or json format.User Friendly. :DNote:If you are interested in implementing this usingSelenium/ChromiumDriver, please check thisProjectContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.LicenseMITFeedbackIf you have any feedback, please reach out to us atakalucifr@protonmail.chChange Log0.1.0 (24/8/2022)initial release
Package: proxynova-scraper
proxynova_scraperGet free proxies for your projects fromproxinova.comGetting startedInstalling the packagepip install proxynova-scrapperUseYou have two ways to get the proxies:Get the latest onesGet proxies by countryGet the latest ones from different countriesIt will scrap themain pagefrom proxynovafromproxynova_scraperimportget_proxiesproxies=get_proxies()Get proxies by countryIt will scrapproxynova by specifying the country(2 char code), if the table from proxynova doesn't contain any proxy it will return an empty arrayfromproxynova_scraperimportget_proxies_by_countrymexico_proxies=get_proxies_by_country('mx')Proxy itemThis is an example of the content returned byget_proxiesorget_proxies_by_country{'proxyIp':'45.174.77.241','proxyPort':'999','lastCheck':'2022-07-23 22:32:03Z','proxySpeed':2735,'Uptime':'5114','proxyCountry':'Mexico - Chihuahua City','anonymity':'Transparent'}
Package: proxynt
中文ProxyNT is a reverse proxy server that can expose a local server to the internet through NATs and firewallsFeaturesOpen port mapping via browser anytime and anywhereEncrypted transmission between public network server and local network client via WebSocketFew dependencies, one-click installation via pipStable, automatically reconnect, and already in production environmentSupport for rate limitingCommon ScenariosHosting website server at homeManaging IoT devicesInstallationpip install -U proxyntExample: Accessing an Internal Network Machine via SSHSuppose the public server's IP is 192.168.9.224.1. Createconfig_s.jsonfile on the public network machineconfig_s.jsoncontent:{"port":18888,"password":"helloworld","path":"/websocket_path","admin":{"enable":true,"admin_password":"new_password"}}Then start:nt_server -c config_s.jsonExplanation:port: Listening portpassword: Connection passwordpath: WebSocket pathadmin: Management page configuration (optional)admin.enable: Whether to enable management pageadmin.admin_password: Management password2. Createconfig_c.jsonfile on the local network computer that needs to be accessedconfig_c.jsoncontent:{"server":{"port":18888,"host":"192.168.9.224","https":false,"password":"helloworld","path":"/websocket_path"},"client_name":"home_pc","client":[]}Then start:nt_client -c config_c.jsonExplanation:server: Configuration of the server to be connected, including port, IP address, whether to use HTTPS, password, and WebSocket path.client_name: Client name, needs to be unique.client: Empty array.3. Open the server webpagehttp://192.168.9.224:18888/websocket_path/adminand add a port:Explanation: The management page path isWebSocket path + /admin.4. Configuration succeeded, use SSH to connect:ssh-oPort=12222test@192.168.9.224Complete Configuration Instructions (please delete the comments when using)Client config_c.json{"server":{// Server configuration to connect to"port":18888,// Port"host":"192.168.9.224",// IP address"https":false,// Whether the server is using HTTPS"password":"helloworld",// Password"path":"/websocket_path"// WebSocket path},"client":[// List of forwarding configurations{"name":"ssh","remote_port":1222,"local_port":22,"local_ip":"127.0.0.1"},{"name":"mongo","remote_port":1223,"local_port":27017,"local_ip":"127.0.0.1"}],"client_name":"ubuntu1",// Client name, must be unique"log_file":"/var/log/nt/nt.log"// Path to log file}Server config_c.json{"port":18888,// Listening port"password":"helloworld",// Password"path":"/websocket_path",// WebSocket path"log_file":"/var/log/nt/nt.log",// Path to log file"admin":{"enable":true,// Whether to enable admin page"admin_password":"new_password"// Password for admin page}}Stargazers over timeUpdate history1.1.9: Bandwidth limitation1.1.8: Display client version1.1.7: Fixed duplicate client_name1.1.6: Fixed client WebSocketException: socket is already opened.
Package: proxyorderedset
(Proxy-) OrderedSetA simple implementation of ordered sets as a proxy to Python's standard dict class.The implementation is based on the idea to take abaseiterable and create a dict usingdict.fromkeys(base). Keys
are unique, and in newer versions of Python, the order is kept; values areNoneand ignored.This package has no external dependencies. The OrderedSet class overwrites allsetmethods.ExamplefromorderedsetimportOrderedSets:OrderedSet[int]=OrderedSet([3,1,4,1])list(s)# yields [3, 1, 4]
Package: proxy-parse
Proxy ParserAboutSynchronous library, for convenient and fast parsing of proxies from different sources.Uses Scrapy as a parser.At the moment the library does not support automatic proxy check, this option will be added in the asynchronous version of the library.InstallationInstalling the latest version of the library:pipinstallproxy-parseExamplefromproxy_parseimportProxyParserproxy_parser=ProxyParser()proxies_list=proxy_parser.parse()If you need, you can add some parameters to the ProxyParser class:path_to_file - optional str parameter, the proxies will be saved to a file at the pathproxy_limit - optional int parameter, the ProxyParser.parse function will return as many proxies as you needscrapy_spiders - optional scrapy.Spider list parameter, you can add your own spiders, which will work together with the othersscrapy_settings - optional dict parameter, you can replace the library rules with your ownContributionAny changes from you will be good for the life of the library
Package: proxyparser-checker
ProxyParserProxyParseris a library for checking/parsing proxies.>>>fromProxyParserimportProxyParser>>>>>>working_proxies=[]>>>proxies=ProxyParser.parseProxy()>>>>>>forproxyinproxies:>>>ifProxyParser.checkProxy(proxy):>>>working_proxies.append(proxy)>>>print("Working proxies:")>>>print(working_proxies)Workingproxies:['101.132.25.152:5678','120.196.188.21:9091']Installing ProxyParserProxyParser is available on PyPi:$python-mpipinstallProxy-ParserGitHub
Package: proxy-pool-redis
proxy_pool_redis 使用redis作为后端存储的ip代理池能够根据对应付费ip的请求Url加载对应的Proxy ip到池中，并能够提供多个应用使用，对应每个应用需要传入特定的应用名字做标示，可对特定的应用使用的Ip报告被ban,质量差的ip使用方法fromproxy_pool_redisimportXunProxyPoolpool=XunProxyPool(api_url='xxx',name='crawl_name',redis_host='xx',redis_port=xx,redis_password='xx',report_num=10)pool.start()# 获取一个ipip=pool.get_ip()# 报告ip已经被当前应用ban了pool.report_ban_ip(ip)# 报告当前ip的质量不好# 如果报告的次数达到了report_num的值，则将该ip ban掉pool.report_bad_ip(ip)
Package: proxyport
Proxy Port SDKProxyport Python package provides interfaces to theProxy PortAPI.InstallationInstall viapip:$pipinstallproxyportGetting StartedPackage can be used as module:$python-mproxyport

AR181.119.301.272:3128
...
UA194.242.310.289:3128
US162.144.277.318:3838Example:fromproxyportimportget_random_proxyprint(get_random_proxy())http://139.180.281.313:3128
Package: proxyport2
Proxy Port SDKproxyport2Python package provides interfaces to theProxy PortAPI.PrerequisitesTo use this package you will need a free API key. Get your API keyhere.
Detailed instructionshere.InstallationInstall viapip:$pipinstallproxyport2Getting StartedBefore you get your first proxy, you need to assign an API key.
This can be done either through an environment variable$exportPROXY_PORT_API_KEY=<API_KEY>or directly in the code.fromproxyport2importset_api_key,get_proxyset_api_key('<API_KEY>')# hereprint(get_proxy())'139.180.281.313:3128'Package can be used as module:$python-mproxyport2['181.119.301.272:3128',...]
Package: proxypypi
WarningDon’t use this software. Usedevpiinstead!Use proxypypi to set up a transparent caching proxy to PyPI
(https://pypy.python.org/) to:make your installs res/ilient against Internet/PyPI issues,speed up your installs significantly (after the first one),prevent problems installing packages that are removed from distribution by
the author,allow installation of packages from within a firewalled environment where
the host performing the installation does not have Internet access, andallow hosting and installation of private packages.When the proxy is asked about a package it doesn’t know it automatically goes
off and fetches the file download list for the package, rewriting all
references (PyPI and external) so they appear to be local. On request of one
of those now-local package file references it performs a background fetch of
the file contents and serves up the new file data to the pip request (thus
keeping that request alive despite its very short timeout duration).Why Another One?There’s a lot of PyPI-alike implementations and “proxy” servers out there.
This proxy differs from almost all others (save devpi-server) in that it
automatically relocates package download files from the Internet to your local
server. It differs from devpi-server in that it has no external database
requirement (just a filesystem is needed) making deployment much simpler (at
least for me).UsageSetup of proxypypi requires:“pip install proxypypi”cd to some directory you would like files to be cached inproxypypi runThere are some command-line arguments (see “proxypypi -h”) that allow some
run-time behaviour control. Notably “-d” which can configure the directory
to cache in.Additionally the proxy may run in the background, in which case you need to
supply three additional command-line arguments: “-P” (PID file), “-l” (log
file) and “-o” (console output file) and use one of the daemon control commands
instead of “run”. See “proxypypi -h” for the commands.Once running you may perform pip installs using the proxy by including the
proxy in the pip command line using the “-i” argument:pip install -ihttp://proxy_host:proxy_port/simple/package_to_installAny packages not cached will be fetched into the cache and the install will
continue as normal.Private (or Manually Downloaded) PackagesPackage distribution files added to the root of the package directory served
by proxypypi will be served alongside those it proxies requests for. You may,
if you wish, make package-named subdirectories, but it’s not necessary.Upgrading Cached PackagesThis isn’t done yet :-)Version History1.0 first public releae1.0.1 clear up handling of paths to daemon files1.2 handle external files that don’t exist (issue 1, thanks George Hickman)1.2.1 fix a name collision issue1.2.2 remove partial download for missing external file
Package: proxy.py-uxspoilers-plugin
proxy.py-uxspoilers-pluginA plugin forproxy.pyto spoil UX of web browsingYou can spoil User Experience of e.g. YouTube with a combination ofproxy.pyplusproxy.py-uxspoilers-pluginand a kind of followingPAC/WPADfile.functionFindProxyForURL(url,host){if(shExpMatch(host,"*youtube*")||shExpMatch(host,"*.googlevideo.com")){return"PROXY ip-of-your-proxy:8899";}return"DIRECT";}Run your proxy server like this:proxy --hostname=0.0.0.0 --pac-file /path/to/wpad.dat --plugin uxspoilers.FixedRustyPumpPlugin --pause-seconds 3Your/etc/dhcp/dhcpd.confshould look like this:option wpad code 252=text;host target-device {option wpad "http://ip-of-your-proxy:8899/wpad.dat";hardware ethernet 2e:8d:8d:xx:xx:xx;}Or you may configure your browser'sPACwithhttp://ip-of-your-proxy:8899/wpad.datmanually.Installationpip3 install proxy.py-uxspoilers-pluginproxy --help --plugin uxspoilers.RandomRustyPumpPluginDockerdocker run -it sakuraiyouhei/proxy.py-uxspoilers-plugin --help --plugin uxspoilers.RandomRustyPumpPlugin