Saravanan666/pypi-org huggingface dataset:
Package: urlfetch
urlfetch is a simple, lightweight and easy to use HTTP client for Python.
It is distributed as a single file module and has no depencencies other than the Python Standard Library.HighlightsDistributed as a single file module, has no depencencies other than the Python Standard Library.Pure Python, works fine withgevent.Small codebase, about 1000 lines and 30% are comments and blank lines. Only 10 minutes you can know every detail.Random user-agent.Support streaming.Installation$ pip install urlfetchHello, worldimporturlfetchresponse=urlfetch.get('http://python.org/')printresponse.status,response.reasonprintlen(response.content)Uploading filesimporturlfetchresponse=urlfetch.post('http://127.0.0.1:8888/upload',headers={'Referer':'http://127.0.0.1/',},files={'fieldname1':open('/path/to/file','rb'),#'fieldname2': 'file content', # file must have a filename'fieldname3':('filename',open('/path/to/file2','rb')),'fieldname4':('filename','file content'),},data={'foo':'bar'},)printresponse.status,response.contentTestingTo run the tests, urlfetch relies onbottleandgunicorn.
If the tests are run by callingpython setup.py test,  the
dependencies will be handled automatically (viatests_require).
So, if you want to run the tests directly, that is,python tests/testall.py, make sure bottle and gunicorn are installed
under the PYTHONPATH.
Package: urlfile
urlfileLazily reading a file from a url.
Allows random access via http range requests so that not the whole file has to be downloaded first.Example Usageimporturlfileimportzipfilewithzipfile.ZipFile(urlfile.UrlFile(url=...))asf:f.printdir()CachingBuffering and caching is provided viaurlfile.BufferedUrlFile.f=urlfile.BufferedUrlFile(url=...,cache_size_bytes=...)By default, uses a cache size of10MB.Other optionsThese are arguments ofUrlFile/BufferedUrlFileverbose: whether to show progress bars during fetching of data (usingtqdm, default:False)session: arequests.Sessionto use (default:None, creates a new session)
Package: url-finder
url-finderA small python module to detect all real links in a given string.Exampleimporturl_findertext="""Picking out google.com is trivial. As well as youtube.com,and even knows what is a real tld, and what is not.a.real.top-level-domain!"""asserturl_finder.get_urls(text)==["https://google.com","https://youtube.com"]Installpip install url-finder
Package: urlfinderlib
urlfinderlibThis is a Python (3.6+) library for finding URLs in documents and checking their validity.Supported DocumentsExtracts URLs from the following types of documents:Binary files (finds URLs within strings)CSV filesHTML filesiCalendar/vCalendar filesPDF filesText files (ASCII or UTF-8)XML filesEvery extracted URL is validated such that it contains a domain with a valid TLD (or a valid IP address) and does not contain any invalid characters.URL PermutationsThis was originally written to accommodate finding both valid and obfuscated or slightly malformed URLs used by malicious actors and using them as indicators of compromise (IOCs). As such, the extracted URLs will also include the following permutations:URL with any Unicode characters in its domainURL with any Unicode characters converted to its IDNA equivalentFor both domain variations, the following permutations are also returned:URL with its path %-encodedURL with its path %-decodedURL with encoded HTML entities in its pathURL with decoded HTML entities in its pathURL with its path %-decoded and HTML entities decodedChild URLsThis library also attempts to extract or decode child URLs found in the paths of URLs. The following formats are supported:Barracuda protected URLsBase64-encoded URLs found within the URL's pathGoogle redirect URLsMandrill/Mailchimp redirect URLsOutlook Safe Links URLsProofpoint protected URLsURLs found in the URL's path query parametersBasic usagefrom urlfinderlib import find_urls

with open('/path/to/file', 'rb') as f:
    print(find_urls(f.read())base_url ParameterIf you are trying to find URLs inside of an HTML file, the paths in the URLs are often relative to their location on the server hosting the HTML. You can use thebase_urlparameter in this case to extract these "relative" URLs.from urlfinderlib import find_urls

with open('/path/to/file', 'rb') as f:
    print(find_urls(f.read(), base_url='http://example.com')
Package: url-for-int
url_for_Intv2.3.0简介这是 url-for Interpreter 的开源使用版，遵循 MIT 协议。开发者：HifiveGithub:https://github.com/Hifive55555/url_for_IntPypi:https://pypi.org/project/url-for-int它总共分为两个部分：GUI 和 API，使用方法见下新版内容2.2.4 修复 ui 数组 bug，增加逆转函数 omit_int2.3.0 修复 omit_int 函数 bug，完善逆转义功能并添加到GUI上使用方法API函数及类int_main 主函数输入：origin_text原始字符返回：x:str处理后的字符file_catcher ：导入文件处理输入：origin_file,to_path,prefix,if_preto_path默认为None，表示生成于原文件目录。若指定目录，则会加入pre_folderprefix默认为"int_"if_pre默认为False。若为True，则文件目录加上pre_folder返回：bool值，表示是否成功omit_int ：删除所有url_forGUI ：用户界面GUI是一个继承了tk.Tk的窗口类教程安装：pip install url_for_int导入库：from url_for_int import start_gui, file_catcher, int_main启用GUI界面：start_gui()或直接运行 __init__.py 文件转义一个文件转录到当前文件夹：file_catcher("test.html")转录到指定文件夹：file_catcher("test.html", "D:/test")转录并覆盖原文件：file_catcher("test.html", prefix="")转录并指定前缀：file_catcher("test.html", prefix="xxx-")转录并增加pre_folder：file_catcher("test.html", "D:/test", if_pre=True)逆转录一个文件同上理，不过要加入参数if_omit=True转义/逆转录一串字符content="xxx"after_1=int_main(content)# 转义after_2=omit_int(content)# 逆转义
Package: urlformat
urlformatURL formatter for web addresses written in PythonThis URL parsing/building class can handle any and all correctly formed URL's. Feel free to use any of the code and if you have any improvements please feel free to fork this repo. and push updates to me. Thank you.It should be noted that I try to maintain a high level of PEP codestyle and linting guidelines. I used the PIP package manager with the pylint, pycodestyle and flake8 packages.Installation:python3 pip -m install urlformatorpip install urlformatUsage:importurlformatx=urlformat.UrlParser("https://www.google.com")x=x.get_scheme()print(x)Output:https
Package: url-for-s3
Python function that generates a URL to a given S3 resource.Exampleurl_for_s3('static',bucket_name='my-cool-foobar-bucket',scheme='https',filename='pics/logo.png')Will return:https://my-cool-foobar-bucket.s3.amazonaws.com/static/pics/logo.pngNote: this function assumes that the given resource exists on S3 and is publicly accessible.Based loosely on Flask-S3’surl_for().For a complete, working Flask app that demonstrates url-for-s3 in action, have a look atflask-s3-save-example.
Package: urlimage
Infer an image for a given URLWhat this does:Given the URL of a website (e.g. "https://www.mozilla.org"), it attempts to find the most appropriate image that represents this site, and returns the URL of that image.How it works:The HTML of the page is parsed (BeautifulSoup) and meta/link tags are extracted which may contain clues as to which image to use. Failing this, the favicon is tried, or else it searches for the first img tag following the first h1 on the page.The default search order is:DEFAULT_CHECK_ORDER = [
    SCHEMA_IMAGE,
    OG_IMAGE,
    TWITTER_IMAGE,
    MS_WIDE_310,
    MS_SQUARE_310,
    ICON,
    APPLE_TOUCH_ICON,
    MS_SQUARE_150,
    MS_SQUARE_70,
    FAV_ICON,
    FIRST_IMAGE,
]Which corresponds to:meta tag with itemprop="image" or property="image"meta tag with property="og:image:secure_url" or property="og:image"meta tag with name="twitter:image"meta tag with name="msapplication-wide310x150logo"meta tag with name="msapplication-square310x310logo"link tag with rel="apple-touch-icon" (if 'sizes' is provided, the link tag with the largest minimum dimension is selected)link tag with rel="icon" (if 'sizes' is provided, the link tag with the largest minimum dimension is selected)meta tag with name="msapplication-square150x150logo"meta tag with name="msapplication-square70x70logo"requesting "{scheme}://{domain}/favicon.ico" to see if it existsthe first img tag siblings with an h1 tag.
Package: urlimage-dcollien
Infer an image for a given URLWhat this does:Given the URL of a website (e.g. "https://www.mozilla.org"), it attempts to find the most appropriate image that represents this site, and returns the URL of that image.How it works:The HTML of the page is parsed (BeautifulSoup) and meta/link tags are extracted which may contain clues as to which image to use. Failing this, the favicon is tried, or else it searches for the first img tag following the first h1 on the page.The default search order is:DEFAULT_CHECK_ORDER = [
    SCHEMA_IMAGE,
    OG_IMAGE,
    TWITTER_IMAGE,
    MS_WIDE_310,
    MS_SQUARE_310,
    ICON,
    APPLE_TOUCH_ICON,
    MS_SQUARE_150,
    MS_SQUARE_70,
    FAV_ICON,
    FIRST_IMAGE,
]Which corresponds to:meta tag with itemprop="image" or property="image"meta tag with property="og:image:secure_url" or property="og:image"meta tag with name="twitter:image"meta tag with name="msapplication-wide310x150logo"meta tag with name="msapplication-square310x310logo"link tag with rel="apple-touch-icon" (if 'sizes' is provided, the link tag with the largest minimum dimension is selected)link tag with rel="icon" (if 'sizes' is provided, the link tag with the largest minimum dimension is selected)meta tag with name="msapplication-square150x150logo"meta tag with name="msapplication-square70x70logo"requesting "{scheme}://{domain}/favicon.ico" to see if it existsthe first img tag siblings with an h1 tag.
Package: urlimport
[ urlimport.py ]Enables remote module importing through urllib2 and perforce.author:    Jure Vrscaj <jure@codeshift.net>last contributed versions:version:   0.72c (myevan <myevan_net@naver.com> fixes and improvements)version:   0.85b (alex <alexbodn@012.net.il> fixes and improvements)version:   0.72d (jure <jure@codeshift.net> found a bug and alex fixed it).homepage:  http://urlimport.codeshift.netlicense:   MITdownload: all the project's files may be downloaded from bitbucket with hg:hg clone http://bitbucket.org/alexbodn/urlimportUsage:import sys, urlimporturlimport.config(**{'ssl_key.https://your.url':'key for this url''ssl_cert.https://your.url':'cert for this url','opener.https://your.url':'opener for this url','cache_time.http://your.url':'timedelta to keep files in cache for this url','no_cache.url':True, # don't cache modules from this url'user_agent.url':'user agent header for this url','ssl_key.host':'key for this host''ssl_cert.host':'cert for this host','opener.host':'opener for this host','cache_time.host':'timedelta to keep files in cache for this host','no_cache.host':True, # don't cache modules from this host'user_agent.host':'user agent header for this host','ssl_key':'key for unspecified urls','ssl_cert':'cert for unspecified urls','opener':'opener for unspecified urls','no_cache':'True, #don't cache modules for unspecified urls'user_agent':'user agent header for unspecified urls','cache_time':'timedelta to keep files in cache for unspecified urls',#if cache_time is missing, keep forever, but check last_modified and etag'cache_dir':'root of the cache dir', # if missing, a temp dir will be created,# if empty, no cache will be used'py_version_string': '$PYTHON_VERSION', # string in path to be replaced by# $major.$minor'debug': int(level of output text. see debug() function)})# note: if you supply your own opener, i'd recommand inheriting/implementing the# status handling features from class DefaultErrorHandler below.sys.path.insert(0, "http://your.url") #this may already be there, like setting#PYTHONPATH#but there is a danger default import#would try it beforeexpect to have access to your modules at http://your.urlTODO:- check what's mpath all about.TODO later:- python 3.1 support. preliminary work done- url import of zipped modules- support of other urls, like ssh, svn, hg, bzr, cvs etc.- reload c extensions. well, they don't allways reload in local import either,but my module does reload locally, so why not remotely?introspection will help.- swithch the usage of debug with some logging, and do some cleanup of thecalls.
Package: url-into-images
Automatically  Download Multiple Images.Package Installation :pip install url-into-imagesUrl Into Images :provide image download link through csv file and it will download all the images. also you can chose different extension.• jpg

 • jpeg

 • png

 • bmp

 • gifHow to import the module:URL_PATH=('')# csv file where image URL containIMAGE_PATH=('')# Download image pathDownload JPGfromurl_into_images.url_into_imagesimporturl_jpgurl_jpg(URL_PATH,IMAGE_PATH)Download JPEGfromurl_into_images.url_into_imagesimporturl_jpegurl_jpeg(URL_PATH,IMAGE_PATH)Download PNGfromurl_into_images.url_into_imagesimporturl_pngurl_png(URL_PATH,IMAGE_PATH)Download BMPfromurl_into_images.url_into_imagesimporturl_bmpurl_bmp(URL_PATH,IMAGE_PATH)Download GIFfromurl_into_images.url_into_imagesimporturl_gifurl_gif(URL_PATH,IMAGE_PATH)Required package’s:• pip install pandasLicense:MIT LicensedAuthor:Sujit MandalGitHubPyPiLinkedIn
Package: urllib3
urllib3 is a powerful,user-friendlyHTTP client for Python. Much of the
Python ecosystem already uses urllib3 and you should too.
urllib3 brings many critical features that are missing from the Python
standard libraries:Thread safety.Connection pooling.Client-side SSL/TLS verification.File uploads with multipart encoding.Helpers for retrying requests and dealing with HTTP redirects.Support for gzip, deflate, brotli, and zstd encoding.Proxy support for HTTP and SOCKS.100% test coverage.urllib3 is powerful and easy to use:>>>importurllib3>>>resp=urllib3.request("GET","http://httpbin.org/robots.txt")>>>resp.status200>>>resp.datab"User-agent: *\nDisallow: /deny\n"Installingurllib3 can be installed withpip:$python-mpipinstallurllib3Alternatively, you can grab the latest source code fromGitHub:$gitclonehttps://github.com/urllib3/urllib3.git
$cdurllib3
$pipinstall.Documentationurllib3 has usage and reference documentation aturllib3.readthedocs.io.Communityurllib3 has acommunity Discord channelfor asking questions and
collaborating with other contributors. Drop by and say hello 👋Contributingurllib3 happily accepts contributions. Please see ourcontributing documentationfor some tips on getting started.Security DisclosuresTo report a security vulnerability, please use theTidelift security contact.
Tidelift will coordinate the fix and disclosure with maintainers.Maintainers@sethmlarson(Seth M. Larson)@pquentin(Quentin Pradet)@illia-v(Illia Volochii)@theacodes(Thea Flowers)@haikuginger(Jess Shapiro)@lukasa(Cory Benfield)@sigmavirus24(Ian Stapleton Cordasco)@shazow(Andrey Petrov)👋SponsorshipIf your company benefits from this library, please considersponsoring its
development.For EnterpriseProfessional support for urllib3 is available as part of theTidelift
Subscription.  Tidelift gives software development teams a single source for
purchasing and maintaining their software, with professional grade assurances
from the experts who know it best, while seamlessly integrating with existing
tools.
Package: urllib3-ext-hface
PyPIThis project is a fork of the akamai/hface project but highly slimmed.
The purpose of that project is to enable basic support for HTTP/1.1, HTTP/2 and HTTP/3 in urllib3.HTTP/1.1, HTTP/2, and HTTP/3 support through respectively h11, h2 and aioquicSans-IOcore with pluggable protocol implementationsLayered design with well-defined APIsClient-oriented onlyDocumentationThis library is pretty straight forward and is very easy to understand.fromurllib3_ext_hfaceimportHTTPProtocolFactory,HTTP1Protocol,HTTP2Protocol,HTTP3Protocolprotocol=HTTPProtocolFactory.new(HTTP2Protocol)# ...protocol.bytes_to_send()# Out to your network I/O implementation# ...protocol.bytes_received()# Whatever comes from your network I/O implementation# ...protocol.next_event()# Output the next event in order that the state-machine generatedHereprotocolis either of typeHTTPOverQUICProtocolorHTTPOverTCPProtocol, depending
on the foremost, you will either be on a DGRAM (UDP) or STREAM (TCP) socket.Find just bellow a somewhat stupid example that can help you get started.fromurllib3_ext_hfaceimportHTTPProtocolFactory,HTTP1Protocol,HTTP2Protocol,HTTP3ProtocolimportsslimportsocketimportcertifiRECV_LENGTH=4096if__name__=="__main__":ctx=ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)ctx.set_alpn_protocols(["h2"])ctx.load_default_certs()ctx.load_verify_locations(certifi.where())sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)sock=ctx.wrap_socket(sock,server_hostname="httpbin.org")sock.connect(("httpbin.org",443))sock.settimeout(3)protocol=HTTPProtocolFactory.new(HTTP2Protocol)# Start HTTP2*PRIwhileTrue:data_out=protocol.bytes_to_send()ifnotdata_out:breaksock.sendall(data_out)protocol.bytes_received(sock.recv(RECV_LENGTH))stream_id=protocol.get_available_stream_id()protocol.submit_headers(stream_id,[(b":authority",b"httpbin.org"),(b":scheme",b"https"),(b":path",b"/headers"),(b":method",b"GET"),(b"User-Agent",b"Awesome Sans-IO!")],True)sock.sendall(protocol.bytes_to_send())whileTrue:try:protocol.bytes_received(sock.recv(RECV_LENGTH))exceptTimeoutError:protocol.connection_lost()event=protocol.next_event()ifhasattr(event,"data"):print(event.data)print(event)ifhasattr(event,"end_stream")andevent.end_streamisTrue:breakprotocol.submit_close()sock.sendall(protocol.bytes_to_send())sock.close()LicenseCopyright 2022 Akamai Technologies, Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Package: urllib3-future
urllib3.future is as BoringSSL is to OpenSSL but to urllib3 (except support is available!)✨🍰 Enjoy HTTP like its 2024 🍰✨💰 Promotional offer, get everything and more for40k0$!⚡ urllib3.future is a powerful,user-friendlyHTTP client for Python.⚡ urllib3.future goes beyond supported features while remaining compatible.⚡ urllib3.future brings many critical features that are missing from both the Python standard librariesand urllib3:Async.Task safety.Thread safety.Connection pooling.Client-side SSL/TLS verification.Highly customizable DNS resolution.File uploads with multipart encoding.DNS over UDP, TLS, QUIC, or HTTPS. DNSSEC protected.Helpers for retrying requests and dealing with HTTP redirects.Support for gzip, deflate, brotli, and zstd encoding.HTTP/1.1, HTTP/2 and HTTP/3 support.Proxy support for HTTP and SOCKS.Detailed connection inspection.Multiplexed connection.Mirrored Sync & Async.urllib3.future is powerful and easy to use:>>>importurllib3>>>pm=urllib3.PoolManager()>>>resp=pm.request("GET","https://httpbin.org/robots.txt")>>>resp.status200>>>resp.datab"User-agent: *\nDisallow: /deny\n">>>resp.version20or using asyncio!importasyncioimporturllib3asyncdefmain()->None:asyncwithurllib3.AsyncPoolManager()aspm:resp=awaitpm.request("GET","https://httpbin.org/robots.txt")print(resp.status)# 200body=awaitresp.dataprint(body)# # b"User-agent: *\nDisallow: /deny\n"print(resp.version)# 20asyncio.run(main())Installingurllib3.future can be installed withpip:$python-mpipinstallurllib3.futureYou either doimporturllib3Or...importurllib3_futureDoingimport urllib3_futureis the safest option for you as there is a significant number of projects that
requireurllib3.NotesIt's a fork⚠️ Installing urllib3.future shadows the actual urllib3 package (depending on installation order).
The semver will always be likeMAJOR.MINOR.9PPlike 2.0.941, the patch node is always greater or equal to 900.Support for bugs or improvements is served in this repository. We regularly sync this fork
with the main branch of urllib3/urllib3 against bugfixes and security patches if applicable.OS Package ManagersFellow OS package maintainers, you cannotjustbuild and ship this package to your package registry.
As it overrideurllib3and due to its current criticality, you'll have to set:URLLIB3_NO_OVERRIDE=true python -m build. SetURLLIB3_NO_OVERRIDEvariable with "true" in it.It will prevent the override.Compatibility with downstreamYou shouldalwaysinstall the downstream project prior to this fork.e.g. I wantrequeststo be use this package.python -m pip install requests
python -m pip install urllib3.futureNowadays, we suggest using the packageNiquestsas a drop-in replacement forRequests.
It leverages urllib3.future capabilities appropriately.Documentationurllib3.future has usage and reference documentation aturllib3future.readthedocs.io.Contributingurllib3.future happily accepts contributions.Security DisclosuresTo report a security vulnerability, please use the GitHub advisory disclosure form.SponsorshipIf your company benefits from this library, please consider sponsoring its
development.
Package: urllib3-mock
A utility library for mocking out theurllib3Python library.This is an adaptation of theresponseslibrary.Response body as stringfromurllib3_mockimportResponsesimportrequestsresponses=Responses('requests.packages.urllib3')@responses.activatedeftest_my_api():responses.add('GET','/api/1/foobar',body='{"error": "not found"}',status=404,content_type='application/json')resp=requests.get('http://twitter.com/api/1/foobar')assertresp.json()=={"error":"not found"}assertlen(responses.calls)==1assertresponses.calls[0].request.url=='/api/1/foobar'assertresponses.calls[0].request.host=='twitter.com'assertresponses.calls[0].request.scheme=='http'Request callbackimportjsonfromurllib3_mockimportResponsesimportrequestsresponses=Responses('requests.packages.urllib3')@responses.activatedeftest_calc_api():defrequest_callback(request):payload=json.loads(request.body)resp_body={'value':sum(payload['numbers'])}headers={'request-id':'728d329e-0e86-11e4-a748-0c84dc037c13'}return(200,headers,json.dumps(resp_body))responses.add_callback('POST','/sum',callback=request_callback,content_type='application/json')resp=requests.post('http://calc.com/sum',json.dumps({'numbers':[1,2,3]}),headers={'content-type':'application/json'},)assertresp.json()=={'value':6}assertlen(responses.calls)==1assertresponses.calls[0].request.url=='/sum'assertresponses.calls[0].request.host=='calc.com'assert(responses.calls[0].response.headers['request-id']=='728d329e-0e86-11e4-a748-0c84dc037c13')Instead of passing a string URL intoresponses.addorresponses.add_callbackyou can also supply a compiled regular expression.importrefromurllib3_mockimportResponsesimportrequestsresponses=Responses('requests.packages.urllib3')# Instead ofresponses.add('GET','/api/1/foobar',body='{"error": "not found"}',status=404,content_type='application/json')# You can do the followingurl_re=re.compile(r'/api/\d+/foobar')responses.add('GET',url_re,body='{"error": "not found"}',status=404,content_type='application/json')A response can also throw an exception as follows.fromurllib3_mockimportResponsesfromrequests.packages.urllib3.exceptionsimportHTTPErrorexception=HTTPError('Something went wrong')responses=Responses('requests.packages.urllib3')responses.add('GET','/api/1/foobar',body=exception)# All calls to 'http://twitter.com/api/1/foobar' will throw exception.
Package: urllibcache
This is a simple caching handler for urllib2.UsageEither use the build_opener function which takes the path to the cache folder
and returns an OpenerDirector instance. You can then use it’s open method to
open urls normally. If the file was already downloaded, then it will be taken
from the cache.You can also use the CachedHandler wherever it’s useful within urllib2. This
also takes the path to cache folder as an argument of the constructor.Be aware that the caching is very simple. It detects incomplete downloads but
doesn’t take any caching headers into account. To invalidate the cache you
have to delete the files in the cache folder.Changelog1.0.1 - 2008-07-23Fixed packaging.
[fschulze]1.0 - 2008-07-22Initial release.
[fschulze]