Saravanan666/pypi-org huggingface dataset:
Package: websockets
What iswebsockets?websockets is a library for buildingWebSocketservers and clients in Python
with a focus on correctness, simplicity, robustness, and performance.Built on top ofasyncio, Python’s standard asynchronous I/O framework, the
default implementation provides an elegant coroutine-based API.An implementation on top ofthreadingand a Sans-I/O implementation are also
available.Documentation is available on Read the Docs.Here’s an echo server with theasyncioAPI:#!/usr/bin/env pythonimportasynciofromwebsockets.serverimportserveasyncdefecho(websocket):asyncformessageinwebsocket:awaitwebsocket.send(message)asyncdefmain():asyncwithserve(echo,"localhost",8765):awaitasyncio.Future()# run foreverasyncio.run(main())Here’s how a client sends and receives messages with thethreadingAPI:#!/usr/bin/env pythonfromwebsockets.sync.clientimportconnectdefhello():withconnect("ws://localhost:8765")aswebsocket:websocket.send("Hello world!")message=websocket.recv()print(f"Received:{message}")hello()Does that look good?Get started with the tutorial!Why should I usewebsockets?The development ofwebsocketsis shaped by four principles:Correctness:websocketsis heavily tested for compliance withRFC 6455. Continuous integration fails under 100% branch coverage.Simplicity: all you need to understand ismsg = await ws.recv()andawait ws.send(msg).websocketstakes care of managing connections
so you can focus on your application.Robustness:websocketsis built for production. For example, it was
the only library tohandle backpressure correctlybefore the issue
became widely known in the Python community.Performance: memory usage is optimized and configurable. A C extension
accelerates expensive operations. It’s pre-compiled for Linux, macOS and
Windows and packaged in the wheel format for each system and Python version.Documentation is a first class concern in the project. Head over toRead the
Docsand see for yourself.Why shouldn’t I usewebsockets?If you prefer callbacks over coroutines:websocketswas created to
provide the best coroutine-based API to manage WebSocket connections in
Python. Pick another library for a callback-based API.If you’re looking for a mixed HTTP / WebSocket library:websocketsaims
at being an excellent implementation ofRFC 6455: The WebSocket Protocol
andRFC 7692: Compression Extensions for WebSocket. Its support for HTTP
is minimal — just enough for an HTTP health check.If you want to do both in the same server, look at HTTP frameworks that
build on top ofwebsocketsto support WebSocket connections, likeSanic.What else?Bug reports, patches and suggestions are welcome!To report a security vulnerability, please use theTidelift security
contact. Tidelift will coordinate the fix and disclosure.For anything else, please open anissueor send apull request.Participants must uphold theContributor Covenant code of conduct.websocketsis released under theBSD license.
Package: websockets-assistant
websockets-assistantHow to use itfrom websockets_assistant import client, run, sleep

async def hello(ws):
    await ws.send("hello")
    await sleep(1)
    await ws.send("websocket")
    await sleep(0.1)
    await ws.close()

# test 1
async def main():
    await asyncio.gather(
        client("wss://echo.websocket.org/", log, hello, True),
        client("wss://echo.websocket.org/", log, hello, True),
        client("wss://echo.websocket.org/", log, hello, True),
    )
run(main)

# test 2
def main():
    client("wss://echo.websocket.org/", log, hello, True),
    client("wss://echo.websocket.org/", log, hello, True),
run(main)
Package: websockets-groups
WebSockets GroupsМодуль реализующих менеджер групп WebSocket подключенийФункционал:Регистрация / Удаление WSСоздание / Удаление групп WSПодключение WS в группуПоддержка реестров: memory, redisQuick startУстановка:pipinstallwebsockets-groupsПодключение:fromfastapiimportWebSocketfromwebsockets_groupsimportWSGroupsManager,MemoryStorage,BaseDispatcherws_groups_manager=WSGroupsManager(MemoryStorage())classChatDispatcher(BaseDispatcher):pass@app.websocket('/chats/')asyncdefws_view(webdocket:WebSocket,chat_name:str):awaitws_groups_manager.register_ws(websocket,ChatDispatcher())
Package: websockets-proxy
webosckets_proxyThis module will enable you to usewebsocketspackage with proxies.Proxy heavy-lifting is done bypython-sockspackage.UsageTo install, use:pip install websockets_proxyAll you need is create a proxy object and pass it intoproxy_connectconstructor:fromwebsockets_proxyimportProxy,proxy_connectproxy=Proxy.from_url("...")asyncwithproxy_connect("wss://example.com",proxy=proxy)asws:...proxy_connectconstructor accepts the same arguments as regularconnectconstructor,
with the addition ofproxyandproxy_conn_timeout, which are self-explanatory.With this patch you cannot usesockkeyword argument, because it is used to connect to a proxy server.You must create yourProxyobjects within a running event loop(inside anasyncfunction).
This is apython-sockslimitation.
If you define yourProxyobjects outside of running event loop, you'll get this kind of error:RuntimeError: Task <Task pending name=...> attached to a different loop.WebSocket proxy checkerServerHere is a simpleaiohttpserver, which allows you to check, if your proxy connections work as expected:importasynciofromaiohttpimportweb,WSMsgType,WSMessageHOST='0.0.0.0'PORT=54321app=web.Application()asyncdefwebsocket_handler(request:web.Request)->web.StreamResponse:ws=web.WebSocketResponse()awaitws.prepare(request)awaitws.send_str(str(request.remote))awaitws.close()print('accepted connection from',request.remote)returnwsapp.add_routes([web.get('/',websocket_handler)])asyncdefmain():runner=web.AppRunner(app)awaitrunner.setup()site=web.TCPSite(runner,host=HOST,port=PORT)awaitsite.start()print('server is running')awaitasyncio.Future()if__name__=='__main__':asyncio.run(main())ClientAn example of a client-side proxy checker script:importasynciofromwebsockets_proxyimportProxy,proxy_connectfromwebsocketsimportconnect# this script is written with the above checker server in mindCHECKER_URL='ws://address:port'asyncdefmain():asyncwithconnect(CHECKER_URL)asws:asyncformsginws:ip_no_proxy=msgprint("Your IP:",ip_no_proxy)print('.')# be sure to create your "Proxy" objects inside an async functionproxy=Proxy.from_url("socks5://username:password@address:port")asyncwithproxy_connect(CHECKER_URL,proxy=proxy)asws:asyncformsginws:ip_with_proxy=msgprint("Proxy IP",ip_with_proxy)print('.')if__name__=="__main__":asyncio.run(main())
Package: websockets-routes
websockets-routeswebsocketsdoes not do routing, and I don't like Sanic, so I rolled my own.Routing backed byRoutes.UsageDecorate your handlers by path, and serve the router.importasyncioimportwebsocketsimportwebsockets_routesrouter=websockets_routes.Router()@router.route("/thing/")asyncdefthing_list(ws,path):...start_server=websockets.serve(router.handle,...)loop=asyncio.get_event_loop()loop.run_until_complete(start_server)loop.run_forever()By default, connections are closed immediately with 4040 if the URL does not match any of the registered routes.Block connections during handshakeThe router has its ownserve()method that overrides theprocess_request()hook, making the server return an HTTP 404 during the handshake phase instead:start_server=router.serve(...)This way, a non-matching client never connects to the websocket handler at all.The override is implemented via a customWebSocket protocol, so you can
subclass that if you need further customisation:classMyProtocol(websockets_routes.Protocol):...start_server=websockets.serve(router,...,create_protocol=MyProtocol,...,)Access route parametersThe handler's second parameter is aRoutedPathinstead of a plainstr. This is astrsubclass, so you can do anything you could as inwebsockets. There is one additional attribute,params, that allows you to access the matched route parameters.@router.route("/thing/{id}")asyncdefthing_detail(ws,path):# Asumming this is accessed with "/thing/123".awaitws.send(path)# This sends a text frame "/thing/123".awaitws.send(path.params["id"])# Text frame "123".Per-view handshake hooksDecorate a class to provide per-view validation and additional processing:importhttp@router.route("/thing/{id}")classThingDetail:asyncdefprocess_request(self,path,headers):thing_id=path.params["id"]thing=get_thing_or_none(thing_id)ifthingisnotNone:# Pass additional context to handle().path.context["thing"]=thingreturnNonemessage=f"thing{thing_id!r}not found\n"return(http.HTTPStatus.NOT_FOUND,[],message.encode("utf-8"))asyncdefhandle(self,ws,path):"""Now this is only called if thing is found."""thing=path.context["thing"]# Retrieve the context to use.
Package: websocket-training
Installationpip install websocket-trainingUsageimport uvicorn
from fastapi import FastAPI

from websocket_training.ws_router import connect_ws, get_ws_manager, ConnectionManager

app = FastAPI()
connect_ws(app)

manager: ConnectionManager = get_ws_manager()
print(manager.count())


if __name__ == '__main__':
    uvicorn.run('main:app')
Package: websockify-LibvirtDomain
InfoWebsockify token plugin for Libvirt domains, token is the Libvirt domain uuid.
This plugin lookups the domain xml using the Libvirt api and gets the vnc port.UsageSee thewebsockify wikiThe--token-sourceoption in Websockify can be used to specify a Libvirt Uri, without this option specified the uri isqemu:///systemExample UsageRuns a Websockify deamon for noVNC using this pluginRun a server on port 6080:websockify --web=/usr/share/novnc/ 6080 --token-plugin websockify_LibvirtDomain.LibvirtDomainRun a server on port 6080 with a custom libvirt uri:websockify --web=/usr/share/novnc/ 6080 --token-plugin websockify_LibvirtDomain.LibvirtDomain --token-source 'LibvirtUri'Use your browser to navigate to the Websockify server with thetokenargument being the domain uuid:http://some-ip:6080/vnc.html?token=some-uuid
Package: websocks
websocks基于隧道与拟态流量混淆的匿名通信系统。可对传输层的流量数据进行加密混淆，保护用户上网时的信息、隐私安全。TCP: 使用隧道流量混淆技术，将需要传递的数据放在 WebSocket 的有效载荷中，作为二进制帧传递。UDP: 使用拟态流量混淆技术，将需要传递的数据混淆后传递与服务器。关于本项目使用方法、详细设计介绍以及其他内容请访问websocks:wiki
Package: web-source-compiler
Web Source CompilerUsageWhen first using the web-source-compiler in a project, runwsc initto generate a.wscfile.After that, customize the file with the directories where your source files will be located (don't worry about ones that don't exist yet). Then, runwsc setupTo create all missing directories for you.When writing your code, you can create sub-modules for javascript, html, and css. Place them in themodulessubdirectory with any main pages in themainsubdirectory. You can then import the modules into any main file with the following comments:Javascript// import <module name>html<!-- import <module name> -->css/* import <module name> -->When you're ready to compile your code, runwsc compileS
Package: webSpider-Douban
## webSpider_Douban## V1:
返回排行榜电影名字Output the title-lists of top ten latest movie charts to a csv file## V2:
返回豆瓣排行榜上前十部电影的数据细节(不再只是电影名!); 表头新增更新时间Output the information of each movie in the top ten latest movie charts to a csv file### V2.1:
添加适当注释并优化了有利于分析的数据细节## V3:
Coming soon…
Package: webSpiders
Developed by Dev Gautam KumarImproved version of html-mitraRequirementsUsageYou can use this library to create an HTML file if you are new to HTMLImport|from webSpiders import html |Create HTML file with preformatted headershtml.create_html(file_name)Create HTML elementshtml.create_element(element_name,inner_content)Append elements and contentshtml.append(file_name,content)Overwrite a filehtml.overwrite(file_name,content)Delete a filehtml.delete(file_name)Complete and serve the filehtml.finish(file_name)ContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Package: webspinner
webspinnerPython utilities for working with data source types used by the dsgrid project
(i.e., AWS, PostgreSQL, and .parquet)Install|Configure|UninstallInstallpipinstallwebspinnerorpipinstallgit+ssh://git@github.com/dsgrid/webspinner.git@v1.1.0orpipinstallgit+https://github.com/dsgrid/webspinner.git@masterTo get dependencies required for certain types of resources, add extras as in:pip install webspinner[pgres]
pip install webspinner[aws]
pip install webspinner[parquet]
pip install webspinner[pgres,aws,parquet]Configurewebspinner provides code in support of accessing various dsgrid data sources.
That said, you may only need to access one or two data sources for your particular
project. Please configure access to the resources you need.PostgreSQL|AWS|.parquetPosgreSQLDependenciespip install pgpasslib psycopg2InstallpgAdminor another PostgreSQL client.
Then identify or create yourpgpass file.
On Mac and Linux, the file to edit or create is~/.pgpass. On Windows it is%APPDATA%/Roaming/postgresql/pgpass.conf, where%APPDATA%is the AppData
subdirectory under your user profile (i.e.,C:/Users/$USER/AppData). If the
file does not yet exist, simply create a new text file namedpgpass.conf.
Once the file exists, add the lines like:POSTGRES_SERVER_ADDRESS:*:*:$USER:YOUR_PASSWORDreplacingPOSTGRES_SERVER_ADDRESSwith the PostgreSQL server to connect to,$USERwith your actual username, andYOUR_GISPGDB_PASSWORDwith your actual
password. The dsgrid team typically connects to10.20.5.28or its aliasgds_edit.nrel.gov.On Mac and Linuxbe sure to set the permissions tochmod 600 ~/.pgpassIf the permissions are too permissive, your run may not start.Configuration file argumentsProject-specific defaults can be stored in a text file, e.g.,webspinner.configorconfig.iniwith a[PGRES]section and any or all of the following arguments:[PGRES]
user = your_user_name
dbase = database_name
host = host_address
port = port_numberThe defaults can then be loaded in at any time (e.g., at the top of a script or
a notebook) by passing the configuration filepath into thewebspinner.configurefunction.AWSDependenciespip install pyathena awscliSet up your AWS access credentials by issuing the following command in the terminal.
(FYI there areother waysto set up your credentials if you're interested.)>> aws configure
AWS Access Key ID [None]: <your key>
AWS Secret Access Key [None]: <your secret key>
Default region name [None]: us-west-2
Default output format [None]: jsonConfiguration file argumentsProject-specific defaults can be stored in a text file, e.g.,webspinner.configorconfig.iniwith an[AWS]section and any or all of the following arguments:[AWS]
s3_staging_dir = data_staging_dir_on_s3
region_name = aws_region_name
schema_name = aws_database_schema_name
work_group = aws_work_group_nameThe defaults can then be loaded in at any time (e.g., at the top of a script or
a notebook) by passing the configuration filepath into thewebspinner.configurefunction..parquetIf you plan to work with .parquet files, also install pyarrow.pip install pyarrowUninstallpip uninstall webspinner
Package: webspot
WebspotWebspot is an intelligent web service to automatically detect web content and extract information from it.Demo中文ScreenshotsDetected ResultsExtracted FieldsExtracted DataGet StartedDockerMake sure you have installedDockerandDocker Compose.# clone git repogitclonehttps://github.com/crawlab-team/webspot# start docker containersdocker-composeup-dThen you can access the web UI athttp://localhost:9999.API ReferenceOnce you started Webspot, you can go tohttp://localhost:9999/redocto view the API reference.ArchitectureThe overall process of how Webspot detects meaningful elements from HTML or web pages is shown in the following figure.graph LR
    hr[HtmlRequester]
    gl[GraphLoader]
    d[Detector]
    r[Results]

    hr --"html + json"--> gl --"graph"--> d --"output"--> rDevelopmentYou can follow the following guidance to get started.Pre-requisitesPython >=3.8 and <=3.10Go 1.16 or higherMongoDB 4.2 or higherInstall dependencies# dependenciespipinstall-rrequirements.txtConfigure Environment VariablesDatabase configuration is located in.envfile. You can copy the example file and modify it.cp.env.example.envStart web server# start development serverpythonmain.pywebCode StructureThe core code is located inwebspotdirectory. Themain.pyfile is the entry point of the web server.webspot
├── cmd     # command line tools
├── crawler # web crawler
├── data    # data files (html, json, etc.)
├── db      # database
├── detect  # web content detection
├── graph   # graph module
├── models  # models
├── request # request helper
├── test    # test cases
├── utils   # utilities
└── web     # web serverTODOsWebspot is aimed at automating the process of web content detection and extraction. It is far from ready for production
use. The following features are planned to be implemented in the future.Table detectionNested list detectionExport to spidersAdvanced browser requestDisclaimerPlease follow the local laws and regulations when using Webspot. The author is not responsible for any legal issues
caused by. Please read theDisclaimerfor details.CommunityIf you are interested in Webspot, please add the author's WeChat account "tikazyq1" noting "Webspot" to enter the
discussion group.
Package: webstack-django-endless-pagination
Django Endless Pagination can be used to provide Twitter-style or
Digg-style pagination, with optional Ajax support and other features
like multiple or lazy pagination.The initial idea, which has guided the development of this application,
is to allow pagination of web contents invery few steps.Documentationisavailable online, or in the docs
directory of the project.To filebugs and requests, please usehttps://github.com/frankban/django-endless-pagination/issues.Thesource codefor this app is hosted athttps://github.com/frankban/django-endless-pagination.
Package: webstack-django-pagination
About the forkThis project is a fork of apparently dead “django-pagination” project
originally written by ‘Eric Florenzano’. It is maintained by the Linaro
Validation/Infrastructure team. Latest releases can be found on launchpad and
pypi.We are in the process of deciding how to properly host and manage the project.
Feel free to hop to #linaro on irc.freenode.net and ask either ‘zyga’ or
‘mwhudson’ about this.About top-level package name changeThe top level package name was changed frompaginationtolinaro-django-pagination. I did this to ensure that existing users of the
original package could easily transition, on their own terms. Since both
packages are co-installable you are free to check things out and see if the new
pagination code works well for you or not.Also, since the implementation of the actual tags was rewritten I would not
feel entirely confident that the new package is a drop-in replacement.ContributorsWe would like to welcome any and all contributors. Please use the pull request
feature on github.How to use linaro-django-paginationSee our official documentation for more details.http://packages.python.org/linaro-django-pagination/
Package: webstag
Convert pictures and videos from a directory to another directory
where the pictures are recompressed and resized and the videos are
converted to WebM & MP4, then generate the HTML pages.Browsing is done usingColorbox, a jQuery-based Lightbox clone.Videos are displayed usingVideo.js.TODO:modify colorbox to insert a link to the original image (there’s something
like this currently but it’s a hack)parralelize conversions using a thread pool (when work is done by forking
commands, not when using PIL)
Package: web-stats
web-statsgenerate website reportsweb-stats is a simple library that allows you to generate website reports in less than 60 seconds , by scraping data frominsites.comonly10requests/ip are allowedyou will have to use proxies ­ƒÿó­ƒöÑonly60 secondsto generate a report­ƒöÑInstallationpipinstallweb-statsUsageimportgenerate_reportfromweb-statsgenerate_report('https://www.dafk.net/what/')Output example[{'name':'Overall','rating':'good','value':'6.7'},{'name':'Accessibility','rating':'better','value':'8.3'},{'name':'Experience','rating':'good','value':'5.2'},{'name':'Marketing','rating':'good','value':'5.1'},{'name':'Technology','rating':'better','value':'8.1'},{'name':'Twitter','value':'0.0'},{'name':'Analytics','value':'0.0'},{'name':'Printability','value':'0.0'},{'name':'Freshness','value':'0.0'},{'name':'Meta tags','value':'1.0'},{'name':'Mobile','value':'3.0'},{'name':'Popularity','value':'5.6'},{'name':'Amount of content','value':'7.8'},{'name':'Server behavior','value':'10.0'},{'name':'Page titles','value':'10.0'},{'name':'Headings','value':'10.0'},{'name':'Incoming links','value':'10.0'},{'name':'Images','value':'10.0'},{'name':'Internal links','value':'10.0'},{'name':'URL format','value':'10.0'},{'name':'Domain age','value':'i'}]LicenseMITif you are reading this then here a kuukie ­ƒì¬
Package: webstor
A script to quickly enumerate all websites across all of your organization’s networks, store their responses, and query for known web technologies, such as those with zero-day vulnerabilities.Free software: MIT licenseDocumentation:https://RossGeerlings.github.com/webstor.FeaturesWebStor is a tool implemented in Python under the MIT license for quickly enumerating all websites across all of your organization’s networks, storing their responses, and querying for known web technologies and versions, such as those with zero-day vulnerabilities. It is intended, in particular, to solve the unique problem presented in mid to large sized organizations with decentralized administration, wherein it can be almost impossible to track all of the web technologies deployed by various administrators distributed across different units and networks.WebStor achieves its goal by performing the following actions:Performs DNS zone transfers to collect an organization’s A and CNAME records.Uses Masscan to scan for open HTTP/HTTPS ports on an organization’s net ranges, as well as any IP addresses outside those ranges that were present in the organization’s A and CNAME records.Uses the Python requests library to collect all responses and store in a MariaDB database. All DNS names corresponding to an IP with open HTTP/HTTPS ports will be included in requests in addition to the IP address, so that sites using different headers will not cause a website to be missed.Downloads Wappalyzer web technologies database and stores in MariaDB database, enabling users to query the location(s) of a common web technology by name.Allows users to query the location(s) where custom regexes are contained within stored responses.
Package: webstring
webstringis a template engine for programmers whose favorite template
language is Python. This release ofwebstringreplaces the standalone packageswebstring,lwebstring, TurboWebstring, and LturboWebstring.webstringsupports:plain text templatingadvanced XML and HTML templating usinglxmlor cElementTreetemplating with web frameworks that support the TurboGears template plugin APIWSGI templating middlewarewebstringis designed to:Separate model and view logic from controller logic.Separate template logic from template data.Require only Python for templating.Use Python syntax, idiom, and patterns.Play well with other Python software.webstring’sXML and HTML templating support is designed to:Use XML as a data format, not a programming language.Use existing XML formats.Hide the parts of an XML document that are not being used.Use the best Python XML processing libraries.Bridge HTML and XML.
Package: webstruct
Webstruct is a library for creating statisticalNERsystems that work
on HTML data, i.e. a library for building tools that extract named
entities (addresses, organization names, open hours, etc) from webpages.Unlike most NER systems, webstruct works on HTML data, not only
on text data. This allows to define features that use HTML structure,
and also to embed annotation results back into HTML.Read thedocsfor more info.License is MIT.ContributingSource code:https://github.com/scrapinghub/webstructBug tracker:https://github.com/scrapinghub/webstruct/issuesTo run tests, make suretoxis installed, then runtoxfrom the source root.