Saravanan666/pypi-org huggingface dataset:
Package: webtorrent-checker-scraper
webtorrent_checker_scraperDescriptionA simple script to scrape results ofWebTorrent Checker.UsagePut your magnet links in a text file, one entry per line (empty lines are ignored) and pass it with the-ioption.Options$ python3 webtorrent_checker_scraper.py -i links.txt
usage: webtorrent_checker_scraper.py [-h] [-i INPUT_FILE] [-t]

version: 1.0

optional arguments:
  -h, --help            show this help message and exit

Input parameters:
  -i INPUT_FILE, --input-file INPUT_FILE
                        Input file containing a magnet link per line
  -t, --autoadd-trackers
                        Automatically add popular trackersExamples$ cat links.txt
magnet:?xt=urn:btih:ada00c203f72ddf9603e49ad1c996f83cf6695db&dn=kali-linux-2022.1-installer-everything-amd64.iso&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.opentrackr.org:1337/announce
magnet:?xt=urn:btih:28c55196f57753c40aceb6fb58617e6995a7eddb&dn=debian-11.2.0-amd64-netinst.iso&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.opentrackr.org:1337/announce

$ python3 webtorrent_checker_scraper.py -i links.txt
DN:   "kali-linux-2022.1-installer-everything-amd64.iso"
Hash: "ADA00C203F72DDF9603E49AD1C996F83CF6695DB"
Seeds: 4
Peers: 3
magnet:?xt=urn:btih:ada00c203f72ddf9603e49ad1c996f83cf6695db&dn=kali-linux-2022.1-installer-everything-amd64.iso&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.opentrackr.org:1337/announce

----------------------------------

DN:   "debian-11.2.0-amd64-netinst.iso"
Hash: "28C55196F57753C40ACEB6FB58617E6995A7EDDB"
Seeds: 7
Peers: 0
magnet:?xt=urn:btih:28c55196f57753c40aceb6fb58617e6995a7eddb&dn=debian-11.2.0-amd64-netinst.iso&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.opentrackr.org:1337/announce

----------------------------------Changelogversion 1.0 - 02/27/2022: Initial commitCopyright and licensewebtorrent_checker_scraper is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.webtorrent_checker_scraper is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the GNU General Public License for more details.You should have received a copy of the GNU General Public License along with webtorrent_checker_scraper.
If not, seehttp://www.gnu.org/licenses/.ContactThomas Debize < tdebize at mail d0t com >
Package: web-to-struct
web-to-structA tool for data structuring, mainly for web data.
将数据格式化的小工具，主要处理web数据。安装pip install web-to-struct环境>= python3.6使用DEMOimportrequestsimportjsonfromweb_to_structimportParserif__name__=='__main__':headers={"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36",}r=requests.get("https://copymanga.org/recommend",headers=headers)config={"name":"data","map":[{"function":"string-to-element"},{"function":"css","kwargs":{"patterns":["#comic > .row > .exemptComicItem"]}},],"children":[{"name":"title","map":[{"function":"css","kwargs":{"patterns":["p[title]"]}},]},{"name":"img","map":[{"function":"css","kwargs":{"patterns":[".exemptComicItem-img > a > img"]}},{"function":"attr","kwargs":{"attr_name":"data-web_to_struct"}},]},{"name":"comic_id","map":[{"function":"css","kwargs":{"patterns":[".exemptComicItem-img > a"]}},{"function":"attr","kwargs":{"attr_name":"href"}},{"function":"regex","kwargs":{"pattern":r"comic/(.*?)$"}},]},{"name":"author","map":[{"function":"css","kwargs":{"patterns":[".exemptComicItem-txt > span.exemptComicItem-txt-span > a[href^=\"/author\"]"]}},],}]}parser=Parser()resp=parser.parse(r.text,config)print(json.dumps(resp,ensure_ascii=False,indent=2))returns{
  "data": [
    {
      "title": "見到你之後該說什麼呢",
      "img": "https://mirror277.mangafuna.xyz:12001/comic/jiandaonizhihougaishuoshenmene/cover/e54e3f14-8425-11eb-869d-00163e0ca5bd.jpg!kb_w_item",
      "comic_id": "jiandaonizhihougaishuoshenmene",
      "author": "ねむようこ"
    } //,...
  ]
}Config参数{
  "name": "",
  "map": [
    { "function": "", "kwargs": {} } // 内置函数，上一个的输出作为下一个的输入
  ],
  "children": [{}] // optional 子节点，结构同本结构。
}内置函数 FunctionsFunction 函数名Accepted Returns 可接受的上一个函数的返回类型Extra Args 额外的参数Returns 返回类型Description 描述string-to-elementUnion[str, bytes]feature: str = "html5lib"Element-cssElementpatterns: Union[str, List[str]][Element, None]-indexUnion[Dict, Tuple, List]pattern: str  # eg."[1].x"Any-textElement-Stringget pure strings inside the current elementshtmlElement-Stringget HTML strings inside the current elementattrElementattr_name: strstrget attribute value of the current elementregexstrpattern: strUnion[str, tuple, None]regex match resulttuple-to-stringTuplepattern: strStringuse $1,$2,... to replace tuple elements, eg. "hello $1, $2" for tuple ("a", "b") returns "hello a, b"json-parsestr-Union[Dict, List]parse json string to dict其他行为返回值如果是list，且有children，则处理为返回值叉乘children参考部分内置函数参考了Yealico
Package: webtraffic
TrafficGives you the current traffic stats on your system.Shows average speed and total data transferred at the exit.Installationpip3 install webtrafficUsagetraffic - a tool to view your network speed
Usage: traffic [options] [device]

- see the network stats on your main device
    $ traffic
- specify a network device
    $ traffic en0
- list all network devices
    $ traffic -l
- view this help
    $ traffic -h
Package: web-traffic-monitor
web_traffic_monitorunder constructionDocumentationPyPIUsageimportweb_traffic_monitorclient=web_traffic_monitor.Client()Client functionalitylog_visit( slug, dt: defaults to current datetime )Logs a visit for a given slug at a specified datetime.get_active_redirect( slug )Returns the active redirect for a given slug, or None if it doesn't exist.add_redirect( slug, redirect, dt: defaults to current time )Deactives existing redirect for given slug, and adds a new redirect at given datetime.deactivate_redirect( slug, dt: defaults to current time )Deactivates active redirect for a given slug, by specifying and END_DATETIME = dtDescriptionThis repo serves a couple of basic purposes:Logs visits to a webpageRedirects slugs to new addressesThe functionality is pretty basic, and the most important piecs are contained in the database schema. ADBclass (built for sqlite) is provided with the package. Feel free to overwrite the class functionality if you want to use your own database engine.Schemavisitsslugdatetime/home2023-09-22 08:15:00+0000/about-us2023-09-23 14:32:00+0000/contact2023-09-24 10:45:00+0000/products/latest-deals2023-09-25 18:20:00+0000/blog/post-12023-09-26 09:10:00+0000/services2023-09-27 12:55:00+0000/blog/post-22023-09-28 16:40:00+0000/portfolio/project-xyz2023-09-29 11:25:00+0000/products/sale-items2023-09-30 08:50:00+0000/blog/post-32023-10-01 13:05:00+0000redirectsidslugredirectstart_datetimeend_datetime1/old-page/new-page12023-09-22 08:15:00+00002023-09-23 14:32:00+00002/old-page/new-page22023-09-23 14:32:00+00002023-09-24 10:45:00+00003/old-page/new-page32023-09-24 10:45:00+00004/coming-soon/modern-version2023-09-23 14:32:00+00002023-09-24 10:45:00+00005/coming-soon/modern-version22023-09-24 10:45:00+00006/outdated-info/updated-info2023-09-25 18:20:00+00007/contact-old/contact-new2023-09-26 09:10:00+00008/expired1/events/event-20212023-09-22 08:15:00+00002023-09-23 14:32:00+00009/expired2/new-offer2023-09-23 14:32:00+00002023-09-24 10:45:00+000010/expired3/fallback-page2023-09-24 10:45:00+0000AuthorJames Kabbes
Package: webtranspose
Web TransposeWeb Crawler & AI Web Scraper APIs for building new web experiences.pipinstallwebtransposeIntroduction·Installation·DocsIntroductionIn the near future,nobody will open websites. Instead, we will be directly served the information we are seeking. New web experiences will combine the information from many websites into a single, unified experience.Web Transposeis a collection of API tools that allow building these new web experiences simple.Webᵀ Crawl: Distributed Web CrawlerWebᵀ Scrape: AI Web ScraperCrawlimportwebtransposeaswebtimportosos.environ['WEBTRANSPOSE_API_KEY']="YOUR WEBT API KEY"crawl=webt.Crawl("https://www.example.com",max_pages=100,render_js=True,)awaitcrawl.crawl()# crawl.queue_crawl() for asyncScrapeimportwebtransposeaswebtimportosos.environ['WEBTRANSPOSE_API_KEY']="YOUR WEBT API KEY"schema={"Merchant Name":"string","Title of Product":"string","Product Photo URL":"string",}scraper=webt.Scraper(schema,render_js=True,)out_json=scraper.scrape("https://www.example.com")Web Search (AI SERP API)importwebtransposeaswebtimportosos.environ['WEBTRANSPOSE_API_KEY']="YOUR WEBT API KEY"results=webt.search("what caused the fourth great ninja war?")# results.keys()# ['results']# AI Filterresults=webt.search_filter("Paul Graham's Blog")# results.keys()# ['results', 'filtered_results']InstallationNon-Python Users:📄 API Docs.This repo contains a localliteinstallation of Web Transpose. This is a good option if you want to run Web Transpose locally on your machine for quick use cases.pipinstallwebtransposeHowever, if you wish to leverage the full tools of Web Transpose and use in production, you should add your API key to add thefullversion.os.environ["WEBTRANSPOSE_API_KEY"]="YOUR_API_KEY_HERE"Enterprise SupportWeb Transpose serves enterprises small and large. We partner with companies for the long term with hands-on support and custom solutions.Please email me directly atmike@webtranspose.comfor enquiries.
Package: webtraversallibrary
Web Traversal LibraryThe Web Traversal Library (WTL) is a Python library for abstracting web interactions on top of a base execution layer such as Selenium.InstallationRunpip install webtraversallibrary. That's it.Usage exampleGlossaryYou will find more information in the API docs. As a high-level overview, common terms in the documentation are:Workflow:The main orhcestrating class handling the main "event loop". Sometimes "schema" is also used for the specification of a certain workflow.View:A static snapshot of a current website in a tab, with metadata associated to the page and its elements, possibly augmented with certain ML classifiers.Policy:WTL is based on principles of reinforcement learning, where a policy is a function of the current state (here, the snapshots of current open tabs) to a set of actions.Classifier:These, along withpreload_callbacksandpostload_callbacksare arbitray code that is executed on each workflow iteration. A classifier takes a set of elements and returns either a subset or a mapping from elements to numeric scores.Config:A helper class containing string, numeric, or boolean values for a number of configurations related to WTL. Some are pregrouped under certain umbrella names, such asdesktop(running as a Desktop browser, the default is mobile emulation), but all values can be arbitrarily set. See the documentation for theConfigclass for more information.Getting startedSee the documentation atwebtraversallibrary.readthedocs.io!Also watch"Machine Learning to Auto-Navigate Websites"given at PyCon SE 2020 for an introduction and examples.General architectureThe flow in a workflow is as follows:Initialize the workflow with given configNavigate to given URLsSnapshot the pagesRun all classifiersCheck if the goal is fulfilled, if so exitCall policy with the current view(s)Execute the returned action(s)Goto 3For more examples and usage, please runmake docsand look atdocs/build/html/index.html.Development setupAll development requirements are in requirements.txt. Install the packages from pip. Helper commands are available to create a virtual environment -make env-createandmake env-update.To lint the JavaScript files (not required unless you're editing them) you needjshintwhich is available from npm.How to contributeSee our guide oncontributing.Release HistorySee ourchangelog.LicenseCopyright © 2020 Klarna Bank ABFor license details, see theLICENSEfile in the root of this project.
Package: webtree
WebTreeProject DescriptionAll HTML documents are structured as trees. This project provides code to parse and visualise HTML documents as graphs.Features of the project are:View any html page (using page source) as a graph (collection of connected nodes).Search graph by html tags, tag attributes, strings etc.Find shortest path between HTML node to any other node in graph, for easier web-scraping.Usagepip install webtreewebtree scrape --site=https://www.google.com
Package: webtypes
A smart Web API framework, for Python 3.Community:https://discuss.apistar.org🤔 💭 🤓 💬 😎Documentation:https://docs.webtypes.com📘FeaturesWhy might you consider using API Star for your next Web API project?Schema generation- Support for automatically generating OpenAPI schemas.Expressive- Type annotated views, that make for expressive, testable code.Performance- Dynamic behaviour for determining how to run each view makes API Star incredibly efficient.Throughput- Support for asyncio allows for building high-throughput non-blocking applications.QuickstartInstall API Star:$pip3installwebtypesCreate a new project inapp.py:fromwebtypesimportApp,Routedefwelcome(name=None):ifnameisNone:return{'message':'Welcome to API Star!'}return{'message':'Welcome to API Star,%s!'%name}routes=[Route('/',method='GET',handler=welcome),]app=App(routes=routes)if__name__=='__main__':app.serve('127.0.0.1',5000,debug=True)Openhttp://127.0.0.1:5000/docs/in your browser:API Star isBSD licensedcode.Designed & built in Brighton, England.
Package: webtypy
webtypyWebtypy is a library created with the aim of enhancing the capabilities of your IDE during web development. This module contains the DOM types for a majority of the web APIs commonly used in a web browser, allowing developers to gain insights about the Web API.With webtypy, developers can receive hints from their IDE about available functions, classes, function parameters, and their associated types. These features are intended to assist in the development process and potentially improve code accuracy.The APIs within webtypy are generated from the specifications for CSS, HTML, and JavaScript, providing developers with a resource that keeps up with current web standards.InstallationTo install webtypy, you can use the following command:pip install webtypyRemark: Webtypy is primarily a development-time library. It is designed to assist developers during the development process by providing insights about the Web API. As such, it is not typically required in a production environment or at runtime. Its main function is to enhance the development experience and is not needed once the application is deployed.UsageWebtypy is designed to enhance your IDE's capabilities and streamline your development process. It integrates directly into your development environment and can be utilized in various ways. Here are a couple of examples:Code Completion: One way to leverage webtypy is through code completion. As you type, your IDE can suggest possible completions based on webtypy's API data. This feature can be typically invoked by pressing 'Control+Space'.Hovering Information: Another example is the detailed information provided when you hover over an element in your code. By placing your mouse pointer over an element, you can view additional details like associated types and the definition of the element.These are just a couple of examples, and the exact usage might vary based on the IDE you're using. Be sure to check your IDE's documentation for more specific instructions on utilizing features like code completion and hovering information.GenerationA few details about the generation process of the .pyi (Python Interface file) can be foundhere.Licensewebtypy uses theMozilla Public License Version
2.0.
Package: webui2
Python WebUI v2.4.5Use any web browser as GUI, with Python in the backend and HTML5 in the frontend, all in a lightweight Python package.FeaturesPortable (Needs only a web browser at runtime)Lightweight (Few Kb library) & Small memory footprintFast binary communication protocolMulti-platform & Multi-BrowserUsing private profile for safetyOriginal library is written in Pure CDocumentationOnline DocumentationInstallpipinstallwebui2ExamplefromwebuiimportwebuiMyWindow=webui.window()MyWindow.show('<html><script src="webui.js"></script> Hello World! </html>')webui.wait()
Package: webuipy
Aboutwebuipyis a Python API client forAUTOMATIC1111/stable-diffusion-webuiInstallationYou can installwebuipyusingpip:pip install webuipywebuipyis supported on Python 3.7 and above.Testpipinstallpytest
python-mpytestPackaging and Uploadpython setup.py sdist
twine upload dist/*
Package: webull
WebullAPIs for webull, you are free to use, but code not extensively checked and Webull may update the APIs or the endpoints at any time.https://www.webull.com/Feel free to sign-up for a webull account through here, you will be able to help me to get referral stocks. You can also get 2 stocks for free:https://www.webull.com/activity?inviteCode=oqJvTY3rJNyR&source=invite_gw&inviteSource=wb_overseaSorry for procrastinating in answering the questions and updating the packages. But if you really like the package or really like to motivate me. Materialist appreciations would really motivate me to responding you faster 😂:Installpip install webull

or

python3 -m pip install webullRun testspip install pytest requests_mock
python -m pytest -vUsageHow to login with your emailWebull has made Multi-Factor Authentication (MFA) mandatory since 2020/05/28, if you are having issues, take a look at here:https://github.com/tedchou12/webull/wiki/MFA-&-SecurityOr Authenticate without Login completely 2021/02/14:https://github.com/tedchou12/webull/wiki/Workaround-for-Loginfrom webull import webull # for paper trading, import 'paper_webull'

wb = webull()
wb.login('test@test.com', 'pa$$w0rd')How to login with your mobilefrom webull import webull # for paper trading, import 'paper_webull'

wb = webull()
wb.login('+1-1112223333', 'pa$$w0rd') # phone must be in format +[country_code]-[your number]How to order stockfrom webull import webull
wb = webull()
wb.login('test@test.com', 'pa$$w0rd')

wb.get_trade_token('123456')
wb.place_order(stock='AAPL', price=90.0, qty=2)How to check standing ordersfrom webull import webull
wb = webull()
wb.login('test@test.com', 'pa$$w0rd')

wb.get_trade_token('123456')
orders = wb.get_current_orders()How to cancel standing ordersfrom webull import webull
wb = webull()
wb.login('test@test.com', 'pa$$w0rd')

wb.get_trade_token('123456')
wb.cancel_all_orders()FAQThank you so much, I have received Emails and messages on reddit from many traders/developers that liked this project. Thanks to many that helped and contributed to this project too! There are quite a few repeated questions on the same topic, so I have utilized the Wiki section for them. If you have troubles regardingLogin/MFA Logins,Real Time Quote Data,What is Trade PIN/Trade Token, orHow to buy me a coffeeplease take a look at the Wiki pages first.https://github.com/tedchou12/webull/wikiStream Quoteshttps://github.com/tedchou12/webull/wiki/How-to-use-Streaming-Quotes%3FDisclaimerThis software is not extensively tested, please use at your own risk.DevelopersIf you are interested to join and help me improve this, feel free to message me.
Package: webull-12
WebullAPIs for webull, you are free to use, but code not extensively checked and Webull may update the APIs or the endpoints at any time.https://www.webull.com/Feel free to sign-up for a webull account through here, you will be able to help me to get referral stocks. You can also get 2 stocks for free:https://www.webull.com/activity?inviteCode=oqJvTY3rJNyR&source=invite_gw&inviteSource=wb_overseaSorry for procrastinating in answering the questions and updating the packages. But if you really like the package or really like to motivate me. Materialist appreciations would really motivate me to responding you faster 😂:Installpip install webull

or

python3 -m pip install webullRun testspip install pytest requests_mock
python -m pytest -vUsageHow to login with your emailWebull has made Multi-Factor Authentication (MFA) mandatory since 2020/05/28, if you are having issues, take a look at here:https://github.com/tedchou12/webull/wiki/MFA-&-SecurityOr Authenticate without Login completely 2021/02/14:https://github.com/tedchou12/webull/wiki/Workaround-for-Loginfrom webull import webull # for paper trading, import 'paper_webull'

wb = webull()
wb.login('test@test.com', 'pa$$w0rd')How to login with your mobilefrom webull import webull # for paper trading, import 'paper_webull'

wb = webull()
wb.login('+1-1112223333', 'pa$$w0rd') # phone must be in format +[country_code]-[your number]How to order stockfrom webull import webull
wb = webull()
wb.login('test@test.com', 'pa$$w0rd')

wb.get_trade_token('123456')
wb.place_order(stock='AAPL', price=90.0, qty=2)How to check standing ordersfrom webull import webull
wb = webull()
wb.login('test@test.com', 'pa$$w0rd')

wb.get_trade_token('123456')
orders = wb.get_current_orders()How to cancel standing ordersfrom webull import webull
wb = webull()
wb.login('test@test.com', 'pa$$w0rd')

wb.get_trade_token('123456')
wb.cancel_all_orders()FAQThank you so much, I have received Emails and messages on reddit from many traders/developers that liked this project. Thanks to many that helped and contributed to this project too! There are quite a few repeated questions on the same topic, so I have utilized the Wiki section for them. If you have troubles regardingLogin/MFA Logins,Real Time Quote Data,What is Trade PIN/Trade Token, orHow to buy me a coffeeplease take a look at the Wiki pages first.https://github.com/tedchou12/webull/wikiStream Quoteshttps://github.com/tedchou12/webull/wiki/How-to-use-Streaming-Quotes%3FDisclaimerThis software is not extensively tested, please use at your own risk.DevelopersIf you are interested to join and help me improve this, feel free to message me.
Package: webunit
This release includes:send correct newline in mimeEncode (thanks Ivan Kurmanov)handle Max-Age set to 0 (thanks Matt Chisholm)Webunit is a framework for unit testing websites:Browser-like page fetching including fetching the images and stylesheets
needed for a page and following redirectsCookies stored and trackable (all automatically handled)HTTP, HTTPS, GET, POST, basic auth all handled, control over expected
status codes, …DOM parsing of pages to retrieve and analyse structure, including simple
form re-postingTwo-line page-fetch followed by form-submit possible, with error checkingAbility to register error page content across multiple testsUses python’s standard unittest module as the underlying frameworkMay also be used to regression-test sites, or ensure their ongoing
operation once in production (testing login processes work, etc.)