Saravanan666/pypi-org huggingface dataset:
Package: webcrawlers
===============`Web Crawlers`_===============Copyright (c) 2016 Jeremie DECOCK (http://www.jdhp.org)* Web site: http://www.jdhp.org/projects_en.html#web-crawlers* Source code: https://github.com/jeremiedecock/web-crawlers* Issue tracker: https://github.com/jeremiedecock/web-crawlers/issues* Web Crawlers on PyPI: https://pypi.python.org/pypi/webcrawlersDescription===========Some web crawlers written with Python, feedparser and Beautifulsoup.Note:This project is still in beta stage, so the API is not finalized yet.Dependencies============-  Python >= 3.0-  Beautifulsoup-  Feedparser-  WaWA.. _install:Installation============Gnu/Linux---------You can install, upgrade, uninstall Web Crawlers with these commands (in aterminal)::pip install --pre webcrawlerspip install --upgrade webcrawlerspip uninstall webcrawlersOr, if you have downloaded the Web Crawlers source code::python3 setup.py install.. There's also a package for Debian/Ubuntu::....     sudo apt-get install webcrawlersWindows-------Note:The following installation procedure has been tested to work with Python3.4 under Windows 7.It should also work with recent Windows systems.You can install, upgrade, uninstall Web Crawlers with these commands (in a`command prompt`_)::py -m pip install --pre webcrawlerspy -m pip install --upgrade webcrawlerspy -m pip uninstall webcrawlersOr, if you have downloaded the Web Crawlers source code::py setup.py installMacOSX-------Note:The following installation procedure has been tested to work with Python3.5 under MacOSX 10.9 (*Mavericks*).It should also work with recent MacOSX systems.You can install, upgrade, uninstall Web Crawlers with these commands (in aterminal)::pip install --pre webcrawlerspip install --upgrade webcrawlerspip uninstall webcrawlersOr, if you have downloaded the Web Crawlers source code::python3 setup.py installBug reports===========To search for bugs or report them, please use the Web Crawlers Bug Tracker at:https://github.com/jeremiedecock/web-crawlers/issuesLicense=======This project is provided under the terms and conditions of the`MIT License`_... _MIT License: http://opensource.org/licenses/MIT.. _Web Crawlers: http://www.jdhp.org/projects_en.html#web-crawlers
Package: web-crawling
WeatherWebCrawlingThis API is used for fetching weather conditions  name as a string to the respective locality's weather data.
Import "code.start(place1,dateinp="",type="<Any one from daily/monthly/5days/10days/datweise in string format>") from package "web_crawling".
You can pass ANY ONE OF the keyword arguments - Date in "YYYY-MM-DD" format/ forecast type required ('daily' for today {Default Selection}, 'monthly' for monthly,'5days' for 5-day , '10days' for 10-day,'datewise' for datewise). We are relying on APIs used by weather.com internally for extracting the respective data.For ex:1.If you just want today's weather parameters with place name:from web_crawling import code
code.start("Place-Name")2.If you just want the current month's weather details with place name:from web_crawling import code
code.start("Place-Name",type="monthly")3.If you just want some date's weather prediction with place name:from web_crawling import code
code.start("Place-Name",dateinp="date string in yyyy-mm-dd format")package is installable as "pip install web_crawling"If no/wrong keyword arguments given, it may result in wrong output
Package: webcronmon
Shows websites monitored onWebcron.orgin kiosk mode.AtSeantiswe useWebcron.orgto monitor our sites. For our customers, as well as for our office monitor screen, we wanted a simple auto-refreshing page that shows the status of our sites at a glance. We therefore wrote this smallFlaskapplication looking like this:The monitor only shows the bare minimum of information to keep things simple:list monitors by grouplink the monitors to the actual websiteshow the current state (online / offline since)show the uptime of the last 30 daysThe site will update the list every 60 seconds by default, unless configured otherwise or unless the timer on the upper right is toggled with a mouse-click.Demohttp://monitor.seantis.chRequirementsPython 2.7InstallationCreate a new folder:mkdir webcronmonOpen it:cd webcronmonInitialize a virtual environment:virtualenv --no-site-packages -p python2.7 .Install webcronmon:pip install webcronmonInitialize the configuration:init-webcronmonConfigurationBefore you run webcronmon you need to edit the config.ini file created byinit-webcronmon. At the least you need to enter your webcron API credentials, but there are other configuration entries you might wish to adjust.All the configuration options are described in the config.ini that was created for you.RunningTo start webcronmon simply executerun-webcronmonin the folder in which you have edited your config.iniDeploymentWe recommend to only run one webcronmon instance and offer it through a reverse proxy. Personally we useCircusby Mozilla with the following circus.ini:[watcher:webcronmon]
gid = webcronmon
uid = webcronmon
cmd = /home/webcronmon/app/bin/run-webcronmon
virtualenv = /home/webcronmon/app
working_dir = /home/webcronmon/app
numprocesses = 1
singleton = True
copy_env = TrueAs a reverse proxy we use Nginx as follows:server {
    server_name monitor.seantis.ch;

    listen      80;

    location / {
        proxy_pass http://localhost:8081;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_redirect off;
    }
}To start the server reload nginx:sudo /etc/init.d/nginx reloadAnd start webcronmon through circus:sudo circusctl start webcronmonCopyrightSeantis GmbHLicenseMITHistory1.0.2 (2014-01-22)Fixes wrong monitor urls when the protocol is defined redundantly.Adds clickable group-headers for groups defined in routes. Fixes #2.1.0.1 (2013-05-03)Fixes the app crashing when a monitor reports an offline server.1.0 (2013-04-22)Initial Release
Package: webctf
Web-CTF-HelpDescriptionA simple set of scripts (mainly scrapers for now) intended for aid in web CTFs. Find image sources (and alts), javascript sources and comments.Usageusage:webctf[-h][-v][--comments][--scripts][--images][--headers][-f]url

positionalarguments:urlURLofthetargetwebsite

optionalarguments:-h,--helpshowthishelpmessageandexit-v,--versionshowprogram'sversionnumberandexit--commentsonlydisplayHTMLcomments(default:False)--scriptsonlydisplayscriptsources(default:False)--imagesonlydisplayimagesources(default:False)--headersonlydisplayinterestingresponseheaders(combinewith-ftodisplayall)(default:False)-f,--fullenablefulloutputforalloptions(default:False)webctfhttps://example.comOnly display HTML commens and script sources:webctf--comments--scriptshttps://example.com=============
COMMENTS
=============

[+] 1 :   Internal game scripts/styles, mostly boring stuff
[+] 2 :   This is our database of messages

=============
SCRIPTS
=============

[+] 1 : /static/game-frame.js
[+] 2 : /static/post-store.jsOnly display interesting headers:webctf--headershttps://example.com===================
INTERESTING HEADERS
===================

Server : Google FrontendDisplay all headers:webctf--headers-fhttps://example.com=============
ALL HEADERS
=============

Content-Type : text/html; charset=utf-8
Cache-Control : no-cache
X-Cloud-Trace-Context : df6d1758c776f4fbc4b3bab158818b64;o=1
Date : Fri, 01 Jan 2021 20:19:57 GMT
Server : Google Frontend
Content-Length : 2689
Alt-Svc : h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"Installationgitclonehttps://github.com/xnomas/web-ctf-help
pipinstall-rrequirements.txt
pipinstall-e.Future plansI plan to implement a download function, to download the images, js files and so on. If you have any suggestions feel free to give them to me,
but please keep in mind that I am still a student :)
Package: web-ctf-analyser
\n# Web Analyser
Analyses Websites for you. It can grab a limited amount of things, but it's better than doing them manually:Robots and SitemapCookies and JWTsRedirectsParameters in redirects are analysed using regex for potential LFI/RFI/SSRF vulnerabilitiesCommentsURLs in the sourceResources in the source, e.g./api/v2Differences in responses between User-AgentsAlso allows you to specify your own:User-AgentCookiesUsername and Password (for Basic Authentication)Installinggit clone https://github.com/ir0nstone/web-analyser.git
cd web-analyser
pip3 install -r requirements.txt
Package: webcubers
WebCubers Package & CLI Toolthis is interface package for WebCubers portal.Usage_________
///\ \ \___||__/__\_||___________\ \/\//_\'_ \ / / | | | | '_\/_\'__/__|\/\/__/|_)//__||_|||_)|__/|\__\\/\/\___|_.__/\____/\__,_|_.__/\___|_||___/<WebCubers-CLIv1.8.1>

usage:cubers[-h]{login,snippets,logout,profile}...

WebCubersCLIConnector.

positionalarguments:{login,snippets,logout,profile}optionalarguments:-h,--helpshowthishelpmessageandexitLoginCubersloginMyUsernameMyApiKeyProfileCubersprofileLeadersCuberssnippetsOTPCubersotpConnectionCubersconnectionSnippetsCuberssnippetsRepositoryCubersrepo
Cubersrepo--downloadserverfile.py
Cubersrepo--uploadlocalfile.py--forceLogoutCuberslogout
Package: webcui
webcuiA Python package for creating Webcui apps. A Webcui app is to the web what a Command Line Interface app is for the command line.
The goal of Webcui is to make it as easy as possible for Python developers to share their Python app with the world.Webcui idea:A web page with a HTML form is generated from a basic Python function.Function parameters become input fields.The function is executed when the user submit the form.The result is displayed on the web page.InstallationInstall Python 3.6 or later and then webcui using pip.$ pip install webcuiUsageHere's an example of a simple Webcui app:importwebcuidefcmd(number_of_spam:int,side:str="eggs"):"""Calculate the price of a breakfast order."""spams=number_of_spam*["spam"]dish=f"{', '.join(spams)}and{side}"price=number_of_spam*1.5+2returnf"The price of an order of{dish}is â‚¬{price:.2f}"if__name__=='__main__':webcui.run(cmd)To run the Webcui app on your own computer run:$ python app.py --run
Package: webcv
WebcvWebcv is a simple wrapper that can implements displaying images, text, and tables via web browsers.This project is based onquartthat requires python 3.7+. For older version of Python, please usewebcv-geventinstead.Installationpip install webcvSample Usageimportsysimportnumpyasnpfromwebcvimportwebcvwebcv.head_show("h1","Test image show")image=np.random.randint(0,256,(256,256,3),dtype=np.uint8)webcv.imshow("image",image)webcv.imshow("resize using webcv",webcv.resize(image,(512,512)))# response in 30 secondswebcv.waitKey(30000)webcv.head_show("h1","Test table show")webcv.table_show("versions",[["python","opencv","webcv"],[f"version{'.'.join(str(v)forvinsys.version_info)}",cv.__version__,webcv.__version__],])webcv.waitKey()By running the code above, your terminal will display something like:#####################################################################
#                                                                   #
#   The server is running at: http://my-server-ip:4732              #
#                                                                   #
#####################################################################
 * Serving Quart app 'webcv'
 * Environment: production
 * Please use an ASGI server (e.g. Hypercorn) directly in production
 * Debug mode: False
 * Running on http://0.0.0.0:4732 (CTRL + C to quit)Copy paste the address into your browser to view (you may need ssh forward if you are using a remote server):AcknowledgementThe project was inspired by a Megvii tool.LicenseMIT License
Package: web-data-extraction
A unified scraper collection of web dataPurpose of the packageThe purpose of the package is to provide a web scraping funstions that will nbe able to deploy data to dbFeaturesApartment for sale featuresRef_numberBuild_YearFloorsSitting_RoomsDining_RoomsBedroomsWardrobesBathroomsCar_ParkingAncillaryPlot_numberLandSizePriceDistrictSectoDateGetting startedThe package can be found on pypi hense you can install it using pipInstallationpipinstallweb_data_extractionAPIContributionContributions are welcome
Notice a bug, let me know. ThanksAuthorMain Maintainer: Lisa Yvette INYANGE
Package: webdav
webdavRelease NotesVersion 1.1.5 - 05.10.2018Use connection pool byhttps://github.com/kamikazeVersion 1.1.4 - 05.10.2018Handle HTTP 405 response code for MKCOL byhttps://github.com/kamikazeVersion 1.1.2 - 07.06.2018Check for is_dir after confirmed that resource exists byhttps://github.com/kamikazeDownload remote file for open() only if it exists byhttps://github.com/kamikazeCreate directories recursively if file is being opened in write mode byhttps://github.com/kamikazeVersion 1.1.1 - 07.06.2018Bug fixes byhttps://github.com/kamikazeVersion 1.1.0 - 07.06.2018Added open() support for transparent file manipulations byhttps://github.com/kamikazeVersion 1.0.0 - 06.06.2018Added support for custom authentication like NTLM byhttps://github.com/kamikaze
Package: webdav3-cli
webdav3-cliSimple command-line interface for interacting with a WebDAV server.This was tested and developed against the WebDAV interface for aRedmineserver.UsageConfiguringYou can configure a set of default values for the optional arguments using theconfigcommandExamples:webdav3 config set hostname={hostname}webdav3 config set user={username}webdav3 config set pass={password}Uploading filesFiles can be uploaded to the WebDAV server using theuploadcommandExamples:webdav3 upload {local_path} {remote_path} --hostname {server address} --root "/dmsf/webdav" --user {username} --pass {password}webdav3 upload {local_path} {remote_path}
Package: webdav4
webdav4Webdav API with an (optional)fsspecimplementation and aCLI.Installation$pipinstallwebdav4Usagewebdav4 Clientfromwebdav4.clientimportClientclient=Client("https://webdav.com",auth=("username","password"))client.exists("Documents/Readme.md")client.ls("Photos",detail=False)client.upload_file("Gorilla.jpg","Photos/Gorilla.jpg")Check outClient APIreference for more information.fsspecfsspectries to provide a
consistent APIs to different storage backends, by defining standard
interfaces, so that other code using them could operate without depending
on the implementations of the backends. This package, in the same way,
wraps the abovewebdav clientwith a consistent file-system API.To use it, you need to installfsspecadditionally which you can do as
follows:$pipinstallwebdav4[fsspec]fromwebdav4.fsspecimportWebdavFileSystemfs=WebdavFileSystem("https://webdav.com",auth=("username","password"))fs.exists("Documents/Readme.md")fs.ls("Photos",detail=False)Check outWebdavFileSystem APIreference for more information.CLIwebdav4 also provides a CLI similar toaws s3to make it easier to work with webdav servers.Please checkoutCLI Usagefor more information.ContributingContributions are welcome. Please take a look atContributing Guidefor more details.
Package: webdavconvert
webdavconvertInstallationpip install webdavconvertCLI UsageFirst create a settings file withpython -m webdavconvert --save settings.iniAfter setting all parameters in settings.ini you can run it withpython -m webdavconvert --load settings.iniYou can also directly run webdavconvert by setting all options through the CLI. Mixing save presets and command line options is not supported at the time, if you use--loadall CLI options are ignored.Available commands$ python -m webdavconvert -h

Usage:
  webdavconvert [options]
  webdavconvert (--save | --load) FILE [options]
  webdavconvert (-h | --help | --version)

Arguments:
  FILE                      Configuration file.

Options:
  -h --help                 Show this screen.
  --version                 Show version.
  --loglevel LEVEL          Set a specific log level. [default: INFO]

Target options:
  --target-url TURL         Url for the target WebDAV host. [default: https://cloud.example.com]
  --target-user TUSER       User for the target WebDAV host. [default: username]
  --target-password TPW     Password for the target user. [default: password]
  --target-dir TDIR         Directory to store files. [default: /]
  --target-root TROOT       Root of the WebDAV host. [default: /public.php/webdav/]

Source options:
  --source-url SURL         Url for the target WebDAV host. [default: https://cloud.example.com]
  --source-user SUSER       User for the target WebDAV host. [default: username]
  --source-password SPW     Password for the source user. [default: password]
  --source-dir SDIR         Directory to store files in [default: /]
  --source-root SROOT       Root of the WebDAV host. [default: /public.php/webdav/]
  --source-filter SFILTER   Only convert files matching this regex. [default: mp4]
  --source-recursive        Recursivly browse all folders.

Convert options:
  --output-format FORMAT    Video output format. [default: mp4]
  --output-timecode         Overlay the video with the timecode.
  --output-height HEIGHT    Resize the video while keeping proportions to HEIGHT. [default: 480]
  --output-prefix PREFIX    Add this prefix to the output filename. [default: ]
  --output-suffix SUFFIX    Add this suffix to the output filename. [default: ]

System options:
  --ffmpeg-path PATH        Path to the ffmpeg executable. [default: /usr/local/bin/ffmpeg]

Config options:
  --save                    Overwrite the existing configuration file.
  --load                    Load the configuration file and ignore all other parameters.As Python modulesource = CloudFiles(url, user, password, root)
target = CloudFiles(url, user, password, root)
source.dir = ''
target.dir = ''
source.filter = 'mp4'

converter = Converter(source, target)
converter.get_files_to_download(source.filter)
converter.download_files()
converter.convert()
converter.get_files_to_upload(source.filter)
converter.upload_files()
converter.cleanup()
Package: webdeployer
DEPLOYER### This webdeployer allows you to deploy fast and easy to your webserver.
Automate deployments of your webapps using :GUNICORNNGINXSUPERVISORDYou need to use a directory format like :`/projects/client/client_name/project/source`requirements :
- Python 2.7
- Fabric
- Jinja2On your server you need to have already installed and running:
-   nginx
-   gunicorn(not realy only you need to have the package installed)
-   supervisordThis is being set free from Private so it may take time to free out all the mess from my private repoTO INSTALL` pip install webdeployer `If you are deploying for the first time :
go in your project directory and type : webdeployer initedit the webdeployer-config.pythen type webdeployer deployMILESTONE/ROADMAPI want to make it more customizable
Package: webdetect
License: MIT|Author:
0xcryptowebdetectDetects technologies of a web page using Wappalyzer in a headless
browser.InstallationpipinstallwebdetectUsageAs an application:Usage:webdetect[OPTIONS][URLS]webdetectv0.0.1by@0xcrypto<vi@hackberry.xyz>Options:--fileFILENAME--json/--active-dataPrintresultinJSONformat.--tabsINTEGERMaximumtabstoopeninchromium.--headless/--no-headlessWhethertoopenchromiuminheadlessmode.--helpShowthismessageandexit.In your scripts:importasynciofromwebdetectimportWebDetectasyncdefgetTech():returnawaitWebDetect().detect(domains,maxtabs=20,headless=True,logging=False)asyncio.run(getTech())
Package: webdev
AboutThis is a small web server for development or sharing purposes.
It allows to serve through HTTP any directory in your machine.
It is based on Python famousSimpleHTTPServerand it only depends on a
Python 2.7 interpreter and the standard library.In contrast toSimpleHTTPServerthis development web server:Allows the serving folder can be passed as the first positional argument.
Defaults to current directory.Will respond with 404 if the serving folder is removed; instead of crashing.Will return back to normal operation if the folder is recreated.
Very useful with build systems that perform clean actions.Bind IP can be specified in case security is a must while developing.As withSimpleHTTPServerthis web server can:Specify binding port.List directories.Serveindex.htmlorindex.htmlautomatically as the directory entry.Installationpip install webdevUsage$ webdev --help
usage: webdev [-h] [-v] [--version] [-i IP] [-p PORT] [-f] [-r RANGE] [-u]
              [path]

Development Web Server

positional arguments:
  path                  Path to serve. Default: Current directory.

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Increase verbosity level
  --version             show program's version number and exit
  -i IP, --ip IP        IP to bind to. Default: 0.0.0.0
  -p PORT, --port PORT  Port to listen to. Default: 8080.
  -f, --force-port      Force the use of the given port.
  -r RANGE, --range RANGE
                        Number of ports to try from the base port
  -u, --future          Ignore if the folder to server doesn't exists yetChangelog1.1.0NewAdded support for Python 3 (issue #5).Added support for incremental automatic port selection (issue #2).Added the--futureflag that allows the webserver to serve a non-existent
folder, assuming it will be created in the future (issue #3).ChangedCtrl+C is now handled (issue #4).1.0.0Initial release.LicenseCopyright (C) 2015-2016 Carlos Jenkins <carlos@jenkins.co.cr>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
Package: webdi
A Simple Dependency Injection Container for Web AppsThis module contains a simple dependency injection container that is optimized
for web apps. You define relationships once, and then each request (probably
via middleware) receives a container that references the once-defined
definition to build memoized services for just that request.Please note: documentation on this project is currently not providedInteracting With the Projectmakeis the best way to interact with the project.Setting Up:make installRunning Tests:make testRunning Coveragemake coverage