Saravanan666/pypi-org huggingface dataset:
Package: backblaze
Welcome to Backblaze's unofficial python wrapper!This is a unofficial asynchronous & synchronous wrapper for Backblaze's B2 API.The awaiting version requires Python3.7or above!Features:Full API coverage.Static typing withPyRight.Background authentication.Asynchronous & synchronous support.Easy to use with an object oriented design.Unit testing.Upload URL caching.Rate limit handler.InstallPip:pip3 install backblazeGit:pip3 install git+https://github.com/WardPearce/backblaze.gitExamplesUpload fileHave an example? Make a PR!DocumentationDocumentationThanks toEncode for creating HTTPX
Package: backcall
Specifications for callback functions passed in to an APIIf your code lets other people supply callback functions, it’s important to
specify the function signature you expect, and check that functions support that.
Adding extra parameters later would break other peoples code unless you’re careful.backcall provides a way of specifying the callback signature using a prototype
function:from backcall import callback_prototype

@callback_prototype
def handle_ping(sender, delay=None):
    # Specify positional parameters without a default, and keyword
    # parameters with a default.
    pass

def register_ping_handler(callback):
    # This checks and adapts the function passed in:
    callback = handle_ping.adapt(callback)
    ping_callbacks.append(callback)If the callback takes fewer parameters than your prototype,backcallwill wrap
it in a function that discards the extra arguments. If the callback expects
more arguments, a TypeError is thrown when it is registered.For more details, see thedocsor
theDemo notebook.The tests are run withpytest. In the root directory,
execute:py.test
Package: back-cleaner
back-cleanerServer-side Python tool for escaping script tags and converting characters into HTML entity equivalents (no regex).def escape_script_tags(input_str)This function escapes script tags with backslashes.Samplefrom back_cleaner.cleaner import escape_script_tagssource = "<script>Hey, how are you doing?</script>"
result = escape_script_tags(source)
print(result)def replace_with_ents(input_str)This function converts the following characters into the HTML entity equivalents.Ampersand (&)Less than (<)Greater than (>)Double quote (")Single quote (')Samplefrom back_cleaner.cleaner import replace_with_entssource = "<script>Hey, how are you doing?</script>"
result = replace_with_ents(source)
print(result)
Package: backend.ai-accelerator-cuda
Backend.AI Accelerator Plugin for CUDAJust install this along with Backend.AI agents, using the same virtual environment.
This will allow the agents to detect CUDA devices on their hosts and make them
available to Backend.AI kernel sessions.$pipinstallbackend.ai-accelerator-cudaThis open-source edition of CUDA plugins support allocation of one or more CUDA
devices to a container, slot-by-slot.Compatibility MatrixBackend.AI AgentCUDA Plugin20.09.x2.0.020.03.9 ~2.0.020.03.0 ~ 20.03.80.14.x19.09.17 ~0.13.x19.06.x, 19.09.0 ~ 19.09.160.11.x, 0.12.x19.03.x0.10.xIn the versions released after the above matrix, the agent will set the required
version range of this plugin as an extra requirements set "cuda".
Package: backend.ai-accelerator-cuda-mock
backend.ai-accelerator-cuda-mockA mockup plugin for CUDA acceleratorsThis plugin deceives the agent and manager to think as if there are CUDA devices.
The configuration followscuda-mock.tomlplaced in the same location ofagent.toml.
Please refer the sample configurations in theconfigs/acceleratordirectory and copy one of them as a starting point.The statistics are randomly generated in reasonable ranges, but it may seem like "jumping around" because there is no smoothing mechanism of generated values.
The configurations for fractional/discrete mode, fraction size, and device masks in etcd are exactly same as the original plugin.The containers are created without any real CUDA device mounts but withBACKENDAI_MOCK_CUDA_DEVICESandBACKENDAI_MOCK_CUDA_DEVICE_COUNTenvironment variables.
Since the manager does not know if the reported devices are real or not, you can start any CUDA-only containers (but of course they won't work as expected).
Package: backend.ai-accelerator-cuda-open
Backend.AI Accelerator Plugin for CUDAJust install this along with Backend.AI agents, using the same virtual environment.
This will allow the agents to detect CUDA devices on their hosts and make them
available to Backend.AI kernel sessions.$pipinstallbackend.ai-accelerator-cudaThis open-source edition of CUDA plugins support allocation of one or more CUDA
devices to a container, slot-by-slot.Compatibility MatrixBackend.AI AgentCUDA Plugin20.09.x2.0.020.03.9 ~2.0.020.03.0 ~ 20.03.80.14.x19.09.17 ~0.13.x19.06.x, 19.09.0 ~ 19.09.160.11.x, 0.12.x19.03.x0.10.xIn the versions released after the above matrix, the agent will set the required
version range of this plugin as an extra requirements set "cuda".
Package: backend.ai-accelerator-mock
backend.ai-accelerator-mockA mockup plugin for acceleratorsThis plugin deceives the agent and manager to think as if there are accelerator devices.
The configuration followsmock-accelerator.tomlplaced in the same location ofagent.toml.
Please refer the sample configurations in theconfigs/acceleratordirectory and copy one of them as a starting point.The statistics are randomly generated in reasonable ranges, but it may seem like "jumping around" because there is no smoothing mechanism of generated values.
The configurations for fractional/discrete mode, fraction size, and device masks in etcd are exactly same as the original plugin.Notes when setting up mock CUDA devicesThe containers are created without any real CUDA device mounts but withBACKENDAI_MOCK_CUDA_DEVICESandBACKENDAI_MOCK_CUDA_DEVICE_COUNTenvironment variables.
Since the manager does not know if the reported devices are real or not, you can start any CUDA-only containers (but of course they won't work as expected).
Package: backend.ai-accelerator-rocm
Backend.AI Accelerator Plugin for ROCmJust install this along with Backend.AI agents, using the same virtual environment.
This will allow the agents to detect ROCm devices on their hosts and make them
available to Backend.AI kernel sessions.$pipinstall-e<Pathtothisrepository>This open-source edition of ROCm plugins support allocation of one or more ROCm
devices to a container, slot-by-slot.Compatibility MatrixTBD
Package: backend.ai-aiotusclient
tus.io uploader for Python asyncioaiotusclient is a fork of tus-py-client rewritten for Python asyncio and aiohttp.
It is used to communicate with storage proxies that handle large transfers for vfolder uploads and downloads.Package Structureaiotusclientclient: The client instance class which communicates between Backend.AI ManagerbaseuploaderanduploaderResponsible for chunking the file and asynchronously uploading to tus serverrequestHandles the uploading requestInstallationPrequisitesPython 3.7 or higher withpyenvandpyenv-virtualenv(optional but recommneded)Installation ProcessFirst, prepare the source clone of this agent:#gitclonehttps://github.com/lablup/aiotusclientFrom now on, let's assume all shell commands are executed inside the virtualenv. And we located in backend.ai root directory.Now install dependencies:#pipinstall-UaiotusclientWhen done, import into your code the aiotusclientfromaiotusclientimportclienttus_client=client.TusClient(session_create_url,session_upload_url,rqst.headers,params)ReferenceThis library was forked fromtus-py-clientand customized in order to facilitate asynchronous communication with our TUS server.
Package: backend.ai-cli
backend.ai-cliUnified command-line interface for Backend.AIHow to adopt in CLI-enabled Backend.AI packagesAn examplesetup.cfgin Backend.AI Manager:[options.entry_points]
backendai_cli_v10 =
    mgr = ai.backend.manager.cli.__main__:main
    mgr.start-server = ai.backend.gateway.server:mainDefine your package entry points that returns a Click command group using a
prefix, and add additional entry points that returns a Click command using a
prefix followed by a dot and sub-command name for shortcut access, under thebackendai_cli_v10entry point group.Then addbackend.ai-clito theinstall_requireslist.You can do the same insetup.pyas well.
Package: backend.ai-common
Backend.AI CommonsCommon utilities library for Backend.AIInstallation$pipinstallbackend.ai-commonFor development$pipinstall-Upipsetuptools$pipinstall-U-rrequirements/dev.txtRunning test suite$python-mpytestWith the default halfstack setup, you may need to set the environment variableBACKEND_ETCD_ADDRto specify the non-standard etcd service port (e.g.,localhost:8110).The tests forcommon.redismodule requires availability of local TCP ports 16379, 16380, 16381,
26379, 26380, and 26381 to launch a temporary Redis sentinel cluster viadocker compose.In macOS, they require a localredis-serverexecutable to be installed, preferably viabrew,
becausedocker composein macOS does not support host-mode networking and Rediscannotbe
configured to use different self IP addresses to announce to the cluster nodes and clients.
Package: backend.ai-console-server
backend.ai-console-serverA minimal webapp to convert web session requests to API requestsInstallationPrepare a Python virtualenv (Python 3.7 or higher) and a Redis server (5.0 or higher).$gitclonehttps://github.com/lablup/backend.ai-console-serverconsole-server$cdconsole-server$pipinstall-U-e.$cpconsole-server.sample.confconsole-server.confModeIfservice.modeis set "webconsole" (the default), the console server handles
PWA-style fallbacks (e.g., servingindex.htmlwhen there are no matching
files for the requested URL path).
The PWA must exclude/serverand/funcURL prefixes from its own routing
to work with the console server's web sessions and the API proxy.If it is set "static", the console server serves the static files as-is,
without any fallbacks or hooking, while preserving the/serverand/funcprefixed URLs and their functionalities.If you want to serve console in console-server with "webconsole" mode, prepare static console source by choosing one of the followings.Option 1: Build console from sourceBuildbackend.ai-consoleand copy all files underbuild/bundleinto thesrc/ai/backend/console/staticdirectory.Option 2: Use pre-built consoleTo download and deploy console from pre-built source, do the following:git submodule initgit submodule updatecd src/ai/backend/console/staticgit checkout main  # or target branchgit fetchgit pullSetup configuration for console-serverYou don't have to writeconfig.tomlfor the console as this console server auto-generates it on-the-fly.Editconsole-server.confto match with your environment.Usage$python-mai.backend.console.server
Package: backend.ai-install
Backend.AI InstallerPackage Structureai.backend.install: The installer packageDevelopmentUsing the textual debug modeFirst, install the textual-dev package in thepython-defaultvenv../py-mpipinstalltextual-devOpen two terminal sessions.
In the first one, run:dist/export/python/virtualenvs/python-default/3.11.6/bin/textualconsoleWarningYou should use thetextualexecutable createdinside the venv'sbindirectory../py -m textualonly shows the demo instead of executing the devtool command.In the second one, run:TEXTUAL=devtools,debug./backend.aiinstall
Package: backend.ai-kernel-runner
A common base runner for various programming languages.It manages an internal task queue so that multiple command/code execution requests
are processed in the FIFO order, without garbling the console output.How to write a new computation kernelInheritai.backend.kernel.BaseRunnerand implement the following methods:async def init_with_loop(self)Called after the asyncio event loop becomes available.Mostly justpass.If your kernel supports interactive user input, then put setself.user_input_queueas anasyncio.Queueobject.  It’s your job
to utilize the queue object for waiting for the user input.  (Seehandle_input()method inai/backend/kernel/python/inproc.pyfor
reference)  If it’s not set, then any attempts for getting interactive user
input will simply return"<user-inputis unsupported>".async def build_heuristic(self)(Batch mode)Write a heuristic code to find some build script or run a
good-enough build command for your language/runtime.(Blocking)You don’t have to worry about overlapped execution since the
base runner will take care of it.async def execute_heuristic(self)(Batch mode)Write a heuristic code to find the main program.(Blocking)You don’t have to worry about overlapped execution since the
base runner will take care of it.async def query(self, code_text)(Query mode)Directly run the given code snippet. Depending on the language/runtime,
you may need to create a temporary file and execute an external program.(Blocking)You don’t have to worry about overlapped execution since the
base runner will take care of it.async def complete(self, data)(Query mode)Take a dict data that includes the current line of code where
the user is typing and return a list of strings that can auto-complete it.(Non-blocking)You should implement this method to run asynchronously with
ongoing code execution.async def interrupt(self)(Query mode)Send an interruption signal to the running program. The implementation
is up to you. The Python runner currently spawns a thread for in-process
query-mode execution and use a ctypes hack to throw KeyboardInterrupt
exception into it.(Non-blocking)You should implement this method to run asynchronously with
ongoing code execution.NOTE: Existing codes are good referecnes!How to use in your Backend.AI computation kernelsInstall this package using pip via aRUNinstruction in Dockerfile.
Then, set theCMDinstruction like below:CMD["/home/backend.ai/jail","-policy","/home/backend.ai/policy.yml",\"/usr/local/bin/python","-m","ai.backend.kernel","<language>"]where<language>should be one of the supported language names defined inlang_mapvariable inai/backend/kernel/__main__.pyfile.
Package: backend.ai-krunner-alpine
backend.ai-krunner-alpineBackend.AI Kernel Runner Package for Alpine-based KernelsDevelopment & UpdatingPlease referthe README of krunner-static-gnu package.Making a minimal Alpine-based image compatibile with this krunner packageUse Alpine 3.17 or later and install this list of packages.NotesmuslDOES NOTsupport dynamic loading of 3rd-party libraries (i.e., Python binary modules) when the CPython interpreter is built statically.So we keep the CPython interpreter as dynamic, using the standard build procedure taken fromthe Docker Hub's Python library.As Alpine Linux is the only well-known musl-based Linux distribution, we keep our version compatibility of this package based on the musl's ABI compatibility, which is currently 1.2 in Alpine 3.17.
Package: backend.ai-krunner-ubuntu
backend.ai-krunner-ubuntuBackend.AI Kernel Runner Package for Ubuntu-based KernelsHow to read below{distro}is a string likeubuntu,centos,alpine, etc.{distro-version}is a string likeubuntu18.04,centos7.6,alpine3.10, etc.Development$gitclonehttps://github.com/lablup/backend.ai-krunner-{distro}krunner-{distro}$cdkrunner-{distro}$pyenvvirtualenv3.8.6venv-krunner# you may share the same venv with other krunner projects$pyenvlocalvenv-krunner$pipinstall-Upipsetuptools$pipinstall-Uclick-e.How to updateModify Dockerfile and/or other contents.Incrementthe volume version numberspecified as a labelai.backend.krunner.vesrioninsrc/ai/backend/krunner/ubuntu/krunner-env.{distro-version}.dockerfile(there may be multiple dockerfiles).Runscripts/build.py {distro-version}.Repeat the above steps for each distro version.Incrementthe package version numberinsrc/ai/backend/krunner/ubuntu/__init__.pyrm -r dist/* build/*(skip if these directories do not exist and or are empty)python setup.py sdist bdist_wheeltwine upload dist/*Note thatsrc/ai/backend/krunner/{distro}/krunner-version.{distro-version}.txtfiles are
overwritten by the build script from the label.
Package: backend.ai-monitor-datadog
Backend.AI Statistics Monitoring Plugin with DatadogInstallationJustpip install backend.ai-monitor-datadoginside the virtualenv of the Backend.AI Manager and Agent.ConfigurationSet the following etcd configurations via the manager cli:backend.ai mgr etcd put config/plugins/stats_monitor/datadog/datadog_api_key "..."
backend.ai mgr etcd put config/plugins/stats_monitor/datadog/datadog_app_key "..."
Package: backend.ai-monitor-sentry
Backend.ai Error Monitoring Plugin with SentryInstallationJustpip install backend.ai-monitor-sentryinside the virtualenv of the Backend.AI Manager and Agent.ConfigurationSet a key with the Sentry DSN string in the cluster's etcd, by running the following command:backend.ai mgr etcd put config/plugins/error_monitor/sentry/dsn "<dsn>"
Package: backend.ai-test
Backend.AI Testing ToolkitAutomated test suites to validate an installation and integration of clients and serversHow to run CLI-based integration testIf there is no configuration script namedenv-tester.sh, copy it fromsample-env-tester.shand check the configuration env.$sourceenv-tester.sh$backend.aitestrun-cliNOTE: The ENV script is parsed with"^export (\w+)=(.*)$"regex pattern line by line.
Package: backend.ai-webserver
Backend.AI Web ServerA webapp hosting daemon which serves ourwebuias a SPA and proxies API requestsInstallationPrepare a Python virtualenv (Python 3.9 or higher) and a Redis server (6.2 or higher).$gitclonehttps://github.com/lablup/backend.ai-webserverwebserver$cdwebserver$pipinstall-U-e.$cpwebserver.sample.confwebserver.confModeIfservice.modeis set "webui" (the default), the webserver handles
PWA-style fallbacks (e.g., servingindex.htmlwhen there are no matching
files for the requested URL path).
The PWA must exclude/serverand/funcURL prefixes from its own routing
to work with the webserver's web sessions and the API proxy.If it is set "static", the webserver serves the static files as-is,
without any fallbacks or hooking, while preserving the/serverand/funcprefixed URLs and their functionalities.If you want to serve web UI in webserver with "webui" mode, prepare static web UI source by choosing one of the followings.Option 1: Build web UI from sourceBuildbackend.ai-webuiand copy all files underbuild/bundleinto thesrc/ai/backend/web/staticdirectory.Option 2: Use pre-built web UITo download and deploy web UI from pre-built source, do the following:git submodule initgit submodule updatecd src/ai/backend/web/staticgit checkout main  # or target branchgit fetchgit pullSetup configuration for webserverYou don't have to writeconfig.tomlfor the web UI as this webserver auto-generates it on-the-fly.Editwebserver.confto match with your environment.UsageTo execute web server, run command below. (for debugging, append a--debugflag)$python-mai.backend.web.server
Package: backend-base
Backend BaseAllows you to implement basic backend functionality for your application. Based on theDjango Framework. It contains the followings packages of the Django Framework:django.confdjango.core.exceptionsdjango.core.maildjango.utils.functionalDisclaimer: This project is not affiliated with the Django Framework. This package contains code extracted from the Django framework, with the purpose of using certain functionality in other projects that do not have to do with django. The code has certain modifications but in theory it is based on the django structureUsage in your projectConfigurationYou need to define the following environment variables:SETTINGS_MODULE_ENVIRONMENT_VARIABLE(default:config.settigs): The environment variable that contains the path to the settings module of your projectEnvironment variablesThe following environment variables are used:NameDefault ValueOverwrite requiredDescriptionReferenceSETTINGS_MODULE_ENVIRONMENT_VARIABLEconfig.settingsYesThe path to the settings module of your projectDjango DocumentationEMAIL_BACKENDbackend_base.mail.backends.smtp.EmailBackendYesThe email backend to use (Replacedjangoforbackend_base)Django DocumentationEMAIL_HOSTlocalhostYesThe host to use for sending email.Django DocumentationEMAIL_PORT25YesThe port to use for the SMTP server specified in EMAIL_HOST.Django DocumentationEMAIL_USE_LOCALTIMEFalseNoWhether to use the local time when formatting dates in email messages.Django DocumentationEMAIL_HOST_USER''YesUsername to use for the SMTP server specified in EMAIL_HOST.Django DocumentationEMAIL_HOST_PASSWORD''YesPassword to use for the SMTP server specified in EMAIL_HOST.Django DocumentationEMAIL_USE_TLSFalseNoWhether to use a TLS (secure) connection when talking to the SMTP server.Django DocumentationEMAIL_USE_SSLFalseNoWhether to use an implicit TLS (secure) connection when talking to the SMTP server.Django DocumentationEMAIL_TIMEOUTNoneNoA timeout in seconds for blocking operations like the connection attempt.Django DocumentationEMAIL_SSL_KEYFILENoneNoThe filename of a private key file for use in SSL connections.Django DocumentationEMAIL_SSL_CERTFILENoneNoThe filename of a certificate file for use in SSL connections.Django DocumentationDEFAUL_CHARSETutf-8NoThe default character set to use for the email.Django DocumentationSERVER_EMAILroot@localhostNoThe email address that error messages come from, such as those sent to ADMINS and MANAGERS.Django Documentation
Package: backendpy
BackendpyPython web framework for building the backend of your project!Some features:Asynchronous programming (ASGI-based projects)Application-based architecture and the ability to install third-party applications in a projectSupport of middlewares for different layers such as Application, Handler, Request or ResponseSupports events and hooksData handler classes, including validators and filters to automatically apply to request input dataSupports a variety of responses including JSON, HTML, file and… with various settings such as stream, gzip and…Router with the ability to define urls as Python decorator or as separate filesApplication-specific error codesOptional default database layer by the Sqlalchemy async ORM with management of sessions for the scope of each requestOptional default templating layer by the Jinja template engine…RequirementsPython 3.8+DocumentationDocumentation is available athttps://backendpy.readthedocs.io.Quick StartInstallation$pip3installbackendpyOr use the following command to install optional additional libraries:$pip3installbackendpy[full]You also need to install an ASGI server such as Uvicorn, Hypercorn or Daphne:$pip3installuvicornCreate Projectproject/main.pyfrombackendpyimportBackendpybp=Backendpy()Create Applicationproject/apps/hello/main.pyfrombackendpy.appimportAppfrom.handlersimportroutesapp=App(routes=[routes])project/apps/hello/handlers.pyfrombackendpy.routerimportRoutesfrombackendpy.responseimportTextroutes=Routes()@routes.get('/hello-world')asyncdefhello(request):returnText('Hello World!')Activate Applicationproject/config.ini[networking]allowed_hosts=127.0.0.1:8000[apps]active=project.apps.helloRun ProjectInside the project root path:$uvicornmain:bpCommand lineThe basic structure of a project mentioned above can also be created by commands:$backendpycreate_project--namemyprojectTo create a project with more complete sample components:$backendpycreate_project--namemyproject--fullOr to create an application:$backendpycreate_app--namemyapp$backendpycreate_app--namemyapp--full
Package: backend-setuptool
Packaging using setuptoolUpgrading buildpython -m pip install --upgrade buildBuild the packagepython -m buildPackage versioningpython -m pip install bumpver
bumpver init
bumpver update --minorAdd Resource Files to Your Packageinclude src/reader/*.tomlLicense Your PackageInstall Your Package Locallypython -m pip install -e .Uploading the distribution archivespython -m pip install --upgrade twine
python -m twine upload --repository testpypi dist/*Installing your newly uploaded packagepython -m pip install --index-url https://test.pypi.org/simple/ --no-deps backend_setuptool