Saravanan666/pypi-org huggingface dataset:
Package: workerprocess
Worker ProcessThis package provides a wrapper to create standalone worker processes.Example WorkerWorkers are created by extending the BaseWorker class and implementing a tick
method to execute then calling .main() on the class. This will start an
infinite loop calling that function.The worker can be rate limited withmax_ticks_per_second.The worker can be stopped gracefully by sending a SIGTERM to the process.>>> import time
...
... from workerprocess import BaseWorker
...
...
... class ExampleWorker(BaseWorker):
...
...     max_ticks_per_second = 10
...
...     def tick(self):
...         print 'Tick!'
...         time.sleep(1)
...
... ExampleWorker.main()Asighupmethod on the function will be called if the process receives a
SIGHUPRunning the WorkerThe easiest way to be able to run the worker is by adding a console_script
entry point in your setup.py:entry_points=”””
[console_scripts]
example_worker_process = yourpackage.yourmodule:ExampleWorker.main
“””,After installing your package you will be able to run the commandexample_worker_processfrom the command line.
Package: workerqueue
workerqueueA thread-based worker queueInstallationpipinstallworkerqueueUsagefromworkerqueueimportWorkerQueuefromtimeimportsleepdefmyfunc(arg1,arg2):print(f"{arg1}*2={arg2}")sleep(0.5)myqueue=WorkerQueue(myfunc,thread_count=2)foriinrange(10):myargs=i,i*2myqueue.put(myargs)sleep(1)print("Calling stop()...")myqueue.stop()print("Finished processing")
Package: workers-kv.py
What is this?This is the api wrapper of Cloudflare Workers KV.Thisrepowasn't compatible with bulk import, so I made my own wrapper instead.You can use this to store and retrieve data from your Cloudflare Workers KV much easier.Usageimportworkers_kv# get these values from your Cloudflare account# Create namespacenamespace=workers_kv.Namespace(account_id="WORKERS_KV_ACCOUNT",namespace_id="WORKERS_KV_NAMESPACE",api_key="TOKEN")# List all keyskeys=namespace.list_keys()# Get value with keyvalue=namespace.read("keyhere")# Set value with key# automatically use bulk import if you have more than 2 key-value pairsnamespace.write({"key1":"value1","key2":"value2"})# Delete single keynamespace.delete_one("key1")# Delete multiple keys (bulk delete)namespace.delete_many(["key1","key2"])Installationpip install -U git+https://github.com/Alpaca131/workers-kv-py.git
Package: workfl
workflWorkflow MarkupLanguageIntroductionworkfl is a lightweight markup language for simple workflow diagrams. It's designed to be quick to write and easy to read.Example:A
B
D

A
C
DYou can try it out for yourself atworkfl.ws.The workfl python package can convert workfl markup into dictionaries of nodes and connections. There is also a method to export the workflow intomermaidsyntax.Installationpip install workflUsageimportworkflmarkup="A\nB"flow=workfl.ws(markup)print(flow.nodes)# A dictionary of nodesprint(flow.connections)# A dictionary of connectionsmermaid=flow.to_mermaid()print(mermaid)By defaultto_mermaid()will render a chart flowing from Top to Bottom. This can be changed by declaring adirectione.g.to_mermaid(direction="LR")(Left to Right). There are 4 options:OptionDirectionTBTop to Bottom (Default)BTBottom to TopLRLeft to RightRLRight to LeftSyntaxA full guide can be found herehttps://workfl.ws/?help=true.To DoTest scripts.
Package: workflomics-benchmarker
Workflomics BenchmarkerLibrary used to execute workflows (in CWL) and benchmark them as part of the Workflomics ecosystem.CreditsTheworkflomics benchmarkerscript was developed byNauman Ahmedas part of thecontainersrepository, but was since migrated to its own repository (seePR #49) to be published as a stand-alone package.RequirementsPython 3.9+PoetryDocker or Singularity runningInstallationpoetryinstallUsageThe command is used with Docker or Singularity service running. It will execute the workflow and benchmark it.Dockerworkflomicsbenchmarktests/data/which is equivalent topythonsrc/benchmarker/workflomics.pybenchmarktests/data/The results will be stored in the./tests/datafolder.SingularityFinally, you can run the test with Singularity. This will require you to have Singularity installed and running, and to use the--singularityflag.pythonsrc/benchmarker/workflomics.pybenchmarktests/data/--singularityTestingTo run the tests, you can use the following command:poetryrunpytest-sThe tests will execute a workflow and benchmark it (require Docker running). The results will be stored in the./tests/datafolder.
Package: workflow2executable
workflow2executableTurn Galaxy Workflows into standalone scripts.Free software: MIT licenseDocumentation:https://workflow2executable.readthedocs.io.Turn Galaxy Workflows into standalone scripts.Works only with regular dataset inputs for now. Might ultimately fit better into planemo.The basic idea is that a workflow with completely defined input section
describes itself:workflow2executable28d1e2d715476c2chttps://usegalaxy.org--script_pathworkflow.pyShould create a python script that will run a workflow on a given Galaxy server.
Useage can be seen withpythonworkflow.py--helpUsage:workflow.py[OPTIONS]RunSelectlastnlinesworkflowOptions:--input_datasetPATH--number_of_lines_to_keepINTEGER-a,--api_keyTEXTAPIkeytouseforrunningworkflow-g,--galaxy_urlTEXTGalaxyURLtouseforrunningworkflow[default:https://usegalaxy.org]-h,--history_idTEXTHistoryIDthatwillcontainworkflowresults-n,--new_history_nameTEXTCreateanewhistorywiththisname.WillnotbeusedifhistoryIDisprovided.--publish/--privatePublishhistory?--monitor_invocation/--no_monitor_invocationPrintworkflowinvocationstatusupdates?--workflow_id_overrideTEXTOverrideworkflowtorunwiththisworkflowidoruuid--quietTEXTDecreaseloggingoutput--helpShowthismessageandexit.CreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2019-08-13)First release on PyPI.
Package: workflow-engine
Workflow EngineintroductionThis module is used to implement workflows.
Every workflow executes a list of tasks.Installationpython-mpipinstallworkflow_engineImplementationProcessorA Workflow processor is the main instance which will handle a list of entities
and passes them to the assigned tasks. The handled entities should inherit
from the workflow model type.Since the workflow processor is an abstract class, the following methods have
to be implemented in the inherited class:get_default_entriesreturns a list of unparsed entriesget_entryparses the raw entries retrieved by the previous method to an
object which inherits from theWorkflowModelfilterdefine some preconditionsget_tasksreturns a list of instances which inherit from the workflow
taskwrite_entryhandle a processed (successfully or unsuccessfully) entry.
Write it to the cache or to the diskclosehandle the finale state of the workflowModelBasic implementation of an entity. Delivers field and methods to check, wether
the task has already been executed, if there is a related error or if the
entry should be executed again.TaskBasic implementation of a task which should be executed for a workflowStatusThe following use cases can be handled:flow typeExceptionStatusExceptionOnSuccessskip--xinterruptxx-repeat(-)xxExceptionin case of an Exception which is not an instance ofStatusExceptionthe workflow will be interrupted. An operator has the
possibility to set the repeat propertyStatusExceptionwill interrupt the workflow and set theStatusobject
which can contain any repeat propertyOnSuccesscan return aStatuscan contain any repeat property. Toskipthe task it is not necessary to implement a dedicated status. Just
return aStatusto exit the taskException HandlingIn case of unexpected Exceptions the workflow for the handled entry will be
cancelled. The stacktrace will be logged. An unique id is assigned to eachexception. The error id will be persisted into your status file and into the
logs.
Package: workflowhub
A Community Framework for Enabling Scientific Workflow Research and DevelopmentThis Python package provides a collection of tools for:Analyzing traces of actual workflow executions;Producing recipes structures for creating workflow recipes for workflow generation; andGenerating synthetic realistic workflow traces.InstallationWorkflowHub is available onPyPI.
WorkflowHub requires Python3.6+ and has been tested on Linux and macOS.RequirementsGraphvizWorkflowHub usespygraphvizand thus needs the graphviz package installed (version 2.16 or later).
You can install graphviz easily on Linux with your favorite package manager,
for example for Debian-based distributions:sudo apt-get install graphviz libgraphviz-devand for RedHat-based distributions:sudo yum install python-devel graphviz-develOn macOS you can usebrewpackage manager:brew install graphvizInstallation using pipWhilepipcan be used to install WorkflowHub, we suggest the following
approach for reliable installation when many Python environments are available:$ python3 -m pip install workflowhubRetrieving the latest unstable versionIf you want to use the latest WorkflowHub unstable version, that will contain
brand new features (but also contain bugs as the stabilization work is still
underway), you may consider retrieving the latest unstable version.Cloning fromWorkflowHub's GitHubrepository:$ git clone https://github.com/workflowhub/workflowhub
$ cd workflowhub
$ pip install .Get in TouchThe main channel to reach the WorkflowHub team is via the support email:support@workflowhub.org.Bug Report / Feature Request:our preferred channel to report a bug or request a feature is viaWorkflowHub'sGithub Issues Track.Citing WorkflowHubWhen citing WRENCH, please use the following paper. You should also actually read
that paper, as it provides a recent and general overview on the framework.@inproceedings{ferreiradasilva2020works,
  title = {WorkflowHub: Community Framework for Enabling Scientific Workflow Research and Development},
  author = {Ferreira da Silva, Rafael and Pottier, Lo\"ic and Coleman, Tain\~a and Deelman, Ewa and Casanova, Henri},
  booktitle = {2020 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS)},
  year = {2020},
  pages = {49--56},
  doi = {10.1109/WORKS51914.2020.00012}
}
Package: workflow-machinery
machineryState-aware workflow orchestrator.This project aims to bring stateful workflows, by defining a list of actions
representing multiple microservices.Define a suite of actions to process events, abstract intelligence for your microservices.Once defined, use your workflows to process your Events.ExampleWe have 3 microservices:Theuppermicroservice that transforms texts into uppercaseThereversemicroservice that reverses a textThespace-countermicroservice that returns the number of space in a stringSuppose we want to process some text by making it uppercase and reversing it.We will define the following definitions:examples/definitions/service-upper.jsonexamples/definitions/service-reverse.jsonexamples/definitions/service-space-counter.jsonThen, we can define a Workflow inexamples/definitions/workflow-process-text.json.After having created our Workflow, we can send our first Event !We will send the content ofexamples/definitions/event-text.jsonGoalsRegister services (ex: a new API)Define a simple Workflow (subsequent calls to different services)Allow async (expose endpoint for webhook?)VisualisationComponentsAPI (WIP)Could be divided into Consumer API and Management API.Register servicesDefine Workflows (json with arbitrary service definition)Treat EventData structuresService{"name":"my-service-name","address":"http://my-service:5000/","spec":{"inputs":{"message":{"type":"string","description":"The text to transform"}},"outputs":{"message":{"type":"string","description":"The transformed text"}}}}input:name: The name of the Serviceaddress: The URL of the Servicespec: The inputs/outputs of this Servicespec.inputs: Each input with its type and descriptionspec.outputs: Each output with its type and descriptionoutput: TBD
(service-id, ack date)EventPOST/e/<workflow-id>{"data":{"message":"The Sun will rise in the morning !"}}POSTing an Event will return anevent-id, an ack date and a status.
input:data: The Event's data thatmustmatch the Workflow's inputs.Workflow{"name":"my-workflow","spec":{"services":["service-1","service-2"],"inputs":{"message":{"type":"string","description":"The text to process"}},"outputs":{"message":{"type":"string","description":"The processed text"}},"operations":{}}}input:name: The name of the Workflowspec: The spec of the Workflowspec.services: The Services to be called, in orderspec.inputs: The inputs of this Workflowspec.outputs: The outputs of this workflowoutput: TBD
(workflow-id, ack date)
Package: workflow_manager
# Workflow managerPython implementation of task-based workflow manager.This package enables an easy wrap of any functionality that has dependencies on other functionality within your codebase.A simple use case would be a step by step wizard that has multiple success and failure scenarios.  This package enables instantiation of all task rules in the wizard and then a simple manager wrapper to execute the workflow in one call.The package also provides an ability to view the history of the workflow for debugging purposes.## HowGiven the following business tasks:```task onetask twotask threetask fourtask fivetask six```And the following business rules:```if task one succeeds, task two and task three will execute in sequenceif task one fails, task five will executeif task two succeeds, task four will executeif task two fails, task five will executeif task three fails, task six will executeif task four fails, task six will executeif task five succeeds, task six will execute```This module will set up a workflow that, based on status of the task, will execute the proper dependencies in the correct order.The module will also short circuit any calls on failure scenarios but will execute all failure dependencies required to completely clean up your workflow.For concrete examples, check out `tests/test_workflow.py`.## Setup`pip install workflow_manager`Create your task, inhertit from `workflow_manager.task.Task` class, and overwrite the `execute` method with your own logic:```pythonfrom workflow_manager.task import Taskclass CustomTask(Task):def __init__(self):super().__init__('my custom task')def execute(self, **kwargs):# your logic hereif success:return (Task.success_state(), 'result', 'in', 'a', 'list')else:return (Task.failure_state(), 'this failed because of reasons')class AnotherTask(Task):def __init__(self):super().__init__('some other task')def execute(self, **kwargs):# your logic hereif success:return (Task.success_state(), 'result', 'in', 'a', 'list')else:return (Task.failure_state(), 'this failed because of reasons')```Then, add your business rules.```pythoncustomTask = CustomTask()anotherTask = AnotherTask()customTask.on_success(anotherTask, someOtherTask)customTask.on_failure(cleanupTask)anotherTask.on_success(keepItGoingTask)anotherTask.on_failure(cleanupTask)```You can validate your workflow by printing your initial task (the one that will initiate the workflow):```pythonstr(customTask) # prints the entire workflow as jsoncustomTask.to_dict() # returns a dictionary of the workflow```Finally, simply register the initial task (the one that will initiate the workflow), and call `run` fuction:```pythonfrom workflow_manager.manager import Managermanager = Manager()manager.register_initial_task(customTask)manager.run()```If you want to see what happened after the workflow ends, you can call `show_executed_flow` method, which will return a list of tasks and the parameters.`manager.show_executed_flow()`
Package: workflow-nodes
workflow-nodesworkflow-nodesis a collection of various tools written in Python 3, which
are also usable inside aworkflowas nodes. Each node is an executable
command line tool providing the--xmlhelpinterface, which can be used to
obtain a machine readable representation of any command line tool and its
parameters (see alsoxmlhelpy). There are nodes for
many different tasks, including data conversion, transport and visualization
tools.For installation and usage instructions, please see the documentation:Stable (reflecting the latest release):https://workflow-nodes.readthedocs.io/en/stable/Latest (reflecting thedevelopbranch):https://workflow-nodes.readthedocs.io/en/latest/
Package: workflow-notification
workflow-notificationused to send notification and logs  to email, groupme and...siasky.create a file called dev.notification.yaml and include the required variables
notifications:
  - groupme:
      bot_id: ""
  - email:
      to: ""
      from: ""
      smtp_host: ""
      smtp_port:
      smtp_username: ""
      smtp_password: ""
      smtp_tls: true
      subject: "hello from {{program.notifications}}"Usagepip install workflow-notificationfromsecret_assistantimportnotificationnotification("testing notifications").info().sendmessage().send_mail()notification("testing notifications").warning().sendmessage().send_mail()notification("testing notifications").critical().sendmessage().send_mail()TODO:siasky for logs sendinguse trace to hook into stdout for logging .
Package: workflow-rdf
workflow-rdfA collection ofpyironJobs with RDF elements. Structures and jobs are automatically annotated usingCMSOandPROV-O, allowing for querying of the generated data.InstallationWe strongly recommend creating a conda environment for the installation.Mambacan allow the creation of conda environments. If you have existing conda installation, you can do:conda install -c conda-forge mambaOnce a conda distribution is available, the following steps will help set up an environment to usepyscal_rdf. First step is to clone the repository.git clone https://github.com/pyscal/rdfjobs.gitAfter cloning, an environment can be created from the included file-cd workflow_rdf
mamba env create -f environment.ymlThis will create an environment calledrdfjobs. It can be activated by,conda activate workflow_rdfthen, installworkflow_rdfusing,pip install .Usingworkflow_rdfA detailed documentation is coming soon. Please check theexamplesfolder for some examples.AcknowledgementsThis work is supported by theNFDI-Matwerkconsortia.
Package: workflows
Workflows enables light-weight services to process tasks in a message-oriented
environment.It is comprised of a communications layer (workflows.transport) that provides a
common interface to queues and topics over different transport providers, a
service abstraction layer (workflows.frontend) which encapsulates a service in
a separate process, tools for status reporting, logging and controlling
services via a transport mechanism, and a service class skeleton alongside
example services (workflows.services.common_serviceet al.).
To achieve more complex workflows services can be interconnected usingrecipeswhereby the output of one service can be directed
onwards to other services.
Package: workflow-sandbox
workflow-sandboxDummy project used to test GitHub actions and Cirrus CI for building and publishing releases of Python packages.This will be a (dummy) Python package on test PyPI in its own right, with package nameworkflow-sandboxand module nameworkflow_sandbox.PracticesThis dummy projectlooselyadopts therelease flowbranching model/strategy. (Loosley as I'm not going to always use feature branches.)Versioning is introspected using the excellentsetuptools_scm, using therelease-branch-semverversion scheme.When a release candidate was identified (a574270, C below):A new branchmaintenance/0.2.xwas created, to service the minor release0.2.The commit was taggedv0.2.0rc1, the first release candidate of this version.Subsequent untagged commits are versioned (with local schemeno-local-version):Onmasteras0.3.0devN, e.g.0.3.0dev1(ee03672, E below);Onmaintenance/0.2.xas0.2.0rc2.dev1(6006a6f, D below).In this example, the release candidate was deemed satisfactory and a dummy commit was used to create6006a6f(F below) so that it could be uniquely tagged asv0.2.0.In pictures, this looks like:D -- F [maintenance/0.2.x]
      /     ^ [v0.2.0]
B -- C---E [master]
     ^
[v0.2.0rc1]Update:See#13for a discussion of why this was necessary and a better alternative (tag C/D with both v0.2.0rc1 and v0.2.0).Actual releases, e.g.0.2.0and bugfix releases0.2.1,0.2.2, etc. will be tagged onmaintenance/0.2.xalone. This branch will contain no development, but cherry pick bug-fixes from master.RoadmapGitHub ActionsCreate release usingactions/create_release.Publish release on GitHub usingactions/upload-release-asset...... with thebodyabove extracted from the appropriate release notes based on git tags (see#14).Usingpypa/gh-action-pypi-publish:All tagged releases (including candidates) are published on PyPI and TestPyPI automatically.All untagged pushes tomasterandmaintenance/*branches are published on TestPyPI automatically with an appropriatedevversion suffix.Investigate triggering off creation vs branch push events (see#8).Build multiple wheels using a branching workflowactions/[up/down]load-artifact(see e.g.chrisjbillington/desktop-app#7) and/ormanylinux(GitHub ActionRalfG/python-wheels-manylinux-build).Automated documentation build and push togh-pagesbranch.CirrusCIFormatting usingblack.Linting usingflake8.AcknowledgementsMuch of this is guided by the practices of other development communities, and many helpful conversations with@chrisjbillingtonand@philipstarkey.
Package: workflow-tools
CLI tools for GitHub Actions.Automate writing GitHub workflow configs with a generator toolAutomate setting GitHub secrets for repositoriesIntegrate the tools into your pipeline for setting up new microservicesRationaleMicroservice architecture may have dozens and dozens of lookalikes services that require similar CI/CD workflows.
With infrastructure as code approach taken by theGitHub Actions, why not using workflows generation?
Setting up a new microservice repository may also be automated. This is whereworkflow-toolscome in handy.ExamplesLet’s setGitHub Secretsfor a repository. First, get apersonal access tokenin GitHub settings.
Then set up a secret usingworkflow-tools:workflow_secret--owner=anna-money--repo=workflow-tools\--token="YOUR-PERSONAL-ACCESS-TOKEN"\update--key=MY_SECRET_KEY--value=MY_VALUENow let’s use a fragment ofJinja2template for a GitHub Actions workflow to generate resulting config:WORKFLOW_RUNNER_VERSION=ubuntu-18.04WORKFLOW_PYTHON27=2.7WORKFLOW_PYTHON37=3.7\workflow_generator# Press Enter to start pasting Jinja2 workflow template into stdinjobs:test:runs-on:[[workflow.runner_version]]strategy:matrix:python:-[[workflow.python27]]-[[workflow.python37]]# Press Ctrl+D to render resulting workflow
# For real workflow templates use reading/writing from/to a file, load variables from envfilejobs:test:runs-on:ubuntu-18.04strategy:matrix:python:-2.7-3.7HelpSeedocumentationfor more details. Use--helpflag for each tool in the package.InstallationJust run:pipinstall-Uworkflow-toolsContributingSeeCONTRIBUTING.rst.Changelog0.6.0 (2020-04-01)Update readthedocs config (#8) by @pilosus0.5.0 (2020-04-01)Fix autodocs (#7) by @pilosusAdd check docs step in CI/CD (#7) by @pilosusAdd badges to README (#7) by @pilosus0.4.0 (2020-04-01)Extend CI/CD flow with package check (#5) by @pilosusFix bugs with setup.py long description by @pilosus0.3.0 (2020-03-31)Add documentation and ReadTheDocs integration (#3) by @pilosus0.2.0 (2020-03-30)Use GitHub Actions for the project (#2) by @pilosus0.1.0 (2020-03-30)Move internal ANNA project to open source (#1) by @pilosusOriginally developed by Vitaly Samigullin (@pilosus) as internal ANNA project