semvec/pypi-packages huggingface dataset:
flowtask 5.9.50
Framework for Task orchestration
License: Apache-2.0
Requires Python: >=3.10.1
Keywords: Flowtask, Data Integration, Task Orchestration, Task-Runner, Pipelines

# FlowTask #

FlowTask is a plugin-based, component-driven task execution framework for create complex Tasks.

FlowTask runs Tasks defined in JSON, YAML or TOML files, any Task is a combination of Components,
and every component in the Task run sequentially or depend of others, like a DAG.

Can create a Task combining Commands, Shell scripts and other specific Components (as TableInput: Open a Table using a datasource, DownloadFromIMAP: Download a File from a IMAP Folder, and so on), any Python Callable can be a Component inside a Task, or can extends UserComponent to build your own componets.

Every designed Task can run from CLI, programmatically, via RESTful API (using our aioHTTP-based Handler), called by WebHooks or even dispatched to a external Worker using our built-in Scheduler.

## Quickstart ##

```console
pip install flowtask
```

Tasks can organizated into directory structure like this:

tasks /
    ├── programs /
      ├── test /
           ├── tasks /

The main reason of this structure, is maintain organized several tasks by tenant/program, avoiding filling a directory with several task files.

FlowTask support "TaskStorage", a Task Storage is the main repository for tasks, main Task Storage is a directory in any filesystem path (optionally you can syncronize that path using git), but Tasks can be saved onto a Database or a S3 bucket.

## Dependencies ##

 * aiohttp (Asyncio Web Framework and Server) (required by navigator)
 * AsyncDB
 * QuerySource
 * Navigator-api
 * (Optional) Qworker (for distributing asyncio Tasks on distributed workers).

## Features ##

* Component-based Task execution framework with several components covering several actions (download files, create pandas dataframes from files, mapping dataframe columns to a json-dictionary, etc)
* Built-in API for execution of Tasks.

### How I run a Task? ###

Can run a Task from CLI:
```console
task --program=test --task=example
```

on CLI, you can pass an ENV (enviroment) to change the environment file on task execution.
```console
ENV=dev task --program=test --task=example
```

or Programmatically:
```python
from flowtask import Task
import asyncio

task = Task(program='test', task='example')
results = asyncio.run(task.run())
# we can alternatively, using the execution mode of task object:
results = asyncio.run(task())
```

### Requirements ###

* Python >= 3.9
* asyncio (https://pypi.python.org/pypi/asyncio/)
* aiohttp >= 3.6.2

### Contribution guidelines ###

Please have a look at the Contribution Guide

* Writing tests
* Code review
* Other guidelines

### Who do I talk to? ###

* Repo owner or admin
* Other community or team contact

### License ###

Navigator is licensed under Apache 2.0 License. See the LICENSE file for more details.
infrahub-sdk 1.19.0b0
Python Client to interact with Infrahub
License: Apache-2.0
Requires Python: <3.15,>=3.10


<!-- markdownlint-disable -->
![Infrahub Logo](https://assets-global.website-files.com/657aff4a26dd8afbab24944b/657b0e0678f7fd35ce130776_Logo%20INFRAHUB.svg)
<!-- markdownlint-restore -->

# Infrahub by OpsMill

[Infrahub](https://github.com/opsmill/infrahub) by [OpsMill](https://opsmill.com) is taking a new approach to Infrastructure Management by providing a new generation of datastore to organize and control all the data that defines how an infrastructure should run.

At its heart, Infrahub is built on 3 fundamental pillars:

- **Powerful Schema**: that's easily extensible
- **Unified Version Control**: for data and files
- **Data Synchronization**: with traceability and ownership

## Infrahub SDK

The Infrahub Python SDK greatly simplifies how you can interact with Infrahub programmatically.

More information can be found in the [Infrahub Python SDK Documentation](https://docs.infrahub.app/python-sdk/introduction).

## Installation

The Infrahub SDK can be installed using the pip package installer. It is recommended to install the SDK into a virtual environment.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install infrahub-sdk
```

### Installing optional extras

Extras can be installed as part of the Python SDK and are not installed by default.

#### ctl

The ctl extra provides the `infrahubctl` command, which allows you to interact with an Infrahub instance.

```bash
pip install 'infrahub-sdk[ctl]'
```

#### tests

The tests extra provides all the components for the testing framework of Transforms, Queries and Checks.

```bash
pip install 'infrahub-sdk[tests]'
```

#### all

Installs infrahub-sdk together with all the extras.

```bash
pip install 'infrahub-sdk[all]'
```

### Development setup with UV

If you're developing the SDK and using UV for dependency management, you can install specific dependency groups:

```bash
# Install development dependencies
uv sync --all-groups --all-extras

# Install specific groups
uv sync --group tests               # Testing dependencies only
uv sync --group lint                # Linting dependencies only
uv sync --group ctl                 # CLI dependencies only
uv sync --all-groups --all-extras   # All optional dependencies
```