zyra 0.1.45
A tool to ingest data from various sources and formats, create imagery or video based on that data, and send the results to various locations for dissemination.
License: Apache-2.0
Requires Python: <4.0,>=3.10

# Zyra

[![PyPI version](https://img.shields.io/pypi/v/zyra.svg)](https://pypi.org/project/zyra/) [![Docs](https://img.shields.io/badge/docs-GitHub_Pages-0A7BBB)](https://noaa-gsl.github.io/zyra/) [![Chat with Zyra Assistant](https://img.shields.io/badge/ChatGPT-Zyra%20Assistant-00A67E?logo=openai&logoColor=white)](https://chatgpt.com/g/g-6897a3dd5a7481918a55ebe3795f7a26-zyra-assistant) [![DOI](https://zenodo.org/badge/854215643.svg)](https://doi.org/10.5281/zenodo.16923322)

[![Zyra Presentation](https://github.com/user-attachments/assets/24b250cd-f4f1-4f47-a378-abba43af253d)](https://docs.google.com/presentation/d/1hdB2qLgzdiHQUzB3u_Mv2gU1wdh8xbcWsOh-dXjz9ME/present?usp=sharing)

Zyra is an open-source Python framework for modular, reproducible data workflows — helping you import, process, visualize, and export scientific and environmental data.

## Highlights
- **Modular pieces:** connectors, processors, visualizers, and utilities.
- **CLI-first & streaming-friendly:** compose stages with pipes and configs.
- **Install what you need:** keep core small; add extras per stage.

## Quickstart

Install

```
pip install zyra
```

Try the CLI

```
zyra --help
```

Minimal example

```
# Acquire → Process → Visualize
zyra acquire http https://example.com/sample.grib2 -o sample.grib2
zyra process convert-format sample.grib2 netcdf --stdout > sample.nc
zyra visualize heatmap --input sample.nc --var VAR --output plot.png
```

### More Examples per Module

See module-level READMEs under `src/zyra/` for focused examples and options:
- Connectors/Ingest: `src/zyra/connectors/ingest/README.md`
- Processing: `src/zyra/processing/README.md`
- API (service): `src/zyra/api/README.md`
- MCP Tools: `src/zyra/api/mcp_tools/README.md`

## Learn More (Wiki)
- Getting Started: https://github.com/NOAA-GSL/zyra/wiki/Getting-Started-with-Zyra
- Workflow Stages: https://github.com/NOAA-GSL/zyra/wiki/Workflow-Stages
- Stage Examples: https://github.com/NOAA-GSL/zyra/wiki/Stage-Examples
- Install & Extras: https://github.com/NOAA-GSL/zyra/wiki/Install-Extras
- Pipeline Patterns: https://github.com/NOAA-GSL/zyra/wiki/Pipeline-Patterns
- Containers Overview: https://github.com/NOAA-GSL/zyra/wiki/Zyra-Containers-Overview-and-Usage
- API Routers & Endpoints: https://github.com/NOAA-GSL/zyra/wiki/Zyra-API-Routers-and-Endpoints
 - Zyra Assistant Instructions: https://github.com/NOAA-GSL/zyra/wiki/Zyra-Assistant-Instructions

## Stage Map
- Overview: https://github.com/NOAA-GSL/zyra/wiki/Workflow-Stages

### Swarm Orchestration
- `zyra swarm --plan samples/swarm/mock_basic.yaml --dry-run` prints the instantiated agents.
- Remove `--dry-run` to execute mock simulate→narrate agents; use `--memory provenance.db` to persist provenance and `--guardrails schema.rail` to enforce structured outputs.
- Guardrail validation requires the optional extra: `pip install "zyra[guardrails]"` (or `poetry install --with guardrails`) before using `--guardrails schema.rail`.
- To exercise the skeleton simulate/decide flow end-to-end, try `zyra swarm --plan samples/swarm/simulate_decide.yaml --dry-run` (or drop `--dry-run` to run the mock pipeline).
- Add `--log-events` to echo provenance events live, and `--dump-memory provenance.db` to inspect existing runs without executing new stages.
- Target specific stages or experiment with partial DAGs using `--agents acquire,visualize,narrate`; unknown stage names are rejected early so typos do not silently skip work.
- Control concurrency explicitly with `--parallel/--no-parallel` (the latter forces `max-workers=1`) and use `--max-workers N` when you want a fixed pool size.
- Override LLM settings for proposal/narrate stages with `--provider mock|openai|ollama|gemini|vertex`, `--model <name>`, and `--base-url <endpoint>` instead of editing `.env` or wizard config files. Gemini runs with either `GOOGLE_API_KEY` (Generative Language API) or Vertex credentials (`VERTEX_PROJECT`, `GOOGLE_APPLICATION_CREDENTIALS`, etc.).
- A real-world example lives in `samples/swarm/drought_animation.yaml`; run it with
  `poetry run zyra swarm --plan samples/swarm/drought_animation.yaml --memory drought.db`.
  Create `data/drought/` ahead of time, place `earth_vegetation.jpg` in your working directory (or adjust the manifest),
  and ensure Pillow is installed for `process pad-missing`.
- Preview planner output (JSON manifest with augmentation suggestions) before running: `poetry run zyra plan --intent "mock swarm plan"`.
- Plans are YAML/JSON manifests listing agents, dependencies (`depends_on`), and CLI args; see `samples/swarm/` to get started.

### Import (acquire/ingest)
- Docs: https://noaa-gsl.github.io/zyra/api/zyra.connectors.html
- Examples: https://github.com/NOAA-GSL/zyra/wiki/Stage-Examples

### Process (transform)
- Docs: https://noaa-gsl.github.io/zyra/api/zyra.processing.html
- Examples: https://github.com/NOAA-GSL/zyra/wiki/Stage-Examples

### Simulate
- Examples: https://github.com/NOAA-GSL/zyra/wiki/Stage-Examples

### Decide (optimize)
- Roadmap: https://github.com/NOAA-GSL/zyra/wiki/Roadmap-and-Tracking

### Visualize (render)
- Docs: https://noaa-gsl.github.io/zyra/api/zyra.visualization.html
- Examples: https://github.com/NOAA-GSL/zyra/wiki/Stage-Examples

### Narrate
- Roadmap: https://github.com/NOAA-GSL/zyra/wiki/Roadmap-and-Tracking

### Verify
- Roadmap: https://github.com/NOAA-GSL/zyra/wiki/Roadmap-and-Tracking

### Export (disseminate; legacy: decimate)
- Docs: https://noaa-gsl.github.io/zyra/api/zyra.connectors.html
- Examples: https://github.com/NOAA-GSL/zyra/wiki/Stage-Examples

### LLM Providers for Wizard/Narrate

Zyra’s Wizard, narrative swarm, and discovery semantic search all share the same provider configuration. Install `google-auth` when needed with `pip install "zyra[llm]"` (or `poetry install --with llm`). Key options:

| Provider | Configuration |
| --- | --- |
| OpenAI | `OPENAI_API_KEY`, optional `OPENAI_BASE_URL` |
| Ollama | Local server on `http://localhost:11434` (override with `OLLAMA_BASE_URL`) |
| Gemini (Generative Language API) | Set `GOOGLE_API_KEY`; optional `GENLANG_BASE_URL`, `VERTEX_MODEL` (default `gemini-2.5-flash`) |
| Gemini (Vertex AI) | Provide `VERTEX_PROJECT` (aliases: `GOOGLE_PROJECT_ID`, `GOOGLE_CLOUD_PROJECT`), `VERTEX_LOCATION` (default `us-central1`), and Application Default Credentials (`GOOGLE_APPLICATION_CREDENTIALS` pointing to a service-account JSON). Optional `VERTEX_MODEL`, `VERTEX_ENDPOINT`, `VERTEX_PUBLISHER`. |
| Mock | No credentials needed; deterministic offline responses. Useful for testing. |

Example `~/.zyra_wizard.yaml` targeting Gemini via Vertex:

```yaml
provider: gemini
model: gemini-2.5-flash
project: my-vertex-project
location: us-central1
```

## API & Reference Docs
- GitHub Pages: https://noaa-gsl.github.io/zyra/
- CLI reference: https://noaa-gsl.github.io/zyra/api/zyra.cli.html
- Modules:
  - Connectors: https://noaa-gsl.github.io/zyra/api/zyra.connectors.html
  - Processing: https://noaa-gsl.github.io/zyra/api/zyra.processing.html
  - Visualization: https://noaa-gsl.github.io/zyra/api/zyra.visualization.html
  - Transform: https://noaa-gsl.github.io/zyra/api/zyra.transform.html
  - Utils: https://noaa-gsl.github.io/zyra/api/zyra.utils.html
  - API (service): https://noaa-gsl.github.io/zyra/api/zyra.api.html

## Contributing
- Guide: [CONTRIBUTING.md](CONTRIBUTING.md)
- Discussions: https://github.com/NOAA-GSL/zyra/discussions
- Issues: https://github.com/NOAA-GSL/zyra/issues

## License
Apache-2.0 — see LICENSE