py-azpe 0.1.7
Minimal Azure Policy engine with file and sdk backends
License: MIT
Requires Python: >=3.7

[![CI](https://github.com/leonavevor/AZURE-POLICY-ENGINE/actions/workflows/ci.yml/badge.svg)](https://github.com/leonavevor/AZURE-POLICY-ENGINE/actions/workflows/ci.yml)

# Azure Policy Engine (minimal)

This is a small Python package that helps to harmonize Azure Policy management and deployment workflows programmatically.

There is no dedicated PyPI package specifically named "Azure Policy Deployment Engine," but the closest and officially supported option for managing and deploying Azure Policy programmatically in Python is the azure-mgmt-policyinsights package, which provides access to Azure Policy Insights and allows programmatic interaction with policies, assignments, and compliance data within Azure. For actual policy management and deployment (creating, updating, assigning policies), you generally use the Azure SDK for Python and various management libraries.​
Additionally, there are community projects such as the "Azure-Policy-Engine" on GitHub, which offers Python tooling to manage Azure Policies outside the portal, CLI, or REST API, using the Azure SDK for Python as a backend. This is not a PyPI package but can be installed from source.​

For broader deployment requirements, you may also need packages like azure-mgmt-resource or azure-mgmt-deploymentmanager, but these are focused on resource and deployment management, not policy-specific automation.​ So this is where py_azpe (Azure Policy Engine) comes in.

| Package/Tool                                     | Purpose                              | PyPI Availability | Notes                                         |
|--------------------------------------------------|--------------------------------------|-------------------|-----------------------------------------------|
| azure-mgmt-policyinsights                        | Policy Insights API (compliance/data) | Yes               | Official Azure SDK; compliance only           |
| Azure-Policy-Engine (GitHub)                     | Programmatic Policy CRUD operations  | No                | Community tool; install from source           |
| azure-mgmt-resource/deploymentmanager            | ARM resource + deployment mgmt       | Yes               | Not policy-specific                           |
| py_azpe - Azure Policy Engine (this Github Repo) | Create/manage/deploy Azure Policies  | Yes               | Lightweight Python package for policy mgmt    |


This README documents the current feature set, CLI usage, packaging/dev setup, examples and a short HOWTO for mapping JSON into SDK models.

## Overview

The **AZURE-POLICY-ENGINE** (available as `py_azpe` on PyPI) is a lightweight Python package designed to help users **create, manage, and optionally deploy Azure Policy artifacts**. The repository includes documentation covering:

- Current features
- Command-line interface (CLI) usage
- Development setup and packaging
- Practical examples
- A brief guide on converting JSON into SDK models

It's a handy tool for streamlining Azure Policy workflows in Python environments.

Supported backends

- file: author and manage policy artifacts as local JSON files. The default folder structure (under the configured `policy_dir`, defaults to the package `policies/` folder) is:
  - `definitions/` — policyDefinition JSON files
  - `assignments/` — policyAssignment JSON files
  - `initiatives/` — policySetDefinition (initiative) JSON files

- sdk: when the Azure management SDKs are available the engine uses `azure.mgmt.resource.PolicyClient` and related models to perform operations. If the SDK is not present (or an SDK call fails and a credential is available) the engine falls back to ARM REST calls using an ARM token acquired from the provided credential.

- azcli: when the Azure CLI (`az`) is installed and the user is authenticated the engine can invoke `az` to perform policy operations. This backend uses `az rest` under the hood to call ARM endpoints and is useful when you prefer CLI tooling or when SDK/REST usage is constrained. It requires `az` installed and an authenticated session (e.g., `az login`).

Key features

- Create/list/get policy definitions and initiatives (policy sets).
- Create/update/delete/list policy assignments.
- Assign initiatives (policy sets) via assignment resources.
- Scope validation and support for different assignment scopes: subscription, resource-group and management-group levels.
- Dry-run mode that prints the prepared payload and effective scope without making API calls.
- SDK model construction: the engine attempts to construct SDK model objects (PolicyAssignment, PolicyDefinition, PolicySetDefinition) when the SDK is available; otherwise it works with raw dicts.

## CLI quick reference

The CLI entry point is `python -m azure_policy_engine.cli`.

Common flags

- `--backend` (file|sdk|azcli) — choose local file backend, SDK-backed operations, or Azure CLI-backed operations (default: file).
- `--policies-dir` — override the policies directory (defaults to package `policies/`).
- `--subscription-id` — subscription id used by SDK/ARM REST calls when a scope isn't explicitly provided.
- `--scope` — provide an explicit scope for assignment operations (must start with `/`).
- `--mgmt-group` — convenience shorthand; builds the management group scope `/providers/Microsoft.Management/managementGroups/{id}`.
- `--dry-run` — print the prepared payload and scope, do not perform the SDK/REST call.

Commands (high level)

- `list` — list local or SDK policy definitions
- `get <name>` — get a single policy definition
- `deploy <name> <file>` — create/update a policy definition
- `assignments` — list assignments
- `create-assignment <name> <file>` — create an assignment (supports `--dry-run` and `--scope` / `--mgmt-group`)
- `update-assignment <name> <file>`
- `delete-assignment <name>`
- `add-assignment <name> <file>` — write file directly into assignments/ (file backend only)
- `list-initiatives` — list initiatives (policy sets)
- `assign-initiative <initiative-name> <assignment-name> <file>` — create an assignment that references an initiative

Examples

Dry-run creating an assignment at a management group (shorthand) using `azcli` backend:

```bash
python -m py_azpe.cli --backend azcli --mgmt-group my-mg --dry-run create-assignment myAssign examples/assign-mg.json
```

Create an assignment at a subscription scope using the SDK backend:

```bash
python -m py_azpe.cli --backend sdk --scope /subscriptions/0000 create-assignment myAssign examples/assign.json
```

Create an assignment at a subscription scope using the Azure CLI backend:

```bash
python -m py_azpe.cli --backend azcli --scope /subscriptions/0000 create-assignment myAssign examples/assign.json
```

Assign an initiative (dry-run):

```bash
python -m py_azpe.cli --backend sdk --mgmt-group my-mg --dry-run assign-initiative myInitiative myAssign examples/assign-mg.json
```

## Scope validation and supported forms

The engine implements a pragmatic scope validator to catch common user mistakes. Accepted forms include:

- subscription root: `/subscriptions/{subscriptionId}`
- resource group: `/subscriptions/{subscriptionId}/resourceGroups/{rg}`
- management group: `/providers/Microsoft.Management/managementGroups/{mgId}`

If you pass an explicit scope with `--scope`, it must start with `/`. Use `--mgmt-group` as a convenience shorthand to build management-group scopes.

## Dry-run details

`--dry-run` is available for create/update/delete assignment and assign-initiative commands. When used the CLI prints a JSON object that contains:

- `dry_run`: true
- `action`: the intended action
- `scope`: effective normalized scope
- `name`: the assignment name
- `payload`: the assignment body or SDK-model-as-dict (if SDK model was constructed)

This helps you validate payload shape and scope before making changes.

## Examples folder

An `examples/` folder is provided with a sample management-group assignment JSON and a small helper script:

- `examples/assign-mg.json` — example assignment JSON that references an initiative at management-group scope.
- `examples/run-assign-mg.sh` — example script that ensures `AZURE_SUBSCRIPTION_ID` is set, runs `az login` interactively if no SP credentials are present, and executes a dry-run create-assignment against `my-mg`.

Make the script executable locally and run it:

```bash
chmod +x examples/run-assign-mg.sh
bash examples/run-assign-mg.sh
```

## Packaging, pinned SDKs and dev setup

This project includes minimal packaging and pinned dependencies to make local development and CI reproducible:

- `pyproject.toml` — basic project metadata and pinned runtime dependencies used by the analyzer/CI.
- `requirements.txt` — pinned versions for local `pip install -r requirements.txt` convenience.
- `tox.ini` — tiny tox configuration to run tests in an isolated environment.

Pinned SDKs used here (change if you need different versions):

- azure-identity==1.12.0
- azure-mgmt-resource==23.0.0
- azure-mgmt-policyinsights==1.0.0
- requests==2.31.0

Dev/test

Install editable package + dev deps in a venv:

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e .
pip install -r requirements.txt
```

Run tests:

```bash
PYTHONPATH=. pytest -q
```

or with tox:

```bash
tox
```

## CI

A GitHub Actions workflow is added at `.github/workflows/ci.yml` that:

- Checks out the repository
- Sets up Python
- Installs the package in editable mode and the pinned requirements
- Runs the test suite with pytest

You can extend the workflow (matrix Python versions, caching, linting) as needed.

## Release and publishing (CI)

Detailed release and publishing instructions are maintained in `RELEASE.md` to keep the main README focused. See `RELEASE.md` for the full release workflow, repository variables, secrets, and example commands.

For convenience, the workflow file is at `.github/workflows/release.yml`.

## HOWTO: map a policy JSON into SDK model fields

When deploying via the SDK it's more robust to construct SDK model objects and pass them to the client methods.

- Ensure `properties.policyDefinitionId` contains the full resource id of the definition or initiative.
- Construct the SDK model (PolicyAssignment / PolicyDefinition / PolicySetDefinition) using the JSON `properties` as kwargs where possible.
- Use `model.as_dict()` to verify the final payload shape.
- The engine contains helper methods (`_make_policy_assignment_model`, `_make_policy_definition_model`, `_make_policy_set_definition_model`) that attempt to construct SDK models when the SDK is available and fall back to dicts otherwise.

## Troubleshooting

- If `--backend sdk` fails due to missing credentials, provide a `DefaultAzureCredential` compatible credential (e.g., `az login` for interactive flows, or set `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`, and `AZURE_CLIENT_SECRET` for a service principal).
- If using `--backend azcli` ensure `az` is installed and authenticated:
  - Install: follow https://aka.ms/azcli
  - Authenticate: `az login` or `az login --service-principal -u <id> -p <secret> --tenant <tenant>`

## Authentication via environment variables

The engine supports authenticating using Azure environment variables. You can either provide a service principal (recommended for CI/non-interactive flows) or rely on DefaultAzureCredential (interactive `az login`, managed identity, etc.).

Service principal (export into your shell):

```bash
export AZURE_CLIENT_ID="<YOUR_CLIENT_ID_HERE>"
export AZURE_CLIENT_SECRET="<YOUR_CLIENT_SECRET_HERE>"
export AZURE_TENANT_ID="<YOUR_TENANT_ID_HERE>"
export AZURE_SUBSCRIPTION_ID="<YOUR_SUBSCRIPTION_ID_HERE>"
```

Quick dry-run example (SDK backend)

```bash
# with SP creds exported as above
python -m py_azpe.cli --backend sdk --mgmt-group my-mg --dry-run create-assignment myAssign examples/assign-mg.json
```

Default credential (interactive / managed identity):

- For local interactive auth, run:

```bash
az login
export AZURE_SUBSCRIPTION_ID="<YOUR_SUBSCRIPTION_ID_HERE>"
```

How the code uses these variables:

- If `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, and `AZURE_TENANT_ID` are present, the engine will create an `azure.identity.ClientSecretCredential` using those values.
- Otherwise the engine falls back to `azure.identity.DefaultAzureCredential` (so `az login` or a managed identity will work).
- `AZURE_SUBSCRIPTION_ID` is used when the SDK/backends need a subscription context.

Quick try-it commands (zsh/bash):

```bash
# install dependencies into a venv
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# export SP creds and run the CLI using SDK backend
export AZURE_CLIENT_ID="<ID>"
export AZURE_CLIENT_SECRET="<SECRET>"
export AZURE_TENANT_ID="<TENANT>"
export AZURE_SUBSCRIPTION_ID="<SUB>"
python -m py_azpe.cli --backend sdk list
```

## Contributing

Thank you for considering contributing to the Azure Policy Engine project — contributions are welcome! To make collaborating smooth please follow these simple guidelines:

- Issues: Open an issue to discuss bugs, feature requests or design changes before implementing larger work.
- Branches: Use a short, descriptive branch name (e.g. `fix/readme-typo`, `feat/azcli-backend`).
- Commits: Keep commits focused and use clear messages. Prefer small commits that make review easier.
- Pull requests:
  - Target the `main` branch (or the branch named in the repository's contribution guidelines).
  - Include a brief description of the change and the motivation.
  - Link any related issue(s).
  - Add unit tests for new behavior and update existing tests if behavior changes.
- Tests: Run the test suite locally before opening a PR:

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
PYTHONPATH=. pytest -q
```

- Style: The project aims for straightforward, idiomatic Python. Keep changes readable and add docstrings where helpful.
- CI: Pull requests should pass the repository CI checks (tests + any linters) before merging. If you need help getting a branch to pass CI, describe the issue in the PR and maintainers can assist.

If you want to help but are unsure where to start, check the issue tracker for labels like "good first issue" or "help wanted".

## License

This project is licensed under the MIT License — see the accompanying `LICENSE` file for the full text. In short, you are free to use, copy, modify and distribute this software with attribution and without warranty.