Project: Prefect (Orchestration & Management / Scheduling & Orchestration)
File: aci.md

description: Run an agent and flows in the cloud with Azure Container Instances. tags: Docker containers agents cloud search: boost: 2 Microsoft Azure Container Instances (ACI) provides a convenient and simple service for quickly spinning up a Docker container that can host a Prefect Agent and execute flow runs. To follow this quickstart, you'll need the following: A A Prefect Cloud (Prefect Cloud Pro and Custom tier accounts can use a API key) A Azure CLI and Like most Azure resources, ACI applications must live in a resource group. If you dont already have a resource group youd like to use, create a new one by running the `az group create` command. For example, this example creates a resource group called `prefect-agents` in the `eastus` region: <div class='terminal'> ```bash az group create --name prefect-agents --location eastus ``` </div> Feel free to change the group name or location to match your use case. You can also run `az account list-locations -o table` to see all available resource group locations for your account. Prefect provides you can use to quickly stand up a container instance. These Docker images include Python and Prefect. For example, the image `prefecthq/prefect:2-python3.10` includes the latest release version of Prefect and Python 3.10. To create the container instance, use the `az container create` command. This example shows the syntax, but you'll need to provide the correct values for `[ACCOUNT-ID]`,`[WORKSPACE-ID]`, `[API-KEY]`, and any dependencies you need to `pip install` on the instance. These options are discussed below. <div class='terminal'> ```bash az container create \ --resource-group prefect-agents \ --name prefect-agent-example \ --image prefecthq/prefect:2-python3.10 \ --secure-environment-variables PREFECTAPIURL='https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/[WORKSPACE-ID]' PREFECTAPIKEY='[API-KEY]' \ --command-line "/bin/bash -c 'pip install adlfs s3fs requests pandas; prefect agent start -p default-agent-pool -q test'" ``` </div> When the container instance is running, go to Prefect Cloud and select the . Select default-agent-pool, then select the Queues tab to see work queues configured on this work pool. When the container instance is running and the agent has started, the `test` work queue displays "Healthy" status. This work queue and agent are ready to execute deployments configured to run on the `test` queue. !!! info "Agents and queues" The agent running in this container instance can now pick up and execute flow runs for any deployment configured to use the `test` queue on the `default-agent-pool` work pool. Let's break down the details of the `az container create` command used here. The `az container create command` creates a new ACI container. `--resource-group prefect-agents` tells Azure which resource group the new container is created in. Here, the examples uses the `prefect-agents` resource group created earlier. `--name prefect-agent-example` determines the container name you will see in the Azure Portal. You can set any name youd like here to suit your use case, but container instance names must be unique in your resource group. `--image prefecthq/prefect:2-python3.10` tells ACI which Docker images to run. The script above pulls a public Prefect image from Docker Hub. You can also build custom images and push them to a public container registry so ACI can access them. Or you can push your image to a private Azure Container Registry and use it to create a container

`--secure-environment-variables` sets environment variables that are only visible from inside the container. They do not show up when viewing the containers metadata. You'll populate these environment variables with a few pieces of information to of the container instance so it can communicate with your Prefect Cloud workspace: A Prefect Cloud API key.) The value specifying the API endpoint of your Prefect Cloud workspace. `--command-line` lets you override the containers normal entry point and run a command instead. The script above uses this section to install the `adlfs` pip package so it can read flow code from Azure Blob Storage, along with `s3fs`, `pandas`, and `requests`. It then runs the Prefect agent, in this case using the default work pool and a `test` work queue. If you want to use a different work pool or queue, make sure to change these values appropriately. Following the example of the tutorial, let's create a deployment that can be executed by the agent on this container instance. In an environment where you have , create a new folder called `healthtest`, and within it create a new file called `healthflow.py` containing the following code. ```python import prefect from prefect import task, flow from prefect import getrunlogger @task def say_hi(): logger = getrunlogger() logger.info("Hello from the Health Check Flow! ") @task def logplatforminfo(): import platform import sys from prefect.server.api.server import SERVERAPIVERSION logger = getrunlogger() logger.info("Host's network name = %s", platform.node()) logger.info("Python version = %s", platform.python_version()) logger.info("Platform information (instance type) = %s ", platform.platform()) logger.info("OS/Arch = %s/%s", sys.platform, platform.machine()) logger.info("Prefect Version = %s ", prefect.version) logger.info("Prefect API Version = %s", SERVERAPIVERSION) @flow(name="Health Check Flow") def healthcheckflow(): hi = say_hi() logplatforminfo(wait_for=[hi]) ``` Now create a deployment for this flow script, making sure that it's configured to use the `test` queue on the `default-agent-pool` work pool. ```bash prefect deployment build --infra process --storage-block azure/flowsville/healthtest --name health-test --pool default-agent-pool --work-queue test --apply healthflow.py:healthcheckflow ``` Once created, any flow runs for this deployment will be picked up by the agent running on this container instance. !!! note "Infrastructure and storage" This Prefect deployment example was built using the infrastructure type and Azure Blob Storage. You might wonder why your deployment needs process infrastructure rather than infrastructure when you are deploying a Docker image to ACI. A Prefect deployments infrastructure type describes how you want Prefect agents to run flows for the deployment. With `DockerContainer` infrastructure, the agent will try to use Docker to spin up a new container for each flow run. Since youll be starting your own container on ACI, you dont need Prefect to do it for you. Specifying process infrastructure on the deployment tells Prefect you want to agent to run flows by starting a process in your ACI container. You can use any storage type as long as you've configured a block for it before creating the deployment. Note that ACI instances may incur usage charges while running, but must be running for the agent to pick up and execute flow runs. To stop a container, use the `az container stop` command: <div class='terminal'> ```bash az container stop --resource-group prefect-agents --name prefect-agent-example ``` </div> To delete a container, use the `az container delete` command: <div class='terminal'> ```bash az container delete --resource-group prefect-agents --name prefect-agent-example ``` </div>