Package: prefect-azure
prefect-azureVisit the full docshereto see additional examples and the API reference.Welcome!prefect-azure is a collection of prebuilt Prefect tasks that can be used to quickly construct Prefect flows.Getting StartedPython setupRequires an installation of Python 3.7+We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-azurewithpippipinstallprefect-azureTo use Blob Storage:pipinstall"prefect-azure[blob_storage]"To use Cosmos DB:pipinstall"prefect-azure[cosmos_db]"To use ML Datastore:pipinstall"prefect-azure[ml_datastore]"A list of available blocks inprefect-azureand their setup instructions can be foundhere.Download a blobfromprefectimportflowfromprefect_azureimportAzureBlobStorageCredentialsfromprefect_azure.blob_storageimportblob_storage_download@flowdefexample_blob_storage_download_flow():connection_string="connection_string"blob_storage_credentials=AzureBlobStorageCredentials(connection_string=connection_string,)data=blob_storage_download(blob="prefect.txt",container="prefect",azure_credentials=blob_storage_credentials,)returndataexample_blob_storage_download_flow()Usewith_optionsto customize options on any existing task or flow:custom_blob_storage_download_flow=example_blob_storage_download_flow.with_options(name="My custom task name",retries=2,retry_delay_seconds=10,)Run a command on an Azure container instancefromprefectimportflowfromprefect_azureimportAzureContainerInstanceCredentialsfromprefect_azure.container_instanceimportAzureContainerInstanceJob@flowdefcontainer_instance_job_flow():aci_credentials=AzureContainerInstanceCredentials.load("MY_BLOCK_NAME")container_instance_job=AzureContainerInstanceJob(aci_credentials=aci_credentials,resource_group_name="azure_resource_group.example.name",subscription_id="<MY_AZURE_SUBSCRIPTION_ID>",command=["echo","hello world"],)returncontainer_instance_job.run()Use Azure Container Instance as infrastructureIf we havea_flow_module.py:fromprefectimportflow,get_run_logger@flowdeflog_hello_flow(name="Marvin"):logger=get_run_logger()logger.info(f"{name}said hello!")if__name__=="__main__":log_hello_flow()We can run that flow using an Azure Container Instance, but first create the infrastructure block:fromprefect_azureimportAzureContainerInstanceCredentialsfromprefect_azure.container_instanceimportAzureContainerInstanceJobcontainer_instance_job=AzureContainerInstanceJob(aci_credentials=AzureContainerInstanceCredentials.load("MY_BLOCK_NAME"),resource_group_name="azure_resource_group.example.name",subscription_id="<MY_AZURE_SUBSCRIPTION_ID>",)container_instance_job.save("aci-dev")Then, create the deployment either on the UI or through the CLI:prefectdeploymentbuilda_flow_module.py:log_hello_flow--nameaci-dev-ibcontainer-instance-job/aci-devVisitPrefect Deploymentsfor more information about deployments.For more tips on how to use tasks and flows in a Collection, check outUsing Collections!Azure Container Instance WorkerThe Azure Container Instance worker is an excellent way to runPrefect projectson Azure.To get started, create an Azure Container Instances typed work pool:prefect work-pool create -t azure-container-instance my-aci-work-poolThen, run a worker that pulls jobs from the work pool:prefect worker start -n my-aci-worker -p my-aci-work-poolThe worker should automatically read the work pool's type and start an
Azure Container Instance worker.ResourcesIf you encounter and bugs while usingprefect-azure, feel free to open an issue in theprefect-azurerepository.If you have any questions or issues while usingprefect-azure, you can find help in either thePrefect Discourse forumor thePrefect Slack communityFeel free to star or watchprefect-azurefor updates too!ContributingIf you'd like to help contribute to fix an issue or add a feature toprefect-azure, pleasepropose changes through a pull request from a fork of the repository.Here are the steps:Fork the repositoryClone the forked repositoryInstall the repository and its dependencies:pip install -e ".[dev]"Make desired changesAdd testsInsert an entry toCHANGELOG.mdInstallpre-committo perform quality checks prior to commit:pre-commit installgit commit,git push, and create a pull request