Saravanan666/pypi-org huggingface dataset:
Package: prefect-docker
prefect-dockerWelcome!Prefect integrations for working with Docker.Note! TheDockerRegistryCredentialsinprefect-dockeris a unique block, separate from theDockerRegistryinprefectcore. WhileDockerRegistryimplements a few functionality from bothDockerHostandDockerRegistryCredentialsfor convenience, it does not allow much configuration to interact with a Docker host.Do not useDockerRegistrywith this collection. Instead, useDockerHostandDockerRegistryCredentials.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-dockerwithpip:pipinstallprefect-dockerThen, register toview the blockon Prefect Cloud:prefectblockregister-mprefect_dockerNote, to use theloadmethod on Blocks, you must already have a block documentsaved through codeorsaved through the UI.Pull image, and create, start, log, stop, and remove Docker containerfromprefectimportflow,get_run_loggerfromprefect_docker.imagesimportpull_docker_imagefromprefect_docker.containersimport(create_docker_container,start_docker_container,get_docker_container_logs,stop_docker_container,remove_docker_container,)@flowdefdocker_flow():logger=get_run_logger()pull_docker_image("prefecthq/prefect","latest")container=create_docker_container(image="prefecthq/prefect",command="echo 'hello world!' && sleep 60")start_docker_container(container_id=container.id)logs=get_docker_container_logs(container_id=container.id)logger.info(logs)stop_docker_container(container_id=container.id)remove_docker_container(container_id=container.id)returncontainerUse a custom Docker Host to create a Docker containerfromprefectimportflowfromprefect_dockerimportDockerHostfromprefect_docker.containersimportcreate_docker_container@flowdefcreate_docker_container_flow():docker_host=DockerHost(base_url="tcp://127.0.0.1:1234",max_pool_size=4)container=create_docker_container(docker_host=docker_host,image="prefecthq/prefect",command="echo 'hello world!'")create_docker_container_flow()ResourcesIf you encounter any bugs while usingprefect-docker, feel free to open an issue in theprefect-dockerrepository.If you have any questions or issues while usingprefect-docker, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to ⭐️ or watchprefect-dockerfor updates too!DevelopmentIf you'd like to install a version ofprefect-dockerfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/PrefectHQ/prefect-docker.gitcdprefect-docker/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-firebolt
prefect-fireboltWelcome!prefect-fireboltis a collection of Prefect integrations for working with Firebolt data warehouses.Fireboltis a Cloud Data Warehousing solution that helps its users streamline their Data Analytics and access to insights. For more information about getting started with Firebolt, readFirebolt's getting started guide.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-fireboltwithpip:pipinstallprefect-fireboltThen, register toview the Firebolt blockson Prefect Cloud:prefectblockregister-mprefect_fireboltNote, to use theloadmethod on Blocks, you must already have a block documentsaved through codeorsaved through the UI.Write and run a flowExecute a query against a Firebolt database:fromprefectimportflowfromprefect_fireboltimportFireboltCredentials,FireboltDatabase,query_firebolt@flowdefrun_firebolt_query():firebolt_database_block=FireboltDatabase(database="travel",credentials=FireboltCredentials(username="arthur.dent@hitchhikers.com",password="dont42panic"),)results=query_firebolt(database=firebolt_database_block,query="SELECT * FROM ex_intergalactic_trips LIMIT 100",)returnresultsrun_firebolt_query()ResourcesIf you encounter any bugs while usingprefect-firebolt, feel free to open an issue in theprefect-fireboltrepository.If you have any questions or issues while usingprefect-firebolt, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to ⭐️ or watchprefect-fireboltfor updates too!DevelopmentIf you'd like to install a version ofprefect-fireboltfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/PrefectHQ/prefect-firebolt.gitcdprefect-firebolt/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-fivetran
prefect-fivetranWelcome!Prefect integrations with FivetranGetting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-fivetranwithpip:pipinstallprefect-fivetranWrite and run a flowfromprefectimportflowfromprefect_fivetranimportFivetranCredentialsfromprefect_fivetran.connectorsimportfivetran_sync_flow@flowdefmy_flow():...fivetran_credentials=FivetranCredentials(api_key="my_api_key",api_secret="my_api_secret",)fivetran_result=awaitfivetran_sync_flow(fivetran_credentials=fivetran_credentials,connector_id="my_connector_id",schedule_type="my_schedule_type",poll_status_every_n_seconds=30,)...my_flow()ResourcesIf you encounter any bugs while usingprefect-fivetran, feel free to open an issue in theprefect-fivetranrepository.If you have any questions or issues while usingprefect-fivetran, you can find help in either thePrefect Discourse forumor thePrefect Slack community.DevelopmentIf you'd like to install a version ofprefect-fivetranfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/pubchimps/prefect-fivetran.gitcdprefect-fivetran/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-gcp
prefect-gcpprefect-gcpmakes it easy to leverage the capabilities of Google Cloud Platform (GCP) in your flows, featuring support for Vertex AI, Cloud Run, BigQuery, Cloud Storage, and Secret Manager.Visit the full docshere.InstallationTo start usingprefect-gcp:pipinstallprefect-gcpTo install extras, seehere.ContributingThanks for thinking about chipping in! Check out thisstep-by-step guideon how to get started.
Package: prefect-github
prefect-githubWelcome!Prefect integrations interacting with GitHub.The tasks within this collection were created by a code generator using the GitHub GraphQL schema.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-githubwithpip:pipinstallprefect-githubThen, register toview the blockon Prefect Cloud:prefectblockregister-mprefect_githubNote, to use theloadmethod on Blocks, you must already have a block documentsaved through codeorsaved through the UI.Write and run a flowfromprefectimportflowfromprefect_githubimportGitHubCredentialsfromprefect_github.repositoryimportquery_repositoryfromprefect_github.mutationsimportadd_star_starrable@flow()defgithub_add_star_flow():github_credentials=GitHubCredentials.load("github-token")repository_id=query_repository("PrefectHQ","Prefect",github_credentials=github_credentials,return_fields="id")["id"]starrable=add_star_starrable(repository_id,github_credentials)returnstarrablegithub_add_star_flow()ResourcesIf you encounter any bugs while usingprefect-github, feel free to open an issue in theprefect-githubrepository.If you have any questions or issues while usingprefect-github, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to ⭐️ or watchprefect-githubfor updates too!DevelopmentIf you'd like to install a version ofprefect-githubfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/PrefectHQ/prefect-github.gitcdprefect-github/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-gitlab
prefect-gitlabWelcome!prefect-gitlabis a Prefect collection for working with GitLab repositories.Getting StartedPython setupRequires an installation of Python 3.8 or higher.We recommend using a Python virtual environment manager such as pipenv, conda, or virtualenv.This integration is designed to work with Prefect 2.3.0 or higher. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-gitlabwithpip:pipinstallprefect-gitlabThen, register theblock types) in this integration to view the storage block type on Prefect Cloud:prefectblockregister-mprefect_gitlabNote, to use theloadmethod on a block, you must already have a block documentsaved.Creating a GitLab storage blockIn Pythonfromprefect_gitlabimportGitLabRepository# public GitLab repositorypublic_gitlab_block=GitLabRepository(name="my-gitlab-block",repository="https://gitlab.com/testing/my-repository.git")public_gitlab_block.save()# specific branch or tag of a GitLab repositorybranch_gitlab_block=GitLabRepository(name="my-gitlab-block",reference="branch-or-tag-name"repository="https://gitlab.com/testing/my-repository.git")branch_gitlab_block.save()# private GitLab repositoryprivate_gitlab_block=GitLabRepository(name="my-private-gitlab-block",repository="https://gitlab.com/testing/my-repository.git",access_token="YOUR_GITLAB_PERSONAL_ACCESS_TOKEN")private_gitlab_block.save()In the UIClick on theBlocksmenu, then click the+button in the page header to open the block catalog:Then, find theGitLabblock and click theAddbutton:Finally, enter your repository information in the form and clickCreate:ResourcesIf you encounter any bugs while usingprefect-gitlab, feel free to open an issue in theprefect-gitlabrepository.If you have any questions or issues while usingprefect-gitlab, you can find help in thePrefect Slack community.Feel free to ⭐️ or watchprefect-gitlabfor updates!DevelopmentIf you'd like to install a version ofprefect-gitlabfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/prefecthq/prefect-gitlab.gitcdprefect-gitlab/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-great-expectations
prefect-great-expectationsWelcome!Prefect integration for interacting with Great Expectations.Great Expectationsis a Python library for data quality. It provides a framework to validate your state of data.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda, or virtualenv.These tasks are designed to work with Prefect 2. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-great-expectationswithpip:pipinstallprefect-great-expectationsWrite and run a flowfromprefectimportflowfromprefect_great_expectationsimportrun_checkpoint_validation@flowdefexample_flow():run_checkpoint_validation(checkpoint_name="my_checkpoint")example_flow()TutorialFor a larger example, check out thetutorial.ResourcesIf you encounter any bugs while usingprefect-great-expectations, feel free to open an issue in theprefect-great-expectationsrepository.If you have any questions or issues while usingprefect-great-expectations, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to ⭐️ or watchprefect-great-expectationsfor updates too!DevelopmentIf you'd like to install a version ofprefect-great-expectationsfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/PrefectHQ/prefect-great-expectations.gitcdprefect-great-expectations/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-hightouch
prefect-hightouchWelcome!Prefect integrations for interacting with Hightouch.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-hightouchwithpip:pipinstallprefect-hightouchThen, register toview the blockon Prefect Cloud:prefectblockregister-mprefect_hightouch.credentialsNote, to use theloadmethod on Blocks, you must already have a block documentsaved through codeorsaved through the UI.Trigger a sync run and wait for completionfromprefectimportflowfromprefect_hightouchimportHightouchCredentialsfromprefect_hightouch.syncsimporttrigger_sync_run_and_wait_for_completion@flowdefsync_flow():hightouch_credentials=HightouchCredentials.load("hightouch-token")sync_metadata=trigger_sync_run_and_wait_for_completion(hightouch_credentials=hightouch_credentials,sync_id=12345,full_resync=True,max_wait_seconds=1800,poll_frequency_seconds=10,)returnsync_metadatasync_flow()List, get, and trigger syncsfromprefectimportflowfromprefect_hightouchimportHightouchCredentials,api_modelsfromprefect_hightouch.syncsimport(list_sync,get_sync,list_sync_runs,trigger_run,trigger_run_custom,)@flowdefhightouch_sync_flow():hightouch_credentials=HightouchCredentials.load("hightouch-token")# list all syncssyncs=list_sync(hightouch_credentials,order_by=api_models.ListSyncOrderBy.CREATEDAT)# get first syncsync_id=syncs[0].idsync=get_sync(hightouch_credentials,sync_id)# list previous runssync_runs=list_sync_runs(hightouch_credentials,sync_id)# trigger by idsync_run=trigger_run(hightouch_credentials,sync_id,json_body=api_models.TriggerRunInput(full_resync=False),)# trigger by slugsync_slug=syncs[0].slugsync_run_2=trigger_run_custom(hightouch_credentials,json_body=api_models.TriggerRunCustomInput(sync_slug=sync_slug,full_resync=False,),)returnsync_runshightouch_sync_flow()Call API endpointsIf an API endpoint is not exposed as a task, you can call the underlying API endpoint functions, but note, these arenotPrefect tasks.fromprefect_hightouch.credentialsimportHightouchCredentialsfromprefect_hightouch.api_client.api.defaultimportlist_destinationcredentials=HightouchCredentials.load(token="my-service-token")client=credentials.get_client()response=list_destination.sync_detailed(client=client)data=response.parsed.dataResourcesIf you encounter any bugs while usingprefect-hightouch, feel free to open an issue in theprefect-hightouchrepository.If you have any questions or issues while usingprefect-hightouch, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to ⭐️ or watchprefect-hightouchfor updates too!DevelopmentIf you'd like to install a version ofprefect-hightouchfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/PrefectHQ/prefect-hightouch.gitcdprefect-hightouch/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-huaweicloud
Prefect Huawei Cloud BlockWelcome!The prefect-huaweicloud collection enables you to use HUAWEI CLOUD cloud service capabilities in Prefect more efficiently.Getting Startedpipinstall-U"prefect"gitclonehttps://gitee.com/HuaweiCloudDeveloper/huaweicloud-prefect-block-python.gitcdhuaweicloud-prefect-block-python

prefectblockregister--file__init__.pyContributingThank you for wanting to contribute, you can contribute tothis code repository.If you have any questions, consult the community.
Package: prefect-hwc
Prefect Huawei Cloud BlockWelcome!The prefect-hwc collection enables you to use HUAWEI CLOUD service capabilities in Prefect more efficiently.Getting Startedpipinstall-U"prefect"pipinstallprefect-hwc

prefectblockregister--moduleprefect_huaweicloudContributingThank you for wanting to contribute, you can contribute tothis code repository.If you have any questions, consult the community.
Package: prefect-jinja
prefect-jinjaWelcome!prefect-jinjais a collection of pre-built Prefect tasks that can be used to quickly build Prefect flows to interact
withJinja.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-jinjawithpip:pipinstallprefect-jinjaThen, register toview the blockon Prefect Cloud:prefectblockregister-mprefect_jinja.blocks!!! note "Load Block"
To use theloadmethod on Blocks, you must already have a block documentsaved through codeorsaved through the UI.Write and run a flowRender templates from a directoryUsing theJinjaEnvironmentBlockblock and thejinja_render_from_templatefunction to render an HTML page.!!! note "Remote storage"
We recommend configuringremote file storagefor task execution withJinjaEnvironmentBlockblock and thejinja_render_from_templatefunction. This ensures tasks have access to
templates files, particularly when accessing a instance outside the execution environment.fromprefectimportflowfromprefect_jinjaimportJinjaEnvironmentBlock,jinja_render_from_template@flowdefsend_welcome_flow(username:str):jinja_environment=JinjaEnvironmentBlock(search_path="templates",namespace={"company_name":"Acme",})returnjinja_render_from_template("welcome.html",jinja_environment,username=username)print(send_welcome_flow(username="Neymar"))Render templates from a stringUsing thejinja_render_from_stringfunction to render a string.fromprefectimportflowfromprefect_jinjaimportjinja_render_from_string@flowdefsend_hello_flow(username:str):returnjinja_render_from_string("Hello, {{name}}!",username=username)print(send_hello_flow(username="Robinho"))ResourcesIf you encounter any bugs while usingprefect-jinja, feel free to open an issue in theprefect-jinjarepository.If you have any questions or issues while usingprefect-jinja, you can find help in either thePrefect Discourse forumor thePrefect Slack community.DevelopmentIf you'd like to install a version ofprefect-jinjafor development, clone the repository and perform an editable
install withpip:gitclonehttps://github.com/hallenmaia/prefect-jinja.gitcdprefect-jinja/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall
Package: prefect-jupyter
prefect-jupyterWelcome!Prefect integrations interacting with Jupyter.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-jupyterwithpip:pipinstallprefect-jupyterRun a parameterized notebookprefect-jupyteruses papermill under the hood. Check outpapermill's documentationto learn how to parametrize the notebook.The following code shows how to run a parameterized notebook:fromprefectimportflowfromprefect_jupyterimportnotebook@flowdefexample_execute_notebook():nb=notebook.execute_notebook("test_notebook.ipynb",parameters={"num":5})body=notebook.export_notebook(nb)output_path="executed_notebook.ipynb"withopen(output_path,"w")asf:f.write(body)returnoutput_pathexample_execute_notebook()ResourcesFeedbackIf you encounter any bugs while usingprefect-jupyter, feel free to open an issue in theprefect-jupyterrepository.If you have any questions or issues while usingprefect-jupyter, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to star or watchprefect-jupyterfor updates too!ContributingIf you'd like to help contribute to fix an issue or add a feature toprefect-jupyter, pleasepropose changes through a pull request from a fork of the repository.Here are the steps:Fork the repositoryClone the forked repositoryInstall the repository and its dependencies:pipinstall-e".[dev]"Make desired changesAdd testsInsert an entry toCHANGELOG.mdInstallpre-committo perform quality checks prior to commit:pre-commitinstallgit commit,git push, and create a pull request
Package: prefect-kv
prefect-kvWelcome!A simple key-value store for use with Prefect.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-kvwithpip:pipinstallprefect-kvUsing a key-value storeThe entire store is saved to Prefect's API each time a key is updated.
Each time a key is read, the entire store is loaded from Prefect's API.
This exposes a simple way to persist and retrieve values across flow runs and tasks, independently of your dataflow.fromprefect_kvimportKVStore# Create a new KV store named 'demo-store'store=KVStore(name="demo-store")# Set a value in the storestore.set("foo","test")# Get a value from the storeprint(store.get("foo"))# Get a value from the store with a defaultprint(store.get("does-not-exist, "that's okay!"))# Set a value in the store using item syntaxstore["bar"]="another test"# Get a value from the store using item syntaxprint(store["bar"])# View the whole storeprint(f"Here's the whole thing:{store.dict()}")If you exit the process and create a new store object with same name, you will still be able to access your values:fromprefect_kvimportKVStore# Get the existing KV store named 'demo-store'store=KVStore(name="demo-store")print(store["foo"])print(store["bar"])ResourcesIf you encounter any bugs while usingprefect-kv, feel free to open an issue in theprefect-kvrepository.If you have any questions or issues while usingprefect-kv, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to ⭐️ or watchprefect-kvfor updates too!DevelopmentIf you'd like to install a version ofprefect-kvfor development, clone the repository and perform an editable install withpip:gitclonehttps://github.com/madkinsz/prefect-kv.gitcdprefect-kv/

pipinstall-e".[dev]"# Install linting pre-commit hookspre-commitinstall