Package: prefect-monte-carlo
prefect-monte-carloVisit the full docshereto see additional examples and the API reference.Welcome!A collection of Prefect tasks and flows to interact with Monte Carlo from workflows.Getting StartedPython setupRequires an installation of Python 3.7+.We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.These tasks are designed to work with Prefect 2.0. For more information about how to use Prefect, please refer to thePrefect documentation.InstallationInstallprefect-monte-carlowithpip:pipinstallprefect-monte-carloThen, register this collection's blocks toview them on Prefect Cloud:prefectblockregister-mprefect_monte_carloNote, to use theloadmethod on Blocks, you must already have a block documentsaved through codeorsaved through the UI.Write and run a flowExecute a query against the Monte Carlo GraphQL APIfromprefectimportflowfromprefect_monte_carlo.graphqlimportexecute_graphql_operationfromprefect_monte_carlo.credentialsimportMonteCarloCredentials@flowdefexample_execute_query():monte_carlo_credentials=MonteCarloCredentials.load("my-mc-creds")result=execute_graphql_operation(monte_carlo_credentials=monte_carlo_credentials,operation="query getUser { getUser { email firstName lastName }}",)Create or update Monte Carlo lineagefromprefectimportflowfromprefect.contextimportget_run_contextfromprefect_monte_carlo.credentialsimportMonteCarloCredentialsfromprefect_monte_carlo.lineageimportcreate_or_update_lineage,MonteCarloLineageNode@flowdefmonte_carlo_orchestrator():current_flow_run_name=get_run_context().flow_run.namesource=MonteCarloLineageNode(node_name="source_dataset",object_id="source_dataset",object_type="table",resource_name="some_resource_name",tags=[{"propertyName":"dataset_owner","propertyValue":"owner_name"}],)destination=MonteCarloLineageNode(node_name="destination_dataset",object_id="destination_dataset",object_type="table",resource_name="some_resource_name",tags=[{"propertyName":"dataset_owner","propertyValue":"owner_name"}],)# `create_or_update_lineage` is a flow, so this will be a subflow run# `extra_tags` are added to both the `source` and `destination` nodescreate_or_update_lineage(monte_carlo_credentials=MonteCarloCredentials.load("my-mc-creds)source=source,destination=destination,expire_at=datetime.now()+timedelta(days=10),extra_tags=[{"propertyName":"flow_run_name","propertyValue":current_flow_run_name}])Conditionally execute a flow based on a Monte Carlo circuit breaker rulefromprefectimportflowfromprefect_monte_carlo.circuit_breakersimportskip_if_circuit_breaker_flippedfromprefect_monte_carlo.credentialsimportMonteCarloCredentialsmy_mc_creds=MonteCarloCredentials.load("my-mc-creds")rule_name="myRule"@flow@skip_if_circuit_breaker_flipped(monte_carlo_credentials=my_mc_credsrule_name=rule_name,)defconditional_flow():logger=get_run_logger()logger.info("If you see this, your circuit breaker rule was not breached!")ResourcesIf you encounter any bugs while usingprefect-monte-carlo, feel free to open an issue in theprefect-monte-carlorepository.If you have any questions or issues while usingprefect-monte-carlo, you can find help in either thePrefect Discourse forumor thePrefect Slack community.Feel free to star or watchprefect-monte-carlofor updates too!ContributingIf you'd like to help contribute to fix an issue or add a feature toprefect-monte-carlo, pleasepropose changes through a pull request from a fork of the repository.Here are the steps:Fork the repositoryClone the forked repositoryInstall the repository and its dependencies:pip install -e ".[dev]"Make desired changesAdd testsInsert an entry toCHANGELOG.mdInstallpre-committo perform quality checks prior to commit:pre-commit installgit commit,git push, and create a pull request