Package: prophecy-build-tool
prophecy-build-toolProphecyis designed to enable all users to be productive with data engineering. It also replaces legacy ETL
products.
To learn more about Prophecy visithttps://docs.prophecy.io/.Prophecy-build-tool(PBT) allows you to quickly build projects generated by Prophecy (your standard Spark Scala and
PySpark pipelines) to integrate them with your own CI / CD (e.g. Github Actions), build system (e.g. Jenkins), and
orchestration (e.g. Databricks Workflows).For the latest information on how to use Prophecy-build-tool please visithttps://docs.prophecy.io/releases-and-deployment/prophecy-build-tool/.InstallationTo install PBT, simply run:pip3installprophecy-build-toolRunningTo build and deploy your Prophecy project containing Python/Scala projects and Databricks Jobs runpbtdeploy--path/path/to/your/prophecy_project/Sample output:Prophecy-build-toolv1.0.0

Found1pipelines:customers_orders(python)Found1jobs:daily_report

Building1pipeline🚰Buildingpipelinepipelines/customers_orders[1/1]runningbuildrunningbuild_py

✅Buildcomplete!

Deploying1job⏱Deployingjobjobs/daily_report[1/1]Uploadingcs-1.0-py3-none-any.whltodbfs:/FileStore/prophecy/artifacts/...Updatinganexistingjob:daily_report

✅Deploymentcomplete!BuildTo just build your Prophecy project containing Python/Scala projects and Databricks Jobs runpbtbuild--path/path/to/your/prophecy_project/DeployTo build and Deploy all jobs in your project, run deploy commandpbtdeploy--path/path/to/your/prophecy_project/It's also possible to only deploy jobs associated with givenfabrics, you can provide fabrics id's (comma separated)
to just deploy only those jobs.pbtdeploy--fabric-ids3,7--path/path/to/your/prophecy_project/By default,deploycommand builds all pipelines and then deploys them, if you want to skip building all pipelines
( this could be useful, if you are running adeploycommand  after runningdeployorbuildpreviously.)pbtdeploy--skip-builds--path/path/to/your/prophecy_project/TestTo run all unit tests in your Prophecy project containing Python/Scala projects and Databricks Jobs runpbttest--path/path/to/your/prophecy_project/Sample output:Prophecy-build-toolv1.0.1

Found1jobs:daily
Found1pipelines:customers_orders(python)UnitTestingpipelinepipelines/customers_orders[1/1]=============================testsessionstarts==============================platformdarwin--Python3.8.9,pytest-7.1.2,pluggy-1.0.0--/Library/Developer/CommandLineTools/usr/bin/pythoncachedir:.pytest_cachemetadata:Nonerootdir:/Users/kiran/proj2/pipelines/customers_orders/codeplugins:html-3.1.1,metadata-2.0.2collecting...collected1itemtest/TestSuite.py::CleanupTest::test_unit_test_0PASSED[100%]==============================1passedin17.42s==============================✅Unittestforpipeline:pipelines/customers_orderssucceeded.ValidateTo quickly validate if all your pipelines are not brokenpbtvalidate--path/path/to/your/prophecy_project/Sample output:Prophecy-build-toolv1.0.3.4

Projectname:resources.HelloWorld
Found1jobs:default_schedule
Found4pipelines:customers_orders(python),report_top_customers(python),join_agg_sort(python),farmers-markets-irs(python)Validating4pipelinesValidatingpipelinepipelines/customers_orders[1/4]Pipelineisvalidated:customers_ordersValidatingpipelinepipelines/report_top_customers[2/4]Pipelineisvalidated:report_top_customersValidatingpipelinepipelines/join_agg_sort[3/4]Pipelineisvalidated:join_agg_sortValidatingpipelinepipelines/farmers-markets-irs[4/4]Pipelineisvalidated:farmers-markets-irsIntegrating with Github actionsProphecy-built-tool (PBT) can be integrated with CI/CD (eg: Github Actions). A sample Github Actions .yml file that builds, tests and deploys the project on every code push is mentioned below:name:Example CIon:[push]env:DATABRICKS_HOST:"https://<<databricks-host>>.databricks.com"DATABRICKS_TOKEN:"<<databricks-token>>"FABRIC_NAME:"fabric-name"jobs:build:runs-on:ubuntu-lateststeps:-uses:actions/checkout@v3-name:Set up JDK 11uses:actions/setup-java@v3with:java-version:'11'distribution:'adopt'-name:Set up Python 3.xuses:actions/setup-python@v4with:python-version:'3.x'# Install all python dependencies# prophecy-libs not included here because prophecy-build-tool# takes care of it by reading each pipeline's setup.py-name:Install dependenciesrun:|python3 -m pip install --upgrade pippip3 install build pytest wheel pytest-html pyspark prophecy-build-tool-name:Run PBT buildrun:pbt build --path .-name:Run PBT testrun:pbt test --path .-name:Run PBT deployrun:pbt deploy --path .