On Demand DAG through the REST API
Channel: Apache Airflow
Date: 15/06/2022

Session presented by Mocheng Guo at Airflow Summit 2022

In this talk we want to present how Airbnb extends the REST api to support on-demand workload. A DAG object is created from a local environment like Jupyter notebook, serialized into binary and transported to the API. The API persists the DAG object into the meta DB and Airflow scheduler and worker are extended to process this new kind of DAG.