Apache Airflow's versatility allows you to set up any type of workflow. Airflow can run ad hoc workloads not related to any interval or schedule. However, it is most suitable for pipelines that change slowly, are related to a specific time interval, or are pre-scheduled.
In this context, slow change means that once the pipeline is deployed, it is expected to change from time to time (once every several days or weeks, not hours or minutes). This has to do with the lack of versioning for Airflow pipelines.
Airflow is best at handling workflows that run at a specified time or every specified time interval. You can trigger the pipeline manually or using an external trigger (e.g. via REST API).
You can use Apache Airflow to schedule the following:
ETL pipelines that extract data from multiple sources, and run Spark jobs or other data transformations
Machine learning model training
Automated generation of reports
Backups and other DevOps tasks. Learn more about Machine Learning Operations