GitHub repository: kiran7041/incremental-load-delta-tables-gcp-databricks-event-driven-pipeline
Primary language: Jupyter Notebook
Description: This project sets up an event-driven pipeline to load data incrementally into Delta Tables using GCP and Databricks. The pipeline triggers on file arrivals, stages data in Delta tables, merges into a target table, and archives processed files. It leverages Delta Lake for efficient and reliable data processing.