GitHub repository: kamal-marouane/Spark-Data-Analysis
Primary language: Jupyter Notebook
Description: Automated cloud-based data pipeline using Apache Spark and Kafka for large-scale cluster analysis. Infrastructure provisioned with Terraform, leveraging Google Cloud Data processing for batch jobs and real-time streaming. Includes CI/CD workflows and monitoring for optimized performance.