Position: Big Data Engineer
Category: Python
English level: upper
Years of experience: 4

- Optimized Spark pipelines that led to x3 time reduction.
- Improved data ingestion logic that allowed the user to reload data hourly instead of daily.
- Refactored architecture of CDC pipelines that allowed schema evolution.
- Led the migration from NiFi to Airflow that reduced data indigestion infra costs by x15 times.
- Led the migration from NiFi to Airflow that reduced data indigestion infra costs by x15 times.
- Silver and Gold layer processing with 700+ TB of data.
- Dropshipping Framework for top worldwide markets (Amazon, eBay, Fnac, Rakuten)
Product, complex problems, modern technologies