Position: Data Engineer
Category: Data Engineer
English level: fluent
Years of experience: 3


- Gathered the business requirements from the stakeholders to design the data-sourcing project outcome.

- Integrated data sources below by creating a Databricks Python operator to run on the Databricks cluster, processed the data using Apache Spark, wrote the data into AWS S3 buckets, and automated the tasks using Airflow DAGs.

- Created an operator to push the Braze events into Mixpanel, and saved 30K USD / year by replacing the paid connector provided by Braze with my operator.

- Sourced the Leads, Accounts, and Opportunities data from Salesforce using the Bulk API.

- Sourced the data from Tickets, Users, Organizations, Groups, and Ticket Management endpoints using the Zendesk API via the cursor-based pagination method.

- Sourced the Campaign and Traffic Source data from Hubspot using the Analytics API.

- Sourced the App Performance data of our Apps and Competitor's Apps from the AppAnnie using the data.ai App Performance V2.0 API.

- Sourced the Amenity, Shop, Tourism, Leisure, and Landuse GIS data for 27 cities from OpenStreetMap (OSM) using the Overpass API.

- Sourced the required financial data using the Microsoft Business Central API.