Position: Data Engineer
Category: Data Engineer
English level: upper
Years of experience: 6


Last job description
My job was to develop and maintain the whole data engineering part of the analytics department.
The data lake model was used where data came in to Redshift dwh or S3 (+Spectrum) from backend sources like MySQL and Mongodb replicas and RabbitMQ. It then got transformed with Lambda jobs and Airflow tasks. The final set of data was used in Tableau reports, in alerting, web API Dreamfactory.
The infrastructure heavily relied on AWS.

Technologies used:
- Python, some JS, some Java, some Scala
- Airflow for scheduling tasks. Written many custom plugin Operators
- Redshift + Spectrum. Querying and maintenance
- RabbitMQ, binlog MySQL and oplog Mongo streaming. NiFi
- Timescale + Grafana. Influx for small legacy use case
- AWS. EC2, Kinesis, Lambda, S3, Glue, LEX, Cloudformation
I want to get a challenging Data Engineering position solving business needs with cutting-edge on-premises and cloud technologies.