Position: Middle/Senior Data Engineer
Category: Data Engineer
English level: upper
Years of experience: 3

- Designed, led and co-implemented a switch to new aggregation key in one of our projects. It required data models changes and affected more than 50% of one of our spark projects. The feature has been smoothly delivered in time.
 - Significantly improved an algorithm of files import. We had issues with 1TB+ files import, cluster scaling didn’t work. After the improvement it easily runs on 40 m5.4xlarge EC2 instances.
4 years of Java experience, 3 years of Spark experience. 2 years working with Python as a secondary language. Have a short experience and willing to learn Scala.

Databricks certifications.

Last position:
 - Implementing of data pipelines based on Airflow + EMR 
 - Taking part in project architecture/design
 - Taking part in short/long term project planning
 - Performance tuning
 - Writing unit and integration tests

Technologies:
Java, Spark, Airflow, Python, Spring Boot, AWS s3, AWS EMR, AWS Athena, PostgreSQL
Cloud based (AWS or Azure) project using Apache Spark with Java or Scala