Position: Senior Data Engineer
Company: EPAM Systems
Experience required: 3y
Category: Data Engineer
English level: upper
Published: 2023-03-01T00:00:00+02:00

PLEASE NOTE THE VACANCY IS OPEN in GEORGIA and POLAND.

Migration of Data Products pipelines from Oracle workloads to Databricks.

Develop, monitor, and operate the most used and most critical curated data pipeline.
This pipeline is processing hundreds of millions of records to provide high-quality datasets for analytical and machine learning use-cases. 
Consulting with analysts, data scientists, and product managers.
Redevelop old legacy pipelines to new, advanced and standard versions that are easy to maintain and scalable for future demands.
Leverage and improve a cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala.

Requirements:
- Expertise in Apache Spark along with Spark streaming;
- Good hands on experience with Databricks and delta-lake;
- Fluency in Scala programming language;
- Expertise in SQL;
- Good understanding & hands-on experience with CI/CD;
- Rich working experience with Github; 
- Fluency working with AWS landscape;
- Ability to build Apache Airflow pipelines.

Nice to have:
- Presto;
- Superset;
- Starburst;
- Oracle & Exasol.

We offer:
- Medical insurance;
- Stable income;
- Flexible roles;
- For your comfortable work;
- 100% remote work;
- EPAM hardware;
- EPAM software licenses
- Access to offices and co-workings;
- Stable workload;
- Flexible engagement models;
- For your growth;
- Free trainings for technical and soft skills;
- Free access to LinkedIn Learning platform;
- Language courses;
- Free access to internal and external e-Libraries;
- Certification opportunities;
- Skill advisory service.