Position: BigData Developer (Python, PySpark, Kafka, GCP)
Company: Grid Dynamics
Category: Python
Experience required: 3y
English level: intermediate
Published: 2021-05-01T00:00:00+03:00

The project is about re-platforming from AWS to GCP. The project goal is:

-to build an initial version of infrastructure using GCP that will host specific services and applications to be migrated from AWS during the next project phases;
-complete the initial research, identify the first applications to be migrated on later program phases, and build a migration plan for selected applications or services.

Project scope includes establishing of services for ingestion of data into GCS storage layer, data transformation using Spark, implementation of Cloud Composer (managed by Airflow), Google Data Catalog.

**Required skills:**
2+ years of solid experience working with Python
Good knowledge PySpark
Extensive knowledge of Kafka and Spark Streaming
Working experience with GCP
Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation
English level: Upper-Intermediate and higher

**Nice to have:**
Working experience with Apache Hadoop, Apache Airflow
Good knowledge of Dataproc, GCS, BigQuery
Experience with DataBricks

**We offer:**
Opportunity to work on bleeding-edge projects
Work with a highly motivated and dedicated team
Competitive salary
Flexible schedule
Medical insurance
Benefits program
Corporate social events