Position: Middle/Senior Big Data Engineer (Java)
Company: Grid Dynamics
Experience required: 3y
Category: Java
English level: intermediate
Published: 2022-02-01T00:00:00+02:00

**Project description:**
We are starting a brand new project with one of the biggest automotive parts retailers in the world to work on a major re-platforming initiative to leverage our expertise in turnkey digital transformation initiatives. The idea is to fully migrate to GCP from an on-premise ATG environment and leverage the cloud capabilities of the #1 retail ecosystem in the world.

**Details on the tech stack:**
GCP, migration from on-prem ATG, BigQuery, Cloud Dataflow, Cloud Dataproc

**Min requirements to the candidate:**
1. Strong skills in SQL/Java and proven experience working with at least one of the major databases platforms required
2. 2-3 years of experience with the following GCP or equivalent cloud services: Cloud Storage, Google Kubernetes Engine, DataProc, Dataflow, CloudSQL, BigQuery

**Nice to have requirements for the candidate:**
1. GCP Data Engineer cloud certification or equivalent preferred
2. Python

**Team:** 4 people to start with as a part of a bigger team

**Tasks:**
1. Support development and deployment of the customer products and services across multiple cloud environments.
2. Gather, store and maintain big data sets for all digital marketing channels.
3. Collaborate with a cross-functional team of client leads, application developers, operations engineers, and architects to translate complex product requirements into technical specs and design requirements.
4. Maintain a high degree of knowledge in cloud data architecture and ETL best practices especially across Google Cloud Platform products and services.
5. Develop and deploy automated scripts in BigQuery and other Google cloud services to be used by other teams to increase productivity.
6. Facilitate architectural discussions and initiatives to ensure cloud-based products are optimally deployed with maximum availability of design features.
7. Formulate and execute robust UAT protocols to identify and address latent errors in product functions.
8. Design, build and deploy ETL and data management processes with reliable error/exception handling and rollback framework.
9. Provide production support for data load jobs and develop a customized query to generate automatic periodic reports.
10. Build applications writing SQL/Python scripts to manipulate data and/or writing specific instructions for off-shore programmers to write the scripts.

**Pros of the project:**
1. the brand new project, cloud migration
2. bleeding-edge technologies
3. an engaged, professional, and diverse team