Position: Big Data Developer
Company: Grid Dynamics
Category: Data Engineer
Experience required: 3y
English level: upper
Published: 2022-09-01T00:00:00+03:00

Project is in the process of a data lake implementation in AWS after spending 18 months using GCP Big Query as the main data platform.

Details on tech stack
- AWS
- Lambdas
- Redshift
- Python
- Spark
- Postgres
- StepFunctions
- EMR clusters
- Terraform

Min requirements to the candidate
- Hands-on experience with AWS cloud platform required
- Experience using terraform lake formation
- Can describe experiences working with AWS Console
- Demonstrated examples of building Lambdas for 3rd party data sources, enabling CDC and ingesting RDS/postgres data sources, implementing Glue Crawler, creating JSON parsers, etc.

Nice to have requirements to the candidate
- Experience with e-commerce data a plus

Do you have client interview?
- Yes, 2 round of technical interviews

Current team size, their locations and whether team is going to grow?
- 3 engineers on client side + 2 that should join from GD

What tasks the candidate will work with?
- The current phase of work is focused on ingesting data sources ranging from source data RDS/postgres to 3rd party sources (e.g. Google Analytics, Stripe, Shopify, Iterable, Mailchimp, etc), and any post-ingestion data transformation and validation needed for the source to be analytics ready.
- We are looking for data engineers who are able to help us complete this work, from ingesting 3rd party sources with AWS lambda/kinesis etc to data transformations and push to redshift.