Position: Middle/Senior Big Data Engineer
Company: Grid Dynamics
Experience required: 3y
Category: Python
English level: intermediate
Published: 2022-07-01T00:00:00+03:00

A global technology and data company that builds verification, optimization, and analytics solutions for the advertising industry and we are looking for a Big Data Engineer to join our Brand Safety Engineering team. If you are interested in building machine learning-based data pipelines; driving multiple billions of decisions per day, then this is the place for you!

**Client description:**
A global technology company that offers data and solutions to establish a safer, more effective advertising ecosystem. 

**Details on the tech stack:**
Python, Java, Spark, Airflow, AWS, Machine Learning, Snowflake
ECS, Docker, SQS, ElasticSearch, CloudFormation

**Min requirements:**
- 2+ years of experience designing and building data-intensive applications
- 3+ years of recent hands-on in an object-oriented language (Java, Scala, Python)
- 2+ years of recent experience developing data pipelines in AWS or Google Cloud
- Expertise in using Big Data frameworks (e.g., Hadoop, Spark) and MPP databases (e.g., Redshift, Vertica, Snowflake) for complex data assembly and transformation
- Superb understanding of algorithms and data structures
- Experience working with Apache Airflow
- Bachelors or Masters in Computer Engineering, Computer Science, Electronics Engineering, or related fields
- Excellent interpersonal and communication skills
- Understanding of full software development life cycle, agile development, and continuous integration

**Nice to have requirements:**
- In-depth understanding of algorithms, scalability, and various tradeoffs in a Big Data setting
- Knowledge of database technology, SQL, schema design, and query optimization techniques
- Experience in designing for performance, scalability, and reliability
- Digital advertising or web technology experience, especially video advertising
- Experience working on data science-driven projects and/or implementing machine learning algorithms

**What tasks you will work with?**
- Design and implementation of ETL pipelines (processing on Spark, orchestration in Airflow)
- Collaboration with DS team (moving models to production, wiring of all the components together,  performance optimizations)