Position: Middle Big Data Engineer
Company: Grid Dynamics
Experience required: 3y
English level: intermediate

Client description
Integral Ad Science (IAS) is a global technology company that offers data and solutions to establish a safer, more effective advertising ecosystem.
They partner with advertisers and publishers to protect their investments, capture consumer attention, and drive business impact.
Founded in 2009, IAS is headquartered in New York with global operations in 13 countries.
Their growth and innovation have been recognized in the Inc. 5000, Crain’s Fast 50, Forbes America’s Most Promising Companies, and I-COM’s Smart Data Marketing Technology Company.
Learn more at www.integralads.com.

Details on the tech stack
Python, Spark, Airflow, AWS, Machine Learning, Snowflake
ECS, Docker, SQS, ElasticSearch, CloudFormation

Min requirements
- 2+ years of experience designing and building data-intensive applications
- 3+ years of recent hands-on in an object-oriented language (Java, Scala, Python)
- 2+ years of recent experience developing data pipelines in AWS Cloud
- Expertise in using Big Data frameworks (e.g., Hadoop, Spark) and MPP databases (e.g., RedShift, Vertica, Snowflake) for complex data assembly and transformation
- Understanding of algorithms and data structures
- Experience working with Apache Airflow
- Good Interpersonal and communication skills
- Understanding of full software development life cycle, agile development, and continuous integration

Tasks:
- Design and implementation of ETL pipelines (processing on the Spark, orchestration in Airflow)
- Collaboration with the DS team (moving models to production, wiring of all the components together, performance optimizations)