Position: Data Engineer
Company: Blackbird Lab
Category: Data Science
Experience required: 2y
English level: upper

The role entails working with advanced technologies like Apache Spark, Apache Kafka and AWS ecosystem and provide data solutions for Retail and Ecommerce problems. You will be working on building an enterprise data warehouse that finds an optimal balance between the 4Vs (Volume, Variety, Velocity and Veracity) of the Big Data enabling analysts & business to make informed decisions

Most of the things you will work on:

- Implement real-time and batch processing data processing data frameworks
- Develop and manage scalable data processing platforms used by Data Science, Engineering and Analytics organizations
- Consult with Data Science and Analytics management to provide best practices for data modelling
- Implement data pipelines through collection, storage, transformation and normalization of large data sets.
- Planning and estimating features the roadmap to determine the next sprint goals
- Design and implement automated unit and integration test strategies

You'll need to have:

- 2+ years of working in a Data engineering department with experience writing ETL’s on big data platforms such as Hadoop, HDFS, Kafka
- 2+ years of designing and managing big data platforms such as Hadoop, Kafka or cloud based managed solutions.
- 2+ years of experience working with data processing languages such as pySpark, Hive, Scala Spark
- Track record of shipping quality code and delivering high quality features