Position: Big Data Engineer (Automotive)
Company: Sigma Software
Category: Data Engineer
Experience required: 3y
English level: intermediate
Published: 2022-04-01T00:00:00+03:00

**Customer**

Our customer is a company with solid experience in setting standards in the automotive aftermarket worldwide as a digital innovator and a provider of leading expert solutions.

The client is distinguished by in-depth knowledge of the industry and requirement for the highest levels of quality and security based on sustainable technologies.

**Project**

You will work with the Data Platform, which includes Data Lakehouse solution designed to become a unified storage and Data Warehouse for structured and unstructured data generated by the various solutions. The solution includes processing pipelines aimed at collecting data and providing it to the end customers via the unified interfaces.

As a Big Data Engineer, you will design and develop a robust and scalable Data Platform. This includes data processing, data versioning and distribution, orchestration of data, merge and conflict handling processes, as well as providing Data Dashboards and analytical reports for end-users.

You are going to be a part of the distributed team that uses agile methodologies best practices, Continuous Integration, develops ETL pipelines, takes care of performance testing, capacity planning.

**Requirements:**

3+ years of professional software development experience with Python
2+ years of experience with relational and/or NoSQL databases
1+ years of experience with processing framework(s), such as Apache Spark
1+ years of experience with at least one of mainstream cloud providers (Amazon, GCP, Azure)
Knowledge of serverless concept capabilities and features
At least Intermediate level of spoken and written English
 
Nice to have:

Experience with Apache Hudi, Serverless Framework, AWS services (Glue, Lambda, Step Functions, S3, SNS/SQS, CloudFormation)
Experience with designing databases and/or Data Warehouses

**Responsibilities:**

Building up an infrastructure for data platform with the level of customization and scalability that allows applying quick changes
Designing and developing data pipelines
Creating and customizing ETL scripts with AWS technologies and frameworks
Performance testing and optimization of the solutions
Living by agile principles and collaborating with team members using agile techniques

Are you in?