Position: Data Engineer
Company: INCOAlliance
Experience required: 1y
English level: upper

Requirements: 
1+ years experience in working with Apache Spark cluster;
1+ years experience and knowledge of Spark scripting languages: PySpark/Scala/Java;
2+ years  experience with SQL;
1+ years of experience with data transformation, validations, cleansing, and ML feature engineering in a Big Data Engineer role;
Experience working with and optimizing ‘big data’ data pipelines, architectures, and data sets;
Strong analytic skills related to working with structured and semi-structured datasets;
Build processes supporting data transformation, data structures, metadata, dependency, and workload management;
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement;
Business oriented and able to work with external customers and cross-functional teams;
Upper Intermediate English both written and spoken.

Nice to have:
Experience in building Machine Learning pipeline
Experience with Elasticsearch
Experience with Zeppelin/Jupyter
Experience with Linux
Experience with workflow automation platforms such as Jenkins or Apache Airflow
Experience with Microservices architecture components, including Docker and Kubernetes.

Responsibilities:
Implement and maintain data pipeline flows in production within the system based on the data scientist’s design;
Design and implement solution-based data flows for specific use cases, enabling applicability of implementations within the product
Building a Machine Learning data pipeline;
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader;
Work with product, R&D, data, and analytics experts to strive for greater functionality in systems;
Train customer data scientists and engineers to maintain and amend data pipelines within the product;
Build and manage technical relationships with customers and partners.

We offer:
Сompetitive compensation based on your skills and experience.
24 days of annual paid vacation and 5 sick leaves.
Accounting as a service.
Long-term employment.

Recruitment process:
A screening call with with HR (20mins).
Interview with the team manager (30 mins).
Test task - 3- 4 hours.
A interview with VP Customer Data (30 mins).
Final feedback received within a week after the interview.