Position: Data Engineer
Company: Abto Software
Category: Data Engineer
Experience required: 5y
English level: upper
Published: 2022-10-01T00:00:00+03:00

We are looking for a Data Engineer to join our team.

About project: 

The products our client are the global standard for process intelligence and automation at more than 10,000 organizations across 90 countries. You’ll help customers accelerate their digital transformation, bringing them a quicker, easier, better way to work.

About the Role:

The Data Engineer work on the next generation of our analytics and AI product, where they will spend most of their time working on ETL’s, Data Ingestion, Apache Spark, Scala and mentoring other members of the team to design and implement our Analytics and AI/ML powered features.

Requirements:
- Hands-on expertise with Apache Spark (Spark structured streaming), Scala and Spark SQL
- Experience in data munging, manipulation, cleansing and blending from multiple data sources and exposure to various data types and storage paradigms
- Recent experience as a Data Engineer
- Experience with Azure technologies
- MS Databricks experience it highly advantageous
- Python experience will be a bonus
- Familiarity with the Delta Lake House paradigm is preferable
- Familiarity with streaming data sources is preferable

Your contribution will be: 

- Design, code, and unit test data engineering components and features that can be released / deployed through automation
- Ensure data completeness and correctness through cross-referencing to original data source
- Design and develop ETL’s in line with best practices and techniques for data transformation and storage
- Deliver data engineering functions that ingest, sanitize, and join data from multiple sources
- Create data models that meets the needs of management stakeholders, data scientists, and analysts as required
- Analyze and evaluate data product/output to identify and recommend optimizations as required in its extraction, cleansing, and organization
- Work with data providers to document the incoming data stream’s domain boundaries, field datatypes, and term definitions in sufficient detail to enable transformation of the data for end-user consumption
- Design and implement data engineering solutions that adhere to guidelines in support of security, disaster recovery, scalability, availability, reliability, and durability.

Our benefits:
- Professional and career growth promotion;
- Competitive salary;
- Paid vacations and sick leaves;
- Medical Insurance (partial compensation);
- Flexible working hours;
- Regular corporate social activities;
- Regular technical training at our office;
- English courses;
- Gym, etc.