Position: Data Engineer (Middle+/Senior)
Company: IT-Rex - IT Staff Recruitment
Category: Data Engineer
Experience required: 3y
English level: upper
Published: 2022-06-01T00:00:00+03:00

Requirements:
Python, Numpy, Pandas, Scipy (for bugs);
Deep understanding of SQL;
DB: PostgreSQL, MySQL; Vertica, ClickHouse, and also noSQL: MongoDB, Redis;
ORM and data migration systems: SQLAlchemy, Flyway, Liquibase;
ETL: Spark, Airflow / Luigi (additional work with Hive / Impala will be a plus);
Chergi: Kafka, RabbitMQ and ZMQ or their analogues;
Visualization: plot, dash or analogues; matplotlib
CI / CD: Gitlab CI / CD, Docker, K8s;
A great plus will be the development of blockchains (Bitcoin, Ethereum only), the completion of programming in C # and the work with the BlockSci library for parsing data.

Candidate Responsibilities
Design and development of blockchain data processing pipelines (data processing pipelines) for new charts on the company's platform, including the development and optimization of data architecture (data architecture);
Integration of developments into existing solutions;
Development and support of the infrastructure necessary for optimal extraction, transformation and loading of data from various sources (ETL/ELT);
Debugging, testing and fixing errors in your own and other people's code;
Interaction with team members, including reviews of technical specifications and code;
Participation in data processing solutions grooming, ETL/ELT processes and data processing workflow automation, including coordination with back-end developers, DS, DA and QA engineers;
Data validation: calculation of historical data using BlockSci, automation of data validation.

Team:
2 products, 2 back, 1 front, 1 qa + hr, ceos, marketing

Number of stages of the interview:
1. ceo
2. tech
3. with team (optional)