Position: Middle Data Engineer
Category: Data Engineer
English level: upper
Years of experience: 2


Experienced IT professional with 4+ years in the IT industry(3+ years as a Business/System Analyst and 1.5 years as a Data Engineer)
Currently, I've worked as a Middle Big Data Software Engineer with practical experience in Databricks, AWS, Azure, Apache Spark, and Airflow.

The outsourcing company, Feb 2022 - Present
The last project: 
Fintech domain, Implementation of Databricks for the customer
Responsibilities:
- created and tested Delta Live Tables(DLT) pipelines and jobs
- performed bugfixes and improvements
- developed notebook for automatic DLT pipelines and jobs creation through Databricks API for MySQL, MariaDB, and MongoDB sources
- developed notebooks to ingest big tables(3 TB) from MariaDB source
- developed Airflow DAGs to call Databricks jobs through Airflow
- prepared configuration files for DLT pipelines
- performed data validation of Delta tables
- communicated with internal client teams to resolve issues