Position: Data Engineer
Category: Data Engineer
English level: upper
Years of experience: 6

- PhD in Life Science;
- Certifications:
        AWS: Data Analytics Speciality, Developer Associate;
        Coursera: Deep Learning Specialization;
AWS Certified Data Engineer with 6+ years of experience in Data Engineering. Strong knowledge of data architecture, including data ingestion pipeline design, data modelling, machine learning and advanced data processing.

Technologies: Python, SQL, PySpark, AWS services (Redshift, Glue, Lambda, S3, etc), Azure (Data Factory, Synapse), Databricks, Kafka, Pandas, Sk-Learn, PyTorch, etc.

Certifications:
        AWS: Data Analytics Speciality, Developer Associate
        Coursera: Deep Learning Specialization 

Recent projects:
Project 1:
        Data from different sources were collected in Data Lake with real-time and batch data ingestion jobs, with following transformation and loading into Data Warehouse.
    • Optimized 3 modules in company's platform library that decreased data processing time for the R&D teams;
    • Decreased data processing time and reduced expenditures (from 10% to 120%) by splitting logic between Batch and Streaming mode;
    • Designed and automated QA dashboard that allowed to catch data issues on the earliest steps.

Project 2:
        Data Lake migration: The goal of the project is to build a Data Lake that allows to monitor and control product logistic and distributions. Our team developed and installed a Monitoring Solution to automatically detect potential issues with the product.
    • Designed and developed new advertise module that clusterizes incoming events into similar group and integrates with other services;
    • Deployed and supported architecture to detect product frauds that is now being actively used in more than 15 countries; 
    • Increased data reprocessing and fault tolerance by adding silver (intermediate) Data Lake in pipelines;
    • Provided leadership in a sub-team (4 engineers) for 6+ months;
    • Optimized ML models with accuracy increase up to 12% by implementing assembled models.