Position: Data Engineer
Category: Data Engineer
English level: intermediate
Years of experience: 1.5


- Project #1
Translating and implementing commercial requirements into queries to database, collection and clear the necessary data.
Implementing the logic of transforming data and forming the final result that are published to a visualization tool.
Optimizing the existing queries.
Writing the logic widgets on Javascript for visualization a received information.
tech: PostgreSQL, Google Cloud SQL, Amazon RDS, Javascript

- Project #2
Migrating the databases and managing the database.
Designing, building, maintaining data pipelines to migrate and transform data.
Tracking pipelines stability and ETL testing.
Creating indexes for faster retrieval of the information and enhance the database performance.
Building database schemas, tables, procedures and functions.
tech: MS SQL, Oracle, Azure Data Factory, Azure Storage/Data Lake, Azure SQL, Azure Function App (Python), Bicep (ARM templates)

- Project #3 (domain: pharmacy management system)
Building Data Warehouse for gather data from across the organization.
Creating ETL/ELT code to populate the Data Warehouse.
Creating Spark jobs for data transformation and aggregation.
Creating Airflow DAG to drive ETL/ETL code.
Creating merge and purge functions.
Working on teams operating under an Agile Scrum delivery methodology.
tech: PySpark, Spark Streaming, Kafka, Airflow, PostgreSQL, ADLS, Docker, K8s, Grafana, Liquibase, Jaspersoft
Data Engineering tasks and tech. Focusing on the cloud products