Position: Big Data Developer
Category: Data Engineer
English level: upper
Years of experience: 5

Google Certified Cloud Developer (2019 - 2021)
Past projects:

• Enterprise Data Lake
Technologies: Java, Python, AWS (S3, Lambda, SQS, SNS, CloudFormation, MSK), Apache Spark
Databases: MS SQL
Responsibilities:
- Supporting existing ETL pipelines
- Developing reporting tool modules
- Developing data monitoring solution
- Implementing unit tests for application modules

• POC: Enterprise Data Lake / Data Warehouse
Technologies: Python, AWS (S3, Glue, Redshift)
Responsibilities:
- participation in the migration of the existing ETL pipeline to AWS

• Cloud Data Lake / Data Warehouse
Technologies: Python, AWS (Glue, S3, Lambda, API Gateway)
Databases: Amazon Redshift, MS SQL
Responsibilities:
- Developing ETL pipelines
- Implementing unit and integration tests for application modules
- API developing

• Data Lake Application
Technologies: Java 8, Python, AWS, Apache Airflow, Apache Spark, Spring (Core, Data), JUnit, Docker
Databases: Oracle DB, MS SQL, PostgreSQL
Responsibilities:
- Developing application modules;
- Automation of data quality checks
- Implementing unit and integration tests for application modules
- API developing and testing

• Hybrid recommendation system
Technologies: Java 8, Maven, Apache Spark (Core, SQL, ML)
Responsibilities:
- Developing application modules
- Implementing several machine learning recommendation algorithms
- Input data preprocessing and output data persisting algorithms
Preference is given to product companies

The "Big Data Engineer" role is not the same as the "Data Warehouse Engineer" role or "Database Developer" role.