Position: Data engineer intern
Company: Luxoft
Experience required: no_exp
Category: Data Engineer
English level: upper
Published: 2023-06-01T00:00:00+03:00

As a part of data migration into Azure Cloud - extract data from Oracle Hive cluster, convert into Parquet format and load to the Azure Blob Storage.
Utilize Spark for further data manipulations and analysis in a distributed manner.

**Responsibilities**
• Take part in designing data and analytics architectures in Microsoft Azure cloud;
Use Big Data technologies like Spark, Hive, Kafka etc.;
•data warehousing, data ingestion, and data profiling.
Preferable knowledge of programming languages such as Python, Scala,
•teamwork, strong communication skills, and collaborative in complex engineering projects.

**Mandatory Skills**

Azure Services:
-DataBricks
-Azure DevOps ( simple GIT and CI/CD)
-Azure Key Vault
-Azure Blob Storage Kafka Streams

Python
PySpark - Mainly understanding the PySpark specifics, difference between actions and transformations, reading data from external sources like Parquet files, SQL databases etc
Kafka Streams

**Nice to have skills**
Big Data
SQL: Tables in 3NF , Joins and CTE ( common table expression)