Position: Data Engineer (Databricks)
Company: GlobalLogic
Experience required: 3y
Category: Data Engineer
English level: intermediate
Published: 2022-02-01T00:00:00+02:00

**Description:**
Our client is one of the leaders of the industry. We are looking to create new and innovative solutions to manage service interactions from across endpoints. Our team consists of highly motivated, innovative, customer-focused engineers and engineering leaders.

As a developer, you will be part of the team that will define what the future great customer service looks like around the world! 

**Requirements:**
3+ years of data engineering or ML pipelines and Apache Spark
Hands-on experience in Azure stack (Azure Data Lake, Azure Data Factory, Azure Databricks)
Good Understanding of Modern Data Warehouse/Lakehouse Architecture, Data warehousing concepts
Python
Proficient in AWS (S3, Glue Data Catalog, EMR, EC2, Cloudformation, IAM) 
Ability to work with SageMaker service

**Preferences:**
Azure data factory, SQL, AWS IAM

**Responsibilities:**
Take a major part in designing and implementing complex systems, data pipelines, and algorithms using a variety of technologies
Development using Databricks (Spark) platform and its tools
Assist with architecture and design for new workloads destined for Databricks and/or utilizing data for storage/events (Kafka, Event Hub)
Mentor teams on Spark, and complex DataFrames and SparkSQL (developers, testers, data analysis)
Monitor and troubleshoot Databricks workloads
Assist with DevOps pipelines
Help teams with data schemas, and data storage details
Ensure applications are tested through the usage of automated test tools
Collaborating with the Dev Leads and Product Manager to ensure that the requirements are met