Position: Big Data Engineer
Company: Halian International
Category: Data Engineer
Experience required: 5y
English level: fluent
Published: 2022-12-01T00:00:00+02:00

Big data engineer for top client in the UAE

Our Client

A leading investment banking company is looking for an experienced Data Engineer to join their fast paced team.





Your Responsibilities

Work closely with the data science and business intelligence teams to develop data models and pipelines for research, reporting, and machine learning
Assemble large, complex data sets that meet functional / non-functional business requirements
Establish and implement data governance processes and methodologies that will adhere to our customers compliance needs
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, streaming and ‘Big data’ technologies 


Your Qualifications

Masters Degree in relevant technical field 
Minimum of 6 years of experience 
Strong foundations of data management systems including SQL (Postgres, MySQL, etc), NoSQL (Elastic, Mongo, etc) and object storage (S3)
Proven experience working with and maintaining Data lakes with vast amounts of unstructured data
Knowledge of programming languages (e.g. Java, Scala and/or Python)
Strong analytic skills related to working with unstructured datasets
Working experience with stream-processing systems: Spark, Flink, Kafka-streams, etc.
Demonstrable working experience with Kubernetes and the deployment of complex systems