Position: Data Engineer, Investment Management Solution
Company: DataArt
Experience required: 2y
Category: Data Engineer
English level: intermediate
Published: 2022-09-01T00:00:00+03:00

Responsibilities
Manage existing and create new Spark jobs (Scala)
Optimazing and performance tuning of Spark jobs
Creating and update NiFi pipelines, manage huge NiFi environment
Manage Hive/Impala tables in CDH infrastructure
Support during creation and implementation of the architecture
Data analysis
Participation in Scrum ceremonies
Interaction with BA

Requirements
Technical expertise with Big Data tools and methodologies
Hands-on experience with Spark 2.0 or greater (+ Scala)
Programming experience with Scala
Experience working with Hadoop ecosystem (CDP/Cloudera Data Platform)
Proficiency in SQL
Spoken English

Nice to have
Experience working with Hive And Impala
Experience working with Apache NiFi
Experience working with Kafka
Experience working in an agile environment and understanding of various phases of the software development life cycle
Experience with Data modelling
Skills working with Airflow (preferred) or Oozie