Position: Scala/Big Data Engineer (#00005)
Company: Avenga
Experience required: 2y
Category: Scala
English level: intermediate
Published: 2022-01-01T00:00:00+02:00

We expect the successful candidate to demonstrate high motivation and ability to solve complex problems and achieve set goals.

**Skills and Qualifications**
At least 2-3 years of a relevant experience as Scala/Big Data Software Engineer;
Good knowledge and hands-on Scala Programming;
Proficiency in Spark Scala for technical development and implementation;
Experience with AWS components and services, particularly EMR, S3, and Lambda;
Data pipeline using Spark Scala;
Load disparate data sets by leveraging Kafka consumers;
Good understanding of Big Data technologies like Hadoop, Spark, Scala, Hive, HBase, Pig, Cascading;
Experience with application architecture in a big data environment;
Excellent in writing shell scripts;
Knowledge on Cassandra Architecture and CQL is desirable;
Ability to utilize Hive, Spark, Cassandra, Mesos and Kafka;
Good knowledge on Hive and HiveQL;
Experience with messaging and complex event-processing systems such as Kafka and Storm;
Good Hands on in Hadoop stack (MapReduce, Hive, Sqoop, Oozie);
Excellent in writing shell scripts;
Good understanding of file formats including JSON, Parquet, Avro, and others;
Experience with open source technologies such as Cassandra;
Extensive hands on Data frame and Dataset operations of Spark;
Build reusable code, with the ability to scale with very large data volumes;
Data Migration/ETL knowledge is desirable;
Experience with AWS technologies or equivalent cloud stack As a hands-on engineer, influence all architecture decisions;
Knowledge on Java Programming is desirable;
English – Upper-Intermediate.

**Responsibilities**
Programs, tests, and documents systems in accordance with programming standards and validation procedures;
For more senior candidate – to serve as technical lead on a project;
Plans and coordinates the complex design, development, implementation, maintenance, and level 4 support of associated systems;
Provides software, system and/or programming solutions across projects and/or technologies to meet internal and external needs;
Engineer systems and/or software, and/or transfer data for internal and external clients;
Plans and coordinates peer review of work products such as code, designs, and test plans produced by other team members;
Interact with corporate and project team members to negotiate timelines, responsibilities, and deliverables;
Provides expert technical advice and solutions for internal and external clients;
Provides expert review of database development work with regard to programming standards and validation procedures;
Provides long term evaluations of systems, company wide;
Meets personal and professional milestones as assigned, and interacts with the project team to organize timelines, responsibilities and deliverables;
Performs other duties as directed.