Position: Big Data Engineer
Company: DataArt
Category: Data Engineer
Experience required: 3y
English level: upper
Published: 2022-07-01T00:00:00+03:00

**Client:**

Our client is a US-based investment management business of an American Fortune Global 500 and Fortune 500 company whose subsidiaries provide insurance, investment management, and other financial products and services to both retail and institutional customers throughout the United States and in over 40 other countries.

**Specifyings of the project:**

The client has recently started the development of a system that appeared to be in need of enhancements before its final implementation. The system is a warehouse consolidating data flows, and processing information so that it can be presented to the end user in a readable format. Along with the optimization, the project is aimed at migration from on-premise software to cloud services (AWS). We are the only vendor working on this task, and we need to expand the team.

**Specifyings of the positions:**

 As the project is new, it provides a room for selecting the best technologies and stacks. Still, the client has certain set processes which include the following technologies: Apache Spark, AWS (S3, EMR, Redshift, Kinesis), ElasticSearch, Git (BitBucket), Scala programming language. We closely work with the Architecture team on the client's side. The head of the Architecture team is our key stakeholder. The team is dedicated to searching and making decisions on best practices to be implemented in development, and you can contribute to the start of large-scale development and proceed with the implementation of suggested approaches.

**Technology stack:**

Apache Spark, AWS (S3, EMR, Redshift, Kinesis), ElasticSearch, Git (BitBucket), Scala programming language


**Requirements:**

 — Understanding of software development company functioning
 — Knowledge of standard development tools specified by project needs
 — Knowledge of procedural and Object-Oriented Programming
 — Understanding of basic RDBMS concepts
 — Hands-on experience working with Apache Spark and data transformations
 — Good English 

**Will be a plus:**

 — Experience with Scala
 — Experience with AWS (S3, EMR)
 — Experience with Databricks
 — A desire to learn fast