Position: Middle Java Engineer (Big Data)
Company: SoftConstruct
Category: Java
Experience required: 2y
Published: 2020-04-01T00:00:00+03:00

Lambda architecture, designed to handle massive quantities of data by taking advantage of both batch and stream-processing methods. This approach to architecture attempts to balance latency, throughput, and fault-tolerance by using batch processing to provide comprehensive and accurate views of batch data, while simultaneously using real-time stream processing to provide views of online data.

In many organizations, data consumption processes for timely reporting or crucial analytical requirements are hindered by both delays in query execution and information presentation. The main challenge is to minimize decision delay with the help of flexible solutions with open and simplified architecture and high performance based on future-oriented technologies using AI models and forecasting analytics.

In numbers:
Up to 1 PB of data from 500+ partners in long-term storage
30+ M new records daily
Latency for insight based on data delivery time <5 sec

Requirements:
Knowledge of Java and Java SDK;
Knowledge of data structures and algorithms;
Knowledge of OOP/OOD;
Knowledge of software design concepts, principles, and patterns
Good understanding of Kafka Stack (Kafka Streams, Kafka Connect, Kafka Schema Registry)
Experience in working with relational and non-relational databases
Mathematical thinking, analytical and problem-solving skills, self-motivated, organized team player

Nice to have:
Some experience with IoC Tools (Terraform, Ansible)
Understanding Big Data technologies (MapReduce, Hadoop, Spark, Presto, Drill...)
Knowledge of data warehouse architecture, infrastructure components, ETL

Responsibilities:
Development, implementation and automation of distributed system deployment for collecting and processing events from several sources.
Data schema development and management of internal data warehouses and database systems (SQL/NoSQL)
Writing ETL-jobs to calculate business metrics
Reviewing project documentation, analyzing requirements;
Participating in code reviews, teamwork, helping colleagues.

We offer:
Interesting and challenging work in a product and data-driven company;
Plenty of opportunities to learn, grow and progress in your career;
We keep work-life balance;
Health insurance;
Free English classes;
Paid vacation 24 days and paid sick leave;
Silent office near Lukyanivska metro station;
Participate in R&D projects.

We in media: 
https://bit.ly/2C8K9ZZ
https://bit.ly/2OpjJZp
https://bit.ly/2qUdAMo
https://bit.ly/2DLMGtu