Position: Senior Big Data Engineer
Company: Mobilunity
Experience required: 5y
Category: Java
English level: upper
Published: 2021-02-01T00:00:00+02:00

The company represented itself as a globally oriented customer engagement platform that leverages customer data to enable high value-added communication channels such as email, push notifications, and in-app pop-ups. They belong to mark tech and have big traffic (10M - 16M monthly active users). They are the most popular such platform on the Japanese market.

The project in brief requires to have the ability to design and implement a system that requires quality as B2B SaaS while handling high traffic load and Big data.

Most of the system runs on AWS in a VPC. Some of the application components use BigQuery as a data store for the raw data, as well as GCS to forward intermediate Avro files to S3 for later use, everything related to Kafka/Kafka Streams App runs purely on AWS.

The client structures the development process organization following the LeSS (large scale scrum) framework. Rather than dividing teams by technology stack/components such as backend, frontend, and native application development, members with each technology stack gather as a functional development team to grasp business issues and refine specifications. 

CTO mostly works solo as a system architect replacing existing solutions or implementing core parts of the overall system. Some of these tasks may eventually be delegated to the SRE team or possibly one of the feature teams. Having said that, it is crucial for the members of this project to align the architectural direction and other critical aspects of the system with the CTO.

So, to sum up, they are in need of Big Data Engineer who'll cover: 
・Web application development using any language
・API development such as marketing campaign API and external cooperation API
・Development of stream processing application using Kafka Stream
・Checking the source code of the framework and library used for development and solving the problems. Returning resolved issues to the OSS community
・Active communication to receive feedback from internal and external stakeholders
・Review and try to continuously improve the development process and products
・Be ready to learn Go, JVM, Mobile and Web SDK development if needed, expand the range of skills and contribute to improving team productivity

Required skills (MUST):
・ Experience in developing web applications using Java
・ Experience in working with Apache Kafka/ Big query/ Big Data
・ Development experience using SQL and related databases
・ Development experience using NoSQL or key-value datastore
・ Experience in developing asynchronous systems using Queueing Service

Good to have:
・ Experience in developing or operating marketing-related services such as ad tech and MA
・ Development experience using the public cloud (AWS or GCP). Especially development experience using CDN
・Experience related to microservice architecture and authentication/authorization
・Experience in deepening understanding of the target business area and context based on business requirements and use cases, incorporating it into system design while appropriately performing business modelling
・ Experience involved in product specification formulation through direct communication with stakeholders, such as hearing requirements and consensus-building on development scope.
・ Experience in improving development processes and products through Scrum or similar development systems
・ Experience of using RoR will be a great advantage OR at least your desire to work with it.