Position: Big Data developer
Company: Data Science UA
Category: Scala
Experience required: 3y
English level: intermediate
Published: 2020-06-01T00:00:00+03:00

**Skills:**
● have 4-5 years experience in software development
● have a strong desire to improve your skills with modern development and production environment — Unix/Linux, bash, python, virtualisation (docker, Kubernetes),
● have deployment orchestration (chef, puppet, ansible), configuration servers ZooKeeper, monitoring and distributed logging (Prometheus, Grafana, GrayLog) experience
● have strong knowledge of Scala /Java
● have knowledge of distributed data processing frameworks Hadoop, Spark, Kafka.
● have proven experience of building production-ready ETL processing pipelines
● have familiarity with Big Data persistent distributed storages Impala, Elasticsearch, Aerospike, HBase
● have proven working record with relational databases (e.g. PostgreSQL (stored functions, triggers))
● have good written and verbal English communication skills


**Responsibilities:**
Big Data team is enabling the optimal storage, organisation and timely retrieval of our data.

Your responsibilities will include:
● Carrying out efficient integration with our data providers via various API endpoints and data representation formats.
● Building and deploying an in-house distributed ETL pipeline for processing petabytes of data per day
● Enable an accurate, comprehensive and reliable data storage in our distributed data warehouses based on the needs of other teams
● Providing continuous improvements in the way data is being processed and stored based on the feedback and needs of the business or other teams
● Setting up monitoring for key performance metrics and overall systems’ behaviour to promptly react in case any anomaly detected
● You will be responsible for optimisation of ETL pipelines, maintaining over 60 Spark jobs. Building a data lake for data scientists and analysts.
● Experimenting with new tools and technologies to produce cutting-edge solutions to business problems
● Be a part of a self-organising, results-oriented agile team using Kanban to complete new product launches

**As a plus:**
● Experience in AWS S3 environment (S3 storage)
● Some experience in the AdTech sector is advantageous

**We offer:**
• Location — Kyiv
• Good compensation;
• Cozy office (on og the best coworking in Kyiv);
• Benefit package