Position: Data Engineer
Company: deventor.io
Category: Data Science
Experience required: 3y
English level: upper
Published: 2021-08-01T00:00:00+03:00

**What you will be doing:**

● Collaborate with business, engineers, and product managers to understand data needs.
● Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using cloud-native data engineering principles
● Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, as well as cloud-based), distributed/elastic environments, and downstream applications and/or self-service solutions
● Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
● Implement the appropriate design patterns while optimizing performance, cost, security, and scale and end user experience.
● Participate in development sprints, demos, and retrospectives, as well as release and deployment.

**What we are looking for:**

● Knowledge and expertise with design, implementation and maintenance of data ingestion pipelines using such AWS technologies as EC2/S3/RDS/Lambda, Glue/GlueCrawler/Athena, JSON/Avro.
● Experience with stream processing implementation based on Apache Kafka, AWS Kinesis and Kubernetes, and Python queuing experience.
● Knowledge and expertise with data modelling skills using RabbitMQ.
● Proven ability to build and support large-scale data solutions and high availability stacks.
● Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations.
● Solid written and verbal communication skills.
● Ability to work in a team environment that promotes collaboration.