Position: Lead Data Engineer
Company: Crunch
Experience required: 5y
Category: Data Engineer
English level: upper
Published: 2022-04-01T00:00:00+03:00

**Description**
Our customer is US-based company. The platform will allow advertisers to run individually targeted campaigns, hosts can upload their own content and schedule time, when they want to show their own non-advertising content or run the advertisements.

As a Lead Data Engineer, you would bring your database design and data systems expertise to build out company's data capabilities. More than just understanding the strategic importance of data in an advertising, sales and marketing context, you’re eager to roll up your sleeves and build the infrastructure to support it.

The team currently consists of two Senior full-stack developers, one frontend developer from Crunch and CTO on client side.

**Mandatory requirements:**
5+ years of experience in a Data Engineering role, with a focus on building data pipelines or conducting data intensive analysis
Hands-on experience with ETL and ELT methodologies
Python, SQL, and (Bash) scripting languages
AWS Cloud-based services (Glue, Redshift, Lambda)
Orchestration tools (Airflow, Luigi)
Logging, Monitoring, and Alerting tools (DataDog, PagerDuty)
Experience solving problems and resolving outages in a large scale production environment

**Will be a plus:**
Experience working with data at scale (TB and PB of data)
CI/CD Pipelines (Jenkins)
Stream Processing (Dataflow, PubSub, Kinesis)
Infrastructure-as-a-code tools (CloudFormation, Terraform)
RabbitMQ
Containers and container orchestration (e.g. Docker, ECS, Kubernetes)

**Main responsibilities:**
Identify data needs for business and product teams, understand their specific requirements for metrics and analysis, and build efficient and scalable data pipelines to enable data-driven decisions across the company
Design, develop, and own batch and streaming data pipelines and models that power internal analytics for product and business teams
Implement shared infrastructure used by all services and teams within the company's platform
Contribute to and improve internal standards for style, maintainability, and best practices

**We offer:**
Annual paid leave – 18 working days (100% compensation)
Annual paid sick leave – 10 working days (100% compensation)
Medical insurance
Opportunity to become a part of our professional team
Opportunity to participate in various events (educational programs, seminars, training sessions)
Flexible working schedule
Free parking place
Massage in the office
Competitive compensation