Position: Big Data Team Lead
Company: Intellias
Experience required: 5y
Category: Data Engineer
English level: intermediate
Published: 2022-11-01T00:00:00+02:00

Our fast-growing and professional team is looking for Data Engineer Team Leader. We are looking for an experienced and results-driven Engineer. The successful candidate will lead a Data Team that develops DWH, and also build Python applications to ingest streaming and extracted data and persist to cloud storage. We use Python and SQL with AWS cloud technologies to automate the delivery of reports generation and data transformation. Our client is one of the leading telecommunications companies based in the United States. By joining our team, you will get exceptional technical\multinational experience working with highly scalable, resilient solutions, and innovative products that already on production. 

Responsibilities
Team Lead will play a vital role in collaborating as part of this cross-functional Agile team to create and enhance data ingestion pipelines and address big data challenges. The Team Lead will work closely with the Data Architect and Chief Architect, systems engineers, software engineers, and data scientists on the following key tasks:

• Provide Extraction, Transformation, and Load (ETL) experience coupled with enterprise search capabilities to solve Big Data challenges
• Managing the day-to-day activities of the team
• Scope Management
• Participate in the process of designing, data engineering, and developing data services (Streaming, ETL/ELT, Real-time analytics, Reporting) using Python, SQL and AWS services
• Leverage strategic and analytical skills to understand and solve customer and business centric questions
• Collaborate with data team members to ensure all services are reliable, maintainable, and well-integrated into existing platforms
• Work effectively with remote teams in various remote time zones
• Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it

Requirements:
• 2 years of experience on leading position
• 2 years of experience with big data tools: Hadoop, Spark, Kafka
• 1 years of experience with object-oriented programming with Java or Python
• 3 years of experience with and managing data across relational SQL and NoSQL databases like MySQL, Postgres, Cassandra, HDFS, Redis, and Elasticsearch
• Experience designing, building, and maintaining data processing systems
• At least 3 years’ experience with modern DevOps automation ecosystems, preferably Git/GitHub/Bitbucket, Buildkite or Ansible (or similar)
• Experience in designing/developing platform components like caching, messaging, event processing, automation, transformation and tooling frameworks

• At least 3 years experience configuring, using, and performance tuning AWS cloud services: preferably S3, Glue, Athena, Kinesis, Firehose, Lambda, Cloudwatch, ECS, API Gateway, RDS (Postgres), SQS, SNS, SES
• Experience transforming data in various formats, including JSON, XML, CSV, and zipped files
• Demonstrate understanding of modern APIs and endpoints, like REST and GraphQL
• Strong interpersonal and communication skills necessary to work effectively with customers and other team members
• Real-world experience handling large data volumes (terabytes of storage and billions of rows)
• Experience with CloudFormation or TerraForm

• Experience using C*, Spark, Kafka, KSQL, Confluent, Pulsar and/or Kinesis helpful

• Proficient with Linux and shell scripting

• Confidence and skillful negotiating skills
• Good organizational skills
• Results-Oriented approach