Position: Lead Data Engineer
Company: GlobalLogic
Category: Data Engineer
Experience required: 5y
English level: upper
Published: 2023-07-01T00:00:00+03:00

Job Description
0) 5+ years of experience in the position or a related occupation.
1) Delivery of multi-tier, highly scalable, distributed Data Platforms.
2) Architecture, Design, Development and Implementation of Data Warehousing, Big Data, Data Analytics solutions in on-premises or cloud environments
3) Distributed computing frameworks: Hive/Hadoop, Apache Spark, Kafka, Airflow.
4) Working with one of the following programming languages: Python, Scala, Java, or Unix and SQL.
5) Working on building ETL (Extraction Transformation and Loading) jobs using pySpark or any ETL tools like Data Stage, Informatica, SQL.
6) Software development and Data Integration working on Data warehouse, big data processing and distributed systems.
7) Developing CI/CD pipelines.
8) SQL and NoSQL database design, Data Modeling and Performance Tuning.
9) SQL Databases like Teradata, Amazon Redshift, Oracle, SQL Server, PostgreSQL and NoSQL Databases like Amazon DynamoDB, Cassandra
10) At least one from the following list of Cloud services: AWS Services EC2, S3, RDS, DynamoDB, VPC, DMS, EMR, IAM, or Amazon Managed Workflows for Apache Airflow
11) Business Intelligence tools like Tableau, QuickSight, to build reporting Dashboards.

 
Job Responsibilities
Design, develop and deploy scalable web applications and services, being part of everything from contributing to web technology and framework stack, to use case and data model design.

Drive architecture discussions, take ownership and responsibility over new projects and deliver high quality software with tight timeline.

Partner with customer security and compliance teams to ensure each application meets compliance, security and audit requirements.

Participate in the continual improvement of engineering tools, cloud-based technologies/services, and processes.

Define and participate in Agile development process, including scoping, technical design, estimation effort, coding, testing, debugging, code reviews, maintenance and support.

Collaborate with cross-function teams. Apply advanced troubleshooting techniques to provide unique solutions for critical, highly complex, customer problems that span multiple AWS services.

Assist business users on a wide variety of topics, including Data warehousing, AWS cloud architecture, migration and troubleshooting complex performance issues.

Architect, Design, Develop and Implement Data Integration solutions in Data Warehouse, Data Lake and Data Analytics platforms.

 

Department/Project Description
Our customer is one of the biggest manufactures of a variety digital media players for video streaming. Project responsibilities will be to - design and develop scalable platform services on which customer run company and ensure it's scaleble - to meet rapid employee and business growth. Design, develop, and maintain efficient large-scale data pipelines to collect and transform rich data using big data technologies.