Position: Senior Data Engineer
Company: United Software
Category: Data Engineer
Experience required: 5y
English level: upper
Published: 2022-04-01T00:00:00+03:00

This means working with source system owners in understanding the data outputs, defining the optimal way to ingest and store the data, and based on downstream use cases implement the relevant data models to analytics systems.

The Senior Data Engineer is responsible for defining and implementing end-to-end data pipelines, transforming the source data into data models suitable for analysis. This covers batch as well as real-time data sources for storage in data lake/warehouse. In order to enable successful downstream analytics, it is key to design and implement robust and reliable pipelines, with built-in quality checks and recovery in case of failures in data sources or processing steps. 

Senior Data Engineer fills a critical function in having the end-to-end view of the data flows, identifying and rectifying potential bottlenecks or points of failure.

In performing the data engineering function, the Senior Data Engineer also collaborates with Business Intelligence and other analytics teams in translating analytics requirements into data models. To do this successfully, the data models and pipelines need to be flexible enough to handle a wide variety of use cases, both for standardized reporting and for custom analytics or data science projects. The role also closely works with the Data Governance function in defining the meta-data, data catalog and quality elements to ensure the data delivered to analytics functions is reliable, correct and well understood.

**Project description:**
Salam mobile is a leading Saudi based telecom provider offering next-generation ICT solutions for broadband, connectivity, cloud computing, managed services, security services, satellite services and internet services for government, businesses, wholesale and consumers sectors.

We help businesses be smarter, more digital, more integrated and secure. We support our community in reaching new heights as a digital society. We offer our employees the opportunity to bring ideas and drive commercial and cultural impact.

**Responsibilities:**

- Collaborate with source system data owners in interpretation and definition of the data sources
- Definition and implementation of batch and real-time data pipelines for ingestion of source data into data eco-systems
- Operational management and oversight of pipelines to ensure data flows as designed, and rectification of operational issues as they occur
- Specification and implementation of data lake/warehouse infrastructure suitable for the data sources and analytics use cases
- Definition of data storage schemas across data lake
- Successful implementation of required data pipelines and storage infrastructure as per analytics requirements
- High degree of end user satisfaction with data provided to downstream analytics end users and systems
- Proactive and effective management of any incidents regarding data pipelines, ensuring high availability and reliability and data warehouse
- Management and monitoring of data storage infrastructure to ensure high degree of reliability and performance
- Data modeling based on logical models for source systems and downstream analytics applications

**OPERATIONAL DATA MANAGEMENT**

- Manage Data Engineering team members
- Responsible for the integrity of data flows across the business, collaborating with data governance function to ensure high degrees of reliability in data provided to downstream applications
- Proactive management and monitoring of data outputs, identifying inefficiencies and bottlenecks in the data flows

** What you'll need**

- 5+ years of experience in Data
- Technical knowledge of ETL/ELT systems and data collection systems from different data sources (Informatica, Talend, Microsoft SSIS, etc.)
-  Technical knowledge about data warehousing and data lake technologies (Vertica, Microsoft, Hadoop/Spark)
- Knowledge about real-time data processing frameworks (Apache Kafka or similar)
- Knowledge about Databases like MS SQL, MySQL, Oracle, etc., high degree of fluency in SQL
- Skilled in working across business and technical domains, interacting effectively with business stakeholders
- Python programming experience preferred
-  Experienced in managing complex data pipelines and storage systems, as well as data modeling practices for a variety of use cases
- Exposure to managing internal and external projects for data infrastructure and systems
- Experience with cloud architectures and technologies (AWS, Microsoft Azure, etc.) a plus
-  Experience working with data science teams and advanced analytics projects preferred
- Telecoms experience a significant plus