Position: Senior Data Engineer
Company: Data Science UA
Experience required: 5y
Category: Data Engineer
English level: fluent
Published: 2022-08-01T00:00:00+03:00

**Responsibilities and essential job functions include but are not limited to the following:**
- Demonstrate deep knowledge of the data engineering domain, including non-interactive (batch, distributed) & real-time, highly available data, data pipelines.
- Deep knowledge of data as a concept and the development of domain driven data products.
- Optimization of data products to service customer personas, Data science, AI/ML and data visualization.
- Knowledge of semantic data concepts.
- Build fault-tolerant, self-healing, adaptive, and highly accurate data computational pipelines.
- Provide consultation and lead the implementation of complex programs.
- Develop and maintain documentation relating to all assigned systems and projects.
- Perform root cause analysis to identify permanent resolutions to software or business process issues.

**Qualifications:**
- Bachelor’s degree in Computer Science, management information systems, or related discipline, or equivalent work experience.

**Must have technology skills (7/10 or higher):**
- Strong/expert Spark (PySpark) Using Jupyter Notebooks, Colab or DataBricks.
- Hands-on data pipeline development, ingest patterns in Azure.
- Orchestration tools, ADF or Airflow.
- SQL.
- Denormalized Data modeling for big data systems.

**Must have competencies:**
- Collaborative, able to work remotely, and still be an engaging team member.
- Strong analytical and design skills.
- Architect and design large-scale high performance distributed systems (7-10).
- Knowledge of SQL Platform (7-10).
- 3+ years of experience in No-SQL Platform.
- 3+ years of experience in Spark.
- 3+ years of experience in Data platform implementation on Azure or AWS.
- 2+ years of experience in CI/CD experience.
- 2+ years of experience in Exposure to SOA architecture.