Position: Lead Data Architecht / Data Modeler (AWS) (Python)
Company: at NIX Solutions Ltd.
Category: Lead
Experience required: 1y
English level: intermediate
Published: 2021-02-01T00:00:00+02:00

**WE NEED A HERO:**
- someone who has played a role at a Lead /arch level and has been involved with dealing with data at scale in PROD;
- who is able to analyze, assess, document, and design scalable and sustainable data architecture and data transformation processes within a large analytics pipeline;
- who will review the implementation of improvements to the data architecture, providing feedback and guidance to product developers, data scientists, and product owners;
- is comfortable collaborating with various teams/regions in driving facilitating data design, identifying architectural risks, and developing and refining data models and architecture frameworks.
**REQUIRED SKILLS:**
- Production experience with Spark at scale;
- Building data pipelines, CICD pipelines, and fit for purpose data stores;
- Dimensional Data Modeling;
- Building data pipelines that process more than 1TB both in streaming and batch mode;
- Working with data consumption patterns;
- Working with automated build and continuous integration systems.
**TECHNOLOGIES:**
- Microservices development in two of these languages: Python, Java, Scala;
- Big Data Technologies: Apache Spark, Hadoop;
- Relational Databases: Postgres, MySQL;
- NoSQL: MongoDB, DynamoDB;
- Data-warehousing products: Snowflake or Redshift;
- Cloud technologies: AWS (Terraform, S3, EMR, EC2, Glue, Athena);
- Orchestration: Apache Airflow and MLFlow.
**NICE TO HAVE:**
- KubeFlow;
- Spark on Kubernetes.