Position: Senior Data Engineer
Company: Sigma Software
Experience required: 5y
Category: Data Engineer
English level: upper
Published: 2022-12-01T00:00:00+02:00

We have a great opportunity for an experienced Data Engineer to collaborate with clients on a daily basis and help improve the existing solution.

If you are ready to show a high level of engagement in making technical decisions using cutting-edge data collection and solve challenging problems that are unique to the DEI world – this opportunity is for you. 

Sounds like you? We are waiting for you on our team! 

CUSTOMER
Our clients are building an analytics platform for Diversity, Equity, and Inclusion (DEI). Their main goal is to solve the main problem in the DEI space: data. 

Our client helps businesses gain access to over 10 million novel DEI insights. Companies can run pay audits in minutes, generate annual reports at the push of a button, and see—maybe for the first time—what’s really happening across the employee lifecycle. 

PROJECT
As a Data Engineer, you will be working on customer’s core systems, engines, and product services using the most sophisticated cutting-edge technologies to solve challenging problems that are unique to the DEI world.  

Our customer develops a system, which currently processes TBs/s of data with upwards of 2m writes per second – and as they grow those numbers are ought to grow with them.  

The customer’s engineering infrastructure is built on top of GCP utilizing cutting edge technologies & software such as Bigtable, BigQuery, Apache Spark, AI Platform & Tensorflow, Apache Airflow, K8s to mention a few. 

**REQUIREMENTS**
- B.S. in Computer Science or a higher degree 
- Advanced knowledge of different non-relational schema models (Column Family, Graph, Document, Object) 
- Experience with advanced topologies and architecture design such as KAPPA and LAMBDA architectures 
- Experience in Scala or any similar JVM based language 
- Experience in Python 
- Experience with high throughput systems 
- Upper-Intermediate level of spoken and written English
 
**WILL BE A PLUS:**
- Experience in Go is a great bonus to have 

**RESPONSIBILITIES**
- Work with Apache Spark batch & real-time streaming to process data at scale 
- Work with Scala microservices hosted on K8s (GKE) to support our products 
- Work with MlLib and Tensorflow on Vertex AI to solve forecasting, simulations in addition to a variety of machine learning challenges 
- Deploy new models and engines for new ideas and analyses