Position: Principal/Architect Big Data Engineer (The Human Defense Platform)
Company: Sigma Software
Experience required: 5y
Category: Data Engineer
English level: fluent
Published: 2023-09-01T00:00:00+03:00

We are looking for a passionate Senior Data Engineer with expertise in cloud technologies, Big Data, and distributed systems to join our Data Engineering team. In this role, you will be responsible for the end-to-end development of the data pipeline and data models, working with major data flows that include structured and unstructured data. Your focus will be on developing and integrating systems that retrieve and analyze data. 
 
What we offer: 

project from scratch
leading experienced team
ability to make technical decision, choose technologies 
 
You will be a part of the HUMAN front line in our commitment to helping protect companies (and their customers in turn) from both revenue and reputation risk caused by malicious bots. 
 
Our Technologies stack: Golang, Airflow, BigQuery, Snowflake, GCP, Docker, Kafka, MongoDB, Redis, and more. 

**CUSTOMER**
HUMAN was founded in 2012 in Brooklyn. Our humble beginnings led to the creation of the Human Defense Platform, the backbone of all our products that safeguards enterprises from sophisticated bot attacks, fraud, and account abuse. Today we verify the humanity of more than 20 trillion interactions per week for some of the largest companies and internet platforms.

**PROJECT**
You’ll be joining us at an exciting moment in the HUMAN story: we joined forces with two big companies in a market-changing merger. Together under the HUMAN brand, we will disrupt the economics of cyber crime. We hope you can join us in that mission.

**REQUIREMENTS**
At least 7+ years of professional software development experience 
At least 5+ years of working experience of developing data applications in Golang, Scala, Java, and Python. Hands-on coding experience required
Strong knowledge of data modeling 
Previous success experience of building data warehouse, data lake or other data modelling paradigm Strong data governance practices, with understanding of how to implement: data stewardship, data quality, data lifecycle management, data security
Proficiency in creating batch and streaming (both real-time and micro-batch) data pipelines
Deep knowledge of building, scaling, and managing a large-scale data pipeline in the cloud (GCP preferred) from the ground up
Experience in creating solution designs and large-scale architecture balancing between various types of constraints
Strong database knowledge. Experienced with relational DBs (OLTP and OLAP) and with NoSQL solutions. Hands on DDL/ DML/ SQL queries
Well-versed in large-scale messaging tools such as Kafka (must)
Expert at cloud based modern OLAP database solutions (e.g., Snowflake, BigQuery, Redshift) for managing data warehouse
Ability to take responsibility for technical and engineering decisions on the project
Experience in proposals and write-up with proven ability to determine pros and cons
Advanced level of English 
 
NICE TO HAVE:
Experience in similar migrations before would be a huge plus  
PERSONAL PROFILE
We are looking for a person who is permanently working on self-development, has a goal-oriented way of thinking, strives to provide high-quality solutions, is open to feedback, keeps his or her commitment, and has effective communication skills 

**RESPONSIBILITIES**
Design an architectural solution that will merge data platforms of the two giant companies (and later the third one)
Achieve the best-fit solution statement
Work tightly with different teams from different departments, discussing and confirming feasibility, consistency, innovation, and efficiency of the solution
Lead the whole work-cycle by taking active part in development pipelines