Position: Senior Big Data Engineer
Company: RecruitGarden
Category: Python
Experience required: 3y
English level: upper
Published: 2021-04-01T00:00:00+03:00

Our client is a New York City-based FinTech startup that is building a trading and risk software platform for the investment banking industry to sell as a service.

We’re looking for a Big Data engineer who will work on building new parts of scalable Data Lake and expand the existing ones based on cutting edge technologies (Spark, Hadoop, Databricks). The Data Lake is used by quantum analysts for ad hoc queries and by data scientists to build models for predicting investment opportunities. Data enters the Data Lake using ETL ingestion pipelines implemented with Apache Airflow. The specialist will join a distributed team on DataArt’s side that consists of 30 people from Ukraine, Russia, Poland, and the US.

**Must have**

Excellent knowledge of the general principles of Big Data
Experience with distributed systems
Excellent knowledge of Spark (Core, SQL, Streaming)
Excellent knowledge of Java, Scala, or Python
Experience with AWS or other cloud providers
Experience with Big Data solutions
Good spoken English

**Would be a plus**

Knowledge of Kafka
Knowledge of Apache Airflow
Infrastructure management (DevOps)
Knowledge of PySpark
Knowledge of Databricks
Experience with Kubernetes