Position: Big Data Engineer
Company: SQRD.tech
Category: Data Engineer
Experience required: 5y
English level: upper
Published: 2023-09-01T00:00:00+03:00

Location: Remote from Poland

Industry: Banking and Finance

Job Description:

Job Summary:
We are seeking an experienced Big Data Engineer to join our project dedicated to a prestigious banking client. The ideal candidate should have a broad understanding of Big Data technologies and a minimum of 4 years of experience in this field. You will be responsible for working with Hortonworks HDP/HDF, ELK stack, programming languages such as Java and Scala, messaging technologies like Kafka, and data processing frameworks like Apache Spark. If you also have experience with Google Cloud Platform (GCP), Python, DataFlow, BigTable, BigQuery, PubSub, Kubernetes, Airflow, and machine learning libraries such as TensorFlow, that's a plus!

Key Responsibilities:

- Develop and maintain Big Data solutions and infrastructure.
- Utilize Hortonworks HDP/HDF, ELK stack, Kafka, and other relevant technologies.
- Work with programming languages including Java and Scala.
- Implement and optimize data processing using Apache Spark.
- Optionally, deploy services on Google Cloud Platform (GCP) and work with additional tools like Python, DataFlow, BigTable, BigQuery, PubSub, Kubernetes, Airflow, and TensorFlow.

Technical Requirements:

- Fluent English language skills.
- Broad understanding of Big Data technologies.
- Experience with Hortonworks HDP/HDF.
- Experience with ELK stack.
- Experience with programming languages (Java, Scala).
- Experience with messaging technologies (Kafka).
- Experience with data processing frameworks (Apache Spark).

Nice to Have:

- Experience of deploying services on Google Cloud Platform (GCP).
- Experience with programming languages (Python).
- Knowledge of DataFlow, BigTable (HBase), BigQuery, PubSub, Kubernetes, Airflow.
- Knowledge of machine learning libraries such as TensorFlow.

Required Technical Skills:

- Java
- Scala
- Kafka
- Apache Spark

Preferred Technical Skills:

- GCP (Google Cloud Platform)
- Python
- BigTable (HBase)
- Kubernetes
- Airflow
- TensorFlow

About the Project:

The Big Data Engineer role is dedicated to serving our esteemed banking client. The project involves working on Big Data solutions and infrastructure, leveraging technologies such as Hortonworks HDP/HDF, ELK stack, Kafka, and Apache Spark. Optionally, the role may involve deploying services on Google Cloud Platform (GCP) and working with a range of other cutting-edge tools.