Position: Data Engineer
Company: EPAM
Category: SQL
Experience required: 3y
English level: intermediate
Published: 2021-09-01T00:00:00+03:00

DESCRIPTION
Our client connects companies, customers, and communities across the UK, delivering a ‘one-price-goes-anywhere’ universal postal service to over 29 million addresses. As one of the UK's leading companies, the customer is focused on being recognized as the best delivery company in the UK and across Europe.

We are looking for an Experienced Data Engineer (or ETL Developer) who is proficient in building and maintaining data pipelines in GCP. Our ideal candidate is willing to work fully with GCP Proficient in ETL/ELT process, data migration from on-premise to GCP.

On current position, you are expected to have a good match (60-70%) with the following technologies: 
• ETL or ELT (end to end understanding) 
• Apache Nifi
• GCP (Google Cloud Platform): Cloud DataFlow or Apache Beam, Data Composer or Apache Airflow, BigQuery 
Responsibilities
Data ingestion with Apache Nifi
Data Job scheduling and orchestration with Apache Airflow or Google Data Composer
Data processing and preparation with Apache Beam or Google DataFlow
Messaging systems based on Pub/Sub or Kafka
Data processing with Apache Hadoop/Cloudera Hadoop or Google DataProc (to run Spark jobs)
Requirements
Strong knowledge of SQL
Proficiency in DataFlow
Expertise in Data Composer or Apache Airflow
Experience in Apache Beam
Good understanding of GCP
Familiarity with DWH solutions, preferably BQ
Experience in Agile software development
Good communication skills
Intermediate or higher English level, both spoken and written (B1+)