Position: Data Engineer
Company: Techstack
Category: Data Science
Experience required: 2y
English level: intermediate
Published: 2021-06-01T00:00:00+03:00

Our company is looking for a Data Engineer to join a global industrial project, with the passion to drive the design, execution, and ongoing support of data services enabling large-scale data collection, near real-time and offline analytics, distributed search, using AI and ML for security, engineering, and business intelligence purposes.

**Stack of Technologies:**

GCP;
BigData;
Spark or mapReduce;
SQL/BigQuery/PubSub
BigQuery, Cloud Composer, Data Fusion, GCS and GKE;
Python or JavaScript;
Terraform scripting.

**About the role:**
As a Data Engineer you will work with one of the largest B2B data sets in the world using the latest cutting-edge technologies. You will have an opportunity to build data pipelines and analyze data structures. And connect sources to data lake and work with Geodata.

**Responsibilities:**
• Maintain and support existing ingestion pipelines;
• Optimize queries based on performance testing;
• Design and implement new ingestion pipelines that bring data from external data sources (HTTPS, SFTP) or internal data sources (JDBC, HTTP, MQQT);
• Work with Application engineers and product managers on refining data requirements;
• Implement and test fine grained access control setup (per dataset, per column).

**Required skills:**
• Strong experience in building cloud-native data engineering solutions using GCP or AWS platforms;
• 2+ years of development experience with BigData;
• Prior experience of building data ingestion pipelines of telemetry data in GCP/AWS including app monitoring, performance monitoring and network monitoring logs;
• Background in building data integration applications using Spark or mapReduce frameworks;
• Track record of producing software artifacts of exceptional quality by adhering to coding standards, design patterns and best practices;
• Strong background in SQL / BigQuery / PubSub;
• Experience with GCP products such as BigQuery, Cloud Composer, Data Fusion, GCS and GKE or corresponding technologies on AWS platform;
• High proficiency in working with Git, automated build and CI/CD pipelines;
• ETL scripting in Python or Javascript;
• Knowledge in some terraform scripting (adding new datasets, buckets, IAM);
• Intermediate level of English or higher.

**Our benefits:**
• Strong opportunities for professional and career growth - Meetups, TechClubs, Professional Library and more;
• Challenging tasks with a friendly experienced team;
• Flat hierarchy without micromanagement — our doors are open, and all teammates are approachable;
• Direct communication with stakeholders and the ability to influence product development;
• Up to 50% compensation of educational courses and conferences price for professional growth;
• Free English classes and English Business Course;
• 23 business days leave and medical Covid support;
• Regular team events and activities;
• Gifts for significant life events.