Position: Data Engineer
Company: GlobalLogic
Experience required: 2y
Category: Data Science
English level: intermediate
Published: 2022-02-01T00:00:00+02:00

The core of the project is working with the data; the target is to re-design, transition to cloud and enhance data-driven solutions.

You will work in a small team responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. 

The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will partner with our software developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. If you are excited by the prospect of optimizing or even re-designing our data architecture to support our next generation of products and data initiatives, we would be thrilled to have you apply!

**Requirements:**

● Good working SQL knowledge and experience working with relational databases, query authoring (SQL) as

well as working familiarity with a variety of databases

● A successful history of manipulating, processing and extracting value from large disconnected datasets

● Experience building and optimizing data pipelines, architectures and data sets

● Experience performing root cause analysis on internal and external data and processes to answer specific

business questions and identify opportunities for improvement

● Strong analytic skills related to working with unstructured datasets

● Experience supporting and working with cross-functional teams

**Nice to have experience:**

● Advertising technology experience including data integrations with ad exchange(s) and identity partners

○ for example Crossix, ComScore, Liveramp, Google Ad Manager or others


**Technologies:**

● AWS: SES, RDS (MySQL), EC2, Elasticsearch Service, Route 53, VPC, etc

● Snowflake

● Fivetran

● DBT

● Airflow

● Snowplow

● Graph database (Amazon Neptune)

● CDP platforms (cencus, hightouch)

**Responsibilities:**

● Manage and optimize core data infrastructure

o Develop custom data infrastructure not available off-the-shelf

o Build monitoring infrastructure to give visibility into the pipeline’s status

o Monitor all jobs for impact on cluster performance

o Run maintenance routines regularly

o Tune table schemas to minimize costs and maximize performance

● Build and maintaining custom ingestion pipelines

● Support data team resources with design and performance optimization

o Work with data and analytics experts to strive for greater functionality in our data systems

● Build non-SQL transformation pipelines

● Build processes supporting data transformation, data structures, metadata, dependency and workload management