Position: Lead Data Engineer
Company: Datarob
Experience required: 5y
Category: Data Engineer
English level: fluent
Published: 2022-03-01T00:00:00+02:00

Responsibilities:
* Design, implement, test, deploy, and maintain stable, secure, and scalable data engineering solutions and pipelines in bigdata environment with GCP solutions and kubernetes
* Build reports and data visualizations, using data from the data warehouse and other sources.
* Produce scalable, replicable code and engineering solutions that help automate repetitive data management tasks.
* Integrating new sources of data into our central data warehouse, and moving data out to applications.
* Analyze and benchmark data tools to improve data team efficiency.
* Insure that data is delivered with good format and structure in the pipelines
* Produce quality code allowing resusing and solution scalibility
* Select and implement the GCP services to guaranty the good data pipelines processing
* Help data team to implement data solutions including SQL query modification and python coding good practice
* Maintain pipeline auto documentation
* Produce automated tests adapted to timeseries bigdata environment
* Help to deploy data pipelines in continuous deployment
* Help and challenge the data team to improve the quality of deliverables
* Train data team members on data engineer skills
* Help product owner, scrum master and CTO to manage project and sprint execution


Experience:
* 6+ years of experience as an Engineer and 3+ years of experience in a similar role (e.g. Data Engineer)
* Good understanding of database technology (postgres, mysql, time series DBs, bigquery)
* Good knowledge of SQL and data modelling in a data warehouse, data lake or data mart environment
* Confident working with APIs (ideally marketing oriented APIs e.g. LinkedIn & Google Ads)
* Python with some experience using Python oriented data tools such as Kedro, nbdev
* Proficient in data modelling / object-oriented programming
* Used & worked with data pipeline & data management tools such as Argo or Apache Airflow
* Data-driven and good with numbers whilst being able to explain complex concepts
* Fluent in English