Position: Middle or Senior Data Engineer
Company: AgileEngine
Category: Data Engineer
Experience required: 2y
English level: intermediate
Published: 2022-06-01T00:00:00+03:00

Qualifications
● Data engineering/architecture/warehousing experience in Google Cloud
● Experience implementing production Google analytics platforms
● The ability to apply the above to solve business problems using one or more Google
Cloud data services in combination with building data pipelines, data streams, and
system integration
● Familiarity with tools like Tableau, Looker, Data Studio, Power BI or other cloud data/BI
tools
● Proficient in Python
Requirements:
● Knowledge with ETS, SQL, GCP, Spark. ...
● Knowledge of Business Intelligence and Data Warehousing best practices, including
Data Cataloging, Data Security and Data Quality
● Proven passion for everything you build to be elegant and maintainable, in architecture
and implementation
● Intuition for and experience in design of highly fault-tolerant architecture of
loosely-coupled services
● Strong individual contributor, self-motivated, and a proven team player
● Demonstrated eagerness to learn and develop new experience

Will be a plus:
● Expertise with some of the following technologies:
○ Experience with BigQuery ML
○ Container-based virtualization (Docker)
○ Distributed pub/sub frameworks (Kafka), distributed high-throughput key-value
storage (Redis)
○ Big data storage (Cassandra, Redshift, BigQuery) and processing (Cloud
Dataflow, Spark)
○ High-concurrency relational databases (PostgreSQL)
○ Distributed log collection and analysis
● Advanced academic training in distributed systems, security, or a related field

Responsibilities:

● Implement data services including Google BigQuery, Apache Spark & Hadoop
(Dataproc), Data Fusion / Dataflow, Cloud Composer, Cloud Function, Cloud Run, Cloud
Pub/Sub, Google Data Studio
● Design and develop scalable data ingestion frameworks to transform a wide variety of
datasets
● Research, analyze, recommend and select technical approaches for solving challenging
and complex development and integration problems
● Assist in designing multi-phased data strategies, including designing multi-phased
implementation roadmaps
●
● Own the full software development lifecycle for user-facing features
● Constantly pay down technical debt and improve software development processes
through tool development
● Constantly learn the latest trends and tools applicable to PatternData problem,
evaluating and integrating them as appropriate
● Work with management to define and refine backend service technology roadmap
● Evangelize and educate best practices for maintainable high-concurrency server code
among the rest of the software organization