Position: Senior Data Engineer
Company: GM Consulting GROUP
Category: Data Engineer
Experience required: 3y
English level: upper

Specific roles and responsibilities:

· Designs data models, builds-out models in warehouse, and develops views/queries for reports

· Implement PySpark processes that leverage SQL DML and wraps them in AWS Glue

· Writes AWS Lambda functions to trigger AWS Glue jobs from AWS S3 actions

· Develop and monitor data processes (ETL pipelines)

· Participates in planning, analysis, and design of new data solutions and tooling for platform

· Investigates, troubleshoots, and remedies performance bottlenecks in the data platform/pipelines

· Learns new technologies and keeps up with both traditional and emerging best practices

· Works independently and collaborates with other members of the team to ensure successful delivery of business features

Qualifications:

· 3-5+ years of production experience in data engineering

· Working knowledge of Python, PySpark, and SQL

· Engineering solutions involving relational, key/value, and big data systems

· Experience operating data platforms in cloud environments, AWS preferred

· Hands-on experience with Git version control system

· Excellent debugging and testing skills, and interest in quickly learning new technologies

· Experience with Agile methodology and software development (pull requests and code reviews)

· Commitment to following security and performance best practices

· Strong sense of collaboration, teamwork, and accountability

· Experience building ETL pipelines in AWS, using Glue, Lambda, EMR, Redshift, etc.

· Experience with designing and building scalable data solutions in AWS