Position: Data Engineer (Sweden)
Company: DataJob
Category: DevOps
Experience required: 3y
English level: upper
Published: 2021-06-01T00:00:00+03:00

The platform will be serving hundreds of data product teams creating data assets to be shared within the corporate; you understand the challenge of design and developing pipelines that ad-hear to shared glossary and linked data sets. Passionate about excellence in pipeline engineering; scalability and quality assured data assets. We are looking for someone who is up for a challenge and help Group grow.

The team is building the first MVP, and along the way we will have a lot of discussions on functionality and architectural set up. We hope that you are comfortable in sharing your experiences and we value your opinion. The team own the development life cycle, and we aim to continuously bring value to the product teams who will use this new platform. The team is autonomous, and we hope that you appreciate that. We also think it is important to have a hands-on mindset and thinking that no challenge is too big with the right support and knowledge at hand.

We are looking for you who:
• Have significant experience in developing backend solutions within Agile teams using DevOps strategies. Your knowledge in this area provides you with an outstanding view on different technology stacks and architecture paradigms.
• Understand the challenge of design and development large scale distributed system.
• Have an open mindset and used to giving and receiving feedback on suggested improvements as part of your daily work.
• Use your data engineering skills to build our new internal platform for large-scale data processing needs in a scalable and robust way.
• Can identify long-term impacts on quality, security and cost and using common best practices for testing, deployment and run and constantly analyse and benchmark for bottlenecks and plan execution improvements.
• Are thrilled to learn about new technology and pass on this knowledge to your colleagues.
• Have a hands-on approach when building solution prototypes when needed.
 
Skills and abilities:
• Expert skills in working with distributed processing platforms (Spark).
• Expert in python programming.
• Excellent skills in SQL programming.
• Expert in working with Azure Data Lake Gen 2 and Azure Messaging Services.
• Proficient skills in Data Modelling.
• Proficient experience of building cloud-based solutions in Azure.
• Skilled in software engineering practices like CI/CD, unit testing, source control, design patterns etc.
• Experience with Terraform, Databricks, NoSQL Databases (preferably CosmosDB) & Airflow is good to have.
• Knowledge on how to work with data Governance and Data Catalog Systems is good to have.
• Experience with Kubernetes and Azure Durable Functions is a bonus.
• Fluent in English in speech and writing is required.