Position: DevOps Engineer
Company: Grid Dynamics
Category: DevOps
Experience required: 3y
English level: intermediate
Published: 2022-10-01T00:00:00+03:00

Current software delivery process for EDAP ETL pipelines implies monthly production releases, lengthy multi-step integration and manual deployment. The preceding "CI/CD foundation for EDAP ETL" project implements a toolkit and a change delivery framework for ETL applications. The framework should enable same day delivery of application features with less effort and make the development scalable to more applications and engineering teams.

It is an American publicly traded consumer retail company that sells kitchen-wares and home furnishings. It is headquartered in San Francisco, California, United States. The company has 625 brick and mortar stores and distributes to more than 60 countries with brands operated through eight corresponding websites and a gift registry. The company is one of the largest e-commerce retailers in the U.S., and one of the biggest multi-channel specialty retailers in the world. In 2019, Williams-Sonoma, Inc. was named as a Fortune 500 company for the first time in its history.

Details on tech stack
- Jenkins automation server: provisioning and configuration, Jenkins pipelines, Groovy DSL for pipeline extension.
- HashiCorp Terraform.
- Kubernetes for an application developer and operator. Helm.
- Public clouds:
   Application deployment in public clouds.
   Azure:
- BigData tools: Azure Data Lake, Azure Data Factories.
- Automation tools: Azure DevOps.
- IaaS: identity and access management, compute, storage, networking.
   HashiCorp Terraform for Azure.
- Programming:
   Java: Acquaintance with Java programming, Java build tools.
   Python: Python programming, test automation.
- Linux: power user.
- Apache Spark: power user.
- Databricks: Azure Databricks, Spark jobs, Spark SQL, Code Notebooks.

Min requirements to the candidate
- Design and implement devops solutions for medium-complex (2 weeks of work) problems. Without supervision.
- Understand and employ QA measures for application, deployment and process automation.
- Experience with SDLC processes and automation for Java or Linux applications.
- Experience with deployment automation for public clouds and Kubernetes.
- Acquaintance with BigData applications.
- Create documentation for their solutions.

Do you have client interview?
Yes

Current team size, their locations and whether team is going to grow?
Currently 6, offshore,
it's going to grow to 8 engineers + pert time architect and technical writer + 2 qa

What tasks the candidate will work with?
Deployment and operations:
Deployment environment onboarding and support (production and non-production).
- Production deployment environment onboarding.
- Deployment environment planning.
- Deployment environment provisioning self-service.
- Deployment environment monitoring implementation.
- Deployment environment maintenance.
- Deployment environment configuration management documentation, training.
SDLC infrastructure provisioning, maintenance and support.
- SDLC infrastructure provisioning.
- SDLC infrastructure maintenance and troubleshooting.
- SDLC infrastructure monitoring implementation.
- SDLC infrastructure documentation, training.
- EDAP ETL pipelines migration.
Pipeline operation support for migrated pipelines and TDM.
Data pipelines monitoring implementation.
- Data pipelines troubleshooting.
- Operation automation for data pipelines, documentation.
Tooling:
- Pipeline development tools.
- Data pipeline application scaffolding/bootstrapping.
- Developer experience - GitHub integration for Databricks notebooks and ADF.
- Pipeline migration assistance tools.
- CI/CD framework maintenance and improvement.
- Data pipeline onboarding and development support.
Test Data Management for ETL.
- TDM for ETL implementation. Incl data preparation and data source provisioning.
- Test data provisioning self-service.

Selling points for this demand (what is good about project / client / team / etc)
We are developing a world class ML/AI ETL CI/CD pipeline engine that will manage big data, versioning that and train ML models with different data sets, verify training quality and put ML models into production.
There are no standard solution on the market and we are one of the first who is going to make that as a standard approach engaging Grid Dynamics accelerators, industry best practices and company know how.
We eager to deliver a new platform that will show excellence in DevOps practices, citing edge in technologies, cloud agnostic and reusable for companies that consider to use automated CI/CD solutions for ML models, data and ETL pipelines