Position: Senior/Middle Systems Engineer
Company: EPAM
Experience required: 3y
Category: Sysadmin
English level: intermediate
Published: 2021-01-01T00:00:00+02:00

PROJECT DESCRIPTION
Principal Duties and Responsibilities:

As a systems engineer with focus on data you will work as an integral part of our Cloud & DevOps team for a customer. The ideal candidate has solid background in system administration and is able to work as a part of DevOps team running an analytical ecosystem in modern cloud-based big data systems in a fast-paced, agile environment.

The specific focus is on processing engine development as well as the design, development, automation and optimisation of its usage in client’s analytic ecosystem:

* Participate in a continuous delivery pipeline to fully automate deployment of the highly available cloud platform that supports multiple teams/projects

* Build tools for deployment, monitoring and operations. Troubleshoot and resolve issues in our development, test and production environments

* Work with platform architects on software and system optimisations, helping to identify and remove potential performance bottlenecks

* Stay up-to-date on relevant technologies, understand trends and opportunities to ensure we are using the best possible techniques and tools

* Understand, implement, and automate security controls, governance processes, and compliance validation

* Design, manage, and maintain tools to automate operational processes


Qualifications:

* Experience with major Data technologies and frameworks including but not limited to Hadoop, Hive, HBase, Spark, Kafka, Flume, ZooKeeper, Parquet, Avro, etc

* Background at least one of the RDBMS or NoSQL engines: PostgreSQL, MySQL, Cassandra, Elasticsearch, Redis, MongoDB, Impala, Kudu, etc

* Hadoop administration experience, including clusters troubleshooting, migration and consolidation

* Experience with Big Data solutions developed in large cloud computing infrastructures such as Amazon Web Services (EMR, RedShift, Kinesis, Athena, Glue, Lambda), Azure (Databricks, Data Factory, Data Lake, Azure Batch, Azure Function, HDInsight), or Google Cloud Platform (Cloud Dataproc, Cloud Dataflow, BigQuery, Google Cloud Dataprep, Cloud BigTable, Cloud PubSub, Cloud Composer)

* Practical experience with Java, Scala or Python code lifecycle in BigData pipelines

* Strong background in Linux administration and scripting (Bash)

* Strong understanding of Continuous Integration and Continuous Delivery principles and experience building CI/CD processes with one of Gitlab, Travis, Jenkins or Octopus

* Software process automation with popular scripting languages (e.g. Python, Ruby)

* Good understanding of version control systems (Git)

Highly desired, but not required, skills include:

* Experience with containerization platforms such as Docker, Kubernetes/Openshift

* Cloudera CDH 5.1 or higher: installation, set-up, on-prem deployment, cluster balancing and performance optimization, workload monitoring & proactive alerting

* Experience in usage of infrastructure as code and desired state management tools (e.g. Ansible, Chef, Saltstack, Terraform)