Position: Senior/Lead BigData Engineer
Company: EPAM
Experience required: 5y
Category: Scala
English level: upper
Published: 2021-02-01T00:00:00+02:00

Requirements
Advanced experience in software development with Big Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning).
Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security.
Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), Data Warehouse or Database architecture.
Strong experience with data modeling; hands-on development experience with modern Big Data components.
Cloud: experience in designing, automation, provisioning, deploying and administering scalable, available and fault tolerant systems.
Good understanding of CI/CD principles and best practices.
Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills.
Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA.
Motivated, independent, efficient and able to handle several projects; work under pressure with a solid sense for setting priorities.
Ability to work in a fast-paced (startup like) agile development environment.
Strong experience in high load and IoT Data Platform architectures and infrastructures
Vast experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn.
Experience in direct customer communications.
Experience in technology/team leading of data oriented projects.
Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis.
Experienced in different business domains.
English proficiency.
Advanced understanding of distributed computing principles.
Technology stack

Programming Languages: Java/ Scala; Python; SQL; Bash.
Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, Zookeper;
NoSQL: Cassandra/ Hbase; MongoDB.
Queues and Stream processing: Kafka Streams; Flink; Spark Streaming; Storm; Event Hub; IOT Hub; MQTT; Storage Queues; Service Bus; Stream Analytics.
Data Visualization: Tableau, QlikView.
ETL & Streaming Pipelines: Pentaho; Talend; Apache Oozie, Airflow, NiFi; Streamsets.
Operation: Cluster operation, Cluster planning
Search: Solr, Elasticsearch/ELK
InMemory: Ignite, Redis.
Solid Cloud experience with 2 or more leading cloud providers (AWS/Azure/GCP): Storage; Compute; Networking; Identity and Security; NoSQL; RDBMS and Cubes; Big Data Processing; Queues and Stream Processing; Serverless; Data Analysis and Visualization; ML as a service (SageMaker; Tensorflow).
Enterprise Design Patterns (ORM, Inversion of Control etc.).
Development Methods (TDD, BDD, DDD).
Version Control Systems (Git, SVN).
Testing: Component/ Integration Testing, Unit testing (JUnit).
Deep understanding of SQL queries, joins, stored procedures, relational schemas; SQL optimization.
Experience in various messaging systems, such as Kafka, ZeroMQ/ RabbitMQ.
Rest, Thrift, GRPC, SOAP.
Build Systems: Maven, SBT, Ant, Gradle.
Docker, Kubernetes, Yarn, Mesos.