Position: Lead Big Data Engineer
Company: SoftServe
Experience required: 5y
Category: Data Engineer
English level: upper
Published: 2022-06-01T00:00:00+03:00

WE ARE
SoftServe is a leading advisory and IT development service company focused on helping clients to build high-end innovative software solutions.

We are a big distributed team working with a fantastic customer, a giant in the mining industry, located in Canada. Our customer is a diversified resource company committed to responsible mining and mineral development with business units focused on steelmaking coal, copper, zinc, and energy. Headquartered in Vancouver they own or have interests in 13 operating mines, a large metallurgical complex, and several major development projects in the Americas. The client has expertise across a wide range of activities related to exploration, development, mining, and minerals processing, including smelting and refining, health and safety, environmental protection, materials stewardship, recycling, and research.

Our customer’s corporate strategy is focused on exploring for, developing, acquiring, and operating world-class, long-life assets in stable jurisdictions that operate through multiple price cycles.

YOU ARE
Excited about the following technologies

- Azure Data Factory
- Databricks or HDInsight or Synapse Analytics
- Azure SQL DW (a.k.a. Synapse SQL Dedicated Pool)
- Azure Blob Storage and Azure Data Lake
- Azure SQL DB/Azure SQL MI/SQL Server
- Azure Functions
- Apache Spark
- Event Hubs or similar (Kafka)
- Azure Stream Analytics or similar (Spark, Flink, Storm)
- Azure Cosmos DB (nice to have)

In addition, you are proficient in
- Complex cloud services management concepts
- Networking (virtual networks, firewalls, communication between services)
- Authentication and security (IAM, service principals and managed identities), logging and monitoring, event-driven architecture (Event Grid, Azure Functions, Logic Apps)
- Hadoop ecosystem (e.g. Hive, Impala, Kafka, Spark, HBase, Presto), streaming platforms and stream ingestion services (Event Hubs, Kafka), stream analytics concepts and technologies (Flink, Spark, Storm, Azure Stream Analytics)
- Database modeling, DWH design, and implementation
- ETL/ELT process design and implementation
- Massive-parallel-processing (MPP) systems: Azure SQL DW (Synapse), GCP BigQuery, Impala, AWS Redshift
- NoSQL databases
- Optimization of code, ETL processes, and SQL queries
- SQL language and database management techniques (row-level security, built-in database roles, partitioning, index maintenance)
- OLAP systems and reporting tools (Azure Analysis Service, Power BI)
Python, Pyspark, Pandas, Airflow, Git and Gitflow, CI/CD process building tools (Azure DevOps)
- Project management methodologies (Agile, Waterfall), the end-to-end architecture of Big Data solutions design and implementation
- Communication with stakeholders in written and verbal form, suggesting the best solution among different options, and conducting trade-off analysis
- Problem-solving and team leadership
- Upper-Intermediate English level

YOU WANT TO WORK WITH
Modern technologies popular on the market
Interesting customers and challenging projects
Either alone on a subproject or in a team
Both developing new functionality and improving the existing ones
Remote teams located in Ukraine, Poland & Canada

TOGETHER WE WILL
- Accomplish great things
- Get a great deal of learning and development opportunities along with our structured career path
- Process dynamic projects at a stable place of work
- Take part in internal and external events where you can build and promote your personal brand
- Give you access to experienced specialists willing to share their knowledge
- Care about your individual initiatives: we are open to them, just come and share your ideas