Position: Data Engineer (PySpark)
Company: Luxoft
Experience required: 2y
Category: Data Engineer
English level: fluent
Published: 2022-05-01T00:00:00+03:00

Project Description:
Creation of a Risk Management Tool in Cybersecurity. We are fetching data from multiple source systems and combining them to calculate various Metrics and Control Effectiveness measures.


You'll need to gain deep understanding of the diverse data sources we are dealing with, platform and coding aspects, as well as problem solving. Data from very different systems and process on a global scale is integrated in a single product. This diverse set of data needs to be aligned / mapped into a common structure to provide meaningful value to the users.
Responsibilities:
Main responsibility - developing the data pipelines.
Fetching the data is handled by another team, the main responsibility is combining data, calculating the metrics and exporting them to the reporting layer in Tableau.
The challenge of the role is to understand the overall data model (100+ tables) and find efficient ways to process and provide the data (billions of records) to Tableau.
Mandatory Skills Description:
• A PhD or Master's degree in Computer Science, a quantitative field (e.g. Statistics, Mathematics, Physics, Economics), or an equivalent education
• 2+ years of work experience as Data Engineer, preferably in the financial industry
• Proficiency in PySpark and familiarity with assembling and analyzing big data sets from disparate sources applying quantitative methodologies.
• Demonstrated ability in building production-ready data pipelines.
• Experience working in agile software team, good knowledge in software life cycle management including version control (GIT)
• Knowledge of reporting tools (e.g. Tableau, Power BI) and experience collaborating with Front-End developers towards the delivery of products displaying visual analytics is an asset.
• Excellent written and verbal communication skills with proficiency in English
• Dedication to fostering an inclusive culture and value diverse perspectives
Nice-to-Have Skills:
- Experience of working in the banking industry
- Business Analysis
- Java, SQL
- Basic understanding of machine learning algorithms
- A very professional, diligent and can-do attitude with experience in managing and communicating with stakeholders
- German is an asset.