lang-uk/recruitment-dataset-candidate-profiles-english huggingface dataset:
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate

Architected and built a data lake solution utilizing AWS S3 as a storage layer, and Athena and Python as a powerful query and transformation engine, which enabled you to store any data as it is, eliminating the need for costly and time-consuming data transformations.

Designed and implemented a custom ingestion engine that facilitated a 40% increase in data processing efficiency while enabling seamless data ingestion from various production databases into the data lake. That allowed for more streamlined data processing and captured historical data.

Developed a custom Python package that utilizes AWS services and a public Python package to efficiently serve, clean, and transform data from the data lake, resulting in a 60% reduction in data processing time for various teams.

Implemented data documentation using OpenMetadata and hosted it on AWS Lambda to improve data governance and discoverability. As a result, the number of data-related questions to the data engineering team has decreased by 80%.
I have architected and built a data lake solution using AWS S3 as a storage layer, Athena and Python as a powerful query and transformation engine. I have also designed and implemented a custom ingestion engine, developed a custom Python package, implemented data documentation using OpenMetadata, and deployed a credit scoring API using AWS Lambda and microservices architecture. My current role as a Data Engineer at Kifiya Financial Technologies is to ensure that the data stored in the data lake is fit for purpose and meets the diverse needs of data consumers.

In terms of improvement, I am focusing on developing my skills in Big Data Analytics. Additionally, I am working on improving my communication and collaboration skills, which are essential for effective teamwork and stakeholder engagement.
As a Data Engineer, my ideal expectation is to work on challenging data projects that allow me to apply my technical skills and problem-solving abilities. I hope to collaborate with cross-functional teams to design and implement scalable and reliable data solutions that drive business success. Additionally, I would like to contribute to data governance and quality control measures to ensure the trustworthiness and usability of the data. I aspire to become a subject matter expert in the field of data engineering, stay up-to-date with emerging technologies and industry trends, and provide mentorship and guidance to junior data engineers.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: fluent

Certified Python Developer (PCAP-31-03)
I have around 4 years of work experience in  Banking, Telecom, Financial Trading, and Investment companies as a Data Engineer/Analyst. 

The main roles:
• Building Data Quality check pipelines and dashboards
• Design and Implementation of data pipelines
• Development and maintenance of database
• Design, development, and maintenance of ETL workflows
• Big data analysis and insight-driven visualization


The main Tech Stacks:
• Python
• Rest API
• Spark
• Tableau
• SQL
• AWS

The main working methodology: 
• Agile (scrum)

Education:
• MSc in Computer Science with Data Analytics from The George Washington University

Languages:
• English (Fluent)
• Russian (Intermediate)
Position: Data Engineer
Category: Data Engineer
Experience: 2 years | English: fluent


I have been doing data science projects for banking industry.
Projects include client segmentation, transaction analysis, churn prediction, evaluation of bank products (loans, deposits, etc.), report sending and automation tasks
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: pre


I have been employed at Alfa-Bank (Sense Bank) since December 2020 and continue to work there presently.

Responsibilities in the company:
Development of decision-making systems for banking systems using Spark (pyspark), Kafka, and Hadoop technology.
Responsible for ETL processes (MSSQL, Pyspark, BigQuery) and construction of pipeline stack technologies (Airflow, Google Cloud Dataflow).
Development of dashboards (Django) and other types of data visualization (Libraries Seaborn, Matplotlib, and others...).
Administration of Windows servers with big data and Linux servers.
       Big data optimization. For example, optimizing a table with a size  of     6 billion records with a physical size of more than 1 TB.
Position: Data Engineer
Category: Data Science
Experience: 3 years | English: upper

I have created a scrapy crawler that has successfully collected Amazon Mobile Application data.
Also I have ported complex Azure Data Factory pipeline to spark code, that helped to significantly decrease costs and improved overall processing time.
I have been working accross different projects that include data scraping, ETL, streaming and batch processing within AWS/Azure infrastrcture using python and spark under EMR/Databricks environments.
Competitive salary, nice team and interesting project to work on.
Position: Data Engineer
Category: Python
Experience: 2 years | English: fluent


I have been working as a data engineer for past year. I have worked with Big data which was in Petabytes of data in total. I have developed KPIs for the customers in telecom industry to show how their network is performing compared to other customers. I have developed pipelines and scheduled them in airflow. I have been working on machine learning projects in innovation days in our company. I have also done my master thesis on the topic of 'predicting next critical alarm in radio networks using NLP'. I have used spark, airflow, python, sql and bash so far in daily work day.
Position: Data Engineer
Category: Data Engineer
Experience: 1.5 years | English: upper


I have been working for 1.5 years as Data Engineer and Data Analyst. Azure tools has been massively by my side and especially Azure Data Factory is a part of my daily routine. I can confidently say that I am fluent in T-SQL. For me Power BI is a key to giant Business Problems and DAX is a piece of magic. Analysis Service is able to response as fast as Bugatti.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper

- Migration whole data systems to Azure cloud. Many years company used old technologies such as Kalido, SSIS jobs and etc. Company top 1 taxpayer in Kazakhstan.

- 3D visualization of oil fileds using Unity3D, Revit and C#. We made a pilot project for company with revenue more than billion$.
I have been working with data processing and analysis for 4 years. Worked in several areas of IT. Found myself in the position of Data Engineer. Approximately 2 years experience in this position. I was engaged in data migration from local database (Kalido) to cloud (Azure). Gathered data from multiple sources and produced reports for clients. Performed data analysis and reports. Worked with Azure Cloud.
Position: Data Engineer
Category: Data Engineer
Experience: 1 years | English: upper

Finalist in hackathon "Hucuthon", master of sports
I have been working with Data Science Platform in " ELEKS" for more than 1 year with such tasks as:
- Optimised the platform.
- Wrote unit tests and CI/CD jobs.
- Added logging Tools and technologies used: Python, SQL, Kubernetes, Docker, Azure, ML
I think all new knowledge and experience is exiting. So I'm open for various variants and especially interested in Data Science and Data Engineering.
Position: Data Engineer
Category: Data Engineer
Experience: 2 years | English: upper

Successfully designed and implemented a real-time data streaming platform using Kafka, which enabled the efficient processing and storage of large volumes of data. This platform could have helped the organization to obtain real-time insights into business operations and improve decision-making.

Improved the performance and scalability of existing data processing pipelines by optimizing the code and enhancing the infrastructure. This could have resulted in significant cost savings and improved the efficiency of the organization's data processing capabilities.

Developed a custom data visualization tool that helped the business to visualize complex data sets in a more intuitive and accessible way. This could have enabled the business to quickly identify trends and patterns in the data and make more informed decisions.
I have completed various projects and tasks using a range of technologies, including Kafka, Microsoft Azure, Python, MySQL, and Docker. My current role in the team involves creating and managing data pipelines that process large volumes of data to extract meaningful insights for business decision-making.

Some of the notable projects that I have worked on include building a real-time data streaming platform using Kafka and integrating it with Microsoft Azure cloud services to enable efficient data processing and storage. I have also developed data processing pipelines using Python and SQL to extract data from various sources, transform it into a structured format, and load it into data warehouses for analysis.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: fluent


I have developed backend applications using FastApi and PostgreSQL. Developed large scale web scraping applications. Improved performance of existing ETL pipelines and developed new ones. I have deployed and maintained services on Kubernetes cluster using Helm software. I usually do data modeling and data ingestion.
Position: Data Engineer
Category: Data Engineer
Experience: 1.5 years | English: fluent


I have excelled as a data engineer for over a year, leveraging my expertise to build cutting-edge automatic data pipelines and streamline legacy systems through the power of Apache Airflow and Apache Nifi. My role also encompasses efficiently managing data warehouse structures, ensuring seamless integration and optimal performance. Before gaining professional experience, I was a self-motivated developer, dedicatedly acquiring knowledge in ML and Data Science.
In my pursuit of a fulfilling career as a data engineer, I am driven by a profound passion for innovation and cutting-edge technologies such as Big Data and Cloud. My expectation is to be part of an organization that embraces these advancements, enabling me to leverage the latest tools and platforms to push boundaries and drive meaningful impact.Furthermore, I place great value on mentorship and learning opportunities. I believe that continuous growth is crucial in a fast-evolving field like data engineering. Therefore, I am excited about the prospect of being part of a team that fosters a culture of mentorship, where experienced professionals guide and inspire me to reach new heights. I am eager to learn from their expertise and gain insights that will further enrich my skill set.
Position: Data Engineer
Category: Data Engineer
Experience: 6 years | English: intermediate

Implementing qlikview and nprinting , modeling to one the big insurance company ;
Architect crm system, develop and implement
I have experience as database administrator,  and as .net developer.
In ms sql used complex query, all type of function, procedure, trigger, linked object etc. And also worked longtime with BI tools like Qlikview for reporting and nprinting for distributing.
For the last three years, I worked on ASP.NET MVC, with some additions like .NET Core, ASP.NET Web Api and Angular. Also for the time, I've started to take a part xamarin form mobile developing . I am also expert in Blazor. 
Almost all projects, where I took part, included close communication with the client regarding feature analyzing, possible solutions, and difficulties that can occur. I am currently team lead of one insurance company with consist tree part. Analying business wants, developing and testing and then supporting that module.
NET, C#, Entity Framework, ASP.NET, .NET Core, MSSQL, REST API, MS SQL Server, LINQ, Git, OOP, SQL, ASP.NET Web API. Blazor
Not interested in Gaming and mobile development.
Mostly interested in USA company. Have SSN(social security Number)- work permit in USA
Position: Data Engineer
Category: Data Engineer
Experience: 2 years | English: intermediate


I have experience:
• deep understanding of business processes in a wide range of industries (Oil and Gas, Pharmaceutical market, Retail, Logistics, Human Resources, Tourism);
• creating data pipelines as base for solving problems with right data;
• outstanding problem solving, analytical and self-management skills, ability to work in a team as well as independently;
• strong understanding of Agile methodologies (Scrum and Kanban).

Python 3:
- Creating ETL pipelines with Prefect workflow management system;
- Pandas, Dask: dataframes manipulating, load data from/to csv, json, sql; pivot and  unpivot tables;
- Matplotlib and Plotly: analyzing data with different types of charts.
- SQLAlchemy (for PostgreSQL), SQLite3: select, update, create tables and views;
- RegEx: applying regex for find patterns and clearing raw data with drugs names.
- Atoti: used multi-dimensional analysis with OLAP cubes and created dashboards.

GIT Bash: creating and maintaining git-repositories for all my ETL projects.

Jupyter Lab (Notebooks): writing draft versions of ETL pipelines. 

DBeaver: DDL, DML, DQL.

CI/CD: Jira, GitHub.

PowerBI: creating dashboards.

Microsoft SSMS 18, T-SQL:
- studied SQL Query, DDL/DML/DQL;
- creating and filling the tables;
- creating views;
- constructing triggers and creating store procedures;
- working with XML, JSON ans CSV data-types;
- got practice with SQL partitions in database.
Introduction to Business Intelligence and Data Warehouse:
- Created Date Dimension table (Python, SQL).

SSIS:
- studied ETL Processing with SSIS;
- created solution with few packages for ETL data from/to database, flat files, Excel file.
Studied basic course about Shell, Spark, Hadoop and Hive intro:
- developed few scripts for simple ETL pipeline;
- practiced using Databricks Platform.
Key facts about me:
• have solid math and computer science background;
• take motivational, performance driven attitude;
• focus on creating state-of-the-art ways to connect businesses with customers, easily and with outstanding results;
• keep up-to-date with the latest trends in computer science;
• constantly learn how to use advanced technologies at their best.

I want to work in the team, where my computer skill, analytical and presentation skills will be useful and that my knowledge and experience in the enterprise projects will bring a great benefit to the company.
Position: Data Engineer
Category: Data Engineer
Experience: 2 years | English: upper


I have experience in fin-tech as Data Engineer in one of the biggest bank systems in the world. There i worked with such tools as Datastage also with Cloudera, Hive, PL/SQL; It was exciting to develop and troubleshoot ETL in IBM Datastage where i got experience about DWH Design. 
Also i worked more than 1,5 years for American product company in healthcare. It was large and complex analytics system for many clients.
There i took part in developing internal data platform, 
ETL processes and CI/CD pipelines for different purposes.
Those tasks allowed me to learn and practice with following technologies:
ETL orchestrators: AWS Glue, Apache Airflow
AWS Services: Athena, S3, EventBridge, Lambda, Secrets Manager
Snowflake
Jenkins
Groovy
Python 
Git
Position: Data Engineer
Category: Data Engineer
Experience: 1 years | English: fluent


I have got a hands-on experience in using Spark, Spark Streaming, Kafka, Hadoop, Airflow and Hive, ELK stack. I used GCP Dataproc for running services, mentioned above. Also I am experienced in using databases, developing relational database design, working with data warehouses, ETL and BI tools and data analytics. I have worked with GCP BigQuery to aggregate, transform and extract data, and also used Power BI to visualize needed information according to business demand.
I would like to gain more practice in Spark Structured Streaming, working with big amounts of streaming data, ETL/ELT processes, Data Warehouses and Lakes, Cloud Services, Python, and technologies in Data Engineering and Big Data stack. Also, I am keen on working as a part of a team, and seeking opportunities for continuous learning.
Position: Data Engineer
Category: Data Engineer
Experience: 1.5 years | English: upper


I have more than 3 years of experience in the IT industry. Currently employed as a Data Engineer.

In my day-to-day operation I create and maintain data pipelines. I have the most experience with PostgreSQL and Neo4j databases, but also work with MongoDB and Elasticsearch (not a database, but still).

I am looking for a more demanding and interesting position, because there are no growth opportunities at my current company. I am particularly interested in positions involving streaming data (i.e. Apache Kafka) and/or distributed computing (i.e. Spark).
Position: Data Engineer
Category: Data Engineer
Experience: 5 years | English: fluent


I have more than 4 years of experience in big coportations on Data Engineering field. Solid knowledge in SQL and Python. I have worked on cloud infrastructures such as Amazon AWS (Redshift, Glue, S3, EC2, IAM, Matillion, Lambda), IBM Cloud (COS, DB2, IBM Infosphere). Working with great engineering teams in my previous workplaces like HBO MAX, IBM - I have experience with data processing and scheduling tools like Apache Spark, Apache Airflow, Kafka, Firehose, Cron Jobs. Familiar with database/storage solution like Snowflake, PostgreSQL, DynamoDb and MySQL. Created complex ETL pipelines (both batch and streaming) in above-mentioned tools, also with Python and PySpark using Databricks. For cloud infra, I used terraform, GitHub CI/CD, Kubernetes, Docker. Created dashboards both from scratch and templates in Power BI and Tableau. I have experience in modelling and creating DWH. Mentored a small group of data engineer trainees. I am also experienced on PII data management including GDPR regulations. Please refer to my resume for further details.
Position: Data Engineer
Category: Data Engineer
Experience: 7 years | English: upper

SnowPro Core Certification
I have more than 8 years of working with data and data solutions.
Deep understanding of building DWH, ETL/ELT, and data migration in general.
SQL Master, Python Enjoyer
Database - Oracle(Exadata); Postgresql, Snowflake
ETL - Informatica, Pentaho data integration, Matillion
Database Concepts - architecture, objects, statistics, parallel execution
Cloud - AWS 
Business Intelligence - Tableau, SAP BI (Web Intelligence, Universe designer); SAP Dashboards (Information Design Tool, Xcelsius)
Personal information - punctual, responsible, learning on the fly
Position: Data Engineer
Category: Data Engineer
Experience: 1.5 years | English: fluent

I am interested about data Engineering,passoniate about data
I have one year and half experience in area of Artificial Intelligence and now shaping my skills in area of data Engineering ,Certifiying myself with IBM professional data Engineering,I am good at python,data visualization and data preprocessing,ETL and many more
I read articles related to thing i come across,google for it and try to do all my best ,if it doesn;t work ,I ask senior person in that areas
Position: Data Engineer
Category: Data Engineer
Experience: 2.5 years | English: upper


I have over 1.5 year of experience as data engineer. During all that time I participated in long-term project is for building data analytics platform. Lately I've been the main data engineer in the team and was taking on ownership of whole work streams.

Worked on development and maintenance of ETL pipelines to ingest data from various sources (such as structured tables or files) using Airflow, Python and SQL.

Worked on Data Warehouse design, its layers (bronze, silver, gold) and logical structure.

Developed data preparation pipelines for Data Science team and also had an experience of deploying ML models.

Strongly involved in interaction with stakeholders (both technical and business ones).

Actively participated in discussions about domain-specific business challenges such as requesting new data sources, accepting and evaluating new metrics and formulas, investigating and offering solutions to achieve set goals and meet business needs.

Partly involved in migration to a new cloud provider and infrastructure upgrading using Spark and Databricks.

Worked mostly with GCP (Google Cloud Platform) cloud but willing to work with another ones as well.

I also constantly improve my skills by learning best practices and new technologies.
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate


I have over 4 years of experience working as a Python Developer / Data Engineer. During my time at a US based startup, I was involved in the development of data pipelines, which included designing, building, and maintaining systems to process and analyze large amounts of data.

I value honesty and transparency in my interactions with others and strive to build relationships based on trust. As such, I have developed strong communication and collaboration skills, which have enabled me to work effectively as part of Scrum teams and to mentor and interview new team members.
Position: Data Engineer
Category: Data Engineer
Experience: 7 years | English: upper


I have over 7+ years of IT experience in Analysis, Design, Modeling, Development, Testing and Maintenance of Data Warehouse Business Applications. I have worked in different domains like Finance, Commerce, Legal and Retail. 
I have extensive experience in creating PL/SQL Packages, Procedures, Functions, Cursors, Event Handlers, and Database Triggers. I have excellent SQL skills, written complex queries and correlated sub queries. 
In all my projects, I have created database objects like Tables, Views, Sequences and Synonyms. I am proficient in logical and physical Data Modeling. In my recent project with Georgia Pacific I was involved in both physical and logical Data Modeling, Forward and Reverse Engineering. In all my projects, I was involved in Performance Tuning, Partitioning tables, Optimizing Indexes and Queries. I have worked on both UNIX and Windows environments. I am proficient in migrating data and worked with SQL Loader and DTS packages. I have experience in UNIX Shell scripting.  I have an excellent understanding of Iterative, waterfall and Agile Methodologies and was involved in all stages of Software Development Life cycles during my projects.
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate


I have solid experience as Data Engineer in building real-time data processing pipelines. I am skilled at solving complex tasks and I am interested in new challenges.

My main focus on current project is:
- Development of Java applications for real-time data processing and transformation
- Improvement of data quality and accuracy
- Creation and configuration of the necessary infrastructure
- Maintenance of platform
- Communication with stakeholders

I am experienced in full-cycle development - from discerning the customer's needs to delivering a final product.

Most used technologies and services:
- development: Java 11, Kafka Streams, Kafka, SQL
- integrations: Kafka Connect, custom self-developed applications
- deployment: Docker, Gitlab CI, Kubernetes
- monitoring: Grafana, Sentry
- cloud: AWS S3, AWS Athena, AWS MSK, AWS Secrets Manager
Interesting projects and professional growth