lang-uk/recruitment-dataset-candidate-profiles-english huggingface dataset:
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: fluent

- Redesigned flawed cross-cloud ETL using SOLID principals and software architecture patterns. Provided business intelligence and data awareness to client by designing informative dashboards.

- Designed and implemented multiple complex ETL systems that included data enrichment from several sources (DB, REST and scripted lookups), data cleaning and validation. I visualized data in Kibana in different ways: histograms, pie charts and time series.

- Researched several papers and implemented CV mathematical models for image transformation.

- Participated at each stage of ML solution integration from data collection with scrapers to training and deployment.
- I have solid experience in python development and data pipeline creation. I worked on cloud and software architecture.

- I am also skilled in SQL and worked with some DBs like PostgreSQL and ElasticSearch. 

- I have experience in data visualization. 

- I am comfortable at working with data analysis tools like: dask, pandas, numpy, etc.
- I am looking for data-centric project that will have tasks in: BigData tools like PySpark, data modelling, SQL, NoSQL and DWH.
Position: Data Engineer
Category: Data Engineer
Experience: 1.5 years | English: fluent


I have started my professional work experience as a Data Engineering Intern at a local Bank. During my 6 monthes there I worked on banking data. My responsibilities were to gather this data from different branches of the bank, clean it, restructure it and save it in Spark to make processing easier for Machine Learning Algorithms.
After my intern period, I was hired by a US-based company on a contract to carry out web scraping and automation projects related to financial data. I set up ETL pipelines with AWS lambda and Overwatch to set the process to be carried out automatically. 
My final experience was at Robert Bosch Kft. as an academic researcher. I was tasked with Computer Vision project related to autonomous driving. I used PyTorch to accomplish task.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper

- created a desktop application from scratch with a variety of automation scenarios for non-tech users to reduce everyday manual work from a few hours to 10 minutes.
- built an ETL solution with 10+ various data sources, which runs on an everyday basis. Further BI solution is built on top of that.
I have various experience in the data engineering field:

- Implementation of ETL workflows

- Collecting and parsing data from different sources (DB, .csv and .excel files, API endpoints)

- Data Quality testing and Data preprocessing

- Automation of reporting and data processing routine

- Creating a desktop application from scratch with a variety of automation scenarios for non-tech users to reduce everyday manual work.

Besides tech skills, I have quite good communication skills. So I'm absolutely comfortable if a clarification call with a client is needed. I always try to understand what is the background of a problem I'm solving. It helps to find a variety of possible solutions and choose an optimal one. I like meetings with teammates for some research and thoughts sharing purposes.
Would be nice to work with:
- AWS
- Snowflake
- Building Data platform from early stages (development ETL, building Data lakes/warehouses, etc.) to Data visualization or building ML models on top of that in the future. 

Definitely don't want to work with:
- gambling
- dating
- trading
- focus on the post-soviet market (any russian companies, partners, or collaboration on work purposes)
Position: Data Engineer
Category: Data Engineer
Experience: 0.5 years | English: fluent

Studied in an international school in Hong Kong(Ontario curriculum)
Studied in the U.S.
Opened a consulting business as a hobby, serving clients with interest in tourism, relocation, e-sports.
I have worked as a Data Engineering Intern at Ignatica, a Hong Kong based insurtech startup. I was responsible for data engineeringing tasks such as ETL in Spark on top of Azure, creating SQL views, writing cronjobs and various bash code to supplement Spark streaming with internal batch jobs. 

Moreover, I was tasked with setting up Hyperledger Sawtooth blockchain infrastructure and achieving node replication on Azure cloud. Automated spin-up with Terraform, visualised with Grafana.

Technologies Used: SQL/Mongo/Hive, PySpark/Scala, Airflow/Databricks, Azure, Terraform, Grafana, Blockchain

During my time at a university in the U.S. I was a part of LIGO Scientific Collaboration as a Quantitative Student Researcher, working on gravitational wave glitches and categorising them to help scientists find unique signals that correspond to important cosmic events, such as black hole mergers. The technology, and job requirements varied during my 2 years as a student researcher, but it was mostly Data Science and Data Engineering

All of my skills were learned independently, or during the job I was assigned to do, which gives me confidence to quickly pick up the skills unique to your project.
As a first, entry-level job, my expectations are to learn on top of what I already know. I am confident that I can get up to speed with what your company is already doing in a short amount of time and swiftly pick-up new tech skills if the need arises.
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate


I have worked with relational databases and with Big Data technologies on Amazon Web Services. I prepared data for future analysis, developed management decision reports, coordinated tasks with customers, looked for ways to improve the process of calculating indicators.

Professional Experience

Project Description:	The company is one of the biggest players in transforming lives through innovative medical solutions that improve the health of patients around the world.
Role:	Data Engineer
Duration:	Nov 2020 – Apr 2022

Responsibilities:	
- implementing business logic and calculating indicators in AWS Athena using Presto SQL
- creating the ETL processes, data cleaning using the AWS Glue Spark jobs
- moving changes to other environments using AWS Cloud Formation
- developing dashboards and visualizations in Tableau
- conducting Data blending, data preparation using SQL for Tableau consumption, and publishing data sources to Tableau server
- developing and implementing Row-level security in Tableau
- communicating with the customer on a daily basis to clarify priorities and technical requirements.


Project Description:	Development of an application for creating a telecast schedule on TV-channels depending on historical data, views, and prediction of views based on machine learning. Implementation of processes of loading historical and current data on a daily basis.
Role:	DWH engineer
Duration:	Mar 2020 - Oct 2020


Project Description:	Managing and supporting an information and reporting systems of the Bank: preparing the data for future analysis, creating the processes for cleaning data, grouping data due to business rules, the formation of internal reporting.
Role:	Data administration and analysis specialist
Duration:	Sep 2017 - Mar 2020
Looking for an interesting project where I can continue growing as a Data engineer on Cloud services. It would be great to work with data processing and with data visualization as well.
Position: Data Engineer
Category: Data Engineer
Experience: 5 years | English: upper


I have work in the 3shape company since 2018 year on the position Data Engineer / Business Intelligence Enginee.My responsibility  is create ETL pipeline and build  DWH model on the different platforms. I have good communication skills not only in my team and also with other departments. I like read a lot of articles about new technologies in the world. I like sport and my favorite one is football
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper


I know languages: Scala, Java and basic Python.
Have a deep knowledge of Spark(Core/Streaming).  In different projects I was involved in creating a big service for a Big Data project from the scratch, maintain existing ones, I've done a lot of investigation related to technologies in order to choose the best for requirements for a project. Was involved in many discussions regarding a project. My responsibilities were: implementation of program code, creating pipelines, analysis of data, creating documentations, unit-tests, configuration docker, finding and fixing bugs in a code, working with some databases, team supporting.
I would like to be involved in the interesting Big Data projects and participating in the different kind of tasks. :)
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: fluent

- Creation of data pipelines/infrastructure that powered web and mobile reports across the departments from scratch.
- ETL, report automation in Python/SQL in the cloud from scratch. The clients included a multimillion-user e-commerce B2C web app, B2B wholesale platform, fintech companies, food tech.
- Creation of Inbound Marketing from scratch with a stable income of ~10 B2B leads / month.
I like moving data from one place to another. Use all the cool tools while doing it too.
Lots of experience with batch processing and 3rd party tools. Worked across the clouds, so I'm familiar with most of the popular tools. Learn new things really fast; all I need is a bit of trust in me taking responsibility.
- Looking for streaming/Spark, or at least an opportunity for one of those. Okay with working alone, managing architecture and writing code.
- Being a one-man army, in terms of managing projects and also coding, is not ok =/.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: intermediate


I'm a Database Developer with more than 4 years of MS SQL 
Server(Azure SQL), have strong experience in T-SQL, solid analytical skills based on Math background. I like being a team member, a good team inspires me
Work with databases
I'm not an Administrator (DBA) or Power BI developer
Coding time more than 50% of work time
Good team
Complex tasks
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: fluent


I’m a Data Engineer with 4.5 years of production experience in data engineering and machine learning. I also have experience teaching Python programming and Data Science in a non‐profit academy. My career goal is to build expertise in the whole process of data‐intensive applications development.
Position: Data Engineer
Category: Data Engineer
Experience: 6 years | English: intermediate


I'm a Data Engineer with a total of 6 years of
experience in software development in the banking domain and 
2 years of experience in Data Engineering.
Technical stack: 
- Apache Spark, Pyspark, Apache Nifi, Airflow, Kafka
- Python (Flask), Scala (Akka), Java
- Kubernetes, Docker, Hadoop
- MongoDB, PostgreSQL, BigQuery, Minio

Last workplace responsibilities: 
- As part of a team of 4 data engineers, worked to provide data to 2 data scientists.
- Developed ETL pipelines using Apache Nifi, and Apache Spark, and orchestrated them using Apache Airflow. Thanks to this, data scientists quickly received data for further analytics and model building, reducing the time to provide results to the business
- Automated the deployment of ETL pipelines using GitLab CI/CD, which made it possible to quickly fix errors and deploy new ETL pipelines.
- Built a GPU model training environment for a chatbot development project using Gitlab CI/CD and Docker and then Kubernetes. This allowed data scientists to quickly deploy changes to the chatbot model.
- Integrated the chatbot with internal banking systems, and developed plug-ins for it in Python. Also engaged in debugging various issues, working with colleagues from other departments. Thus, I identified critical errors in other bank systems that affected the work of the chatbot.
I am looking for a Data Engineer or a Backend Engineer position, I want to work with Apache Spark, Python, Scala, or Java, both on-premise and clouds. I don't want to work with reporting.
Position: Data Engineer
Category: Data Engineer
Experience: 2 years | English: fluent

• Databricks certified data engineer associate;
• Microsoft certified data engineer associate;
• English - C1;
• Exceptionally great feedback from the past projects;
I'm a Microsoft and Databricks certified data engineer passionate about delivering efficient and innovative data solutions. I have hands-on production experience working with Python, SQL, Spark, and Docker, and proficient in utilizing Cloud Platform services, such as Databricks, Google BigQuery, Google Composer, Google Dataproc, Azure DevOps, etc. I enjoy designing and building data pipelines and systems that deliver valuable insights. As a dedicated problem solver, I thrive in collaborative environments and am skilled in working with teams to develop effective data-driven solutions. In terms of my interpersonal skills, I would consider teamwork, discipline, willingness to help, and stress resistance my main strengths.

The current projects domain - Retail & Distribution, implies me following responsibilities:
• Contribute to the project by collaborating with a team using the Scrum framework and Jira dashboard;
• Gather requirements from customers;
• Support the architecture development and configuration-driven generic ELT/ETL framework. Develop Delta extension features;
• Model data sources and layers for efficient storage and retrieval;
• Develop an orchestration system for sources with inner and outer dependencies using Google Composer;
• Cover code with thorough unit and integration tests to ensure high quality;
• Document knowledge on the Confluence.

From the most challenging tasks that I have faced, I can point out 3 of the most interesting:
• Development of release reprocessing dag, that uses graph structures to set complex dependencies between multiple sources;
• Creation of data layer logic in multihop architecture;
• Development of an advanced multisource filtering engine that outputs data to SCD2.

As a data engineer, my desirable project would be one that challenges me to leverage my technical skills and creativity to develop robust and scalable data infrastructure solutions. Ultimately, my ideal project would allow me to work collaboratively with a team of skilled professionals and make a meaningful impact on the organization's data-driven initiatives.
Position: Data Engineer
Category: Data Engineer
Experience: 1 years | English: intermediate

Optimization of data processing
Improving the performance of the ETL process
Creating dashboards to track performance
Optimization of SQL queries
I'm an ambitious Data Engineer with experience indeveloping and optimizing data processing processes. Istrive to create effective systems for collecting, storing andanalyzing large amounts of data in order to providebusiness with valuable information for decision making
It is important for me to avoid such jobs where there is no opportunity to use advanced technologies and develop new solutions to optimize data processing. My goal is to develop and apply best practices in the field of data processing and analysis, and I am looking for opportunities where this goal will be achieved.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper


I'm an experienced Data Engineer skilled in data architecture, database management, and ETL processes. My expertise includes designing data pipelines, warehouses, and models. Over my career, I've proven adept at tackling data challenges and maintaining data integrity.

Currently, I'm pursuing a Data Engineer role to leverage my technical skills, build strong data solutions, and grow in the field. With a reputation for precision, a strong work ethic, and reliability, I'm confident in consistently exceeding expectations.
Position: Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper

Data Engineer
I’m a team member with experience in building and implementing data platform architectures for the biggest companies in the world. I worked on implementation of ETL pipelines, modeling of data warehouses and used to DevOps techniques closely.
I’d like to work on project where is modern data stack and there is option to build architecture from scratch.
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate


I`m currently a data engineer with 3+ year experience, as following:
Designed and built Data Lake and Delta Lake on AWS S3. 
Migrated data from MSSQL Server to Amazon S3 using AWS DMS service. 
Designed and created batch and stream processing data pipelines using various AWS services like Lambda, Glue/Glue Streaming, DMS, Kinesis Data Stream, S3, Redshift, SQS and Step Function.

Skills: SQL, Python, AWS, Airflow, Spark, ETL, Kinesis
Position: Data Engineer
Category: Data Engineer
Experience: 2 years | English: upper


I'm data analyst with strong engineering background in data. I'd like to growth like data engineer.

On the last position I've been building analytics in company from scratch:
- data infrastructure set up (remote server with PostgreSQL RDBMS; GCP services like Compute Engine VM's, Cloud Storage, BigQuery; Apache AirFlow; GitLab)
- System analysis and data modeling
- Building ETL pipelines with python and orchestrator like Airflow
- writing test and data quality implementing 
- Developing dashboards on Tableau 
- Ad-hoc reports
- Documentation

Also, I took data engineering course from Karpov Courses where I got more theory and hands-on experience in this field, and some of them I applied in company
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: upper


I'm Data Enginner with 3 years of experience in bulding and maintaining ETL pipelines.
My key resposibilities are:
- Ingesting data sources in different file formats (JSON, Parquet, CSV, Delta) and perform data -- - Inspection and analyzing using Athena, Zeppelin notebook
- Engineering and developing storing data in Data Lake using S3, Delta, Athena.
- Developing and maintaining data pipelines using Spark and Scala. 
- Automating ETL processing on AWS Glue platform with Step Functions, Lambda
Position: Data Engineer
Category: Data Analyst
Experience: 3 years | English: intermediate


I'm experienced Middle Data Engineer in Teamwork Commerce. My job is building and extending data base architecture using:

- Apache Airflow runned on Google Cloud Platform's Composer for building ETL pipelines.

- DBT to keep and deploy all architecture in one place.

- Google Cloud Platform's BigQuery as a warehouse for the data. 

- Looker to manage and visualize the data.
Position: Data Engineer
Category: Data Engineer
Experience: 5 years | English: upper


Implementation of ETL processes and DWH from scratch in a big logistics company.
Core bank system implementation and migration in a few large banks in Ukraine and Azerbaijan as a technical business analyst.
Development and implementation of a new B2C application for an insurance company in OAE as a system analyst.
Position: Data Engineer
Category: Data Engineer
Experience: 2.5 years | English: upper

the analyst team of my company faced the problem of verification and analysis of large text files data. I implemented a solution that saved $61,000 in annual costs in analyst team work-hours
in general, my work as ETL Developer consists in moving data from the source to the destination, which can be C3, FTP, SFTP, DB. During this process, the input data is validated and, if necessary, transformed.
I am comfortable working on monotonous jobs, on multi-tasking and diverse tasks, individually or in close cooperation with other team members. However, I would not like to be a pioneer in finding and implementing a completely new development environment.
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: upper


In my previous role as a Data Engineer, I was responsible for designing and implementing data pipelines and data warehousing solutions. I have experience working with a variety of data technologies, including SQL, Kafka, and Spark. Additionally, I am skilled in programming languages such as Python and Java, and have experience using cloud-based data platforms such as AWS, some GCP. I am passionate about using data to drive insights and improve business performance. I enjoy working in a fast-paced environment, and am comfortable managing large datasets and complex data structures.
Position: Data Engineer
Category: Data Engineer
Experience: 3 years | English: fluent

Optimizing ETL Processes: I led an initiative at Pasha Insurance to optimize ETL processes, resulting in a 30% reduction in data processing time. By fine-tuning SQL queries and introducing parallel processing techniques, I significantly enhanced the efficiency of data ingestion and transformation.

Data Pipeline Scalability: At Agile Solutions, I played a pivotal role in designing a scalable data pipeline architecture for a high-velocity data stream. This architecture allowed us to handle a 50% increase in data volume while maintaining performance, ensuring uninterrupted data flow for critical business operations.

Cross-Functional Collaboration: As part of the IFRS 17 project, I collaborated closely with actuaries and financial analysts to ensure the accurate representation of insurance contracts in our data systems. This collaboration resulted in compliance with regulatory standards and provided valuable insights for financial reporting.

Technical Mentorship: I've actively mentored junior team members, sharing best practices in data engineering and ETL development. Several team members have progressed in their careers and contributed effectively to our projects, a testament to the impact of mentorship.
In my role as an Data Engineer at Pasha Insurance, I have gained extensive experience in data management and ETL (Extract, Transform, Load) processes. My responsibilities encompass:

Writing complex T-SQL queries, including recursive CTEs, temp tables, dynamic queries, subqueries, and intricate joins for creating Stored Procedures, User-defined Functions, Views, and Cursors.
Implementing robust error and event handling mechanisms, such as precedence constraints, breakpoints, checkpoints, and logging.
Importing/exporting data from various sources (Python files, flat files, Excel) using SSIS/DTS utility.
Utilizing C# within the SSIS script component.
Developing dynamic SSIS packages.
Defining data warehouse structures, including star and snowflake schemas, fact tables, cubes, and dimensions.
Resolving technical and data-related challenges in the IFRS 17 project.
I am an integral part of the team, contributing to the technical aspects of the project and collaborating with colleagues to ensure its success. I continually strive to enhance my skills and aim to improve my expertise in ETL processes, data warehousing, and database management.
Seeking a challenging data engineering role with a focus on complex data transformations, ETL development, and database design, including cloud-based solutions. I value opportunities that allow leadership in designing data warehouses and thrive in dynamic, learning-oriented environments. Prefer roles that minimize administrative tasks to focus on technical contributions.