lang-uk/recruitment-dataset-candidate-profiles-english huggingface dataset:
Position: Big Data Engineer
Category: Data Engineer
Experience: 8 years | English: upper


In my current role as a data engineer, I've been involved in a variety of projects and tasks that revolve around data processing and pipeline development.

I've also been responsible for designing and maintaining ETL (Extract, Transform, Load) pipelines, primarily using Python and Apache Spark, to process large volumes of data from various sources and load it into our data warehouse. This has involved optimizing the performance of Spark jobs and ensuring data quality and consistency.
In my new data engineering role, I have several expectations:

1. Challenging projects.
2. Cutting-edge technologies.
3. Collaborative environment.
4. Professional growth.
5. Clear goals and expectations.

Overall, I'm excited to bring my data engineering skills and enthusiasm to my new role, and I look forward to contributing to the success of the organization while advancing my own career in the field of data engineering.
Position: Big Data Engineer
Category: Data Engineer
Experience: 2 years | English: upper

In current position I worked on project called Notification. In that project We find interests of the users by their activities and used that interest for advertising. And another thing that I am proud of is that during simple analysis of logs I found that one of the clients have interest on other our product, after that our product manager prepare additional offer to that client, and our company made extra money for that quarter.
**Jan 2022 - present Epam Data Analyst**
-Analyzing on SQL logs
- Monitoring tables
-Refactoring and engineering data based on cloud platform called palantir platform (Swiss Reinsurance Company)
-Analyzing and engineering data based on cloud platforms like snowflake, AWS and Teradata for Data Analytics and Engineering team (Nike)

**Nov 2020 - Jan 2022   BigData Analyst in telecom company. 
**
Projects:
- Fraud detection algorithm
- Business Notification
- Client support
- Verification/Scoring

**Nov 2017 - Nov 2020 Self Employed 
**
**June 2016 - Aug 2017 Teacher of programming and robotics classes**
In new position I am expecting interesting analytical tasks, like 
- engineering cloud platforms
- analyzing root sources and table structure
Position: Big Data Engineer
Category: Python
Experience: 4 years | English: upper

- Optimized Spark pipelines that led to x3 time reduction.
- Improved data ingestion logic that allowed the user to reload data hourly instead of daily.
- Refactored architecture of CDC pipelines that allowed schema evolution.
- Led the migration from NiFi to Airflow that reduced data indigestion infra costs by x15 times.
- Led the migration from NiFi to Airflow that reduced data indigestion infra costs by x15 times.
- Silver and Gold layer processing with 700+ TB of data.
- Dropshipping Framework for top worldwide markets (Amazon, eBay, Fnac, Rakuten)
Product, complex problems, modern technologies
Position: Big Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper

Last achievements I’m proud of:
- designed and drove a migration of large business-critical system
- passed GCP Professional Data Engineer certification
Most important projects:
1. Telecom - contributed into in-memory data processing system: Java, Kafka, Ignite
2. Sports - developed data warehousing system based on cassandra, spark streaming(scala) and kafka
3. Financial technologies - developed ETL pipelines for data marts based on GCP PubSub, Dataflow and BigQuery

Current responsibilities:
Driving migration of large system from MR stack to Spark from design doc to working prototype. Stack: scala, spark, hive
Perfect opportunity:
It would be perfect to find an interesting big data project with opportunities to grow in Data Science area.
Position: Big Data Engineer
Category: Data Science
Experience: 11 years | English: upper

A lot of successful projects, up to production deployment, using most powerful development practices and Agile methodology.
Programming languages: Python, Java, SQL
Relational databases: Oracle, MSSQL, MySQL, PostgreSQL, Sybase, Teradata, SAP HANA, Greenplum
Big data/NoSQL: Hadoop, Hive, HBase, MongoDB, Redshift
Cloud: AWS(EC2, RDS, S3)
ETL: Apache Airflow, Informatica
Version control tools: Git, GitHub, Bitbucket

Responsibility areas: Scope defining, requirements definition, database design, data analysis, ETL pipelines, data migration, data integration, DWH design
I'm Interesting to develop complex products using modern cloud technologies, AWS is preferable. It would be great to work within the team of talented developers with high tech/soft skills level.
Position: Big Data Engineer
Category: Data Science
Experience: 7 years | English: upper

7+ years experience in software and data development on various projects
Hands on infrastructure and development experience in building distributed and scalable solutions
Steaming and batching processing data
Data orchestration experience using Airflow
Kubernetes Spark and Airflow operators
Expert in Python
Good experience in Scala
Experience with building ETL
Experience in Data Science (Image recognition using OpenCV)
DevOps practical experience (Ansible, K8s)
Participant in opensource projects (Airflow Spark Operator on top of K8s)
- Python(6+ years)
- Scala(1+ year)
- Spark, Hadoop, HDFS, Hive, Presto, Airflow
- Kafka/RabbitMQ
- NoSQL
- ELK
- RDBMS
- BI(Pentaho, Jasper)
- Docker, Docker Swarm, Kubernetes, Ansible
- AWS/GCP
take a job on project to build scalable platform;
project from scratch is a big plus;

Don't like working with REST API, web frameworks and extracting data using just native libraries of any programing languages.
Position: Big Data Engineer
Category: Python
Experience: 3 years | English: intermediate

Created system for keeping databases up to date:
-scrapping open web sources;
- data cleaning and transforming;
- upload data

Automated flow for data extraction and insights creation:
- extraction from DB;
- data processing with business requirements;
- sending to requester
Retail project:
Creation of ETL pipelines for Data Lake.
Performance improvements for Spark jobs.
Mobile advertising project:
Creation of web scrapping tools, data cleaning and verification, ETL processes.Data processing with decision trees and multiprocessing. Creation of PySpark applications for huge data processing. Producing data insights for more business value.
Position: Big Data Engineer
Category: Data Engineer
Experience: 3 years | English: upper

- Designed architecture for the Big Data project that was successfully deployed and working on production, that helps clients to explore telemetric data and they can see possible ways on how to improve performance of some specific part of work to spend less money and at the same time earn more
- Improved performance of SQL queries execution on 30%
- Implemented a new async service to enrich information about entities that were added to the production
- Implemented a script to transfer specific data from one environment to another to avoid errors that can occur due to constraints or triggers on DB. This allows us to quickly start working with the system in a new environment, with the data that was set up in the old environment
Software Big Data Engineer with 3 years in software development. Proficient in the back-end development of applications based on microservice architecture. Provided performance improvements of both API and Database (PostgreSQL). Worked with documentation like software requirements specifications, tech design, and implementation notes for project features. I'm working with the frameworks and technologies used in Big Data: HDFS, MapReduce, PySpark, Hive, Apache Airflow.
I have experience in designing the architecture for Big Data projects, most of the time I use AWS to deploy the system. 
Currently, I'm team lead for the Big Data engineers team.
Position: Big Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper


Software developer with over 5 years of experience of the design and development of products.
Main experience with Python, PySpark, Azure, AWS, GIT in Telecom, Web, and Financials areas. Involved in more than 9 projects of various scales and size, as a part of a team with 5-10 members.
Experience working according to the Scrum process as a part of XFT (cross functional team) structure.
Additionally, learn new technologies such as GraphQL, Redis and try to always be familiar with newness. About myself: flexible/adaptable, acting as a team member, ability to learn fast, competitive.
Technical skills
 Programming languages: Python, JavaScript, HTML, CSS, PLEX, PLEX-M.
 Frameworks: Django, Flask, Bootstrap, DRF, GraphQL.
 Platforms: Mac OS X, Windows XP, Linux.
 Development tools: Spark, Hive, Hadoop, Airflow, ADF, Celery, Redis, RabbitMQ, SnowFlake,
SnowPark.
 Core libraries: Pandas, NumPy, Sklearn.
 CI/CD: GitLab, GitHub actions
 Databases: MySQL, PostgreSQL, MongoDB, Redis, Redshift, Aurora.
 Issue Tracking: Jira, Azure DevOps.
 Cloud platforms: AWS, Azure
Position: Big Data Engineer
Category: Data Engineer
Experience: 1.5 years | English: upper


The vast majority of my experience in the speciality was gained on university courses, that is the place where mistakes and skills were gained. I have used Python as the main language, worked with relational and NoSQL databases, asynchronous libraries such as aiohttp, asyncpg, aiogram. Also used numpy, pandas, scipy, plotly, Jupyter Notebooks for data analysis, optimization methods studying and visualisation of results. I mostly use Docker for apps and have successfully deployed some of the apps on Heroku. Studied parallel computations (mostly on OpenMP and QT). Participated in programming olympiads. Also, I have a Telegram Gmail Bot as a pet project, which uses Google API and pub/sub notifications to forward emails from Gmail to Telegram chat.
Position: Big Data Engineer
Category: Data Engineer
Experience: 5 years | English: upper

The two startups I've worked for are now unicorns with an overall evaluation of ~10B.
T-Shaped Product Engineer having 5 years of Machine Learning, DataScience, and 12 years of Software Engineering. Took part in a variety of Data Science projects for e-commerce and advertising companies. My work covered both Engineering and Product parts. On the engineering side, I did R&Ds, prepared system design specs with recommendations, and of cause built solutions. On the Product side I worked with stakeholders to formalize requirements and present results.
I'm looking for rather engineering kind of work such as MLOps, Machine Learning, Big Data, then related to the theoretical part of DS. Companies with strong Data Science/Engineering cultures or solid revenue streams are preferred.     
Sorry, I'm not considering any Data Science or Analytics positions any longer
Position: Big Data Engineer
Category: Data Engineer
Experience: 4 years | English: fluent


Various projects at previous clients in Luxembourg includes:
- Creating in-house framework to transition in different curated stage of the company datalake. Reducing license costs and dependency to exterior data provider.Reduced manpower through automatization and deployment of open-source technologies (Apache Trino, Apache Spark, HDFS, MariaDB ...).
-Building a data-vault to ensure data consistency and adapt to GDPR compliance. Design the layout and define keys relationships between entities.
-Ingest massive amount of critical data on a daily basis, responsible for pipeline deployment and maintenance (Apache Airflow) and curation related tasks (Python/Pyspark). Testing deployment on DevOps technologies (Kubernetes, Docker, Kind)
Part-time teaching at the ESGF-Finance as a Big Data Professor for Master 2 Students and at ECE for DevOps modules (Kubernetes, Docker, Ansible, Vagrant...).
Position: Big Data Engineer
Category: Data Engineer
Experience: 1 years | English: fluent

Getting aws certified and surviving 3 hour-long retros)
Worked in a huge team as a developer whose duty was to implement scalable data pipelines, which extract, cleanse, transform various types of data in the data lake on AWS. Besides, created a bunch of API’s for data consumers and other teams to approach different parts of the lake. At the moment, I’m looking forward to be a part of cloud-based project, preferably AWS, but am open to other similar opportunities.
I’d love to work in a nice and friendly team. Ability to grow is a must, having some learning budget (for e.g. for certifications/courses) is a plus.
Position: Big Data Engineer
Category: Data Engineer
Experience: 2.5 years | English: upper

Certified cloud engineer (AWS Data Analytics Specialty and Azure Data Engineer Associate).
Worked in an international team as a key developer whose duty was to implement highly scalable data pipelines, which produce and extract various types of data. Besides, integrated the solution with Azure Data Factory orchestration. Have commercial experience with Hadoop, Kafka and Spark (java and scala). Currently, I’m looking forward to be a part of AWS-based project, but am open to other similar opportunities.
I want to improve my skills in Data Engineering field, especially in the cloud.
Position: BigData Engineer
Category: Scala
Experience: 2.5 years | English: upper

Google Cloud Certified Professiondal Data Engineer
Solid experience with Spark, Hive, Hadoop(HDP), Hbase, Redshift, Firehose, Lambda.
Wrote spark ETL jobs, optimized sql queries, wrote tests.
Position: Big Data Engineer and Architect
Category: .NET
Experience: 0 years | English: 


C#, F#, .NET, ASP.NET, MVC, SQL, Entity, Framework, Web, API, HTML, 5, CSS, JavaScript, Architecture, MongoDB
Ability to grow professionaly
Position: Big Data engineer / architect
Category: Scala
Experience: 11 years | English: fluent


13+ years of software engineering experience

Interested in: Big Data, high load, distributed systems, stream processing.

Primary skills: Spark, Scala, Kafka, Cassandra.
Position: Big data engineering
Category: Data Engineer
Experience: 2 years | English: upper

Languages skills: English - Upper Intermediate (speaking and writing). French – conversational. Ukrainian -
native language.
Technical skills:
 SAS: SAS Management Console, SAS Information Map Studio, SAS Real-Time Decision Manager, SAS
Marketing Automation (SAS MA), SAS DIS (beginner);
 ETL with SAS;
 SQL: Oracle SQL Developer; MS SQL Server(beginner);
 Python(beginner);
 Kannel (SMPP);
 Kafka Console Utilities (beginner);
 Mongo DB (beginner);
 Evam: Evam Designer, Evam Console;
 Linux CMD, Linux Server administration;
 WinScp/Putty;
 Grafana/Graphite;
 Tableau;
 MS Office (Word, Excel, Power Point, Visio, Teams, Access) ;
 JIRA/ Knowledgebase;
 Outlook;
 IBM Lotus;
 Mathlab(beginner);
 Mathcad(beginner);
 Altium designer (beginner);
 SDTM; ADaM (beginner)
Besides that, i have experience completing technical documentation, negotiating with Business or CRM department, Linux and 
database administrators, stakeholders.
Passed training courses:
 freecodecamp.org (HTML courses),
 moonexcel.com.ua (SQL Lessons)
WORK EXPERIENCE
Big Data Engineer
Jan 2021 – Dec 2022
• Administrating of a real-time marketing communications system (more than 50 nodes) based on 
SAS Real-Time Decision Manager & EVAM products.
• Data lifecycle management (processes for obtaining, hosting, storing, distributing, migrating, archiving and
deleting data).
• Solving business problems when planning, testing and conducting marketing campaigns.
• Ensuring correct work with third parties (external or internal systems) - development of methods and
regulations in order to process large volumes of data (data streams) from heterogeneous data sources

Associate Statistical Programmer Intern
Mar 2020 – Jun 2020
• Created data tables and graphs according to the required specification for clinical, pharmacokinetic and
statistical reports.
• Programmed on SAS and related languages (e.g. SQL)to present and analyze clinical trial data.
Position: Big Data Engineer (Python)
Category: Data Engineer
Experience: 9 years | English: upper

Publications:
• Multistage SVR-RBF-Based Model for Heart Rate Prediction of Individuals
• SGD-Based Cascade Scheme for Higher Degrees Wiener Polynomial Approximation of Large Biomedical Datasets

Completed courses:
• Google Cloud Skill Boost:
  - App Deployment, Debugging, and Performance
  - Securing and Integrating Components of your Application
  - Google Cloud Fundamentals: Core Infrastructure
• Data Warehouse and Business Intelligence
• SQL for Data Science
• Using Git for Distributed Development
• Design Patterns

Hackathons:
• BEST Hackathon
• AI Hackathon
Bachelor of Computer Science in Artificial Intelligence with more than two years of experience in Python, including 1.5 years in the Data Engineering field.
I also worked with Clouds and Big Data technologies such as Spark, Kafka, Hadoop, etc. I have completed a wide range of courses in my specialization and keep staying motivated to improve myself constantly.
Can be either a good team player or individual contributor and become an initiative leader if necessary.
Challenging position and opportunity to influence and deep dive into product development.
Potentially interested in Data Science and ML-oriented projects.

Desired stack: Python + GCP/Azure + (Big) Data Engineering

Open to new technologies if the learning environment is available. Intent getting cloud certification.
Position: Big Data Engineer/Python Back end Developer
Category: Python
Experience: 6 years | English: upper

I took an active role in designing and building complex web platforms based on microservice architecture. Also, I build highly intensive streaming data pipelines and complex batch processes. I gained two GCP certifications: GCP Professional Cloud Architect and Data Engineer.
I have experience working as a web backend developer with Python Flask, FastAPI, Sanic, Sqlalchemy PostgreSQL, and others. Currently, I'm working as a BigData Engineer with the following technologies GCP (BigQuery, Dataflow, Pub/Sub, Big table, Composer, GCS), PySpark, Apache Beam, Apache Airflow, and others.
Position: Big Data Engineer (Scala)
Category: Data Engineer
Experience: 6 years | English: upper


Big Data Engineer with 6 years of experience in software development of high load distributed systems. My current responsibilities includes design and implementation of the new data pipelines for mobility intelligence platform, developing data integrity tools, work closely with data science team, mentoring newcomers etc. Current stack: Scala, Spark, Delta Lake, Airflow, Kafka, DynamoDB, Python, S3, EMR, Athena.
Would prefer to work for a product or outstaff company
Position: Big Data Engineer / Scala developer
Category: Scala
Experience: 11 years | English: intermediate

Create and maintain big data solutions using Spark, Spark streaming, Kafka, Hive, Solr. I have some pilot projects with technologies such Kafka, Cassandra, Scala. I have some works in machine learning fields (random forests, logistic regression, sentimental analysis and etc). Work with twitter streaming (Spark streaming, sentimental analysis).
Big data: Hadoop, Spark, Flume, Kafka, Impala, Cassandra, Hive, Solr, Zookeeper, Ambari server
languages: Scala, Python, Golang
Development platforms: Linux, Windows.
Development tools: InteliJ.
Framework: Play
Database: MySQL, PostgreSQL
NoSQL: Redis, Memcache, MongoDB
Other: Google API, Elasticsearch, Sphinx, Kibana, Logstash, Filebeat, Ether Blockchain, Bitcoin Blockchain
I am looking for any big data and data science opportunities in general and Spark+Scala opportunities in partial.
Position: BigData engineer Scala/Java/Python
Category: Data Engineer
Experience: 7 years | English: fluent


Scala, Java,  Python

Apache Flink(Flink SQL, Datastream)

Apache Kafka using both Java and Scala APIs (Confluent platform,  Schema registry, KSQL, Streams API, Processor API,Kafka Connect)

Apache Spark (RDD,Datasets, Dataframes, SQL, Streaming,Structured Streaming)

Apache Airflow (DataProc, Spark, Python, BigQuery and other operators)

Apache Hadoop (Google cloud Dataproc, Amazon EMR, HDP 2.5)
Athena(Presto), Glue, Hive
Apache Beam (Java and Scala API, Dataflow, Flink, Scio)

AWS, Google Cloud platforms
Akka (Actors, Http, Play)
Google Pub/Sub, AWS MSK, RabbitMQ 
Apache Avro, Parquet
DB: Google BigQuery, 
BigTable(Hbase), Apache Phoenix,  MySQL, MS SQL Server,

Development tools: IntelliJ IDEA, Git, Maven, SBT, Docker/ Docker Compose
Agile development approach, Confluence, Jira
A product company, stock options plan, real-time, high load
Position: Big Data Engineer \ Software Developer
Category: Python
Experience: 3 years | English: upper

Was involved to developing in all parts of project. Perpetually took part in fixing bugs and architecting new functionality.
ETL Developer, Software Developer

Developing ETL processes in PDI. Creating reports for Business. Creating dashboards in Tableau. Implementing and ongoing support our application ClevaDesk (frontend and backend). Creating Excel and Word templates. Creating BPMN diagrams and logic for approve.

Technology: Pentaho Data Integration, SQL(PostgreSQL, MariaDB), DB Schema, JS, Python, Lua, Domino Notes Formula, ClevaDesk, BPMN, Tableau
__________________________________________________
HCM Developer (Oracle Cloud)

Implementing Oracle Cloud HCM. Ongoing support for existing Oracle HCM clients. Processes analysis and customization of the system according to customer’s needs. Development and producing high level reports according to the technical design. OBIEE\OTBI high-level knowledge.  Developing ETL processes in Oracle HCM.

Technology: Oracle Cloud HCM, Oracle Reports , Fast Formula, XML / XPath / XSLT, SQL(PL SQL), Java, dot-Project, Redmine, BI Publisher
I'm looking for an interesting project in which I can use all my skills. I would admire to be a part of result-oriented professional team.
Position: Big Data/ETL developer
Category: SQL
Experience: 11 years | English: intermediate


Total IT experience 11 years.

Enterprise software development based on Oracle/Informatica/Hadoop, database modeling, working with large data volumes, performance optimization, technical analysis.

Databases: Oracle 12c, Oracle 11g, Oracle 10g, Oracle 9i, MySQL, MS SQL, MongoDB

ETL: Informatica PowerCenter 8.6/9.5.1/9.6.1, Informatica BDE 9.5.1/9.6.1

BigData: Cloudera Hadoop, Spark (Scala), MapReduce, Hive, Impala, HBase

CI: TeamCity

Version control: git, SVN
Position: Big Data / ETL / DWH developer
Category: Other
Experience: 6 years | English: intermediate

I have experience to perform Team Lead role on projects with 4-5 devs.
Experience with SQL, various ETL tools (SSIS, Pentaho Data Integration, Informatica), AWS (Amazon Redshift, AWS S3, EMR)  MSSQL, Hadoop, Spark, Scala (core), Java (core). Also I have an  experience to lead small teams in DWH projects and to work in international environment.
Current project is ingestion and processing XML files with Spark and Spark SQL.
I have an experience to work in various project with Insurance, Banking, Travel, Oil & Gas companies.
Right now I'm living abroad but I would be glad to come back to Ukraine.
I'm Senior Engineer but I'm looking for opportunity to switch from my current knowledge in DWH/ETL to things like Scala/Spark/Java etc... I have some production experience in this area but my motivation is to enrich my skills. Projects related to data (Big Data, Data Warehousing, ETL processes etc) or mixed. Ability to code on Spark or using other cutting-edge technologies.
Position: Big Data/Golang/Java Lead
Category: Java
Experience: 5 years | English: fluent


I've developed microservices on golang using best practice and microservice patterns.
Actually I've worked with Hadoop and Spark.
Maintained k8s nodes.
Implemented collaborative filtering.