lang-uk/recruitment-dataset-candidate-profiles-english huggingface dataset:
Position: Big Data engineer
Category: Java
Experience: 8 years | English: upper


Results-driven Big Data Engineer with over 8 years of experience in developing and implementing data solutions. Skilled in utilizing HDFS, MapReduce, Spark, Hive, Kafka, Java, and Python. Strong knowledge of AWS Cloud and a passion for learning innovative technologies. Proven expertise in software development, system implementation, debugging, testing, and maintenance. Experienced in research, development, and implementing proof of concept (POC) projects.
Challenging Projects: Seek complex data engineering tasks to expand skills.
Learning Opportunities: Value continuous learning. Cutting-Edge Technologies: Look for a company that stays up to date with the latest data engineering tools to stay relevant and explore new approaches. Professional Growth: Ensure a clear career path with opportunities for promotion. Feedback and Reviews: Expect regular performance assessments and constructive feedback.
Position: Big Data engineer
Category: Data Engineer
Experience: 5 years | English: upper

Upgraded project from python 2.* to python 3.*
Experience in scraping and parsing with selenium, beautifulSoup, lxml
Created GUI with pyqt
Created new and updated existing data pipelines with dataflow, orchestrate a jobs with airflow and other data lake related job.
Worked as a PHP developer on freelance. 
1 year of experience in web scraping, in office, using Web Query Language. Experience in Jira and SVN.
Last 5 years work with python, 2 of them as a GCP Data engineer in data lake.
Certified Google data engineer.
Position: BigData engineer
Category: Data Engineer
Experience: 9 years | English: intermediate


Development of ETL  processes for related databases. 
Writing SQL queries 
Troubleshooting and support existing systems 
Creating pipelines based on AWS Step Functions
Writing PySpark scripts for AWS Glue and Azure Databricks
Position: Big Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate

In one of project I have optimised data pipeline and reduce time to process data in 4x. I have good experience with relative new tool - DataHub. Recently I have gotten AWS Certified Data Analytics - Specialty (03.02.2022-03.02.2025)
1. Creating Data Platform with using DataHub as the metadata storage and visualisation.

Responsibilities:
- Collect metadata and fill DataHub, build data lineage.
- Creating and maintaining airflow pipelines to fill RedShift data warehouse and datalake.
- Develop IaC using Terraform

2. Creating airflow pipelines to fill RedShift data warehouse and datalake.
Responsibilities:
- Creating and maintaining airflow pipelines to fill RedShift data warehouse and datalake.
- Creating AWS Glue job
- Debugging and bug fixing

3. ETL systems. Data analysis and conversion. Creating Data Marts based on the data.
Responsibilities:
- Creating serverless aplication on AWS (AWS Lambda, AWS Glue)
- Increased code quality(integrated sonar, decomposition complex area of codes) 
- Rump-up of new members.
- Creating stage table using AWS Crawler.
- Covering with tests
4. Large enterprise project with microservice architectureintended to optimize calculating the rebates, chargeback and
other types of fees that are used in the USA marketing. The
complex platform for business data management and
analysis aim is to assists customers in the delivery of managed services for the health care industry.

Responsibilities:
- Increase code quality(added docstrings, integrated sonar,decomposition some complex area of codes).
- Implement inside tools for parsing title of commit and sync with JIRA ticket between different release.
- Fix bugs and noted root causes of bugs.
Position: Big Data Engineer
Category: Data Science
Experience: 2 years | English: upper

Helped the customer to enter the market in another region faster by building an Ml platform in a short time
1) Development of a service for automatic detection of anomalies in the fuel system. 
Responsibilities:
Creating data lake and MLOps process
Development of PySpark pipelines for data preparation for training models
Implementation of the orchestration of model training and prediction pipelines using Airflow 
Mentoring interns and knowledge transfer to new project participants during their onboarding 
2) Development of an Ml platform for a pharmaceutical company.
Responsibilities:
Development of Terraform scripts
Development of pipelines for model training
Leading a team of 4 people
Discussion of requirements with the customer
Conducting a demo
3) Development of a data governance solution for all internal services of the company. 
Responsibilities:
Development of connectors for data ingestion from various internal services
Development of data access differentiation
Architecture development
4) Development of a recommendation system for e-commerce
Responsibilities:
Building pipelines for data processing
Delta Lake development
Building pipelines for setting ML models for production
Interested in any complex tasks related to the construction of highly loaded pipelines
Position: Big Data Engineer
Category: Data Engineer
Experience: 7 years | English: upper

1. Significant performance enhancements for different clients.
2. Expenditure reductions.
3. Participated in a few large customers acquisition for the company.
1. Development of different infrastructures and processes
for Big Data purposes (bare metal, cloud, k8s) with
different stack types (Spark, Kafka, Hive, HBase, Yarn, Spark Standalone, Spark on k8s, MongoDB, Mysql, some
specific NoSql and SQL databases, Livy)

2. Cloud system development

3. DevOps related tasks (large K8S clusters setup and
administrating, CI process management with Jenkins, development, and management of internal infrastructure
project on Ansible, networks management), docker images
refactoring, helm charts development
Involving in apache products improvements
QA-related tasks (performance testing, setting up the
the testing process)

4. Teams management

5. Projects audit and performance enhancement

6. Presale activities (presentations, communications with clients, etc.)
I want to be engaged in near to life project that makes some real stuff with a lot of benefits for real world
Position: Big Data Engineer
Category: Python
Experience: 3 years | English: fluent

- project to parse Amazon data using Tor network of free proxies as an alternative to paid services (client could decrease monthly costs by a few thousand US dollars)

- background before switching to IT - economics & finance (passed 2 out of 3 levels of Chartered Financial Analyst [CFA] qualification), data analytics roles in various Ukraine-based western-capital top-tier banks / investment & insurance companies.
- 4th commercial project (~6 months) - global privacy protection-based search engine (DuckDuckGo alternative). I defined architecture and built integrated and automated parsing of Wikipedia articles big data dumps within AWS Serverless infrastructure (AWS Step Functions, Lambda, Glue/PySpark, DynamoDB, AWS SAM) - to use it for displaying Wikipedia widget in a search engine results.

- 3rd commercial project (~3 months) - NFT-market based project in transition from Flask to Django. Writing basic Django replica (basic fullstack Django/Bootstrap) app from scratch with functionality of existing project in Flask (using docker & docker compose, consisting of Django web app, nginx, celery, Postgres, Redis, Rabbitmq), celery jobs of parsing of data from Opensea, endpoints and views, user dashboards & authentication etc.) 

- 2nd commercial project (~14 months) - US e-commerce platform for managing products marketing on Amazon - new functionality, performance & speed improvements, debugging, Pytest tests from scratch, ad hoc scripts.

- 1st commercial project (~7 months) - US e-commerce platform for managing logistics & goods deliveries costs & efficiency for e-commerce traders - improvements and debugging.

- pet-project (~6 months) - e-commerce platform for managing drop-shipping products sales in Ukraine - myself full-stack building from scratch to MVP.
Focus on back-end.

However, I have some practical experience with integrating back-end with HTML / Bootstrap / CSS / Javascript / JQuery / Ajax / Jinja.
Position: Big Data Engineer
Category: Scala
Experience: 6 years | English: upper

7+ experience years.
-saved 200k $ for the company
-deployed service 2 months faster than expected
-report speaker
6+ commercial dev experience.
-Scala(Certified)
-Spark/Hadoop (Certified)
-Hive, MS SQL, MongoDB, Redis
-RabbitMQ, Kafka
-Microservices
-Highload
-DevOps
TDD, BDD, DRY, KISS, SOLID, GoF
living in the USA
Position: Big Data Engineer
Category: Data Engineer
Experience: 1 years | English: intermediate

I've been engaged in ETL projects holding responsibilities of setting up fault-tolerant and reliable environments with Docker containers, extracting and delivering data from various sources for further processing using Kafka and Spark, building scalable and automated data pipelines with Airflow, and analyzing data using Kibana.
As a Big Data Engineer, I utilized different tools such as Java, Go, Scala, Python, Pandas, Spark, TimescaleDB, Postgres, Kafka, Azure Cloud, Amazon Cloud, and NodeRed to develop and maintain big data solutions.

- Involved in various projects, including real-time message processing, end-to-end metrics analysis platforms, and building an IoT device analytics platform.
- Gained experience in designing, deploying, and managing big data solutions in cloud environments, resulting in improved efficiency and cost savings for the organization.
- Collaborated with cross-functional teams including Data Science, AQA, Backend, Frontend, and BI to deliver end-to-end big data solutions.
- Participated in interviewing and training junior specialists, contributing to the growth and development of the team.
I have a strong passion to improve business and operational processes by leveraging data, and creating reliable and scalable Big Data Platforms using technologies like Amazon Web Services (AWS), Spark, Scala, Airflow, and many more.
Position: Big Data Engineer
Category: Data Engineer
Experience: 3 years | English: upper


BigData developer, with more than 10 years experience in e-commerce development. More than 3 years in Big data development, 7 as a QC Automation Engineer. Experienced in ETL pipelines implementation (Big Data stack: Apache Spark, Apache Hadoop, Apache Kafka + AWS).
  	Experienced in team leading, designing of automation frameworks, implementation and maintenance of complex ecosystems and QC frameworks based on Java stack.
 	Experienced in long term onsite-offshore team effort coordination with success.
Position: Big Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper

Migrated to cloud the full scale multistack system for geospatial data processing in real time. The system consists from a bunch of ETL processes, both batch and streaming, Geoserver as backend, React app as frontend and Acumulo as distributed database, continuously debugging and upgraded the system.
Big Data Engineer at N-IX
February 2021
Data Engineering activities  - developing, maintaining and troubleshooting big data pipelines (ETL, ELT) using Spark, Hadoop, Airflow. 
Experinced with AWS cloud tech: EMR, EC2, S3, Kinesis, Lambda, S3, SES, Athena, Redshift.
Get known with Scala and JS, working with geospatial data.
Tech Stack:
Python/Scala/JS/Spark/Hadoop/Airflow/AWS/Git/Acumulo/Geomesa/Geoserver

Database Researcher at AllStarsIT 
October 2020 - Junuary 2021
Research new types of database vulnerabilities, applying best practices and official security policies to the on-prem and cloud databases (vSphere, AWS, Azure). Developing bash scripts for OS security checks, python parsers and other internal tools, sql-queries. DBA activities, tuning databases according to the best security practices. 
Bash/Python/SQL/DBA/Linux/Windows Server/AWS/Azure  

Data Analyst/Research Engineer  at Ring Ukraine  April 2018 - September 2020   
Implemented anomaly detection applications, developed web applications for Edge devices performance analytics and Data Collection Process. Dashboards developing. developed analytics applications for computer vision algorithms (ex: people/vehicle detection/tracking) based on Cloud platform AWS). Data Collection on real world conditions. Applying Machine Learning and Deep Learning techniques. 
Python/Linux/SQL/AWS Athena/Redshift/Scikit-learn/Pytorch   

Data Annotation Specialist  at Ring Ukraine  
October 2017 - April 2018   
Data Annotation and Data Quality control using internal tools. Data Entry activities. 
 Mac OS/JSON
Azure cloud
Position: Big Data Engineer
Category: Data Engineer
Experience: 8 years | English: upper

Confluent Certified Developer for Apache Kafka, 2020
Big Data Engineer with more than 8 years of commercial experience with a key
focus on big data and streaming technologies (Apache Kafka, Apache Spark, Hadoop),
cloud (AWS), and containerization (Docker, Kubernetes). Expertise in working as a part
of distributed teams with customers from the US and EU. Participated in projects of such domains as banking, IOT, healthcare, oil, energy.

Worked with Data lakes, DWHs, ETL pipelines, Streaming processing. 

As engineer, my main responsibilities included:
 Requirements gathering
 Evaluation of available solutions on the market
 Software solutions design
 POC development
 Implementing batch & event-driven streaming data pipelines
 Development and support of data migration solutions.
Not interested in positions that involve lots of SQL-scripts development.
Position: Big Data Engineer
Category: Java
Experience: 5 years | English: upper


Big Data Software Engineer 
- Creating and maintaining BigQuery views.
- Leading on sharding, partitioning, clustering optimization in BigQuery
- Denormalization of structures into BigQuery
- Creating dashboards into DataStudio
- Optimizing DataFlow batch/streaming jobs
- Developing ETL pipelines using Apache Airflow & Python
- Optimizing scheduled queries into BigQuery
- Configuring AirFlow DAGs
Position: Big Data Engineer
Category: Data Engineer
Experience: 8 years | English: upper


Dec 2021 - present:
Big Data engineer, GlobalLogic
Project overview:
Data platform that provides a capability for hospitality industry businesses to get an
unified view of their customers.
Key technologies:
Microsoft Azure Cloud (ADLS2, Data Factory), Databricks, Delta Lake, Spark, Scala,
Elasticsearch

May 2021 - Dec 2021:
Big Data engineer, Luxoft
Project overview:
Data quality solution for corporate Big Data platform.
Key technologies:
Kafka, Apache Flink, AWS, Java 11, Python, Gitlab CI/CD, Kubernetes

June 2020 – May 2021
Big Data engineer, Global Logic
Project overview:
Datalake migration from on-premises environments to AWS.
Key technologies:
Airflow, AWS (EMR, S3, DynamoDB, Glue), Python

Oct 2019 – June 2020
Big Data engineer, EPAM Systems
Project overview:
Migration of data warehouses from on-premises environments to GCP.
Key technologies:
Google Dataflow, Airflow, Apache Beam, Google Cloud Storage,
BigQuery, Java, Avro, Scala, Apache Spark.

Aug 2019 - Oct 2019
Software engineer, Global Logic
Project overview:
Medical-domain project. Web application that allows patient to submit
observations and predict which type of treatment is the most suitable.
Key technologies:
Spring Boot, Spring Data, REST, MySQL, Java 9.

May 2018 - Aug 2019
Software engineer, Global Logic
Project overview:
Security analytic platform, analysis of incoming stream of event logs,
detection of potential threats. Comprises streaming part and micro-services.
Key technologies:
Apache Spark, Google Dataflow, Bigtable, Google Cloud Storage,
Kafka, Apache Airflow, NiFi, Spring Boot, Spring Data, Elasticsearch,
MySQL, Java, Python.

Feb 2015 - Apr 2018
Software engineer, Nix Solutions
Project overview:
Web project (medical domain).
Key technologies:
Spring, Hibernate, jsp, MSSql, BackboneJS, REST
Position: Big Data Engineer
Category: Data Engineer
Experience: 2 years | English: upper

·	Designed, developed, and migrated ETL and ELT processes in Snowflake.
·	Designed solutions for complex data processing from the AWS layer to Snowflake
Developing an application that drives End-to-End Global Supply Chain Performance with batch processing, sensitive data, and orchestration of all data processing. Creating Glue jobs. Developing Snowpark jobs. Troubleshooting and fixing complex defects. Firefighting during deployment processes
Position: Big Data Engineer
Category: Data Engineer
Experience: 6 years | English: upper


Experienced Big Data Engineer/Data Engineer with MLOps skills. Using Scala, Python, Spark, and AWS. Have team-leading experience in a small DE team.
My current role is Big Data Engineer/MLOps.

On a current project, I am developing infrastructure for ML and NLP models(deployment, API, feature engineering etc.) and data storage, also working on the integration of customers' internal services.

On a previous project, I developed an engineering part(API, data storage, spark jobs, ci/cd) for the NLP model.
Also last year I lead an engineering team and was a mentor for new team members. As a team leader, I planned and designed new features with customer stakeholders and my team successfully completed the development.
Position: Big Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper

AWS Developer Certificate
Oracle Java Certificate
Hello! 

Are you looking for a qualified specialist?
I've quite a good experience working in different domains. On practice always try to follow best software design and development practices. Such approach allows to provide the best quality product and value for the customers.

Can't find skill or tech in my bio? 
Just write me to clarify.

Compensation looks like too high? :) 
Lets discuss it :) 

Data Engineer 
1. Prepare DataLake for Fin-tech customer. 
2. Build automatic medical adverse event detection. 
3. Bioinformatics project for efficient storing genetic data

Software Engineer
1. Web-page crawler for Anti-piracy project.
2. Platform for learning languages online.
3. CRM for finance/investment managers. 

Tech: Java/Scala/Python
Airflow
PostgreSQL, MongoDB, Snowflake
AWS EMR/Glue/ECS
Hadoop Ecosystem, Docker
SQL/NoSQL DBs
Spring Framework
JUnit/Spock Frameworks
Interesting long-running project with qualified team/management.
Position: Big Data Engineer
Category: Data Engineer
Experience: 5 years | English: upper


* Hi I'm Big Data Enginner from Tbilisi, Georgia.
 Proficiency in  Oracle database Ms SQL, MySQL, PostgreSQL, Big Data technologies.

* I've strong functional knowledge of IT and I'm looking for 
  challenging projects to help you build something great.

* Hourly rate also depends on the complexity and duration of the job, so the rate may be higher or lower.

* Reporting an ETL Developer over 3 years of exprerience.

* PowerBI / DAX Developer with 2 years of experience.

* Big Data Enginner with 2 years of experience.

* Experiences with Banking for 4 Years.

* Experiences with telecommunications for 3 Years.
Position: Big Data Engineer
Category: Data Engineer
Experience: 3 years | English: upper


I have almost 2 years of experience in Big Data and 3 years in software development. I'm a Big Data Engineer with knowledge of OOP principles, Java 11, Python, RDBMS and SQL. I have experience in developing modern lakehouse solutions. Also, I am deeply familiar
with the ETL process, worked with various sources - structured and semi-structured data for different purposes. Had an experience with AI, including ML&DL. Worked with modern cloud solutions providers such as GCP and Azure. Scrum and Kanban as process methodologies were used to manage product development and other knowledge work in projects where I was involved.
I’m result-oriented, embrace critical thinking and have a growth mindset
Position: Big Data Engineer
Category: Python
Experience: 3 years | English: upper


I have been working in the field of Big Data for 3 years. Have hands-on experience with Spark, Hive, HDFS, YARN, Apache Livy, HBase, HiFi, Airflow, Docker, Kubernetes, MQTT, and more. Most often use Python, SQL, and Scala programming languages, also have a basis knowledge of C/C++, Go, Java, Kotlin, JS, and R. Have been using AWS, Digital Ocean, and Databricks cloud providers.
I have practical experience in all phases of product implementation: initiation, research, requirement definition, design, establishment of milestones, development, integration, testing and further support. Have experience in firefighting with a highly loaded data processing system.
I’m playing an active and constructive role within the team. After some time I become a good connoisseur of business model and business sphere, which allows me to share knowledge among team members.
For 1.5 years was working in a telecommunication company; have experience working with IoT events. Currently work in the Game Industry as a Big Data Engineer.
Position: Big Data Engineer
Category: Data Engineer
Experience: 2 years | English: upper


I have experience as Python Developer, Machine Learning Engineer and Big Data Engineer. Work on startup project as a Python Developer for almost 1 year. Next 1 year worked as a freelancer Machine Learning Engineer. I was a part of team, we worked on different projects. Our biggest project was an Extract Summarization for lawyer documents. Worked as a Big Data Engineer in Grid Dynamics on big commercial project for 1 year. Now a Big Data Engineer in Trinetix.
Position: Big Data Engineer
Category: Data Engineer
Experience: 4 years | English: upper


I have experience in developing Big Data infrastructues, troubleshooting the errors. I got the experience of leading a small team of developers. Gain the two AWS certifications. I'm passionate to learn new technologies and to solve engineering problems.
Position: Big Data Engineer
Category: Data Engineer
Experience: 3 years | English: intermediate


I have experience with Big Data. I participated in the creation of the new data process and the improvement of existing ones, as well as in the bug fix. I have experience in organizing teamwork.  Punctual, polite, plodding, attentive to details, who likes to work in a team, help teammates and learn new technologies.
Position: Big Data Engineer
Category: Data Engineer
Experience: 5 years | English: upper

Detailed list of my achievements available in resume.
Mainly:
- developed complex systems from scratch
- broke existing monolithic apps to microservices
- proposed changes that led to revenue increase
- leaded a team
I have extensive experience of work in startups with frequent requirement changes. Mostly my experience related to Big Data and Full Stack development areas.
Main technologies in my tech stack are: Java + Spring, BigData (Spark, Flink, Hadoop), Kafka, Vert.x
I have experience of team leadership.
More details about my achievements on various projects are available in my resume.
I want to work on interesting project with new technologies (at least > Java 8). If it's migration from legacy to new technologies I might consider this.
I prefer to work with product companies.
I don't like a lot of bureaucracy.
Position: Big Data Engineer
Category: Data Engineer
Experience: 1 years | English: intermediate

Microsoft Certified: Azure Fundamentals
Microsoft Certified: Azure Data Fundamentals
I'm a junior big data engineer with a great interest in the data world. I started my career as a python engineer but quickly became fascinated with big data and clouds, so I started exploring them. 

I have successfully passed two Azure certifications (as well as multiple online courses) and I’m preparing for the next one: “Data Engineering on Microsoft Azure”. After that, I’m planning to pass “Databricks Lakehouse Fundamentals”. I’m constantly improving my skills in Python, SQL, and clouds.
I am interested in learning Python, SQL, clouds for software engineering development and data analysis.