Position: Data Engineer
Category: Data Engineer
English level: upper
Years of experience: 5

Designed scalable data processing solutions capable of accommodating exponential data growth. Collected TBs of data from different sources. In my current projects, I have processed substantial amounts of data, reaching hundreds of gigabytes. This data includes diverse formats, such as videos and images.
I am currently working as a SW & Data Engineer, responsible for creating ETL pipelines to process and collect data (video, image, audio). Overall, my role requires me to have a strong understanding of both software engineering and data analysis, allowing me to design and implement efficient and scalable solutions for our team. In my current projects, I have processed substantial amounts of data, reaching hundreds of gigabytes. 

English: Upper-Intermediate level
Programming language: Python
Software development tools: Jira, Confluence
VCS: Git/GitHub, BitBucket
Web Scraping tools: Selenium WebDriver, Python + BeautifulSoup, proxies
NoSQL Databases: MongoDB, ElasticSearch, DynamoDB
SQL Databases: PostgreSQL, MySQL
Cloud Storage: Amazon S3
Key-value db: redis
Virtualization: Docker, Docker Compose, Kubernetes
Messaging systems: SQS
Message Queuing and Task Processing: celery+redis(rabbitMQ)
Working with clouds: AWS (lambda, SQS, MWAA etc.)
API: Flask framework, FastAPI
Work with the integration of different data tools into your project, and improve the quality of your data. Design and Develop ETL Pipelines: Create efficient and scalable ETL pipelines to process diverse text, image, video, and audio data, ensuring seamless data flow from source to destination.
Implement Best Practices: Utilize industry best practices to optimize data extraction, transformation, and loading processes, ensuring data quality, accuracy, and reliability.
Incorporate Newest Technologies: Stay up-to-date with the latest advancements in data engineering technologies and tools, integrating cutting-edge solutions to enhance pipeline performance and maintain competitiveness.