Position: Data Engineer -Dev Ops Specialist
Company: DataFlik
Category: Data Engineer
Experience required: 3y
English level: fluent
Published: 2023-06-01T00:00:00+03:00

Company: DataFlik

Location: Fully Remote

Type: Full-Time

About DataFlik:

DataFlik is a fast-growing, data-driven company offering unique data solutions to our clients. We are expanding and seeking a Data Engineer who specializes in Operations and Data Deployment to join our dynamic team.

Job Description:
We're seeking a detail-oriented and innovative Data Engineer to manage, optimize, and deploy our data systems. The successful candidate will create and maintain data ingestion and processing pipelines, transition our data storage to the Medallion Architecture, and deploy data files to our current users, ensuring data is readily available, of high quality, and well-organized. The candidate will occasionally conduct web scraping to further build and enhance our database.

Key Responsibilities:

Create and manage data pipelines for large-scale data processes
Transition our data storage to the Medallion Architecture

Efficiently and cost-effectively manage data storage (currently at 9.7 TB/year)

Deploy data files to our current users, ensuring data usability and accessibility.

Collaborate with Machine Learning and Software Engineering teams to optimize data systems.

Implement and monitor data systems to ensure data availability, quality, and usability.

Occasionally perform web scraping to enhance our database
Requirements:

Degree in Computer Science, Engineering, or a related field
Experience with large-scale data processing and data storage systems

Proficiency with AWS S3, AuroraDB, and other cloud-based data solutions

Familiarity with the Medallion Architecture and data ingestion pipelines

Proven ability to collaborate with various teams and manage multiple tasks concurrently

Excellent problem-solving skills and attention to detail
Preferred Qualifications:

Prior experience in a similar role focused on Operations and Data Deployment

Proven track record of implementing efficient and cost-effective data organization systems

Experience with web scraping tools and techniques