Position: Cloud Infrastructure and Data Operations Engineer
Company: Trust Sourcing
Experience required: 5y
Category: Data Engineer
English level: upper
Published: 2023-09-01T00:00:00+03:00

We are seeking a highly skilled and motivated Cloud Infrastructure and Data Operations Engineer to join our team. In this role, you will be responsible for managing and optimizing our AWS cloud infrastructure, including AWS Redshift, as well as supporting data pipelines and ETL processes. Your expertise in AWS management and data engineering will play a critical role in ensuring the reliability, security, and performance of our systems. You work with the backend and data science teams to support development, testing, and deployment processes.


Key Responsibilities:
AWS Infrastructure Management
● Manage and maintain all aspects of our AWS cloud infrastructure, including EC2 instances, RDS databases, VPCs, security groups, and more.
● Configure and optimize networking components, including VPNs, VPC peering, and security groups to ensure secure and efficient data transfer.

Security and Compliance
● Implement and maintain security best practices, ensuring AWS resources are secure and compliant with industry standards.

Monitoring and Alerting
● Set up monitoring systems to proactively identify and address infrastructure issues. Configure alerts for critical events

Documentation
● Maintain clear and up-to-date documentation for infrastructure and operational procedures.

Data Engineering
● Design, develop, and maintain robust data pipelines for the extraction, transformation, and loading (ETL) of data from various sources into AWS Redshift.

AWS Redshift Management
● Manage and optimize our AWS Redshift data warehouse to ensure high performance, scalability, and cost-effectiveness.

ETL
● Utilize Apache Airflow to automate and schedule ETL workflows, ensuring data is processed efficiently and on time.
● Monitor and troubleshoot ETL processes, addressing issues promptly to minimize downtime.



Requirements
● Proven experience in AWS infrastructure management.
● Strong knowledge of AWS services, networking, and security.
● Experience with data engineering, ETL processes, and data warehousing (AWS Redshift preferred).
● Proficiency in scripting languages (e.g., Python) for automation tasks.
● Familiarity with Apache Airflow or similar workflow orchestration tools.
● Excellent problem-solving and communication skills.
● Ability to work collaboratively in a team and independently when required.
● AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Data Analytics) are a plus.