Position: Data Engineer (Scraping)
Company: Serious Software
Category: Data Engineer
Experience required: 5y
English level: upper
Published: 2021-12-01T00:00:00+02:00

They are turning the rich and diverse Latin American financial ecosystem into a set of developer-friendly tools and APIs.

Responsibilities:
- Design and develop web scraping pipelines via Python and Javascript; 
- Perform network traffic analysis and infer implicit API's from web and mobile applications;
- Understand and be able to reverse engineer authentication processes both in web and mobile applications;
- Have a very good understanding of scraping techniques and how to analyze crawling processes;
- Build parsing and normalization solutions for multiple data sources;
- Stay on top of reliability and data quality by implementing automated quality assurance and monitoring solutions.

Requirements:
- 5+ years of experience as a Developer, even better if with Python;
- Strong experience with techniques and tools for crawling, extracting, and processing data (e.g. Scrapy, Puppeteer, BeautifulSoup, etc.);
- Good spoken and written English.
 
As a plus:
- Fintech and SaaS past experience;
- have mobile scraping expertise (APK decompilation, code analysis and replication, etc.);
- Asynchronous programming experience with Python`s asyncio;
- Experience designing and developing RESTful APIs;
- Experience building large-scale platforms with a global reach.


Project tech stack:
- They’re building our platform using modern technologies, putting the focus on reliability and long-term maintainability;
- They primarily use Python on the backend. They use battle-tested technologies such as Django, and we are heavy users of Python’s asyncio for some parts of our stack;
- They use Javascript, Vue.js and Sass on the frontend. They are developing and leveraging our own design system and component library;
- They run our infrastructure on top of Amazon Web Services, leveraging managed services. They tend to favor the use of managed services wherever possible, in order to focus on our business problems;
- They observe and monitor our services using Datadog;
- They follow Continuous Integration and Continuous Delivery best practices.