Position: Python Developer
Company: Johannisberg Partners AG
Experience required: 5y
Category: Python
English level: upper
Published: 2022-02-01T00:00:00+02:00

Overview

Our team is looking for a Python Developer who is an expert in web scraping, with a particular focus on accessing large numbers of pages to create large datasets (more than 1 mn data points). A strong candidate will be proficient in both creating the functions for accessing, extracting and parsing the ‘raw’ data, as well as providing strong postprocessing, such as cleaning the data, writing checks, and providing the output in a high-quality format, suitable for database entry.

Key responsibilities:
- Build crawlers for various websites to collect structured data
- Define parsers to retrieve data in a professional manner
- Define postprocessing functions, incl. for data cleansing and quality checks
- Continuous integration of all of the above into the Adler crawler framework/ecosystem (GitHub)
- Integrate into and work with our existing crawler framework (Python), incl. utilities for read/write operations, checks for data de-duplication, etc.
- Work with our developer team to ensure continuous integration of new code
- Ensure you have a local, de-centralized working environment to work on existing crawlers (local env, IDE, GitHub, etc.)
- Potentially long-term: Obtain a leading role (’product owner’), incl. responsibility for multiple crawlers and coaching a team of junior developers

Key requirements:
- Proficiency in Python (key packages to be used include typically data entry/web scraping, such as Pandas, Selenium, BeautifulSoup, Chromedriver)
- Basic workflow management (defining scope of to-be-crawled data, e.g., for loops that span the right sets of URLs)
- Very high attention to data quality and detail (quality beats quantity): use existing and develop specific duplicate checks, data consistency checks, and perform manual checks
- Basic understanding of soccer rules/"an eye for soccer data"
- Strong on-the-fly documentation skills (Python typing, pydantic data models, docstrings)
- High willingness to learn and explore alternative paths for challenging problems
- Preferred: Strong candidates will have experience in data crawling, database record-entry, and a strong understanding of database taxonomies
- Proficiency in English language

What we offer:
- High degree of autonomy/independence — you can do much of the work remotely and set define your own schedule (within reasonable bounds)
- You work in an analytically strong team — fruitful exchange of ideas and continuous learning
- Long-term project with possibilities to develop further — potentially possible to take a leading role in our growing team
- Work directly with founders — bring in own ideas and have an impact on the overall direction of the project/company

If you believe you meet the above requirements and are looking for a new project in a modern environment, please send your CV and indicate your salary.