Refactor API calls in crawler.py