A Django app for managing robots.txt files following the robots exclusion protocol