Merge branch 'semaphore'
Merge branch 'dev'
removed stdout prints for robots.txt errors