dakrone/itsy — pull request #5 by shriphani (2013-10-03 23:09:16+00:00)
Title: Politeness

Crawler must obey robots.txt. User can still override this setting.