dermesser/perl-grepper — Issue #1: robots.txt not respected
Author: eserte
Created: 2012-01-07 17:17:31+00:00
State: open

See title. As perl-grepper is a crawler, it should also respect /robots.txt.