issue #8 - add crawler config, disallow all for now