Adding configuration object for crawler module.
Fixing test in light of this addition.