Updated spec_helper. Added server support in specs. Updated Crawler and its spec.