improved tests for crawler and parser