updating docs and adding new crawler for test