changed /crawl method to post