A web crawler for various purposes