GitHub repository: shaileshpandey11/web-crawler
Primary language: Java
Created: 2019-02-03T05:36:28Z
Description: A JAVA based Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).