Divided web into crawler (crawl function) and web (deal with server requests)