Web Crawler Architecture.
Design and implementation of a high-performance distributed Web crawler