Patent Document ID: 9443019
Application ID: 14601008

Base Claim:
1. A system for providing web domains classification based on progressive crawling with clustering, comprising: a processor configured to: distribute a first Uniform Resource Locator (URL) content categorization data feed to a first plurality of subscribers, comprising to: collect the first URL content categorization data feed based on progressive crawling with clustering to determine which category clusters to publish for each categorized web domain, comprising to: automatically classify content for a first web domain, comprising to: receive the first web domain from a first security device associated with one of the first plurality of subscribers, comprising to: crawl a plurality of pages in the first web domain; and determine a category for the plurality of pages in the first web domain, to group more than one page having the same category into a first cluster, comprising to: determine whether a number of the more than one page of the first cluster exceeds a first threshold; in the event that the number of the more than one page of the first cluster does not exceed the first threshold, select a new page within the web domain to crawl; and classify the new page; and send a classification determined for the content for the first web domain to the first security device associated with one of the first plurality of subscribers; and a memory coupled to the processor and configured to provide the processor with instructions.

---

Claim 4:
4. The system recited in claim 1 , wherein the processor is further configured to: distribute a second URL content categorization data feed to a second plurality of subscribers, wherein the second URL content categorization data feed is collected based on progressive crawling with clustering to determine which category clusters to publish for each categorized web domain.