Patent Document ID: 9443019
Application ID: 14601008

Base Claim:
1. A system for providing web domains classification based on progressive crawling with clustering, comprising: a processor configured to: distribute a first Uniform Resource Locator (URL) content categorization data feed to a first plurality of subscribers, comprising to: collect the first URL content categorization data feed based on progressive crawling with clustering to determine which category clusters to publish for each categorized web domain, comprising to: automatically classify content for a first web domain, comprising to: receive the first web domain from a first security device associated with one of the first plurality of subscribers, comprising to: crawl a plurality of pages in the first web domain; and determine a category for the plurality of pages in the first web domain, to group more than one page having the same category into a first cluster, comprising to: determine whether a number of the more than one page of the first cluster exceeds a first threshold; in the event that the number of the more than one page of the first cluster does not exceed the first threshold, select a new page within the web domain to crawl; and classify the new page; and send a classification determined for the content for the first web domain to the first security device associated with one of the first plurality of subscribers; and a memory coupled to the processor and configured to provide the processor with instructions.

---

Claim 8:
8. The system recited in claim 1 , wherein the automatically classifying of the content for the first web domain further comprises to: receive a second web domain from a second security device associated with one of the first plurality of subscribers; the send a classification determined for the content for the second web domain to the second security device associated with one of the first plurality of subscribers; and store, in the first security device, different URL categorization data than URL categorization data stored in the second security device based on different URL requests passing through first security device.