Patent Document ID: 9448999
Application ID: 13462592

Base Claim:
1. A method that uses a processor for detecting similar documents, comprising: extracting entities from a first web document and a second web document from among web documents; determining importance contribution elements corresponding to the entities in the first web document and the second web document; calculating, by the processor, weights for the entities based on the determined importance contribution elements; calculating, by the processor, characteristic indexes for each of the first web document and the second web document, by extracting hash values for the entities, and subsequently calculating the characteristic indexes by applying the weights to the hash values for the entities; determining whether the first web document and the second web document are similar to each other based on the calculated characteristic indexes; deleting one of the first web document and the second web document if the first web document and the second web document are determined to be similar to each other, and wherein each of the importance contribution elements includes a frequency of which each of the entities is duplicated within the web documents.

---

Claim 4:
4. The method of claim 1 , wherein the entities are extracted from at least one of a text node, an anchor node, an embed node, and an image node the first web document and the second web document.