Patent Document ID: 7685090
Application ID: 11182590
Patent Flag: 1

Claim One:
1. In a database having records stored on a medium, a method for identifying duplicate tuples in data records, the data records containing a plurality of dimensional hierarchical data fields in a respective hierarchical arrangement of relations, the method comprising: identifying a plurality of tuples in a parent field of the data records in a corresponding hierarchical arrangement of relations; identifying the tuples of the plurality of tuples in the parent field as duplicates based on corresponding tuples in a child field of the data records, the child field being a child field of the parent field in the corresponding hierarchical arrangement of relations, wherein the step of identifying the tuples of the plurality of tuples in the parent field as duplicates comprises: identifying a plurality of duplicative tuples in the corresponding child field of the data records in the corresponding hierarchical arrangement of relations; matching each of the duplicative tuples in the child field with a tuple in a corresponding parent field of the data records corresponding hierarchical arrangement of relations; identifying the tuples of the plurality of tuples in the parent field as duplicates comprises identifying a candidate set of records having a first record field from records in a table; and determining a commonality between records having first record fields based on tokens in a set of one or more child record fields related to the first record fields to identify duplicate records from the candidate set of records; determining a record v 1 is a duplicate of another record v 2 if a containment metric of tokens from a field of v 1 in the record v 2 is greater than or equal to a threshold value, wherein a textual similarity between tokens in the first record field is compared in determining a token containment metric; and wherein a textual similarity between tokens in the first field is compared in determining a token containment metric, a textual similarity between tokens in the child record fields is compared in determining a foreign key containment metric; and both the token containment metric and the foreign key containment metric are combined in identifying possible duplicate records in the candidate set of records; and wherein the method is executed in at least one computing device.