Patent ID: 8099415
Filing Date: 2012-01-17
Classification: G06F

Abstract:
1. A method of identifying semantically duplicate job listings that describe a same job, where the duplicate job listings are retrieved from a plurality of online sources, the method comprising the machine-implemented steps of: preprocessing each job listing, comprising: automatically determining sameness-indicating signatures for the respective job listings, where the signatures are shorter in length than the job listings they were derived from and where substantially identical signatures indicate likelihood of job listings that describe a same job; automatically testing the signatures for collisions and among the job listings that have signature collisions, identifying job listings as duplicates of each other if they share at least a predetermined amount of identical text; and automatically segregating job listings that have been identified as duplicates so that one serves as a unique master job listing for storage in a production database and for being searched through by job seeking queries of job seekers.