document: Adaptive duplicate detection using learnable string similarity measures The problem of identifying approximately duplicate records in databases is an essential step for data cleaning and data integration processes . Most existing approaches have relied on generic or manually tuned distance metrics for estimating the similarity of potential duplicates . In this paper , we present a framework for improving duplicate detection using trainable measures of textual similarity . We propose to employ learnable text distance functions for each database field , and show that such measures are capable of adapting to the specific notion of similarity that is appropriate for the field 's domain . We present two learnable text similarity measures suitable for this task : an extended variant of learnable string edit distance , and a novel vector-space based measure that employs a Support Vector Machine ( SVM ) for training . Experimental results on a range of datasets show that our framework can improve duplicate detection accuracy over traditional techniques .
doc_bio_tags: O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O