Patent application EP16191846A (EP), filed 2016-09-30
Publication number: EP3260992A1
Title: SYSTEM AND METHOD FOR DATA CLEANSING
Abstract: System and method for data cleansing are disclosed. The method comprises receiving one or more data records pre-categorized into one or more categories. Identifying at least one concept associated with one or more data records, and grouping, the at least one concept associated with the one or more data records into a plurality of category lists based on the predefined category associated with each of the one or more data records. Determining, one or more intersection sets based on a comparison between each pair of the plurality of category lists, wherein each intersection set comprises a set of one or more common concepts associated with a pair of category lists. The method comprises replacing each of at least one common concept of the set of one or more common concepts associated with each intersection set by a category name based on an occurrence rate of the common concepts.
IPC codes: G06F  17/    27            A I
G06F  17/    30            A I
Claims: number: 1
text: A computer-implemented method of data cleansing, the method comprising:
number: 2
text: The method of claim 1, wherein the at least one concept comprises one or more words.
number: 3
text: The method of claim 1 or claim 2, further comprising determining a largest intersection set from the one or more intersection sets, wherein the largest intersection set includes a highest number of common concepts.
number: 4
text: The method of claim 3, wherein the highest number of common concepts is replaced by the at least one category name based on an occurrence rate of each of the highest number of common concepts associated with the largest intersection set.
number: 5
text: The method of any of the preceding claims, wherein replacing the each of the at least one common concept of the set of one or more common concepts associated with the each intersection set comprises:
number: 6
text: The method of claim 5 wherein replacing the each of the at least one common concept of the set of one or more common concepts associated with the each intersection set further comprises :
number: 7
text: The method of any of the preceding claims, wherein replacing the each of the at least one common concept of the set of one or more common concepts associated with the each intersection set further comprises:
number: 8
text: A system for data cleansing, comprising:
number: 9
text: A non-transitory computer-readable medium storing instructions for data cleansing, wherein upon execution of the instructions by one or more processors, the processors perform the method of any of claims 1 to 7.
Office action: search_report, dated 2017-12-27
Prior art cited: ref_id: - XIAOZHONG LIU: "Proposal of Document Classification with Word Sense Disambiguation", IDEALS ICONFERENCE 2008, 28 February 2008 (2008-02-28), XP055340713, Retrieved from the Internet <URL:https://www.ideals.illinois.edu/bitstream/handle/2142/15145/PO20_iconf08.doc.pdf?sequence=4&isAllowed=y> [retrieved on 20170131]
ref_type: npl
severity: background
ref_id: - HWEE TOU NG: "Exemplar-Based Word Sense Disambiguation: Some Recent Improvements", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 10 June 1997 (1997-06-10), XP080614568
ref_type: npl
severity: background
ref_id: - XINDONG WU ET AL: "Mining With Noise Knowledge: Error-Aware Data Mining", IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS. PART A:SYSTEMS AND HUMANS, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 38, no. 4, 1 July 2008 (2008-07-01), pages 917 - 932, XP011216487, ISSN: 1083-4427
ref_type: npl
severity: background
ref_id: - Xiaozhong Liu, "Proposal of Document Classification with Word Sense Disambiguation", iDEALS iConference 2008, (20080228), URL: https://www.ideals.illinois.edu/bitstream/handle/2142/15145/PO20_iconf08.doc.pdf?sequence=4&isAllowed=y, (20170131), XP055340713 [I] 1-9 * the whole document *
ref_type: npl
severity: background
ref_id: - HWEE TOU NG, "Exemplar-Based Word Sense Disambiguation: Some Recent Improvements", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (19970610), XP080614568 [A] 1-9 * abstract * * page  4, column  lhc, paragraph  1 *
ref_type: npl
severity: background
ref_id: - XINDONG WU ET AL, "Mining With Noise Knowledge: Error-Aware Data Mining", IEEE TRANSACTIONS ON SYSTEMS, MAN AND CYBERNETICS. PART A:SYSTEMS AND HUMANS, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, (20080701), vol. 38, no. 4, ISSN 1083-4427, pages 917 - 932, XP011216487 [A] 1-9 * abstract * * Section I "Introduction" * * page  922; figure 2 *
ref_type: npl
severity: background
Final disposition: pending (2017-12-27)