Patent application EP20177617A (EP), filed 2020-05-29
Publication number: EP3745276A1
Title: DISCOVERING A SEMANTIC MEANING OF DATA FIELDS FROM PROFILE DATA OF THE DATA FIELDS
Abstract: [0001]    A data processing system for discovering a semantic meaning of a field included in one or more data sets is configured to identify a field included in one or more data sets, with the field having an identifier. For that field, the system profiles data values of the field to generate a data profile, accesses a plurality of label proposal tests, and generates a set of label proposals by applying the plurality of label proposal tests to the data profile. The system determines a similarity among the label proposals and selects a classification. The system identifies one of the label proposals as identifying the semantic meaning. The system stores the identifier of the field with the identified one of the label proposals that identifies the semantic meaning.
IPC codes: G06F  16/    21            A I
Claims: number: 1
text: A data processing system for discovering a semantic meaning of a field included in one or more data sets, the system including:
number: 2
text: The system of claim 1, wherein profiling the one or more data values of the field includes determining a format of a data value of the field.
number: 3
text: The system of claim 1, wherein profiling the data values of the field includes determining a statistical value representing the data values included in the field.
number: 4
text: The system of claim 3, wherein the statistical value comprises at least one of a minimum length of the data values of the field, a maximum length of the data values of the field, a most common data value of the field, a least common data value of the field, a maximum data value of the field, and a minimum data value of the field.
number: 5
text: The system of claim 1, wherein applying the plurality of label proposal tests includes:
number: 6
text: The system of claim 1, wherein applying the plurality of label proposal tests includes:
performing a metadata comparison of data values of the field to terms in a glossary of terms.
number: 7
text: The system of claim 1, wherein applying the plurality of label proposal tests includes:
number: 8
text: The system of claim 1, wherein applying the plurality of label proposal tests includes:
number: 9
text: The system of claim 1, the operations further including:
determining, in response to applying the plurality of label proposal tests, a relationship between the field and another field of the one or more data sets.
number: 10
text: The system of claim 9, wherein the relationship includes one of an indication that a first data value the field determines a second data value stored in the other field, an indication that the first data value correlates to the second data value, or an indication that the first data value is identical to the second data value.
number: 11
text: The system of claim 1, wherein the plurality of label proposal tests are each associated with at least one weight value, the operations further including:
number: 12
text: The system of claim 1, the operations further including:
number: 13
text: The system of claim 1, wherein comparing the label proposals generated from the label proposal tests includes:
number: 14
text: The system of claim 1, the operations further including:
number: 15
text: The system of claim 1, the operations further including:
reducing, based on the identified one of the label proposals, a number of errors for processing data for the field using data quality rules from the data quality environment relative to another number of errors for processing the data for the field without using the identified one of the label proposals.
Office action: search_report, dated 2020-12-02
Prior art cited: ref_id: US201962855233P
ref_type: patent
severity: background
ref_id: US202016794361A
ref_type: patent
severity: background
ref_id: EP3407207A1
ref_type: patent
severity: background
ref_id: WO2015187155A1
ref_type: patent
severity: background
ref_id: US2011173149A1
ref_type: patent
severity: background
ref_id: EP2261820A2
ref_type: patent
severity: background
ref_id: - SABBIR M RASHID ET AL: "The Semantic Data Dictionary Approach to Data Annotation & Integration", 1 October 2017 (2017-10-01), XP055721287, Retrieved from the Internet <URL:https://semsci.github.io/semSci2017/openReview/SemSci_2017_paper_10.pdf> [retrieved on 20200810]
ref_type: npl
severity: background
ref_id: - ZHIYU CHEN ET AL: "Generating Schema Labels through Dataset Content Analysis", THE WEB CONFERENCE 2018, INTERNATIONAL WORLD WIDE WEB CONFERENCES STEERING COMMITTEE, REPUBLIC AND CANTON OF GENEVASWITZERLAND, 23 April 2018 (2018-04-23), pages 1515 - 1522, XP058397736, ISBN: 978-1-4503-5640-4, DOI: 10.1145/3184558.3191601
ref_type: npl
severity: background
ref_id: - PHAM MINH ET AL: "Semantic Labeling: A Domain-Independent Approach", 23 September 2016, ANNUAL INTERNATIONAL CONFERENCE ON THE THEORY AND APPLICATIONS OF CRYPTOGRAPHIC TECHNIQUES, EUROCRYPT 2018; [LECTURE NOTES IN COMPUTER SCIENCE; LECT.NOTES COMPUTER], SPRINGER, BERLIN, HEIDELBERG, PAGE(S) 446 - 462, ISBN: 978-3-642-17318-9, XP047357760
ref_type: npl
severity: background
ref_id: - ANONYMOUS: "Rule-based system", 16 May 2019 (2019-05-16), XP055721374, Retrieved from the Internet <URL:https://en.wikipedia.org/w/index.php?title=Rule-based_system&oldid=897374778> [retrieved on 20200810]
ref_type: npl
severity: background
ref_id: - FELIX NAUMANN: "Data profiling revisited", SIGMOD RECORD, ACM, NEW YORK, NY, US, vol. 42, no. 4, 28 February 2014 (2014-02-28), pages 40 - 49, XP058045944, ISSN: 0163-5808, DOI: 10.1145/2590989.2590995
ref_type: npl
severity: background
ref_id: - ZHIYU CHEN ET AL, "Generating Schema Labels through Dataset Content Analysis", THE WEB CONFERENCE 2018, INTERNATIONAL WORLD WIDE WEB CONFERENCES STEERING COMMITTEE, REPUBLIC AND CANTON OF GENEVASWITZERLAND, (20180423), doi:10.1145/3184558.3191601, ISBN 978-1-4503-5640-4, pages 1515 - 1522, XP058397736 [I] 1-15 * sections 4 and 5 *
ref_type: npl
severity: background
ref_id: - PHAM MINH ET AL, Semantic Labeling: A Domain-Independent Approach, ANNUAL INTERNATIONAL CONFERENCE ON THE THEORY AND APPLICATIONS OF CRYPTOGRAPHIC TECHNIQUES, EUROCRYPT 2018; [LECTURE NOTES IN COMPUTER SCIENCE; LECT.NOTES COMPUTER], SPRINGER, BERLIN, HEIDELBERG, PAGE(S) 446 - 462, (20160923), ISBN 978-3-642-17318-9, XP047357760 [I] 1-15 * section 3.2 *
ref_type: npl
severity: background
ref_id: - Anonymous, "Rule-based system", (20190516), URL: https://en.wikipedia.org/w/index.php?title=Rule-based_system&oldid=897374778, (20200810), XP055721374 [A] 1-15 * the whole document *
ref_type: npl
severity: background
ref_id: - FELIX NAUMANN, "Data profiling revisited", SIGMOD RECORD, ACM, NEW YORK, NY, US, (20140228), vol. 42, no. 4, doi:10.1145/2590989.2590995, ISSN 0163-5808, pages 40 - 49, XP058045944 [A] 1-15 * the whole document *
ref_type: npl
severity: background
Final disposition: pending (2020-12-02)