Patent application EP20179567A (EP), filed 2020-06-11
Publication number: EP3751427A1
Title: RULES-BASED DATASET CLEANING
Abstract: One or more processors store rules for performing rules-based cleaning operations on a plurality of datasets, wherein each rule comprises one or more functions to be executed against a dataset during the rules-based cleaning operations, the one or more functions each having one or more associated conditions and actions, wherein the one or more actions are performed on the dataset responsive to the one or more associated conditions being satisfied. The one or more processors further apply the rules to each of the plurality of datasets to perform the rules-based cleaning operations. To apply the rules to a given dataset, the one or more processors identify an ordered list of the one or more functions to be executed with respect to the given dataset during the rules-based cleaning operations and determine, for each of the one or more functions, whether the given dataset satisfies one or more conditions associated with a respective function of the one or more functions. Responsive to the given dataset satisfying the one or more conditions associated with the respective function, the one or more processors perform, on the given dataset, one or more actions associated with the respective function and provide a derived dataset comprising at least one modification to the given dataset resulting from the one or more actions associated with the respective function.
IPC codes: G06F  16/   215            A I
Claims: number: 1
text: A method comprising:
number: 2
text: The method of claim 1, wherein each of the plurality of datasets comprises one or more columns each having corresponding column metadata, and wherein performing the one or more actions comprises modifying at least one column metadata of at least one of the one or more columns from the dataset.
number: 3
text: The method of claim 2, wherein each column metadata comprises at least one of a column name or a column data type, and wherein performing the one or more actions comprises modifying at least one of a column name or a column data type of at least one of the one or more columns from the first dataset.
number: 4
text: The method of any preceding claim, wherein determining whether the given dataset satisfies the one or more conditions associated with the respective function comprises determining whether at least one of a name, identifier, path, or storage location associated with the given dataset matches a defined parameter.
number: 5
text: The method of any preceding claim, wherein the rules are applied to each of the plurality of datasets automatically responsive to a triggering event, the triggering event comprising at least one of a periodic interval having elapsed since a previous rules-based cleaning operation was performed, an addition of a new dataset to the plurality of datasets, or receipt of a request to perform a rules-based cleaning operation.
number: 6
text: The method of any preceding claim, wherein applying the rules to a given dataset further comprises:
number: 7
text: The method of any preceding claim, further comprising:
number: 8
text: The method of any preceding claim, wherein the rules are stored as an object model, the object model comprising one or more objects that each represent an action of the one or more actions to be performed with respect to the plurality of datasets, one or more objects that each represent a function of the one or more functions that invokes the one or more actions, and one or more objects that each represent a condition of the one or more conditions defining whether to perform the one or more actions.
number: 9
text: The method of claim 8, wherein the object model further comprises one or more objects that each represent a sequencer defining the ordered list of the one or more functions to be executed, and an object representing a cleaner that orchestrates the rules-based cleaning operations.
number: 10
text: A system configured to:
number: 11
text: The system of claim 10, wherein each of the plurality of datasets comprises one or more columns each having corresponding column metadata, wherein to perform the one or more actions, the system is configured to modify at least one column metadata of at least one of the one or more columns from the dataset, and optionally wherein each column metadata comprises at least one of a column name or a column data type, and wherein to perform the one or more actions, the system is configured to modify at least one of the column name or the column data type of at least one of the one or more columns from the first dataset.
number: 12
text: The system of claim 10 or claim 11, wherein to determine whether the given dataset satisfies the one or more conditions associated with the respective function, the system is configured to determine whether at least one of a name, identifier, path, or storage location associated with the given dataset matches a defined parameter.
number: 13
text: The system of any of claims 10 to 12, wherein the rules are applied to each of the plurality of datasets automatically responsive to a triggering event, the triggering event comprising at least one of a periodic interval having elapsed since a previous rules-based cleaning operation was performed, an addition of a new dataset to the plurality of datasets, or receipt of a request to perform a rules-based cleaning operation.
number: 14
text: The system of any of claims 10 to 13, wherein to apply the rules to a given dataset, the system is configured further to:
number: 15
text: Computer program instructions that, when executed by one or more processors, cause the one or more processors to perform a method as claimed in any of claims 1 to 9.
Office action: search_report, dated 2020-12-16
Rejection reasons: novelty_epc_54
Prior art cited: ref_id: US2018075115A1
ref_type: patent
severity: background
ref_id: US2011029546A1
ref_type: patent
severity: background
ref_id: US2005278270A1
ref_type: patent
severity: background
ref_id: US2013117202A1
ref_type: patent
severity: background
ref_id: US2005278270
ref_type: patent
severity: background
ref_id: US2011029546
ref_type: patent
severity: novelty_destroying
ref_id: US2018075115
ref_type: patent
severity: novelty_destroying
Final disposition: pending (2020-12-16)