Patent ID: 11861462
Assignee: nan
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method for consistently preparing data for a machine learning (ML) system, comprising:
receiving a tabular training data set, the training data set including a set of one or more source columns;
identifying column labels from the training data set, the column labels associated with a source column of data points;
determining, for each identified column label, a root category based on at least one of a user specification, data types, or distribution properties associated with the data points in each column of the set of source columns;
performing one or more data transformations for data points in a source column in an order based on defined primitives of a transformation tree to obtain a transformed data set, the transformation tree including defined primitive category entries associated with each root category, wherein the defined primitives associated with the source column are based on a root category associated with the source column, wherein the defined primitive category entries for the root category are associated with a defined transformation function set, wherein the transformation function set includes transformation functions for training data sets and test data sets, wherein the primitives are used to specify the type, order, and retention of univariate transformations performed on a training data basis in a hierarchy associated with a root category applied to the source column to return zero, one, or more columns derived from a source column, wherein the set of primitives include upstream and downstream primitives, wherein the set of primitives include primitives that may supplement or replace the input data representation, wherein the set of upstream primitives include primitives with or without offspring, wherein a category entry to an upstream primitive with offspring is used to access a separate transformation tree to access a set of corresponding downstream primitive category entries, wherein this set of downstream primitives is treated as a set of upstream primitives for their state of received data representation;
recording the column categories determined for each identified column label and properties of the data transformations performed for each source column in a metadata database;
outputting the metadata database and transformed training data set, wherein the transformed training data set is for training a ML system, and wherein the metadata database is output for use by a user for additional data sets;
receiving a tabular additional data set and the metadata database;
performing the one or more data transformations for data points in corresponding additional columns of the tabular additional data set using the recorded column categories and properties of the data transformations from the metadata database to obtain a transformed additional data set on a basis derived from a corresponding training data set; and
outputting the transformed additional data set for use with the ML system.