Package: dsbox-primitives
ISI PrimitivesISI DSBox Cleaning PrimitivesThe git repository containing DSBox cleaning related primitives ishere. The git repository for DSBox primitives related to featurization is locatedhere.Data cleaning primitivesd3m.primitives.dsbox.CleaningFeaturizerThis is a multi-purpose cleaning featurizer primitive. This primitive requires metadata annotations from ISI's profiling primitive, seed3m.primitives.dsbox.Profilerbelow. The cleaning featurization operations supported include:Split date column into multiple columns, e.g. year, month, date, daySplit US phone number into multiple columns.Split column with consistent alpha-numeric value patterns, e.g. '2days' into multiple columns.Split column with consistent puntucation value patterns, e.g. 'NY_US' into multiple columns.d3m.primitives.dsbox.FoldColumnsFold multiple columns into one column based on common column name prefix. For example, fold columns with names 'month-jan', 'month-feb', 'month-mar' and so on, into one column named 'month'.Encoding primitivesd3m.primitives.dsbox.EncoderPerforms one-hot encoding for categorical attributes. This encoder can handle missing values, and it allows user to specify the upper limit of columns to generate per cagtegorical attribute,n_limit.d3m.primitives.dsbox.UnaryEncoderPerforms unary encoding, which useful for ordinal data.Imputation primitivesd3m.primitives.dsbox.MeanImputationPerforms mean missing value imputation for numerical columns, and mode imputation for categorical columns.d3m.primitives.dsbox.GreedyImputationPerforms missing value imputation by greedy search over simple imputation methods, i.e. mean, min, max, and zero.d3m.primitives.dsbox.IterativeRegressionImputationPerforms missing value imputation by regression, then improve the imputation by iterating over columns with missing values.Profiling Primitived3m.primitives.dsbox.ProfilerThis primitive generates metadata by examining the given data. The types of metadata include:Column contains values tokenizable as an American phone numberColumn contains values tokenizable by puntucationColumn contains values tokenizable into numerical tokens and alpha tokensColumn value tokenization features (most common tokens, number of distinct tokens, ratio of distinct tokens, and so on)Column value features (most common values, number of distinct values, ration of distinct values, and so on)Column contains filename-like valuesColumn contains missing values (number of missing values, ratio of missing values)Number of outlier valuesCorrelation between columns (Pearson, Spearman)Datamart Primitivesd3m.primitives.dsbox.QueryDataframeQueries datamart for available datasets. The JSON query specification is definedDatamart Query API. The primitive returns a list of dataset metadata.d3m.primitives.dsbox.JoinJoins two dataframes into one dataframe. The primtive takes two dataframes, left_dataframe and right_dataframe, and two lists specifing the join columns, left_columns and right_columns.ISI DSBox Featurization PrimitivesThe git repository for DSBox primitives related to featurization is locatedhere. The git repository containing DSBox cleaning related primitives ishere.Image Featurization Primitivesd3m.primitives.dsbox.ResNet50ImageFeatureGenerate features using pre-trained ResNet50 deep neural network. Use hyperparameterlayer_indexto select the network layer to use for featurization.d3m.primitives.dsbox.Vgg16ImageFeatureGenerate features using pre-trained VGG16 deep neural network. Use hyperparameterlayer_indexto select the network layer to use for featurization.d3m.primitives.dsbox.DataFrameToTensorReads in image files and generates a tensor that suitable as input tod3m.primitives.dsbox.ResNet50ImageFeatureandd3m.primitives.dsbox.Vgg16ImageFeature.Timeseries Featuration Primitivesd3m.primitives.dsbox.RNNTimeSeriesPerforms forecasting of one timeseries using recursive neural network.d3m.primitives.dsbox.AutoArimaPerforms forecasting of one timeseries using AutoArima.d3m.primitives.dsbox.GroupUpByTimeSeriesPerforms forecasting of one timeseries using Group Up.d3m.primitives.dsbox.RandomProjectionTimeSeriesFeaturizationGenerate features of multiple timeseries by random projecting the timeseries matrix into lower dimendions.d3m.primitives.dsbox.TimeseriesToListReads in timeseries csv files and generate output List that is suitable as input tod3m.primitives.dsbox.RandomProjectionTimeSeriesFeaturization.Multi-table Join Primitived3m.primitives.dsbox.MultiTableFeaturizationAutomatically detect foriegn key relationships among multiple tables, and join the tables into one table using aggregation.Miscellaneousd3m.primitives.dsbox.DoNothingThis an identity function primitive that returns the input dataframe as output. This useful for bypassing a step in a pipeline without having to modify the pipeline structure.