Package: autoembedder
The AutoembedderIntroductionThe Autoembedder is an autoencoder with additional embedding layers for the categorical columns. Its usage is flexible, and hyperparameters like the number of layers can be easily adjusted and tuned. The data provided for training can be either a path to aDaskorPandasDataFrame stored in the Parquet format or the DataFrame object directly.InstallationIf you are usingPoetry, you can install the package with the following command:poetryaddautoembedderIf you are usingpip, you can install the package with the following command:pipinstallautoembedderInstalling dependenciesWithPoetry:poetryinstallWithpip:pipinstall-rrequirements.txtUsage0. Some importsfromautoembedderimportAutoembedder,dataloader,fit1. Create dataloadersFirst, we create twodataloaders. One for training, and the other for validation data. Assourcethey either accept a path to a Parquet file, to a folder of Parquet files or aPandas/DaskDataFrame.train_dl=dataloader(train_df)valid_dl=dataloader(vaild_df)2. Set parametersNow, we need to set the parameters. They are going to be used for handling the data and training the model. In this example, only parameters for the training are set.Hereyou find a list of all possible parameters. This should do it:parameters={"hidden_layers":[[25,20],[20,10]],"epochs":10,"lr":0.0001,"verbose":1,}3. Initialize the autoembedderThen, we need to initialize theautoembedder. In this example, we are not using any categorical features. So we can skip theembedding_sizesargument.model=Autoembedder(parameters,num_cont_features=train_df.shape[1])4. Train the modelEverything is set up. Now we canfitthe model.fit(parameters,model,train_dl,valid_dl)ExampleCheck outthis Jupyter notebookfor an applied example using theCredit Card Fraud Detectionfrom Kaggle.ParametersThis is a list of all parameters that can be passed to the Autoembedder for training. When using the training script the_needs to be replaced with-and the parameters need to be passed as arguments. For boolean values please have a look at theCommentcolumn for understanding how to pass them.Run the training scriptYou can also simply use the training script::python3training.py\--epochs20\--train-input-path"path/to/your/train_data"\--test-input-path"path/to/your/test_data"\--hidden-layers"[[12, 6], [6, 3]]"for help just run:python3training.py--helpArgumentTypeRequiredDefault valueCommentbatch_sizeintFalse32drop_lastboolFalseTrue--drop-last / --no-drop-lastpin_memoryboolFalseTrue--pin-memory / --no-pin-memorynum_workersintFalse00 means that the data will be loaded in the main processuse_mpsboolFalseFalse--use-mps / --no-use-mpsmodel_titlestrFalseautoembedder_{datetime}.binmodel_save_pathstrFalsen_save_checkpointsintFalselrfloatFalse0.001amsgradboolFalseFalse--amsgrad / --no-amsgradepochsintTruedropout_ratefloatFalse0Dropout rate for the dropout layers in the encoder and decoder.layer_biasboolFalseTrue--layer-bias / --no-layer-biasweight_decayfloatFalseFalsel1_lambdafloatFalse0xavier_initboolFalseFalse--xavier-init / --no-xavier-initactivationstrFalsetanhActivation function; eithertanh,relu,leaky_reluorelutensorboard_log_pathstrFalsetrim_eval_errorsboolFalseFalse--trim-eval-errors / --no-trim-eval-errors; Removes the max and min loss when calculating themean loss diffandmedian loss diff. This can be useful if some rows create very high losses.verboseintFalse0Set this to1if you want to see the model summary and the validation and evaluation results. set this to2if you want to see the training progress bar.0means no output.targetstrFalseThe target column. If not set no evaluation will be performed.train_input_pathstrTruetest_input_pathstrTrueeval_input_pathstrFalsePath to the evaluation data. If no path is provided no evaluation will be performed.hidden_layersstrTrueContains a string representation of a list of list of integers which represents the hidden layer structure. E.g.:"[[64, 32], [32, 16], [16, 8]]"activationcat_columnsstrFalse"[]"Contains a string representation of a list of list of categorical columns (strings). The columns which use the same encoder should be together in a list. E.g.:"[['a', 'b'], ['c']]".drop-cat-columnsboolFalse--drop-cat-columns / --no-drop-cat-columnsWhy additional embedding layers?The additional embedding layers automatically embed all columns with the Pandascategorydata type. If categorical columns have another data type, they will not be embedded and will be handled like continuous columns. Simply encoding the categorical values (e.g., with the usage of a label encoder) decreases the quality of the outcome.