Package: mlexp
MLexpMLexp is the framework to automate model training, hyperparameters
optimization and logging results to a server, which allows to speed up
hypothesis testing and helps to maintain the reproducibility of experiments.InstallationInstallation with pip$pipinstallmlexpBy default, MLexp is installed without support of torch training.To support training torch models install MLexp as:$pipinstallmlexp[torch]This waypytorch-lightningwill also be installed.It is advised to install desired version of torch before installing mlexp[torch]
because pytorch-lightning depends on torch and by default downloads
torch with CUDA support (and heavy dependencies).DocumentationDocumentation is hosted onRead the DocsSimple exampleTrain model, optimize hyperparameters and log resultsIn this example we will train lightgbm model, optimize hyperparameters
and log results to mlflow server.First of all, we have to start mlflow server. Run the following command
in your console.$mlflowserver--port5000Running this command will start mlflow server on port 5000 in the
current directory.Now we should define function which returns set of parameters.def params_func(trial):
    return(
        {
            'model_params': {'boosting': trial.suggest_categorical('boosting', ['gbdt', 'dart', 'goss']),
                             'feature_fraction': trial.suggest_float('feature_fraction', 0.01, 1),
                             'min_child_samples': trial.suggest_int('min_child_samples', 2, 256),
                             'n_estimators': 500},
            'lgb_data_set_params': {}
        }
    )According to this function:boostingwill be chosen from ['gbdt', 'dart', 'goss']
during hyperparameter optimizationfeature_fractionwill be chosen from uniform distribution from
0.01 to 1 during hyperparameter optimizationmin_child_sampleswill be chosen integers from 2 to 256 during
hyperparameter optimizationno parameters will be passed to lightgbm.Dataset, except fordataandlabel, which are passed from training data automaticallyNow we can use python API of mlexp.trainers.LgbTrainer to train model,
optimize hyperparameters and log results to mlflow server.from mlexp.trainers import LgbTrainer
from sklearn.metrics import mean_squared_error

# Initialise trainer object
trainer = LgbTrainer(
    # MSE will be used as validation metric
    validation_metric = mean_squared_error,
    # MSE should be minimised during hyperparameters optimization
    direction = 'minimize',
    # Before logging to server files will be saved to /home/logged_files
    saved_files_path = r'/home/logged_files',
    # During training model on test fold n_estimators will be set to the mean n_estimators on validation_folds
    use_average_n_estimators_on_test_fold = True,
    # During hyperparameters' optimization, mean metric on validation fold will be optimized
    optimization_metric = 'metric_mean_cv')

# Initialise mlflow run
trainer.init_run(
    # Init run on mlflow server
    logging_server = 'mlflow',
    # Run will be started in experiment 'example_exp'
    experiment_name = 'example_exp',
    # URI of mlflow server (it will be printed in console after starting mlflow server)
    tracking_uri = 'http://127.0.0.1:5000/',
    # Let's set run_name to 'Example. LGBM' and add tag Hypothesis = 1
    mlflow_run_params = {'run_name': 'Example. LGBM', 'tags': {'Hypothesis': '1'}},
    # Let's also log current_data_config.txt to mlflow server
    upload_files = ['/home/current_data_config.txt'])

# Hyperparameters' sampler
sampler = optuna.samplers.TPESampler(seed = 1234)

# Start model training, hyperparameters optimization and logging results to mlflow server
trainer.train(
    X = X,
    y = y,
    cv = cv_list,
    # Will optimize hyperparameters during 20 iterations
    n_trials = 20,
    params_func = params_func,
    sampler = sampler)Model inferenceLet's get trained model, parameters and hyperparameters from mlflow
server with the help of mlexp.inference.LgbInferencefrom mlexp.inference import LgbInference

# Initialize inference object
inference = LgbInference(
    # Let's download models and aprameters to /home/downloaded_files
    downloaded_files_path = '/home/downloaded_files',
    inference_server_params={
    'tracking_uri': 'http://127.0.0.1:5000/',
    'run_id': '1325ca558ec14277b0f39b0f8134d17e'},
    server='mlflow')
# Download params and model
inference.get_params_model(
    # Get parameters and models from best step of hyperparameters optimization (with minimal MSE)
    step = 'best',
    # Get model for test fold
    fold_num = 'test',
    # Get trained model
    trained_model = True)After running this code we will get dictionary with downloaded
parameters and trained model.ContributingFor contributing guidelines checkCONTRIBUTING.md