Package: esr-dt-model
ESR_DT_MODELThis package serves as a hub for consolidating all individual model developments associated with the Digital Twin project. Its primary objective is to generate unified and ensemble-based model outputs, which can be seamlessly integrated into any downstream applications.1. Install the packageThe package can be installed usingpip:pip install esr_dt_model2. Usage:This package serves as a repository for preserving modeling development processes and allows for the retrieval of information from previous developments.The following provides a simple example of using the package:from pandas import read_csv
from xgboost import XGBRegressor
from esr_dt_model import esr_dt_model

# load dataset
url = 'https://raw.githubusercontent.com/jbrownlee/Datasets/master/housing.csv'
df = read_csv(url, header=None)

# split training and target dataset
x, y = df.iloc[:, :-1], df.iloc[:, -1].to_frame()

# train the model
model = XGBRegressor()
trained_model = model.fit(x, y)

# save the data:
esr_dt_model.export_model(
    project_name="example",
    user="Sijin",
    trained_models=trained_model,
    training_data=x,
    target_data=y
)

# view the model:
esr_dt_model.view_model()

# load the model:
model = esr_dt_model.import_model(model_version="Simple_example")2.1 Save model and related dataset:The model, training dataset and test dataset can be saved as below:from esr_dt_model import esr_dt_model
    esr_dt_model.export_model(
        <Project Name>,
        <User>, 
        <Trained Models>, 
        <Training Dataset>, 
        <Target Dataset>)Where here<Project Name>is the project name,<User>is the user name,<Trained Models>is a trained model,<Training Dataset>is the dataset used for training the model,<Target Dataset>is the target dataset in the training process. Note thatproject name,user,trained model,training datasetandtarget datasetare mandatory arguments.By default, the model and related dataset will be saved in the development channel. When a model is well tested, the model can be saved in the production channel by settingprodtoTrue. For example:esr_dt_model.export_model(
        ...
        prod=True)Also, by default, for each training process, a unique ID will be assigned. However, one can fix the ID by setting the argument ofexp_id. For example:esr_dt_model.export_model(
        ...
        exp_id="Simple_example")2.2 List model and related dataset:We can list all stored model and related dataset as below:from esr_dt_model import esr_dt_model
    esr_dt_model.view_model(
        filters = {
            "project_name": [<Project name1>, <Project name2>, ...],
            "datetime_start": <Start date>,
            "datetime_end": <End date>,
        }
    )Thefiltershere indicates the conditions that we want to put when list the model. The fullfilterscan incldude the arguments includingproject_name,datetime_start,datetime_end,user,fmt,output_type, for example:filters = {
        "project_name": ["DT"],
        "datetime_start": "20231112T0149",
        "datetime_end": "20231112T0250",
        "user": ["Sijin"],
        "fmt": ["pkl", "onnx"],
        "output_type": ["dev", "prod"]
    }An optional argumentkeycan also be used to specify the columns that you want to view. The full columns include['project_name', 'version', 'datetime', 'user', 'type', 'fmt', 'output', 'output_type', 'training_data', 'test_data']. By default, all columns will be shown.2.3 Load the model:The saved model can be loaded as:esr_dt_model.load_model("D7QVDT")whereD7QVDTis the model version (a unique ID) that can be obtained from runningesr_dt_model.view_model. By default, if we find multiple models share the same model version (e.g., set byexp_id), only the latest version of the model will be returned. However, we can return all models by settinglatestargument toFalse. For example,esr_dt_model.load_model("D7QVDT", latest=False)Appendix: Publish the package (for development only)The package can be published as:make publishNote the athe API token must be set up in~/.pypirc.