Package: akerbp-mlpet
akerbp.mlpetPreprocessing tools for Petrophysics ML projects at EurekaInstallationInstall the package by running the following (requires python 3.8 or later)pip install akerbp.mlpetQuick startFor a short example of how to use the mlpet Dataset class for pre-processing data see below. Please refer to the tests folder of this repository for more examples as well as some examples of thesettings.yamlfile:import os
    from akerbp.mlpet import Dataset
    from akerbp.mlpet import utilities

    # Instantiate an empty dataset object using the example settings and mappings provided
    ds = Dataset(
            settings=os.path.abspath("settings.yaml"), # Absolute file paths are required
            folder_path=os.path.abspath(r"./"), # Absolute file paths are required
    )

    # Populate the dataset with data from a file (support for multiple file formats and direct cdf data collection exists)
    ds.load_from_pickle(r"data.pkl") # Absolute file paths are preferred

    # The original data will be kept in ds.df_original and will remain unchanged
    print(ds.df_original.head())

    # Split the data into train-validation sets
    df_train, df_test = utilities.train_test_split(
            df=ds.df_original,
            target_column=ds.label_column,
            id_column=ds.id_column,
            test_size=0.3,
    )

    # Preprocess the data for training according to default workflow
    # print(ds.default_preprocessing_workflow) <- Uncomment to see what the workflow does
    df_preprocessed = ds.preprocess(df_train)The procedure will be exactly the same for any other dataset class. The only difference will be in the "settings". For a full list of possible settings keys see either thebuilt documentationor the akerbp.mlpet.Dataset class docstring. Make sure that the curve names are consistent with those in the dataset.The loaded data is NOT mapped at load time but rather at preprocessing time (i.e. when preprocess is called).Recommended workflow for preprocessingDue to the operations performed by certain preprocessing methods in akerbp.mlpet, the order in which the different preprocessing steps can sometimes be important for achieving the desired results. Below is a simple guide that should be followed for most use cases:Misrepresented missing data should always be handled first (usingset_as_nan)This should then be followed by data cleaning methods (e.g.remove_outliers,remove_noise,remove_small_negative_values)Depending on your use case, once the data is clean you can then impute missing values (seeimputers.py). Note however that some features depend on the presence of missing values to provide better estimates (e.g.calculate_VSH)Add new features (using methods fromfeature_engineering.py) or usingprocess_wellsfrompreprocessors.pyif the features should be well specific.Fill missing values if any still exist or were created during step 4. (usingfillna_with_fillers)Scale whichever features you want (usingscale_curvesfrompreprocessors.py). In some use cases this step could also come before step 5.Encode the GROUP & FORMATION column if you want to use it for training. (usingencode_columnsfrompreprocessors.py)Select or drop the specific features you want to keep for model training. (usingselect_columnsordrop_columnsfrompreprocessors.py)NOTE:The dataset classdropsall input columns that are not explicitly named in your settings.yaml or settings dictionary passed to the Dataset class at instantiation. This is to ensure that the data is not polluted with features that are not used. Therefore, if you have features that are being loaded into the Dataset class but are not being preprocessed, these need to be explicitly defined in your settings.yaml or settings dictionary under the keyword argumentkeep_columns.API DocumentationFull API documentaion of the package can be found under thedocsfolder once you have run the make html command.For developersto make the API documentation, from the root directory of the project run (assuming you have installed all development dependencies)cd docs/
  make htmlto install mlpet in editable mode for use in another project, there are two
possible solutions dependent on the tools being used:If the other package uses poetry, please refer to thisguideIf you are not using poetry (using conda, pyenv or something else), just revert to usingpip install -e .from within the root directory (Note: you need to have pip version >= 21.3).Licenseakerbp.mlpet Copyright 2021 AkerBP ASALicensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.