Package: igann
IGANN - Interpretable Generalized Additive Neural Networks--- This project is under active development ---IGANN is a novel machine learning model from the family of generalized additive models (GAMs). This GAM is special in the sense that it uses a combination of extreme learning machines and gradient boosting.Some of the main features are:Training is very fast and can be performed on CPU and GPUThe rate of change of so-called shape functions can be influenced through hyperparameter ELM_scaleThe initial model is simply linear, thus IGANN generally also performs well on small datasetsMain developers of this project are:Mathias Kraus, FAU Erlangen-NürnbergDaniel Tschernutter, ETH ZurichSven Weinzierl, FAU Erlangen-NürnbergPatrick Zschech, FAU Erlangen-NürnbergNico Hambauer, FAU Erlangen-NürnbergSven Kruschel, FAU Erlangen-NürnbergLasse Bohlen, FAU Erlangen-NürnbergJulian Rosenberger, FAU Erlangen-NürnbergTheodor Stöcker, FAU Erlangen-NürnbergInstallationAvailable throughpip install igannFor the latest features, use this repository.DependenciesThe project depends on PyTorch and abess (version 0.4.5).UsageIGANN can in general be used similar to sklearn models. The methods to interact with IGANN are the following:.fit(X, y) for training IGANN on (X, y) dataset. Categorical variables are derived based on their dtypes. That's why the feature marix X needs to be a pandas DataFrame..predict_raw(X) to compute simple prediction for regression or raw logits for classification tasks. Per default values greater than 0 could be interpreted as belonging to class 1 and values smaller than 0 as belonging to class -1..predict(X) to compute simple prediction for regression or class prediction in {-1, 1} for classification tasks. If the IGANN parameter optimize_threshold is set to True, the threshold for the class prediction is optimized on the training data and hence deviates from the decision boundary of 0..predict_proba(X) to compute probability estimates.plot_single() to show shape functions.plot_learning() to show the learning curve on train and validation setParametersWhen initializing IGANN, the following parameters can be set:task: defines the task, can be 'regression' or 'classification'n_hid: the number of hidden neurons for one featuren_estimators: the maximum number of estimators (ELMs) to be fitted.boost_rate: Boosting rate.init_reg: the initial regularization strength for the linear model.elm_scale: the scale of the random weights in the elm model.elm_alpha: the regularization strength for the ridge regression in the ELM model.act: the activation function in the ELM model. Can be 'elu', 'relu' or a torch activation function.early_stopping: If there has been no improvements for 'early_stopping' number of iterations, training is stopped.device: the device on which the model is optimized. Can be 'cpu' or 'cuda'random_state: random seed.optimize_threshold: if True, the threshold for the classification is optimized using train data only and using the ROC curve. Otherwise, per default the raw logit value greater 0 means class 1 and less 0 means class -1.verbose: verbosity level. Can be 0 for no information, 1 for printing losses, and 2 for plotting shape functions every 5 iterations.ExamplesBasic regression exampleIn the following, we use the common diabetes dataset from sklearn (https://scikit-learn.org/0.16/modules/generated/sklearn.datasets.load_diabetes.html). After loading the dataset viaimport pandas as pd
from sklearn.datasets import load_diabetes
from sklearn.preprocessing import StandardScaler

X, y = load_diabetes(return_X_y=True, as_frame=True)
scaler = StandardScaler()
X_names = X.columns

X = scaler.fit_transform(X)
X = pd.DataFrame(X, columns=X_names)
X['sex'] = X.sex.apply(lambda x: 'w' if x > 0 else 'm')and !important scale the target valuesy = (y - y.mean()) / y.std()we can simply initialize and fit IGANN withfrom igann import IGANN
model = IGANN(task='regression')
model.fit(X, y)Withmodel.plot_single(plot_by_list=['age', 'bmi', 'bp', 'sex', 's1', 's2'])we obtain the following shape functionsScikit-Learn Integration with IGANNor scikit-learn users, we offer IGANNClassifier and IGANNRegressor classes. These are optimized for scikit-learn's ecosystem, ensuring full compatibility with its tools and conventions. IGANNClassifier is ideal for classification tasks, and IGANNRegressor for regression. Both integrate smoothly with scikit-learn's features like cross-validation and grid search, allowing easy incorporation of IGANN's capabilities into your machine learning projects.Import them directly from the igann package:from igann import IGANNClassifier, IGANNRegressor
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import load_iris

# Load sample data
X, y = load_iris(return_X_y=True)

# Initialize the IGANNClassifier
igann_classifier = IGANNClassifier()

# Define the parameter grid to search
param_grid = {
    'learning_rate': [0.01, 0.1, 0.2],
    'n_iter_no_change': [10, 20, 30],
    # add other parameters you wish to tune
}

# Create GridSearchCV object
grid_search = GridSearchCV(igann_classifier, param_grid, cv=5, scoring='accuracy')

# Perform grid search on the data
grid_search.fit(X, y)

# Best parameters and score
print("Best parameters:", grid_search.best_params_)
print("Best score:", grid_search.best_score_)Citations@article{kraus2023interpretable,
  title={Interpretable Generalized Additive Neural Networks},
  author={Kraus, Mathias and Tschernutter, Daniel and Weinzierl, Sven and Zschech, Patrick},
  journal={European Journal of Operational Research},
  year={2023},
  publisher={Elsevier}}