Isolation Forest Contamination Rate has no effect on AUC

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/28172

<div type='discussions-op-text'>

<sup>Originally posted by **robertken** January 18, 2024</sup>
I am experiencing some unexpected behavior with the Isolation Forest. I'm using sklearn 1.3.0. When i adjust the contamination rate, the only hyperparameter of IF, there is no effect on AUC or AUPRC, which are score based metrics, but the predicted value-based metrics, like TPR, does change. See my example below. 

Not sure if posting here in _Discussions_ or as an _Issue_ is most appropriate.

```python
import numpy as np
import pandas as pd
from sklearn.ensemble import IsolationForest
from sklearn.metrics import roc_auc_score, average_precision_score, confusion_matrix, accuracy_score, f1_score
from sklearn.model_selection import StratifiedKFold
from sklearn.datasets import make_classification

# Set a random seed
random_seed = 42
np.random.seed(random_seed)

# Generate a synthetic dataset
X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=random_seed)
y = np.where(y == 0, -1, 1)  # Adjusting labels for Isolation Forest

# Define contamination rates
contamination_rates = [0.01, 0.02, 0.05, 0.1, 0.2]

# Stratified K-Fold for cross-validation
n_splits = 5
skf = StratifiedKFold(n_splits=n_splits, shuffle=True, random_state=random_seed)

# Function to calculate TPR, FPR, TNR, FNR
def calculate_rates(cm):
    TN, FP, FN, TP = cm.ravel()
    TPR = TP / (TP + FN)
    FPR = FP / (FP + TN)
    TNR = TN / (TN + FP)
    FNR = FN / (TP + FN)
    return TPR, FPR, TNR, FNR

# Iterate over different contamination rates
for contamination in contamination_rates:
    print(f"\nEvaluating model with contamination rate: {contamination}")
    auc_scores = []
    auprc_scores = []
    f1_scores = []
    accuracy_scores = []
    rates = []

    for train_index, test_index in skf.split(X, y):
        X_train, X_test = X[train_index], X[test_index]
        y_train, y_test = y[train_index], y[test_index]

        # Train Isolation Forest
        model = IsolationForest(contamination=contamination, random_state=random_seed)
        model.fit(X_train)

        # Predict
        y_pred = model.predict(X_test)
        y_pred = np.where(y_pred == -1, 1, 0)  # Convert to binary format

        # Adjust y_test for binary classification metrics
        y_test_adj = np.where(y_test == -1, 1, 0)

        # Calculate AUC and AUPRC
        scores_pred = -model.decision_function(X_test)
        auc = roc_auc_score(y_test_adj, scores_pred)
        auprc = average_precision_score(y_test_adj, scores_pred)
        auc_scores.append(auc)
        auprc_scores.append(auprc)

        # Calculate F1 and Accuracy
        f1 = f1_score(y_test_adj, y_pred)
        accuracy = accuracy_score(y_test_adj, y_pred)
        f1_scores.append(f1)
        accuracy_scores.append(accuracy)

        # Calculate Confusion Matrix and Rates
        cm = confusion_matrix(y_test_adj, y_pred)
        rates.append(calculate_rates(cm))

    # Average metrics
    average_auc = np.mean(auc_scores)
    average_auprc = np.mean(auprc_scores)
    average_f1 = np.mean(f1_scores)
    average_accuracy = np.mean(accuracy_scores)
    average_rates = np.mean(rates, axis=0)

    print(f"Average AUC for contamination {contamination}: {average_auc}")
    print(f"Average AUPRC for contamination {contamination}: {average_auprc}")
    print(f"Average F1 for contamination {contamination}: {average_f1}")
    print(f"Average Accuracy for contamination {contamination}: {average_accuracy}")
    print(f"Average Rates (TPR, FPR, TNR, FNR) for contamination {contamination}: {average_rates}")
```

This outputs:

```
Evaluating model with contamination rate: 0.01
Average AUC for contamination 0.01: 0.45271999999999996
Average AUPRC for contamination 0.01: 0.48626406127580835
Average F1 for contamination 0.01: 0.023233444321124674
Average Accuracy for contamination 0.01: 0.5
Average Rates (TPR, FPR, TNR, FNR) for contamination 0.01: [0.012 0.012 0.988 0.988]

Evaluating model with contamination rate: 0.02
Average AUC for contamination 0.02: 0.45271999999999996
Average AUPRC for contamination 0.02: 0.48626406127580835
Average F1 for contamination 0.02: 0.038230738316403653
Average Accuracy for contamination 0.02: 0.501
Average Rates (TPR, FPR, TNR, FNR) for contamination 0.02: [0.02  0.018 0.982 0.98 ]

Evaluating model with contamination rate: 0.05
Average AUC for contamination 0.05: 0.45271999999999996
Average AUPRC for contamination 0.05: 0.48626406127580835
Average F1 for contamination 0.05: 0.10474786386720276
Average Accuracy for contamination 0.05: 0.5069999999999999
Average Rates (TPR, FPR, TNR, FNR) for contamination 0.05: [0.058 0.044 0.956 0.942]

Evaluating model with contamination rate: 0.1
Average AUC for contamination 0.1: 0.45271999999999996
Average AUPRC for contamination 0.1: 0.48626406127580835
Average F1 for contamination 0.1: 0.16572749935850517
Average Accuracy for contamination 0.1: 0.5
Average Rates (TPR, FPR, TNR, FNR) for contamination 0.1: [0.1 0.1 0.9 0.9]

Evaluating model with contamination rate: 0.2
Average AUC for contamination 0.2: 0.45271999999999996
Average AUPRC for contamination 0.2: 0.48626406127580835
Average F1 for contamination 0.2: 0.24898883246003756
Average Accuracy for contamination 0.2: 0.469
Average Rates (TPR, FPR, TNR, FNR) for contamination 0.2: [0.178 0.24  0.76  0.822]

```
</div>

COMMENT:
I'm closing since I don't think that we should consider it as an issue until it is investigated.

Labels: Needs Triage