User Should Have An Option To Assign Different criterions With Different Percentage Of Trees In Random Forest

## Describe the workflow you want to enable

### **Detailed Explanation Of Proposed Workflow**

User can mention how many percentage of trees in `sklearn.ensemble.RandomForestClassifier` & `sklearn.ensemble.RandomForestRegressor` will follow which `criterion`

### **Advantages Of Implementing Above Functionality**

Better results can be achieved in certain domains and this feature will help reserchers

## Describe your proposed solution

### **This Is How The Feature Will Look At User's End When Coding In Python3**

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

# Load dataset
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Give multiple criterion
n_estimators = 100

rfc = RandomForestClassifier(n_estimators=n_estimators, criterion={"gini": 0.4, "entropy": 0.3, "random": 0.3}, random_state=42)

# Model training
rfc.fit(X_train, y_train)

# Prediction
print(rfc.predict(X_test))
```

### **Explanation Of Above Code**

After implementation of this new feature, `criterion` parameter will also accept a `dict` where percentage can be passed as value for a particular criterion as key
If sum of all values is less than 1 then percentage of trees left will follow default `criterion`
and if it's more than 1 then an error will be raised

In above code, other than `gini` and `entropy`, there is a `random` criterion also where each tree falling under `random` criterion can have any random `criterion`


## Describe alternatives you've considered, if relevant

### **Alternative Code Using `np.argmax`**

```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.ensemble import VotingClassifier

# Load dataset
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Create Random Forest classifiers with Gini and Entropy
n_estimators = 100

rf_gini = RandomForestClassifier(n_estimators=n_estimators // 2, criterion="gini", random_state=42)
rf_entropy = RandomForestClassifier(n_estimators=n_estimators // 2, criterion="entropy", random_state=42)

# Fit the models
rf_gini.fit(X_train, y_train)
rf_entropy.fit(X_train, y_train)

# Predict with both models
pred_gini = rf_gini.predict(X_test)
pred_entropy = rf_entropy.predict(X_test)

# Average the predictions (convert to probabilities first, then average, then take the argmax)
avg_proba = (rf_gini.predict_proba(X_test) + rf_entropy.predict_proba(X_test)) / 2
avg_pred = np.argmax(avg_proba, axis=1)

# Majority voting (directly count the most common prediction)
from scipy.stats import mode
majority_pred = mode([pred_gini, pred_entropy])[0][0]

# Evaluate accuracy
accuracy_avg = accuracy_score(y_test, avg_pred)
accuracy_majority = accuracy_score(y_test, majority_pred)

print(f"Accuracy using averaged probabilities: {accuracy_avg:.2f}")
print(f"Accuracy using majority voting: {accuracy_majority:.2f}")
```

## Additional context

**I Will Be Able To Make A PR Once The Issue Gets Approval**

COMMENT:
I would say this is solved by the `VotingClassifier`, see https://scikit-learn.org/stable/modules/ensemble.html#voting-classifier. Each tree in a RF is independent anyway, so you can train 2 RF with the criteria of your choice and the combine by the VotingClassifier.

Labels: New Feature, Needs Triage