How data normalization affects your Random Forest algorithm | by Javier Fernandez | Towards Data Science

Recently, I was implementing a Random Forest regressor when I faced the classical question of: Should I implement data normalization?.
As far as I knew, decision tree-based algorithms do not need, in general, normalization. But, I realized that every time I heard this statement it was related to classifiers so, what about regressors? Do they need data normalization?
If you search on the internet Random Forest normalization, the first links indicate that:
Stack Overflow: (1) No, scaling is not necessary for random forests, (2) Random Forest is a tree-based model and hence does not require feature scaling.
Stack Exchange: Thanks for the clarification by commenting. Tree-based models do not care about the absolute value that a feature takes. They only care about the order of the values. Hence, normalization is used mainly in linear models/KNN/neural networks because they’re affected by absolute values taken by features.
However, after deepening my research, I discovered that data normalization may affect the output. So, here I leave my takeaway and a short demonstration in Python.
For classification tasks, the output of the random forest is the class selected by most trees. For regression tasks, the mean or average prediction of the individual trees is returned.
Therefore, data normalization won’t affect the output for Random Forest classifiers while it will affect the output for Random Forest regressors.
Regarding the regressor, the algorithm will be more affected by the high-end values if the data is not transformed. This means that they will probably be more accurate in predicting high values than low values.
Consequently, transformations such as log-transform will reduce the relative importance of these high values, hence generalizing better.
Here I present a short experiment that shows that the output changes for regressors when implementing data normalization but it is the same for classifiers.
Importing the libraries
Loading the dataset
The dataset selected is the Boston House-prices dataset, used for regression tasks.
Keys: dict_keys(['data', 'target', 'feature_names', 'DESCR', 'filename'])Feature names: ['CRIM' 'ZN' 'INDUS' 'CHAS' 'NOX' 'RM' 'AGE' 'DIS' 'RAD' 'TAX' 'PTRATIO' 'B' 'LSTAT']Data shape: (506, 13)Target shape: (506,)
To use this dataset also for classification, the target variable has been rounded.
Random Forest classifier
Here is the implementation of the Random Forest classifier under three conditions: (1) no normalization, (2) min-max normalization, and (3) standardization.
As observed, the data normalization does not affect the accuracy score.
Accuracy score without normalization: 0.12574850299401197Accuracy score with min-max normalization: 0.12574850299401197Accuracy score with standardization: 0.12574850299401197
Random Forest regressor
Here is the implementation of the Random Forest regressor under three conditions: (1) no normalization, (2) min-max normalization, and (3) standardization.
In this case, data normalization affects the mean squared score of the regressor.
Accuracy score without normalization: 13.38962275449102Accuracy score with min-max normalization: 13.478456820359284Accuracy score with standardization: 13.38586179640719