[Chapter-2] Fine-Tune Your Machine Learning Model — Part-5🧙🏻‍♂️
By: Vishvdeep Dasadiya
Tags: Scikit Learn, Machine Intelligence, Data Science, Machine, Python

Grid Search 🕵️‍♂️

One way to tune your machine learning model that would be to fiddle with the hyper-parameters manually 👷, until you find a good combination of hyper-parameter values. This work will be very tedious and you may not have time to explore plethora of combinations. Sounds very hectic right🥴.

Instead you can use Scikit-Learn’s GridSearchCV to do that for you😍. All we need to do is tell which hyper-parameters we want to experiment with, and what values to try out, sounds very cool right. Let’s have quick look at the small example.

from sklearn.model_selection import GridSearchCV param_grid = [

{'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},

{'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},

] forest_reg = RandomForestRegressor() grid_search = GridSearchCV(forest_reg, param_grid, cv=5,

scoring='neg_mean_squared_error',

return_train_score=True) grid_search.fit(housing_prepared, housing_labels)

When you have no idea what value s hyper-parameter should have, a simple approach would be much better😮. Here, param_grid tells Scikit-Learn to evaluate 3 x 4 = 12 combinations of n_estimators and max_features hyper-parameter values specified in the dictionary. Do not worry about what these hyper-parameters mean as of now. After that try all 2 x 3 = 6 combinations of hyper-parameter values in the second dictionary, however this time with bootstrap hyper-parameter set to False instead of True🤗.

All in all, the grid search will explore combinations of Random Forest Regression hyper-parameter values, and it will train each model five times😲.

>>> grid_search.best_params_

{'max_features': 8, 'n_estimators': 30}

We can find out about best parameter by trying out. Below will be to find about best estimator.👩‍🏫

>>> grid_search.best_estimator_ RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,

max_features=8, max_leaf_nodes=None, min_impurity_decrease=0.0,

min_impurity_split=None, min_samples_leaf=1,

min_samples_split=2, min_weight_fraction_leaf=0.0,

n_estimators=30, n_jobs=None, oob_score=False, random_state=None,

verbose=0, warm_start=False)

And of course the evaluation results are also available:👨‍💼

>>> cvres = grid_search.cv_results_ >>> for mean_score, params in zip(cvres["mean_test_score"], cvres["params"]):



>>> print(np.sqrt(-mean_score), params)



63669.05791727153 {'max_features': 2, 'n_estimators': 3}

55627.16171305252 {'max_features': 2, 'n_estimators': 10}

53384.57867637289 {'max_features': 2, 'n_estimators': 30}

60965.99185930139 {'max_features': 4, 'n_estimators': 3}

52740.98248528835 {'max_features': 4, 'n_estimators': 10}

50377.344409590376 {'max_features': 4, 'n_estimators': 30}

58663.84733372485 {'max_features': 6, 'n_estimators': 3}

52006.15355973719 {'max_features': 6, 'n_estimators': 10}

50146.465964159885 {'max_features': 6, 'n_estimators': 30}

57869.25504027614 {'max_features': 8, 'n_estimators': 3}

51711.09443660957 {'max_features': 8, 'n_estimators': 10}

49682.25345942335 {'max_features': 8, 'n_estimators': 30}

62895.088889905004 {'bootstrap': False, 'max_features': 2, 'n_estimators': 3}

54658.14484390074 {'bootstrap': False, 'max_features': 2, 'n_estimators': 10}

59470.399594730654 {'bootstrap': False, 'max_features': 3, 'n_estimators': 3}

52725.01091081235 {'bootstrap': False, 'max_features': 3, 'n_estimators': 10}

57490.612956065226 {'bootstrap': False, 'max_features': 4, 'n_estimators': 3}

51009.51445842374 {'bootstrap': False, 'max_features': 4, 'n_estimators': 10}

In this example, fortunately we obtain the best solution by setting the max_features hyper-parameter to 8 and the n_estimators hyper-parameter to 30. The RMSE score for this is 49,682. Congratulations, you have successfully fine-tuned your best model!👨‍🚀