Automating Hyperparameter Tuning of Keras Model | by Himanshu Sharma | Towards Data Science

Building a model is of no use if you cannot optimize it for a good performance and a higher accuracy. Generally, the model building requires less time than optimizing that model, because during optimization or tuning the model you need to look out for the best parameters which is a time-consuming process.
We can automate this process of finding out the best values for the hyperparameters and getting the highest accuracy of the Keras model. In this article, we will be discussing Hyperas which is an open-source python package used for automating the process of Keras Model Hyperparameter Tuning.
Let’s get started......
Like any other python library, we will use pip installation to install hyperas. Before installation, we need to run a command which is required to avoid errors at a later stage, make sure this command is the first code that you run while building this.
from __future__ import print_function!pip install hyperas
In order to run hyperas, we need to change certain settings and also we will use a file named exactly the same as our collab notebook i.e if my collab notebook is named Hyperas.ipynb then in the below code, I will use Hyperas.ipynb as the file name.
!pip install -U -q PyDrivefrom pydrive.auth import GoogleAuthfrom pydrive.drive import GoogleDrivefrom google.colab import authfrom oauth2client.client import GoogleCredentials# Authenticate and create the PyDrive client.auth.authenticate_user()gauth = GoogleAuth()gauth.credentials = GoogleCredentials.get_application_default()drive = GoogleDrive(gauth)# Copy/download the filefid = drive.ListFile({'q':"title='Hyperas.ipynb'"}).GetList()[0]['id']f = drive.CreateFile({'id': fid})f.GetContentFile('Hyperas.ipynb')
Next, we will import all the required dependencies for this article.
import numpy as npfrom hyperopt import Trials, STATUS_OK, tpefrom keras.datasets import mnistfrom keras.layers.core import Dense, Dropout, Activationfrom keras.models import Sequentialfrom keras.utils import np_utilsfrom hyperas import optimfrom hyperas.distributions import choice, uniform
In this step, we will create two functions that will help us in loading the data and creating the model respectively.
def data():    (x_train, y_train), (x_test, y_test) = mnist.load_data()    x_train = x_train.reshape(60000, 784)    x_test = x_test.reshape(10000, 784)    x_train = x_train.astype('float32')    x_test = x_test.astype('float32')    x_train /= 255    x_test /= 255    nb_classes = 10    y_train = np_utils.to_categorical(y_train, nb_classes)    y_test = np_utils.to_categorical(y_test, nb_classes)    return x_train, y_train, x_test, y_test
def create_model(x_train, y_train, x_test, y_test):        model = Sequential()    model.add(Dense(512, input_shape=(784,)))    model.add(Activation('relu'))    model.add(Dropout({{uniform(0, 1)}}))    model.add(Dense({{choice([256, 512, 1024])}}))    model.add(Activation({{choice(['relu', 'sigmoid'])}}))    model.add(Dropout({{uniform(0, 1)}}))    if {{choice(['three', 'four'])}} == 'four':        model.add(Dense(100))model.add({{choice([Dropout(0.5), Activation('linear')])}})        model.add(Activation('relu'))model.add(Dense(10))    model.add(Activation('softmax'))model.compile(loss='categorical_crossentropy', metrics=['accuracy'],                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})result = model.fit(x_train, y_train,              batch_size={{choice([64, 128])}},              epochs=2,              verbose=2,              validation_split=0.1)    #get the highest validation accuracy of the training epochs    validation_acc = np.amax(result.history['val_accuracy'])     print('Best validation acc of epoch:', validation_acc)    return {'loss': -validation_acc, 'status': STATUS_OK, 'model': model}
This is the final step where we will use hyperas and find out the best parameters and highest accuracy.
if __name__ == '__main__':    best_run, best_model = optim.minimize(model=create_model,                                      data=data,                                      algo=tpe.suggest,                                      max_evals=5,                                      trials=Trials(),                                      notebook_name='Hyperas')    X_train, Y_train, X_test, Y_test = data()    print("Evalutation of best performing model:")    print(best_model.evaluate(X_test, Y_test))    print("Best performing model chosen hyper-parameters:")    print(best_run)
Here you can clearly analyze how hyperas displayed the best performing hyperparameters along with the accuracy of the model. This is easy and can save time and effort.
Go ahead try this and let me know your experiences in the response section.
This article is in collaboration with Piyush Ingale
Thanks for reading! If you want to get in touch with me, feel free to reach me on hmix13@gmail.com or my LinkedIn Profile. You can view my Github profile for different data science projects and packages tutorials. Also, feel free to explore my profile and read different articles I have written related to Data Science.