A High Level Overview of Keras ModelCheckpoint Callback
By: Vishnuvardhan Janapati
Tags: Machine Learning, TensorFlow, Data Science, Keras, Programming

3. How to use ModelCheckpoint callback?

The arguments of ModelCheckpoint callback are as shown below (from the TensorFlow website). Try to play with these arguments to learn more.

filepath: string or PathLike , path to save the model file. filepath can contain named formatting options, which will be filled the value of epoch and keys in logs (passed in on_epoch_end ). For example: if filepath is weights.{epoch:02d}-{val_loss:.2f}.hdf5 , then the model checkpoints will be saved with the epoch number and the validation loss in the filename. monitor: quantity to monitor. verbose: verbosity mode, 0 or 1. save_best_only: if save_best_only=True , the latest best model according to the quantity monitored will not be overwritten. If filepath doesn't contain formatting options like {epoch} then filepath will be overwritten by each new better model. mode: one of {auto, min, max}. If save_best_only=True , the decision to overwrite the current save file is made based on either the maximization or the minimization of the monitored quantity. For val_acc , this should be max , for val_loss this should be min , etc. In auto mode, the direction is automatically inferred from the name of the monitored quantity. save_weights_only: if True, then only the model’s weights will be saved ( model.save_weights(filepath) ), else the full model is saved ( model.save(filepath) ). save_freq: 'epoch' or integer. When using 'epoch' , the callback saves the model after each epoch. When using integer, the callback saves the model at end of this many batches. If the Model is compiled with experimental_steps_per_execution=N , then the saving criteria will be checked every N th batch. Note that if the saving isn't aligned to epochs, the monitored metric may potentially be less reliable (it could reflect as little as 1 batch, since the metrics get reset every epoch). Defaults option is 'epoch' . **kwargs: Additional arguments for backwards compatibility. Possible key is period .

Consider the following simple model with mnist data. The full code used in this article is provided here.

Simple Keras model

3.1 Simple Checkpoint ( tf format) — weights only

In this section we will see how to checkpoint at the end of every epoch and load the checkpoint to verify the performance. Notice that checkpoint name is fixed ( cp.ckpt ) which means the weights are updated after each epoch and overwrite the same checkpoint. Also, notice that checkpoints are saved in tf format or ckpt format. Later we will see how to save checkpoints in h5 format.

Please note that in the above code when we change save_weights_only=True to save_weights_only=False will result in saving entire model.

Please note that saving best model is not possible incase of subclassed model. Most probably this will be implemented soon. However, this is not an issue if your are using Keras Sequential or Functional model APIs.

Loading the above checkpoint and evaluating the model is as shown below

3.2 Checkpoint every epoch ( tf format) — weights only

Checkpointing every epoch is similar to the Section 3.1 except that we need to provide epoch number in the checkpoint path as shown below.

As mentioned in the last section, when we change save_weights_only=True to save_weights_only=False will result in saving entire model.

As we have one checkpoint for every epoch, we can use any checkpoint to evaluate the model. The following example shows how to access latest checkpoint and evaluate model performance.

Moreover, you could also include other parameters like val_loss in the file path as shown below.

3.3 Checkpoint every 100 batches ( tf format) — weights only

Some use-cases requires the entire model or weights to be saved after each batch. Similar to checkpointing after every epoch, you could also checkpoint after every N (for example N =100) batches as shown below.

Similarly, we can also checkpoint every N (say 5) epochs as shown below. The parameter save_freq needs to be set as save_freq=N*batch_size. Please note that N needs to be an integer.

3.4 Checkpoint best model weights ( tf format)

Checkpointing best model is one of the widely used approach by many users. The following code shows how to checkpoint best model. For checkpointing best model, we need to provide the quantity that needs to be monitored ( monitor=’val_accuracy’ ) and select save_best_only = True as shown below. Please note that for demonstrating purpose we have epoch:04d in the checkpoint_path_4 but we don’t need it.

Output from the above code is as follows. After first epoch you can notice val_accuracy improved from ......... saving model to .... where it saved the model as best model. During second epoch, val_accuracy did not improve when compared to first epoch. So, model was not saved after second epoch. After third epoch, val_accuracy improved and the model was saved.

Couple of things to note here is that we must provide save_best_only = True and what to monitor . Based on the loss function defined in model.compile , Keras will infer the mode (minimization or maximization) if that was not provided by the user.

3.5 Checkpoint every epoch ( h5 format) — weights only

This is similar to Section 3.2. Only change is providing h5 extension as shown below.

I think this is more than enough for this article. I have posted the entire code here that I used for this article.