In this blog, we will discuss how to checkpoint your model in Keras using ModelCheckpoint callbacks. Check-pointing your work is important in any field. If by-chance any problem or failure occurs, you don’t need to restart your work from zero, just resume from that checkpoint. This is very important in the field of deep learning where training can take days. So, let’s see how to use this.
Keras Function
Keras provides a built-in function for model check-pointing as
|
1 |
ModelCheckpoint(filepath, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1) |
Let’s discuss in detail each of its arguments:
filepath: This is the path to save your model. Depending on the filepath specified, we can either save only the best model or save models at every epoch. Let’s see what this means.
If you specified the filepath as fixed, for example, ‘D:/best_model.hdf5’, this will overwrite your previous best model and what you end up is the best model up to that epoch.
If you specified a dynamic filepath, say, ‘D:/model{epoch:02d}.hdf5’, this will save the model at every epoch. For instance, for epoch 22, the model will be saved as model22.hdf5. You can only use variables like ‘epoch’ or keys in logs during training such as ‘loss’, ‘acc’, ‘val_loss’ and ‘val_acc’ for formatting the filepath. For example, ‘D:/model-{epoch:02d}-{val_acc:.2f}.hdf5’ is a valid filepath.
monitor: This is the quantity to monitor. This can take one of the values from ‘loss’, ‘acc’, ‘val_loss’ and ‘val_acc’.
verbose: This thing controls whether some information about model saving will be displayed or not. This is either 0 or 1. If 0, nothing will be displayed and for 1 something like this will be displayed depending on the behavior of the monitored quantity.
save_best_only: If set to false, then model after every epoch will be saved whether the monitored quantity increases or decreases. Otherwise, it will save the model depending on the ‘mode’ argument.
mode: This can take one of the values from auto, min, max. For instance, if the mode is ‘max’ and ‘val_acc’ is the monitored quantity, then for save_best_only = True the model will be saved only when ‘val_acc’ improves, otherwise, the model will not be saved at that epoch. For ‘val_loss’, this should be min. ‘auto’ mode automatically decides the direction depending on the monitored quantity.
save_weights_only: if True, then only the model weights will be saved otherwise the full model will be saved.
period: The callback will be applied after the specified period (no. of epochs)
How to use this?
- All the callbacks are available in the keras.callbacks module so first import the ModelCheckpoint function from this module.
- Then properly set up the function arguments.
- Now, to apply this you need to pass this as a list in the .fit() method.
|
1 2 3 4 5 |
from keras.callbacks import ModelCheckpoint filepath = 'D:/downloads/best_weights.hdf5' checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='auto',period=1) model.fit(train_x, train_y, epochs=10, batch_size=128, callbacks=[checkpoint]) |
Let’s take MNIST classification example to understand this
Import Libraries
|
1 2 3 4 5 6 |
import keras import numpy as np import matplotlib.pyplot as plt from keras.layers import Dense, Flatten, Conv2D, MaxPool2D from keras.models import Sequential from keras.datasets import mnist |
Data Loading and Pre-processing
|
1 2 3 4 5 6 7 8 |
(train_x, train_y),(test_x, test_y) = mnist.load_data() # Normalize between 0 and 1 train_x = train_x.astype('float32')/255 test_x = test_x.astype('float32')/255 # Add the channel dimension train_x = np.expand_dims(train_x,axis=-1) test_x = np.expand_dims(test_x,axis=-1) |
Build Model
|
1 2 3 4 5 6 7 8 9 10 |
model = Sequential() model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1))) model.add(MaxPool2D((2,2))) model.add(Conv2D(64,(3,3),activation='relu')) model.add(MaxPool2D((2,2))) model.add(Flatten()) model.add(Dense(512,activation='relu')) model.add(Dense(10,activation='softmax')) model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy']) |
Callbacks
|
1 2 3 |
from keras.callbacks import ModelCheckpoint filepath = 'D:/downloads/best_model.hdf5' checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='max',period=1) |
Fit Model
|
1 |
record = model.fit(train_x, train_y,validation_split=0.20, epochs=10, batch_size=128,callbacks=[checkpoint]) |
Load Weights and Evaluate test set
|
1 2 |
model.load_weights('D:/downloads/best_model.hdf5') model.evaluate(test_x,test_y) |
Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.