Keras is a high level API, can run on top of Tensorflow, CNTK and Theano. Keras is preferable because it is easy and fast to learn. In this blog we will learn a set of functions named as callbacks, used during training in Keras.
Callbacks provides some advantages over normal training in keras. Here I will explain the important ones.
- Callback can terminate a training when a Nan loss occurs.
- Callback can save the model after every epoch, also you can save the best model.
- Early Stopping: Callback can stop training when accuracy stops improving.
Terminate the training when Nan loss occurs
Let’s see the code how to terminate when Nan loss occurs while training:
|
1 |
keras.callbacks.TerminateOnNaN() |
Saving Model using Callbacks
To save model after every epoch in keras, we need to import ModelCheckpoint from keras.callbacks. Let’s see the below code which will save the model if validation loss decreases.
|
1 2 3 4 5 6 7 8 |
from keras.callbacks import ModelCheckpoint filepath="weights-{epoch:02d}-{val_acc:.2f}.hdf5" checkpoint = ModelCheckpoint(filepath=filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max') callbacks_list = [checkpoint] model.compile(optimizer = 'sgd', loss = 'binary_crossentropy') model.fit(train_data, train_label, epochs = 10, batch_size = 64, callbacks = callbacks_list) |
In the above code first we have created a ModelCheckpoint object by passing its required parameters.
- “filepath” defines the path where all checkpoints will be saved. If you want to save only the best model, then directly pass filepath with
name “best_model.hdf5” which will overwrite the previous saved checkpoints. - “monitor” will decide which quantity you want to monitor while training.
- “save_best_only” only saves if validation loss decreases.
- mode with {auto, min, max} when chosen max, will stop training when monitored quantity stops increasing.
Then finally make a callback list and pass it into model.fit() with parameter callbacks.
Early Stopping
Callbacks can stop training when a monitored quantity has stopped improving. Lets see how:
|
1 |
keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto', baseline=None, restore_best_weights=False) |
- min_delta: It is the minimum quantity which will be taken for improvement to be conceded.
- patience: after this number of epochs if training does not improve, it will stop.
- mode: in auto mode, the direction will be decided by monitored quantity.
- “baseline” the baseline value over which no improvement will stop the training.
Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.