Tag Archives: callbacks

Create custom callbacks in Keras

In this blog, we will discuss how to create custom callbacks in Keras. This is actually very simple. You just need to create a class that takes keras.callbacks.Callback() as its base class. The set of methods that we can use is also fixed. We just need to write the logic. Let’s understand this with the help of an example. Here, we will create a callback that stops the training when the accuracy has reached a threshold and prints the message.

class myCallback(keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs={}):
        if logs.get('acc')>0.99:
            self.model.stop_training = True
            print('Stopped training as accuracy above threshold')
    
callbacks1 = myCallback()

class myCallback(keras.callbacks.Callback):

def on_epoch_end(self, epoch, logs={}):

if logs.get('acc')>0.99:

self.model.stop_training = True

print('Stopped training as accuracy above threshold')

callbacks1 = myCallback()

In “Line-1“, we create a class “mycallback” that takes keras.callbacks.Callback() as its base class.

In “Line-2“, we define a method “on_epoch_end”. Note that the name of the functions that we can use is already predefined according to their functionality. For instance, if we define a function by the name “on_epoch_end“, then this function will be implemented at the end of every epoch. If you change this function name to “on_epoch_end1“, then this function will not be implemented.

Below are some of the method names that we can use. The name of these functions is aptly named according to their functionality. The arguments that they can take is already fixed.

The epoch and batch arguments refer to the current epoch and batch number. And “logs” is a dictionary that records all the training events like “loss”, “acc” etc.

In “Line-3,4“, we define a stopping condition and if met stop training the model. Note that we can access the model being trained through the base class. And so we can use any other model properties like save_weights, save, trainable, etc.

At last, we create an instance of this class and pass this instance as a list in the fit() method. Below is the output of applying the above callback.

Below is another example that saves the weights at the end of each epoch.

class myCallback(keras.callbacks.Callback):
    def on_batch_end(self, batch,logs={}):
        self.model.save_weights('D:/downloads/best{}.hdf5'.format(batch))
    
callbacks1 = myCallback()

class myCallback(keras.callbacks.Callback):

def on_batch_end(self, batch,logs={}):

self.model.save_weights('D:/downloads/best{}.hdf5'.format(batch))

callbacks1 = myCallback()

This way you can create many other custom callbacks in keras. This gives you more control over the training process. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – ReduceLROnPlateau

Leave a reply

As we already know that the neural networks suffer greatly from the problem of plateaus which substantially slows down the training process. Thus, the selection of a good learning rate becomes a really challenging task. Many methods have been proposed to counter this problem such as using cyclic learning rates where we vary learning rates between reasonable boundary values, or other methods like Stochastic Gradient Descent with Warm Restarts, etc.

Keras also provides ReduceLROnPlateau callback that reduces the learning rate by some factor whenever the learning stagnates. It is believed that sometimes our model will benefit from lowering the learning rate when trapped in the plateau region. So, let’s discuss its Keras API.

keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=10, verbose=0, mode='auto', min_delta=0.0001, cooldown=0, min_lr=0)

1	keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=10, verbose=0, mode='auto', min_delta=0.0001, cooldown=0, min_lr=0)

This callback “monitors” a quantity and if no improvement is seen for a ‘patience‘ number of epochs, the learning rate is reduced by the “factor” specified. Improvement is specified by the “min_delta” argument. No improvement is considered if the change in the monitored quantity is less than the min_delta specified. This also has an option whether you want to start evaluating the new LR instantly or give some time to the optimizer to crawl with the new LR and then evaluate the monitored quantity. This is done using the “cooldown” argument. You can also set the lower bound on the LR using the “min_lr” argument. No matter how many epochs or what reduction factor you use, the LR will never decrease beyond “min_lr“.

Now, let’s see how to use this.

# Load data, preprocessing and build model
...

# First, create an instance of this ReduceLROnPlateau class
from keras.callbacks import ReduceLROnPlateau
call = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=20, verbose=0, mode='auto', min_delta=0.0001, cooldown=0, min_lr=0.0001)

#Then pass this as a list in the fit() method
record = model.fit(..., callbacks=[call],...)

# Load data, preprocessing and build model

...

# First, create an instance of this ReduceLROnPlateau class

from keras.callbacks import ReduceLROnPlateau

call = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=20, verbose=0, mode='auto', min_delta=0.0001, cooldown=0, min_lr=0.0001)

#Then pass this as a list in the fit() method

record = model.fit(..., callbacks=[call],...)

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – LambdaCallback

Leave a reply

Keras has provided several builtin classes/callbacks that serves our purpose for most of the cases. But let’s say we want to stop training when the accuracy has reached a benchmark or save the model at each batch. These tasks cannot be achieved using the builtin callbacks. In that case, we need to create our own callback function. In Keras, we can easily create custom callbacks using keras.callbacks.Callback() as our base class. But for that case, you need to create a class and write some amount of code.

As an alternative, Keras also provides us with an option to creates simple, custom callbacks on-the-fly. This can be done with ease using the LambdaCallback. So, in this blog, let’s discuss how to use this callback.

Note: For Python 3.8 or higher, lambda function will start supporting assignment expressions and this will make this callback even more powerful.

keras.callbacks.LambdaCallback(on_epoch_begin=None, on_epoch_end=None, on_batch_begin=None, on_batch_end=None, on_train_begin=None, on_train_end=None)

1	keras.callbacks.LambdaCallback(on_epoch_begin=None, on_epoch_end=None, on_batch_begin=None, on_batch_end=None, on_train_begin=None, on_train_end=None)

Here, all the arguments are aptly named according to their work. For instance, in “on_epoch_end” argument, we pass the function that will be called at the end of each epoch.

Now, all of these arguments expect functions with fixed positional arguments which are mentioned below.

Now, let’s see how to use this callback. Below I created a simple callback that saves the model weights when the accuracy is beyond some limit.

from keras.callbacks import LambdaCallback
call = LambdaCallback(on_epoch_end= lambda epochs, logs: model.save_weights('kang.hdf5') if logs.get('acc')>0.99 else None)

1 2	from keras.callbacks import LambdaCallback call = LambdaCallback(on_epoch_end= lambda epochs, logs: model.save_weights('kang.hdf5') if logs.get('acc')>0.99 else None)

Let’s take another example. Here, I’m stopping the training whenever the accuracy has reached a certain point. (For Python 3.8 or higher)

from keras.callbacks import LambdaCallback
call = LambdaCallback(on_epoch_end= lambda epochs, logs: (model.stop_training:=True) if logs.get('acc')>0.99 else None)

1 2	from keras.callbacks import LambdaCallback call = LambdaCallback(on_epoch_end= lambda epochs, logs: (model.stop_training:=True) if logs.get('acc')>0.99 else None)

Similarly, you can create any custom callback. That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – CSVLogger

Leave a reply

In this blog, we will discuss Keras CSVLogger callback. As clear from the name, this streams the training events like ‘loss’, ‘acc’ etc. to a csv file. Using this you can export all the values that can be represented as a string. So, let’s discuss its Keras API.

keras.callbacks.CSVLogger(filename, separator=',', append=False)

1	keras.callbacks.CSVLogger(filename, separator=',', append=False)

Here, the “filename” is the name of the csv file where you want to keep the record. This also gives you an option of how to separate elements in the csv file. You can pass this as a string in the “separator” argument.

This also provides an option of whether to append the training history in an existing file or overwrite the existing file. For instance, if “append=False”, this will overwrite an existing file. Otherwise, it will append the information in the existing file without affecting the previously stored information in that file.

If no existing file is present this will create a new file and then append the information. Now, let’s see how to use this class.

# Load data, preprocessing and build model
...

# First, create an instance of this CSVLogger class
from keras.callbacks import CSVLogger
call = CSVLogger('D:/train.csv', separator=',', append=False)

#Then pass this as a list in the fit() method
record = model.fit(..., callbacks=[call],...)

# Load data, preprocessing and build model

...

# First, create an instance of this CSVLogger class

from keras.callbacks import CSVLogger

call = CSVLogger('D:/train.csv', separator=',', append=False)

#Then pass this as a list in the fit() method

record = model.fit(..., callbacks=[call],...)

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – LearningRateScheduler

Leave a reply

In neural networks, setting up a good learning rate is always a challenging task. If the learning rate is set too high, this can cause undesirable divergent behavior in your loss function or sometimes your model can converge too quickly to a sub-optimal value. If it is set too low, the training process may take a long time. Thus, it often proves sometimes useful to decay the learning rate as the training progresses. This can be done using the Learning rate schedules or the adaptive learning rate methods like SGD, Adam, etc. In this blog, we will only discuss Learning rate schedules.

Learning rate schedules as clear from the name adjusts the learning rates based on some schedule. For instance, time decay, exponential decay, etc. To implement these decays, Keras has provided a callback known as LearningRateScheduler that adjusts the weights based on the decay function provided. So, let’s discuss its Keras API.

keras.callbacks.LearningRateScheduler(schedule, verbose=0)

1	keras.callbacks.LearningRateScheduler(schedule, verbose=0)

Here, the “schedule” is a decay function that takes epoch number and the current learning rate as the input and returns the new learning rate. The verbose argument tells us whether to print the following message when changing the learning rate or not.

Note: This method overwrites the learning rate of the optimizer used.

Now, let’s discuss the schedule argument in more detail. Here, we will use the time decay function. This updates the learning rate by the expression below

Now, let’s see how to use this using the LearningRateScheduler callback. First, create a function that takes epoch and learning rate as arguments as shown below

def time_decay(epoch, initial_lrate):
    decay_rate = 0.01
    new_lrate = initial_lrate/(1+decay_rate*epoch)
    return new_lrate

def time_decay(epoch, initial_lrate):

decay_rate = 0.01

new_lrate = initial_lrate/(1+decay_rate*epoch)

return new_lrate

Then pass this function in the LearningRateScheduler callback as shown below

from keras.callbacks import LearningRateScheduler
lrate = LearningRateScheduler(time_decay,verbose=1)

1 2	from keras.callbacks import LearningRateScheduler lrate = LearningRateScheduler(time_decay,verbose=1)

Now, simply pass this callback as a list in the fit() method as shown below.

record = model.fit(..., callbacks=[lrate], ...)

1	record = model.fit(..., callbacks=[lrate], ...)

You can also check how the learning rate varies by the following command. This returns a list of learning rates over the epochs.

record.history['lr']

1	record.history['lr']

You can also plot the learning rate over epochs using any plotting library.

Similarly, you can use other decays like step decay, exponential decay or any custom decay. Just create a function and pass it to the LearningRateScheduler callback. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – ProgbarLogger

Leave a reply

In this blog, we will discuss Keras ProgbarLogger callback. As clear from the name, this deals with the logging of the progress bar that we usually see during fit() method depending upon the verbosity argument. So, let’s first discuss its API.

keras.callbacks.ProgbarLogger(count_mode='samples', stateful_metrics=None)

1	keras.callbacks.ProgbarLogger(count_mode='samples', stateful_metrics=None)

Here, “count_mode” argument controls whether the progress bar displays the samples seen or the steps. This argument can take one of the values from ‘samples‘ or ‘steps‘. If set equal to ‘steps’, make sure that you provide the “steps_per_epoch” argument in the fit() method. Otherwise, this will give you an error. The ‘steps’ is basically used with generators like fit_generator, etc.

Below is the figure where first I’ve trained on 5000 samples and the count_mode argument is set to “samples“. For the second one, I’ve used 12 steps in the fit_generator for 1 epoch. The count_mode argument is set to “steps“.

The second argument “stateful_metrics” controls whether to display the average value of the metric specified or display its value at the last step of every epoch. This should be passed as an iterable like list etc. For more details, refer to Keras callbacks BaseLogger where we have discussed this argument in detail.

This callback, in turn, calls the Keras Progbar class that controls how to display the progress bar like the width of the progress bar, its update interval, etc. Now, let’s see how to use this.

# Load data, preprocessing and build model
...

# First, create an instance of this ProgbarLogger class
from keras.callbacks import ProgbarLogger
call = ProgbarLogger(count_mode='samples', stateful_metrics=['acc'])

#Then pass this as a list in the fit() method
record = model.fit(..., callbacks=[call],...)

# Load data, preprocessing and build model

...

# First, create an instance of this ProgbarLogger class

from keras.callbacks import ProgbarLogger

call = ProgbarLogger(count_mode='samples', stateful_metrics=['acc'])

#Then pass this as a list in the fit() method

record = model.fit(..., callbacks=[call],...)

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – TerminateOnNaN

Leave a reply

In this blog, we will discuss Keras TerminateOnNaN callback. As clear from the name, this terminates the training when a Nan loss is encountered. Below is the Keras API for this callback.

keras.callbacks.TerminateOnNaN()

1	keras.callbacks.TerminateOnNaN()

This checks the loss at every batch end and if that loss is nan or inf, this callback stops the training. This prints out the batch number at which it stops the training. Something like this will be printed.

Below is the code, taken from Keras that shows how this works.

class TerminateOnNaN(Callback):
    def on_batch_end(self, batch, logs=None):
        logs = logs or {}
        loss = logs.get('loss')
        if loss is not None:
            if np.isnan(loss) or np.isinf(loss):
                print('Batch %d: Invalid loss, terminating training' % (batch))
                self.model.stop_training = True

class TerminateOnNaN(Callback):

def on_batch_end(self, batch, logs=None):

logs = logs or {}

loss = logs.get('loss')

if loss is not None:

if np.isnan(loss) or np.isinf(loss):

print('Batch %d: Invalid loss, terminating training' % (batch))

self.model.stop_training = True

Similarly, you can create your own custom callback that tracks some other metrics. Now, let’s see how to use this callback.

# Load data, preprocessing and build model
...

# First, create an instance of this TerminateOnNaN class
from keras.callbacks import TerminateOnNaN
call = TerminateOnNaN()

#Then pass this as a list in the fit() method
record = model.fit(..., callbacks=[call],...)

# Load data, preprocessing and build model

...

# First, create an instance of this TerminateOnNaN class

from keras.callbacks import TerminateOnNaN

call = TerminateOnNaN()

#Then pass this as a list in the fit() method

record = model.fit(..., callbacks=[call],...)

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – BaseLogger

Leave a reply

As we already know that the values of the metrics such as loss, acc, etc. that we get during the training are the averaged values over the epoch. This averaging of the values are automatically applied to every Keras model using the BaseLogger class present under the Keras callbacks. This class also provides us with the flexibility of not averaging the metrics over an epoch. So, in this blog, let’s discuss this BaseLogger class in more detail.

Keras API

keras.callbacks.BaseLogger(stateful_metrics=None)

1	keras.callbacks.BaseLogger(stateful_metrics=None)

Similar to the History callback, this callback is also automatically applied to every Keras model with the default set of arguments. Only if you want to change the arguments, you need to apply it similar to how we applied other callbacks i.e. pass in the fit() method.

Here, stateful_metrics are the name of the metrics that you don’t want to average over an epoch. All the metrics names should be passed in the form of iterable like lists etc. For instance, stateful_metrics=[‘acc’,’loss’].

The value of the stateful_metrics will be saved as-is in on_epoch_end. In other words, the value of the stateful_metric in the last batch before the epoch end will be saved as the final value. All the other remaining metrics will be averaged over the epoch ends. Let’s take a look at the image below.

Here, I trained the model for 1 epoch on the mnist data. The stateful_metrics used is ‘acc’. Clearly, the final logged accuracy (See record.history) is similar to the last batch accuracy (0.9392) and not the average accuracy obtained on the epoch end (0.9405). Hope this makes everything clear. Let’s see how to apply baselogger callback.

# Load data, preprocessing and build model
...

# First, create an instance of this BaseLogger class
from keras.callbacks import BaseLogger
call = BaseLogger(stateful_metrics= ['acc'])

#Then pass this as a list in the fit() method
record = model.fit(..., callbacks=[call],...)

# Load data, preprocessing and build model

...

# First, create an instance of this BaseLogger class

from keras.callbacks import BaseLogger

call = BaseLogger(stateful_metrics= ['acc'])

#Then pass this as a list in the fit() method

record = model.fit(..., callbacks=[call],...)

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – ModelCheckpoint

Leave a reply

In this blog, we will discuss how to checkpoint your model in Keras using ModelCheckpoint callbacks. Check-pointing your work is important in any field. If by-chance any problem or failure occurs, you don’t need to restart your work from zero, just resume from that checkpoint. This is very important in the field of deep learning where training can take days. So, let’s see how to use this.

Keras Function

Keras provides a built-in function for model check-pointing as

ModelCheckpoint(filepath, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)

1	ModelCheckpoint(filepath, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)

Let’s discuss in detail each of its arguments:

filepath: This is the path to save your model. Depending on the filepath specified, we can either save only the best model or save models at every epoch. Let’s see what this means.

If you specified the filepath as fixed, for example, ‘D:/best_model.hdf5’, this will overwrite your previous best model and what you end up is the best model up to that epoch.

If you specified a dynamic filepath, say, ‘D:/model{epoch:02d}.hdf5’, this will save the model at every epoch. For instance, for epoch 22, the model will be saved as model22.hdf5. You can only use variables like ‘epoch’ or keys in logs during training such as ‘loss’, ‘acc’, ‘val_loss’ and ‘val_acc’ for formatting the filepath. For example, ‘D:/model-{epoch:02d}-{val_acc:.2f}.hdf5’ is a valid filepath.

monitor: This is the quantity to monitor. This can take one of the values from ‘loss’, ‘acc’, ‘val_loss’ and ‘val_acc’.

verbose: This thing controls whether some information about model saving will be displayed or not. This is either 0 or 1. If 0, nothing will be displayed and for 1 something like this will be displayed depending on the behavior of the monitored quantity.

save_best_only: If set to false, then model after every epoch will be saved whether the monitored quantity increases or decreases. Otherwise, it will save the model depending on the ‘mode’ argument.

mode: This can take one of the values from auto, min, max. For instance, if the mode is ‘max’ and ‘val_acc’ is the monitored quantity, then for save_best_only = True the model will be saved only when ‘val_acc’ improves, otherwise, the model will not be saved at that epoch. For ‘val_loss’, this should be min. ‘auto’ mode automatically decides the direction depending on the monitored quantity.

save_weights_only: if True, then only the model weights will be saved otherwise the full model will be saved.

period: The callback will be applied after the specified period (no. of epochs)

How to use this?

All the callbacks are available in the keras.callbacks module so first import the ModelCheckpoint function from this module.
Then properly set up the function arguments.
Now, to apply this you need to pass this as a list in the .fit() method.

from keras.callbacks import ModelCheckpoint
filepath = 'D:/downloads/best_weights.hdf5'
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='auto',period=1)

model.fit(train_x, train_y, epochs=10, batch_size=128, callbacks=[checkpoint])

from keras.callbacks import ModelCheckpoint

filepath = 'D:/downloads/best_weights.hdf5'

checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='auto',period=1)

model.fit(train_x, train_y, epochs=10, batch_size=128, callbacks=[checkpoint])

Let’s take MNIST classification example to understand this

Import Libraries

import keras
import numpy as np
import matplotlib.pyplot as plt
from keras.layers import Dense, Flatten, Conv2D, MaxPool2D
from keras.models import Sequential
from keras.datasets import mnist

import keras

import numpy as np

import matplotlib.pyplot as plt

from keras.layers import Dense, Flatten, Conv2D, MaxPool2D

from keras.models import Sequential

from keras.datasets import mnist

Data Loading and Pre-processing

(train_x, train_y),(test_x, test_y) = mnist.load_data()

# Normalize between 0 and 1
train_x = train_x.astype('float32')/255
test_x = test_x.astype('float32')/255
# Add the channel dimension
train_x = np.expand_dims(train_x,axis=-1)
test_x = np.expand_dims(test_x,axis=-1)

(train_x, train_y),(test_x, test_y) = mnist.load_data()

# Normalize between 0 and 1

train_x = train_x.astype('float32')/255

test_x = test_x.astype('float32')/255

# Add the channel dimension

train_x = np.expand_dims(train_x,axis=-1)

test_x = np.expand_dims(test_x,axis=-1)

Build Model

model = Sequential()
model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(64,(3,3),activation='relu'))
model.add(MaxPool2D((2,2)))
model.add(Flatten())
model.add(Dense(512,activation='relu'))
model.add(Dense(10,activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

model = Sequential()

model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))

model.add(MaxPool2D((2,2)))

model.add(Conv2D(64,(3,3),activation='relu'))

model.add(MaxPool2D((2,2)))

model.add(Flatten())

model.add(Dense(512,activation='relu'))

model.add(Dense(10,activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

Callbacks

from keras.callbacks import ModelCheckpoint
filepath = 'D:/downloads/best_model.hdf5'
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='max',period=1)

from keras.callbacks import ModelCheckpoint

filepath = 'D:/downloads/best_model.hdf5'

checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='max',period=1)

Fit Model

record = model.fit(train_x, train_y,validation_split=0.20, epochs=10, batch_size=128,callbacks=[checkpoint])

1	record = model.fit(train_x, train_y,validation_split=0.20, epochs=10, batch_size=128,callbacks=[checkpoint])

Load Weights and Evaluate test set

model.load_weights('D:/downloads/best_model.hdf5')
model.evaluate(test_x,test_y)

1 2	model.load_weights('D:/downloads/best_model.hdf5') model.evaluate(test_x,test_y)

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Monitoring Training in Keras: Callbacks

Leave a reply

Keras is a high level API, can run on top of Tensorflow, CNTK and Theano. Keras is preferable because it is easy and fast to learn. In this blog we will learn a set of functions named as callbacks, used during training in Keras.

Callbacks provides some advantages over normal training in keras. Here I will explain the important ones.

Callback can terminate a training when a Nan loss occurs.
Callback can save the model after every epoch, also you can save the best model.
Early Stopping: Callback can stop training when accuracy stops improving.

Terminate the training when Nan loss occurs

Let’s see the code how to terminate when Nan loss occurs while training:

keras.callbacks.TerminateOnNaN()

1	keras.callbacks.TerminateOnNaN()

Saving Model using Callbacks

To save model after every epoch in keras, we need to import ModelCheckpoint from keras.callbacks. Let’s see the below code which will save the model if validation loss decreases.

from keras.callbacks import ModelCheckpoint

filepath="weights-{epoch:02d}-{val_acc:.2f}.hdf5"
checkpoint = ModelCheckpoint(filepath=filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')
callbacks_list = [checkpoint]

model.compile(optimizer = 'sgd', loss = 'binary_crossentropy')
model.fit(train_data, train_label, epochs = 10, batch_size = 64, callbacks = callbacks_list)

from keras.callbacks import ModelCheckpoint

filepath="weights-{epoch:02d}-{val_acc:.2f}.hdf5"

checkpoint = ModelCheckpoint(filepath=filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')

callbacks_list = [checkpoint]

model.compile(optimizer = 'sgd', loss = 'binary_crossentropy')

model.fit(train_data, train_label, epochs = 10, batch_size = 64, callbacks = callbacks_list)

In the above code first we have created a ModelCheckpoint object by passing its required parameters.

“filepath” defines the path where all checkpoints will be saved. If you want to save only the best model, then directly pass filepath with name “best_model.hdf5” which will overwrite the previous saved checkpoints.
“monitor” will decide which quantity you want to monitor while training.
“save_best_only” only saves if validation loss decreases.
mode with {auto, min, max} when chosen max, will stop training when monitored quantity stops increasing.

Then finally make a callback list and pass it into model.fit() with parameter callbacks.

Early Stopping

Callbacks can stop training when a monitored quantity has stopped improving. Lets see how:

keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto', baseline=None, restore_best_weights=False)

1	keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto', baseline=None, restore_best_weights=False)

min_delta: It is the minimum quantity which will be taken for improvement to be conceded.
patience: after this number of epochs if training does not improve, it will stop.
mode: in auto mode, the direction will be decided by monitored quantity.
“baseline” the baseline value over which no improvement will stop the training.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

TheAILearner

Mastering Artificial Intelligence

Tag Archives: callbacks

Create custom callbacks in Keras

Keras Callbacks – ReduceLROnPlateau

Keras Callbacks – LambdaCallback

Keras Callbacks – CSVLogger

Keras Callbacks – LearningRateScheduler

Keras Callbacks – ProgbarLogger

Keras Callbacks – TerminateOnNaN

Keras Callbacks – BaseLogger

Keras API

Keras Callbacks – ModelCheckpoint

Keras Function

How to use this?

Import Libraries

Data Loading and Pre-processing

Build Model

Callbacks

Fit Model

Load Weights and Evaluate test set

Monitoring Training in Keras: Callbacks

Terminate the training when Nan loss occurs

Saving Model using Callbacks

Early Stopping