In neural networks, the best idea for debugging is to see the relationship between the cost and the number of iterations. This not only ensures that the optimizer is working properly but can also be very useful in the indication of overfitting. Moreover, we can also debug the learning rate based on this relationship. Thus, one should always keep a track on the loss and the accuracy metrics while training a neural network.
Fortunately, in Keras, we don’t need to write a single extra line of code to store all these values. Keras automatically keeps the record of all the events for each epoch. This includes loss and accuracy metrics for both training and validation sets (if used). This is done using the History callback which is automatically applied to every Keras model. This callback records all the events into a History object that gets returned by the fit() method.
How does this work?
First, at the onset of training, this creates an empty dictionary to store all the events. Then at every epoch end, all the events are appended into the dictionary. Below is the code for this taken from the Keras GitHub.
|
1 2 3 4 5 6 7 8 9 |
def on_train_begin(self, logs=None): self.epoch = [] self.history = {} def on_epoch_end(self, epoch, logs=None): logs = logs or {} self.epoch.append(epoch) for k, v in logs.items(): self.history.setdefault(k, []).append(v) |
How to use this?
Since all the saved records are returned by the fit() method, we can simply store all the events in any variable. Here, I’ve used “record” as the variable name.
|
1 |
record = model.fit(train_x, train_y,validation_split=0.20, epochs=5, batch_size=128) |
Now, using this record object, we can retrieve any information about the training process. For instance, “record.epoch” returns the list of epochs.
|
1 2 |
record.epoch >>> [0, 1, 2, 3, 4] |
“record.history” returns the dictionary containing the event names as the dictionary keys and their values at each epoch in a list.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
record.history >>> {'val_loss': [0.037735904552973806, 0.0383092053757185, 0.03597573842055863, 0.04122187125845812, 0.04817074624549908], 'val_acc': [0.9889166665077209, 0.9899166665077209, 0.9912499998410543, 0.989416666507721, 0.988666666507721], 'loss': [0.01413803466820779, 0.010153079537209123, 0.008668457060974712, 0.008247203289516619, 0.007816496806034896], 'acc': [0.9952291666666667, 0.9967083333333333, 0.9972291666666667, 0.9971875, 0.9975208333333333]} |
You can retrieve all the event names using the following command.
|
1 2 |
record.history.keys() >>> dict_keys(['val_loss', 'val_acc', 'loss', 'acc']) |
You can also get the information about the parameters used while fitting the model. This can be done using the following command.
|
1 2 3 4 5 6 7 8 |
record.params >>> {'batch_size': 128, 'epochs': 5, 'steps': None, 'samples': 48000, 'verbose': 1, 'do_validation': True, 'metrics': ['loss', 'acc', 'val_loss', 'val_acc']} |
Not only this, but one can also check which data is used as the validation data using the following command.
|
1 |
record.validation_data[0] |
These are just a few of functionalities available under the History callback. You can check more of these at Keras GitHub.
Plot the training history
Since all the events are stored in a dictionary, one can easily plot these using any plotting library. Here, I’m using Matplotlib. Below is the code for plotting the loss curves for both training and validation sets.
|
1 2 3 4 5 6 7 8 9 10 11 |
history_dict = record.history train_acc = history_dict['loss'] val_acc = history_dict['val_loss'] epochs = range(1, len(history_dict['loss'])+1) plt.plot(epochs, train_acc,'bo',label='Training Accuracy') plt.plot(epochs, val_acc,'b',label='Validation Accuracy') plt.title('Training and Validation Accuracy') plt.xlabel('Epochs') plt.ylabel('Accuracy') plt.legend() plt.show() |
Similarly, one can plot the accuracy plots. That’s all for History callback. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.