Author Archives: kang & atul

Erosion

Leave a reply

In the previous blog, we touched on the introduction of morphological operations. In this blog and the next blog, we will discuss two of the most fundamental morphological operations – Erosion and Dilation. All other morphological operations can be defined in terms of these two basic operations. So, let’s get started.

Erosion

As clear from the name, this operation erodes or remove the pixels from the object boundary. In this, we ask the simple question of whether the structuring element fits the object or not? (“Fits” here means that all the image pixels underlying the structuring element (SE) should have the same value as that of the corresponding SE) If the image pixel fits, it is assigned 1, otherwise eroded (assigned 0). Thus if we use a square SE (say of size 3×3), then all the object boundary pixels will be eroded away. Now, let’s understand this in terms of the set operation.

In general, the erosion of the binary image A by some SE B is defined as

That is the set of all values of z such that B translated by z, is the subset of A or is contained in A. In other words, you shift the SE over the image and set the positions, where the SE doesn’t share any common element with the background, to 1 and erode all the remaining positions.

Thus this results in a decrease in the object area. If some holes are present in the object, this operation tends to increase the hole area. For binary images, this can be simply applied by taking the minimum of the neighborhood defined by the SE. Now, let’s see how to do this using OpenCV-Python. OpenCV provides a builtin function for this as shown below.

Here, src is the input image with any number of channels( all will be processed independently) and the kernel is the structuring element whose origin is defined by the anchor (default (-1,-1)). You can create the SE using cv2.getStructuringElement() or simply using numpy. Iterations specify how many times to repeat the erosion process. It is sometimes useful to pad the image to account for the boundary pixels or if the image is of non-regular shape and this can be done using the “borderType” and “borderValue” arguments. Below is an example where we erode the image with the rectangular SE.

Different structuring elements, whether in the terms of shape or size, will produce different results. Mostly people prefer disc-shaped structuring element. If the size of the SE exceeds the size of the object, then the entire object will be eroded away. Erosion can be useful in removing noise (subjected to some conditions), detecting the object boundaries (subtracting the eroded image from the original one), separating the connected components or structures of a certain shape or size, etc.

In the next blog, we will discuss another morphological operation known as Dilation in greater detail. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Morphological Image Processing

Leave a reply

In the previous blogs, we discussed various thresholding algorithms like otsu, adaptive, BHT, etc. All these resulted in a binary image which in general are distorted by noise, holes, etc. Thus there is a need to process these images so as to remove the imperfections. And sometimes, we also need to extract image features such as boundaries, etc that are useful in the representation of the object of interest. This all is done using Morphological image processing that applies non-linear transformations based on the image shape. Now, let’s discuss why the name morphology?

In general, morphology stands for the study of the form and the structure of the things. Because as a result of conversion to the binary image, we have lost the intensity information. Thus the only information that remains is the spatial location or the structure of the image. That’s why known as morphological image processing.

Morphological image processing was originally developed for binary images but later this was also extended to the grayscale images. The watershed algorithm is an outcome of this generalization.

Let’s understand the concept behind the morphological image processing (MIP) wrt. convolution operation that we studied earlier.

Remember in convolution, we have a filter/window and we move this filter over the image. The output values are then computed by some linear operation between the filter weights and the pixel values. Similarly, in MIP we have a structuring element and we move this over the entire image. The output values are computed by applying non-linear operations like set operations (intersection, union, etc.) between the structuring element and the underlying pixel values. These non-linear operations are known as morphological operations.

So, the above paragraph contains two terms – structuring element and the morphological operations. So, let’s understand what’s a structuring element?

The structuring element is a binary image (consisting of 0’s and 1’s) that is used to probe an image for finding the region of interest. For instance, if we want to detect lines in an image, we create a linear structuring element. The pattern of 1’s and 0’s specifies the shape of the structuring element. Below is an example of elliptical SE.

Mostly the dimensions of the SE are odd with the origin at the center. OpenCV provides a builtin function for creating SE as shown below.

Here, shape refers to the SE shape. This can take one of the following values

  • cv2.MORPH_RECT – creates a rectangular SE.
  • cv2.MORPH_ELLIPSE – creates an elliptical SE.
  • cv2.MORPH_CROSS – cross-shaped SE.

The “ksize” specifies the size of the SE and anchor specifies the origin position (default is (-1,-1) i.e at the center of the SE). Below is an example that creates an elliptical SE.

To obtain the output, morphological operations are performed between the SE and the underlying pixel values. Morphological operations are nothing but basic set operations like union, intersection, etc. For instance, an example of morphological operations can be the set of all values such that at least one of the SE pixel values is equal to the underlying image pixel values. This operation usually leads to an increase in the size of the object and fills the holes if present in the object. Below figure shows this morphological operation.

If you use other SE, the result would be different. So, select the shape of the SE according to your application. In the next blog, we will discuss various morphological operations in detail. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Create custom callbacks in Keras

2 Replies

In this blog, we will discuss how to create custom callbacks in Keras. This is actually very simple. You just need to create a class that takes keras.callbacks.Callback() as its base class. The set of methods that we can use is also fixed. We just need to write the logic. Let’s understand this with the help of an example. Here, we will create a callback that stops the training when the accuracy has reached a threshold and prints the message.

In “Line-1“, we create a class “mycallback” that takes keras.callbacks.Callback() as its base class.

In “Line-2“, we define a method “on_epoch_end”. Note that the name of the functions that we can use is already predefined according to their functionality. For instance, if we define a function by the name “on_epoch_end“, then this function will be implemented at the end of every epoch. If you change this function name to “on_epoch_end1“, then this function will not be implemented.

Below are some of the method names that we can use. The name of these functions is aptly named according to their functionality. The arguments that they can take is already fixed.

Source: Keras

The epoch and batch arguments refer to the current epoch and batch number. And “logs” is a dictionary that records all the training events like “loss”, “acc” etc.

In “Line-3,4“, we define a stopping condition and if met stop training the model. Note that we can access the model being trained through the base class. And so we can use any other model properties like save_weights, save, trainable, etc.

At last, we create an instance of this class and pass this instance as a list in the fit() method. Below is the output of applying the above callback.

Below is another example that saves the weights at the end of each epoch.

This way you can create many other custom callbacks in keras. This gives you more control over the training process. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – ReduceLROnPlateau

Leave a reply

As we already know that the neural networks suffer greatly from the problem of plateaus which substantially slows down the training process. Thus, the selection of a good learning rate becomes a really challenging task. Many methods have been proposed to counter this problem such as using cyclic learning rates where we vary learning rates between reasonable boundary values, or other methods like Stochastic Gradient Descent with Warm Restarts, etc.

Keras also provides ReduceLROnPlateau callback that reduces the learning rate by some factor whenever the learning stagnates. It is believed that sometimes our model will benefit from lowering the learning rate when trapped in the plateau region. So, let’s discuss its Keras API.

This callback “monitors” a quantity and if no improvement is seen for a ‘patience‘ number of epochs, the learning rate is reduced by the “factor” specified. Improvement is specified by the “min_delta” argument. No improvement is considered if the change in the monitored quantity is less than the min_delta specified. This also has an option whether you want to start evaluating the new LR instantly or give some time to the optimizer to crawl with the new LR and then evaluate the monitored quantity. This is done using the “cooldown” argument. You can also set the lower bound on the LR using the “min_lr” argument. No matter how many epochs or what reduction factor you use, the LR will never decrease beyond “min_lr“.

Now, let’s see how to use this.

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – LambdaCallback

Leave a reply

Keras has provided several builtin classes/callbacks that serves our purpose for most of the cases. But let’s say we want to stop training when the accuracy has reached a benchmark or save the model at each batch. These tasks cannot be achieved using the builtin callbacks. In that case, we need to create our own callback function. In Keras, we can easily create custom callbacks using keras.callbacks.Callback() as our base class. But for that case, you need to create a class and write some amount of code.

As an alternative, Keras also provides us with an option to creates simple, custom callbacks on-the-fly. This can be done with ease using the LambdaCallback. So, in this blog, let’s discuss how to use this callback.

Note: For Python 3.8 or higher, lambda function will start supporting assignment expressions and this will make this callback even more powerful.

Here, all the arguments are aptly named according to their work. For instance, in “on_epoch_end” argument, we pass the function that will be called at the end of each epoch.

Now, all of these arguments expect functions with fixed positional arguments which are mentioned below.

Source: Keras

Now, let’s see how to use this callback. Below I created a simple callback that saves the model weights when the accuracy is beyond some limit.

Let’s take another example. Here, I’m stopping the training whenever the accuracy has reached a certain point. (For Python 3.8 or higher)

Similarly, you can create any custom callback. That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – CSVLogger

Leave a reply

In this blog, we will discuss Keras CSVLogger callback. As clear from the name, this streams the training events like ‘loss’, ‘acc’ etc. to a csv file. Using this you can export all the values that can be represented as a string. So, let’s discuss its Keras API.

Here, the “filename” is the name of the csv file where you want to keep the record. This also gives you an option of how to separate elements in the csv file. You can pass this as a string in the “separator” argument.

This also provides an option of whether to append the training history in an existing file or overwrite the existing file. For instance, if “append=False”, this will overwrite an existing file. Otherwise, it will append the information in the existing file without affecting the previously stored information in that file.

If no existing file is present this will create a new file and then append the information. Now, let’s see how to use this class.

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – LearningRateScheduler

Leave a reply

In neural networks, setting up a good learning rate is always a challenging task. If the learning rate is set too high, this can cause undesirable divergent behavior in your loss function or sometimes your model can converge too quickly to a sub-optimal value. If it is set too low, the training process may take a long time. Thus, it often proves sometimes useful to decay the learning rate as the training progresses. This can be done using the Learning rate schedules or the adaptive learning rate methods like SGD, Adam, etc. In this blog, we will only discuss Learning rate schedules.

Learning rate schedules as clear from the name adjusts the learning rates based on some schedule. For instance, time decay, exponential decay, etc. To implement these decays, Keras has provided a callback known as LearningRateScheduler that adjusts the weights based on the decay function provided. So, let’s discuss its Keras API.

Here, the “schedule” is a decay function that takes epoch number and the current learning rate as the input and returns the new learning rate. The verbose argument tells us whether to print the following message when changing the learning rate or not.

Note: This method overwrites the learning rate of the optimizer used.

Now, let’s discuss the schedule argument in more detail. Here, we will use the time decay function. This updates the learning rate by the expression below

Now, let’s see how to use this using the LearningRateScheduler callback. First, create a function that takes epoch and learning rate as arguments as shown below

Then pass this function in the LearningRateScheduler callback as shown below

Now, simply pass this callback as a list in the fit() method as shown below.

You can also check how the learning rate varies by the following command. This returns a list of learning rates over the epochs.

You can also plot the learning rate over epochs using any plotting library.

Similarly, you can use other decays like step decay, exponential decay or any custom decay. Just create a function and pass it to the LearningRateScheduler callback. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – ProgbarLogger

Leave a reply

In this blog, we will discuss Keras ProgbarLogger callback. As clear from the name, this deals with the logging of the progress bar that we usually see during fit() method depending upon the verbosity argument. So, let’s first discuss its API.

Here, “count_mode” argument controls whether the progress bar displays the samples seen or the steps. This argument can take one of the values from ‘samples‘ or ‘steps‘. If set equal to ‘steps’, make sure that you provide the “steps_per_epoch” argument in the fit() method. Otherwise, this will give you an error. The ‘steps’ is basically used with generators like fit_generator, etc.

Below is the figure where first I’ve trained on 5000 samples and the count_mode argument is set to “samples“. For the second one, I’ve used 12 steps in the fit_generator for 1 epoch. The count_mode argument is set to “steps“.

The second argument “stateful_metrics” controls whether to display the average value of the metric specified or display its value at the last step of every epoch. This should be passed as an iterable like list etc. For more details, refer to Keras callbacks BaseLogger where we have discussed this argument in detail.

This callback, in turn, calls the Keras Progbar class that controls how to display the progress bar like the width of the progress bar, its update interval, etc. Now, let’s see how to use this.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – TerminateOnNaN

Leave a reply

In this blog, we will discuss Keras TerminateOnNaN callback. As clear from the name, this terminates the training when a Nan loss is encountered. Below is the Keras API for this callback.

This checks the loss at every batch end and if that loss is nan or inf, this callback stops the training. This prints out the batch number at which it stops the training. Something like this will be printed.

Below is the code, taken from Keras that shows how this works.

Similarly, you can create your own custom callback that tracks some other metrics. Now, let’s see how to use this callback.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – BaseLogger

Leave a reply

As we already know that the values of the metrics such as loss, acc, etc. that we get during the training are the averaged values over the epoch. This averaging of the values are automatically applied to every Keras model using the BaseLogger class present under the Keras callbacks. This class also provides us with the flexibility of not averaging the metrics over an epoch. So, in this blog, let’s discuss this BaseLogger class in more detail.

Keras API

Similar to the History callback, this callback is also automatically applied to every Keras model with the default set of arguments. Only if you want to change the arguments, you need to apply it similar to how we applied other callbacks i.e. pass in the fit() method.

Here, stateful_metrics are the name of the metrics that you don’t want to average over an epoch. All the metrics names should be passed in the form of iterable like lists etc. For instance, stateful_metrics=[‘acc’,’loss’].

The value of the stateful_metrics will be saved as-is in on_epoch_end. In other words, the value of the stateful_metric in the last batch before the epoch end will be saved as the final value. All the other remaining metrics will be averaged over the epoch ends. Let’s take a look at the image below.

Here, I trained the model for 1 epoch on the mnist data. The stateful_metrics used is ‘acc’. Clearly, the final logged accuracy (See record.history) is similar to the last batch accuracy (0.9392) and not the average accuracy obtained on the epoch end (0.9405). Hope this makes everything clear. Let’s see how to apply baselogger callback.

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.