Tag Archives: image processing

Understanding Structuring Element with Trackbars

As we already discussed that the most important thing in morphological image processing is the Structuring element. This is used to probe an image for finding the region of interest. Different shapes and sizes of SE will produce a different result. Thus it becomes vital to have a good grasp on this for better understanding of morphological image processing. In this blog, let’s create trackbars which makes it really easy to visualize the result for different values. So, let’s get started.

Steps:

Load the image and create a window to attach trackbars
Specify the morphological operations and Structuring elements
Create the trackbars and the callback function

import cv2
import numpy as np

# Load the image
src = cv2.imread('D:/downloads/g1.png')
# Create a window to attach trackbars
title_window = 'Morphology Operations'
cv2.namedWindow(title_window)

# Specify the morphological operations and Structuring elements
morph_op_dic = {0: 'ERODE', 1: 'DILATE', 2: 'OPEN', 3: 'CLOSE', \
                4: 'GRADIENT', 5: 'TOPHAT', 6: 'BLACKHAT', 7: 'HITMISS'}
element_dic = {0: 'RECT', 1: 'CROSS', 2: 'ELLIPSE'}

# Define callback function
def morphology_operations(val):
    # get current trackbar position
    morph_operator = cv2.getTrackbarPos('Operator', title_window)
    kernel_size = cv2.getTrackbarPos('kernel', title_window)
    val_type = cv2.getTrackbarPos('SE', title_window)
    iter_num = cv2.getTrackbarPos('iterations', title_window)
    # create structuring element
    element = cv2.getStructuringElement(val_type, (2*kernel_size + 1, 2*kernel_size+1))
    # apply morphological operation
    dst = cv2.morphologyEx(src, morph_operator, element,iterations=iter_num)
    # Display the operation and structuring element
    font = cv2.FONT_HERSHEY_SIMPLEX
    cv2.putText(dst,'Op:{}, SE: {}'.format(morph_op_dic[morph_operator],element_dic[val_type]),(10,40), font, 0.5,(0,255,255),1,cv2.LINE_AA)
    cv2.imshow(title_window, dst)
    
# Create Trackbars
cv2.createTrackbar('Operator', title_window , 0, 7, morphology_operations)
cv2.createTrackbar('SE', title_window , 0, 2, morphology_operations)
cv2.createTrackbar('kernel', title_window , 0, 10, morphology_operations)
cv2.createTrackbar('iterations', title_window , 0, 10, morphology_operations)

# Instantiate the function
morphology_operations(0)
cv2.waitKey(0)

import cv2

import numpy as np

# Load the image

src = cv2.imread('D:/downloads/g1.png')

# Create a window to attach trackbars

title_window = 'Morphology Operations'

cv2.namedWindow(title_window)

# Specify the morphological operations and Structuring elements

morph_op_dic = {0: 'ERODE', 1: 'DILATE', 2: 'OPEN', 3: 'CLOSE', \

4: 'GRADIENT', 5: 'TOPHAT', 6: 'BLACKHAT', 7: 'HITMISS'}

element_dic = {0: 'RECT', 1: 'CROSS', 2: 'ELLIPSE'}

# Define callback function

def morphology_operations(val):

# get current trackbar position

morph_operator = cv2.getTrackbarPos('Operator', title_window)

kernel_size = cv2.getTrackbarPos('kernel', title_window)

val_type = cv2.getTrackbarPos('SE', title_window)

iter_num = cv2.getTrackbarPos('iterations', title_window)

# create structuring element

element = cv2.getStructuringElement(val_type, (2*kernel_size + 1, 2*kernel_size+1))

# apply morphological operation

dst = cv2.morphologyEx(src, morph_operator, element,iterations=iter_num)

# Display the operation and structuring element

font = cv2.FONT_HERSHEY_SIMPLEX

cv2.putText(dst,'Op:{}, SE: {}'.format(morph_op_dic[morph_operator],element_dic[val_type]),(10,40), font, 0.5,(0,255,255),1,cv2.LINE_AA)

cv2.imshow(title_window, dst)

# Create Trackbars

cv2.createTrackbar('Operator', title_window , 0, 7, morphology_operations)

cv2.createTrackbar('SE', title_window , 0, 2, morphology_operations)

cv2.createTrackbar('kernel', title_window , 0, 10, morphology_operations)

cv2.createTrackbar('iterations', title_window , 0, 10, morphology_operations)

# Instantiate the function

morphology_operations(0)

cv2.waitKey(0)

This will produce the following output

Play with the trackbars to get a feel about the morphological operations. That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time

Thinning and Thickening

1 Reply

In the previous blog, we discussed Hit-or-Miss transformation, that is used for finding desired patterns in an image. In this blog, we will discuss various applications of Hit-or-miss transform such as thinning, thickening, etc. So, let’s get started.

Thinning

This is somewhat similar to erosion or opening operation that we discussed earlier. As clear from the name, this is used to thin the foreground region such that its extent and connectivity is preserved. Preserving extent means preserving the endpoints of a structure whereas connectivity can refer to either 4-connected or 8-connected. Thinning is mostly used for producing skeletons which serve as image descriptors, and for reducing the output of the edge detectors to a one-pixel thickness, etc.

There are various algorithms to implement the thinning operation such as

Zhang Suen fast parallel thinning algorithm
Non-max Suppression in Canny Edge Detector
Guo and Hall’s two sub-iteration parallel Thinning algorithm
Iterative algorithms using morphological operations such as hit-or-miss, opening and erosion, etc

In this blog, we will only discuss the last algorithm, rest we will discuss in the following blogs. So, let’s get started.

In this, we can implement thinning either using erosion and opening operations or by using hit-or-miss operation. Let’s first discuss thinning using erosion and opening. This can be expressed as the union of skeleton subsets where each subset is given by the following expression (A-Binary image and B-structuring element)

Here, n indicates the number of iterations of erosion. N is the last iterative step before A erodes to the empty set (stopping condition). Now, let’s discuss how to implement this using OpenCV-Python.

import cv2
import numpy as np

# Create an image with text on it
img = np.zeros((100,400),dtype='uint8')
font = cv2.FONT_HERSHEY_SIMPLEX
cv2.putText(img,'TheAILearner',(5,70), font, 2,(255),5,cv2.LINE_AA)
img1 = img.copy()

# Structuring Element
kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))
# Create an empty output image to hold values
thin = np.zeros(img.shape,dtype='uint8')

# Loop until erosion leads to an empty set
while (cv2.countNonZero(img1)!=0):
    # Erosion
    erode = cv2.erode(img1,kernel)
    # Opening on eroded image
    opening = cv2.morphologyEx(erode,cv2.MORPH_OPEN,kernel)
    # Subtract these two
    subset = erode - opening
    # Union of all previous sets
    thin = cv2.bitwise_or(subset,thin)
    # Set the eroded image for next iteration
    img1 = erode.copy()
    
cv2.imshow('original',img)
cv2.imshow('thinned',thin)
cv2.waitKey(0)

import cv2

import numpy as np

# Create an image with text on it

img = np.zeros((100,400),dtype='uint8')

font = cv2.FONT_HERSHEY_SIMPLEX

cv2.putText(img,'TheAILearner',(5,70), font, 2,(255),5,cv2.LINE_AA)

img1 = img.copy()

# Structuring Element

kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))

# Create an empty output image to hold values

thin = np.zeros(img.shape,dtype='uint8')

# Loop until erosion leads to an empty set

while (cv2.countNonZero(img1)!=0):

# Erosion

erode = cv2.erode(img1,kernel)

# Opening on eroded image

opening = cv2.morphologyEx(erode,cv2.MORPH_OPEN,kernel)

# Subtract these two

subset = erode - opening

# Union of all previous sets

thin = cv2.bitwise_or(subset,thin)

# Set the eroded image for next iteration

img1 = erode.copy()

cv2.imshow('original',img)

cv2.imshow('thinned',thin)

cv2.waitKey(0)

Now, let’s discuss thinning using hit-or-miss transform. Thinning of set A by SE B can be expressed in terms of hit-or-miss transform as

This means we remove all those pixels whose neighborhood exactly matches the pixels in the SE. Instead of applying this with a single structuring element, it is a common practice to implement it using a sequence of SE so as to produce symmetric results. This operation is mostly applied iteratively until no further changes occur.

Thickening

Thickening is the dual of thinning and thus is equivalent to applying the thinning operation on the background or on the complement of the set A.

In the next blog, we will discuss the remaining thinning algorithms in detail. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Hit-or-Miss Transform

Leave a reply

In this blog, we will discuss Hit-or-Miss transformation. This is basically used for shape detection or finding particular patterns in the given image. The shape or pattern to match has to be provided via the structuring element. This transformation can be easily implemented using the erosion operation. So, let’s get started.

Here, we use two structuring elements (say B1 and B2). In this, we ask a simple question of does B1 fits the object while, simultaneously, B2 misses the object, i.e. fits the background? In other words, we are interested only in those pixels whose neighborhood exactly matches B1 while not matching B2 at the same time. As we already discussed that erosion answers the question of whether the SE fits or not. Thus, Hit-or-Miss operation can be expressed in terms of erosion as

Here, we assume that both SE B1 and B2 don’t intersect otherwise this operation cannot be performed. Because both B1 and B2 are disjoint sets, we can express both in terms of a single structuring element. Doing so makes the operation more interpretable. Let’s see how.

Suppose we want to find a T-shaped pattern in the image defined by the SE B1 such that it does not contain the pattern defined by B2 in its right neighborhood. 0 below represents that we don’t care about these positions. Below figure shows how to combine two SE into one.

Now, just compare the underlying pixels values with the combined SE. If it matches exactly then the pixel underneath the origin of SE is set to 1 else 0. Let’s take an example. Suppose we want to find the above combined SE in the image shown below. Clearly, the one on the lower right matches the pattern defined by the combined SE. The result is shown on the right side.

Below is the code for this. The hit-or-miss can be implemented using the OpenCV cv2.morphologyEx() function by passing the flag cv2.MORPH_HITMISS as shown below.

import cv2
import numpy as np

# Create the input image
input_image = np.array((
    [0, 0, 0, 0, 0, 0, 0, 0],
    [0, 255, 255, 255, 0, 0, 0, 255],
    [0, 0, 255, 255, 0, 0, 0, 0],
    [0, 0, 255, 255, 0, 0, 0, 0],
    [0, 0, 0, 0, 0, 0, 0, 0],
    [0, 255, 255, 0, 0, 255, 255, 255],
    [0,255, 0, 0, 0, 0, 255, 0],
    [0, 255, 0, 0, 0, 0, 255, 0]), dtype="uint8")

# Construct the structuring element
kernel = np.array((
        [1, 1, 1],
        [0, 1, -1],
        [0, 1, -1]), dtype="int")

# Apply hit-or-miss transformation
output_image = cv.morphologyEx(input_image, cv2.MORPH_HITMISS, kernel)

import cv2

import numpy as np

# Create the input image

input_image = np.array((

[0, 0, 0, 0, 0, 0, 0, 0],

[0, 255, 255, 255, 0, 0, 0, 255],

[0, 0, 255, 255, 0, 0, 0, 0],

[0, 0, 0, 0, 0, 0, 0, 0],

[0, 255, 255, 0, 0, 255, 255, 255],

[0,255, 0, 0, 0, 0, 255, 0],

[0, 255, 0, 0, 0, 0, 255, 0]), dtype="uint8")

# Construct the structuring element

kernel = np.array((

[1, 1, 1],

[0, 1, -1],

[0, 1, -1]), dtype="int")

# Apply hit-or-miss transformation

output_image = cv.morphologyEx(input_image, cv2.MORPH_HITMISS, kernel)

This way you can find any patterns in the input image. In the next blog, we will discuss other applications of hit-or-miss operation such as thinning, thickening, convex Hull, etc. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Opening and Closing

Leave a reply

In the previous blogs, we discuss two fundamental morphological operations – Erosion and Dilation. Both have their own advantages and disadvantages. For instance, erosion is useful in removing salt noise and structures of a certain shape but at the same time, if holes or gaps are present in the object, this tends to amplify them. So, what we want is something that removes the structures or fills holes/gaps without affecting the remaining foreground parts.

One plausible solution is to combine erosion and dilation operation. So, for instance, let’s take the case of noise removal. Like we discussed that the erosion will remove the salt noise but at the same time will shrink the foreground region also. To counter this, we apply dilation operation using the same structuring element. Because noise is removed, so dilation will only work on the shrunk foreground area and revert it back to the original. This process of applying erosion followed by dilation is known as Opening and in this blog, we will discuss this method in detail.

So, why is the name Opening? Because this opens up the gap between the objects connected by thin protrusions that are of size less than that of the structuring element. Below is the image where the bridge is 2 pixels wide while the SE is 3 pixels wide.

Now, let’s formulate the opening operation in terms of a set operation. The opening of binary image A by the structuring element B is defined as the erosion of A by B, followed by the dilation of the result by B. This can be stated using any of the two expressions as shown below

The second one states that the opening is the union of all the translations of SE B that fits into A. There is an interesting property associated with the opening known as idempotence. This simply states that if an image has been opened once, performing the subsequent opening operation with the same SE will have no effect on that image. This is because after opening the new boundaries created are such that the SE always fits inside them. Now, let’s discuss another operation – closing that is a dual of opening operation.

Closing

This is the just the reverse of Opening i.e Dilation followed by Erosion. Because this closes the holes/gaps present in the object while keeping the initial object size the same. That’s why the name Closing. Now, let’s formulate this in terms of a set operation.

Actual implementation involves rotating the SE by 180 degrees before performing dilation and erosion. But since our SE is mostly symmetric, we usually don’t care. Similar to the opening, this operator is also idempotent. Unlike opening, this fuses narrow breaks or bridges between the objects. Generally, good for removing pepper noise but not salt noise.

Now, let’s discuss how to implement these using OpenCV-Python. For Opening, one way is to first apply erosion and then dilation using the builtin functions we discussed earlier. Similarly, for closing also. Fortunately, OpenCV provides another function that directly implements these operations as shown below.

cv2.morphologyEx(src, op, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

1	cv2.morphologyEx(src, op, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

Here, src is the input image with any number of channels( all will be processed independently) and the kernel is the structuring element whose origin is defined by the anchor (default (-1,-1)). Don’t know why the iterations argument is given because the opening and closing are idempotent operators. You can create the SE using cv2.getStructuringElement() or simply using numpy. It is sometimes useful to pad the image to account for the boundary pixels or if the image is of non-regular shape and this can be done using the “borderType” and “borderValue” arguments. The “op” argument specifies which type of morphological operation to apply. Following are the types available.

MORPH_OPEN – an opening operation
MORPH_CLOSE – a closing operation
MORPH_GRADIENT – a morphological gradient
MORPH_TOPHAT – “top hat”
MORPH_BLACKHAT – “black hat”

Below is an example where we open and close the image with the rectangular SE.

# Define the structuring element
kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))
# Apply the opening operation
opening = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel)
# Apply the closing operation
closing = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)

# Define the structuring element

kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))

# Apply the opening operation

opening = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel)

# Apply the closing operation

closing = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)

In the next blog, we will discuss other morphological operators like morphological gradient, top hat, etc. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Morphological gradient and Top-hat operators

Leave a reply

In this blog, we will discuss various morphological operators such as morphological gradient, white and black top-hat transform, etc. All these operators can be easily obtained by combining erosion and dilation operations. So, let’s discuss each of these methods in detail.

Morphological Gradient

This is the difference between the dilation and erosion of an image. Because dilation and erosion mostly affect the pixels that are close to the boundary between the foreground and background, their difference generally yields the boundary and thus this is used for edge detection and segmentation tasks. Now, let’s discuss how to implement this using OpenCV-Python.

One approach is to use OpenCV cv2.dilate() and cv2.erode() functions and then subtract these two. Another approach is to use OpenCV cv2.morphologyEx() function with cv2.MORPH_GRADIENT flag as discussed in the previous blog. Both of these approaches are shown below.

# Structuring element
kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))

# Approach-1: Perform erosion and dilation separately and then subtract
erosion = cv2.erode(img, kernel, iterations = 1)
dilation = cv2.dilate(img, kernel, iterations=1)
gradient1 = dilation - erosion

# Approach-2: Use cv2.morphologyEx()
gradient = cv2.morphologyEx(img, cv2.MORPH_GRADIENT, kernel)

# Structuring element

kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))

# Approach-1: Perform erosion and dilation separately and then subtract

erosion = cv2.erode(img, kernel, iterations = 1)

dilation = cv2.dilate(img, kernel, iterations=1)

gradient1 = dilation - erosion

# Approach-2: Use cv2.morphologyEx()

gradient = cv2.morphologyEx(img, cv2.MORPH_GRADIENT, kernel)

Top Hat Transform

This is mainly used to extract small details from the images. There are two types of top hat operators namely

White top-hat transform: This is the difference between the image and its opening.
Black top-hat transform: Difference between the closing and the input image.

As we already know that opening eliminates thin protrusions and salt noise, thus the white top hat transform will return these elements. Similarly, the black hat transform will return the pepper noise and gaps/holes that are filled. That’s why the name “White” and “Black”. Thus depending upon the structuring element used, these can be useful for feature extraction.

Now, let’s discuss how to implement this using OpenCV-Python. This can be implemented either manually for instance first finding the image opening and then subtracting it from the original image. The second approach is to pass the corresponding flag in the cv2.morphologyEx() function as shown below.

# Load the image
img1 = cv2.imread("D:/downloads/adap1.jpg",0)
# Structuring element
kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(25,25))
# Apply the top hat transform
tophat = cv2.morphologyEx(img, cv2.MORPH_TOPHAT, kernel)
# Apply the black hat transform
blackhat = cv2.morphologyEx(img, cv2.MORPH_BLACKHAT, kernel)

# Load the image

img1 = cv2.imread("D:/downloads/adap1.jpg",0)

# Structuring element

kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(25,25))

# Apply the top hat transform

tophat = cv2.morphologyEx(img, cv2.MORPH_TOPHAT, kernel)

# Apply the black hat transform

blackhat = cv2.morphologyEx(img, cv2.MORPH_BLACKHAT, kernel)

This can also be used for tackling non-uniform illumination in images and thus can be a useful pre-processing tool for image segmentation tasks. The main idea is that you select the structuring element large enough that it erodes all the object region. Then the only thing remaining will be the background or the shading pattern. Subtracting this background from the image (or top-hat transform, in other words) will produce an image that reduces the non-uniform illumination effect. Thus the image can now be reasonably segmented. Below is an example of the image corrupted by non-uniform illumination and the result of applying white top-hat transform on that image. The image is thresholded using Otsu’s method.

That’s all for this blog. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Dilation

Leave a reply

In the previous blog, we discussed erosion operation. In this blog, we will cover another morphological operation – Dilation which is the dual of erosion. Dual in the sense that dilating the object region is equivalent to eroding the background region and vice versa. So, let’s get started.

Dilation

As clear from the name, this operation dilates or expands the object region. This is just opposite of erosion. In this, we ask the simple question of whether the structuring element hits the object or not? (“hits” here means that at least one of the image pixels underlying the structuring element (SE) should have the same value as that of the corresponding SE). If it hits, that pixel is set to 1 else set to 0. This is all the concept behind the dilation operation. Now, let’s formulate this in terms of the set operation.

In general, the dilation of the binary image A by some SE B is defined as

That is the set of all values of z such that the intersection of B (translated by z and reflected about its origin) and A is non-empty. In other words, we place the SE over the image so that the origin of the SE coincides with the input pixel position and compare the underlying image pixels with the pixels of the corresponding SE. If the SE shares at least one common element with its underlying image pixels, then the central image pixel is set to 1 else 0.

Thus this increases the size of the object. If some holes or pepper noise is present in the object, this results in bridging the gaps or removing the noise similar to what we discussed in the low pass filtering. The extent of thickening is controlled by the shape and size of the SE.

For binary images, this can be simply done by taking the maximum of the neighborhood defined by the SE. Now, let’s see how to do this using OpenCV-Python. OpenCV provides a builtin function for this as shown below.

cv2.dilate(src, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

1	cv2.dilate(src, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

Here, src is the input image with any number of channels( all will be processed independently) and the kernel is the structuring element whose origin is defined by the anchor (default (-1,-1) i.e at the center of the SE). You can create the SE using cv2.getStructuringElement() or simply using numpy. Iterations specify how many times to repeat the dilation process. It is sometimes useful to pad the image to account for the boundary pixels or if the image is of non-regular shape and this can be done using the “borderType” and “borderValue” arguments. Below is an example where we dilate the image with the rectangular SE.

import cv2
import numpy as np

# load the image
img = cv2.imread('kang.png', 0)

# Following are the two methods by which we can construct
# the structuring element

# Method-1: using numpy
kernel_1 = np.ones((3,1), dtype='uint8')

# Method-2: using opencv getStructuringElement() function
kernel_2 = cv2.getStructuringElement(cv2.MORPH_RECT,(1,3))

# Apply the dilation function
dilation = cv2.dilate(b, kernel_2, iterations = 1)

import cv2

import numpy as np

# load the image

img = cv2.imread('kang.png', 0)

# Following are the two methods by which we can construct

# the structuring element

# Method-1: using numpy

kernel_1 = np.ones((3,1), dtype='uint8')

# Method-2: using opencv getStructuringElement() function

kernel_2 = cv2.getStructuringElement(cv2.MORPH_RECT,(1,3))

# Apply the dilation function

dilation = cv2.dilate(b, kernel_2, iterations = 1)

Similar to erosion, this can also be used to remove noise, detect the object boundary, etc. Although neither erosion nor dilation alone is effective in reducing noise. A more efficient approach is erosion followed by dilation or opening operation in general. Most of the morphological algorithms which we will discuss in the next blogs are also based on dilation and erosion. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Erosion

Leave a reply

In the previous blog, we touched on the introduction of morphological operations. In this blog and the next blog, we will discuss two of the most fundamental morphological operations – Erosion and Dilation. All other morphological operations can be defined in terms of these two basic operations. So, let’s get started.

Erosion

As clear from the name, this operation erodes or remove the pixels from the object boundary. In this, we ask the simple question of whether the structuring element fits the object or not? (“Fits” here means that all the image pixels underlying the structuring element (SE) should have the same value as that of the corresponding SE) If the image pixel fits, it is assigned 1, otherwise eroded (assigned 0). Thus if we use a square SE (say of size 3×3), then all the object boundary pixels will be eroded away. Now, let’s understand this in terms of the set operation.

In general, the erosion of the binary image A by some SE B is defined as

That is the set of all values of z such that B translated by z, is the subset of A or is contained in A. In other words, you shift the SE over the image and set the positions, where the SE doesn’t share any common element with the background, to 1 and erode all the remaining positions.

Thus this results in a decrease in the object area. If some holes are present in the object, this operation tends to increase the hole area. For binary images, this can be simply applied by taking the minimum of the neighborhood defined by the SE. Now, let’s see how to do this using OpenCV-Python. OpenCV provides a builtin function for this as shown below.

cv2.erode(src, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

1	cv2.erode(src, kernel[, anchor[, iterations[, borderType[, borderValue]]]])

Here, src is the input image with any number of channels( all will be processed independently) and the kernel is the structuring element whose origin is defined by the anchor (default (-1,-1)). You can create the SE using cv2.getStructuringElement() or simply using numpy. Iterations specify how many times to repeat the erosion process. It is sometimes useful to pad the image to account for the boundary pixels or if the image is of non-regular shape and this can be done using the “borderType” and “borderValue” arguments. Below is an example where we erode the image with the rectangular SE.

import cv2
import numpy as np

# load the image
img = cv2.imread('kang.png', 0)

# Following are the two methods by which we can construct
# the structuring element

# Method-1: using numpy
kernel_1 = np.ones((3,3), dtype='uint8')

# Method-2: using opencv getStructuringElement() function
kernel_2 = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))

# Apply the erosion function
erosion = cv2.erode(b, kernel_2, iterations = 1)

import cv2

import numpy as np

# load the image

img = cv2.imread('kang.png', 0)

# Following are the two methods by which we can construct

# the structuring element

# Method-1: using numpy

kernel_1 = np.ones((3,3), dtype='uint8')

# Method-2: using opencv getStructuringElement() function

kernel_2 = cv2.getStructuringElement(cv2.MORPH_RECT,(3,3))

# Apply the erosion function

erosion = cv2.erode(b, kernel_2, iterations = 1)

Different structuring elements, whether in the terms of shape or size, will produce different results. Mostly people prefer disc-shaped structuring element. If the size of the SE exceeds the size of the object, then the entire object will be eroded away. Erosion can be useful in removing noise (subjected to some conditions), detecting the object boundaries (subtracting the eroded image from the original one), separating the connected components or structures of a certain shape or size, etc.

In the next blog, we will discuss another morphological operation known as Dilation in greater detail. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Morphological Image Processing

Leave a reply

In the previous blogs, we discussed various thresholding algorithms like otsu, adaptive, BHT, etc. All these resulted in a binary image which in general are distorted by noise, holes, etc. Thus there is a need to process these images so as to remove the imperfections. And sometimes, we also need to extract image features such as boundaries, etc that are useful in the representation of the object of interest. This all is done using Morphological image processing that applies non-linear transformations based on the image shape. Now, let’s discuss why the name morphology?

In general, morphology stands for the study of the form and the structure of the things. Because as a result of conversion to the binary image, we have lost the intensity information. Thus the only information that remains is the spatial location or the structure of the image. That’s why known as morphological image processing.

Morphological image processing was originally developed for binary images but later this was also extended to the grayscale images. The watershed algorithm is an outcome of this generalization.

Let’s understand the concept behind the morphological image processing (MIP) wrt. convolution operation that we studied earlier.

Remember in convolution, we have a filter/window and we move this filter over the image. The output values are then computed by some linear operation between the filter weights and the pixel values. Similarly, in MIP we have a structuring element and we move this over the entire image. The output values are computed by applying non-linear operations like set operations (intersection, union, etc.) between the structuring element and the underlying pixel values. These non-linear operations are known as morphological operations.

So, the above paragraph contains two terms – structuring element and the morphological operations. So, let’s understand what’s a structuring element?

The structuring element is a binary image (consisting of 0’s and 1’s) that is used to probe an image for finding the region of interest. For instance, if we want to detect lines in an image, we create a linear structuring element. The pattern of 1’s and 0’s specifies the shape of the structuring element. Below is an example of elliptical SE.

Mostly the dimensions of the SE are odd with the origin at the center. OpenCV provides a builtin function for creating SE as shown below.

cv2.getStructuringElement(shape, ksize[, anchor]) → retval

1	cv2.getStructuringElement(shape, ksize[, anchor]) → retval

Here, shape refers to the SE shape. This can take one of the following values

cv2.MORPH_RECT – creates a rectangular SE.
cv2.MORPH_ELLIPSE – creates an elliptical SE.
cv2.MORPH_CROSS – cross-shaped SE.

The “ksize” specifies the size of the SE and anchor specifies the origin position (default is (-1,-1) i.e at the center of the SE). Below is an example that creates an elliptical SE.

cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))

1	cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))

To obtain the output, morphological operations are performed between the SE and the underlying pixel values. Morphological operations are nothing but basic set operations like union, intersection, etc. For instance, an example of morphological operations can be the set of all values such that at least one of the SE pixel values is equal to the underlying image pixel values. This operation usually leads to an increase in the size of the object and fills the holes if present in the object. Below figure shows this morphological operation.

If you use other SE, the result would be different. So, select the shape of the SE according to your application. In the next blog, we will discuss various morphological operations in detail. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Adaptive Thresholding

Leave a reply

In the previous blog, we discussed how global thresholding can be a tedious task when dealing with images having non-uniform illumination. This is because you need to ensure that while subdividing an image, each sub-image histogram is bimodal. Otherwise, the segmentation task will fail.

In this blog, we will discuss adaptive thresholding that works well for varying conditions like non-uniform illumination, etc. In this, the threshold value is calculated separately for each pixel using some statistics obtained from its neighborhood. This way we will get different thresholds for different image regions and thus tackles the problem of varying illumination.

The whole procedure can be summed up as:

For each pixel in the image
- Calculate the statistics (such as mean, median, etc.) from its neighborhood. This will be the threshold value for that pixel.
- Compare the pixel value with this threshold

Now, let’s discuss the OpenCV function for adaptive thresholding.

cv2.adaptiveThreshold(src, maxValue, adaptiveMethod, thresholdType, blockSize, C)

1	cv2.adaptiveThreshold(src, maxValue, adaptiveMethod, thresholdType, blockSize, C)

src: 8-bit greyscale image
thresholdType: This tells us what value to assign to pixels greater/less than the threshold. Must be either THRESH_BINARY or THRESH_BINARY_INV. (You can read more about it here).
maxValue: This is the value assigned to the pixels after thresholding. This depends on the thresholding type. If the type is cv2.THRESH_BINARY, all the pixels greater than the threshold are assigned this maxValue.
adaptiveMethod: This tells us how the threshold is calculated from the pixel neighborhood. This currently supports two methods:
- cv2.ADAPTIVE_THRESH_MEAN_C: In this, the threshold value is the mean of the neighborhood area.
- cv2.ADAPTIVE_THRESH_GAUSSIAN_C: In this, the threshold value is the weighted sum of the neighborhood area. This uses Gaussian weights computed using getGaussiankernel() method. You can read more about it here.
blockSize: This is the neighborhood size.
C: a constant which is subtracted from the threshold.

As discussed OpenCV only provides mean and weighted mean to serve as the threshold. But don’t limit yourself to these two statistics. Try other statistics like standard deviation, median, etc. by writing your own helper function. Let’s see how to use this.

import cv2
# Load the image
img1 = cv2.imread("D:/downloads/adap1.jpg",0)
# Apply Otsu method
ret, thres = cv2.threshold(img2,127,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Apply adaptive threshold
th3 = cv2.adaptiveThreshold(img2,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,cv2.THRESH_BINARY,5,2)
# Display the result
cv2.imshow('original',img1)
cv2.imshow('otsu', thres)
cv2.imshow('adaptive', th3)
cv2.waitKey(0)

import cv2

# Load the image

img1 = cv2.imread("D:/downloads/adap1.jpg",0)

# Apply Otsu method

ret, thres = cv2.threshold(img2,127,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)

# Apply adaptive threshold

th3 = cv2.adaptiveThreshold(img2,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,cv2.THRESH_BINARY,5,2)

# Display the result

cv2.imshow('original',img1)

cv2.imshow('otsu', thres)

cv2.imshow('adaptive', th3)

cv2.waitKey(0)

See how effective adaptive thresholding is in the case of non-uniform illumination. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Balanced histogram thresholding

Leave a reply

In the previous blogs, we discussed different methods for automatically finding the global threshold for an image. For instance, the iterative method, Otsu’s method, etc. In this blog, we will discuss another very simple approach for automatic thresholding – Balanced histogram thresholding. As clear from the name, this method tries to automatically find the threshold by balancing the image histogram. Let’s understand this method in detail.

Note: This method assumes that the image histogram is bimodal and a reasonable contrast ratio exists between the background and the region of interest.

Concept

Suppose you have a perfectly balanced histogram i.e. a histogram where the distribution of the background and the roi is the same. If you place such a histogram over the lever, it will be balanced. And the optimum threshold will be at the center of the lever as shown in the figure below

This is the main idea behind the Balanced Histogram Thresholding. This method tries to balance the image histogram and then infer the threshold value from that.

But in real-life situations, we don’t encounter images with such perfectly balanced histograms. So, let’s see how this method balances the unbalanced histograms.

First, it places the histogram over the lever and calculates the center point.
Then this calculates the left side and right side weights from the center point.
Removes weight from the heavier side and adjust the center.
Repeat the above two steps until the starting and the endpoints are equal to the center.

The whole procedure can be summed up in the below gif (taken from Wikipedia)

**Credits: By Power3d – Own work, CC BY-SA 3.0**

Below is the python code for this. Here, i_s, i_e are the starting and the endpoints of the histogram and i_m is the center

def balanced_hist_thresholding(b):
    # Starting point of histogram
    i_s = np.min(np.where(b[0]>0))
    # End point of histogram
    i_e = np.max(np.where(b[0]>0))
    # Center of histogram
    i_m = (i_s + i_e)//2
    # Left side weight
    w_l = np.sum(b[0][0:i_m+1])
    # Right side weight
    w_r = np.sum(b[0][i_m+1:i_e+1])
    # Until starting point not equal to endpoint
    while (i_s != i_e):
        # If right side is heavier
        if (w_r > w_l):
            # Remove the end weight
            w_r -= b[0][i_e]
            i_e -= 1
            # Adjust the center position and recompute the weights
            if ((i_s+i_e)//2) < i_m:
                w_l -= b[0][i_m]
                w_r += b[0][i_m]
                i_m -= 1
        else:
            # If left side is heavier, remove the starting weight
            w_l -= b[0][i_s]
            i_s += 1
            # Adjust the center position and recompute the weights
            if ((i_s+i_e)//2) >= i_m:
                w_l += b[0][i_m+1]
                w_r -= b[0][i_m+1]
                i_m += 1
    return i_m

def balanced_hist_thresholding(b):

# Starting point of histogram

i_s = np.min(np.where(b[0]>0))

# End point of histogram

i_e = np.max(np.where(b[0]>0))

# Center of histogram

i_m = (i_s + i_e)//2

# Left side weight

w_l = np.sum(b[0][0:i_m+1])

# Right side weight

w_r = np.sum(b[0][i_m+1:i_e+1])

# Until starting point not equal to endpoint

while (i_s != i_e):

# If right side is heavier

if (w_r > w_l):

# Remove the end weight

w_r -= b[0][i_e]

i_e -= 1

# Adjust the center position and recompute the weights

if ((i_s+i_e)//2) < i_m:

w_l -= b[0][i_m]

w_r += b[0][i_m]

i_m -= 1

else:

# If left side is heavier, remove the starting weight

w_l -= b[0][i_s]

i_s += 1

# Adjust the center position and recompute the weights

if ((i_s+i_e)//2) >= i_m:

w_l += b[0][i_m+1]

w_r -= b[0][i_m+1]

i_m += 1

return i_m

The above function takes the image histogram as the input and returns the optimum threshold. Let’s take an example to check how this works.

import cv2
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
np.random.seed(7)

# Create a sample image
img = np.random.normal(40,10,size=(500,500)).astype('uint8')
img[img>100]=40
img[100:400,100:400] = np.random.normal(150,20,size=(300,300)).astype('uint8')

# Plot the histogram
b1 = plt.hist(img.ravel(),256,[0,256])
plt.show()

import cv2

import numpy as np

import matplotlib.pyplot as plt

%matplotlib inline

np.random.seed(7)

# Create a sample image

img = np.random.normal(40,10,size=(500,500)).astype('uint8')

img[img>100]=40

img[100:400,100:400] = np.random.normal(150,20,size=(300,300)).astype('uint8')

# Plot the histogram

b1 = plt.hist(img.ravel(),256,[0,256])

plt.show()

Below is the histogram of the image constructed.

Now, let’s apply the Balanced Histogram thresholding method to check what threshold value this outputs.

thresh_value = balanced_hist_thresholding(b1)
>>> 87

1 2	thresh_value = balanced_hist_thresholding(b1) >>> 87

87 looks like a reasonable threshold, check the image histogram above. So, that’s all for this time. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.