In this blog, we will learn how we can sharpen an image or perform edge enhancement using a smoothing filter. Let’s see how this is done
First, we blur the image. We know by smoothing an image we suppress most of the high-frequency components.
Then, we subtract this smoothed image from the original image(the resulting difference is known as a mask). Thus, the output image will have most of the high-frequency components that are blocked by the smoothing filter.
Adding this mask back to the original will enhance the high-frequency components.
Because we are using a blurred or unsharp image to create a mask this technique is known as Unsharp Masking.
Thus, unsharp masking first produces a mask m(x,y) as
where, f(x,y) is the original image and fb(x,y) is the blurred version of the original image.
Then this mask is added back to the original image which results in enhancing the high-frequency components.
where k specifies what portion of the mask to be added. When k= 1 this is known as Unsharp masking. For k>1 we call this as high-boost filtering because we are boosting the high-frequency components by giving more weight to the masked (edge) image.
We can also write the above two equations into one as the weighted average of the original and the blurred image.
Note: Instead of subtracting the blurred image from the original, we can directly use a negative Laplacian filter to obtain the mask.
If the image contains noise, this method will not produce satisfactory results, like most of the other sharpening filters.
Let’s see how to do this using OpenCV-Python
OpenCV-Python
Since in the last equation we described unsharp masking as the weighted average of the original and the input image, we will simply use OpenCV cv2.addWeighted() function.
1
2
3
4
5
6
7
8
import cv2
# Load the image
image=cv2.imread("D:/downloads/kang.jpg")
# Blur the image
gauss=cv2.GaussianBlur(image,(7,7),0)
# Apply Unsharp masking
unsharp_image=cv2.addWeighted(image,2,gauss,-1,0)
The output is shown below
Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
In the previous blog, we discussed Gaussian Blurring that uses Gaussian kernels for image smoothing. This is a low pass filtering technique that blocks high frequencies (like edges, noise, etc.). In this blog, we will see how we can use this Gaussian Blurring to highlight certain high-frequency parts in an image. Isn’t that interesting? So, let’s get started.
In Gaussian Blurring, we discussed how the standard deviation of the Gaussian affects the degree of smoothing. Roughly speaking, larger the standard deviation more will be the blurring or in other words more high frequency components will be suppressed.
Thus if we take 2 Gaussian kernels with different standard deviations, apply separately on the same image and subtract their corresponding responses, we will get an output that highlights certain high-frequency components based on the standard deviations used.
The logic is by blurring we remove some high-frequency components that represent noise, and by subtracting we remove some low-frequency components that correspond to the homogeneous areas in the image. All the remaining frequency components are assumed to be associated with the edges in the images. Thus, the Difference of Gaussian acts like a bandpass filter. Let’s take an example to understand this.
Suppose we have an image as shown below
Suppose we have 2 Gaussian kernels with standard deviation (σ1 > σ2). The kernel (with σ1), when convolved with an image, will blur the high-frequency components more as compared to the other kernel. Subtracting these, we can recover the information that lies between the frequency range which is not suppressed or blurred.
Now, that we saw how this works, let’s also discuss where this is useful(its pros and cons) as compared to other edge detection methods we have discussed.
All the edge detection kernels which we discussed till now are quite good in edge detection but one downside is that they are highly susceptible to noise. Thus if the image contains a high degree of noise, Difference of Gaussian is the way to go. This is because we are actually doing blurring which reduces the effect of noise to a great extent.
One downside of this method is that the edges are not enhanced much as compared to other methods. The output image formed has lower contrast.
OpenCV-Python
Let’s see how to do this using OpenCV-Python
1
2
3
4
5
6
7
8
9
import cv2
img=cv2.imread('D:/downloads/opencv_logo.PNG')
# Apply 3x3 and 7x7 Gaussian blur
low_sigma=cv2.GaussianBlur(img,(3,3),0)
high_sigma=cv2.GaussianBlur(img,(5,5),0)
# Calculate the DoG by subtracting
dog=low_sigma-high_sigma
Note: Using the linear separable property of the Gaussian kernel we can speed up the entire algorithm.
Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
In the previous blogs, we discussed different smoothing filters. Before moving forward, let’s first discuss Image Gradients which will be useful in edge detection, robust feature and texture matching. So, let’s first recall what a gradient is.
In mathematics, the term gradient of a function means how a function is changing wrt. its arguments or independent variables. The gradient term is more frequently used for multi-variable functions. For a single variable function, we refer to this as the slope.
The gradient of an N-variable function at each point is an N-D vector with the components given by the derivatives in the N-directions. e.g. for a 3-variable function (f(x,y,z)), the gradient, if it exists, is given by
Thus, the gradient provides two pieces of information – magnitude and direction. The direction of the gradient tells us the direction of greatest increase while the magnitude represents the rate of increase in that direction.
Because gradients are defined only for continuous functions and Image is a 2-d discrete function (F(x,y)). Thus we need to approximate the gradients and we do this using Finite differences. In this instead of h approaching 0 we assume h to be a fixed (non-zero) value.
Three forms of finite differences are commonly used: forward, backward and central.
But for calculating Image Gradients, we use the central difference to approximate gradients in x and y directions. Below example shows how to calculate the central difference in the x-direction for 200.
In the next blog, we will discuss how to derive different kernels such as Sobel, Prewitt, etc from this central difference formulae and then using convolution to approximate the image gradients.
Thus, at each image point, the gradient vector points in the direction of largest possible intensity increase, and the magnitude corresponds to the rate of change in that direction. Thus for an image f(x,y), the gradient direction and magnitude is given by
Thus in simple words, image gradient in x-direction measures the horizontal change in intensity while the gradient in y measures the vertical change in intensity.
Since edges are an abrupt change in the intensity values thus the largest gradient values will occur across an edge (neglecting noise) in an image. Thus, the x-gradient will find the vertical edges while y-gradient will highlight the horizontal edges as shown below.
Thus, we may conclude that edges are perpendicular to the gradient direction(largest). That’s why gradients are used in edge detection.
In the next blog, we will discuss different edge detection filters. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
Till now, we have discussed various smoothing filters like Averaging, Median, Gaussian, etc. All these filters are effective in removing different types of noises but at the same time produce an undesirable side effect of blurring the edges also. So isn’t it be nice, if we somehow prevent averaging across edges, while still smoothing other regions. This is what exactly Bilateral filtering does.
Let’s first refresh some basic concepts which will be needed to understand Bilateral filtering.
I hope you are all familiar with the domain and range of any function. If not then let’s refresh these concepts. Domain and range are the set of all plausible values that the independent and dependent variables can take respectively. We all know that the image is also a function (a 2-D light intensity function F(x,y)). Thus for an image, the domain is the set of all possible pixel locations and range corresponds to all possible intensity values.
Now, let’s use these concepts to understand Bilateral filtering.
All the filters we read till now like Median, Gaussian, etc. were domain filters. This means that the filter weights are assigned using the spatial closeness (i.e. domain). This has an issue as it will blur the edges also. Let’s take an example to see how.
Below is a small 3×3 patch extracted from a large image having a diagonal edge. Because in domain filters, we are assigning filter weights according to the spatial closeness, more weights are given to the nearer pixels as compared to the distant pixels. This leads to the edge blurring. See how the central pixel value changed from 10 to 4.
Thus, domain filters doesn’t consider whether a pixel is an edge pixel or not. It just assigns weights according to spatial closeness and thus leads to edge blurring.
Now, let’s see what will happen if we consider range filters. In range filters, we assign weights according to the intensity difference. This ensures that only those pixels with similar intensity to the central pixel is considered for blurring. Because in range filtering, we are not considering the spatial relationship. So, now the similar intensity pixels that are far away from the central pixel affect the final value of the central pixel more as compared to the nearby approx. similar pixels. This makes no sense.
Thus, range filtering alone also doesn’t solve the problem of edge blurring.
Now, what if we combine both domain and range filtering. That will solve our problem. Because now, first, the domain filter will make sure that only nearby pixels (say a 3×3 window) are considered for blurring and then the range filter will make sure that the weights in this 3×3 window are given according to the intensity difference wrt. center pixel. This way it will preserve the edges. This is known as Bilateral filtering (bi for both domain and range filtering).
I hope you understood Bilateral filtering. Now, let’s see how to do this using OpenCV-Python
OpenCV-Python
OpenCV provides an inbuilt function for bilateral filtering as shown below. You can read more about it here but a short description is given below
# d - Filter size. If it is non-positive, it is computed from sigmaSpace
# sigmaColor - Filter sigma in the color space (Range Filter)
# sigmaSpace - Filter sigma in the coordinate space (Domain Filter)
If the sigma values are small (< 10), the filter will not have much effect, whereas if they are large (> 150), they will have a very strong effect, making the image look “cartoonish”.
Large filters (d > 5) are very slow, so it is recommended to use d=5 for real-time applications, and perhaps d=9 for offline applications that need heavy noise filtering.
Let’s take an example to understand this
1
2
3
4
5
import cv2
img=cv2.imread('D:/downloads/opencv_logo.PNG')
# Apply the Bilateral filter
blur=cv2.bilateralFilter(img,5,25,25)
There exist several extensions to this filter like the guided filterthat deals with the artifacts generated by this. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
In this blog, we will discuss how we can add different types of noise in an image like Gaussian, salt-and-pepper, speckle, etc. By knowing this, you will be able to evaluate various image filtering, restoration, and many other techniques. So, let’s get started.
1. Using Scikit-image
In Scikit-image, there is a builtin function random_noise that adds random noise of various types to a floating-point image. Let’s first check the function arguments and then we will see how to implement it.
Basic syntax of the random_noise function is shown below. You can read more about the arguments in the scikit-image documentation.
This returns a floating-point image data on the range [0, 1] or [-1, 1] depending on whether the input image was unsigned or signed, respectively.
Let’s take an example to understand how to use this function
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import cv2
import numpy asnp
from skimage.util import random_noise
# Load the image
img=cv2.imread("D:/downloads/opencv_logo.PNG")
# Add salt-and-pepper noise to the image.
noise_img=random_noise(img,mode='s&p',amount=0.3)
# The above function returns a floating-point image
# on the range [0, 1], thus we changed it to 'uint8'
# and from [0,255]
noise_img=np.array(255*noise_img,dtype='uint8')
# Display the noise image
cv2.imshow('blur',noise_img)
cv2.waitKey(0)
The output image with salt-and-pepper noise looks like this
You can add several builtin noise patterns, such as Gaussian, salt and pepper, Poisson, speckle, etc. by changing the ‘mode’ argument.
2. Using Numpy
Image noise is a random variation in the intensity values. Thus, by randomly inserting some values in an image, we can reproduce any noise pattern. For randomly inserting values, Numpy random module comes handy. Let’s see how
In the previous blog, we discussed smoothing filters. In this article, we will discuss another smoothing technique known as Gaussian Blurring, that uses a low pass filter whose weights are derived from a Gaussian function. This is perhaps the most frequently used low pass filter in computer vision applications. We will also discuss various properties of the Gaussian filter that makes the algorithm more efficient. So, let’s get started with a basic background introduction.
We already know that a digital image is obtained by sampling and quantizing the continuous signal. Thus if we were to interpolate a pixel value, more chances are that it resembles that of the neighborhood pixels and less on the distant pixels. Similarly while smoothing an image, it makes more sense to take the weighted average instead of just averaging the values under the mask (like we did in Averaging).
So, we should look for a distribution/function that assigns more weights to the nearest pixels as compared to the distant pixels. This is the motivation for using Gaussian distribution.
A 2-d Gaussian function is obtained by multiplying two 1-d Gaussian functions (one for each direction) as shown below
2-d Gaussian function with mean=0 and std. deviation= σ
Now, just convolve the 2-d Gaussian function with the image to get the output. But for that, we need to produce a discrete approximation to the Gaussian function. Here comes the problem.
Because the Gaussian function has infinite support (meaning it is non-zero everywhere), the approximation would require an infinitely large convolution kernel. In other words, for each pixel calculation, we will need the entire image. So, we need to truncate or limit the kernel size.
For Gaussian, we know that 99.3% of the distribution falls within 3 standard deviations after which the values are effectively close to zero. So, we limit the kernel size to contain only values within 3σ from the mean. This approximation generally yields a result sufficiently close to that obtained by the entire Gaussian distribution.
Note: The approximated kernel weights would not sum exactly 1 so, normalize the weights by the overall kernel sum. Otherwise, this will cause darkening or brightening of the image.
A normalized 3×3 Gaussian filter is shown below (See the weight distribution)
Later we will see how to obtain different Gaussian kernels. Now, let’s see some interesting properties of the Gaussian filter that makes it efficient.
Properties
First, the Gaussian kernel is linearly separable. This means we can break any 2-d filter into two 1-d filters. Because of this, the computational complexity is reduced from O(n2) to O(n). Let’s see an example
Applying multiple successive Gaussian kernels is equivalent to applying a single, larger Gaussian blur, whose radius is the square root of the sum of the squares of the multiple kernels radii. Using this property we can approximate a non-separable filter by a combination of multiple separable filters.
The Gaussian kernel weights(1-D) can be obtained quickly using the Pascal’s Triangle. See how the third row corresponds to the 3×3 filter we used above.
Because of these properties, Gaussian Blurring is one of the most efficient and widely used algorithm. Now, let’s see some applications
Applications
Computer Graphics
Before edge detection (Canny Edge Detector)
Before down-sampling an image to reduce the ringing effect
Now let’s see how to do this using OpenCV-Python
OpenCV-Python
OpenCV provides an inbuilt function for both creating a Gaussian kernel and applying Gaussian blurring. Let’s see them one by one.
To create a Gaussian kernel of your choice, you can use
1
2
3
4
cv2.getGaussianKernel(ksize,sigma[,ktype])
# ksize - kernel size, should be odd and positive (3,5,...)
# sigma - Gaussian standard deviation. If it is non-positive, it is computed from ksize as sigma = 0.3*((ksize-1)*0.5 - 1) + 0.8
# sigmaY - Optional,if sigmaY is zero, it is set to be equal to sigmaX
This first creates a Gaussian kernel and then convolves it with the image.
Now, let’s take an example to implement these two functions. First, use the cv2.getGaussianKernel() to create a 1-D kernel. Then use the cv2.sepFilter() to apply these kernels to the input image.
1
2
3
4
5
6
7
8
9
10
11
12
13
import cv2
img=cv2.imread('D:/downloads/opencv_logo.PNG')
# Creates a 1-D Gaussian kernel
a=cv2.getGaussianKernel(5,1)
# Apply the above Gaussian kernel. Here, I
# have used the same kernel for both X and Y
b=cv2.sepFilter2D(img,-1,a,a)
# Display the Image
cv2.imshow('a',b)
cv2.waitKey(0)
The second method is quite easy to use. Just one line as shown below
1
2
3
4
5
import cv2
img=cv2.imread('D:/downloads/opencv_logo.PNG')
# Apply the Gaussian blur
c=cv2.GaussianBlur(img,(5,5),1)
Both these methods produce the same result but the second one is more easy to implement. Try using this for a different type of noises and compare the results with other techniques.
That’s all about Gaussian blurring. Hope you enjoy reading. In the next blog, we will discuss Bilateral filtering, another smoothing technique that preserves edges also.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
In the previous blog, we briefly introduced Low Pass filters. In this blog, let’s discuss them in detail. Low Pass filters (also known as Smoothing or averaging filter) are mainly used for blurring and noise reduction. Both of these can serve as a useful pre-processing step in many applications.
In general, the Low Pass filters block high-frequency parts of an image. Because noise typically consists of sharp transitions in intensity values, this results in noise reduction. But one downside is that edges are also blurred (Later we will see the blurring techniques which don’t blur the edges).
Now, let’s discuss some of the most commonly used blurring techniques
1. Averaging
In this, each pixel value in an image is replaced by the weighted average of the neighborhood (defined by the filter mask) intensity values. The most commonly used filter is the Box filter which has equal weights. A 3×3 normalized box filter is shown below
It’s a good practice to normalize the filter. This is to make sure that the image doesn’t get brighter or darker. You can also use an unnormalized box filter.
OpenCV provides two inbuilt functions for averaging namely:
cv2.blur() that blurs an image using only the normalized box filter and
cv2.boxFilter() which is more general, having the option of using either normalized or unnormalized box filter. Just pass an argument normalize=False to the function
The basic syntax of both the functions are shown below
# ddepth - the output image depth (Pass -1 to use that of input)
Let’s take an example
1
2
3
4
5
6
7
import cv2
import numpy asnp
img=cv2.imread('D:/downloads/opencv.png')
blur=cv2.boxFilter(img,-1,(5,5),normalize=True)
# or use cv2.blur() as shown below
# blur = cv2.blur(img,(5,5))
The output looks like this
2. Median Blurring
This is a non-linear filtering technique. As clear from the name, this takes a median of all the pixels under the kernel area and replaces the central element with this median value. This is quite effective in reducing a certain type of noise (like salt-and-pepper noise) with considerably less edge blurring as compared to other linear filters of the same size.
Because we are taking a median, the output image will have no new pixel values other than that in the input image.
Note: For an even number of entries, there is more than one possible median, thus kernel size must be odd and greater than 1 for simplicity.
See how effectively median blurring is able to remove salt and pepper noise and still able to preserve the edges.
In the next blog, we will discuss Gaussian Blurring, another blurring technique which is widely used in computer graphics and is computationally very efficient. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
In the previous blog, we discussed filters and convolution operation. Before moving forward, let’s discuss an important concept “Frequency”, which is widely used in spatial filtering.
Frequency in images is the rate of change of intensity values. Thus, a high-frequency image is the one where the intensity values change quickly from one pixel to the next. On the other hand, a low-frequency image may be one that is relatively uniform in brightness or where intensity changes very slowly. Most images contain both high-frequency and low-frequency components. Let’s see by an example below
Clearly, in the above image, the zebra pattern has a high frequency as the intensity changes very rapidly from white to black. While the intensity changes very gradually in the sky thus it has low frequency.
It’s not hard to conclude that edges in an image represents high frequency because the intensity changes drastically across an edge.
Based on the frequency, we can classify the filters as
Low Pass Filters
High Pass Filters
Low Pass filters block high-frequency parts of an image and thus results in blurring or image smoothing. This is shown below
On the other hand, a high pass filter enhances high-frequency parts of an image (i.e. edges) and thus results in image sharpening.
In the next blog, we will discuss in detail different low pass and high pass filters, how to construct them and enhance an image. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
Before reading, please refer to this blog for better understanding.
In this blog, we will discuss how to perform a geometric transformation using OpenCV-Python. In geometric transformation, we move the pixels of an image based on some mathematical formulae. This involves translation, rotation, scaling, and distortion (or undistortion!) of images. This is frequently used as a pre-processing step in many applications where the input is distorted while capturing like document scanning, matching temporal images in remote sensing and many more.
There are two basic steps in geometric transformation
Spatial Transformation: Calculating the spatial position of pixels in the transformed image.
Intensity Interpolation: Finding the intensity values at the newly calculated positions.
OpenCV has built-in functions to apply the different geometric transformations to images like translation, rotation, affine transformation, etc. You can find all the functions here: Geometric Transformations of Images
In this blog, we will learn how to change the apparent perspective of an image. This will make the image look more clear and easy to read. Below image summarizes what we want to do. See how easily we can read the words in the corrected image.
For perspective transformation, we need 4 points on the input image and corresponding points on the output image. The points should be selected counterclockwise. From these points, we will calculate the transformation matrix which when applied to the input image yields the corrected image. Let’s see the steps using OpenCV-Python
Steps:
Load the image
Convert the image to RGB so as to display via matplotlib
Select 4 points in the input image (counterclockwise, starting from the top left) by using matplotlib interactive window.
Apply the perspective transformation to the input image using cv2.warpPerspective()to obtain the corrected image.
Code:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import cv2
import numpy asnp
import matplotlib.pyplot asplt
# To open matplotlib in interactive mode
%matplotlibqt
# Load the image
img=cv2.imread('D:/downloads/geometric.jpg')
# Create a copy of the image
img_copy=np.copy(img)
# Convert to RGB so as to display via matplotlib
# Using Matplotlib we can easily find the coordinates
# of the 4 points that is essential for finding the
# transformation matrix
img_copy=cv2.cvtColor(img_copy,cv2.COLOR_BGR2RGB)
plt.imshow(img_copy)
By running the above code you will get an interactive matplotlib window popup. Now select any four points(better to select corner points) for the inputs. Then specify the corresponding output points.
1
2
3
4
5
6
7
8
9
10
11
12
13
# Specify input and output coordinates that is used
In the previous blogs, we discussed Intensity Transformation, a point processing technique for image enhancement. In this blog, we will discuss another image enhancement method known as Spatial Filtering, that transforms the intensity of a pixel according to the intensities of the neighboring pixels.
First let’s discuss what is a spatial filter?
The spatial filter is a window with some width and height that is usually much less than that of the image. Mostly 3×3, 5×5 or 7×7 size filters are used. The values in the filter are called coefficients or weights. There are other terms to call filters such as mask, kernel, template, or window. A 3×3 spatial filter is shown below
Now, let’s see the mechanism of Spatial Filtering.
The spatial filtering can be characterized as a ‘shift-and-multiply’ operation. First, we place the filter over a portion of an image. Then we multiply the filter weights (or coefficients) with the corresponding image pixel values, sum these up. The center image pixel value is then replaced with the result obtained. Then shift the filter to a new location and repeat the process again.
For the corner image pixels, we pad the image with 0’s. The whole process is shown below where a 3×3 filter is convolved with a 5×5 input image (blue color below) to produce a 7×7 output image.
This process is actually known as “correlation” but here, we refer to this as “convolution” operation. This should not be confused with mathematics convolution.
Note: The mathematics convolution is similar to correlation except that the mask is first flipped both horizontally and vertically.
Mathematically, the result of convolving a filter mask “w” of size mxn with an image “f” of size MxN is given by the expression
Here, we assume that filters are of odd size thus m=2a+1 and n=2b+1, where a and b are positive integers.
Again remember that this function does actually compute the correlation, not the convolution. If you need a real convolution, flip the kernel both horizontally and vertically and then apply the above function.
If you want the output image to be of the same size as that of the input, then you must change the padding as shown below
1
2
3
4
# assuming kernel size is odd
m,n=kernel.shape
pad_y=(m-1)//2
pad_x=(n-1)//2
You can also do this using scipy or other libraries.
OpenCV
OpenCV has a builtin function cv2.filter2D() to convolve a kernel with an image. It’s arguments are
ddepth: desired depth of the output image. If it is negative, it will be the same as that of the input image.
borderType: pixel extrapolation method.
This returns the output image of the same size and the same number of channels as the input image. Depending on the border type, you may get different outputs.
