Tag Archives: image interpolation

Image Processing – Bicubic Interpolation

1 Reply

In the last blog, we discussed what is Bi-linear interpolation and how it is performed on images. In this blog, we will learn Bi-cubic interpolation in detail.

Note: We will be using some concepts from the Nearest Neighbour and Bilinear interpolation blog. Check them first before moving forward.

Difference between Bi-linear and Bi-cubic:

  1. Bi-linear uses 4 nearest neighbors to determine the output, while Bi-cubic uses 16 (4×4 neighbourhood).
  2. Weight distribution is done differently.

So, the only thing we need to know is how weights are distributed and rest is same as Bi-linear.

In OpenCV, weights are distributed according to the following code (whole code can be found here)

x used in the above code is calculated from below code where x = fx

Similarly, for y, replace x with fy and fy can be obtained by replacing dx and scale_x in the above code by dy and scale_y respectively (Explained in the previous blog).

Note: For Matlab, use A= -0.50

Let’s see an example. We take the same 2×2 image from the previous blog and want to upscale it by a factor of 2 as shown below

Steps:

  • In the last blog, we calculated for P1. This time let’s take ‘P2’. First, we find the position of P2 in the input image as we did before. So, we find P2 coordinate as (0.75,0.25) with dx = 1 and dy=0.
  • Because cubic needs 4 pixels (2 on left and 2 on right) so, we pad the input image.
  • OpenCV has different methods to add borders which you can check here. Here, I used cv2.BORDER_REPLICATE method. You can use any. After padding the input image looks like this
After padding, Blue square is the input image
  • To find the value of P2, let’s first visualize where P2 is in the image. Yellow is the input image before padding. We take the blue 4×4 neighborhood as shown below
  • For P2, using dx and dy we calculate fx and fy from code above. We get, fx=0.25 and fy=0.75
  • Now, we substitute fx and fy in the above code to calculate the four coefficients. Thus we get coefficients = [-0.0351, 0.2617,0.8789, -0.1055] for fy =0.75 and for fx=0.25 we get coefficients = [ -0.1055 , 0.8789, 0.2617, -0.0351]
  • First, we will perform cubic interpolation along rows( as shown in the above figure inside blue box) with the above calculated weights for fx as
    -0.1055 *10 + 0.8789*10 + 0.2617*20 -0.0351*20 = 12.265625
    -0.1055 *10 + 0.8789*10 + 0.2617*20 -0.0351*20 = 12.265625
    -0.1055 *10 + 0.8789*10 + 0.2617*20 -0.0351*20 = 12.265625
    -0.1055 *30 + 0.8789*30 + 0.2617*40 -0.0351*40 = 32.265625
  • Now, using above calculated 4 values, we will interpolate along columns using calculated weights for fy as
    -0.0351*12.265 + 0.2617*12.265 + 0.8789*12.265 -0.1055*32.625 = 10.11702
  • Similarly, repeat for other pixels.

The final result we get is shown below:

This produces noticeably sharper images than the previous two methods and balances processing time and output quality. That’s why it is used widely (e.g. Adobe Photoshop etc.)

In the next blog, we will see these interpolation methods using OpenCV functions on real images. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Image Processing – Bilinear Interpolation

11 Replies

In the previous blog, we learned how to find the pixel coordinate in the input image and then we discussed nearest neighbour algorithm. In this blog, we will discuss Bi-linear interpolation method in detail.

Bi-linear interpolation means applying a linear interpolation in two directions. Thus, it uses 4 nearest neighbors, takes their weighted average to produce the output

So, let’s first discuss what is linear interpolation and how it is performed?

Linear interpolation means we estimate the value using linear polynomials. Suppose we have 2 points having value 10 and 20 and we want to guess the values in between them. Simple Linear interpolation looks like this

More weight is given to the nearest value(See 1/3 and 2/3 in the above figure). For 2D (e.g. images), we have to perform this operation twice once along rows and then along columns that is why it is known as Bi-Linear interpolation.

Algorithm for Bi-linear Interpolation:

Suppose we have 4 pixels located at (0,0), (1,0), (0,1) and (1,1) and we want to find value at (0.3,0.4).

  1. First, find the value along rows i.e at position A:(0,0.4) and B:(1,0.4) by linear interpolation.
  2. After getting the values at A and B, apply linear interpolation for point (0.3,0.4) between A and B and this is the final result.

Let’s see how to do this for images. We take the same 2×2 image from the previous blog and want to upscale it by a factor of 2 as shown below

Same assumptions as we took in the last blog, pixel is of size 1 and is located at the center.

  • Let’s take ‘P1’. First, we find the position of P1 in the input image. By projecting the 4×4 image on the input 2×2 image we get the coordinates of P1 as (0.25,0.25). (For more details, See here)
  • Since P1 is the border pixel and has no values to its left, so OpenCV replicates the border pixel. This means the row or column at the very edge of the original is replicated to the extra border(padding). OpenCV has different methods to add borders which you can check here.
  • So, now our input image (after border replication) looks like this. Note the values in red shows the input image.
  • To find the value of P1, let’s first visualize where P1 is in the input image (previous step image). Below figure shows the upper left 2×2 input image region and the location of P1 in that.
Image-1
  • Before applying Bi-linear interpolation let’s see how weights are distributed.

Both Matlab and OpenCV yield different results for interpolation because their weight distribution is done differently. Here, I will only explain for OpenCV.

In OpenCV, weights are distributed according to this equation

Where dx is the column index of the unknown pixel and fx is the weight that is assigned to the right pixel, 1-fx is given to the left pixel. Scale_x is the ratio of input width by output width. Similarly, for y, dy is the row index and scale_y is the ratio of heights now.

After knowing how weights are calculated let’s get back to the problem again.

  • For P1, both row and column index i.e dx, and dy =0 so, fx = 0.75 and fy =0.75.
  • We apply linear interpolation with weights fx for both A and B(See Image-1) as 0.75*10(right) + 0.25*10 = 10 (Explained in the Algorithm above)
  • Now, for P1 apply linear interpolation between A and B with the weights fy as 0.75*10(B) +0.25*10(A) = 10
  • So, we get P1 =10. Similarly, repeat for other pixels.

The final result we get is shown below:

This produces smoother results than the nearest neighbor but, the results for sharp transitions like edges, are not ideal.

In the next blog, we will discuss Bi-cubic interpolation. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.