In the previous blog, we discussed image pyramids, and how to construct a Laplacian pyramid from the Gaussian. In this blog, we will discuss how image pyramids can be used for image blending. This produces more visually appealing results as compared to different blending methods we discussed until now. Below are the steps for image blending using image pyramids.
Steps:
- Load the two images and the mask.
- Find the Gaussian pyramid for the two images and the mask.
- From the Gaussian pyramid, calculate the Laplacian pyramid for the two images as explained in the previous blog.
- Now, blend each level of the Laplacian pyramid according to the mask image of the corresponding Gaussian level.
- From this blended Laplacian pyramid, reconstruct the original image. This is done by expanding the level and adding it to the below level as shown in the figure below. Here LS0, LS1, LS2, and LS3 are the levels of the blended Laplacian pyramid obtained in step 4.
Now, let’s implement the above steps using OpenCV-Python. Suppose we want to blend the two images corresponding to the mask as shown below.
So, we will clip the jet image from the second image and blend it to the first image. Below is the code for the steps explained above.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
import cv2 import numpy as np # Step-2 # Find the Gaussian pyramid of the two images and the mask def gaussian_pyramid(img, num_levels): lower = img.copy() gaussian_pyr = [lower] for i in range(num_levels): lower = cv2.pyrDown(lower) gaussian_pyr.append(np.float32(lower)) return gaussian_pyr # Step-3 # Then calculate the Laplacian pyramid def laplacian_pyramid(gaussian_pyr): laplacian_top = gaussian_pyr[-1] num_levels = len(gaussian_pyr) - 1 laplacian_pyr = [laplacian_top] for i in range(num_levels,0,-1): size = (gaussian_pyr[i - 1].shape[1], gaussian_pyr[i - 1].shape[0]) gaussian_expanded = cv2.pyrUp(gaussian_pyr[i], dstsize=size) laplacian = np.subtract(gaussian_pyr[i-1], gaussian_expanded) laplacian_pyr.append(laplacian) return laplacian_pyr # Step-4 # Now blend the two images wrt. the mask def blend(laplacian_A,laplacian_B,mask_pyr): LS = [] for la,lb,mask in zip(laplacian_A,laplacian_B,mask_pyr): ls = lb * mask + la * (1.0 - mask) LS.append(ls) return LS # Step-5 # Reconstruct the original image def reconstruct(laplacian_pyr): laplacian_top = laplacian_pyr[0] laplacian_lst = [laplacian_top] num_levels = len(laplacian_pyr) - 1 for i in range(num_levels): size = (laplacian_pyr[i + 1].shape[1], laplacian_pyr[i + 1].shape[0]) laplacian_expanded = cv2.pyrUp(laplacian_top, dstsize=size) laplacian_top = cv2.add(laplacian_pyr[i+1], laplacian_expanded) laplacian_lst.append(laplacian_top) return laplacian_lst # Now let's call all these functions if __name__ == '__main__': # Step-1 # Load the two images img1 = cv2.imread('D:/downloads/cloud1.jpg') img1 = cv2.resize(img1, (1800, 1000)) img2 = cv2.imread('D:/downloads/jet.jpg') img2 = cv2.resize(img2, (1800, 1000)) # Create the mask mask = np.zeros((1000,1800,3), dtype='float32') mask[250:500,640:1440,:] = (1,1,1) num_levels = 7 # For image-1, calculate Gaussian and Laplacian gaussian_pyr_1 = gaussian_pyramid(img1, num_levels) laplacian_pyr_1 = laplacian_pyramid(gaussian_pyr_1) # For image-2, calculate Gaussian and Laplacian gaussian_pyr_2 = gaussian_pyramid(img2, num_levels) laplacian_pyr_2 = laplacian_pyramid(gaussian_pyr_2) # Calculate the Gaussian pyramid for the mask image and reverse it. mask_pyr_final = gaussian_pyramid(mask, num_levels) mask_pyr_final.reverse() # Blend the images add_laplace = blend(laplacian_pyr_1,laplacian_pyr_2,mask_pyr_final) # Reconstruct the images final = reconstruct(add_laplace) # Save the final image to the disk cv2.imwrite('D:/downloads/pp2.jpg',final[num_levels]) |
The blended output is shown below
Still, there is some amount of white gaze around the jet. Later, we will discuss gradient-domain blending methods which improve the result even more. Now, compare this image with a simple copy and paste operation and see the difference.
You can do a side-by-side blending also. In the next blog, we will discuss how to perform image enhancement and image compression using the Laplacian pyramids. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.