Category Archives: Image Processing

Understanding Color Models using OpenCV-Python

In this blog, we will see how to convert images from one color model to another and how different colors can be obtained from these color models.

In OpenCV, the command for converting an image to another color-space is

cv2.cvtColor(input_image, conversion_method) 

for example, BGR to HSV conversion can be done by using cv2.COLOR_BGR2HSV method

In OpenCV, more than 150 color-space conversion methods are available. To get the other conversion methods, type the following commands

In the previous blog, we learned how we can construct all colors from each model. Now, let’s get the feeling of this with OpenCV.

Here, I will create three trackbars to specify each of B, G, R colors and a window which shows the color obtained by combining different proportions of B, G, R. Similarly for HSI and CMYK models.

In OpenCV, Trackbar can be created using the cv2.createTrackbar() and its position at any moment can be found using cv2.getTrackbarPos(). 

RGB Trackbar

You can move these trackbars to obtain different colors. A snapshot of output is shown below

HSI Trackbar

We get the following output as

Similarly, you can create trackbar for any color model. Play with these trackbars to get intuition about color models. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Object Tracking Using Color Models OpenCV-Python

In this tutorial, we will learn how we can use color models for object tracking. You can use any color model. Here, I have used HSI because it is easier to represent a color using the HSI model (as it separates the color component from greyscale). Let’s see how to do this

Steps

  1. Open the camera using cv2.VideoCapture()
  2. Create 3 Trackbars of H, S, and I using cv2.createTrackbar()
  3. Read frame by frame
  4. Record the current trackbar position using cv2.getTrackbarPos()
  5. Convert from BGR to HSV using cv2.cvtColor()
  6. Threshold the HSV image based on current trackbar position using cv2.inRange()
  7. Extract the desired result

Code

Open in full screen and play with trackbar to get more intuition about HSI model. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Color Models

In the previous blogs, we represented the color image using the RGB components but this is not the only way available. There are different color models ( A color model is simply a way to define the color) available, each having their own pros and cons.

There are two types of color models available: Additive and Subtractive. Additive uses light (transmitted) to display color while subtractive models use printing inks. These models are fitted into different shapes to obtain new models (See HSI model below).

In this blog, we’ll discuss the three that are most commonly used in the context of digital image processing: RGB, CMY, and HSI

The RGB Color Model

In this, we construct a color cube whose 3 axes denote R, G, and B  respectively as shown below

Normalized RGB Color Cube; Source: Researchgate

This is an additive model, i.e. the colors present in the light add to form new colors. For example, Yellow has coordinate of (1,1,0) which means Yellow =  Red + Green. Similarly, for other colors like cyan = Blue + Green and magenta = Red +Blue.

R, G, and B are added together in varying proportions to produce an extensive range of colors. Mixing equal proportions of R, G, and B falls on the grayscale line.

Use: color monitors and most video cameras.

The CMYK Color Model

CMY stands for cyan, magenta, and yellow also known as secondary colors of light. K refers to black. An equal proportion of C, M, and Y produce muddly black and not pure black. That’s why we use CMYK instead of CMY model.

This is a subtractive model i.e colors are perceived as a result of reflected light. e.g. when light falls on a cyan coated surface, red is absorbed (or subtracted) while Green and Blue are reflected and thus G + B = Cyan. Similarly for magenta and yellow.

Thus, CMY can be obtained from RGB by subtracting RGB from the max intensity.

Use: Printing like books, magazines etc.

The HSI Color Model

HSI stands for Hue, Saturation, and Intensity. This model is similar to how humans perceive color. Let’s understand HSI terms

Hue: Color attribute that describes the pure color or dominant wavelength.

Saturation: Purity of Color or how much a pure color is diluted by white light.

Intensity: Amount of light

H and S tell us about the chromaticity (color information) of the light while I carries the greyscale information.

HSI model can be obtained by rotating the RGB cube such that Black is at the bottom and white at the top.

H varies from 0 to 120 degrees for Red, 120 – 240 for Green, and 240 -360 for Blue. Saturation can take value from 0 to 100%. Intensity value varies according to the bit size of an image.

Pros: Easier to represent the color than the RGB model.

Note: We can also use these color models for object tracking (See here).

In the next blog, we will see how different colors can be generated from these color models with the help of OpenCV. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Changing Video Resolution using OpenCV-Python

In this tutorial, I will show how to change the resolution of the video using OpenCV-Python. This blog is based on interpolation methods (Chapter-5) which we have discussed earlier.

Here, I will convert a 640×480 video to 1280×720. Let’s see how to do this

Steps:

  1. Load a video using cv2.VideoCapture()
  2. Create a VideoWriter object using cv2.VideoWriter()
  3. Extract frame by frame
  4. Resize the frames using cv2.resize()
  5. Save the frames to a video file using cv2.VideoWriter()
  6. Release the VideoWriter and destroy all windows

Code:

Here, I have used Bicubic as the interpolation method, you can use any. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Image Interpolation using OpenCV-Python

In the previous blogs, we discussed the algorithm behind the

  1. nearest neighbor 
  2. bilinear and
  3. bicubic interpolation methods using a 2×2 image.

Now, let’s do the same using OpenCV on a real image. First, let’s take an image, either you can load one or can make own image. Loading an image from the device looks like this

This is a 20×22 apple image that looks like this.

Now, let’s zoom it 10 times using each interpolation method. The OpenCV command for doing this is

where fx and fy are scale factors along x and y, dsize refers to the output image size and the interpolation flag refers to which method we are going to use. Either you specify (fx, fy) or dsize, OpenCV calculates the other automatically. Let’s see how to use this function

Nearest Neighbor Interpolation

In this we use cv2.INTER_NEAREST as the interpolation flag in the cv2.resize() function as shown below

Output: 

Clearly, this produces a pixelated or blocky image. Also, it doesn’t introduce any new data.

Bilinear Interpolation

In this we use cv2.INTER_LINEAR flag as shown below

Output: 

This produces a smooth image than the nearest neighbor but the results for sharp transitions like edges are not ideal because the results are a weighted average of 2 surrounding pixels.

Bicubic Interpolation

In this we use cv2.INTER_CUBIC flag as shown below

Output: 

Clearly, this produces a sharper image than the above 2 methods. See the white patch on the left side of the apple. This method balances processing time and output quality fairly well.

Next time, when you are resizing an image using any software, wisely use the interpolation method as this can affect your result to a great extent. Hope you enjoy reading.

If you have any doubts/suggestions please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Image Demosaicing or Interpolation methods

In the previous blog, we discussed the Bayer filter and how we can form a color image from a Bayer image. But we didn’t discuss much about interpolation or demosaicing algorithms so in this blog let’s discuss these algorithms in detail.

According to Wikipedia, Interpolation is a method of constructing new data points within the range of a discrete set of known data points. Image interpolation refers to the “guess” of intensity values at missing locations.

The big question is why we need interpolation if we are able to capture intensity values at all the pixels using Image sensor?

  1.  Bayer filter, where we need to find missing color information at each pixel.
  2.  Projecting low-resolution image to a high-resolution screen or vice versa. For example, we prefer watching videos in the full-screen mode.
  3.  Image Inpainting, Image Warping etc.
  4.  Geometric Transformations.

There are plenty of Interpolation methods available but we will discuss only the frequently used. Interpolation algorithms can be classified as

Non-adaptive perform interpolation in a fixed pattern for every pixel, while adaptive algorithms detect local spatial features, like edges, of the pixel neighborhood and make effective choices depending on the algorithm.

Let’s discuss the maths behind each interpolation method in the subsequent blogs.

In the next blog, we will see how the nearest neighbor method works. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Bayer filter

In the previous blog, we saw different methods by which color image can be obtained from an image sensor. Out of these methods, the Bayer filter is most widely used today and in this blog, we will discuss it in detail.

To form a color image, we need to collect information at RGB wavelengths for all the pixels or sensors. But this process is expensive both in terms of time and money.

So in 1976, Bayer thought of an alternative. Instead of capturing all RGB information at each pixel, Bayer thought of capturing one out of RGB for each pixel. Now, each pixel will contain either R, G or B. To be able to form a color image, he decided 50% pixels be Green and rest equally to Red and Blue (to mimic human eye) and these are arranged in a pattern as shown below

He would then use interpolation or color demosaicing algorithm to find the missing information for example pixel capturing Red will need Green and Blue and so on. We will study in more detail about interpolation algorithms in next blog.

The overall procedure from Bayer to RGB color image can be summarized as

Source: Thesis

So, with Bayer filter, we are only storing one color information(either R, G or B) at each pixel which reduces the computation time and cost while maintaining the image quality. That’s why it is used widely.

Hope you understand the Bayer filter, why it is used and how the color image is obtained from the Bayer image.Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Understanding Images with OpenCV-Python

In the previous blogs, we learned about pixels, intensity value, color, and greyscale image. Now, with OpenCV and numpy, let’s try to visualize these concepts.

First, we need an image, either you can load one or can make own image. Loading an image from the device looks like this

To access the pixel location, we must first know the shape of the image. This can be done by

It returns a tuple of the number of rows, columns, and channels (if the image is color).

Total number of pixels can be found either by multiplying rows, columns, channels found using img.shape or by using the following command

After knowing the image shape, we can access the pixel location by its row and column coordinates as

This returns the intensity value at that pixel location. For a greyscale image, intensity or pixel value is a single integer while for a color image, it is an array of Blue, Green, Red values.

Note: OpenCV reads the color image in BGR mode and not in RGB mode. Be careful

We know that intensity levels depend on the number of bits that can be found by

To access the RGB channels separately, use numpy indexing as shown below

You can change the pixel value just by normal assignment as shown below

You can change the color image to greyscale using the following command

All the operations that you can perform on the array like add, subtract etc apply to images also.

Play with all these commands to understand better. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Greyscale and Color Image

In the previous blog, we learned how the digital image is formed but we didn’t discuss whether the image formed was greyscale or colored. In this blog, let’s go through this.

A greyscale image is the one where pixels contains only the intensity information and not wavelength. For example, a pixel value of 165 represents the amount of light captured by the sensor(pixel). It doesn’t tell how much of this light belongs to red wavelength or any other.

For an 8 bit image, there will be 28=256 intensity levels where 0 means no light(black) and 255 means White light and in between levels represents shades of grey as shown below

The above image is created using numpy and OpenCV (See here for code).

A color image, on the other hand, contains intensity information corresponding to three wavelengths red, green, and blue (RGB)collectively called primary colors or channels. The reason for choosing these 3 colors lies in the fact that cone cells in the human eye, that is responsible for color vision, are sensitive to red, green, and blue light.

Note: Combining the primary colors(their wavelength may vary) in various intensity proportions, we can produce all visible colors.

So, in color image, corresponding to every pixel, we have 3 intensity values Red, Green, Blue (for example 160,30,45) that produces different colors in the image.

How a Color image is formed?

To form a color image, we want something that absorbs light in red, green, and blue wavelengths. The advancements in color image formation can be summarized by the figure below

Source: Foveon

The drastic advancement in color image formation came with the introduction of digital sensors like CCD and CMOS. Most of the digital cameras today only contain one imaging sensor so they cannot collect red, green, and blue information simultaneously at each pixel location. One possible solution is we use 3 imaging sensors with RGB filters but that is expensive in terms of money and time.

So, in 1976 Bryce Bayer of Eastman Kodak, invented the Bayer filter that revolutionized color image formation. Bayer filter is a color filter array(CFA) in which RGB color filters are arranged in a pattern on the sensor. Below figure shows a Bayer filter.

Source: Wikipedia

In the bayer filter, there are twice as many green elements as red or blue to mimic the human eye’s greater resolving power with the green light. In order to construct an RGB picture, we calculate Green and Blue values for each Red pixel, Blue and Red values for each Green pixel and Red and Green values for each Blue pixel through interpolation or color demosaicing algorithm (For more details See here).

Foveon X3 direct image sensor is the most advanced color image sensor. This combines the power of digital sensor with the essence of the film. Like the film, this has three layers of pixels embedded in silicon. The layers are positioned to take advantage of the fact that silicon absorbs red, green, and blue light to different depths. The bottom layer records red, the middle layer records green and the top layer records blue. Pros are superior quality and less processing power.

Now, you might have got some feeling about the Color image and how it is formed. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Create own image using Numpy and OpenCV

In this tutorial, we will learn how we can create our own image using numpy and OpenCV. This will help you in understanding the image concepts more. Let’s see how the 256 intensity levels for an 8-bit image looks like.

Steps:

  1.  Create an array of any desired size using numpy. Always specify the ‘datatype’
  2.  Fill the values of the array using some logic
  3.  Show the image using cv2.imshow() or matplotlib.

Code:

The resulting image looks like this

To create a color image just specify the third dimension of the array (for example (100,256,3)) and rest is same.

Now, you are ready to create your own images. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.