Author Archives: kang & atul

Genetic Algorithm and its usage in neural network

Leave a reply

You might have heard about the theory of evolution by natural selection. If not then read this quote by Charles Darwin ” It is not the strongest of the species that survives, nor the most intelligent; it is the one most adaptable to change. ” Genetic Algorithm is also based on this theory.

In the 1970’s, Jon Holland tries to mimic some processes observed in natural evolution by introducing genetic algorithm. This algorithm can be used in optimization and search problems both.

A typical genetic algorithm requires some population in the solution domain and a fitness function to find the fittest individual. To evolve individuals in the population genetic algorithm uses some operations like crossover, mutation, and selection.

Genetic algorithm starts with some random initial population. Then tries to produce offspring from the best individuals in the population. The concept is that, if the fittest individuals are selected then chances of producing a better offspring is more. This process keeps on iteration until your target is not achieved. Each iteration is known as the generation.

Initial Population

Initial Population refers to a set of possible solutions. Each member (individual) of the population is usually known as the chromosome (phenotypes) and represents a solution for the problem to be investigated. The chromosome is represented as a set of parameters (features or genes or weights) that defines the individual. Size of the population depends totally on your problem. Random selection of initial population makes sure that it covers a wide range of possible solution.

Evaluation and Fitness Function

Now, we have a random initial population, next thing is to evaluate the fitness of these individuals. To evaluate the fitness of these individuals, you need to define some fitness function. You need to choose the fitness function according to your problem. Fitness function measures the quality of each individual.

Selection

Some best individuals are selected from the evaluated population. These selected individuals are mated to produce some new offspring.

Crossover

Each individual selected in the previous step has some quality. Our objective is to produce better offspring so that our algorithm can evolve and find a better solution to the problem. To do that two individuals from the best population are selected and a new child (offspring) is produced with features of both as shown above. This is known as Crossover.

Mutation

Mutation is applied to maintain the diversity within the population and inhibit premature convergence. With some low probability, a portion of the new individual is subjected to mutation as shown in the figure above.

Replacement

New population replaces a previous one for the next generation. This process keeps on iterating until a certain target is not achieved.

Application of genetic algorithm in neural networks:

  1. Training of a neural network ( instead of using gradient descent, Adam etc.)
  2. Selection of neural network architecture ( Hyperparameters selection)

Now, you might have got some feeling about the genetic algorithm. In the next blog, we will see how this concept can be applied to train the neural network to play a snake game. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Bayer filter

Leave a reply

In the previous blog, we saw different methods by which color image can be obtained from an image sensor. Out of these methods, the Bayer filter is most widely used today and in this blog, we will discuss it in detail.

To form a color image, we need to collect information at RGB wavelengths for all the pixels or sensors. But this process is expensive both in terms of time and money.

So in 1976, Bayer thought of an alternative. Instead of capturing all RGB information at each pixel, Bayer thought of capturing one out of RGB for each pixel. Now, each pixel will contain either R, G or B. To be able to form a color image, he decided 50% pixels be Green and rest equally to Red and Blue (to mimic human eye) and these are arranged in a pattern as shown below

He would then use interpolation or color demosaicing algorithm to find the missing information for example pixel capturing Red will need Green and Blue and so on. We will study in more detail about interpolation algorithms in next blog.

The overall procedure from Bayer to RGB color image can be summarized as

Source: Thesis

So, with Bayer filter, we are only storing one color information(either R, G or B) at each pixel which reduces the computation time and cost while maintaining the image quality. That’s why it is used widely.

Hope you understand the Bayer filter, why it is used and how the color image is obtained from the Bayer image.Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Understanding Images with OpenCV-Python

Leave a reply

In the previous blogs, we learned about pixels, intensity value, color, and greyscale image. Now, with OpenCV and numpy, let’s try to visualize these concepts.

First, we need an image, either you can load one or can make own image. Loading an image from the device looks like this

To access the pixel location, we must first know the shape of the image. This can be done by

It returns a tuple of the number of rows, columns, and channels (if the image is color).

Total number of pixels can be found either by multiplying rows, columns, channels found using img.shape or by using the following command

After knowing the image shape, we can access the pixel location by its row and column coordinates as

This returns the intensity value at that pixel location. For a greyscale image, intensity or pixel value is a single integer while for a color image, it is an array of Blue, Green, Red values.

Note: OpenCV reads the color image in BGR mode and not in RGB mode. Be careful

We know that intensity levels depend on the number of bits that can be found by

To access the RGB channels separately, use numpy indexing as shown below

You can change the pixel value just by normal assignment as shown below

You can change the color image to greyscale using the following command

All the operations that you can perform on the array like add, subtract etc apply to images also.

Play with all these commands to understand better. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Greyscale and Color Image

Leave a reply

In the previous blog, we learned how the digital image is formed but we didn’t discuss whether the image formed was greyscale or colored. In this blog, let’s go through this.

A greyscale image is the one where pixels contains only the intensity information and not wavelength. For example, a pixel value of 165 represents the amount of light captured by the sensor(pixel). It doesn’t tell how much of this light belongs to red wavelength or any other.

For an 8 bit image, there will be 28=256 intensity levels where 0 means no light(black) and 255 means White light and in between levels represents shades of grey as shown below

The above image is created using numpy and OpenCV (See here for code).

A color image, on the other hand, contains intensity information corresponding to three wavelengths red, green, and blue (RGB)collectively called primary colors or channels. The reason for choosing these 3 colors lies in the fact that cone cells in the human eye, that is responsible for color vision, are sensitive to red, green, and blue light.

Note: Combining the primary colors(their wavelength may vary) in various intensity proportions, we can produce all visible colors.

So, in color image, corresponding to every pixel, we have 3 intensity values Red, Green, Blue (for example 160,30,45) that produces different colors in the image.

How a Color image is formed?

To form a color image, we want something that absorbs light in red, green, and blue wavelengths. The advancements in color image formation can be summarized by the figure below

Source: Foveon

The drastic advancement in color image formation came with the introduction of digital sensors like CCD and CMOS. Most of the digital cameras today only contain one imaging sensor so they cannot collect red, green, and blue information simultaneously at each pixel location. One possible solution is we use 3 imaging sensors with RGB filters but that is expensive in terms of money and time.

So, in 1976 Bryce Bayer of Eastman Kodak, invented the Bayer filter that revolutionized color image formation. Bayer filter is a color filter array(CFA) in which RGB color filters are arranged in a pattern on the sensor. Below figure shows a Bayer filter.

Source: Wikipedia

In the bayer filter, there are twice as many green elements as red or blue to mimic the human eye’s greater resolving power with the green light. In order to construct an RGB picture, we calculate Green and Blue values for each Red pixel, Blue and Red values for each Green pixel and Red and Green values for each Blue pixel through interpolation or color demosaicing algorithm (For more details See here).

Foveon X3 direct image sensor is the most advanced color image sensor. This combines the power of digital sensor with the essence of the film. Like the film, this has three layers of pixels embedded in silicon. The layers are positioned to take advantage of the fact that silicon absorbs red, green, and blue light to different depths. The bottom layer records red, the middle layer records green and the top layer records blue. Pros are superior quality and less processing power.

Now, you might have got some feeling about the Color image and how it is formed. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Create own image using Numpy and OpenCV

Leave a reply

In this tutorial, we will learn how we can create our own image using numpy and OpenCV. This will help you in understanding the image concepts more. Let’s see how the 256 intensity levels for an 8-bit image looks like.

Steps:

  1.  Create an array of any desired size using numpy. Always specify the ‘datatype’
  2.  Fill the values of the array using some logic
  3.  Show the image using cv2.imshow() or matplotlib.

Code:

The resulting image looks like this

To create a color image just specify the third dimension of the array (for example (100,256,3)) and rest is same.

Now, you are ready to create your own images. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Recording a specific Window using OpenCV-Python

Leave a reply

In this blog, we will learn how to record any window using OpenCV-Python.

Installing Libraries:

  1. To install PIL and pywin32, write

    in the cmd(See here). pywin32 will install win32gui, win32api, and win32con.
  2. For installing winGuiAuto, download winGuiAuto.py from here. Save this as winGuiAuto.py in the python -> lib -> site-packages.

Steps:

  1.  Create the window that you want to record (I used cv2.imshow() for that)
  2.  Give the window name that you want to record in winGuiAuto.findTopWindow()
  3.  Keep the window on top and set its position using win32gui.SetWindowPos()
  4.  Get the coordinates of the window using win32gui.GetWindowPlacement()
  5.  Grab an image of the area using ImageGrab.grab()
  6.  Append all these images into a list.
  7.  Create a VideoWriter object using cv2.VideoWriter()
  8.  Convert each image color and save it.

Code:

If you want to make a .gif file, uncomment the last part.

Note: This works well for windows 8.1, but you might find some difficulty in capturing full window in windows 10.

Hope you enjoy reading. If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Extracting and Saving Video Frames using OpenCV-Python

29 Replies

In this tutorial, I will show you how to extract and save frames from a video file. Prerequisites are Chapter 1 and 2.

Steps:

  1.  Open the Video file or camera using cv2.VideoCapture()
  2.  Read frame by frame
  3.  Save each frame using cv2.imwrite()
  4.  Release the VideoCapture and destroy all windows

Now, Let’s code

If you don’t have any video, no need to worry. Open the camera instead of the file using cv2.VideoCapture(0) and start extracting frames.

Use: We can perform a number of operations on these frames like crop, flip, reverse etc. save them into a list and iterate over them to get cropped/flipped/reversed video. (See How you can make Video from images)

Hope you enjoy reading. If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time

Creating Video from Images using OpenCV-Python

43 Replies

In this blog, we will do a small project using OpenCV-Python where we will be creating video from image sequences. This project is entirely based on what we read in Chapter 1 and 2. Let’s start

Steps:

  1.  Fetch all the image file names using glob
  2.  Read all the images using cv2.imread()
  3.  Store all the images into a list
  4.  Create a VideoWriter object using cv2.VideoWriter()
  5.  Save the images to video file using cv2.VideoWriter().write()
  6.  Release the VideoWriter and destroy all windows.

Let’s see the code

glob.glob(Pathname) fetches all the filenames present in that path. ‘*.jpg’ means all the jpg files. So, in code glob.glob() fetches the filename of all the jpg files present in that path.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python Generators and Yield

Leave a reply

In the previous blog, we have seen how we can build an iterator using __iter__() and __next__() method. In this blog, we will learn an easy and fast way to make our own iterator.

Generators return an iterator object in a simple fashion. Let’s see how we can make generators in Python:

Generator Function: This is the same as the normal function. The only difference is that we use yield statement instead of return. The return statement terminates the function while the yield statement pauses the function and starts from that point again whenever called. Let’s see by an example

The output looks like this

The Odd() function used above is of type generator and ‘a’ is a generator object.

Calling yield, saves all states(x in above code) and continues from that point on calling again ( next(a) returns 3). To iterate again we have to create a new generator object.

We can also use generators with for loop directly.

So, whenever we use yield instead of return, we end up making a generator function.

Generator Expression: This is similar to the list comprehension. Only replace the square bracket with the round. Let’s see an example

Pros: The generator expression gives one element at a time(Lazy evaluation), unlike list comprehensions that output the entire list. Thus, generator expressions are more memory efficient than list comprehensions.

To find the next element in generator expression, use the next() function. Let’s take above code

So, you have learned 3 methods to build an iterator namely:

  1. using __iter__() and__next__() methods(See Python Iterators)
  2. Generator functions
  3. Generator expressions

Using Generators to create an iterator is a fast, easy way but if you want to add extra methods to the iterator object, better switch to the first method.

Now, you might have got some feeling about the Python Generators and yield. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

 

Python Iterators

Leave a reply

Iterators, this name might be new for most of us, but What if I tell you that you have already used this within for loops, comprehensions etc. So, in this blog, let’s see what are iterators, how they work and how we can build our own iterator.

What is an iterator and how it works?

To understand this, we first need to know what is an iterable

Iterable: It is anything that you are able to loop over like lists, tuples, strings etc.

Iterator: It is the object that does the actual iterating.

So, in order to iterate, you must first convert an iterable into an iterator. This is done by using built-in iter() function.

To get the next element, use built-in next() function on that iterator. next() function raises a StopIteration exception when there are no more elements in that iterator.

Note: If we directly apply next() on iterable like lists etc. it will give a typeerror as

Thus, __iter__() and __ next__() (called the iterator protocol)are the two special methods implemented by the iterator.

Pros:  Iterators are lazy, meaning, they don’t determine what their next item is until you ask them for it. This saves memory, time and can work with infinite sequences.

Building our own iterator:

To build an iterator, we only need the methods __iter__() and __next__(). Below, we show an example that will give us the next Fibonacci term in each iteration.

Now we can create an iterator and iterate through it as follows.

Here, I haven’t called the iter() method because it returns the object itself. You may call this, as this will not affect anything.

We can also use for loop to iterate over our Fib Class as:

Here, for loop doesn’t give stopiteration exception because it uses try-except statement(See here how the for loop works)

Now, you might have got some feeling about the Python iterators and iterable. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.