Category Archives: Gaming with Deep Learning

Creating a Snake Game using OpenCV-Python

Isn’t it interesting to create a snake game using OpenCV-Python? And what if I tell you that you only gonna need

  • cv2.imshow()
  • cv2.waitKey()
  • cv2.putText()
  • cv2.rectangle()

So, let’s get started.

Import Libraries

For this we only need four libraries

Displaying Game Objects

  • Game Window: Here, I have used a 500×500 image as my game window.
  • Snake and Apple: I have used green squares for displaying a snake and a red square for an apple. Each square has a size of 10 units.

Game Rules

Now, let’s define some game rules

  • Collision with boundaries: If the snake collides with the boundaries, it dies.
  • Collision with self: If the snake collides with itself, it should die. For this, we only need to check whether the snake’s head is in snake body or not.
  • Collision with apple: If the snake collides with the apple, the score is increased and the apple is moved to a new location.

Also, on eating apple snake length should increase. Otherwise, snake moves as it is.

  • Snake game has a fixed time for a keypress. If you press any button in that time, the snake should move in that direction otherwise continue moving in the previous direction. Sadly, with OpenCV cv2.waitKey() function, if you hold down the left direction button, the snake starts moving fast in that direction. So, to make the snake movement uniform, i did something like this.

Because cv2.waitKey() returns -1 when no key is pressed, so this ‘k’ stores the first key pressed in that time. Because the while loop is for a fixed time, so it doesn’t matter how fast you pressed a key. It will always wait a fixed time.

  • Snake cannot move backward: Here, I have used the w, a, s, d controls for moving the snake. If the snake was moving right and we pressed the left button, it will continue moving right or in short snake cannot directly move backwards.

After seeing which direction button is pressed, we change our head position

Displaying the final Score

For displaying the final score, i have used cv2.putText() function.

Finally, our snake game is ready and looks like this

The full code can be found here.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game Using Python Curses

In this tutorial we will learn how to create a snake game using python and curses.

What is Curses?

The curses is a library that can be used to create text user interface application. It is terminal controlled library i.e. the code written using curses can only be run through terminal.

Import Libraries

To start with creating a snake game using curses, we first need to import the following libraries:

The above import will work fine for Linux based systems, to make it compatible for windows you need to install curses. To do this you need to download curses for windows according to your python version from python extension packages and then run the following command:

Initializing Game Screen

After importing required libraries, first we need to initialize game screen and get the maximum height and width of the opened terminal screen. Using these height and width we will create a window for the game.

In the above code win.keypad(1) will initiate the keyboard to take the user’s input for the game. Also curses.curs_set(0) will set the cursor mode to be invisible in the screen.

Initialize snake and apple initial positions

Next, we will initialize the snake and apple(food) starting positions in the game screen. Also, we will initialize our game score to be zero.

In the above code win.addch() will add a diamond like symbol in the game screen according to specified apple position.

Specifying Game Over Conditions

For the snake game there are basically two conditions which defines how game will end. First if snake collide with one of the game window boundaries and second if snake collides with itself.

Playing the Game

In this game we will use four keyboard buttons, ‘up’, ‘down’, ‘left’ and ‘right’. To get a user input from keyboard we need to use win.getch() function.

In the above code win.border(0) will create a border around our game screen.

Now we will see the logic to move snake and eat apple. According to the game rules, snake will continue to move in the same direction if user do not press any button. Also snake can not move backward. In case user press any button, we need to update snake head’s position. Let’s see code:

Next there are two situations. One, in next step snake will move to a new position and second, in next step snake will eat apple. In case snake only moves to a new position, we will add one unit at it’s head and remove one unit from its tail according to the pressed direction. Another situation, if snake eats apple then we will add one unit at snake’s head and do not remove from it’s tail. We will also display a new apple at different location. Let’s see the code:

Then finally we will display the score in the screen and quit the game window.

Here are some images of the game that we have just created.

The full code can be found here.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game Using Tensorflow Object Detection API – Part IV

In the last blog we have trained the model and saved the inference graph. In this blog we will learn how to use this inference graph for object detection and how to run our snake game using this trained object detection model.

To play snake game using this trained model, you first need to develop a snake game. But don’t worry you need not to develop it from scratch, you can clone this repository. And if you want to know algorithm behind this code you can follow this blog.

Now we have our snake game next thing is to use this object detection model to play the snake game. To do this we need to run both snake game file and following script from models/research folder simultaneously.

In the above code we need to specify path to our inference graph using ” PATH_TO_CKPT ” variable. Also we need to specify ” PATH_TO_LABELS ” variable with path of object-detection.pbtxt file. Then specify number of classes i.e. 4 in our case.

In the above script we have used ” pyautogui ” to press the button when particular hand gesture for a particular direction is detected.

Finally you can play snake game using your hand gestures. Let see some of the results.

Pretty well yeah. This is all for playing snake game using tensorflow object detection API. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game Using Tensorflow Object Detection API – Part III

In the previous blogs we have seen how to generate data for object detection and convert it into TFRecord format to train the model. In this blog we will learn how to use this data to train the model.

To train the model we will use the pre-trained model and then use transfer learning to train it on our dataset. I have used mobilenet pre trained model. Here is mobilenet model. For its configuration file you can go to model -> research -> object_detection -> samples -> configs ->> ssd_mobilenet_v1_pets.config.
The configuration file that we have downloaded, needs to be edited as per our requirement. In configuration file we have changed the no. of classes, no. of steps in training, path to model checkpoint and path to pbtxt files as shown below.

For the object-detection.pbtxt file, create a pbtxt file and put following text inside it to specify our labels for the problem.

Now go to models -> research -> object detection -> legecy and copy train.py file to models -> research folder.

Then create a folder named images inside models -> research folder. Put your mobilenet model, configuration file, train and test image data folders, and train and test csv label files. Inside training_data folder, create a folder named data and put your train and test TFRecord files. The hierarchy will look like this:

Also create a training folder inside the images folder where model will save its checkpoints. Now run the following command to train the model from models -> research folder.

Time for training your model will depend upon your machine configuration and no. of steps that you have mentioned in the configuration file.

Now we have our trained model and its checkpoints are saved inside the models/research/images/training folder. In order to test this model and use this model to detect objects we need to export the inference graph.

To do this first we need to copy models/research/object_detection/export_inference_graph.py to models/research/ folder. Then inside models/research folder create a folder named “snake” which will save the inference graph. From models -> research folder run the following command:

Now we are having forzen_inference_graph.pb inside models/research/snake folder which will be used to detect object using trained model.

This is all for training the model and saving the inference graph, in the next blog we will see how to use this inference graph for object detection and how to run our snake game using this trained object detection model.

Next Blog: Snake Game Using Tensorflow Object Detection API – Part IV

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game Using Tensorflow Object Detection API – Part II

In the previous blog, we did two things. First, we create a dataset and second we split this into training and test. In this blog, we will learn how to convert this dataset into TFRecord format for training.

Before creating TFRecord file, we just need to do one more step. In the last blog, we have generated XML files using LabelImg. To get labels for training and test dataset, we need to convert these XML files into CSV format. To do this we will use the following code which has been taken from this repository.

In the above main function, you should specify your XML files path for both train and test folder. The generated CSV files will contain columns as filename, width, and height of images, output label of images and coordinates of the annotated rectangular box as shown in the figure below

Once you have your train and test images with labels in CSV format, let’s convert data in TFRecord format.

A TFRecord file store your data as a sequence of binary strings. It has many advantages over normal data formats. To do this we will use the following code which has been taken from this repository. According to your requirement, you need to change the condition for labels at line 31 below.

Save this code in a file named generate_tfrecord.py. Now in order to use this code, first we need to clone tensorflow object detection API. For that do the following:

Then we need to do the following steps to avoid getting error of protoc:

  1. Go to this release link and download protobuf according to your operating system.
  2. Extract the downloaded file and go to bin folder inside it.
  3. Copy protoc.exe file and put in models -> research -> object_detection -> protos folder.
  4. In protos folder run the following command for .proto files.

After cloning this repository, copy generate_tfrecord.py inside models -> research folder and run the following command.

Above commands will generate two files named train.record and test.record which will be used for training of model.

This is all for generating TFRecord file, in the next blog we will perform training and testing of object detection model.

Next Blog: Snake Game Using Tensorflow Object Detection API – Part III

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game Using Tensorflow Object Detection API

Here, we will learn how to use tensorflow object detection API with the computer’s webcam to play a snake game. We will use hand gestures instead of the keyboard.

We will use following steps to play snake game using tensorflow object detection API:

  1. Generate dataset.
  2. Convert train and test datasets into tfrecord format.
  3. Train a pre-trained model using generated data.
  4. Integrate trained model with snake game.
  5. Play the snake game using your own hand gestures.

In this blog, we will cover only the first step i.e. how to create your own training data and the rest steps will be covered in the subsequent blogs.

You can find code here.

Generate Dataset

A snake game problem generally contains four directions to move i.e. up, down, right and left. For each of the four directions, we need to generate at least 100 images per direction. You can use your phone or laptop camera to do this. Try to generate images with a different background for better generalization. Below are some examples of images of hand gestures.

Hand Gestures

Now we have our captured images of hand gestures. The next thing is to annotate these images according to their classes. Which means we need to create rectangular boxes around hand gestures and label them appropriately. Don’t worry there is a tool named LabelImg, which is highly helpful to annotate images to create training and test dataset. To start with LabelImg you can follow there GitHub link. The start screen of Labellmg would look like this.

At the left side of the screen, you can find various options. Click on Open dir and choose the input image folder. Then click on Change save dir and select output folder where generated XML files will be saved. This XML file will contain coordinates of your generated rectangular box in the image, something like this.

To create a rectangular box in the image using Labellmg, you just need to press ‘W’ and then create a box and save it. You can create one or multiple boxes in one image as shown in the figure below. Repeat this for all the images.

Now we have images and their corresponding XML files. Then we will separate this dataset into training and testing in the 90/10 ratio. To do this we need to put 90% of images of each class ‘up’, ‘right’, ‘left’ and ‘down’ and their corresponding XML files in one folder and other 10% in other folder.

That’s all for creating dataset, in the next blog we will see how to create TFRecord files from these datasets which will be used for training of the model.

Next Blog: Snake Game Using Tensorflow Object Detection API – Part II

 Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game using Real Time Speech Recognition

Speech recognition can be very helpful in your daily activities like you can switch on and off your laptops, control your T.V and A.C., and handle other home appliances. In this blog, we will learn a fun activity to play a snake game using voice control. By learning this method you can apply it on other real life applications.

To perform speech recognition, you need to train a model. And training a model requires a large amount of data which requires a lot of time. To save this time I was searching for a pre-trained model. There are some open source api, but those are not that much accurate and fast. Luckily, I got to know about Porcupine. Porcupine is a self-service, highly-accurate, and lightweight wake word (voice control) engine. In this blog, I will show you how to play a snake game using porcupine GitHub repository.

To play a snake game, You first need to develop a snake game. But don’t worry you need not to develop it from scratch, you can clone this repository. And if you want to know algorithm behind this code you can follow this blog.

Now you have a snake game, the next thing is how to use speech recognition to play this game. First, clone the Porcupine repository into your system. It has some pre-trained wake words define in it. You can also use your own wake words. For this problem, I have used four wake words “go left”, “go right”, “go down” and “snake up”.

Here are the steps to play snake game using voice control:

  • First go to Porcupine directory that you have cloned.
  • Then go to tools -> optimizer -> System(windows or linux or mac) -> os type(64 ar 32 bit)
  • Then use pv_porcupine_optimizer.exe file to create wake word files. To do this you need following command
  • Here -r corresponds to resource directory which you can find inside Porcupine directory, -w corresponds to your wake word that you can choose, -o corresponds to the output directory of your wake word and -p corresponds to your platform(windows, linux or mac)
  • To generate four different wake words, I have run the above command four times.
  • Now you have created your wake words, the next thing is to integrate it with your snake game python code.
  • Inside Porcupine folder go to binding -> python. There is a file named porcupine.py
  • Open porcupine. py file and append following code at the last of it. You will also be needed to install pyaudio and pyautogui using pip.
  • Now run both porcupine.py file and your snake game code to play it with your voice control.

Now you have got an idea to use real time speech recognition. Hope you can find some real life applications to apply it.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Genetic Algorithm

Before moving forward, let’s first summarize what we have done till now. In the first blog, we created a snake game using pygame. Then in the next blog, using backpropagation, we let the neural network learn how to play snake game.

In this blog, we will let the genetic algorithm (GA) and neural network(NN) play the snake game (if you are new to genetic algorithm please refer to this blog).

But let’s first clear some doubts which are obvious to come in mind.

What is the advantage of using GA + NN?

We don’t need any training data.

Note: I am not saying this is better than backpropagation but for snake game problem creating a valid training data is a tough task so this is a good option other than reinforcement learning, which we will see later.

Where we will use GA in NN?

Instead of backpropagation, we will update weights using GA.

How we will use GA in NN?

This whole process can be easily summarized in 7 steps:

  1. Creating a snake game and deciding neural network architecture.
  2. Creating an initial population.
  3. Deciding the fitness function.
  4. Play a game for each individual in the population and sort each individual in the population based on the fitness function score.
  5. Select a few top individuals from the population and create the remaining population from these top selected individuals using Crossover and mutation.
  6. The new population is created (meaning the next generation).
  7. Go to step 4 and repeat until the stopping criteria are not satisfied.

Now, I hope most of the things might be clear to you. Let’s see these steps in more detail

Creating a snake game and deciding neural network architecture

Snake game is created with pygame and network architecture is same as that of the previous blog with 7 units in the input layer, 3 units in the output layer with ‘softmax’ and used 2 hidden layers one of 9 units and other of 15 units with ‘relu’ as shown below.

Creating Initial Population

Here, I have chosen 50 individuals in the population and each individual is an array of weights of the neural network. Randomly initialize these individuals. See code below

Deciding the Fitness Function

You can use any fitness function. Here, I have used the following fitness function

On every grasp of food, I have given 5000 reward points and if it collides with the boundary or itself, I have awarded a penalty of 150 points

You can find this inside the run_game_with_ML() code (Last line).

Evaluating the population (Step – 4)

For each individual, a game is played and the fitness function is calculated which is then appended in the list as shown in the code below

Selection, Crossover, and Mutation (Step – 5)

Selection: Now, according to fitness value, some best individuals will be selected from the population and are stored in the ‘parents’ array as shown in the code below

Crossover: To produce children for the next generation, the crossover is used. First, two individuals are randomly selected from the best, then I randomly choose some values from first and some from the second individual to produce new offspring. This process is repeated until the total population size is not achieved as shown below

Mutation: Then, some variations are being added to the newly formed offspring. Here, for each child, I randomly selected 25 weights and mutated them by adding some random value as shown in the code below

New Population Created

With the help of fitness function, crossover and mutation, new population for the next generation is created. Now, next thing is to replace the previous population with this newly formed. Below is the code for this

Now we will repeat this process until our target for certain game score is not achieved. In this problem, I have used 100 generations for training and 2500 steps in a game, which was able to achieve a maximum score of 40. You can choose more number of steps per game and can achieve more score.

The full code can be found here.

In the next blog, we will see how the genetic algorithm can be applied for neural network architecture search. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Genetic Algorithm and its usage in neural network

You might have heard about the theory of evolution by natural selection. If not then read this quote by Charles Darwin ” It is not the strongest of the species that survives, nor the most intelligent; it is the one most adaptable to change. ” Genetic Algorithm is also based on this theory.

In the 1970’s, Jon Holland tries to mimic some processes observed in natural evolution by introducing genetic algorithm. This algorithm can be used in optimization and search problems both.

A typical genetic algorithm requires some population in the solution domain and a fitness function to find the fittest individual. To evolve individuals in the population genetic algorithm uses some operations like crossover, mutation, and selection.

Genetic algorithm starts with some random initial population. Then tries to produce offspring from the best individuals in the population. The concept is that, if the fittest individuals are selected then chances of producing a better offspring is more. This process keeps on iteration until your target is not achieved. Each iteration is known as the generation.

Initial Population

Initial Population refers to a set of possible solutions. Each member (individual) of the population is usually known as the chromosome (phenotypes) and represents a solution for the problem to be investigated. The chromosome is represented as a set of parameters (features or genes or weights) that defines the individual. Size of the population depends totally on your problem. Random selection of initial population makes sure that it covers a wide range of possible solution.

Evaluation and Fitness Function

Now, we have a random initial population, next thing is to evaluate the fitness of these individuals. To evaluate the fitness of these individuals, you need to define some fitness function. You need to choose the fitness function according to your problem. Fitness function measures the quality of each individual.

Selection

Some best individuals are selected from the evaluated population. These selected individuals are mated to produce some new offspring.

Crossover

Each individual selected in the previous step has some quality. Our objective is to produce better offspring so that our algorithm can evolve and find a better solution to the problem. To do that two individuals from the best population are selected and a new child (offspring) is produced with features of both as shown above. This is known as Crossover.

Mutation

Mutation is applied to maintain the diversity within the population and inhibit premature convergence. With some low probability, a portion of the new individual is subjected to mutation as shown in the figure above.

Replacement

New population replaces a previous one for the next generation. This process keeps on iterating until a certain target is not achieved.

Application of genetic algorithm in neural networks:

  1. Training of a neural network ( instead of using gradient descent, Adam etc.)
  2. Selection of neural network architecture ( Hyperparameters selection)

Now, you might have got some feeling about the genetic algorithm. In the next blog, we will see how this concept can be applied to train the neural network to play a snake game. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Creating a Snake Game with pygame

In this tutorial, we will create a snake game with python and pygame. You can find full code here.

What is pygame?

It is a free and open source Python programming language library used for making a multimedia application like games. It is great for beginners as it’s simple and easy to use.

Installing pygame in Python:

There are two ways to do this.

  1. You can directly use pip install pygame in the command window or
  2. Download whl file from this link python extension packages according to your python version. Then use pip install whl_file_name.

Let’s create a snake game using pygame library. To Start with this, we first need to initialize pygame modules as shown below.

pygame.init() will attempt to initialize all the pygame modules for you. Not all pygame modules need to be initialized, but this will automatically initialize the ones that do.

Displaying Game Objects:

After initializing pygame modules, it’s time to display the game window. For that, we need to initialize its dimensions(width and height). Here I have used 500 x 500, but you can choose yours.

display.fill(window_color) will fill white color into game window and pygame.display.update() allows only a portion of the screen to be updated, instead of the entire area. If no argument is passed, it updates the entire Surface area.

This will create a game window as shown below.

Now it’s time to display snake and apple. Here I have used red color for snake and an image for apple. At the start of each game, we want snake’s starting position to be fixed while apple can take any random locationStarting length of the snake is 3 units where each unit is a 10×10 block.

pygame.draw.rect() will draw a rectangle corresponding to given arguments which will represent our snake and display.blit() will show the image of an apple.

This will create a game window as shown below.

The next thing is to decide at what frame rate we want to play our game. This is done using pygame.time.Clock() that creates a “clock” object. Now whenever clock.tick() is called and passed with some number, then it will limit the run time speed of the game. As an example, if clock.tick(20) is called, then the program will never run at more than 20 frames per second.

Game Logic:

Now it’s time to define game rules. As you know, there must be some rules which makes a game playable.

Rule 1: If the snake collapses with itself or with boundaries, game over.

Rule 2:  Moving the snake to a new position

In our game, the snake will continue to move in one direction until a button is pressed. To move the snake, we need to add one unit to the head and remove one unit from the tail.

Another case is when a button is pressed to move in a direction(left, right, up or down). Also, the Snake cannot move backward.

After seeing which direction button is pressed, we need to change our snakes head position.

Rule 3. If the snake eats an apple, the apple moves to a new position and snake length increases.

When the snake eats an apple, we increase our snake’s length and improve the score. To increase snake size, we add one unit at snake’s head. Also, we need to create another apple at random location to proceed the game.

Displaying Final Score:

After the game is over, we want to display the final score. Here, I have defined a function ‘display_final_score’, which takes final score and text to be displayed as its arguments to display in the game window.

This will display our final score as shown below.

Finally, our snake game is created, now it’s time to play it.

Now, you might have got some feeling about creating games using pygame. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.