Tag Archives: pygame

Snake Game using Real Time Speech Recognition

Speech recognition can be very helpful in your daily activities like you can switch on and off your laptops, control your T.V and A.C., and handle other home appliances. In this blog, we will learn a fun activity to play a snake game using voice control. By learning this method you can apply it on other real life applications.

To perform speech recognition, you need to train a model. And training a model requires a large amount of data which requires a lot of time. To save this time I was searching for a pre-trained model. There are some open source api, but those are not that much accurate and fast. Luckily, I got to know about Porcupine. Porcupine is a self-service, highly-accurate, and lightweight wake word (voice control) engine. In this blog, I will show you how to play a snake game using porcupine GitHub repository.

To play a snake game, You first need to develop a snake game. But don’t worry you need not to develop it from scratch, you can clone this repository. And if you want to know algorithm behind this code you can follow this blog.

Now you have a snake game, the next thing is how to use speech recognition to play this game. First, clone the Porcupine repository into your system. It has some pre-trained wake words define in it. You can also use your own wake words. For this problem, I have used four wake words “go left”, “go right”, “go down” and “snake up”.

Here are the steps to play snake game using voice control:

First go to Porcupine directory that you have cloned.
Then go to tools -> optimizer -> System(windows or linux or mac) -> os type(64 ar 32 bit)
Then use pv_porcupine_optimizer.exe file to create wake word files. To do this you need following command

 pv_porcupine_optimizer.exe -r D:/Porcupine/resources/ -w "wake word" -o D:/Porcupine -p windows

1	pv_porcupine_optimizer.exe -r D:/Porcupine/resources/ -w "wake word" -o D:/Porcupine -p windows

Here -r corresponds to resource directory which you can find inside Porcupine directory, -w corresponds to your wake word that you can choose, -o corresponds to the output directory of your wake word and -p corresponds to your platform(windows, linux or mac)
To generate four different wake words, I have run the above command four times.
Now you have created your wake words, the next thing is to integrate it with your snake game python code.
Inside Porcupine folder go to binding -> python. There is a file named porcupine.py
Open porcupine. py file and append following code at the last of it. You will also be needed to install pyaudio and pyautogui using pip.

import pyaudio
import struct
import pyautogui  # to press a button to play the game

# below are the four wake word's path that you have generated earlier
key1 = 'go_down_windows.ppn'
key2 = 'snake_up_windows.ppn'
key3 = 'go_right_windows.ppn'
key4 = 'go_left_windows.ppn'

# this is the library path that you can fnd inside Porcupine -> lib -> system(windows or linux or mac) -> os type( 64 or 32 bit)
library_path = '/Porcupine/lib/windows/amd64/libpv_porcupine.dll' 

# this is model file path can be find inside Porcupine -> lib -> common
model_file_path = '/Porcupine/lib/common/porcupine_params.pv'
keyword_file_paths = [key1, key2, key3, key4]
sensitivities = [0.5,0.5,0.5,0.5]
handle = Porcupine(library_path, model_file_path, keyword_file_paths=keyword_file_paths, sensitivities=sensitivities)

def get_next_audio_frame():
    pa = pyaudio.PyAudio()
    audio_stream = pa.open(rate=handle.sample_rate,channels=1,format=pyaudio.paInt16,input=True,frames_per_buffer=handle.frame_length,input_device_index=None)
    pcm = audio_stream.read(handle.frame_length)
    pcm = struct.unpack_from("h" * handle.frame_length, pcm)
    return pcm

while True:
    pcm = get_next_audio_frame()
    keyword_index = handle.process(pcm)
    if keyword_index==1:
        print(keyword_index)
        pyautogui.press('up')
    if keyword_index==3:
        print(keyword_index)
        pyautogui.press('left')
    if keyword_index==2:
        print(keyword_index)
        pyautogui.press('right')
    if keyword_index==0:
        print(keyword_index)
        pyautogui.press('down')

import pyaudio

import struct

import pyautogui # to press a button to play the game

# below are the four wake word's path that you have generated earlier

key1 = 'go_down_windows.ppn'

key2 = 'snake_up_windows.ppn'

key3 = 'go_right_windows.ppn'

key4 = 'go_left_windows.ppn'

# this is the library path that you can fnd inside Porcupine -> lib -> system(windows or linux or mac) -> os type( 64 or 32 bit)

library_path = '/Porcupine/lib/windows/amd64/libpv_porcupine.dll'

# this is model file path can be find inside Porcupine -> lib -> common

model_file_path = '/Porcupine/lib/common/porcupine_params.pv'

keyword_file_paths = [key1, key2, key3, key4]

sensitivities = [0.5,0.5,0.5,0.5]

handle = Porcupine(library_path, model_file_path, keyword_file_paths=keyword_file_paths, sensitivities=sensitivities)

def get_next_audio_frame():

pa = pyaudio.PyAudio()

audio_stream = pa.open(rate=handle.sample_rate,channels=1,format=pyaudio.paInt16,input=True,frames_per_buffer=handle.frame_length,input_device_index=None)

pcm = audio_stream.read(handle.frame_length)

pcm = struct.unpack_from("h" * handle.frame_length, pcm)

return pcm

while True:

pcm = get_next_audio_frame()

keyword_index = handle.process(pcm)

if keyword_index==1:

print(keyword_index)

pyautogui.press('up')

if keyword_index==3:

print(keyword_index)

pyautogui.press('left')

if keyword_index==2:

print(keyword_index)

pyautogui.press('right')

if keyword_index==0:

print(keyword_index)

pyautogui.press('down')

Now run both porcupine.py file and your snake game code to play it with your voice control.

Now you have got an idea to use real time speech recognition. Hope you can find some real life applications to apply it.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Genetic Algorithm

Creating a snake game and deciding neural network architecture

Snake game is created with pygame and network architecture is same as that of the previous blog with 7 units in the input layer, 3 units in the output layer with ‘softmax’ and used 2 hidden layers one of 9 units and other of 15 units with ‘relu’ as shown below.

Creating Initial Population

Here, I have chosen 50 individuals in the population and each individual is an array of weights of the neural network. Randomly initialize these individuals. See code below

# n_x   no. of input units
# n_h   no. of units in hidden layer 1
# n_h2  no. of units in hidden layer 2
# n_y   no. of output units

# The population will have sol_per_pop chromosome where each chromosome has num_weights genes.
sol_per_pop = 50
num_weights = n_x*n_h + n_h*n_h2 + n_h2*n_y

# Defining the population size.
pop_size = (sol_per_pop,num_weights)
#Creating the initial population.
new_population = np.random.choice(np.arange(-1,1,step=0.01),size=pop_size,replace=True)

# n_x no. of input units

# n_h no. of units in hidden layer 1

# n_h2 no. of units in hidden layer 2

# n_y no. of output units

# The population will have sol_per_pop chromosome where each chromosome has num_weights genes.

sol_per_pop = 50

num_weights = n_x*n_h + n_h*n_h2 + n_h2*n_y

# Defining the population size.

pop_size = (sol_per_pop,num_weights)

#Creating the initial population.

new_population = np.random.choice(np.arange(-1,1,step=0.01),size=pop_size,replace=True)

Deciding the Fitness Function

You can use any fitness function. Here, I have used the following fitness function

“On every grasp of food, I have given 5000 reward points and if it collides with the boundary or itself, I have awarded a penalty of 150 points“

You can find this inside the run_game_with_ML() code (Last line).

Evaluating the population (Step – 4)

For each individual, a game is played and the fitness function is calculated which is then appended in the list as shown in the code below

def cal_pop_fitness(pop):
    fitness = []
    for i in range(pop.shape[0]):
        fit = run_game_with_ML(display,clock,pop[i])
        print('fitness value of chromosome '+ str(i) +' :  ', fit)
        fitness.append(fit)
    return np.array(fitness)

fitness = cal_pop_fitness(new_population)

def cal_pop_fitness(pop):

fitness = []

for i in range(pop.shape[0]):

fit = run_game_with_ML(display,clock,pop[i])

print('fitness value of chromosome '+ str(i) +' : ', fit)

fitness.append(fit)

return np.array(fitness)

fitness = cal_pop_fitness(new_population)

Selection, Crossover, and Mutation (Step – 5)

Selection: Now, according to fitness value, some best individuals will be selected from the population and are stored in the ‘parents’ array as shown in the code below

def select_mating_pool(pop, fitness, num_parents):
    # Selecting the best individuals in the current generation as parents for producing the offspring of the next generation.
    parents = np.empty((num_parents, pop.shape[1]))
    for parent_num in range(num_parents):
        max_fitness_idx = np.where(fitness == np.max(fitness))
        max_fitness_idx = max_fitness_idx[0][0]
        parents[parent_num, :] = pop[max_fitness_idx, :]
        fitness[max_fitness_idx] = -99999999
    return parents

parents = select_mating_pool(new_population, fitness, num_parents_mating)

def select_mating_pool(pop, fitness, num_parents):

# Selecting the best individuals in the current generation as parents for producing the offspring of the next generation.

parents = np.empty((num_parents, pop.shape[1]))

for parent_num in range(num_parents):

max_fitness_idx = np.where(fitness == np.max(fitness))

max_fitness_idx = max_fitness_idx[0][0]

parents[parent_num, :] = pop[max_fitness_idx, :]

fitness[max_fitness_idx] = -99999999

return parents

parents = select_mating_pool(new_population, fitness, num_parents_mating)

Crossover: To produce children for the next generation, the crossover is used. First, two individuals are randomly selected from the best, then I randomly choose some values from first and some from the second individual to produce new offspring. This process is repeated until the total population size is not achieved as shown below

def crossover(parents, offspring_size):
    offspring = np.empty(offspring_size)

    for k in range(offspring_size[0]):

        while True:
            parent1_idx = random.randint(0, parents.shape[0] - 1)
            parent2_idx = random.randint(0, parents.shape[0] - 1)
            if parent1_idx != parent2_idx:
                for j in range(offspring_size[1]):
                    if random.uniform(0, 1) < 0.5:
                        offspring[k, j] = parents[parent1_idx, j]
                    else:
                        offspring[k, j] = parents[parent2_idx, j]
                break
    return offspring

# Generating next generation using crossover.
offspring_crossover = crossover(parents, offspring_size=(pop_size[0] - parents.shape[0], num_weights))

def crossover(parents, offspring_size):

offspring = np.empty(offspring_size)

for k in range(offspring_size[0]):

while True:

parent1_idx = random.randint(0, parents.shape[0] - 1)

parent2_idx = random.randint(0, parents.shape[0] - 1)

if parent1_idx != parent2_idx:

for j in range(offspring_size[1]):

if random.uniform(0, 1) < 0.5:

offspring[k, j] = parents[parent1_idx, j]

else:

offspring[k, j] = parents[parent2_idx, j]

break

return offspring

# Generating next generation using crossover.

offspring_crossover = crossover(parents, offspring_size=(pop_size[0] - parents.shape[0], num_weights))

Mutation: Then, some variations are being added to the newly formed offspring. Here, for each child, I randomly selected 25 weights and mutated them by adding some random value as shown in the code below

def mutation(offspring_crossover):

    for idx in range(offspring_crossover.shape[0]):
        for _ in range(25):
            i = randint(0,offspring_crossover.shape[1]-1)

        random_value = np.random.choice(np.arange(-1,1,step=0.001),size=(1),replace=False)
        offspring_crossover[idx, i] = offspring_crossover[idx, i] + random_value

    return offspring_crossover

offspring_mutation = mutation(offspring_crossover)

def mutation(offspring_crossover):

for idx in range(offspring_crossover.shape[0]):

for _ in range(25):

i = randint(0,offspring_crossover.shape[1]-1)

random_value = np.random.choice(np.arange(-1,1,step=0.001),size=(1),replace=False)

offspring_crossover[idx, i] = offspring_crossover[idx, i] + random_value

return offspring_crossover

offspring_mutation = mutation(offspring_crossover)

New Population Created

With the help of fitness function, crossover and mutation, new population for the next generation is created. Now, next thing is to replace the previous population with this newly formed. Below is the code for this

# Creating the new population based on the parents and offspring.
new_population[0:parents.shape[0], :] = parents
new_population[parents.shape[0]:, :] = offspring_mutation

# Creating the new population based on the parents and offspring.

new_population[0:parents.shape[0], :] = parents

new_population[parents.shape[0]:, :] = offspring_mutation

Now we will repeat this process until our target for certain game score is not achieved. In this problem, I have used 100 generations for training and 2500 steps in a game, which was able to achieve a maximum score of 40. You can choose more number of steps per game and can achieve more score.

The full code can be found here.

In the next blog, we will see how the genetic algorithm can be applied for neural network architecture search. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Creating a Snake Game with pygame

7 Replies

In this tutorial, we will create a snake game with python and pygame. You can find full code here.

What is pygame?

It is a free and open source Python programming language library used for making a multimedia application like games. It is great for beginners as it’s simple and easy to use.

Installing pygame in Python:

There are two ways to do this.

You can directly use pip install pygame in the command window or
Download whl file from this link python extension packages according to your python version. Then use pip install whl_file_name.

Let’s create a snake game using pygame library. To Start with this, we first need to initialize pygame modules as shown below.

pygame.init()

1	pygame.init()

pygame.init() will attempt to initialize all the pygame modules for you. Not all pygame modules need to be initialized, but this will automatically initialize the ones that do.

Displaying Game Objects:

After initializing pygame modules, it’s time to display the game window. For that, we need to initialize its dimensions(width and height). Here I have used 500 x 500, but you can choose yours.

display_width = 500 
display_height = 500    
display = pygame.display.set_mode((display_width,display_height))
window_color= (200,200,200)
display.fill(window_color)
pygame.display.update()

display_width = 500

display_height = 500

display = pygame.display.set_mode((display_width,display_height))

window_color= (200,200,200)

display.fill(window_color)

pygame.display.update()

display.fill(window_color) will fill white color into game window and pygame.display.update() allows only a portion of the screen to be updated, instead of the entire area. If no argument is passed, it updates the entire Surface area.

This will create a game window as shown below.

Now it’s time to display snake and apple. Here I have used red color for snake and an image for apple. At the start of each game, we want snake’s starting position to be fixed while apple can take any random location. Starting length of the snake is 3 units where each unit is a 10×10 block.

red = (255,0,0)  
black = (0,0,0)
apple_image = pygame.image.load('apple.jpg') 
snake_head = [250,250] 
snake_position = [[250,250],[240,250],[230,250]] 
apple_position = [random.randrange(1,50)*10,random.randrange(1,50)*10]<strong>

def display_snake(snake_position):
    for position in snake_position:
        pygame.draw.rect(display,red,pygame.Rect(position[0],position[1],10,10))

def display_apple(display,apple_position, apple):
    display.blit(apple,(apple_position[0], apple_position[1]))
</strong>

red = (255,0,0)

black = (0,0,0)

apple_image = pygame.image.load('apple.jpg')

snake_head = [250,250]

snake_position = [[250,250],[240,250],[230,250]]

apple_position = [random.randrange(1,50)*10,random.randrange(1,50)*10]<strong>

def display_snake(snake_position):

for position in snake_position:

pygame.draw.rect(display,red,pygame.Rect(position[0],position[1],10,10))

def display_apple(display,apple_position, apple):

display.blit(apple,(apple_position[0], apple_position[1]))

</strong>

pygame.draw.rect() will draw a rectangle corresponding to given arguments which will represent our snake and display.blit() will show the image of an apple.

This will create a game window as shown below.

The next thing is to decide at what frame rate we want to play our game. This is done using pygame.time.Clock() that creates a “clock” object. Now whenever clock.tick() is called and passed with some number, then it will limit the run time speed of the game. As an example, if clock.tick(20) is called, then the program will never run at more than 20 frames per second.

clock = pygame.time.Clock()
clock.tick(20)

1 2	clock = pygame.time.Clock() clock.tick(20)

Game Logic:

Now it’s time to define game rules. As you know, there must be some rules which makes a game playable.

Rule 1: If the snake collapses with itself or with boundaries, game over.

def collision_with_boundaries(snake_head):
    if snake_head[0]>=500 or snake_head[0]<0 or snake_head[1]>=500 or snake_head[1]<0 :
        return 1
    else:
        return 0

def collision_with_self(snake_position):
    snake_head = snake_position[0]
    if snake_head in snake_position[1:]:
        return 1
    else:
        return 0

def collision_with_boundaries(snake_head):

if snake_head[0]>=500 or snake_head[0]<0 or snake_head[1]>=500 or snake_head[1]<0 :

return 1

else:

return 0

def collision_with_self(snake_position):

snake_head = snake_position[0]

if snake_head in snake_position[1:]:

return 1

else:

return 0

Rule 2: Moving the snake to a new position

In our game, the snake will continue to move in one direction until a button is pressed. To move the snake, we need to add one unit to the head and remove one unit from the tail.

 snake_position.insert(0,list(snake_head))
 snake_position.pop()

1 2	snake_position.insert(0,list(snake_head)) snake_position.pop()

Another case is when a button is pressed to move in a direction(left, right, up or down). Also, the Snake cannot move backward.

 if event.type == pygame.KEYDOWN:
    if event.key == pygame.K_LEFT and prev_button_direction != 1:
        button_direction = 0
    elif event.key == pygame.K_RIGHT and prev_button_direction != 0:
        button_direction = 1
    elif event.key == pygame.K_UP and prev_button_direction != 2:
        button_direction = 3
    elif event.key == pygame.K_DOWN and prev_button_direction != 3:
        button_direction = 2
    else:
        button_direction = button_direction

if event.type == pygame.KEYDOWN:

if event.key == pygame.K_LEFT and prev_button_direction != 1:

button_direction = 0

elif event.key == pygame.K_RIGHT and prev_button_direction != 0:

button_direction = 1

elif event.key == pygame.K_UP and prev_button_direction != 2:

button_direction = 3

elif event.key == pygame.K_DOWN and prev_button_direction != 3:

button_direction = 2

else:

button_direction = button_direction

After seeing which direction button is pressed, we need to change our snakes head position.

if button_direction == 1:
    snake_head[0] += 10
elif button_direction == 0:
    snake_head[0] -= 10
elif button_direction == 2:
    snake_head[1] += 10
elif button_direction == 3:
    snake_head[1] -= 10
else:
    pass

snake_position.insert(0,list(snake_head))
snake_position.pop()

if button_direction == 1:

snake_head[0] += 10

elif button_direction == 0:

snake_head[0] -= 10

elif button_direction == 2:

snake_head[1] += 10

elif button_direction == 3:

snake_head[1] -= 10

else:

pass

snake_position.insert(0,list(snake_head))

snake_position.pop()

Rule 3. If the snake eats an apple, the apple moves to a new position and snake length increases.

When the snake eats an apple, we increase our snake’s length and improve the score. To increase snake size, we add one unit at snake’s head. Also, we need to create another apple at random location to proceed the game.

def collision_with_apple(apple_position, score):
    apple_position = [random.randrange(1,50)*10,random.randrange(1,50)*10]
    score += 1
    return apple_position, score

if snake_head == apple_position:
    apple_position, score = collision_with_apple(apple_position, score)
    snake_position.insert(0,list(snake_head))

def collision_with_apple(apple_position, score):

apple_position = [random.randrange(1,50)*10,random.randrange(1,50)*10]

score += 1

return apple_position, score

if snake_head == apple_position:

apple_position, score = collision_with_apple(apple_position, score)

snake_position.insert(0,list(snake_head))

Displaying Final Score:

After the game is over, we want to display the final score. Here, I have defined a function ‘display_final_score’, which takes final score and text to be displayed as its arguments to display in the game window.

def display_final_score(display_text, final_score):
    largeText = pygame.font.Font('freesansbold.ttf',35)
    TextSurf = largeText.render(display_text, True, black)
    TextRect = TextSurf.get_rect()
    TextRect.center = ((display_width/2),(display_height/2))
    display.blit(TextSurf, TextRect)
    pygame.display.update()
    time.sleep(2)

def display_final_score(display_text, final_score):

largeText = pygame.font.Font('freesansbold.ttf',35)

TextSurf = largeText.render(display_text, True, black)

TextRect = TextSurf.get_rect()

TextRect.center = ((display_width/2),(display_height/2))

display.blit(TextSurf, TextRect)

pygame.display.update()

time.sleep(2)

This will display our final score as shown below.

Finally, our snake game is created, now it’s time to play it.

Now, you might have got some feeling about creating games using pygame. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Neural Network Architecture Selection Using Genetic Algorithm

9 Replies

In the previous blog, I have discussed the genetic algorithm and one of its application in the neural network (Training a neural network with a genetic algorithm ). In this blog, I have used a genetic algorithm to solve the problem of neural network architecture search.

You can find full code here .

Genetic Algorithm is really helpful if you do not want to waste your time in using brute force trial and error method for selecting hyperparameters. To know more about the genetic algorithm you can read this blog.

In this tutorial, to demonstrate the use of genetic algorithm I have used Snake Game with Deep Learning where it’s been difficult to find out which neural network architecture will give us the best results. So, the genetic algorithm can be used to find out the best network architecture among the number of hyperparameters.

Different values of hyperparameters are used to create an initial population. I have used the following parameters in the genetic algorithm to find the best value for them.

Number of hidden Layers.
Units per hidden layer
Activation function
Network optimizer

def hyperparameters():
    parameters = []
    units_per_layer = [20,25,30,35,40]
    layers = [1, 2, 3, 4, 5,6,7]
    activation = ['relu', 'tanh', 'sigmoid']
    optimizer = ['adam', 'rmsprop']
    parameters.append(units_per_layer)
    parameters.append(layers)
    parameters.append(activation)
    parameters.append(optimizer)
    
    return parameters

def hyperparameters():

parameters = []

units_per_layer = [20,25,30,35,40]

layers = [1, 2, 3, 4, 5,6,7]

activation = ['relu', 'tanh', 'sigmoid']

optimizer = ['adam', 'rmsprop']

parameters.append(units_per_layer)

parameters.append(layers)

parameters.append(activation)

parameters.append(optimizer)

return parameters

Creating Initial Population

Random parameters are used to create the Initial population. For creating the population first, you have to decide population size. Each individual in the population will have four values.

I have taken 20 chromosomes in the population.

def generate_population(size):
    parameters = hyperparameters()
    
    population = []
    i=0   
    while i < size:
        chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),
                    random.choice(parameters[2]), random.choice(parameters[3])]
        if chromosome not in population:
            population.append(chromosome)
            i+=1
    return population

def generate_population(size):

parameters = hyperparameters()

population = []

i=0

while i < size:

chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),

random.choice(parameters[2]), random.choice(parameters[3])]

if chromosome not in population:

population.append(chromosome)

i+=1

return population

Fitness Function

Fitness function can vary as per the need of different genetic algorithms. Here, I have used the average score for different network architectures. Individuals with the highest average score are fittest ones.

Selection

After evaluating each individual in the population, I have selected top 5 fittest individuals from the population. And also selected 3 individuals from the non-top performers. This will keep us away from getting stuck in the local maximum.

Remaining 12 individuals are created from these 8 individuals using Crossover.

Crossover and Mutation

To produce better offspring for the next generation, I have selected two parents randomly from the 8 individuals selected above and generated other 12 individuals.

def children_population(size, population):
    
    child_population = []
    k = 0
    population_size = len(population)
    while k < size :
        i, j = random.sample(range(0, population_size), 2)
        parent1 = population[i]
        parent2 = population[j]
         
        child = new_child(parent1, parent2) 
        if child not in child_population and child not in population:
            child_population.append(child)
            k +=1
            
    for newest_child in child_population:
        population.append(newest_child)
        
    return population

def new_child(parent1, parent2):
    child = []
    parent_size = len(parent1)
    i=0
    while i<parent_size:
        child.append(random.choice([parent1[i],parent2[i]]))
        i += 1
    return child

def children_population(size, population):

child_population = []

k = 0

population_size = len(population)

while k < size :

i, j = random.sample(range(0, population_size), 2)

parent1 = population[i]

parent2 = population[j]

child = new_child(parent1, parent2)

if child not in child_population and child not in population:

child_population.append(child)

k +=1

for newest_child in child_population:

population.append(newest_child)

return population

def new_child(parent1, parent2):

child = []

parent_size = len(parent1)

i=0

while i<parent_size:

child.append(random.choice([parent1[i],parent2[i]]))

i += 1

return child

In certain new children formed, some of their genes can be subjected to a mutation with a low random probability. Mutation is required to maintain some amount of randomness in the genetic algorithm.

def mutation(population):
    parameters = hyperparameters()
    for chromosome in population:
        if random.random() < 0.1 :
            key = random.choice([0,1,2,3])
            parameters = hyperparameters()
            mutate_key = random.choice(parameters[key])
            chromosome[key] = mutate_key
    
    return population

def mutation(population):

parameters = hyperparameters()

for chromosome in population:

if random.random() < 0.1 :

key = random.choice([0,1,2,3])

parameters = hyperparameters()

mutate_key = random.choice(parameters[key])

chromosome[key] = mutate_key

return population

Now we have created all the necessary functions required for a genetic algorithm. Now, we define a model function using keras library. Then we will train this model with different hyperparameters and search for the best using genetic algorithm.

def train_model(parameters):
    units_per_layer = parameters[0]
    layers = parameters[1]
    activation = parameters[2]
    optimizer = parameters[3]

    model = Sequential()
    model.add(Dense(units=9,input_dim=9))

    for _ in range(layers):
        model.add(Dense(units=units_per_layer, activation=activation))
        
    model.add(Dense(units = 3,  activation = 'softmax'))
    return model

population_with_avg_score = []

for chromosome in population:
    model = train_model(chromosome)
    model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])
    model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)
    max_score, avg_score = run_game_with_ML(model,display,clock)
    chromosome.append(avg_score)
    chromosome.append(max_score)
    population_with_avg_score.append(chromosome)

def train_model(parameters):

units_per_layer = parameters[0]

layers = parameters[1]

activation = parameters[2]

optimizer = parameters[3]

model = Sequential()

model.add(Dense(units=9,input_dim=9))

for _ in range(layers):

model.add(Dense(units=units_per_layer, activation=activation))

model.add(Dense(units = 3, activation = 'softmax'))

return model

population_with_avg_score = []

for chromosome in population:

model = train_model(chromosome)

model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])

model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

max_score, avg_score = run_game_with_ML(model,display,clock)

chromosome.append(avg_score)

chromosome.append(max_score)

population_with_avg_score.append(chromosome)

Here, I have used 10 generations and 20 individuals in the population. It can vary according to your need.

Now, you might have got some feeling about how the genetic algorithm can be applied to find neural architecture instead of using the brute-force method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Deep Learning

4 Replies

Developing a neural network to play a snake game usually consists of three steps.

Training data generation
Training neural network
Testing

The full code can be found here

In this tutorial, I will guide you to generate training data. To do this, first, we need to develop a snake game for which you can follow this blog.

Training data consists of inputs and corresponding outputs. Here, I have used the following inputs and outputs.

Input is comprised of 7 nodes:

Is left blocked or is there any obstacle in left ( 1 or 0)
is front blocked or is there any obstacle in front (1 or 0)
Is right blocked or is there any obstacle in right(1 or 0)
Apple direction vector from snake (X)
Apple direction vector from snake (Y)
Snake’s current direction vector (X)
Snake’s current direction vector (Y)

our input data will look like this:

The output is comprised of 3 node:

[1,0,0] will move snake left
[0,1,0] will continue snake in same direction
[0,0,1] will move snake right

Now the big question, how to generate this data? You can sit and play as many games as you can, but it is always good when you can generate data automatically. Let’s see how to do this.

Generating Training Data

Here I have generated training data automatically. To do this I have used angle between snake and apple. On the basis of that angle, I have decided in which direction snake should move. First, let’s calculate these.

Calculating angle b/w snake and apple:

To calculate the angle between snake and apple we only require two parameters, snake position and apple position.

In the following code, I have first calculated the snake’s current direction vector and Apple’s direction from the snake’s current position. Snake direction vector can be calculated by simply subtracting 0th index of the snake’s list from the 1st index. And to calculate apple direction from the snake, just subtract 0th index of snake’s list from Apple’s position.

Then normalize these direction vectors and calculate the angle with the help of the math library. The code is as follows:

def angle_with_apple(snake_position, apple_position):
    apple_direction_vector = np.array(apple_position)-np.array(snake_position[0])
    snake_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])
    
    norm_of_apple_direction_vector = np.linalg.norm(apple_direction_vector)
    norm_of_snake_direction_vector = np.linalg.norm(snake_direction_vector)
    if norm_of_apple_direction_vector == 0:
        norm_of_apple_direction_vector = 10
    if norm_of_snake_direction_vector == 0:
        norm_of_snake_direction_vector = 10
        
    apple_direction_vector_normalized = apple_direction_vector/norm_of_apple_direction_vector
    snake_direction_vector_normalized = snake_direction_vector/norm_of_snake_direction_vector
    angle = math.atan2(apple_direction_vector_normalized[1] * snake_direction_vector_normalized[0] - apple_direction_vector_normalized[0] * snake_direction_vector_normalized[1], apple_direction_vector_normalized[1] * snake_direction_vector_normalized[1] + apple_direction_vector_normalized[0] * snake_direction_vector_normalized[0]) / math.pi
    return angle

def angle_with_apple(snake_position, apple_position):

apple_direction_vector = np.array(apple_position)-np.array(snake_position[0])

snake_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])

norm_of_apple_direction_vector = np.linalg.norm(apple_direction_vector)

norm_of_snake_direction_vector = np.linalg.norm(snake_direction_vector)

if norm_of_apple_direction_vector == 0:

norm_of_apple_direction_vector = 10

if norm_of_snake_direction_vector == 0:

norm_of_snake_direction_vector = 10

apple_direction_vector_normalized = apple_direction_vector/norm_of_apple_direction_vector

snake_direction_vector_normalized = snake_direction_vector/norm_of_snake_direction_vector

angle = math.atan2(apple_direction_vector_normalized[1] * snake_direction_vector_normalized[0] - apple_direction_vector_normalized[0] * snake_direction_vector_normalized[1], apple_direction_vector_normalized[1] * snake_direction_vector_normalized[1] + apple_direction_vector_normalized[0] * snake_direction_vector_normalized[0]) / math.pi

return angle

After calculating the angle, next thing is to decide in which direction snake should move.

Calculating direction according to the angle:

If above-calculated angle > 0, this means Apple is on the right side of the snake. So snake should move to the right. For < 0, move left and =0 means continue in same direction. I have used 1, – 1 and 0 for the right, left and front respectively.

I have used the following steps to get the correct button direction (up, down, right, left or 3, 2, 1, 0 respectively) for the next step of the snake.

First, I have calculated the snake’s current direction.
Then to turn the snake to the left or right direction, I have calculated left direction vector or right direction vector from snake’s current direction vector.
Then I have converted the above-calculated direction vector into the button direction.

def generate_next_direction(snake_position, angle_with_apple):
    direction = 0
    
    if angle_with_apple > 0:
        direction = 1
    elif angle_with_apple < 0:
        direction = -1
    else:
        direction = 0
        
    current_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])
    left_direction_vector = np.array([current_direction_vector[1],-current_direction_vector[0]])
    right_direction_vector = np.array([-current_direction_vector[1], current_direction_vector[0]])
    
    new_direction = current_direction_vector
    if direction == -1:
        new_direction = left_direction_vector
    if direction == 1:
        new_direction = right_direction_vector

    button_direction = generate_button_direction(new_direction)
    
    return direction, button_direction

def generate_button_direction(new_direction):
    button_direction = 0
    if new_direction.tolist() == [10,0]:
        button_direction = 1
    elif new_direction.tolist() == [-10,0]:
        button_direction = 0
    elif new_direction.tolist() == [0,10]:
        button_direction = 2
    else:
        button_direction = 3
        
    return button_direction

def generate_next_direction(snake_position, angle_with_apple):

direction = 0

if angle_with_apple > 0:

direction = 1

elif angle_with_apple < 0:

direction = -1

else:

direction = 0

current_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])

left_direction_vector = np.array([current_direction_vector[1],-current_direction_vector[0]])

right_direction_vector = np.array([-current_direction_vector[1], current_direction_vector[0]])

new_direction = current_direction_vector

if direction == -1:

new_direction = left_direction_vector

if direction == 1:

new_direction = right_direction_vector

button_direction = generate_button_direction(new_direction)

return direction, button_direction

def generate_button_direction(new_direction):

button_direction = 0

if new_direction.tolist() == [10,0]:

button_direction = 1

elif new_direction.tolist() == [-10,0]:

button_direction = 0

elif new_direction.tolist() == [0,10]:

button_direction = 2

else:

button_direction = 3

return button_direction

Now, for every step, angle and corresponding next direction are calculated and snake moves according to that. And for each step inputs and outputs are calculated which are appended to a list of training data.To generate training data, we need to keep a record of 7 inputs and 3 outputs for every step the snake takes. First, let’s see how I have calculated the inputs for every step the snake takes.

To check if the direction is blocked, we look one step ahead in each direction.
Snake direction vector = Snake’s Head (0th index) – Snake’s 1st index
Apple direction from the snake = Apple’s position – Snake’s head position (See the figure below)

For every step, the output is generated by first calculating the direction for the given snake and apple position, using angle between them. Now, we need to convert our directions( -1, 0 or 1 ) to output(Y), a one hot vector. For every predicted direction we need to see that if that direction is blocked or not and according to that create output (Y) for training data. The code given below seems to be a bit longer but it calculates our training data output (Y).

def generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked):
    if direction == -1:
        if is_left_blocked == 1:
            if is_front_blocked == 1 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)
                training_data_y.append([0,0,1])
            elif is_front_blocked == 0 and is_right_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)
                training_data_y.append([0,1,0])
            elif is_front_blocked == 0 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1) 
                training_data_y.append([0,0,1])
             
        else:
            training_data_y.append([1,0,0])

    elif direction == 0:
        if is_front_blocked == 1:
            if is_left_blocked == 1 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)
                training_data_y.append([0,0,1])
            elif is_left_blocked == 0 and is_right_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])
            elif is_left_blocked == 0 and is_right_blocked == 0:
                training_data_y.append([0,0,1])
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)           
        else:
            training_data_y.append([0,1,0])
    else:
        if is_right_blocked == 1:
            if is_left_blocked == 1 and is_front_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)
                training_data_y.append([0,1,0])
            elif is_left_blocked == 0 and is_front_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])
            elif is_left_blocked == 0 and is_front_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])               
        else:
            training_data_y.append([0,0,1])
    
    return direction, button_direction, training_data_y

def generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked):

if direction == -1:

if is_left_blocked == 1:

if is_front_blocked == 1 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

elif is_front_blocked == 0 and is_right_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)

training_data_y.append([0,1,0])

elif is_front_blocked == 0 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

else:

training_data_y.append([1,0,0])

elif direction == 0:

if is_front_blocked == 1:

if is_left_blocked == 1 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

elif is_left_blocked == 0 and is_right_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

elif is_left_blocked == 0 and is_right_blocked == 0:

training_data_y.append([0,0,1])

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

else:

training_data_y.append([0,1,0])

else:

if is_right_blocked == 1:

if is_left_blocked == 1 and is_front_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)

training_data_y.append([0,1,0])

elif is_left_blocked == 0 and is_front_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

elif is_left_blocked == 0 and is_front_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

else:

training_data_y.append([0,0,1])

return direction, button_direction, training_data_y

Here, I have used 1000 games for generating training data, each of which consists of 2000 steps. For every game, I have re-initialized snake position, apple position, and score. Then, created two empty lists, one for input training data(X) and another output training data(Y), those will contain our whole training data.The code is as follows:

def generate_training_data(display,clock):
    
    training_data_x = []
    training_data_y = []
    training_games = 1000
    steps_per_game = 2000
    
    for _ in tqdm(range(training_games)):
        snake_start, snake_position, apple_position, score = starting_positions()
        prev_apple_distance = apple_distance_from_snake(apple_position, snake_position)
        
        for _ in range(steps_per_game):
            angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)
            direction, button_direction = generate_random_direction(snake_position, angle)
            current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)
            
            direction, button_direction, training_data_y = generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked)
            
            if is_front_blocked == 1 and is_left_blocked == 1 and is_right_blocked == 1:
                break

            training_data_x.append([is_left_blocked, is_front_blocked, is_right_blocked,apple_direction_vector_normalized[0], \
                        snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \
                        snake_direction_vector_normalized[1]])
            
            
            
            
            snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)   

    return training_data_x, training_data_y

def generate_training_data(display,clock):

training_data_x = []

training_data_y = []

training_games = 1000

steps_per_game = 2000

for _ in tqdm(range(training_games)):

snake_start, snake_position, apple_position, score = starting_positions()

prev_apple_distance = apple_distance_from_snake(apple_position, snake_position)

for _ in range(steps_per_game):

angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)

direction, button_direction = generate_random_direction(snake_position, angle)

current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)

direction, button_direction, training_data_y = generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked)

if is_front_blocked == 1 and is_left_blocked == 1 and is_right_blocked == 1:

break

training_data_x.append([is_left_blocked, is_front_blocked, is_right_blocked,apple_direction_vector_normalized[0], \

snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \

snake_direction_vector_normalized[1]])

snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)

return training_data_x, training_data_y

You might have got some feeling about the training data generation for the snake game with deep learning. In the next blog, we will use this data to train and test our neural network. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Deep Learning Part-2

2 Replies

This is the second part of the snake game with deep learning series. In my previous blog, we have seen that how to generate training data for the neural network. In this tutorial, we will see training and testing of the neural network from generated training data.

The full code can be found here.

Our neural network is comprised of 7 nodes in the input layer, 3 node in the final layer and some hidden layers.

Network Architecture:

Now, it’s time to choose hidden layers and corresponding hyperparameters. Its always been difficult to find the perfect neural network architecture. There are some algorithms that can help to find the best network architecture for a neural network like a genetic algorithm, NAS, autoML etc. I have explained neural architecture search using the genetic algorithm in this blog.

In this blog, I have used hit and trial method to find network architecture. After some hit and trials, I have found a workable architecture, which consists of 2 hidden layers one of 9 units and other of 15 units. For the hidden layer, I have used the non-linear function ‘relu’ and for the output layer, I have used ‘softmax’.

You can use different libraries to train this model like keras, tflearn, etc. Here I have used keras. Our network architecture is as follows:

model = Sequential()
model.add(Dense(units=9,input_dim=7))

model.add(Dense(units=15, activation='relu'))
model.add(Dense(output_dim=3,  activation = 'softmax'))

model = Sequential()

model.add(Dense(units=9,input_dim=7))

model.add(Dense(units=15, activation='relu'))

model.add(Dense(output_dim=3, activation = 'softmax'))

Train Neural Network

Our model is prepared, now it’s time to train this. For training, we first need to compile this model then call a method model.fit() which will do the rest. Since our training data is a list, we first need to change it into numpy array and then reshape it. The reason for this is, a sequential model from keras expects numpy array or sparse matrix of shape [n_samples,n_features].

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

1 2	model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy']) model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

Now, our model is trained with generated training data. Next thing is to test it and see how much is learned.

Test Snake Game

Now it’s time to test our trained snake. To predict the direction we have fed our model with input values. Then used the predicted direction(Left, Straight or Front) to take the next step in our test games. For the new position, again predict the direction and move the snake. This continues until the snake dies or steps are over.

At last, we have calculated the maximum and average score for all the games in our test set.

def run_game_with_ML(model,display,clock):
    max_score = 3
    avg_score = 0
    test_games = 1000
    steps_per_game = 2000
    
    for _ in range(test_games):
        snake_start, snake_position, apple_position, score = starting_positions()
        
        count_same_direction = 0
        prev_direction = 0

        for _ in range(steps_per_game):
            current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)
            angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)
            predictions = []

            predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \
                                    is_right_blocked,apple_direction_vector_normalized[0], \
                                    snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \
                                            snake_direction_vector_normalized[1]]).reshape(-1,7))))-1
            
            
            if predicted_direction == prev_direction:
                count_same_direction+=1
            else:
                count_same_direction = 0
                prev_direction = predicted_direction
            
            new_direction = np.array(snake_position[0]) - np.array(snake_position[1])
            if predicted_direction == -1:
                new_direction = np.array([new_direction[1],-new_direction[0]])
            if predicted_direction == 1:
                new_direction = np.array([-new_direction[1],new_direction[0]])
            
            button_direction = generate_button_direction(new_direction)
            
            next_step = snake_position[0]+ current_direction_vector
            if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1:
                break
            snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)  
            
                                         
            if score > max_score:
                max_score = score
                
        avg_score += score
       
    return max_score, avg_score/1000

def run_game_with_ML(model,display,clock):

max_score = 3

avg_score = 0

test_games = 1000

steps_per_game = 2000

for _ in range(test_games):

snake_start, snake_position, apple_position, score = starting_positions()

count_same_direction = 0

prev_direction = 0

for _ in range(steps_per_game):

current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)

angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)

predictions = []

predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \

is_right_blocked,apple_direction_vector_normalized[0], \

snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \

snake_direction_vector_normalized[1]]).reshape(-1,7))))-1

if predicted_direction == prev_direction:

count_same_direction+=1

else:

count_same_direction = 0

prev_direction = predicted_direction

new_direction = np.array(snake_position[0]) - np.array(snake_position[1])

if predicted_direction == -1:

new_direction = np.array([new_direction[1],-new_direction[0]])

if predicted_direction == 1:

new_direction = np.array([-new_direction[1],new_direction[0]])

button_direction = generate_button_direction(new_direction)

next_step = snake_position[0]+ current_direction_vector

if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1:

break

snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)

if score > max_score:

max_score = score

avg_score += score

return max_score, avg_score/1000

Now let see how neural network plays snake game.

Summary

I have used 1000 training games and 2000 steps per game. From this, I have generated 1633235 training examples. Then I have tested it on 1000 games and 2000 steps per game. Got the highest score of 61 and got an average score of 23.091. This score can vary since we are using random positions for food and also no of steps are fixed. You can also vary your clock speed as per your need.
You can try with the different number of games but then you have to change your network architecture in order to prevent the model from biasing and overfitting.

Now you might have got some feeling about how neural network plays a snake game. In the next blog, we will use a neural network trained with a genetic algorithm to play snake game. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

TheAILearner

Mastering Artificial Intelligence

Tag Archives: pygame

Snake Game using Real Time Speech Recognition

Snake Game with Genetic Algorithm

Creating a snake game and deciding neural network architecture

Creating Initial Population

Deciding the Fitness Function

Evaluating the population (Step – 4)

Selection, Crossover, and Mutation (Step – 5)

New Population Created

Creating a Snake Game with pygame

Neural Network Architecture Selection Using Genetic Algorithm

Creating Initial Population

Fitness Function

Selection

Crossover and Mutation

Snake Game with Deep Learning

Generating Training Data

Like this:

Snake Game with Deep Learning Part-2

Test Snake Game

Summary

Like this: