Tag Archives: Machine Learning

Neural Network Architecture Selection Using Genetic Algorithm

In the previous blog, I have discussed the genetic algorithm and one of its application in the neural network (Training a neural network with a genetic algorithm ). In this blog, I have used a genetic algorithm to solve the problem of neural network architecture search.

You can find full code here .

Genetic Algorithm is really helpful if you do not want to waste your time in using brute force trial and error method for selecting hyperparameters. To know more about the genetic algorithm you can read this blog.

In this tutorial, to demonstrate the use of genetic algorithm I have used Snake Game with Deep Learning where it’s been difficult to find out which neural network architecture will give us the best results. So, the genetic algorithm can be used to find out the best network architecture among the number of hyperparameters.

Different values of hyperparameters are used to create an initial population. I have used the following parameters in the genetic algorithm to find the best value for them.

Number of hidden Layers.
Units per hidden layer
Activation function
Network optimizer

def hyperparameters():
    parameters = []
    units_per_layer = [20,25,30,35,40]
    layers = [1, 2, 3, 4, 5,6,7]
    activation = ['relu', 'tanh', 'sigmoid']
    optimizer = ['adam', 'rmsprop']
    parameters.append(units_per_layer)
    parameters.append(layers)
    parameters.append(activation)
    parameters.append(optimizer)
    
    return parameters

def hyperparameters():

parameters = []

units_per_layer = [20,25,30,35,40]

layers = [1, 2, 3, 4, 5,6,7]

activation = ['relu', 'tanh', 'sigmoid']

optimizer = ['adam', 'rmsprop']

parameters.append(units_per_layer)

parameters.append(layers)

parameters.append(activation)

parameters.append(optimizer)

return parameters

Creating Initial Population

Random parameters are used to create the Initial population. For creating the population first, you have to decide population size. Each individual in the population will have four values.

I have taken 20 chromosomes in the population.

def generate_population(size):
    parameters = hyperparameters()
    
    population = []
    i=0   
    while i < size:
        chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),
                    random.choice(parameters[2]), random.choice(parameters[3])]
        if chromosome not in population:
            population.append(chromosome)
            i+=1
    return population

def generate_population(size):

parameters = hyperparameters()

population = []

i=0

while i < size:

chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),

random.choice(parameters[2]), random.choice(parameters[3])]

if chromosome not in population:

population.append(chromosome)

i+=1

return population

Fitness Function

Fitness function can vary as per the need of different genetic algorithms. Here, I have used the average score for different network architectures. Individuals with the highest average score are fittest ones.

Selection

After evaluating each individual in the population, I have selected top 5 fittest individuals from the population. And also selected 3 individuals from the non-top performers. This will keep us away from getting stuck in the local maximum.

Remaining 12 individuals are created from these 8 individuals using Crossover.

Crossover and Mutation

To produce better offspring for the next generation, I have selected two parents randomly from the 8 individuals selected above and generated other 12 individuals.

def children_population(size, population):
    
    child_population = []
    k = 0
    population_size = len(population)
    while k < size :
        i, j = random.sample(range(0, population_size), 2)
        parent1 = population[i]
        parent2 = population[j]
         
        child = new_child(parent1, parent2) 
        if child not in child_population and child not in population:
            child_population.append(child)
            k +=1
            
    for newest_child in child_population:
        population.append(newest_child)
        
    return population

def new_child(parent1, parent2):
    child = []
    parent_size = len(parent1)
    i=0
    while i<parent_size:
        child.append(random.choice([parent1[i],parent2[i]]))
        i += 1
    return child

def children_population(size, population):

child_population = []

k = 0

population_size = len(population)

while k < size :

i, j = random.sample(range(0, population_size), 2)

parent1 = population[i]

parent2 = population[j]

child = new_child(parent1, parent2)

if child not in child_population and child not in population:

child_population.append(child)

k +=1

for newest_child in child_population:

population.append(newest_child)

return population

def new_child(parent1, parent2):

child = []

parent_size = len(parent1)

i=0

while i<parent_size:

child.append(random.choice([parent1[i],parent2[i]]))

i += 1

return child

In certain new children formed, some of their genes can be subjected to a mutation with a low random probability. Mutation is required to maintain some amount of randomness in the genetic algorithm.

def mutation(population):
    parameters = hyperparameters()
    for chromosome in population:
        if random.random() < 0.1 :
            key = random.choice([0,1,2,3])
            parameters = hyperparameters()
            mutate_key = random.choice(parameters[key])
            chromosome[key] = mutate_key
    
    return population

def mutation(population):

parameters = hyperparameters()

for chromosome in population:

if random.random() < 0.1 :

key = random.choice([0,1,2,3])

parameters = hyperparameters()

mutate_key = random.choice(parameters[key])

chromosome[key] = mutate_key

return population

Now we have created all the necessary functions required for a genetic algorithm. Now, we define a model function using keras library. Then we will train this model with different hyperparameters and search for the best using genetic algorithm.

def train_model(parameters):
    units_per_layer = parameters[0]
    layers = parameters[1]
    activation = parameters[2]
    optimizer = parameters[3]

    model = Sequential()
    model.add(Dense(units=9,input_dim=9))

    for _ in range(layers):
        model.add(Dense(units=units_per_layer, activation=activation))
        
    model.add(Dense(units = 3,  activation = 'softmax'))
    return model

population_with_avg_score = []

for chromosome in population:
    model = train_model(chromosome)
    model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])
    model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)
    max_score, avg_score = run_game_with_ML(model,display,clock)
    chromosome.append(avg_score)
    chromosome.append(max_score)
    population_with_avg_score.append(chromosome)

def train_model(parameters):

units_per_layer = parameters[0]

layers = parameters[1]

activation = parameters[2]

optimizer = parameters[3]

model = Sequential()

model.add(Dense(units=9,input_dim=9))

for _ in range(layers):

model.add(Dense(units=units_per_layer, activation=activation))

model.add(Dense(units = 3, activation = 'softmax'))

return model

population_with_avg_score = []

for chromosome in population:

model = train_model(chromosome)

model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])

model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

max_score, avg_score = run_game_with_ML(model,display,clock)

chromosome.append(avg_score)

chromosome.append(max_score)

population_with_avg_score.append(chromosome)

Here, I have used 10 generations and 20 individuals in the population. It can vary according to your need.

Now, you might have got some feeling about how the genetic algorithm can be applied to find neural architecture instead of using the brute-force method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Deep Learning

4 Replies

Developing a neural network to play a snake game usually consists of three steps.

Training data generation
Training neural network
Testing

The full code can be found here

In this tutorial, I will guide you to generate training data. To do this, first, we need to develop a snake game for which you can follow this blog.

Training data consists of inputs and corresponding outputs. Here, I have used the following inputs and outputs.

Input is comprised of 7 nodes:

Is left blocked or is there any obstacle in left ( 1 or 0)
is front blocked or is there any obstacle in front (1 or 0)
Is right blocked or is there any obstacle in right(1 or 0)
Apple direction vector from snake (X)
Apple direction vector from snake (Y)
Snake’s current direction vector (X)
Snake’s current direction vector (Y)

our input data will look like this:

The output is comprised of 3 node:

[1,0,0] will move snake left
[0,1,0] will continue snake in same direction
[0,0,1] will move snake right

Now the big question, how to generate this data? You can sit and play as many games as you can, but it is always good when you can generate data automatically. Let’s see how to do this.

Generating Training Data

Here I have generated training data automatically. To do this I have used angle between snake and apple. On the basis of that angle, I have decided in which direction snake should move. First, let’s calculate these.

Calculating angle b/w snake and apple:

To calculate the angle between snake and apple we only require two parameters, snake position and apple position.

In the following code, I have first calculated the snake’s current direction vector and Apple’s direction from the snake’s current position. Snake direction vector can be calculated by simply subtracting 0th index of the snake’s list from the 1st index. And to calculate apple direction from the snake, just subtract 0th index of snake’s list from Apple’s position.

Then normalize these direction vectors and calculate the angle with the help of the math library. The code is as follows:

def angle_with_apple(snake_position, apple_position):
    apple_direction_vector = np.array(apple_position)-np.array(snake_position[0])
    snake_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])
    
    norm_of_apple_direction_vector = np.linalg.norm(apple_direction_vector)
    norm_of_snake_direction_vector = np.linalg.norm(snake_direction_vector)
    if norm_of_apple_direction_vector == 0:
        norm_of_apple_direction_vector = 10
    if norm_of_snake_direction_vector == 0:
        norm_of_snake_direction_vector = 10
        
    apple_direction_vector_normalized = apple_direction_vector/norm_of_apple_direction_vector
    snake_direction_vector_normalized = snake_direction_vector/norm_of_snake_direction_vector
    angle = math.atan2(apple_direction_vector_normalized[1] * snake_direction_vector_normalized[0] - apple_direction_vector_normalized[0] * snake_direction_vector_normalized[1], apple_direction_vector_normalized[1] * snake_direction_vector_normalized[1] + apple_direction_vector_normalized[0] * snake_direction_vector_normalized[0]) / math.pi
    return angle

def angle_with_apple(snake_position, apple_position):

apple_direction_vector = np.array(apple_position)-np.array(snake_position[0])

snake_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])

norm_of_apple_direction_vector = np.linalg.norm(apple_direction_vector)

norm_of_snake_direction_vector = np.linalg.norm(snake_direction_vector)

if norm_of_apple_direction_vector == 0:

norm_of_apple_direction_vector = 10

if norm_of_snake_direction_vector == 0:

norm_of_snake_direction_vector = 10

apple_direction_vector_normalized = apple_direction_vector/norm_of_apple_direction_vector

snake_direction_vector_normalized = snake_direction_vector/norm_of_snake_direction_vector

angle = math.atan2(apple_direction_vector_normalized[1] * snake_direction_vector_normalized[0] - apple_direction_vector_normalized[0] * snake_direction_vector_normalized[1], apple_direction_vector_normalized[1] * snake_direction_vector_normalized[1] + apple_direction_vector_normalized[0] * snake_direction_vector_normalized[0]) / math.pi

return angle

After calculating the angle, next thing is to decide in which direction snake should move.

Calculating direction according to the angle:

If above-calculated angle > 0, this means Apple is on the right side of the snake. So snake should move to the right. For < 0, move left and =0 means continue in same direction. I have used 1, – 1 and 0 for the right, left and front respectively.

I have used the following steps to get the correct button direction (up, down, right, left or 3, 2, 1, 0 respectively) for the next step of the snake.

First, I have calculated the snake’s current direction.
Then to turn the snake to the left or right direction, I have calculated left direction vector or right direction vector from snake’s current direction vector.
Then I have converted the above-calculated direction vector into the button direction.

def generate_next_direction(snake_position, angle_with_apple):
    direction = 0
    
    if angle_with_apple > 0:
        direction = 1
    elif angle_with_apple < 0:
        direction = -1
    else:
        direction = 0
        
    current_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])
    left_direction_vector = np.array([current_direction_vector[1],-current_direction_vector[0]])
    right_direction_vector = np.array([-current_direction_vector[1], current_direction_vector[0]])
    
    new_direction = current_direction_vector
    if direction == -1:
        new_direction = left_direction_vector
    if direction == 1:
        new_direction = right_direction_vector

    button_direction = generate_button_direction(new_direction)
    
    return direction, button_direction

def generate_button_direction(new_direction):
    button_direction = 0
    if new_direction.tolist() == [10,0]:
        button_direction = 1
    elif new_direction.tolist() == [-10,0]:
        button_direction = 0
    elif new_direction.tolist() == [0,10]:
        button_direction = 2
    else:
        button_direction = 3
        
    return button_direction

def generate_next_direction(snake_position, angle_with_apple):

direction = 0

if angle_with_apple > 0:

direction = 1

elif angle_with_apple < 0:

direction = -1

else:

direction = 0

current_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])

left_direction_vector = np.array([current_direction_vector[1],-current_direction_vector[0]])

right_direction_vector = np.array([-current_direction_vector[1], current_direction_vector[0]])

new_direction = current_direction_vector

if direction == -1:

new_direction = left_direction_vector

if direction == 1:

new_direction = right_direction_vector

button_direction = generate_button_direction(new_direction)

return direction, button_direction

def generate_button_direction(new_direction):

button_direction = 0

if new_direction.tolist() == [10,0]:

button_direction = 1

elif new_direction.tolist() == [-10,0]:

button_direction = 0

elif new_direction.tolist() == [0,10]:

button_direction = 2

else:

button_direction = 3

return button_direction

Now, for every step, angle and corresponding next direction are calculated and snake moves according to that. And for each step inputs and outputs are calculated which are appended to a list of training data.To generate training data, we need to keep a record of 7 inputs and 3 outputs for every step the snake takes. First, let’s see how I have calculated the inputs for every step the snake takes.

To check if the direction is blocked, we look one step ahead in each direction.
Snake direction vector = Snake’s Head (0th index) – Snake’s 1st index
Apple direction from the snake = Apple’s position – Snake’s head position (See the figure below)

For every step, the output is generated by first calculating the direction for the given snake and apple position, using angle between them. Now, we need to convert our directions( -1, 0 or 1 ) to output(Y), a one hot vector. For every predicted direction we need to see that if that direction is blocked or not and according to that create output (Y) for training data. The code given below seems to be a bit longer but it calculates our training data output (Y).

def generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked):
    if direction == -1:
        if is_left_blocked == 1:
            if is_front_blocked == 1 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)
                training_data_y.append([0,0,1])
            elif is_front_blocked == 0 and is_right_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)
                training_data_y.append([0,1,0])
            elif is_front_blocked == 0 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1) 
                training_data_y.append([0,0,1])
             
        else:
            training_data_y.append([1,0,0])

    elif direction == 0:
        if is_front_blocked == 1:
            if is_left_blocked == 1 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)
                training_data_y.append([0,0,1])
            elif is_left_blocked == 0 and is_right_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])
            elif is_left_blocked == 0 and is_right_blocked == 0:
                training_data_y.append([0,0,1])
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)           
        else:
            training_data_y.append([0,1,0])
    else:
        if is_right_blocked == 1:
            if is_left_blocked == 1 and is_front_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)
                training_data_y.append([0,1,0])
            elif is_left_blocked == 0 and is_front_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])
            elif is_left_blocked == 0 and is_front_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])               
        else:
            training_data_y.append([0,0,1])
    
    return direction, button_direction, training_data_y

def generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked):

if direction == -1:

if is_left_blocked == 1:

if is_front_blocked == 1 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

elif is_front_blocked == 0 and is_right_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)

training_data_y.append([0,1,0])

elif is_front_blocked == 0 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

else:

training_data_y.append([1,0,0])

elif direction == 0:

if is_front_blocked == 1:

if is_left_blocked == 1 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

elif is_left_blocked == 0 and is_right_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

elif is_left_blocked == 0 and is_right_blocked == 0:

training_data_y.append([0,0,1])

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

else:

training_data_y.append([0,1,0])

else:

if is_right_blocked == 1:

if is_left_blocked == 1 and is_front_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)

training_data_y.append([0,1,0])

elif is_left_blocked == 0 and is_front_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

elif is_left_blocked == 0 and is_front_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

else:

training_data_y.append([0,0,1])

return direction, button_direction, training_data_y

Here, I have used 1000 games for generating training data, each of which consists of 2000 steps. For every game, I have re-initialized snake position, apple position, and score. Then, created two empty lists, one for input training data(X) and another output training data(Y), those will contain our whole training data.The code is as follows:

def generate_training_data(display,clock):
    
    training_data_x = []
    training_data_y = []
    training_games = 1000
    steps_per_game = 2000
    
    for _ in tqdm(range(training_games)):
        snake_start, snake_position, apple_position, score = starting_positions()
        prev_apple_distance = apple_distance_from_snake(apple_position, snake_position)
        
        for _ in range(steps_per_game):
            angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)
            direction, button_direction = generate_random_direction(snake_position, angle)
            current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)
            
            direction, button_direction, training_data_y = generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked)
            
            if is_front_blocked == 1 and is_left_blocked == 1 and is_right_blocked == 1:
                break

            training_data_x.append([is_left_blocked, is_front_blocked, is_right_blocked,apple_direction_vector_normalized[0], \
                        snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \
                        snake_direction_vector_normalized[1]])
            
            
            
            
            snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)   

    return training_data_x, training_data_y

def generate_training_data(display,clock):

training_data_x = []

training_data_y = []

training_games = 1000

steps_per_game = 2000

for _ in tqdm(range(training_games)):

snake_start, snake_position, apple_position, score = starting_positions()

prev_apple_distance = apple_distance_from_snake(apple_position, snake_position)

for _ in range(steps_per_game):

angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)

direction, button_direction = generate_random_direction(snake_position, angle)

current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)

direction, button_direction, training_data_y = generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked)

if is_front_blocked == 1 and is_left_blocked == 1 and is_right_blocked == 1:

break

training_data_x.append([is_left_blocked, is_front_blocked, is_right_blocked,apple_direction_vector_normalized[0], \

snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \

snake_direction_vector_normalized[1]])

snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)

return training_data_x, training_data_y

You might have got some feeling about the training data generation for the snake game with deep learning. In the next blog, we will use this data to train and test our neural network. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Deep Learning Part-2

2 Replies

This is the second part of the snake game with deep learning series. In my previous blog, we have seen that how to generate training data for the neural network. In this tutorial, we will see training and testing of the neural network from generated training data.

The full code can be found here.

Our neural network is comprised of 7 nodes in the input layer, 3 node in the final layer and some hidden layers.

Network Architecture:

Now, it’s time to choose hidden layers and corresponding hyperparameters. Its always been difficult to find the perfect neural network architecture. There are some algorithms that can help to find the best network architecture for a neural network like a genetic algorithm, NAS, autoML etc. I have explained neural architecture search using the genetic algorithm in this blog.

In this blog, I have used hit and trial method to find network architecture. After some hit and trials, I have found a workable architecture, which consists of 2 hidden layers one of 9 units and other of 15 units. For the hidden layer, I have used the non-linear function ‘relu’ and for the output layer, I have used ‘softmax’.

You can use different libraries to train this model like keras, tflearn, etc. Here I have used keras. Our network architecture is as follows:

model = Sequential()
model.add(Dense(units=9,input_dim=7))

model.add(Dense(units=15, activation='relu'))
model.add(Dense(output_dim=3,  activation = 'softmax'))

model = Sequential()

model.add(Dense(units=9,input_dim=7))

model.add(Dense(units=15, activation='relu'))

model.add(Dense(output_dim=3, activation = 'softmax'))

Train Neural Network

Our model is prepared, now it’s time to train this. For training, we first need to compile this model then call a method model.fit() which will do the rest. Since our training data is a list, we first need to change it into numpy array and then reshape it. The reason for this is, a sequential model from keras expects numpy array or sparse matrix of shape [n_samples,n_features].

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

1 2	model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy']) model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

Now, our model is trained with generated training data. Next thing is to test it and see how much is learned.

Test Snake Game

Now it’s time to test our trained snake. To predict the direction we have fed our model with input values. Then used the predicted direction(Left, Straight or Front) to take the next step in our test games. For the new position, again predict the direction and move the snake. This continues until the snake dies or steps are over.

At last, we have calculated the maximum and average score for all the games in our test set.

def run_game_with_ML(model,display,clock):
    max_score = 3
    avg_score = 0
    test_games = 1000
    steps_per_game = 2000
    
    for _ in range(test_games):
        snake_start, snake_position, apple_position, score = starting_positions()
        
        count_same_direction = 0
        prev_direction = 0

        for _ in range(steps_per_game):
            current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)
            angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)
            predictions = []

            predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \
                                    is_right_blocked,apple_direction_vector_normalized[0], \
                                    snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \
                                            snake_direction_vector_normalized[1]]).reshape(-1,7))))-1
            
            
            if predicted_direction == prev_direction:
                count_same_direction+=1
            else:
                count_same_direction = 0
                prev_direction = predicted_direction
            
            new_direction = np.array(snake_position[0]) - np.array(snake_position[1])
            if predicted_direction == -1:
                new_direction = np.array([new_direction[1],-new_direction[0]])
            if predicted_direction == 1:
                new_direction = np.array([-new_direction[1],new_direction[0]])
            
            button_direction = generate_button_direction(new_direction)
            
            next_step = snake_position[0]+ current_direction_vector
            if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1:
                break
            snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)  
            
                                         
            if score > max_score:
                max_score = score
                
        avg_score += score
       
    return max_score, avg_score/1000

def run_game_with_ML(model,display,clock):

max_score = 3

avg_score = 0

test_games = 1000

steps_per_game = 2000

for _ in range(test_games):

snake_start, snake_position, apple_position, score = starting_positions()

count_same_direction = 0

prev_direction = 0

for _ in range(steps_per_game):

current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)

angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)

predictions = []

predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \

is_right_blocked,apple_direction_vector_normalized[0], \

snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \

snake_direction_vector_normalized[1]]).reshape(-1,7))))-1

if predicted_direction == prev_direction:

count_same_direction+=1

else:

count_same_direction = 0

prev_direction = predicted_direction

new_direction = np.array(snake_position[0]) - np.array(snake_position[1])

if predicted_direction == -1:

new_direction = np.array([new_direction[1],-new_direction[0]])

if predicted_direction == 1:

new_direction = np.array([-new_direction[1],new_direction[0]])

button_direction = generate_button_direction(new_direction)

next_step = snake_position[0]+ current_direction_vector

if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1:

break

snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)

if score > max_score:

max_score = score

avg_score += score

return max_score, avg_score/1000

Now let see how neural network plays snake game.

Summary

I have used 1000 training games and 2000 steps per game. From this, I have generated 1633235 training examples. Then I have tested it on 1000 games and 2000 steps per game. Got the highest score of 61 and got an average score of 23.091. This score can vary since we are using random positions for food and also no of steps are fixed. You can also vary your clock speed as per your need.
You can try with the different number of games but then you have to change your network architecture in order to prevent the model from biasing and overfitting.

Now you might have got some feeling about how neural network plays a snake game. In the next blog, we will use a neural network trained with a genetic algorithm to play snake game. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

TheAILearner

Mastering Artificial Intelligence

Tag Archives: Machine Learning

Neural Network Architecture Selection Using Genetic Algorithm

Creating Initial Population

Fitness Function

Selection

Crossover and Mutation

Snake Game with Deep Learning

Generating Training Data

Like this:

Snake Game with Deep Learning Part-2

Test Snake Game

Summary

Like this: