This is the second part of the snake game with deep learning series. In my previous blog, we have seen that how to generate training data for the neural network. In this tutorial, we will see training and testing of the neural network from generated training data.
The full code can be found here.
Our neural network is comprised of 7 nodes in the input layer, 3 node in the final layer and some hidden layers.
Network Architecture:
Now, it’s time to choose hidden layers and corresponding hyperparameters. Its always been difficult to find the perfect neural network architecture. There are some algorithms that can help to find the best network architecture for a neural network like a genetic algorithm, NAS, autoML etc. I have explained neural architecture search using the genetic algorithm in this blog.
In this blog, I have used hit and trial method to find network architecture. After some hit and trials, I have found a workable architecture, which consists of 2 hidden layers one of 9 units and other of 15 units. For the hidden layer, I have used the non-linear function ‘relu’ and for the output layer, I have used ‘softmax’.
You can use different libraries to train this model like keras, tflearn, etc. Here I have used keras. Our network architecture is as follows:
|
1 2 3 4 5 |
model = Sequential() model.add(Dense(units=9,input_dim=7)) model.add(Dense(units=15, activation='relu')) model.add(Dense(output_dim=3, activation = 'softmax')) |
Train Neural Network
Our model is prepared, now it’s time to train this. For training, we first need to compile this model then call a method model.fit() which will do the rest. Since our training data is a list, we first need to change it into numpy array and then reshape it. The reason for this is, a sequential model from keras expects numpy array or sparse matrix of shape [n_samples,n_features].
|
1 2 |
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy']) model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3) |
Now, our model is trained with generated training data. Next thing is to test it and see how much is learned.
Test Snake Game
Now it’s time to test our trained snake. To predict the direction we have fed our model with input values. Then used the predicted direction(Left, Straight or Front) to take the next step in our test games. For the new position, again predict the direction and move the snake. This continues until the snake dies or steps are over.
At last, we have calculated the maximum and average score for all the games in our test set.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
def run_game_with_ML(model,display,clock): max_score = 3 avg_score = 0 test_games = 1000 steps_per_game = 2000 for _ in range(test_games): snake_start, snake_position, apple_position, score = starting_positions() count_same_direction = 0 prev_direction = 0 for _ in range(steps_per_game): current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position) angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position) predictions = [] predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \ is_right_blocked,apple_direction_vector_normalized[0], \ snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \ snake_direction_vector_normalized[1]]).reshape(-1,7))))-1 if predicted_direction == prev_direction: count_same_direction+=1 else: count_same_direction = 0 prev_direction = predicted_direction new_direction = np.array(snake_position[0]) - np.array(snake_position[1]) if predicted_direction == -1: new_direction = np.array([new_direction[1],-new_direction[0]]) if predicted_direction == 1: new_direction = np.array([-new_direction[1],new_direction[0]]) button_direction = generate_button_direction(new_direction) next_step = snake_position[0]+ current_direction_vector if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1: break snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock) if score > max_score: max_score = score avg_score += score return max_score, avg_score/1000 |
Now let see how neural network plays snake game.
Summary
I have used 1000 training games and 2000 steps per game. From this, I have generated 1633235 training examples. Then I have tested it on 1000 games and 2000 steps per game. Got the highest score of 61 and got an average score of 23.091. This score can vary since we are using random positions for food and also no of steps are fixed. You can also vary your clock speed as per your need.
You can try with the different number of games but then you have to change your network architecture in order to prevent the model from biasing and overfitting.
Now you might have got some feeling about how neural network plays a snake game. In the next blog, we will use a neural network trained with a genetic algorithm to play snake game. Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.