Tag Archives: neural network

Is the deconvolution layer the same as a convolutional layer?

Isn’t this an interesting topic? If you have worked with image classification problems( e.g. classifying cats and dogs) or image generation problems( e.g. GANs, autoencoders), surely you have encountered with convolution and deconvolution layer. But what if someone says a deconvolution layer is same as a convolution layer.

This paper has proposed an efficient subpixel convolution layer which works same as a deconvolution layer. To understand this, lets first understand convolution layer , transposed convolution layer and sub pixel convolution layer.

Convolution Layer

In every convolution neural network, convolution layer is the most important part. A convolution layer is consist of numbers of independent filters which convolve independently with input and produce output for the next layer. Let’s see how a filter convolve with the input.

Transposed and sub pixel Convolution Layer

Transposed convolution is the inverse operation of convolution. In convolution layer, you try to extract useful features from input while in transposed convolution, you try to add some useful features to upscale an image. Transposed convolution has learnable features which are learnt using backpropogation. Lets see how to do a transposed convolution visually.

Similarly, a subpixel convolution is also used for upsampling an image. It uses fractional strides( input is padded with in-between zero pixels) to an input and outputs an upsampled image. Let’s see visually.

An efficient sub pixel convolution Layer

In this paper authors have proposed that upsampling using deconvolution layer isn’t really necessary. So they came up with this Idea. Instead of putting in between zero pixels in the input image, they do more convolution in lower resolution image and then apply periodic shuffling to produce an upscaled image.

Authors have illustrated that deconvolution layer with kernel size of (o, i, k*r , k*r ) is same as convolution layer with kernel size of (o*r *r, i, k, k) e.g. (output channels, input channels, kernel width, kernel height) in LR space. Let’s take an example of proposed efficient subpixel convolution layer.

In the above figure, input image shape is (1, 4, 4) and upscaling ratio(r) is 2. To achieve an image of size (1, 8, 8), first input image is applied with kernel size of (4, 1, 2, 2) which produces output of shape (4, 4, 4) and then periodic shufling is applied to get required upscaled image of shape (1, 8, 8). So instead of using deconvolution layer with kernel size of (1, 1, 4, 4) same can be done with this efficient sub pixel convolution layer.

Implementation

I have also implemented an autoencoder(using MNIST dataset) with efficient subpixel convolution layer. Let’s see the code for efficient subpixel convolution.

def _phase_shift(I, r):  # defines periodical shuffling to upscale image
    input_shape = tf.shape(I)
    bsize = input_shape[0]
    a = input_shape[1]
    b = input_shape[2]
    c = tf.cast(input_shape[3]/(r ** 2),tf.int32)
    shape = tf.stack([bsize,a,b,r,r,c])

    input_shape_as_numbers = I.get_shape()

    X = tf.reshape(tensor=I, shape=shape)
    X = tf.transpose(X, (0, 1, 2, 4, 3,5))  
    X = tf.split(X, input_shape_as_numbers[1], axis=1)  
    X = tf.concat([tf.squeeze(x) for x in X], axis=2)  
    X = tf.split(X,input_shape_as_numbers[2], axis=1) 
    X = tf.concat([tf.squeeze(x) for x in X], axis=2) 

    return tf.reshape(X, (bsize, a*r, b*r, c))

def PS(input_shape, r, name, color=False):

    def subpixel_shape(input_shape):
        dims = [input_shape[0],
            input_shape[1] * r,
            input_shape[2] * r,
            int(input_shape[3] / (r ** 2))]
        output_shape = tuple(dims)
        return output_shape
  
    def subpixel(x):
        if color:
            Xc = tf.split(3, 3, X)
            X = tf.concat(3, [_phase_shift(x, r) for x in Xc])
        else:
            x_upsampled = _phase_shift(x, r)
        return x_upsampled

    return Lambda(subpixel, output_shape=subpixel_shape, name=name)


# encoder layers
encoder_inputs = Input(shape = (28,28,1))
conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(encoder_inputs)
pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1)
conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1)
pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2)
flat = Flatten()(pool2)

#decoder layer
dense_layer_d = Dense(7*7*32, activation = 'relu')(enocder_outputs)
output_from_d = Reshape((7,7,32))(dense_layer_d)

# convolution then periodical shuffling to upscale image
sub_up = Conv2D(128, (2,2), activation = 'relu', padding = "SAME")(output_from_d)
upSampled_1 = PS(sub_up.shape, 2, name = 'subpixel1', color=False)(sub_up)

output_from_upSampled_1 = Reshape((14,14,32))(upSampled_1)

# convolution then periodical shuffling to upscale image
sub_up_2 = Conv2D(4, (2,2), activation = 'relu', padding = "SAME")(output_from_upSampled_1)
output = PS(sub_up_2.shape, 2, name = 'subpixel2', color=False)(sub_up_2)

autoencoder = Model(encoder_inputs, output)

# training of model
m = 256
n_epoch = 25
autoencoder.compile(optimizer='adam', loss='mse')
autoencoder.fit(output_X_train,output_X_train, epochs=n_epoch, batch_size=m, shuffle=True)

def _phase_shift(I, r): # defines periodical shuffling to upscale image

input_shape = tf.shape(I)

bsize = input_shape[0]

a = input_shape[1]

b = input_shape[2]

c = tf.cast(input_shape[3]/(r ** 2),tf.int32)

shape = tf.stack([bsize,a,b,r,r,c])

input_shape_as_numbers = I.get_shape()

X = tf.reshape(tensor=I, shape=shape)

X = tf.transpose(X, (0, 1, 2, 4, 3,5))

X = tf.split(X, input_shape_as_numbers[1], axis=1)

X = tf.concat([tf.squeeze(x) for x in X], axis=2)

X = tf.split(X,input_shape_as_numbers[2], axis=1)

X = tf.concat([tf.squeeze(x) for x in X], axis=2)

return tf.reshape(X, (bsize, a*r, b*r, c))

def PS(input_shape, r, name, color=False):

def subpixel_shape(input_shape):

dims = [input_shape[0],

input_shape[1] * r,

input_shape[2] * r,

int(input_shape[3] / (r ** 2))]

output_shape = tuple(dims)

return output_shape

def subpixel(x):

if color:

Xc = tf.split(3, 3, X)

X = tf.concat(3, [_phase_shift(x, r) for x in Xc])

else:

x_upsampled = _phase_shift(x, r)

return x_upsampled

return Lambda(subpixel, output_shape=subpixel_shape, name=name)

# encoder layers

encoder_inputs = Input(shape = (28,28,1))

conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(encoder_inputs)

pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1)

conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1)

pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2)

flat = Flatten()(pool2)

#decoder layer

dense_layer_d = Dense(7*7*32, activation = 'relu')(enocder_outputs)

output_from_d = Reshape((7,7,32))(dense_layer_d)

# convolution then periodical shuffling to upscale image

sub_up = Conv2D(128, (2,2), activation = 'relu', padding = "SAME")(output_from_d)

upSampled_1 = PS(sub_up.shape, 2, name = 'subpixel1', color=False)(sub_up)

output_from_upSampled_1 = Reshape((14,14,32))(upSampled_1)

# convolution then periodical shuffling to upscale image

sub_up_2 = Conv2D(4, (2,2), activation = 'relu', padding = "SAME")(output_from_upSampled_1)

output = PS(sub_up_2.shape, 2, name = 'subpixel2', color=False)(sub_up_2)

autoencoder = Model(encoder_inputs, output)

# training of model

m = 256

n_epoch = 25

autoencoder.compile(optimizer='adam', loss='mse')

autoencoder.fit(output_X_train,output_X_train, epochs=n_epoch, batch_size=m, shuffle=True)

The above periodic shuffling code is given by this github link. Then applied autoencoder layers to generate image. To up-sample image in decoder layers first convolved encoded images then used periodical shuffling.

This type of subpixel convolution layers can be very helpful in problems like image generation( autoencoders, GANs), image enhancement(super resolution). Also there is more to find out what can this efficient subpixel convolution layer offers.

Now, you might have got some feeling about efficient subpixel convolution layer. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Referenced Research Paper : Is the deconvolution layer the same as a convolutional layer?

Referenced Gitub Link : Subpixel

On Calibration of Modern Neural Networks

Leave a reply

Nowadays neural networks are having vast applicability and these are trusted to make complex decisions in applications such as, medical diagnosis, speech recognition, object recognition and optical character recognition. Due to more and more research in deep learning, neural networks accuracy has been improved dramatically.

With the improvement in accuracy, neural network should also be confident in saying when they are likely to be incorrect. As an example, if confidence given by a neural network for disease diagnosis is low, control should be passed to human doctors.

Now what is confidence score in neural network? It is the probability estimate produced by the neural network. Let say, you are working on a multi-class classification task. After applying softmax layer you found out that a particular class is having highest probability with value of 0.7 . It means that you are 70% confident that this should be your actual output.

Here we intuitively mean that, for 100 predictions if average confidence score is 0.8, 80 should be correctly classified. But modern neural networks are poorly calibrated. As you can see in figure there is larger gap between average confidence score and accuracy for ResNet while less for LeNet.

In the paper, author has addresses the followings:

What methods are alleviating poor calibration problem in neural networks.
A simple and straightforward solution to reduce this problem.

Observing Miscalibration:

With the advancement in deep neural networks some recent changes are responsible for miscalibration.

Model Capacity: Although increasing depth and width of neural networks may reduce classification error, but in paper they have observed that these increases negatively affect model calibration.
Batch Normalization: Batch Normalization improves training time, reduces the need for additional regularization, and can in some cases improve the accuracy of networks. It has been observed that models trained with Batch Normalization tend to be more miscalibrated.
Weight Decay: It has been found that that training with less weight decay has a negative impact on calibration.

Temperature Scaling:

Temperature scaling works well to calibrate computer vision models. It is a simplest extension of Platt scaling. To understand temprature scaling we will first see Platt scaling.

Platt Scaling: This method is used for calibrating models. It uses logistic regression to return the calibrated probabilities of a model. Let say you are working on a multi-class classification task and trained it on some training data. Now Platt scaling will take logits(output from trained network before applying softmax layer using validation dataset) as input to logistic regression model. Then Platt scaling will be trained on validation dataset and learns scalar parameters a, b ∈ R and outputs q = σ(az + b) as the calibrated probability(where z are logits.).

Temperature scaling is an extension of Platt scaling having a trainable single parameter T>0 for all classes. T is called the temperature. T is trained with validation dataset not on training dataset. Because if we train T during training, network would learn to make the temperature as low as possible so that it can be very confident on training dataset.

Then temperature will be applied directly to softmax layer by dividing logits with T ( z/T ) and then trained on validation dataset. After adjusting temperature parameter on validation dataset, it will give trained parameter T, which we can use to divide logits and then apply softmax layer to find calibrated probabilities during test data. Now, lets see a simple TensorFlow code to implement temperature scaling.

X = tf.placeholder(np.float32, shape=[None, 10])  # logits
Y = tf.placeholder(np.float32, shape=[None, 10])  # actual output
temp = tf.get_variable("temp", shape=[1], initializer=tf.initializers.constant(1.0)) # T temperature parameter
logits_w_temp = tf.divide(X, temp)  # logits after dividing with T

loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=Y, logits=logits_w_temp))  # loss function with softmax
optimizer = tf.train.AdamOptimizer(learning_rate=0.01).minimize(loss)
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init) 
# input_data are the logits output of validation data using trained network
# labels are actual output of validation dataset
    for epoch in range(100):
        _, c, t = sess.run([optimizer, loss, temp], feed_dict={X: input_data, Y: labels})

        print("Epoch:", (epoch + 1), "cost =", "{:.15f}".format(c))
        print("Epoch:", (epoch + 1), "temprature =", t)

    X_test = input_data

    predictions = sess.run(logits_w_temp, {X: X_test})

sess.close()

X = tf.placeholder(np.float32, shape=[None, 10]) # logits

Y = tf.placeholder(np.float32, shape=[None, 10]) # actual output

temp = tf.get_variable("temp", shape=[1], initializer=tf.initializers.constant(1.0)) # T temperature parameter

logits_w_temp = tf.divide(X, temp) # logits after dividing with T

loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=Y, logits=logits_w_temp)) # loss function with softmax

optimizer = tf.train.AdamOptimizer(learning_rate=0.01).minimize(loss)

init = tf.global_variables_initializer()

with tf.Session() as sess:

sess.run(init)

# input_data are the logits output of validation data using trained network

# labels are actual output of validation dataset

for epoch in range(100):

_, c, t = sess.run([optimizer, loss, temp], feed_dict={X: input_data, Y: labels})

print("Epoch:", (epoch + 1), "cost =", "{:.15f}".format(c))

print("Epoch:", (epoch + 1), "temprature =", t)

X_test = input_data

predictions = sess.run(logits_w_temp, {X: X_test})

sess.close()

Simple techniques can effectively remedy the miscalibration phenomenon in neural networks. Temperature scaling is the simplest, fastest, and most straightforward of the methods,and surprisingly is often the most effective.

Referenced Research Paper : On Calibration of Modern Neural Networks

GitHub: Temperature Scaling

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Denoising Autoencoders

Leave a reply

In my previous blog, we have discussed what is an autoencoder, its applications and a simple implementation in keras. In this blog, we will see a variant of autoencoder – ‘ denoising autoencoders ‘.

A denoising autoencoder is an extension of autoencoders. An autoencoder tries to learn identity function( output equals to input ), which makes it risking to not learn useful feature. One method to overcome this problem is to use denoising autoencoders.

For training a denoising autoencoder, we need to use noisy input data. For that, we need to add some noise to an original image. The amount of corrupting data depends on the amount of information present in data. Usually, 25-30 % data is being corrupted. This can be higher if your data contains less information. Let see how you can add noise to data in code:

# adding some noise to data

input_x_train = output_X_train + 0.5 * np.random.normal(loc=0.0, scale=1.0, size=output_X_train.shape) 
input_x_test = output_X_test + 0.5 * np.random.normal(loc=0.0, scale=1.0, size=output_X_test.shape)

# adding some noise to data

input_x_train = output_X_train + 0.5 * np.random.normal(loc=0.0, scale=1.0, size=output_X_train.shape)

input_x_test = output_X_test + 0.5 * np.random.normal(loc=0.0, scale=1.0, size=output_X_test.shape)

To calculate loss, the output of the denoising autoencoder is then compared to original input instead of the corrupted one. Such a loss function train model to learn interesting features rather than learning identity function.

I have implemented denoising autoencoder in keras using MNIST data, which will give you an overview, how a denoising autoencoder works.

# creating denoising autoencoder model
inputs = Input(shape = (28,28,1))

conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(inputs)
pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1)
conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1)
pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2)

upsampling_1 = Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(pool2)
upsampling_2 = Conv2DTranspose(16, 3, padding='same', activation='relu', strides=(2, 2))(upsampling_1)
outputs = Conv2DTranspose(1, 3, padding='same', activation='relu')(upsampling_2)

autoencoder = Model(inputs, outputs)
m = 256
n_epoch = 10
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
autoencoder.fit(input_x_train,output_X_train, epochs=n_epoch, batch_size=m, shuffle=True)

# creating denoising autoencoder model

inputs = Input(shape = (28,28,1))

conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(inputs)

pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1)

conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1)

pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2)

upsampling_1 = Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(pool2)

upsampling_2 = Conv2DTranspose(16, 3, padding='same', activation='relu', strides=(2, 2))(upsampling_1)

outputs = Conv2DTranspose(1, 3, padding='same', activation='relu')(upsampling_2)

autoencoder = Model(inputs, outputs)

m = 256

n_epoch = 10

autoencoder.compile(optimizer='adam', loss='binary_crossentropy')

autoencoder.fit(input_x_train,output_X_train, epochs=n_epoch, batch_size=m, shuffle=True)

following is the result of denoising autoencoder.

The full code can be find here.

Hope you understand the usefulness of denoising autoencoder. In the next blog, we will feature variational autoencoders. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Autoencoders

Leave a reply

Let’s start with a simple definition of autoencoders. ‘ Autoencoders are the neural networks trained to reconstruct their original input’.

Now, you might be thinking what’s the use of reconstructing same data. Let me give you an example If you want to transfer data of GB’s of size and somehow if you can compress it into MB’s and then able to reconstruct back the data to the original size, isn’t that a better way to transfer data. This is one of the applications of autoencoders.

Autoencoders generally consists of two parts, one is encoder and other is decoder. Encoder downscale data to less number of features and decoder upscale the extracted features to original one.

There are some practical applications of autoencoders:

Dimensionality reduction for data visualization
Image Denoising
Generative Models

Visualizing a 10-dimensional vector is difficult. To overcome this problem we need to reduce that 10-dimensional vector into 2-D or 3-D. One of the famous algorithm PCA (Principal Component Analysis) tries to solve this problem. PCA uses linear transformations while autoencoders can use both linear and non-linear transformations for dimensionality reduction. Which makes autoencoders to generate more complex and interesting features than PCA.

Autoencoders can be used to remove the noise present in the image. It can also be used to generate new images required for a specific task. We will see more about these two applications in the next blog.

Now, let’s start with the simple implementation of autoencoders in Keras using MNIST data. First, let’s download MNIST training and test data and reshape it.

(X_train, Y_train), (X_test, Y_test) = mnist.load_data()
X_train = X_train.astype('float32') / 255.
output_X_train = X_train.reshape(-1,28,28,1)

X_test = X_test.astype('float32') / 255.
output_X_test = X_test.reshape(-1,28,28,1)

print(X_train.shape, X_test.shape)

(X_train, Y_train), (X_test, Y_test) = mnist.load_data()

X_train = X_train.astype('float32') / 255.

output_X_train = X_train.reshape(-1,28,28,1)

X_test = X_test.astype('float32') / 255.

output_X_test = X_test.reshape(-1,28,28,1)

print(X_train.shape, X_test.shape)

Encoder

MNIST data consists of images of digits. So, it is better to use a convolutional neural network in our encoders and decoders. In our encoder, I have used conv and max-pooling layers to extract the compressed representation. Then flatten the encoder output to 32 features. Which will be the input to the decoder.

encoder_inputs = Input(shape = (28,28,1))

conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(encoder_inputs)
pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1)
conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1)
pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2)
flat = Flatten()(pool2)

enocder_outputs = Dense(32, activation = 'relu')(flat)

encoder_inputs = Input(shape = (28,28,1))

conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(encoder_inputs)

pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1)

conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1)

pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2)

flat = Flatten()(pool2)

enocder_outputs = Dense(32, activation = 'relu')(flat)

Decoder

In the decoder, we need to upsample the extracted 32 features into the original size of the image. To achieve this, I have used Conv2DTranspose functions from keras. Then the final layer of the decoder will give the reconstructed output which will be similar to the original input.

dense_layer_d = Dense(7*7*32, activation = 'relu')(enocder_outputs)
output_from_d = Reshape((7,7,32))(dense_layer_d)
conv1_1 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(output_from_d)
upsampling_1 = Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(conv1_1)
upsampling_2 = Conv2DTranspose(16, 3, padding='same', activation='relu', strides=(2, 2))(upsampling_1)
decoded_outputs = Conv2DTranspose(1, 3, padding='same', activation='relu')(upsampling_2)

autoencoder = Model(encoder_inputs, decoded_outputs)

dense_layer_d = Dense(7*7*32, activation = 'relu')(enocder_outputs)

output_from_d = Reshape((7,7,32))(dense_layer_d)

conv1_1 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(output_from_d)

upsampling_1 = Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(conv1_1)

upsampling_2 = Conv2DTranspose(16, 3, padding='same', activation='relu', strides=(2, 2))(upsampling_1)

decoded_outputs = Conv2DTranspose(1, 3, padding='same', activation='relu')(upsampling_2)

autoencoder = Model(encoder_inputs, decoded_outputs)

To minimize reconstruction loss, we train the network with a large dataset and update weights. Now, our model is created, the next thing is to compile and train the model.

m = 256
n_epoch = 10
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
autoencoder.fit(output_X_train,output_X_train, epochs=n_epoch, batch_size=m, shuffle=True)

m = 256

n_epoch = 10

autoencoder.compile(optimizer='adam', loss='binary_crossentropy')

autoencoder.fit(output_X_train,output_X_train, epochs=n_epoch, batch_size=m, shuffle=True)

Below are the results from autoencoder trained above. The first line of digits shows the original input (test images) while the second line represents the reconstructed inputs from the model.

The full code can be find here.

Hope you understand the basics of autoencoders, where these can be used and how a simple autoencoder be implemented. In the next blog, we will see how to denoise an image using autoencoders. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Referenced Research Paper: http://proceedings.mlr.press/v27/baldi12a/baldi12a.pdf

Genetic Algorithm and its usage in neural network

Leave a reply

You might have heard about the theory of evolution by natural selection. If not then read this quote by Charles Darwin ” It is not the strongest of the species that survives, nor the most intelligent; it is the one most adaptable to change. ” Genetic Algorithm is also based on this theory.

In the 1970’s, Jon Holland tries to mimic some processes observed in natural evolution by introducing genetic algorithm. This algorithm can be used in optimization and search problems both.

A typical genetic algorithm requires some population in the solution domain and a fitness function to find the fittest individual. To evolve individuals in the population genetic algorithm uses some operations like crossover, mutation, and selection.

Genetic algorithm starts with some random initial population. Then tries to produce offspring from the best individuals in the population. The concept is that, if the fittest individuals are selected then chances of producing a better offspring is more. This process keeps on iteration until your target is not achieved. Each iteration is known as the generation.

Initial Population

Initial Population refers to a set of possible solutions. Each member (individual) of the population is usually known as the chromosome (phenotypes) and represents a solution for the problem to be investigated. The chromosome is represented as a set of parameters (features or genes or weights) that defines the individual. Size of the population depends totally on your problem. Random selection of initial population makes sure that it covers a wide range of possible solution.

Evaluation and Fitness Function

Now, we have a random initial population, next thing is to evaluate the fitness of these individuals. To evaluate the fitness of these individuals, you need to define some fitness function. You need to choose the fitness function according to your problem. Fitness function measures the quality of each individual.

Selection

Some best individuals are selected from the evaluated population. These selected individuals are mated to produce some new offspring.

Crossover

Each individual selected in the previous step has some quality. Our objective is to produce better offspring so that our algorithm can evolve and find a better solution to the problem. To do that two individuals from the best population are selected and a new child (offspring) is produced with features of both as shown above. This is known as Crossover.

Mutation

Mutation is applied to maintain the diversity within the population and inhibit premature convergence. With some low probability, a portion of the new individual is subjected to mutation as shown in the figure above.

Replacement

New population replaces a previous one for the next generation. This process keeps on iterating until a certain target is not achieved.

Application of genetic algorithm in neural networks:

Training of a neural network ( instead of using gradient descent, Adam etc.)
Selection of neural network architecture ( Hyperparameters selection)

Now, you might have got some feeling about the genetic algorithm. In the next blog, we will see how this concept can be applied to train the neural network to play a snake game. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Neural Network Architecture Selection Using Genetic Algorithm

9 Replies

In the previous blog, I have discussed the genetic algorithm and one of its application in the neural network (Training a neural network with a genetic algorithm ). In this blog, I have used a genetic algorithm to solve the problem of neural network architecture search.

You can find full code here .

Genetic Algorithm is really helpful if you do not want to waste your time in using brute force trial and error method for selecting hyperparameters. To know more about the genetic algorithm you can read this blog.

In this tutorial, to demonstrate the use of genetic algorithm I have used Snake Game with Deep Learning where it’s been difficult to find out which neural network architecture will give us the best results. So, the genetic algorithm can be used to find out the best network architecture among the number of hyperparameters.

Different values of hyperparameters are used to create an initial population. I have used the following parameters in the genetic algorithm to find the best value for them.

Number of hidden Layers.
Units per hidden layer
Activation function
Network optimizer

def hyperparameters():
    parameters = []
    units_per_layer = [20,25,30,35,40]
    layers = [1, 2, 3, 4, 5,6,7]
    activation = ['relu', 'tanh', 'sigmoid']
    optimizer = ['adam', 'rmsprop']
    parameters.append(units_per_layer)
    parameters.append(layers)
    parameters.append(activation)
    parameters.append(optimizer)
    
    return parameters

def hyperparameters():

parameters = []

units_per_layer = [20,25,30,35,40]

layers = [1, 2, 3, 4, 5,6,7]

activation = ['relu', 'tanh', 'sigmoid']

optimizer = ['adam', 'rmsprop']

parameters.append(units_per_layer)

parameters.append(layers)

parameters.append(activation)

parameters.append(optimizer)

return parameters

Creating Initial Population

Random parameters are used to create the Initial population. For creating the population first, you have to decide population size. Each individual in the population will have four values.

I have taken 20 chromosomes in the population.

def generate_population(size):
    parameters = hyperparameters()
    
    population = []
    i=0   
    while i < size:
        chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),
                    random.choice(parameters[2]), random.choice(parameters[3])]
        if chromosome not in population:
            population.append(chromosome)
            i+=1
    return population

def generate_population(size):

parameters = hyperparameters()

population = []

i=0

while i < size:

chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),

random.choice(parameters[2]), random.choice(parameters[3])]

if chromosome not in population:

population.append(chromosome)

i+=1

return population

Fitness Function

Fitness function can vary as per the need of different genetic algorithms. Here, I have used the average score for different network architectures. Individuals with the highest average score are fittest ones.

Selection

After evaluating each individual in the population, I have selected top 5 fittest individuals from the population. And also selected 3 individuals from the non-top performers. This will keep us away from getting stuck in the local maximum.

Remaining 12 individuals are created from these 8 individuals using Crossover.

Crossover and Mutation

To produce better offspring for the next generation, I have selected two parents randomly from the 8 individuals selected above and generated other 12 individuals.

def children_population(size, population):
    
    child_population = []
    k = 0
    population_size = len(population)
    while k < size :
        i, j = random.sample(range(0, population_size), 2)
        parent1 = population[i]
        parent2 = population[j]
         
        child = new_child(parent1, parent2) 
        if child not in child_population and child not in population:
            child_population.append(child)
            k +=1
            
    for newest_child in child_population:
        population.append(newest_child)
        
    return population

def new_child(parent1, parent2):
    child = []
    parent_size = len(parent1)
    i=0
    while i<parent_size:
        child.append(random.choice([parent1[i],parent2[i]]))
        i += 1
    return child

def children_population(size, population):

child_population = []

k = 0

population_size = len(population)

while k < size :

i, j = random.sample(range(0, population_size), 2)

parent1 = population[i]

parent2 = population[j]

child = new_child(parent1, parent2)

if child not in child_population and child not in population:

child_population.append(child)

k +=1

for newest_child in child_population:

population.append(newest_child)

return population

def new_child(parent1, parent2):

child = []

parent_size = len(parent1)

i=0

while i<parent_size:

child.append(random.choice([parent1[i],parent2[i]]))

i += 1

return child

In certain new children formed, some of their genes can be subjected to a mutation with a low random probability. Mutation is required to maintain some amount of randomness in the genetic algorithm.

def mutation(population):
    parameters = hyperparameters()
    for chromosome in population:
        if random.random() < 0.1 :
            key = random.choice([0,1,2,3])
            parameters = hyperparameters()
            mutate_key = random.choice(parameters[key])
            chromosome[key] = mutate_key
    
    return population

def mutation(population):

parameters = hyperparameters()

for chromosome in population:

if random.random() < 0.1 :

key = random.choice([0,1,2,3])

parameters = hyperparameters()

mutate_key = random.choice(parameters[key])

chromosome[key] = mutate_key

return population

Now we have created all the necessary functions required for a genetic algorithm. Now, we define a model function using keras library. Then we will train this model with different hyperparameters and search for the best using genetic algorithm.

def train_model(parameters):
    units_per_layer = parameters[0]
    layers = parameters[1]
    activation = parameters[2]
    optimizer = parameters[3]

    model = Sequential()
    model.add(Dense(units=9,input_dim=9))

    for _ in range(layers):
        model.add(Dense(units=units_per_layer, activation=activation))
        
    model.add(Dense(units = 3,  activation = 'softmax'))
    return model

population_with_avg_score = []

for chromosome in population:
    model = train_model(chromosome)
    model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])
    model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)
    max_score, avg_score = run_game_with_ML(model,display,clock)
    chromosome.append(avg_score)
    chromosome.append(max_score)
    population_with_avg_score.append(chromosome)

def train_model(parameters):

units_per_layer = parameters[0]

layers = parameters[1]

activation = parameters[2]

optimizer = parameters[3]

model = Sequential()

model.add(Dense(units=9,input_dim=9))

for _ in range(layers):

model.add(Dense(units=units_per_layer, activation=activation))

model.add(Dense(units = 3, activation = 'softmax'))

return model

population_with_avg_score = []

for chromosome in population:

model = train_model(chromosome)

model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])

model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

max_score, avg_score = run_game_with_ML(model,display,clock)

chromosome.append(avg_score)

chromosome.append(max_score)

population_with_avg_score.append(chromosome)

Here, I have used 10 generations and 20 individuals in the population. It can vary according to your need.

Now, you might have got some feeling about how the genetic algorithm can be applied to find neural architecture instead of using the brute-force method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Deep Learning

4 Replies

Developing a neural network to play a snake game usually consists of three steps.

Training data generation
Training neural network
Testing

The full code can be found here

In this tutorial, I will guide you to generate training data. To do this, first, we need to develop a snake game for which you can follow this blog.

Training data consists of inputs and corresponding outputs. Here, I have used the following inputs and outputs.

Input is comprised of 7 nodes:

Is left blocked or is there any obstacle in left ( 1 or 0)
is front blocked or is there any obstacle in front (1 or 0)
Is right blocked or is there any obstacle in right(1 or 0)
Apple direction vector from snake (X)
Apple direction vector from snake (Y)
Snake’s current direction vector (X)
Snake’s current direction vector (Y)

our input data will look like this:

The output is comprised of 3 node:

[1,0,0] will move snake left
[0,1,0] will continue snake in same direction
[0,0,1] will move snake right

Now the big question, how to generate this data? You can sit and play as many games as you can, but it is always good when you can generate data automatically. Let’s see how to do this.

Generating Training Data

Here I have generated training data automatically. To do this I have used angle between snake and apple. On the basis of that angle, I have decided in which direction snake should move. First, let’s calculate these.

Calculating angle b/w snake and apple:

To calculate the angle between snake and apple we only require two parameters, snake position and apple position.

In the following code, I have first calculated the snake’s current direction vector and Apple’s direction from the snake’s current position. Snake direction vector can be calculated by simply subtracting 0th index of the snake’s list from the 1st index. And to calculate apple direction from the snake, just subtract 0th index of snake’s list from Apple’s position.

Then normalize these direction vectors and calculate the angle with the help of the math library. The code is as follows:

def angle_with_apple(snake_position, apple_position):
    apple_direction_vector = np.array(apple_position)-np.array(snake_position[0])
    snake_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])
    
    norm_of_apple_direction_vector = np.linalg.norm(apple_direction_vector)
    norm_of_snake_direction_vector = np.linalg.norm(snake_direction_vector)
    if norm_of_apple_direction_vector == 0:
        norm_of_apple_direction_vector = 10
    if norm_of_snake_direction_vector == 0:
        norm_of_snake_direction_vector = 10
        
    apple_direction_vector_normalized = apple_direction_vector/norm_of_apple_direction_vector
    snake_direction_vector_normalized = snake_direction_vector/norm_of_snake_direction_vector
    angle = math.atan2(apple_direction_vector_normalized[1] * snake_direction_vector_normalized[0] - apple_direction_vector_normalized[0] * snake_direction_vector_normalized[1], apple_direction_vector_normalized[1] * snake_direction_vector_normalized[1] + apple_direction_vector_normalized[0] * snake_direction_vector_normalized[0]) / math.pi
    return angle

def angle_with_apple(snake_position, apple_position):

apple_direction_vector = np.array(apple_position)-np.array(snake_position[0])

snake_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])

norm_of_apple_direction_vector = np.linalg.norm(apple_direction_vector)

norm_of_snake_direction_vector = np.linalg.norm(snake_direction_vector)

if norm_of_apple_direction_vector == 0:

norm_of_apple_direction_vector = 10

if norm_of_snake_direction_vector == 0:

norm_of_snake_direction_vector = 10

apple_direction_vector_normalized = apple_direction_vector/norm_of_apple_direction_vector

snake_direction_vector_normalized = snake_direction_vector/norm_of_snake_direction_vector

angle = math.atan2(apple_direction_vector_normalized[1] * snake_direction_vector_normalized[0] - apple_direction_vector_normalized[0] * snake_direction_vector_normalized[1], apple_direction_vector_normalized[1] * snake_direction_vector_normalized[1] + apple_direction_vector_normalized[0] * snake_direction_vector_normalized[0]) / math.pi

return angle

After calculating the angle, next thing is to decide in which direction snake should move.

Calculating direction according to the angle:

If above-calculated angle > 0, this means Apple is on the right side of the snake. So snake should move to the right. For < 0, move left and =0 means continue in same direction. I have used 1, – 1 and 0 for the right, left and front respectively.

I have used the following steps to get the correct button direction (up, down, right, left or 3, 2, 1, 0 respectively) for the next step of the snake.

First, I have calculated the snake’s current direction.
Then to turn the snake to the left or right direction, I have calculated left direction vector or right direction vector from snake’s current direction vector.
Then I have converted the above-calculated direction vector into the button direction.

def generate_next_direction(snake_position, angle_with_apple):
    direction = 0
    
    if angle_with_apple > 0:
        direction = 1
    elif angle_with_apple < 0:
        direction = -1
    else:
        direction = 0
        
    current_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])
    left_direction_vector = np.array([current_direction_vector[1],-current_direction_vector[0]])
    right_direction_vector = np.array([-current_direction_vector[1], current_direction_vector[0]])
    
    new_direction = current_direction_vector
    if direction == -1:
        new_direction = left_direction_vector
    if direction == 1:
        new_direction = right_direction_vector

    button_direction = generate_button_direction(new_direction)
    
    return direction, button_direction

def generate_button_direction(new_direction):
    button_direction = 0
    if new_direction.tolist() == [10,0]:
        button_direction = 1
    elif new_direction.tolist() == [-10,0]:
        button_direction = 0
    elif new_direction.tolist() == [0,10]:
        button_direction = 2
    else:
        button_direction = 3
        
    return button_direction

def generate_next_direction(snake_position, angle_with_apple):

direction = 0

if angle_with_apple > 0:

direction = 1

elif angle_with_apple < 0:

direction = -1

else:

direction = 0

current_direction_vector = np.array(snake_position[0])-np.array(snake_position[1])

left_direction_vector = np.array([current_direction_vector[1],-current_direction_vector[0]])

right_direction_vector = np.array([-current_direction_vector[1], current_direction_vector[0]])

new_direction = current_direction_vector

if direction == -1:

new_direction = left_direction_vector

if direction == 1:

new_direction = right_direction_vector

button_direction = generate_button_direction(new_direction)

return direction, button_direction

def generate_button_direction(new_direction):

button_direction = 0

if new_direction.tolist() == [10,0]:

button_direction = 1

elif new_direction.tolist() == [-10,0]:

button_direction = 0

elif new_direction.tolist() == [0,10]:

button_direction = 2

else:

button_direction = 3

return button_direction

Now, for every step, angle and corresponding next direction are calculated and snake moves according to that. And for each step inputs and outputs are calculated which are appended to a list of training data.To generate training data, we need to keep a record of 7 inputs and 3 outputs for every step the snake takes. First, let’s see how I have calculated the inputs for every step the snake takes.

To check if the direction is blocked, we look one step ahead in each direction.
Snake direction vector = Snake’s Head (0th index) – Snake’s 1st index
Apple direction from the snake = Apple’s position – Snake’s head position (See the figure below)

For every step, the output is generated by first calculating the direction for the given snake and apple position, using angle between them. Now, we need to convert our directions( -1, 0 or 1 ) to output(Y), a one hot vector. For every predicted direction we need to see that if that direction is blocked or not and according to that create output (Y) for training data. The code given below seems to be a bit longer but it calculates our training data output (Y).

def generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked):
    if direction == -1:
        if is_left_blocked == 1:
            if is_front_blocked == 1 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)
                training_data_y.append([0,0,1])
            elif is_front_blocked == 0 and is_right_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)
                training_data_y.append([0,1,0])
            elif is_front_blocked == 0 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1) 
                training_data_y.append([0,0,1])
             
        else:
            training_data_y.append([1,0,0])

    elif direction == 0:
        if is_front_blocked == 1:
            if is_left_blocked == 1 and is_right_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)
                training_data_y.append([0,0,1])
            elif is_left_blocked == 0 and is_right_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])
            elif is_left_blocked == 0 and is_right_blocked == 0:
                training_data_y.append([0,0,1])
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)           
        else:
            training_data_y.append([0,1,0])
    else:
        if is_right_blocked == 1:
            if is_left_blocked == 1 and is_front_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)
                training_data_y.append([0,1,0])
            elif is_left_blocked == 0 and is_front_blocked == 1:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])
            elif is_left_blocked == 0 and is_front_blocked == 0:
                direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)
                training_data_y.append([1,0,0])               
        else:
            training_data_y.append([0,0,1])
    
    return direction, button_direction, training_data_y

def generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked):

if direction == -1:

if is_left_blocked == 1:

if is_front_blocked == 1 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

elif is_front_blocked == 0 and is_right_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)

training_data_y.append([0,1,0])

elif is_front_blocked == 0 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

else:

training_data_y.append([1,0,0])

elif direction == 0:

if is_front_blocked == 1:

if is_left_blocked == 1 and is_right_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

training_data_y.append([0,0,1])

elif is_left_blocked == 0 and is_right_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

elif is_left_blocked == 0 and is_right_blocked == 0:

training_data_y.append([0,0,1])

direction, button_direction = direction_vector(snake_position, angle_with_apple, 1)

else:

training_data_y.append([0,1,0])

else:

if is_right_blocked == 1:

if is_left_blocked == 1 and is_front_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, 0)

training_data_y.append([0,1,0])

elif is_left_blocked == 0 and is_front_blocked == 1:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

elif is_left_blocked == 0 and is_front_blocked == 0:

direction, button_direction = direction_vector(snake_position, angle_with_apple, -1)

training_data_y.append([1,0,0])

else:

training_data_y.append([0,0,1])

return direction, button_direction, training_data_y

Here, I have used 1000 games for generating training data, each of which consists of 2000 steps. For every game, I have re-initialized snake position, apple position, and score. Then, created two empty lists, one for input training data(X) and another output training data(Y), those will contain our whole training data.The code is as follows:

def generate_training_data(display,clock):
    
    training_data_x = []
    training_data_y = []
    training_games = 1000
    steps_per_game = 2000
    
    for _ in tqdm(range(training_games)):
        snake_start, snake_position, apple_position, score = starting_positions()
        prev_apple_distance = apple_distance_from_snake(apple_position, snake_position)
        
        for _ in range(steps_per_game):
            angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)
            direction, button_direction = generate_random_direction(snake_position, angle)
            current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)
            
            direction, button_direction, training_data_y = generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked)
            
            if is_front_blocked == 1 and is_left_blocked == 1 and is_right_blocked == 1:
                break

            training_data_x.append([is_left_blocked, is_front_blocked, is_right_blocked,apple_direction_vector_normalized[0], \
                        snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \
                        snake_direction_vector_normalized[1]])
            
            
            
            
            snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)   

    return training_data_x, training_data_y

def generate_training_data(display,clock):

training_data_x = []

training_data_y = []

training_games = 1000

steps_per_game = 2000

for _ in tqdm(range(training_games)):

snake_start, snake_position, apple_position, score = starting_positions()

prev_apple_distance = apple_distance_from_snake(apple_position, snake_position)

for _ in range(steps_per_game):

angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)

direction, button_direction = generate_random_direction(snake_position, angle)

current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)

direction, button_direction, training_data_y = generate_training_data_y(snake_position, angle_with_apple, button_direction, direction, training_data_y, is_front_blocked, is_left_blocked ,is_right_blocked)

if is_front_blocked == 1 and is_left_blocked == 1 and is_right_blocked == 1:

break

training_data_x.append([is_left_blocked, is_front_blocked, is_right_blocked,apple_direction_vector_normalized[0], \

snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \

snake_direction_vector_normalized[1]])

snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)

return training_data_x, training_data_y

You might have got some feeling about the training data generation for the snake game with deep learning. In the next blog, we will use this data to train and test our neural network. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Snake Game with Deep Learning Part-2

2 Replies

This is the second part of the snake game with deep learning series. In my previous blog, we have seen that how to generate training data for the neural network. In this tutorial, we will see training and testing of the neural network from generated training data.

The full code can be found here.

Our neural network is comprised of 7 nodes in the input layer, 3 node in the final layer and some hidden layers.

Network Architecture:

Now, it’s time to choose hidden layers and corresponding hyperparameters. Its always been difficult to find the perfect neural network architecture. There are some algorithms that can help to find the best network architecture for a neural network like a genetic algorithm, NAS, autoML etc. I have explained neural architecture search using the genetic algorithm in this blog.

In this blog, I have used hit and trial method to find network architecture. After some hit and trials, I have found a workable architecture, which consists of 2 hidden layers one of 9 units and other of 15 units. For the hidden layer, I have used the non-linear function ‘relu’ and for the output layer, I have used ‘softmax’.

You can use different libraries to train this model like keras, tflearn, etc. Here I have used keras. Our network architecture is as follows:

model = Sequential()
model.add(Dense(units=9,input_dim=7))

model.add(Dense(units=15, activation='relu'))
model.add(Dense(output_dim=3,  activation = 'softmax'))

model = Sequential()

model.add(Dense(units=9,input_dim=7))

model.add(Dense(units=15, activation='relu'))

model.add(Dense(output_dim=3, activation = 'softmax'))

Train Neural Network

Our model is prepared, now it’s time to train this. For training, we first need to compile this model then call a method model.fit() which will do the rest. Since our training data is a list, we first need to change it into numpy array and then reshape it. The reason for this is, a sequential model from keras expects numpy array or sparse matrix of shape [n_samples,n_features].

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

1 2	model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy']) model.fit((np.array(training_data_x).reshape(-1,7)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

Now, our model is trained with generated training data. Next thing is to test it and see how much is learned.

Test Snake Game

Now it’s time to test our trained snake. To predict the direction we have fed our model with input values. Then used the predicted direction(Left, Straight or Front) to take the next step in our test games. For the new position, again predict the direction and move the snake. This continues until the snake dies or steps are over.

At last, we have calculated the maximum and average score for all the games in our test set.

def run_game_with_ML(model,display,clock):
    max_score = 3
    avg_score = 0
    test_games = 1000
    steps_per_game = 2000
    
    for _ in range(test_games):
        snake_start, snake_position, apple_position, score = starting_positions()
        
        count_same_direction = 0
        prev_direction = 0

        for _ in range(steps_per_game):
            current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)
            angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)
            predictions = []

            predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \
                                    is_right_blocked,apple_direction_vector_normalized[0], \
                                    snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \
                                            snake_direction_vector_normalized[1]]).reshape(-1,7))))-1
            
            
            if predicted_direction == prev_direction:
                count_same_direction+=1
            else:
                count_same_direction = 0
                prev_direction = predicted_direction
            
            new_direction = np.array(snake_position[0]) - np.array(snake_position[1])
            if predicted_direction == -1:
                new_direction = np.array([new_direction[1],-new_direction[0]])
            if predicted_direction == 1:
                new_direction = np.array([-new_direction[1],new_direction[0]])
            
            button_direction = generate_button_direction(new_direction)
            
            next_step = snake_position[0]+ current_direction_vector
            if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1:
                break
            snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)  
            
                                         
            if score > max_score:
                max_score = score
                
        avg_score += score
       
    return max_score, avg_score/1000

def run_game_with_ML(model,display,clock):

max_score = 3

avg_score = 0

test_games = 1000

steps_per_game = 2000

for _ in range(test_games):

snake_start, snake_position, apple_position, score = starting_positions()

count_same_direction = 0

prev_direction = 0

for _ in range(steps_per_game):

current_direction_vector, is_front_blocked, is_left_blocked ,is_right_blocked = blocked_directions(snake_position)

angle, snake_direction_vector, apple_direction_vector_normalized, snake_direction_vector_normalized = angle_with_apple(snake_position, apple_position)

predictions = []

predicted_direction = np.argmax(np.array(model.predict(np.array([is_left_blocked, is_front_blocked, \

is_right_blocked,apple_direction_vector_normalized[0], \

snake_direction_vector_normalized[0], apple_direction_vector_normalized[1], \

snake_direction_vector_normalized[1]]).reshape(-1,7))))-1

if predicted_direction == prev_direction:

count_same_direction+=1

else:

count_same_direction = 0

prev_direction = predicted_direction

new_direction = np.array(snake_position[0]) - np.array(snake_position[1])

if predicted_direction == -1:

new_direction = np.array([new_direction[1],-new_direction[0]])

if predicted_direction == 1:

new_direction = np.array([-new_direction[1],new_direction[0]])

button_direction = generate_button_direction(new_direction)

next_step = snake_position[0]+ current_direction_vector

if collision_with_boundaries(snake_position[0]) == 1 or collision_with_self(next_step.tolist(), snake_position) == 1:

break

snake_position, apple_position, score = play_game(snake_start, snake_position, apple_position, button_direction, score,display,clock)

if score > max_score:

max_score = score

avg_score += score

return max_score, avg_score/1000

Now let see how neural network plays snake game.

Summary

I have used 1000 training games and 2000 steps per game. From this, I have generated 1633235 training examples. Then I have tested it on 1000 games and 2000 steps per game. Got the highest score of 61 and got an average score of 23.091. This score can vary since we are using random positions for food and also no of steps are fixed. You can also vary your clock speed as per your need.
You can try with the different number of games but then you have to change your network architecture in order to prevent the model from biasing and overfitting.

Now you might have got some feeling about how neural network plays a snake game. In the next blog, we will use a neural network trained with a genetic algorithm to play snake game. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

TheAILearner

Mastering Artificial Intelligence

Tag Archives: neural network

Is the deconvolution layer the same as a convolutional layer?

On Calibration of Modern Neural Networks

Observing Miscalibration:

Temperature Scaling:

Denoising Autoencoders

Autoencoders

Encoder

Decoder

Genetic Algorithm and its usage in neural network

Initial Population

Evaluation and Fitness Function

Selection

Crossover

Mutation

Replacement

Neural Network Architecture Selection Using Genetic Algorithm

Creating Initial Population

Fitness Function

Selection

Crossover and Mutation

Snake Game with Deep Learning

Generating Training Data

Like this:

Snake Game with Deep Learning Part-2

Test Snake Game

Summary

Like this: