Category Archives: Keras

ImageDataGenerator – apply_transform method

In this blog, we will discuss ImageDataGenerator “apply_transform” method. Using this method, you can apply any desired transformations to an image. You can find its use in the ImageDataGenerator “flow” method. First of all, let’s discuss its Keras API.

Keras API

apply_transform(x, transform_parameters)

1	apply_transform(x, transform_parameters)

This applies transformations to x (3D tensor) according to the transform parameters specified.

The “transform_parameters” is a dictionary specifying the set of transformations to be applied. Only the following transformations are available

theta, tx, ty, shear, zx, zy, flip_horizontal, flip_vertical, channel_shift_intensity, brightness

1	theta, tx, ty, shear, zx, zy, flip_horizontal, flip_vertical, channel_shift_intensity, brightness

Let’s discuss these in detail.

theta: Rotation angle in degrees. Below is an example that rotates the image by 40 degrees.

tx and ty: These are the shifts in the vertical and the horizontal directions respectively. For instance, tx=20 will shift the image vertically by 20 pixels.

In this, first of all, the translation matrix is calculated. Then affine transformation is applied using the “scipy.ndimage” affine_transformation method.

zx and zy: This zooms the image in the vertical and horizontal directions respectively. If less than 1, the image is zoomed in otherwise zoomed out.

Note: -ve values of zx and zy results in flipping the image in vertical and horizontal directions respectively. For instance, zx=-1 will flip the image vertically.

flip_horizontal and flip_vertical: This flips the image horizontally and vertically. For instance, below is the code for flipping the image horizontally,

b = img.swapaxes(1,0)
c = b[::-1, ...]
flip_img = c.swapaxes(0,1)

b = img.swapaxes(1,0)

c = b[::-1, ...]

flip_img = c.swapaxes(0,1)

channel_shift_intensity: This shifts the channel values by the amount specified. The following code sums up how it works

[np.clip(x_channel + amount, min_img, max_img) for x_channel in img]

1	[np.clip(x_channel + amount, min_img, max_img) for x_channel in img]

brightness: This controls the brightness of the image. An enhancement factor of 0.0 gives a black image. A factor of 1.0 gives the original image.

Hope you understand all the arguments. Now, let’s see how to use this.

How to use this?

Because “apply_transform” is a method inside the ImageDataGenerator class. Thus, one first need to create the instance of this class and then apply this method as shown below

from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array
img = load_img('D:/downloads/opencv_logo.PNG')
img = img_to_array(img)
datagen = ImageDataGenerator()
rotate = datagen.apply_transform(x=img, transform_parameters={'theta':40, 'brightness':0.8, 'zx':0.9, 'zy':0.9})

from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array

img = load_img('D:/downloads/opencv_logo.PNG')

img = img_to_array(img)

datagen = ImageDataGenerator()

rotate = datagen.apply_transform(x=img, transform_parameters={'theta':40, 'brightness':0.8, 'zx':0.9, 'zy':0.9})

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – get_random_transform method

Leave a reply

In the previous blog, we discussed how to generate batches of augmented data using the flow method. We also learned that the key ingredient in the flow method is the “get_random_transform” method. This generates random parameters for a transformation. So, in this blog, let’s discuss this method in detail.

Keras API

get_random_transform(img_shape, seed=None)

1	get_random_transform(img_shape, seed=None)

Now, let’s see how this generates random parameters for transformations by just using the image shape information.

How this works?

This borrows the parameters from the ImageDataGenerator class. For instance, if we define rotation range in the ImageDataGenerator class

datagen = ImageDataGenerator(rotation_range=40)

1	datagen = ImageDataGenerator(rotation_range=40)

then the random parameters for this is obtained as

if self.rotation_range:
    theta = np.random.uniform(-self.rotation_range,self.rotation_range)
else:
    theta = 0

if self.rotation_range:

theta = np.random.uniform(-self.rotation_range,self.rotation_range)

else:

theta = 0

Thus, whenever you generate examples, theta is obtained from the uniform distribution, specified according to the parameters provided in the ImageDataGenerator class.

Similarly, for every transformation provided in the ImageDataGenerator class, we can obtain the random parameters. For more details, refer to the Keras GitHub.

How to use this?

To use this, you first need to provide the transformations in the ImageDataGenerator class. For instance, if I just want to rotate the image, then first specify the parameters as

datagen = ImageDataGenerator(rotation_range=40)

1	datagen = ImageDataGenerator(rotation_range=40)

Now, to get the random parameters, call the “get_random_transform” method as

transform = datagen1.get_random_transform((257, 278, 3),seed = 2)

1	transform = datagen1.get_random_transform((257, 278, 3),seed = 2)

This outputs the following parameters dictionary as

See only the transformations specified in the ImageDataGenerator class i.e. theta value is changed. Rest all values are the default.

So, this way one can generate random parameters for transformations. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – standardize method

Leave a reply

In this blog, we will discuss ImageDataGenerator “standardize” method. This method performs in-place normalization to the batch of inputs. As already discussed, this is an important step in the flow method or data augmentation. So, let’s discuss it in detail.

Keras API

standardize(x)

1	standardize(x)

Here, x is the batch of inputs. This method returns the normalized inputs. Note that x is changed in-place. If you don’t want to change the inputs in-place, pass a copy of the input to this method.

How this works?

While performing data augmentation with ImageDataGenerator, we discussed different normalization techniques. These techniques include centering the entire distribution or a sample, rescaling the input, performing zca whitening, etc. Behind the scenes, these are implemented by the “standardize” method. Let’s see how.

For instance, we want to rescale the input by 1/255. So, first of all we will create an ImageDataGenerator instance as shown below

datagen = ImageDataGenerator(rescale=1/255.)

1	datagen = ImageDataGenerator(rescale=1/255.)

Then for data augmentation, we will use the flow method as

training_generator = datagen.flow(x_train, y_train, batch_size=64,subset='training',seed=7)

1	training_generator = datagen.flow(x_train, y_train, batch_size=64,subset='training',seed=7)

Thus, the training_generator will yield batches of augmented images. That’s all we usually do.

As already discussed in this blog, the flow method consists of three steps, of which the last step is the “standardize” method. All the normalization work in the ImageDataGenerator class is handled by this method.

Now, coming back to the above example, the “standardize” method will first check whether you want to rescale or not. If yes, then this will change the input in-place as shown below.

if self.rescale:
    x *= self.rescale

1 2	if self.rescale: x *= self.rescale

Similarly, this method performs featurewise_center or samplewise_center or any other normalization. For more details, refer to Keras Github.

How to use this?

First of all, create an ImageDataGenerator instance with the desired transformations. Then apply the “standardize” method as shown below.

import matplotlib.pyplot as plt
import numpy as np
from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array

img = load_img('D:/downloads/opencv_logo.PNG')
img = img_to_array(img)
datagen2 = ImageDataGenerator(samplewise_center=True)
standardize = datagen2.standardize(np.copy(img))
plt.imshow(standardize/255.)

import matplotlib.pyplot as plt

import numpy as np

from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array

img = load_img('D:/downloads/opencv_logo.PNG')

img = img_to_array(img)

datagen2 = ImageDataGenerator(samplewise_center=True)

standardize = datagen2.standardize(np.copy(img))

plt.imshow(standardize/255.)

Note: The standardize method only supports transformations that perform normalization such as featurewise_center, rescale, etc. Otherwise, this returns the same image or batch of inputs.

What does in-place means?

As already discussed, this method normalizes the inputs in-place. This is exactly what in-place operators in Python do. Let’s take an example to understand what does in-place means.

For instance, let’s rescale an image of all ones by 2. After the “standardize” method, see how the mean of the “images” change.

from keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(rescale=2)

images = np.ones((10, 128, 128, 3))
print(images.mean()) # 1.0 

images_std = datagen.standardize(images)

print(images.mean(), images_std.mean()) # 2.0 2.0

from keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(rescale=2)

images = np.ones((10, 128, 128, 3))

print(images.mean()) # 1.0

images_std = datagen.standardize(images)

print(images.mean(), images_std.mean()) # 2.0 2.0

That’s all for “standardize” method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – random_transform method

Leave a reply

In the previous blog, we discussed how to generate random parameters for a transformation. In this blog, we will discuss how to apply a random transformation to an image.

Keras API

random_transform(x, seed=None)

1	random_transform(x, seed=None)

This function returns a randomly transformed version of the input image x.

How this method works?

First of all, this generates random parameters for a transformation using the “get_random_transform” method. For more details, refer to this blog.
Then the image is transformed according to the parameters (generated above) using the “apply_transform” method. For more details, refer to this blog.

Below is the code for this (taken from Keras)

def random_transform(self, x, seed=None):
    params = self.get_random_transform(x.shape, seed)
    return self.apply_transform(x, params)

def random_transform(self, x, seed=None):

params = self.get_random_transform(x.shape, seed)

return self.apply_transform(x, params)

How to use this?

To use this, you first need to provide the desired transformations in the ImageDataGenerator class. For instance, let’s say we just want to zoom the image. Firstly, we specify the parameters in the ImageDataGenerator class.

datagen = ImageDataGenerator(zoom_range=0.2)

1	datagen = ImageDataGenerator(zoom_range=0.2)

Then we apply the random_transform method.

random = datagen.random_transform(img)

1	random = datagen.random_transform(img)

Similarly, you can apply any random transformation to the image. Just specify the transformations in the ImageDataGenerator class. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – fit method

Leave a reply

In the previous blog, we discussed how to perform data augmentation using ImageDataGenerator. In that, we saw that some transformations require statistics of the entire dataset. These transformations include featurewise_center, featurewise_std_normalization and zca_whitening.

To calculate these statistics, first of all, one may need to load the entire dataset into the memory. Then calculate the mean, standard deviation, principal components or any other statistics from that data. Fortunately, Keras has a built-in fit method for doing this. Let’s discuss it in detail.

Keras API

fit(x, augment=False, rounds=1, seed=None)

1	fit(x, augment=False, rounds=1, seed=None)

Here, x is the data from which to calculate the statistics. Should have rank 4. Note that the channel axis of x should have value either 1, 3, or 4 depending upon whether the data is greyscale, RGB, or RGBA.

This also provides an option of whether to use the augmented data for calculating statistics or not. This is done using the “augment” argument. If True, then augmented examples are also used for calculating statistics. The number of augmented examples depends upon the “rounds” parameter. For instance, if “rounds=2” and x.shape[0] or data size is 64, then 128 augmented examples are used.

Below code shows its implementation (taken from Keras). First of all, create an array of zeros to handle the augmented examples. Then generate the augmented examples using the random_transform method and append to this array.

if augment:
    ax = np.zeros(
        tuple([rounds * x.shape[0]] + list(x.shape)[1:]),
        dtype=self.dtype)
    for r in range(rounds):
        for i in range(x.shape[0]):
            ax[i + r * x.shape[0]] = self.random_transform(x[i])
    x = ax

if augment:

ax = np.zeros(

tuple([rounds * x.shape[0]] + list(x.shape)[1:]),

dtype=self.dtype)

for r in range(rounds):

for i in range(x.shape[0]):

ax[i + r * x.shape[0]] = self.random_transform(x[i])

x = ax

Here, x is the training data or the data whose statistics we want to calculate. Once we have the data, we can easily calculate the statistics such as mean, standard deviation and principal components using Numpy and Scipy libraries.

Note: Statistics are calculated across all channels in an image. So, don’t calculate the mean separately for each channel.

Now, when we generate batches of augmented data using any method (like flow) these statistics are used to normalize the data as shown below

if self.featurewise_center:
    if self.mean is not None:
        x -= self.mean

if self.featurewise_center:

if self.mean is not None:

x -= self.mean

How to use this?

Let’s take the MNIST digit classification example. Suppose we want to center the distribution i.e. mean equal to 0. For this, we will use the ImageDataGenerator “featurewise_center” transformation. Firstly, load the data and preprocess it.

from keras.datasets import mnist
import numpy as np
from keras.preprocessing.image import ImageDataGenerator

(x_train, y_train),(x_test, y_test) = mnist.load_data()

train_x = x_train.astype('float32')
test_x = x_test.astype('float32')

train_x = np.expand_dims(train_x, axis=-1)
test_x = np.expand_dims(test_x, axis=-1)

from keras.datasets import mnist

import numpy as np

from keras.preprocessing.image import ImageDataGenerator

(x_train, y_train),(x_test, y_test) = mnist.load_data()

train_x = x_train.astype('float32')

test_x = x_test.astype('float32')

train_x = np.expand_dims(train_x, axis=-1)

test_x = np.expand_dims(test_x, axis=-1)

After loading the data, firstly, create an ImageDataGenerator instance. Then fit the training data as shown below

datagen = ImageDataGenerator(featurewise_center=True)
datagen.fit(train_x)

1 2	datagen = ImageDataGenerator(featurewise_center=True) datagen.fit(train_x)

Let’s calculate the mean of the training data manually and using “datagen” mean attribute.

print(train_x.mean())
print(datagen.mean)

1 2	print(train_x.mean()) print(datagen.mean)

As expected these should be the same i.e 33.318447. Now, let’s see what happens to the mean of the distribution after normalization.

datagenerator = datagen.flow(train_x,y_train,batch_size=len(train_x))
for x,y in datagenerator:
    print(x.mean())   # -2.752179e-05
    break

datagenerator = datagen.flow(train_x,y_train,batch_size=len(train_x))

for x,y in datagenerator:

print(x.mean()) # -2.752179e-05

break

Clearly, this centers the distribution. Similarly, we can perform other types of normalizations also.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras ImageDataGenerator Normalization at validation and test time

Leave a reply

Note: This blog should not be confused with Test time augmentation (TTA).

In the previous blogs, we discussed different operations that are available for image augmentation under the ImageDataGenerator class. For instance rotation, translation, zoom, shearing, normalization, etc. By this, our model will be exposed to more aspects of data and thus will generalize better.

But what about validation and prediction time? Since both of these are used to evaluate the model, we want them to be fixed. That is why we don’t apply any random transformation to the validation and test data. But the test and the dev sets should come from the same distribution as the train set. In other words, the test and the dev sets should be normalized using the statistics calculated on the train set.

Since the normalization in Keras is done using the ImageDataGenerator class. So, in this blog, we will discuss how to normalize the data during prediction using the ImageDataGenerator class?

Method-1

We create a separate ImageDataGenerator instance and then fit it on the train data as shown below.

train_datagen = ImageDataGenerator(featurewise_center=True,
                                  featurewise_std_normalization=True,
                                  rotation_range=40,
                                  width_shift_range=0.2,
                                  zoom_range=0.2,
                                  horizontal_flip=True)

# Fit the train_datagen to calculate the train data statistics
train_datagen.fit(x_train)

# Create a separate ImageDataGenerator instance
validation_datagen = ImageDataGenerator(featurewise_center=True,
                                        featurewise_std_normalization=True)

# Fit the validation_datagen on the train data 
validation_datagen.fit(x_train)

# Use any of the flow methods. Here I have used flow()
training_generator = train_datagen.flow(x_train, y_train, batch_size=64, seed=7)
validation_generator = validation_datagen.flow(x_val, y_val, batch_size=64, seed=7)

history = model.fit_generator(training_generator,steps_per_epoch=64, epochs=10, validation_data=validation_generator, validation_steps=32)

train_datagen = ImageDataGenerator(featurewise_center=True,

featurewise_std_normalization=True,

rotation_range=40,

width_shift_range=0.2,

zoom_range=0.2,

horizontal_flip=True)

# Fit the train_datagen to calculate the train data statistics

train_datagen.fit(x_train)

# Create a separate ImageDataGenerator instance

validation_datagen = ImageDataGenerator(featurewise_center=True,

featurewise_std_normalization=True)

# Fit the validation_datagen on the train data

validation_datagen.fit(x_train)

# Use any of the flow methods. Here I have used flow()

training_generator = train_datagen.flow(x_train, y_train, batch_size=64, seed=7)

validation_generator = validation_datagen.flow(x_val, y_val, batch_size=64, seed=7)

history = model.fit_generator(training_generator,steps_per_epoch=64, epochs=10, validation_data=validation_generator, validation_steps=32)

Similarly, we can do this for the test set. Because for validation and test set we need to fit the generator on the train data, this is very time-consuming.

Method-2

We use the “standardize” method provided under the ImageDataGenerator class. As already discussed, the “standardize” method performs in-place normalization to the batch of inputs, which makes it perfect for this work. You can read more about normalization here.

train_datagen = ImageDataGenerator(featurewise_center=True,
                                  featurewise_std_normalization=True,
                                  rotation_range=40,
                                  width_shift_range=0.2,
                                  zoom_range=0.2,
                                  horizontal_flip=True)

# Fit the train_datagen to calculate the train data statistics
train_datagen.fit(x_train)

# Apply standardize method
train_datagen.standardize(x_val)

training_generator = train_datagen.flow(x_train, y_train, batch_size=64, seed=7)

history = model.fit_generator(training_generator,steps_per_epoch=64, epochs=10, validation_data=(x_val,y_val))

train_datagen = ImageDataGenerator(featurewise_center=True,

featurewise_std_normalization=True,

rotation_range=40,

width_shift_range=0.2,

zoom_range=0.2,

horizontal_flip=True)

# Fit the train_datagen to calculate the train data statistics

train_datagen.fit(x_train)

# Apply standardize method

train_datagen.standardize(x_val)

training_generator = train_datagen.flow(x_train, y_train, batch_size=64, seed=7)

history = model.fit_generator(training_generator,steps_per_epoch=64, epochs=10, validation_data=(x_val,y_val))

Method-3

This is similar to the above method but is more explicit. In this we obtain the mean and the standard deviation from the generator and apply the desired normalization.

train_datagen = ImageDataGenerator(featurewise_center=True,
                                  featurewise_std_normalization=True,
                                  rotation_range=40,
                                  width_shift_range=0.2,
                                  zoom_range=0.2,
                                  horizontal_flip=True)

# Fit the train_datagen to calculate the train data statistics
train_datagen.fit(x_train)

# Apply the desired normalization
x_val -= train_datagen.mean
x_val /= train_datagen.std

training_generator = train_datagen.flow(x_train, y_train, batch_size=64, seed=7)

history = model.fit_generator(training_generator,steps_per_epoch=64, epochs=10, validation_data=(x_val,y_val))

train_datagen = ImageDataGenerator(featurewise_center=True,

featurewise_std_normalization=True,

rotation_range=40,

width_shift_range=0.2,

zoom_range=0.2,

horizontal_flip=True)

# Fit the train_datagen to calculate the train data statistics

train_datagen.fit(x_train)

# Apply the desired normalization

x_val -= train_datagen.mean

x_val /= train_datagen.std

training_generator = train_datagen.flow(x_train, y_train, batch_size=64, seed=7)

history = model.fit_generator(training_generator,steps_per_epoch=64, epochs=10, validation_data=(x_val,y_val))

I hope you might have now get some idea of how to apply normalization during prediction time. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Keras Callbacks – ModelCheckpoint

Leave a reply

In this blog, we will discuss how to checkpoint your model in Keras using ModelCheckpoint callbacks. Check-pointing your work is important in any field. If by-chance any problem or failure occurs, you don’t need to restart your work from zero, just resume from that checkpoint. This is very important in the field of deep learning where training can take days. So, let’s see how to use this.

Keras Function

Keras provides a built-in function for model check-pointing as

ModelCheckpoint(filepath, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)

1	ModelCheckpoint(filepath, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)

Let’s discuss in detail each of its arguments:

filepath: This is the path to save your model. Depending on the filepath specified, we can either save only the best model or save models at every epoch. Let’s see what this means.

If you specified the filepath as fixed, for example, ‘D:/best_model.hdf5’, this will overwrite your previous best model and what you end up is the best model up to that epoch.

If you specified a dynamic filepath, say, ‘D:/model{epoch:02d}.hdf5’, this will save the model at every epoch. For instance, for epoch 22, the model will be saved as model22.hdf5. You can only use variables like ‘epoch’ or keys in logs during training such as ‘loss’, ‘acc’, ‘val_loss’ and ‘val_acc’ for formatting the filepath. For example, ‘D:/model-{epoch:02d}-{val_acc:.2f}.hdf5’ is a valid filepath.

monitor: This is the quantity to monitor. This can take one of the values from ‘loss’, ‘acc’, ‘val_loss’ and ‘val_acc’.

verbose: This thing controls whether some information about model saving will be displayed or not. This is either 0 or 1. If 0, nothing will be displayed and for 1 something like this will be displayed depending on the behavior of the monitored quantity.

save_best_only: If set to false, then model after every epoch will be saved whether the monitored quantity increases or decreases. Otherwise, it will save the model depending on the ‘mode’ argument.

mode: This can take one of the values from auto, min, max. For instance, if the mode is ‘max’ and ‘val_acc’ is the monitored quantity, then for save_best_only = True the model will be saved only when ‘val_acc’ improves, otherwise, the model will not be saved at that epoch. For ‘val_loss’, this should be min. ‘auto’ mode automatically decides the direction depending on the monitored quantity.

save_weights_only: if True, then only the model weights will be saved otherwise the full model will be saved.

period: The callback will be applied after the specified period (no. of epochs)

How to use this?

All the callbacks are available in the keras.callbacks module so first import the ModelCheckpoint function from this module.
Then properly set up the function arguments.
Now, to apply this you need to pass this as a list in the .fit() method.

from keras.callbacks import ModelCheckpoint
filepath = 'D:/downloads/best_weights.hdf5'
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='auto',period=1)

model.fit(train_x, train_y, epochs=10, batch_size=128, callbacks=[checkpoint])

from keras.callbacks import ModelCheckpoint

filepath = 'D:/downloads/best_weights.hdf5'

checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='auto',period=1)

model.fit(train_x, train_y, epochs=10, batch_size=128, callbacks=[checkpoint])

Let’s take MNIST classification example to understand this

Import Libraries

import keras
import numpy as np
import matplotlib.pyplot as plt
from keras.layers import Dense, Flatten, Conv2D, MaxPool2D
from keras.models import Sequential
from keras.datasets import mnist

import keras

import numpy as np

import matplotlib.pyplot as plt

from keras.layers import Dense, Flatten, Conv2D, MaxPool2D

from keras.models import Sequential

from keras.datasets import mnist

Data Loading and Pre-processing

(train_x, train_y),(test_x, test_y) = mnist.load_data()

# Normalize between 0 and 1
train_x = train_x.astype('float32')/255
test_x = test_x.astype('float32')/255
# Add the channel dimension
train_x = np.expand_dims(train_x,axis=-1)
test_x = np.expand_dims(test_x,axis=-1)

(train_x, train_y),(test_x, test_y) = mnist.load_data()

# Normalize between 0 and 1

train_x = train_x.astype('float32')/255

test_x = test_x.astype('float32')/255

# Add the channel dimension

train_x = np.expand_dims(train_x,axis=-1)

test_x = np.expand_dims(test_x,axis=-1)

Build Model

model = Sequential()
model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(64,(3,3),activation='relu'))
model.add(MaxPool2D((2,2)))
model.add(Flatten())
model.add(Dense(512,activation='relu'))
model.add(Dense(10,activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

model = Sequential()

model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))

model.add(MaxPool2D((2,2)))

model.add(Conv2D(64,(3,3),activation='relu'))

model.add(MaxPool2D((2,2)))

model.add(Flatten())

model.add(Dense(512,activation='relu'))

model.add(Dense(10,activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

Callbacks

from keras.callbacks import ModelCheckpoint
filepath = 'D:/downloads/best_model.hdf5'
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='max',period=1)

from keras.callbacks import ModelCheckpoint

filepath = 'D:/downloads/best_model.hdf5'

checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=True, mode='max',period=1)

Fit Model

record = model.fit(train_x, train_y,validation_split=0.20, epochs=10, batch_size=128,callbacks=[checkpoint])

1	record = model.fit(train_x, train_y,validation_split=0.20, epochs=10, batch_size=128,callbacks=[checkpoint])

Load Weights and Evaluate test set

model.load_weights('D:/downloads/best_model.hdf5')
model.evaluate(test_x,test_y)

1 2	model.load_weights('D:/downloads/best_model.hdf5') model.evaluate(test_x,test_y)

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – flow_from_directory method

Leave a reply

In the previous blog, we learned how to generate batches of augmented data using the flow method. In that, the data was loaded in the memory. But this is not always the case. Sometimes, the datasets we download contains folders of data corresponding to the respective classes. To use the flow method, one may first need to append the data and corresponding labels into an array and then use the flow method on those arrays. Thus overall it is a tedious task.

This led to the need for a method that takes the path to a directory and generates batches of augmented data. In Keras, this is done using the flow_from_directory method. So, let’s discuss this method in detail.

Keras API

flow_from_directory(directory, target_size=(256, 256), color_mode='rgb', classes=None, class_mode='categorical', batch_size=32, shuffle=True, seed=None, save_to_dir=None, save_prefix='', save_format='png', follow_links=False, subset=None, interpolation='nearest')

1	flow_from_directory(directory, target_size=(256, 256), color_mode='rgb', classes=None, class_mode='categorical', batch_size=32, shuffle=True, seed=None, save_to_dir=None, save_prefix='', save_format='png', follow_links=False, subset=None, interpolation='nearest')

Here, the directory is the path of the directory that contains the sub-directories of the respective classes. Each subdirectory is treated as a different class. The name of the class can either be inferred from the subdirectory name or can be passed using the “classes” argument. The labels to these classes are assigned alphanumerically.

For instance, suppose you have a directory structure as shown below

data/
    train/
        dogs/
            dog001.jpg
            dog002.jpg
            ...
        cats/
            cat001.jpg
            cat002.jpg
            ...

data/

train/

dogs/

dog001.jpg

dog002.jpg

...

cats/

cat001.jpg

cat002.jpg

...

So, in this case, the directory will be the path to the train folder. If we set the “classes=None“, the class names will be inferred from the sub-directory names as “dogs” and “cats”. Because the labels are assigned alphanumerically, the labels for this will be {‘cats’: 0, ‘dogs’: 1}. If we have passed the argument classes=[‘Dog’,’Cat’], then the labels will be {‘Cat’: 0, ‘Dog’: 1}.

To check the class labels, we can use the “class_indices” argument as

train_datagen = ImageDataGenerator(...)
train_generator = train_datagen.flow_from_directory(...)
train_generator.class_indices

train_datagen = ImageDataGenerator(...)

train_generator = train_datagen.flow_from_directory(...)

train_generator.class_indices

This returns a dictionary containing the mapping from class names to class indices.

The labels generated depends on the “class_mode” argument. This can take one of “categorical“, “binary“, “sparse“, “input“, or None. Default is “categorical”.

If “binary“, the labels are “0” and “1”.
For “categorical“, we will have 2D one-hot encoded labels
If “sparse”, 1D integer labels
For autoencoders, pass this as “input“
Since during test time we have no labels, so pass as None.

Sometimes the datasets contain images that are not of the same size. So, using the “target_size” argument, we can resize the images to a fixed size using an interpolation method specified by the “interpolation” argument. Default is the nearest neighbor interpolation method.

You can also convert the color of the images using the “color_mode” argument. Available options are “grayscale“, “rgb“, “rgba“. Default is “rgb“.

You can also save the augmented images to the disk by specifying the “save_to_dir” argument. You can also select which format to save the image files and what prefix to use, using the “save_format” and “save_prefix” arguments respectively.

To see an example of flow_from_directory() method, you can refer to this blog.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – flow_from_dataframe method

Leave a reply

In the previous blogs, we discussed flow and flow_from_directory methods. Both these methods perform the same task i.e. generate batches of augmented data. The only thing that differs is the format or structuring of the datasets. Some of the most common formats (Image datasets) are

Keras builtin datasets
Datasets containing separate folders of data corresponding to the respective classes.
Datasets containing a single folder along with a CSV or JSON file that maps the image filenames with their corresponding classes.

We already know how to deal with the first two formats. In this blog, we will discuss how to perform data augmentation with the data available in the data frame. To do this, Keras provides a builtin flow_from_dataframe method. So, let’s discuss this method in detail.

Keras API

flow_from_dataframe(dataframe, directory=None, x_col='filename', y_col='class', target_size=(256, 256), color_mode='rgb', classes=None, class_mode='categorical', batch_size=32, shuffle=True, seed=None, save_to_dir=None, save_prefix='', save_format='png', subset=None, interpolation='nearest', drop_duplicates=True)

flow_from_dataframe(dataframe, directory=None, x_col='filename', y_col='class', target_size=(256, 256), color_mode='rgb', classes=None, class_mode='categorical', batch_size=32, shuffle=True, seed=None, save_to_dir=None, save_prefix='', save_format='png', subset=None, interpolation='nearest', drop_duplicates=True)

In this, you need to provide the data frame that contains the image names or file paths and the corresponding labels. Now, there are two cases possible:

if the data frame contains image names then you need to specify the directory where these images are residing, using the “directory” argument. See the example below.
if the data frame contains the absolute image paths then set the “directory” argument to None.

Similarly, for the labels column, the values can be string/list/tuple depending on the “class_mode” argument. For instance, if class_mode is binary, then the label column must contain the class values as strings. Note that we can have multiple label columns also. For instance regression tasks like bounding box prediction etc. Then you need to pass these columns as a list in the “y_col” argument.

Rest all the arguments are the same as discussed in the ImageDataGenerator flow_from_directory blog. Now let’s take an example to see how to use this.

We will take the traditional cats vs dogs dataset. First, download the dataset from Kaggle. This dataset contains two folders train and the test each containing 25000 and 12500 images respectively.

Create a Dataframe

The first step is to create a data frame that contains the filename and the corresponding labels column. For this, we will iterate over each image in the train folder and check the filename prefix. If it is a cat, set the label to 0 otherwise 1.

# Path to the train folder
original_train = 'D:/downloads/Data/train/'

filenames = os.listdir(original_train)
categories = []
for filename in filenames:
    category = filename.split('.')[0]
    if category == 'cat':
        categories.append('0')
    else:
        categories.append('1')

# Path to the train folder

original_train = 'D:/downloads/Data/train/'

filenames = os.listdir(original_train)

categories = []

for filename in filenames:

category = filename.split('.')[0]

if category == 'cat':

categories.append('0')

else:

categories.append('1')

Now create a data frame as

data = pd.DataFrame({'filename':filenames,'label':categories})

1	data = pd.DataFrame({'filename':filenames,'label':categories})

Create Generators

Now, we will create the train and validation generator using the flow_from_dataframe method as

datagen = ImageDataGenerator(rescale=1/255., validation_split=0.2)

train_generator = datagen.flow_from_dataframe(dataframe=data, directory=original_train,
                                             x_col='filename',
                                             y_col='label',
                                             target_size=(150,150),
                                             class_mode='binary',
                                             batch_size=100,
                                             subset='training',
                                             seed=7)

validation_generator = datagen.flow_from_dataframe(dataframe=data, directory=original_train,
                                             x_col='filename',
                                             y_col='label',
                                             target_size=(150,150),
                                             class_mode='binary',
                                             batch_size=100,
                                             subset='validation',
                                             seed=7)

datagen = ImageDataGenerator(rescale=1/255., validation_split=0.2)

train_generator = datagen.flow_from_dataframe(dataframe=data, directory=original_train,

x_col='filename',

y_col='label',

target_size=(150,150),

class_mode='binary',

batch_size=100,

subset='training',

seed=7)

validation_generator = datagen.flow_from_dataframe(dataframe=data, directory=original_train,

x_col='filename',

y_col='label',

target_size=(150,150),

class_mode='binary',

batch_size=100,

subset='validation',

seed=7)

Build the Model

model = Sequential()
model.add(Conv2D(32,(3,3),activation='relu',input_shape=(150,150,3)))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(64,(3,3),activation='relu'))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(128,(3,3),activation='relu'))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(128,(3,3),activation='relu'))
model.add(MaxPool2D((2,2)))
model.add(Flatten())
model.add(Dense(512,activation='relu'))
model.add(Dense(1,activation='sigmoid'))

model.compile(loss='binary_crossentropy',optimizer = 'Adam',metrics=['accuracy'])

model = Sequential()

model.add(Conv2D(32,(3,3),activation='relu',input_shape=(150,150,3)))

model.add(MaxPool2D((2,2)))

model.add(Conv2D(64,(3,3),activation='relu'))

model.add(MaxPool2D((2,2)))

model.add(Conv2D(128,(3,3),activation='relu'))

model.add(MaxPool2D((2,2)))

model.add(Conv2D(128,(3,3),activation='relu'))

model.add(MaxPool2D((2,2)))

model.add(Flatten())

model.add(Dense(512,activation='relu'))

model.add(Dense(1,activation='sigmoid'))

model.compile(loss='binary_crossentropy',optimizer = 'Adam',metrics=['accuracy'])

Train the Model

Let’s train the model using the fit_generator method.

train_steps = train_generator.n//train_generator.batch_size
validation_steps = validation_generator.n//validation_generator.batch_size

history = model.fit_generator(train_generator,steps_per_epoch=train_steps, epochs=20,
                              validation_data=validation_generator,validation_steps=validation_steps)

train_steps = train_generator.n//train_generator.batch_size

validation_steps = validation_generator.n//validation_generator.batch_size

history = model.fit_generator(train_generator,steps_per_epoch=train_steps, epochs=20,

validation_data=validation_generator,validation_steps=validation_steps)

Test time

So, for the test time, we can simply use the flow_from_directory method. You can use any method. For this, you need to create a subfolder inside the test folder. Remember not to shuffle the data at the test time. The class_mode argument should be set to None.

test_directory = 'D:/downloads/Data/test/'
test_datagen = ImageDataGenerator(rescale=1/255.)
test_generator = test_datagen.flow_from_directory(test_directory,target_size=(150,150),
                                                 shuffle=False,
                                                 class_mode=None,
                                                 batch_size=1,
                                                 seed=7)

test_directory = 'D:/downloads/Data/test/'

test_datagen = ImageDataGenerator(rescale=1/255.)

test_generator = test_datagen.flow_from_directory(test_directory,target_size=(150,150),

shuffle=False,

class_mode=None,

batch_size=1,

seed=7)

For predictions, we can simply use the predict_generator method.

STEP_SIZE_TEST=test_generator.n//test_generator.batch_size
test_generator.reset()
pred=model.predict_generator(test_generator,steps=STEP_SIZE_TEST,verbose=1)

predictions = []
for i in pred:
    if i >=0.5:
        predictions.append('1')
    else:
        predictions.append('0')

filenames=test_generator.filenames
results=pd.DataFrame({"Filename":filenames,
                      "Predictions":predictions})

STEP_SIZE_TEST=test_generator.n//test_generator.batch_size

test_generator.reset()

pred=model.predict_generator(test_generator,steps=STEP_SIZE_TEST,verbose=1)

predictions = []

for i in pred:

if i >=0.5:

predictions.append('1')

else:

predictions.append('0')

filenames=test_generator.filenames

results=pd.DataFrame({"Filename":filenames,

"Predictions":predictions})

That’s all for the flow_from_dataframe method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

ImageDataGenerator – flow method

2 Replies

In the previous blog, we have discussed how to apply different transformations to augment data using Keras ImageDataGenerator class. In this blog, we will learn how we can generate batches of the augmented data. This is done using the flow method which creates an iterator. We can easily iterate over the iterator to yield the batches of data. Let’s first discuss Keras ImageDataGenerator- flow method API and then we will see how to use this.

Keras API

flow(x, y=None, batch_size=32, shuffle=True, sample_weight=None, seed=None, save_to_dir=None, save_prefix='', save_format='png', subset=None)

1	flow(x, y=None, batch_size=32, shuffle=True, sample_weight=None, seed=None, save_to_dir=None, save_prefix='', save_format='png', subset=None)

Here, x is the Numpy array of rank 4 (batches, image_width, image_height, channels) and y is the corresponding labels. For greyscale image, channels must be equal to 1.

One can also save the augmented images to the disk by specifying the “save_to_dir” argument. You can also select which format to save the image files and what prefix to use, using the “save_format” and “save_prefix” arguments respectively.

For instance, the below code saves the augmented file to the downloads folder with the name as “aug_0_2345” etc.

data_generator = datagen.flow(img, save_to_dir='D:/downloads/', save_format='jpeg', save_prefix='aug')

1	data_generator = datagen.flow(img, save_to_dir='D:/downloads/', save_format='jpeg', save_prefix='aug')

Another interesting thing is that one can weight each sample using the “sample_weight” argument. Now, while calculating the loss each sample has its own weight which controls the gradient direction. This should have the same length as the input array. These sample_weights, if not None, are returned as it is.

“subset” decides whether the data generated is for training or validation. This works as follows:

First of all, depending on the input length and validation_split argument in the ImageDataGenerator, the split index is determined as shown

split_idx = int(len(x) * image_data_generator._validation_split)

1	split_idx = int(len(x) * image_data_generator._validation_split)

Now, if subset is ‘validation’, then the data is splitted as

x = x[:split_idx]

1	x = x[:split_idx]

Rest of the data is reserved for the training. As we can see that splitting is straight i.e. it reserves first n examples for validation and rest for training. So, training and validation may have a different number of classes after the split, if the data is not properly shuffled.

Note for the test set, set shuffle equal to False. Set the batch size carefully for the test set. Make sure that this divides exactly the test set as you don’t want to leave some examples or predict multiple times some examples.

Now, you might have got some idea about the flow method arguments. Next, let’s see how this method works.

How the flow method works?

Firstly, this generates random parameters for a transformation using the “get_random_transform” method.
Then these transformations are applied using the “apply_transform” method.
Finally, the image is standardized using the “standardize” method.

How to use?

Let’s take MNIST digits classification example. Firstly load the required libraries and the data.

1. Load Libraries and Data

from keras.layers import Dense, Flatten, Conv2D, MaxPool2D
from keras.models import Sequential
from keras.datasets import mnist
import numpy as np
import matplotlib.pyplot as plt

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = np.expand_dims(x_train, axis=-1)

from keras.layers import Dense, Flatten, Conv2D, MaxPool2D

from keras.models import Sequential

from keras.datasets import mnist

import numpy as np

import matplotlib.pyplot as plt

(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = np.expand_dims(x_train, axis=-1)

2. Build model

model = Sequential()
model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))
model.add(MaxPool2D((2,2)))
model.add(Conv2D(64,(3,3),activation='relu'))
model.add(MaxPool2D((2,2)))
model.add(Flatten())
model.add(Dense(512,activation='relu'))
model.add(Dense(10,activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

model = Sequential()

model.add(Conv2D(32,(3,3),activation='relu',input_shape=(28,28,1)))

model.add(MaxPool2D((2,2)))

model.add(Conv2D(64,(3,3),activation='relu'))

model.add(MaxPool2D((2,2)))

model.add(Flatten())

model.add(Dense(512,activation='relu'))

model.add(Dense(10,activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

3. Data Augmentation

Create an ImageDataGenerator instance with the set of transformations you want to perform. If you were to perform augmentation using transformation such as rotation, cropping, etc. better create a separate generator for the validation set. Because validation data should be kept fixed. In that case, don’t use the validation_split argument. Instead, use some other methods for splitting, for instance, train_test_split, etc.

datagen = ImageDataGenerator(rescale=1/255.,validation_split=0.2)

1	datagen = ImageDataGenerator(rescale=1/255.,validation_split=0.2)

4. flow method

Based on the validation split argument in the above code, we create a separate training and validation generator using the “subset” argument.

training_generator = datagen.flow(x_train, y_train, batch_size=64,subset='training',seed=7)
validation_generator = datagen.flow(x_train, y_train, batch_size=64,subset='validation',seed=7)

1 2	training_generator = datagen.flow(x_train, y_train, batch_size=64,subset='training',seed=7) validation_generator = datagen.flow(x_train, y_train, batch_size=64,subset='validation',seed=7)

5. Visualize the training generator

Let’s plot the first outcome of 6 batches.

plt.figure(figsize=(10,5))
for i in range(6):
    plt.subplot(2,3,i+1)
    for x,y in training_generator:
        plt.imshow((x[0]/255).reshape(28,28),cmap='gray')
        plt.title('y={}'.format(y[0]))
        plt.axis('off')
        break
plt.tight_layout()
plt.show()

plt.figure(figsize=(10,5))

for i in range(6):

plt.subplot(2,3,i+1)

for x,y in training_generator:

plt.imshow((x[0]/255).reshape(28,28),cmap='gray')

plt.title('y={}'.format(y[0]))

plt.axis('off')

break

plt.tight_layout()

plt.show()

6. Train model

history = model.fit_generator(training_generator,steps_per_epoch=(len(x_train)*0.8)//64, epochs=10, validation_data=validation_generator, validation_steps=(len(x_train)*0.2)//64)

1	history = model.fit_generator(training_generator,steps_per_epoch=(len(x_train)0.8)//64, epochs=10, validation_data=validation_generator, validation_steps=(len(x_train)0.2)//64)

Similarly, you can create the test generator and evaluate the performance of the model on the test set. This is how you can use the flow method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.