The Keras functional API is used to define complex models in deep learning . On of its good use case is to use multiple input and output in a model. In this blog we will learn how to define a keras model which takes more than one input and output.
Multi Output Model
Let say you are using MNIST dataset (handwritten digits images) for creating an autoencoder and classification problem both. In that case, you will be having single input but multiple outputs (predicted class and the generated
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
from keras.layers import Dense, Input from keras.models import Model # creating model inputs = Input(shape = (784,)) dense1 = Dense(512, activation = 'relu')(inputs) dense2 = Dense(128, activation = 'relu')(dense1) dense3 = Dense(32, activation = 'relu')(dense2) # create classification output classification_output = Dense(10, activation = 'softmax')(dense3) # use output from dense layer 3 to create autoencder output up_dense1 = Dense(128, activation = 'relu')(dense3) up_dense2 = Dense(512, activation = 'relu')(up_dense1) decoded_outputs = Dense(784)(up_dense2) |
In the above code we have used a single input layer and two output layers as ‘classification_output’ and ‘decoder_output’. Let’s see how to create model with these input and outputs.
|
1 2 |
model = Model(inputs, [classification_output,decoded_outputs]) model.summary() |
Now we have created the model, the next thing is to compile this model. Here we will define two loss functions for both outputs. Also we can assign weights for both losses. See code.
|
1 2 3 4 |
m = 256 n_epoch = 25 model.compile(optimizer='adam', loss=['categorical_crossentropy', 'mse'], loss_weights = [1.0, 0.5], metrics = ['accuracy']) model.fit(output_X_train,[Y_train, output_X_train], epochs=n_epoch, batch_size=m, shuffle=True) |
Multi Input Model
Let’s take an example where you need to take two inputs: one grayscale image and another RGB image. Using these two images you want to do an image classification. To perform this, we will use Keras functional API. Let’s see
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# feature extraction from gray scale image inputs = Input(shape = (28,28,1)) conv1 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(inputs) pool1 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1) conv2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1) pool2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2) flat_1 = Flatten()(pool2) # feature extraction from RGB image inputs_2 = Input(shape = (28,28,3)) conv1_2 = Conv2D(16, (3,3), activation = 'relu', padding = "SAME")(inputs_2) pool1_2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv1_2) conv2_2 = Conv2D(32, (3,3), activation = 'relu', padding = "SAME")(pool1_2) pool2_2 = MaxPooling2D(pool_size = (2,2), strides = 2)(conv2_2) flat_2 = Flatten()(pool2_2) # concatenate both feature layers and define output layer after some dense layers concat = concatenate([flat_1,flat_2]) dense1 = Dense(512, activation = 'relu')(concat) dense2 = Dense(128, activation = 'relu')(dense1) dense3 = Dense(32, activation = 'relu')(dense2) output = Dense(10, activation = 'softmax')(dense3) # create model with two inputs model = Model([inputs,inputs_2], dense1) |
In the above code, we have extracted two different feature layers from both inputs and then concatenated both to create output layer. And created model with two inputs and one output.
A nice example where you can you use both multi input and multi output is capsule network. If you want to take a look into this, refer this blog.
Hope you enjoy reading.
If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.
Hey, I am trying the develop the multi-output model However while prediction I am getting strange results and unable to visualize it.
I have developed the model having 1 input and 3 output and model is working fine without any error. But while prediction (model.predict(input)) I should get 3 samples, one for each output, however i am getting 516 output samples.
Could you please help me in this.
Thanks
As you are saying that the model is working fine, so the only issue I think is that the output arrays are concatenated in the list. You might have done something like this
One approach is you do pred[0][i],pred[1][i] and pred[2][i] to access the 3 outputs corresponding to the ith example.
Another thing to do is define 3 outputs as
Hope this helps. If it still doesn’t resolve, please provide the code.
Hi,
If you feed 2 images as input, how do you keep the consistency between image pairs? Could you elaborate on that more? Thanks
i was working with vgg16 and mobile net,how do i use “model.fit()” when i have two inputs. to be specific how to pass train data set for both