Decoding the input shapes and output shapes of Multi Layered Perceptron model.
By: Chamanth Mvs
Tags: Deep Learning, Data Science, Neural Networks, Machine Learning

Decoding the input shapes and output shapes of Multi Layered Perceptron model. Chamanth mvs Follow Feb 29 · 7 min read

I assume that, people who are reading this blog already have some idea about deep learning concepts and Multi Layered Perceptrons. If you are completely new to these concepts, you may not understand this blog. kindly, revisit after you learn basics of deep learning.

In very simple terms, important task in training a neural network model is to find best weights (weight matrix), which helps in reducing the error and minimizing the loss to achieve the objective of the given problem (classification or regression).

So, understanding the model summary in detail (output shapes, input shapes and number of parameters) is very important while training a neural network model, because these will help you to understand , what is happening within the model, which you defined.

This is very short blog and will try to explain these topics using example with help of (toy dataset).

Note : Everything discussed in this blog is only with respect to fully connected multi layered perceptron.

Dataset : Iris flower dataset

About dataset :

The Iris flower data set is a multivariate data set introduced by the British statistician and biologist Ronald Fisher in his 1936 paper . Iris is a type of flower plant.

The data set consists of 50 samples for each of three species of Iris (Iris Setosa, Iris virginica, and Iris versicolor). Altogether this dataset consists of 150 samples.

Fig-1 : iris_dataset

Four features were measured from each sample: the length and the width of the sepals and petals (Petal Length, Petal Width, Sepal Length, Sepal Width) in centimeters.

target column represents three values 0,1,2 , which represents ‘setosa’, ‘versicolor’, ‘virginica’ respectively.

Input shape, Output shape and Number of Parameters.

Fig-2 : Data (without target variable)

While training a neural network, we seperate target variable from the dataset and train the model with data and target variable separately. So,now our data appears as shown in Fig -2.

Before discussing about these topics, I wanted to make clear about architecture I am using to explain these parameters.

Model Architecture

MLP model without bias term

Fig-3 : MLP without bias term (only weight matrices)

The simplest function for such models, can be defined as f(x) = W^t * X, where W is Weight matrix and X is data.

MLP model with bias term

Fig-4 : MLP with bias term (Weight matrices as well as bias term)

The simplest function for such models, can be defined as f(x) = W^t * X + b , where W is Weight matrix and X is data and b is Bias term.

Model architecture is 4 -layered network with 2 hidden layers

Fig-5 : Model architecture

‘dense_input’ is the input layer .

‘dense’ , ‘dense_1’ are the hidden layers — with 5 and 4 neurons respectively.

dense_2 is the output layer (softmax layer)with 3 units, as we are solving multi class classification problem (3-classes).

Input shape and output shape

Fig-6 : How data is passed through network (feed forward)

Fig-7 : input and output shapes of MLP (as in Fig-6)

As said earlier, in training phase, data and target are passed separately, when data is passed through the network.

As, each datapoint has 4 features (refer Fig-2),

Input layer:

input shape and output shape of input layer will be same (you can think of it as identity). It is the first step, when you pass your data to the network. So, it will output (which has output shape).

As we are passing through the network with the data, which has 4 features (refer Fig-2). So, the output from input_layer would be (?,4).

“ ?” — specifies batch size.

batch size is nothing but, how many data points will be passed through the neural network (Fig-6) in one batch at a time.

So, if you define batch size as 16, the output shape will be (16,4), which specifies 16 data points each with 4 dimension (or 4 features).

Hidden layer-1 :

This layer is receiving input from Input_layer. So, output_shape of Input_layer becomes input_shape of hidden layer-1. As per our example, output_shape of input_layer is (?,4), which became input_shape to hidden layer-1.

So, intuitively speaking

“Output_shape of preceding layer becomes Input_shape of next layer in Multi Layered Perceptron networks”.

Hidden layer -1 has 5 neurons or units (Fig-6), which contains some activation functions to introduce non-linearity to the model, after the input is passed through these 5 neurons, all 5 neurons generates output. So, the output_shape of hidden layer -1 is (?,5) , where “?” specifies batch size.

Hidden layer-2 :

As said above, where the output_shape of hidden layer-1 becomes input_shape of hidden layer-2. So, the input_shape of hidden layer-2 is (?,5) , in other terms (hidden layer-2 is receiving input from 5 neuron units).

Hidden layer -2 has 4 neurons or units (Fig-6), which contains some activation functions to introduce non-linearity to the model, after the input is passed through these 4 neurons, all 4 neurons generates output. So, the output_shape of hidden layer -2 is (?,4) , where “?” specifies batch size.

Output layer :

As said above, where the output_shape of hidden layer-2 becomes input_shape of output layer. So, the input_shape of output layer is (?,4) , in other terms (output layer is receiving input from 4 neuron units (from hidden layer-2 )).

Output layer has 3 units (Fig-6), as we are dealing with 3-class classification problem (we need to predict whether the given data point belongs to any of these 3 types of flower classes). If we have 10 classes to classify (then output layer contains 10 units). It changes, according to the problem statement. So, in our example, the output_shape of output layer is (?,3) , where “?” specifies batch size.

Number of parameters in a MLP network