How to Create a Custom Loss Function | Keras | by Shiva Verma | Towards Data Science

The Loss Function is one of the important components of Neural Network. Loss is nothing but a prediction error of Neural Net. And the method to calculate the loss is called Loss Function.
Loss is used to calculate the gradients for the neural net. And gradients are used to update the weights. This is how a Neural Net is trained.
Keras has many inbuilt loss functions, which I have covered in one of my previous blog. These loss functions are enough for many typical Machine Learning tasks such as Classification and Regression.
But there might be some tasks where we need to implement a custom loss function, which I will be covering in this Blog.
Before starting, let’s quickly review how we use an inbuilt loss function in Keras.
Following is a simple example of using an inbuilt loss function.
Here we used in-built categorical_crossentropy loss function, which is mostly used for the classification task. We pass the name of the loss function in model.compile() method.
We can create a custom loss function simply as follows.
There are following rules you have to follow while building a custom loss function.
The loss function should take only 2 arguments, which are target value(y_true) and predicted value(y_pred). Because in order to measure the error in prediction(loss) we need these 2 values. These arguments are passed from the model itself at the time of fitting the data.
Loss function must make use of y_pred value while calculating the loss, if you do not do so, then gradient expression will not be defined and you will get an error.
Then you can simply plug this function into model.compile() method to use it.
The first dimension of arguments y_true and y_pred is always the same as batch size. Ex - If you are fitting data with a batch size of 32, and your neural net has 5 output nodes, then the shape of y_pred would be (32, 5). Because there would be 32 outputs, each having 5 values.
Loss function should always return a vector of length batch_size. Because you have to return a loss for each datapoint. Ex - If you are fitting data with a batch size of 32, then you need to return a vector of length 32 from your loss function.
Let’s look at an example for better understanding.
Let’s say, you have designed a Neural Net for some regression task, which outputs a vector [x1, x2] of length 2.
Assume that value x2 is more important than x1, and you want it to be really close to the target value. For this scenario, you can create a custom MSE (mean squared error) loss function which penalizes prediction error for x2 more than x1.
Here I am multiplying loss values with weights 0.3 and 0.7, giving more penalty to the second value, which is the x2. You can decide the weights as per your requirement.
I have also written down the shape of variables in the code snippet. You can see the shape of the final loss would be(batch_size,), means one loss value for each data point.
Now you can simply plug this loss function to your model.
model.compile(loss=custom_mse, optimizer='adam')
I would advise you to use Keras backend functions instead of Numpy functions to avoid any misadventure. Keras backend functions work almost similar to Numpy functions.
Now you already know how to build a custom loss function. You can also create custom evaluation metrics in the exact similar way. Here I have created a Mean Absolute Error(MAE) metrics and plugged it into model.compile() method.
If you want to understand the concept of Dimensions and Shapes. You can go through the following article where I am explaining these concepts in detail.