Jacobian matrix in PyTorch - GeeksforGeeks

15 Jul, 2021
Introduction:
The Jacobian is a very powerful operator used to calculate the partial derivatives of a given function with respect to its constituent latent variables. For refresher purposes, the Jacobian of a given function  with respect to a vector  is defined as
 
Example:
Suppose we have a vector  and a function . To calculate the Jacobian of  with respect to , we can use the above-mentioned formula to get
To achieve the same functionality as above, we can use the jacobian() function from Pytorch’s torch.autograd.functional utility to compute the Jacobian matrix of a given function for some inputs.
Syntax: torch.autograd.functional.jacobian(func, inputs, create_graph=False, strict=False, vectorize=False)
Parameters:
func: A Python function which takes input and outputs a Pytorch Tensor(or a tuple of Tensors).
inputs: The inputs are passed as parameters to the ‘func’ method. The input can be a single Pytorch Tensor(or a tuple of Tensors)
create_graph: If True, the autograd engine creates a backpropable graph for doing further operations on gradients. Defaults to False.
strict: If True, an error will be raised when the engine detects that there exists an input such that all the outputs are independent of it. If False, returns zero gradients for such inputs. Defaults to False.
vectorize: Still in it’s experimental phase if True, the function uses the vmap prototype feature to compute the gradients by only one call of the autograd engine instead of one call per each row of the matrix. Defaults to False.
Installation:
For this article, you only need the torch utility, which can be downloaded through the pip package manager using:
pip install torch
Example usage of the function:
We’ll be using the same function and vector for ease of flow, as discussed in the above example. Since tensors are the basic building blocks of the Pytorch package, we’ll be using them for representing both the inputs vectors and the given function. This article assumes a basic familiarity with Pytorch tensors which can be quickly reviewed by going through Pytorch articles. 
Theoretical verification:
Suppose we have a vector    as a given input. By plugging in the values of  into the above derived equation we will get 
Code: Python implementation to show the working of Jacobian Matrix using Pytorch
Python
from torch.autograd.functional import jacobianfrom torch import tensor #Defining the main functiondef f(x1,x2,x3):    return (x1 + x2, x3*x1, x2**3) #Defining input tensorsx1 = tensor(3.0)x2 = tensor(4.0)x3 = tensor(5.0) #Printing the Jacobianprint(jacobian(f,(x1,x2,x3)))
Output:
((tensor(1.), tensor(1.), tensor(0.)), 
(tensor(5.), tensor(0.), tensor(3.)), 
(tensor(0.), tensor(48.), tensor(0.)))
The output is exactly similar to our theoretical verification! Using a similar approach, we can calculate the Jacobian matrix of any given function using the Pytorch API.
References:
https://pytorch.org/docs/stable/autograd.html#torch.autograd.functional.jacobianhttps://pytorch.org/docs/stable/tensors.html
https://pytorch.org/docs/stable/autograd.html#torch.autograd.functional.jacobian
https://pytorch.org/docs/stable/tensors.html
simmytarika5
Picked
Python Framework
Python-PyTorch
Machine Learning
Python
Machine Learning
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Comments
Old Comments
Difference between Informed and Uninformed Search in AI
Deploy Machine Learning Model using Flask
Support Vector Machine Algorithm
Types of Environments in AI
k-nearest neighbor algorithm in Python
Read JSON file using Python
Adding new column to existing DataFrame in Pandas
Python map() function
How to get column names in Pandas dataframe