[ ](https://pytorch.org/)

  * [ Get Started ](https://pytorch.org/get-started)
  * Ecosystem 

[ Models (Beta)  Discover, publish, and reuse pre-trained models
](https://pytorch.org/hub) [ Tools & Libraries  Explore the ecosystem of tools
and libraries  ](https://pytorch.org/ecosystem)

  * [ Mobile ](https://pytorch.org/mobile)
  * [ Blog ](https://pytorch.org/blog/)
  * [ Tutorials ](https://pytorch.org/tutorials)
  * [ Docs ](https://pytorch.org/docs/stable/index.html)
  * Resources 

[ Developer Resources  Find resources and get questions answered
](https://pytorch.org/resources) [ About  Learn about PyTorch’s features and
capabilities  ](https://pytorch.org/features)

  * [ Github ](https://github.com/pytorch/pytorch)

Table of Contents

[ 1.6.0 ▼ ](http://pytorch.org/docs/versions.html)

Notes

  * [ Automatic Mixed Precision examples ](amp_examples.html)
  * Autograd mechanics 
  * [ Broadcasting semantics ](broadcasting.html)
  * [ CPU threading and TorchScript inference ](cpu_threading_torchscript_inference.html)
  * [ CUDA semantics ](cuda.html)
  * [ Distributed Data Parallel ](ddp.html)
  * [ Extending PyTorch ](extending.html)
  * [ Frequently Asked Questions ](faq.html)
  * [ Features for large-scale deployments ](large_scale_deployments.html)
  * [ Multiprocessing best practices ](multiprocessing.html)
  * [ Reproducibility ](randomness.html)
  * [ Serialization semantics ](serialization.html)
  * [ Windows FAQ ](windows.html)

Language Bindings

  * [ C++ ](../cpp_index.html)
  * [ Javadoc ](https://pytorch.org/javadoc/)

Python API

  * [ torch ](../torch.html)
  * [ torch.nn ](../nn.html)
  * [ torch.nn.functional ](../nn.functional.html)
  * [ torch.Tensor ](../tensors.html)
  * [ Tensor Attributes ](../tensor_attributes.html)
  * [ Tensor Views ](../tensor_view.html)
  * [ torch.autograd ](../autograd.html)
  * [ torch.cuda ](../cuda.html)
  * [ torch.cuda.amp ](../amp.html)
  * [ torch.distributed ](../distributed.html)
  * [ torch.distributions ](../distributions.html)
  * [ torch.futures ](../futures.html)
  * [ torch.hub ](../hub.html)
  * [ torch.jit ](../jit.html)
  * [ torch.nn.init ](../nn.init.html)
  * [ torch.onnx ](../onnx.html)
  * [ torch.optim ](../optim.html)
  * [ Complex Numbers ](../complex_numbers.html)
  * [ Quantization ](../quantization.html)
  * [ Distributed RPC Framework ](../rpc.html)
  * [ torch.random ](../random.html)
  * [ torch.sparse ](../sparse.html)
  * [ torch.Storage ](../storage.html)
  * [ torch.utils.bottleneck ](../bottleneck.html)
  * [ torch.utils.checkpoint ](../checkpoint.html)
  * [ torch.utils.cpp_extension ](../cpp_extension.html)
  * [ torch.utils.data ](../data.html)
  * [ torch.utils.dlpack ](../dlpack.html)
  * [ torch.utils.mobile_optimizer ](../mobile_optimizer.html)
  * [ torch.utils.model_zoo ](../model_zoo.html)
  * [ torch.utils.tensorboard ](../tensorboard.html)
  * [ Type Info ](../type_info.html)
  * [ Named Tensors ](../named_tensor.html)
  * [ Named Tensors operator coverage ](../name_inference.html)
  * [ torch.__config__ ](../__config__.html)

Libraries

  * [ torchaudio ](https://pytorch.org/audio)
  * [ torchtext ](https://pytorch.org/text)
  * [ torchvision ](../torchvision/index.html)
  * [ TorchElastic ](https://pytorch.org/elastic/)
  * [ TorchServe ](https://pytorch.org/serve)
  * [ PyTorch on XLA Devices ](http://pytorch.org/xla/)

Community

  * [ PyTorch Contribution Guide ](../community/contribution_guide.html)
  * [ PyTorch Governance ](../community/governance.html)
  * [ PyTorch Governance | Persons of Interest ](../community/persons_of_interest.html)

  * [ Docs ](../index.html) >
  * Autograd mechanics 
  * [ ![](../_static/images/view-page-source-icon.svg) ](../_sources/notes/autograd.rst.txt)

Shortcuts

#  Autograd mechanics  ¶

This note will present an overview of how autograd works and records the
operations. It’s not strictly necessary to understand all this, but we
recommend getting familiar with it, as it will help you