[ ](https://pytorch.org/)

  * [ Get Started ](https://pytorch.org/get-started)
  * Ecosystem 

[ Models (Beta)  Discover, publish, and reuse pre-trained models
](https://pytorch.org/hub) [ Tools & Libraries  Explore the ecosystem of tools
and libraries  ](https://pytorch.org/ecosystem)

  * [ Mobile ](https://pytorch.org/mobile)
  * [ Blog ](https://pytorch.org/blog/)
  * [ Tutorials ](https://pytorch.org/tutorials)
  * [ Docs ](https://pytorch.org/docs/stable/index.html)
  * Resources 

[ Developer Resources  Find resources and get questions answered
](https://pytorch.org/resources) [ About  Learn about PyTorch’s features and
capabilities  ](https://pytorch.org/features)

  * [ Github ](https://github.com/pytorch/pytorch)

Table of Contents

[ 1.6.0 ▼ ](http://pytorch.org/docs/versions.html)

Notes

  * [ Automatic Mixed Precision examples ](notes/amp_examples.html)
  * [ Autograd mechanics ](notes/autograd.html)
  * [ Broadcasting semantics ](notes/broadcasting.html)
  * [ CPU threading and TorchScript inference ](notes/cpu_threading_torchscript_inference.html)
  * [ CUDA semantics ](notes/cuda.html)
  * [ Distributed Data Parallel ](notes/ddp.html)
  * [ Extending PyTorch ](notes/extending.html)
  * [ Frequently Asked Questions ](notes/faq.html)
  * [ Features for large-scale deployments ](notes/large_scale_deployments.html)
  * [ Multiprocessing best practices ](notes/multiprocessing.html)
  * [ Reproducibility ](notes/randomness.html)
  * [ Serialization semantics ](notes/serialization.html)
  * [ Windows FAQ ](notes/windows.html)

Language Bindings

  * [ C++ ](cpp_index.html)
  * [ Javadoc ](https://pytorch.org/javadoc/)

Python API

  * [ torch ](torch.html)
  * [ torch.nn ](nn.html)
  * [ torch.nn.functional ](nn.functional.html)
  * [ torch.Tensor ](tensors.html)
  * [ Tensor Attributes ](tensor_attributes.html)
  * [ Tensor Views ](tensor_view.html)
  * [ torch.autograd ](autograd.html)
  * [ torch.cuda ](cuda.html)
  * [ torch.cuda.amp ](amp.html)
  * torch.distributed 
  * [ torch.distributions ](distributions.html)
  * [ torch.futures ](futures.html)
  * [ torch.hub ](hub.html)
  * [ torch.jit ](jit.html)
  * [ torch.nn.init ](nn.init.html)
  * [ torch.onnx ](onnx.html)
  * [ torch.optim ](optim.html)
  * [ Complex Numbers ](complex_numbers.html)
  * [ Quantization ](quantization.html)
  * [ Distributed RPC Framework ](rpc.html)
  * [ torch.random ](random.html)
  * [ torch.sparse ](sparse.html)
  * [ torch.Storage ](storage.html)
  * [ torch.utils.bottleneck ](bottleneck.html)
  * [ torch.utils.checkpoint ](checkpoint.html)
  * [ torch.utils.cpp_extension ](cpp_extension.html)
  * [ torch.utils.data ](data.html)
  * [ torch.utils.dlpack ](dlpack.html)
  * [ torch.utils.mobile_optimizer ](mobile_optimizer.html)
  * [ torch.utils.model_zoo ](model_zoo.html)
  * [ torch.utils.tensorboard ](tensorboard.html)
  * [ Type Info ](type_info.html)
  * [ Named Tensors ](named_tensor.html)
  * [ Named Tensors operator coverage ](name_inference.html)
  * [ torch.__config__ ](__config__.html)

Libraries

  * [ torchaudio ](https://pytorch.org/audio)
  * [ torchtext ](https://pytorch.org/text)
  * [ torchvision ](torchvision/index.html)
  * [ TorchElastic ](https://pytorch.org/elastic/)
  * [ TorchServe ](https://pytorch.org/serve)
  * [ PyTorch on XLA Devices ](http://pytorch.org/xla/)

Community

  * [ PyTorch Contribution Guide ](community/contribution_guide.html)
  * [ PyTorch Governance ](community/governance.html)
  * [ PyTorch Governance | Persons of Interest ](community/persons_of_interest.html)

  * [ Docs ](index.html) >
  * Distributed communication package - torch.distributed 
  * [ ![](_static/images/view-page-source-icon.svg) ](_sources/distributed.rst.txt)

Shortcuts

#  Distributed communication package - torch.distributed  ¶

Note

Please refer to [ PyTorch Distributed Overview
](https://pytorch.org/tutorials/beginner/dist_overview.html) for a brief
introduction to all features related to distributed training