ate a helper proxy to easily launch an ` rpc_sync  ` using the owner of the
RRef as the destination to run functions on the object referenced by this
RRef. More specifically, ` rref.rpc_sync().func_name(*args,  **kwargs)  ` is
the same as the following:

    
    
    >>> def run(rref, func_name, args, kwargs):
    >>>   return getattr(rref.local_value(), func_name)(*args, **kwargs)
    >>>
    >>> rpc.rpc_sync(rref.owner(), run, args=(rref, func_name, args, kwargs))
    

Example::

    
    
    
    >>> from torch.distributed import rpc
    >>> rref = rpc.remote("worker1", torch.add, args=(torch.zeros(2, 2), 1))
    >>> rref.rpc_sync().size()  # returns torch.Size([2, 2])
    >>> rref.rpc_sync().view(1, 4)  # returns tensor([[1., 1., 1., 1.]])
    

` to_here ` (  _self: torch.distributed.rpc.RRef_ , _timeout: float = -1.0_ )
→ object  ¶

    

Blocking call that copies the value of the RRef from the owner to the local
node and returns it. If the current node is the owner, returns a reference to
the local value.

Parameters

    

**timeout** ( [ _float_
](https://docs.python.org/3/library/functions.html#float "\(in Python v3.8\)")
_,_ _optional_ ) – Timeout for ` to_here  ` . If the call does not complete
within this timeframe, an exception indicating so will be raised. If this
argument is not provided, the default RPC timeout (60s) will be used.

More Information about RRef

  * [ Remote Reference Protocol ](rpc/rref.html)
    * [ Background ](rpc/rref.html#background)
    * [ Assumptions ](rpc/rref.html#assumptions)
    * [ RRef Lifetime ](rpc/rref.html#rref-lifetime)
      * [ Design Reasoning ](rpc/rref.html#design-reasoning)
      * [ Implementation ](rpc/rref.html#implementation)
    * [ Protocol Scenarios ](rpc/rref.html#protocol-scenarios)
      * [ User Share RRef with Owner as Return Value ](rpc/rref.html#user-share-rref-with-owner-as-return-value)
      * [ User Share RRef with Owner as Argument ](rpc/rref.html#user-share-rref-with-owner-as-argument)
      * [ Owner Share RRef with User ](rpc/rref.html#owner-share-rref-with-user)
      * [ User Share RRef with User ](rpc/rref.html#user-share-rref-with-user)

##  Distributed Autograd Framework  ¶

This module provides an RPC-based distributed autograd framework that can be
used for applications such as model parallel training. In short, applications
may send and receive gradient recording tensors over RPC. In the forward pass,
we record when gradient recording tensors are sent over RPC and during the
backward pass we use this information to perform a distributed backward pass
using RPC. For more details see [ Distributed Autograd Design
](rpc/distributed_autograd.html#distributed-autograd-design) .

_class_ ` torch.distributed.autograd. ` ` context ` [ [source]
](_modules/torch/distributed/autograd.html#context) ¶

    

Context object to wrap forward and backward passes when using distributed
autograd. The ` context_id  ` generated in the ` with  ` statement is required
to uniquely identify a distributed backward pass on all workers. Each worker
stores metadata associated with this ` context_id  ` , which is required to
correctly execute a distributed autograd pass.

Example::

    
    
    
    >>> import torch.distributed.autograd as dist_autograd
    >>> with dist_autograd.context() as context_id:
    >>>   t1 = torch.rand((3, 3), requires_grad=True)
    >>>   t2 = torch.rand((3, 3), requires_grad=True)
    >>>   loss = rpc.rpc_sync("worker1", torch.add, args=(t1, t2)).sum()
    >>>   dist_autograd.backward(context_id, [loss])
    

` torch.distributed.autograd. ` ` backward ` (  _context_id: int, roots:
List[Tensor], retain_graph = False_ )  → None  ¶

    

Kicks off the distributed backward pass using the provided roots. This
currently implements the [ FAST mode algorithm
](rpc/distributed_autograd.html#fast-mode-algorithm) which assumes all RPC
messages sent in the same distributed autograd context across workers would be
part of the autograd graph durin