shrinath-suresh/blogs-docs-splitted huggingface dataset:
local host has a handle to this module and invoke this module similar
to a regular nn.Module. The invocation however incurs RPC calls to the
remote end and can be performed asynchronously if needed via
additional APIs supported by RemoteModule.
class torch.distributed.nn.api.remote_module.RemoteModule(args, *kwargs)
  A RemoteModule instance can only be created after RPC
  initialization. It creates a user-specified module on a
  specified remote node. It behaves like a regular "nn.Module"
  except that the "forward" method is executed on the remote node.
  It takes care of autograd recording to ensure the backward pass
  propagates gradients back to the corresponding remote module.

  It generates two methods "forward_async" and "forward" based on
  the signature of the "forward" method of "module_cls".
  "forward_async" runs asynchronously and returns a Future. The
  arguments of "forward_async" and "forward" are the same as the
"forward" method of the module returned by the "module_cls".
  For example, if "module_cls" returns an instance of "nn.Linear",
  that has "forward" method signature: "def forward(input: Tensor)
  -> Tensor:", the generated "RemoteModule" will have 2 methods
  with the signatures:

     "def forward(input: Tensor) -> Tensor:"
     "def forward_async(input: Tensor) -> Future[Tensor]:"

Parameters:
      * remote_device (str) -- Device on the destination worker
        where we'd like to place this module. The format should be
        "/", where the device field can be parsed
        as torch.device type. E.g., "trainer0/cpu", "trainer0",
        "ps0/cuda:0". In addition, the device field can be optional
        and the default value is "cpu".
  * **module_cls** (*nn.Module*) --

    Class for the module to be created remotely. For example,

    >>> class MyModule(nn.Module):
    >>>     def forward(input):
def forward(input):
    >>>         return input + 1
    >>>
    >>> module_cls = MyModule




  * **args** (*Sequence**, **optional*) -- args to be passed to
    "module_cls".

  * **kwargs** (*Dict**, **optional*) -- kwargs to be passed to
    "module_cls".

Returns:
      A remote module instance which wraps the "Module" created by the
      user-provided "module_cls", it has a blocking "forward" method
      and an asynchronous "forward_async" method that returns a future
      of the "forward" call on the user-provided module on the remote
      side.
Example::
      Run the following code in two different processes:
  >>> # On worker 0:
  >>> import torch
  >>> import torch.distributed.rpc as rpc
  >>> from torch import nn, Tensor
  >>> from torch.distributed.nn.api.remote_module import RemoteModule
  >>>
  >>> rpc.init_rpc("worker0", rank=0, world_size=2)
  >>> remote_linear_module = RemoteModule(
remote_linear_module = RemoteModule(
      >>>     "worker1/cpu", nn.Linear, args=(20, 30),
      >>> )
      >>> input = torch.randn(128, 20)
      >>> ret_fut = remote_linear_module.forward_async(input)
      >>> ret = ret_fut.wait()
      >>> rpc.shutdown()



  >>> # On worker 1:
  >>> import torch
  >>> import torch.distributed.rpc as rpc
  >>>
  >>> rpc.init_rpc("worker1", rank=1, world_size=2)
  >>> rpc.shutdown()

  Furthermore, a more practical example that is combined with
  DistributedDataParallel (DDP) can be found in this tutorial.

get_module_rref()
  Returns an "RRef" ("RRef[nn.Module]") pointing to the remote
  module.

  Return type:
     *RRef*[*Module*]

remote_parameters(recurse=True)
  Returns a list of "RRef" pointing to the remote module's
  parameters. This can typically be used in conjuction with
  "DistributedOptimizer".

  Parameters:
     **recurse** (*bool*) -- if True, then returns parameters of
the remote module and all submodules of the remote module.
         Otherwise, returns only parameters that are direct members of
         the remote module.
  Returns:
     A list of "RRef" ("List[RRef[nn.Parameter]]") to remote
     module's parameters.

  Return type:
     *List*[*RRef*[*Parameter*]]

Distributed Autograd Framework
Warning:
Distributed autograd is not currently supported when using CUDA
  tensors
This module provides an RPC-based distributed autograd framework that
can be used for applications such as model parallel training. In
short, applications may send and receive gradient recording tensors
over RPC. In the forward pass, we record when gradient recording
tensors are sent over RPC and during the backward pass we use this
information to perform a distributed backward pass using RPC. For more
details see Distributed Autograd Design.
details see Distributed Autograd Design.
torch.distributed.autograd.backward(context_id: int, roots: List[Tensor], retain_graph=False) -> None
Kicks off the distributed backward pass using the provided roots.
   This currently implements the FAST mode algorithm which assumes all
   RPC messages sent in the same distributed autograd context across
   workers would be part of the autograd graph during the backward
   pass.
We use the provided roots to discover the autograd graph and
   compute appropriate dependencies. This method blocks until the
   entire autograd computation is done.
We accumulate the gradients in the appropriate
   "torch.distributed.autograd.context" on each of the nodes. The
   autograd context to be used is looked up given the "context_id"
   that is passed in when "torch.distributed.autograd.backward()" is
   called. If there is no valid autograd context corresponding to the
   given ID, we throw an error. You can retrieve the accumulated
gradients using the "get_gradients()" API.
Parameters:
      * context_id (int) -- The autograd context id for which we
        should retrieve the gradients.
  * **roots** (*list*) -- Tensors which represent the roots of the
    autograd computation. All the tensors should be scalars.

  * **retain_graph** (*bool**, **optional*) -- If False, the graph
    used to compute the grad will be freed. Note that in nearly
    all cases setting this option to True is not needed and often
    can be worked around in a much more efficient way. Usually,
    you need to set this to True to run backward multiple times.

Example::
      >>> import torch.distributed.autograd as dist_autograd
      >>> with dist_autograd.context() as context_id:
      >>>     pred = model.forward()
      >>>     loss = loss_func(pred, loss)
      >>>     dist_autograd.backward(context_id, loss)
class torch.distributed.autograd.context
class torch.distributed.autograd.context
Context object to wrap forward and backward passes when using
   distributed autograd. The "context_id" generated in the "with"
   statement  is required to uniquely identify a distributed backward
   pass on all workers. Each worker stores metadata associated with
   this "context_id", which is required to correctly execute a
   distributed autograd pass.
Example::
      >>> import torch.distributed.autograd as dist_autograd
      >>> with dist_autograd.context() as context_id:
      >>>     t1 = torch.rand((3, 3), requires_grad=True)
      >>>     t2 = torch.rand((3, 3), requires_grad=True)
      >>>     loss = rpc.rpc_sync("worker1", torch.add, args=(t1, t2)).sum()
      >>>     dist_autograd.backward(context_id, [loss])
torch.distributed.autograd.get_gradients(context_id: int) -> Dict[Tensor, Tensor]
Retrieves a map from Tensor to the appropriate gradient for that
   Tensor accumulated in the provided context corresponding to the
given "context_id" as part of the distributed autograd backward
   pass.
Parameters:
      context_id (int) -- The autograd context id for which we
      should retrieve the gradients.
Returns:
      A map where the key is the Tensor and the value is the
      associated gradient for that Tensor.
Example::
      >>> import torch.distributed.autograd as dist_autograd
      >>> with dist_autograd.context() as context_id:
      >>>     t1 = torch.rand((3, 3), requires_grad=True)
      >>>     t2 = torch.rand((3, 3), requires_grad=True)
      >>>     loss = t1 + t2
      >>>     dist_autograd.backward(context_id, [loss.sum()])
      >>>     grads = dist_autograd.get_gradients(context_id)
      >>>     print(grads[t1])
      >>>     print(grads[t2])
More Information about RPC Autograd
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


Distributed Autograd Design


Background


Autograd recording during the forward pass


Distributed Autograd Context


Distributed Backward Pass
Distributed Backward Pass


Computing dependencies


FAST mode algorithm


SMART mode algorithm




Distributed Optimizer


Simple end to end example


Distributed Optimizer
See the torch.distributed.optim page for documentation on distributed
optimizers.
Design Notes
The distributed autograd design note covers the design of the RPC-
based distributed autograd framework that is useful for applications
such as model parallel training.

Distributed Autograd Design

The RRef design note covers the design of the RRef (Remote REFerence)
protocol used to refer to values on remote workers by the framework.

Remote Reference Protocol

Tutorials
The RPC tutorials introduce users to the RPC framework, provide
several example applications using torch.distributed.rpc APIs, and
demonstrate how to use the profiler to profile RPC-based workloads.

Getting started with Distributed RPC Framework
torch.special
The torch.special module, modeled after SciPy's special module.
Functions
torch.special.airy_ai(input, *, out=None) -> Tensor
Airy function \text{Ai}\left(\text{input}\right).
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
torch.special.bessel_j0(input, *, out=None) -> Tensor
Bessel function of the first kind of order 0.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
torch.special.bessel_j1(input, *, out=None) -> Tensor
Bessel function of the first kind of order 1.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
torch.special.digamma(input, *, out=None) -> Tensor
Computes the logarithmic derivative of the gamma function on
   input.
input.
  \digamma(x) = \frac{d}{dx} \ln\left(\Gamma\left(x\right)\right)
  = \frac{\Gamma'(x)}{\Gamma(x)}

Parameters:
      input (Tensor) -- the tensor to compute the digamma
      function on
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Note:
 This function is similar to SciPy's *scipy.special.digamma*.

Note:
 From PyTorch 1.8 onwards, the digamma function returns *-Inf* for
 *0*. Previously it returned *NaN* for *0*.

Example:
  >>> a = torch.tensor([1, 0.5])
  >>> torch.special.digamma(a)
  tensor([-0.5772, -1.9635])

torch.special.entr(input, *, out=None) -> Tensor
Computes the entropy on "input" (as defined below), elementwise.
  \begin{align} \text{entr(x)} = \begin{cases}     -x * \ln(x)  &
  x > 0 \\     0 &  x = 0.0 \\     -\infty & x < 0 \end{cases}
  \end{align}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example::
      >>> a = torch.arange(-0.5, 1, 0.5)
      >>> a
      tensor([-0.5000,  0.0000,  0.5000])
      >>> torch.special.entr(a)
      tensor([  -inf, 0.0000, 0.3466])
torch.special.erf(input, *, out=None) -> Tensor
Computes the error function of "input". The error function is
   defined as follows:
  \mathrm{erf}(x) = \frac{2}{\sqrt{\pi}} \int_{0}^{x} e^{-t^2} dt

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.special.erf(torch.tensor([0, -1., 10.]))
  tensor([ 0.0000, -0.8427,  1.0000])

torch.special.erfc(input, *, out=None) -> Tensor
Computes the complementary error function of "input". The
   complementary error function is defined as follows:
  \mathrm{erfc}(x) = 1 - \frac{2}{\sqrt{\pi}} \int_{0}^{x}
  e^{-t^2} dt

Parameters:
e^{-t^2} dt
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.special.erfc(torch.tensor([0, -1., 10.]))
  tensor([ 1.0000, 1.8427,  0.0000])

torch.special.erfcx(input, *, out=None) -> Tensor
Computes the scaled complementary error function for each element
   of "input". The scaled complementary error function is defined as
   follows:
  \mathrm{erfcx}(x) = e^{x^2} \mathrm{erfc}(x)

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.special.erfcx(torch.tensor([0, -1., 10.]))
  tensor([ 1.0000, 5.0090, 0.0561])

torch.special.erfinv(input, *, out=None) -> Tensor
Computes the inverse error function of "input". The inverse error
   function is defined in the range (-1, 1) as:
  \mathrm{erfinv}(\mathrm{erf}(x)) = x

Parameters:
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.special.erfinv(torch.tensor([0, 0.5, -1.]))
  tensor([ 0.0000,  0.4769,    -inf])

torch.special.exp2(input, *, out=None) -> Tensor
Computes the base two exponential function of "input".
  y_{i} = 2^{x_{i}}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.special.exp2(torch.tensor([0, math.log2(2.), 3, 4]))
  tensor([ 1.,  2.,  8., 16.])

torch.special.expit(input, *, out=None) -> Tensor
Computes the expit (also known as the logistic sigmoid function) of
   the elements of "input".
  \text{out}_{i} = \frac{1}{1 + e^{-\text{input}_{i}}}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> t = torch.randn(4)
  >>> t
  tensor([ 0.9213,  1.0887, -0.8858, -1.7683])
  >>> torch.special.expit(t)
  tensor([ 0.7153,  0.7481,  0.2920,  0.1458])

torch.special.expm1(input, *, out=None) -> Tensor
Computes the exponential of the elements minus 1 of "input".
  y_{i} = e^{x_{i}} - 1

Note:
 This function provides greater precision than exp(x) - 1 for
 small values of x.

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.special.expm1(torch.tensor([0, math.log(2.)]))
  tensor([ 0.,  1.])

torch.special.gammainc(input, other, *, out=None) -> Tensor
Computes the regularized lower incomplete gamma function:
  \text{out}_{i} = \frac{1}{\Gamma(\text{input}_i)}
  \int_0^{\text{other}_i} t^{\text{input}_i-1} e^{-t} dt

where both \text{input}_i and \text{other}_i are weakly positive
and at least one is strictly positive. If both are zero or either
   is negative then \text{out}_i=\text{nan}. \Gamma(\cdot) in the
   equation above is the gamma function,
  \Gamma(\text{input}_i) = \int_0^\infty t^{(\text{input}_i-1)}
  e^{-t} dt.

See "torch.special.gammaincc()" and "torch.special.gammaln()" for
   related functions.
Supports broadcasting to a common shape and float inputs.
Note:
 The backward pass with respect to "input" is not yet supported.
 Please open an issue on PyTorch's Github to request it.

Parameters:
      * input (Tensor) -- the first non-negative input tensor
  * **other** (*Tensor*) -- the second non-negative input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a1 = torch.tensor([4.0])
  >>> a2 = torch.tensor([3.0, 4.0, 5.0])
  >>> a = torch.special.gammaincc(a1, a2)
  tensor([0.3528, 0.5665, 0.7350])
  tensor([0.3528, 0.5665, 0.7350])
tensor([0.3528, 0.5665, 0.7350])
      >>> b = torch.special.gammainc(a1, a2) + torch.special.gammaincc(a1, a2)
      tensor([1., 1., 1.])
torch.special.gammaincc(input, other, *, out=None) -> Tensor
Computes the regularized upper incomplete gamma function:
  \text{out}_{i} = \frac{1}{\Gamma(\text{input}_i)}
  \int_{\text{other}_i}^{\infty} t^{\text{input}_i-1} e^{-t} dt

where both \text{input}_i and \text{other}_i are weakly positive
   and at least one is strictly positive. If both are zero or either
   is negative then \text{out}_i=\text{nan}. \Gamma(\cdot) in the
   equation above is the gamma function,
  \Gamma(\text{input}_i) = \int_0^\infty t^{(\text{input}_i-1)}
  e^{-t} dt.

See "torch.special.gammainc()" and "torch.special.gammaln()" for
   related functions.
Supports broadcasting to a common shape and float inputs.
Note:
 The backward pass with respect to "input" is not yet supported.
 Please open an issue on PyTorch's Github to request it.
Parameters:
      * input (Tensor) -- the first non-negative input tensor
  * **other** (*Tensor*) -- the second non-negative input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a1 = torch.tensor([4.0])
  >>> a2 = torch.tensor([3.0, 4.0, 5.0])
  >>> a = torch.special.gammaincc(a1, a2)
  tensor([0.6472, 0.4335, 0.2650])
  >>> b = torch.special.gammainc(a1, a2) + torch.special.gammaincc(a1, a2)
  tensor([1., 1., 1.])

torch.special.gammaln(input, *, out=None) -> Tensor
Computes the natural logarithm of the absolute value of the gamma
   function on "input".
  \text{out}_{i} = \ln \Gamma(|\text{input}_{i}|)

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.arange(0.5, 2, 0.5)
  >>> torch.special.gammaln(a)
  tensor([ 0.5724,  0.0000, -0.1208])
tensor([ 0.5724,  0.0000, -0.1208])
torch.special.i0(input, *, out=None) -> Tensor
Computes the zeroth order modified Bessel function of the first
   kind for each element of "input".
  \text{out}_{i} = I_0(\text{input}_{i}) = \sum_{k=0}^{\infty}
  \frac{(\text{input}_{i}^2/4)^k}{(k!)^2}

Parameters:
      input (Tensor) -- the input tensor
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.i0(torch.arange(5, dtype=torch.float32))
  tensor([ 1.0000,  1.2661,  2.2796,  4.8808, 11.3019])

torch.special.i0e(input, *, out=None) -> Tensor
Computes the exponentially scaled zeroth order modified Bessel
   function of the first kind (as defined below) for each element of
   "input".
  \text{out}_{i} = \exp(-|x|) * i0(x) = \exp(-|x|) *
  \sum_{k=0}^{\infty} \frac{(\text{input}_{i}^2/4)^k}{(k!)^2}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example::
      >>> torch.special.i0e(torch.arange(5, dtype=torch.float32))
      tensor([1.0000, 0.4658, 0.3085, 0.2430, 0.2070])
torch.special.i1(input, *, out=None) -> Tensor
Computes the first order modified Bessel function of the first kind
   (as defined below) for each element of "input".
  \text{out}_{i} = \frac{(\text{input}_{i})}{2} *
  \sum_{k=0}^{\infty} \frac{(\text{input}_{i}^2/4)^k}{(k!) *
  (k+1)!}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example::
      >>> torch.special.i1(torch.arange(5, dtype=torch.float32))
      tensor([0.0000, 0.5652, 1.5906, 3.9534, 9.7595])
torch.special.i1e(input, *, out=None) -> Tensor
Computes the exponentially scaled first order modified Bessel
   function of the first kind (as defined below) for each element of
   "input".
"input".
  \text{out}_{i} = \exp(-|x|) * i1(x) =     \exp(-|x|) *
  \frac{(\text{input}_{i})}{2} * \sum_{k=0}^{\infty}
  \frac{(\text{input}_{i}^2/4)^k}{(k!) * (k+1)!}

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example::
      >>> torch.special.i1e(torch.arange(5, dtype=torch.float32))
      tensor([0.0000, 0.2079, 0.2153, 0.1968, 0.1788])
torch.special.log1p(input, *, out=None) -> Tensor
Alias for "torch.log1p()".
torch.special.log_ndtr(input, *, out=None) -> Tensor
Computes the log of the area under the standard Gaussian
   probability density function, integrated from minus infinity to
   "input", elementwise.
  \text{log\_ndtr}(x) = \log\left(\frac{1}{\sqrt{2
  \pi}}\int_{-\infty}^{x} e^{-\frac{1}{2}t^2} dt \right)

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example::
      >>> torch.special.log_ndtr(torch.tensor([-3., -2, -1, 0, 1, 2, 3]))
      tensor([-6.6077 -3.7832 -1.841  -0.6931 -0.1728 -0.023  -0.0014])
torch.special.log_softmax(input, dim, *, dtype=None) -> Tensor
Computes softmax followed by a logarithm.
While mathematically equivalent to log(softmax(x)), doing these two
   operations separately is slower and numerically unstable. This
   function is computed as:
  \text{log\_softmax}(x_{i}) = \log\left(\frac{\exp(x_i) }{ \sum_j
  \exp(x_j)} \right)

Parameters:
      * input (Tensor) -- input
  * **dim** (*int*) -- A dimension along which log_softmax will be
    computed.

  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. If specified, the input tensor is cast to
    "dtype" before the operation is performed. This is useful for
    preventing data type overflows. Default: None.

Example::
      >>> t = torch.ones(2, 2)
Example::
      >>> t = torch.ones(2, 2)
      >>> torch.special.log_softmax(t, 0)
      tensor([[-0.6931, -0.6931],
              [-0.6931, -0.6931]])
torch.special.logit(input, eps=None, *, out=None) -> Tensor
Returns a new tensor with the logit of the elements of "input".
   "input" is clamped to [eps, 1 - eps] when eps is not None. When eps
   is None and "input" < 0 or "input" > 1, the function will yields
   NaN.
  \begin{align} y_{i} &= \ln(\frac{z_{i}}{1 - z_{i}}) \\ z_{i} &=
  \begin{cases}     x_{i} & \text{if eps is None} \\
  \text{eps} & \text{if } x_{i} < \text{eps} \\     x_{i} &
  \text{if } \text{eps} \leq x_{i} \leq 1 - \text{eps} \\     1 -
  \text{eps} & \text{if } x_{i} > 1 - \text{eps} \end{cases}
  \end{align}

Parameters:
      * input (Tensor) -- the input tensor.
  * **eps** (*float**, **optional*) -- the epsilon for input clamp
    bound. Default: "None"

Keyword Arguments:
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.rand(5)
  >>> a
  tensor([0.2796, 0.9331, 0.6486, 0.1523, 0.6516])
  >>> torch.special.logit(a, eps=1e-6)
  tensor([-0.9466,  2.6352,  0.6131, -1.7169,  0.6261])

torch.special.logsumexp(input, dim, keepdim=False, *, out=None)
Alias for "torch.logsumexp()".
torch.special.multigammaln(input, p, *, out=None) -> Tensor
Computes the multivariate log-gamma function with dimension p
   element-wise, given by
  \log(\Gamma_{p}(a)) = C + \displaystyle \sum_{i=1}^{p}
  \log\left(\Gamma\left(a - \frac{i - 1}{2}\right)\right)

where C = \log(\pi) \cdot \frac{p (p - 1)}{4} and \Gamma(-) is the
   Gamma function.
All elements must be greater than \frac{p - 1}{2}, otherwise the
   behavior is undefiend.
Parameters:
      * input (Tensor) -- the tensor to compute the multivariate
        log-gamma function
  * **p** (*int*) -- the number of dimensions
p (int) -- the number of dimensions

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.empty(2, 3).uniform_(1, 2)
  >>> a
  tensor([[1.6835, 1.8474, 1.1929],
          [1.0475, 1.7162, 1.4180]])
  >>> torch.special.multigammaln(a, 2)
  tensor([[0.3928, 0.4007, 0.7586],
          [1.0311, 0.3901, 0.5049]])

torch.special.ndtr(input, *, out=None) -> Tensor
Computes the area under the standard Gaussian probability density
   function, integrated from minus infinity to "input", elementwise.
  \text{ndtr}(x) = \frac{1}{\sqrt{2 \pi}}\int_{-\infty}^{x}
  e^{-\frac{1}{2}t^2} dt

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example::
      >>> torch.special.ndtr(torch.tensor([-3., -2, -1, 0, 1, 2, 3]))
      tensor([0.0013, 0.0228, 0.1587, 0.5000, 0.8413, 0.9772, 0.9987])