shrinath-suresh/blogs-docs-splitted huggingface dataset:
to which to take diagonal. Default: 0.
  * **dim2** (*int**, **optional*) -- second dimension with
    respect to which to take diagonal. Default: 1.

Note:
 To take a batch diagonal, pass in dim1=-2, dim2=-1.

Examples:
  >>> a = torch.randn(3, 3)
  >>> a
  tensor([[-1.0854,  1.1431, -0.1752],
          [ 0.8536, -0.0905,  0.0360],
          [ 0.6927, -0.3735, -0.4945]])


  >>> torch.diagonal(a, 0)
  tensor([-1.0854, -0.0905, -0.4945])


  >>> torch.diagonal(a, 1)
  tensor([ 1.1431,  0.0360])


  >>> x = torch.randn(2, 5, 4, 2)
  >>> torch.diagonal(x, offset=-1, dim1=1, dim2=2)
  tensor([[[-1.2631,  0.3755, -1.5977, -1.8172],
           [-1.1065,  1.0401, -0.2235, -0.7938]],

          [[-1.7325, -0.3081,  0.6166,  0.2335],
           [ 1.0500,  0.7336, -0.3836, -1.1015]]])
MultiLabelMarginLoss
class torch.nn.MultiLabelMarginLoss(size_average=None, reduce=None, reduction='mean')
Creates a criterion that optimizes a multi-class multi-
   classification hinge loss (margin-based loss) between input x (a 2D
   mini-batch Tensor) and output y (which is a 2D Tensor of target
   class indices). For each sample in the mini-batch:
  \text{loss}(x, y) = \sum_{ij}\frac{\max(0, 1 - (x[y[j]] -
  x[i]))}{\text{x.size}(0)}

where x \in \left{0, \; \cdots , \; \text{x.size}(0) - 1\right},
   y \in \left{0, \; \cdots , \; \text{y.size}(0) - 1\right}, 0 \leq
   y[j] \leq \text{x.size}(0)-1, and i \neq y[j] for all i and j.
y and x must have the same size.
The criterion only considers a contiguous block of non-negative
   targets that starts at the front.
This allows for different samples to have variable amounts of
   target classes.
Parameters:
      * size_average (bool, optional) -- Deprecated (see
"size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
Shape:
      * Input: (C) or (N, C) where N is the batch size and C is
        the number of classes.
  * Target: (C) or (N, C), label targets padded by -1 ensuring
    same shape as the input.

  * Output: scalar. If "reduction" is "'none'", then (N).

Examples:
  >>> loss = nn.MultiLabelMarginLoss()
  >>> x = torch.FloatTensor([[0.1, 0.2, 0.4, 0.8]])
  >>> # for target y, only consider labels 3 and 0, not after label -1
  >>> y = torch.LongTensor([[3, 0, -1, 1]])
  >>> # 0.25 * ((1-(0.1-0.2)) + (1-(0.1-0.4)) + (1-(0.8-0.2)) + (1-(0.8-0.4)))
  >>> loss(x, y)
  tensor(0.85...)
BatchNorm3d
class torch.nn.BatchNorm3d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, device=None, dtype=None)
Applies Batch Normalization over a 5D input (a mini-batch of 3D
   inputs with additional channel dimension) as described in the paper
   Batch Normalization: Accelerating Deep Network Training by Reducing
   Internal Covariate Shift .
  y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}}
  * \gamma + \beta

The mean and standard-deviation are calculated per-dimension over
   the mini-batches and \gamma and \beta are learnable parameter
   vectors of size C (where C is the input size). By default, the
   elements of \gamma are set to 1 and the elements of \beta are set
   to 0. The standard-deviation is calculated via the biased
   estimator, equivalent to torch.var(input, unbiased=False).
Also by default, during training this layer keeps running estimates
   of its computed mean and variance, which are then used for
normalization during evaluation. The running estimates are kept
   with a default "momentum" of 0.1.
If "track_running_stats" is set to "False", this layer then does
   not keep running estimates, and batch statistics are instead used
   during evaluation time as well.
Note:
 This "momentum" argument is different from one used in optimizer
 classes and the conventional notion of momentum. Mathematically,
 the update rule for running statistics here is \hat{x}_\text{new}
 = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times
 x_t, where \hat{x} is the estimated statistic and x_t is the new
 observed value.

Because the Batch Normalization is done over the C dimension,
   computing statistics on (N, D, H, W) slices, it's common
   terminology to call this Volumetric Batch Normalization or Spatio-
   temporal Batch Normalization.
Parameters:
      * num_features (int) -- C from an expected input of size
        (N, C, D, H, W)
(N, C, D, H, W)
  * **eps** (*float*) -- a value added to the denominator for
    numerical stability. Default: 1e-5

  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Can be set to "None" for
    cumulative moving average (i.e. simple average). Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters. Default:
    "True"

  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and
    variance, and when set to "False", this module does not track
    such statistics, and initializes statistics buffers
    "running_mean" and "running_var" as "None". When these buffers
    are "None", this module always uses batch statistics. in both
    training and eval modes. Default: "True"

Shape:
      * Input: (N, C, D, H, W)
Shape:
      * Input: (N, C, D, H, W)
  * Output: (N, C, D, H, W) (same shape as input)

Examples:
  >>> # With Learnable Parameters
  >>> m = nn.BatchNorm3d(100)
  >>> # Without Learnable Parameters
  >>> m = nn.BatchNorm3d(100, affine=False)
  >>> input = torch.randn(20, 100, 35, 45, 10)
  >>> output = m(input)
Softshrink
class torch.nn.Softshrink(lambd=0.5)
Applies the soft shrinkage function elementwise:
  \text{SoftShrinkage}(x) = \begin{cases} x - \lambda, & \text{ if
  } x > \lambda \\ x + \lambda, & \text{ if } x < -\lambda \\ 0, &
  \text{ otherwise } \end{cases}

Parameters:
      lambd (float) -- the \lambda (must be no less than zero)
      value for the Softshrink formulation. Default: 0.5
Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

[image]
Examples:
  >>> m = nn.Softshrink()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.cross
torch.cross(input, other, dim=None, *, out=None) -> Tensor
Returns the cross product of vectors in dimension "dim" of "input"
   and "other".
Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of vectors, for which it computes the product
   along the dimension "dim". In this case, the output has the same
   batch dimensions as the inputs.
If "dim" is not given, it defaults to the first dimension found
   with the size 3. Note that this might be unexpected.
See also:
 "torch.linalg.cross()" which requires specifying dim (defaulting
 to -1).

Warning:
 This function may change in a future PyTorch release to match the
 default behaviour in "torch.linalg.cross()". We recommend using
 "torch.linalg.cross()".

Parameters:
      * input (Tensor) -- the input tensor.
  * **other** (*Tensor*) -- the second input tensor

  * **dim** (*int**, **optional*) -- the dimension to take the
cross-product in.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4, 3)
  >>> a
  tensor([[-0.3956,  1.1455,  1.6895],
          [-0.5849,  1.3672,  0.3599],
          [-1.1626,  0.7180, -0.0521],
          [-0.1339,  0.9902, -2.0225]])
  >>> b = torch.randn(4, 3)
  >>> b
  tensor([[-0.0257, -1.4725, -1.2251],
          [-1.1479, -0.7005, -1.9757],
          [-1.3904,  0.3726, -1.1836],
          [-0.9688, -0.7153,  0.2159]])
  >>> torch.cross(a, b, dim=1)
  tensor([[ 1.0844, -0.5281,  0.6120],
          [-2.4490, -1.5687,  1.9792],
          [-0.8304, -1.3037,  0.5650],
          [-1.2329,  1.9883,  1.0551]])
  >>> torch.cross(a, b)
  tensor([[ 1.0844, -0.5281,  0.6120],
          [-2.4490, -1.5687,  1.9792],
          [-0.8304, -1.3037,  0.5650],
          [-1.2329,  1.9883,  1.0551]])
torch.dequantize
torch.dequantize(tensor) -> Tensor
Returns an fp32 Tensor by dequantizing a quantized Tensor
Parameters:
      tensor (Tensor) -- A quantized Tensor
torch.dequantize(tensors) -> sequence of Tensors
Given a list of quantized Tensors, dequantize them and return a
   list of fp32 Tensors
Parameters:
      tensors (sequence of Tensors) -- A list of quantized
      Tensors
LinearReLU
class torch.ao.nn.intrinsic.quantized.LinearReLU(in_features, out_features, bias=True, dtype=torch.qint8)
A LinearReLU module fused from Linear and ReLU modules
We adopt the same interface as "torch.ao.nn.quantized.Linear".
Variables:
      torch.ao.nn.quantized.Linear (Same as) --
Examples:
  >>> m = nn.intrinsic.LinearReLU(20, 30)
  >>> input = torch.randn(128, 20)
  >>> output = m(input)
  >>> print(output.size())
  torch.Size([128, 30])
FakeQuantizeBase
class torch.quantization.fake_quantize.FakeQuantizeBase
Base fake quantize module Any fake quantize implementation should
   derive from this class.
Concrete fake quantize module should follow the same API. In
   forward, they will update the statistics of the observed Tensor and
   fake quantize the input. They should also provide a
   calculate_qparams function that computes the quantization
   parameters given the collected statistics.
torch.optim.Optimizer.add_param_group
Optimizer.add_param_group(param_group)
Add a param group to the "Optimizer" s param_groups.
This can be useful when fine tuning a pre-trained network as frozen
   layers can be made trainable and added to the "Optimizer" as
   training progresses.
Parameters:
      param_group (dict) -- Specifies what Tensors should be
      optimized along with group specific optimization options.
torch.Tensor.new_ones
Tensor.new_ones(size, *, dtype=None, device=None, requires_grad=False, layout=torch.strided, pin_memory=False) -> Tensor
Returns a Tensor of size "size" filled with "1". By default, the
   returned Tensor has the same "torch.dtype" and "torch.device" as
   this tensor.
Parameters:
      size (int...) -- a list, tuple, or "torch.Size" of
      integers defining the shape of the output tensor.
Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired type of
        returned tensor. Default: if None, same "torch.dtype" as this
        tensor.
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if None, same "torch.device" as this
    tensor.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **layout** ("torch.layout", optional) -- the desired layout of
returned Tensor. Default: "torch.strided".
  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> tensor = torch.tensor((), dtype=torch.int32)
  >>> tensor.new_ones((2, 3))
  tensor([[ 1,  1,  1],
          [ 1,  1,  1]], dtype=torch.int32)
AdaptiveMaxPool3d
class torch.nn.AdaptiveMaxPool3d(output_size, return_indices=False)
Applies a 3D adaptive max pooling over an input signal composed of
   several input planes.
The output is of size D_{out} \times H_{out} \times W_{out}, for
   any input size. The number of output features is equal to the
   number of input planes.
Parameters:
      * output_size (Union[int, None,
        Tuple[Optional[int], Optional[int],
        Optional[int]]]) -- the target output size of the
        image of the form D_{out} \times H_{out} \times W_{out}. Can
        be a tuple (D_{out}, H_{out}, W_{out}) or a single D_{out} for
        a cube D_{out} \times D_{out} \times D_{out}. D_{out}, H_{out}
        and W_{out} can be either a "int", or "None" which means the
        size will be the same as that of the input.
  * **return_indices** (*bool*) -- if "True", will return the
indices along with the outputs. Useful to pass to
        nn.MaxUnpool3d. Default: "False"
Shape:
      * Input: (N, C, D_{in}, H_{in}, W_{in}) or (C, D_{in}, H_{in},
        W_{in}).
  * Output: (N, C, D_{out}, H_{out}, W_{out}) or (C, D_{out},
    H_{out}, W_{out}), where (D_{out}, H_{out},
    W_{out})=\text{output\_size}.

-[ Examples ]-



target output size of 5x7x9
m = nn.AdaptiveMaxPool3d((5, 7, 9))
input = torch.randn(1, 64, 8, 9, 10)
output = m(input)
target output size of 7x7x7 (cube)
m = nn.AdaptiveMaxPool3d(7)
input = torch.randn(1, 64, 10, 9, 8)
output = m(input)
target output size of 7x9x8
m = nn.AdaptiveMaxPool3d((7, None, None))
input = torch.randn(1, 64, 10, 9, 8)
output = m(input)
torch.optim.Optimizer.zero_grad
Optimizer.zero_grad(set_to_none=False)
Sets the gradients of all optimized "torch.Tensor" s to zero.
Parameters:
      set_to_none (bool) -- instead of setting to zero, set the
      grads to None. This will in general have lower memory footprint,
      and can modestly improve performance. However, it changes
      certain behaviors. For example: 1. When the user tries to access
      a gradient and perform manual ops on it, a None attribute or a
      Tensor full of 0s will behave differently. 2. If the user
      requests "zero_grad(set_to_none=True)" followed by a backward
      pass, ".grad"s are guaranteed to be None for params that did not
      receive a gradient. 3. "torch.optim" optimizers have a different
      behavior if the gradient is 0 or None (in one case it does the
      step with a gradient of 0 and in the other it skips the step
      altogether).
torch.nn.modules.module.register_module_backward_hook
torch.nn.modules.module.register_module_backward_hook(hook)
Registers a backward hook common to all the modules.
This function is deprecated in favor of
   "torch.nn.modules.module.register_module_full_backward_hook()" and
   the behavior of this function will change in future versions.
Returns:
      a handle that can be used to remove the added hook by calling
      "handle.remove()"
Return type:
      "torch.utils.hooks.RemovableHandle"
ConstantPad3d
class torch.nn.ConstantPad3d(padding, value)
Pads the input tensor boundaries with a constant value.
For N-dimensional padding, use "torch.nn.functional.pad()".
Parameters:
      padding (int, tuple) -- the size of the padding. If is
      int, uses the same padding in all boundaries. If a 6-tuple,
      uses (\text{padding_left}, \text{padding_right},
      \text{padding_top}, \text{padding_bottom},
      \text{padding_front}, \text{padding_back})
Shape:
      * Input: (N, C, D_{in}, H_{in}, W_{in}) or (C, D_{in}, H_{in},
        W_{in}).
  * Output: (N, C, D_{out}, H_{out}, W_{out}) or (C, D_{out},
    H_{out}, W_{out}), where

    D_{out} = D_{in} + \text{padding\_front} +
    \text{padding\_back}

    H_{out} = H_{in} + \text{padding\_top} +
    \text{padding\_bottom}

    W_{out} = W_{in} + \text{padding\_left} +
    \text{padding\_right}

Examples:
  >>> m = nn.ConstantPad3d(3, 3.5)
m = nn.ConstantPad3d(3, 3.5)
      >>> input = torch.randn(16, 3, 10, 20, 30)
      >>> output = m(input)
      >>> # using different paddings for different sides
      >>> m = nn.ConstantPad3d((3, 3, 6, 6, 0, 1), 3.5)
      >>> output = m(input)
torch.nn.utils.weight_norm
torch.nn.utils.weight_norm(module, name='weight', dim=0)
Applies weight normalization to a parameter in the given module.
  \mathbf{w} = g \dfrac{\mathbf{v}}{\|\mathbf{v}\|}

Weight normalization is a reparameterization that decouples the
   magnitude of a weight tensor from its direction. This replaces the
   parameter specified by "name" (e.g. "'weight'") with two
   parameters: one specifying the magnitude (e.g. "'weight_g'") and
   one specifying the direction (e.g. "'weight_v'"). Weight
   normalization is implemented via a hook that recomputes the weight
   tensor from the magnitude and direction before every "forward()"
   call.
By default, with "dim=0", the norm is computed independently per
   output channel/plane. To compute a norm over the entire weight
   tensor, use "dim=None".
See https://arxiv.org/abs/1602.07868
Parameters:
      * module (Module) -- containing module
name (str, optional) -- name of weight parameter

dim (int, optional) -- dimension over which to
    compute the norm



Returns:
      The original module with the weight norm hook
Return type:
      T_module
Example:
  >>> m = weight_norm(nn.Linear(20, 40), name='weight')
  >>> m
  Linear(in_features=20, out_features=40, bias=True)
  >>> m.weight_g.size()
  torch.Size([40, 1])
  >>> m.weight_v.size()
  torch.Size([40, 20])
torch.cuda.make_graphed_callables
torch.cuda.make_graphed_callables(callables, sample_args, num_warmup_iters=3, allow_unused_input=False)
Accepts callables (functions or "nn.Module"s) and returns graphed
   versions.
Each graphed callable's forward pass runs its source callable's
   forward CUDA work as a CUDA graph inside a single autograd node.
The graphed callable's forward pass also appends a backward node to
   the autograd graph. During backward, this node runs the callable's
   backward work as a CUDA graph.
Therefore, each graphed callable should be a drop-in replacement
   for its source callable in an autograd-enabled training loop.
See Partial-network capture for detailed use and constraints.
If you pass a tuple of several callables, their captures will use
   the same memory pool. See Graph memory management for when this is
   appropriate.
Parameters:
      * callables (torch.nn.Module or Python function*, or
*tuple of these) -- Callable or callables to graph. See
        Graph memory management for when passing a tuple of callables
        is appropriate.  If you pass a tuple of callables, their order
        in the tuple must be the same order they'll run in the live
        workload.
  * **sample_args** (*tuple of Tensors**, or **tuple of tuples of
    Tensors*) -- Samples args for each callable. If a single
    callable was passed, "sample_args" must be a single tuple of
    argument Tensors. If a tuple of callables was passed,
    "sample_args" must be tuple of tuples of argument Tensors.

  * **num_warmup_iters** (*int*) -- The number of warmup
    iterations. Currently, "DataDistributedParallel" needs 11
    iterations for warm up. Default: "3".

  * **allow_unused_input** (*bool*) -- If False, specifying inputs
    that were not used when computing outputs (and therefore their
    grad is always zero) is an error. Defaults to False.

Note:
Note:
 The "requires_grad" state of each Tensor in "sample_args" must
 match the state that's expected for the corresponding real input
 in the training loop.

Warning:
 This API is in beta and may change in future releases.

Warning:
 "sample_args" for each callable must contain only Tensors. Other
 types are not allowed.

Warning:
 Returned callables do not support higher order differentiation
 (e.g., double backward).

Warning:
 In any "Module" passed to "make_graphed_callables()", only
 parameters may be trainable. Buffers must have
 "requires_grad=False".

Warning:
 After you pass a "torch.nn.Module" through
 "make_graphed_callables()", you may not add or remove any of that
 Module's parameters or buffers.

Warning:
 "torch.nn.Module"s passed to "make_graphed_callables()" must not
 have module hooks registered on them at the time they are passed.
 However, registering hooks on modules *after* passing them
through "make_graphed_callables()" is allowed.
Warning:
 When running a graphed callable, you must pass its arguments in
 the same order and format they appeared in that callable's
 "sample_args".

Warning:
 The automatic mixed precision is supported in
 "make_graphed_callables()" only with disabled caching. The
 context manager *torch.cuda.amp.autocast()* must have
 *cache_enabled=False*.
torch.nn.utils.spectral_norm
torch.nn.utils.spectral_norm(module, name='weight', n_power_iterations=1, eps=1e-12, dim=None)
Applies spectral normalization to a parameter in the given module.
  \mathbf{W}_{SN} = \dfrac{\mathbf{W}}{\sigma(\mathbf{W})},
  \sigma(\mathbf{W}) = \max_{\mathbf{h}: \mathbf{h} \ne 0}
  \dfrac{\|\mathbf{W} \mathbf{h}\|_2}{\|\mathbf{h}\|_2}

Spectral normalization stabilizes the training of discriminators
   (critics) in Generative Adversarial Networks (GANs) by rescaling
   the weight tensor with spectral norm \sigma of the weight matrix
   calculated using power iteration method. If the dimension of the
   weight tensor is greater than 2, it is reshaped to 2D in power
   iteration method to get spectral norm. This is implemented via a
   hook that calculates spectral norm and rescales weight before every
   "forward()" call.
See Spectral Normalization for Generative Adversarial Networks .
Parameters:
Parameters:
      * module (nn.Module) -- containing module
  * **name** (*str**, **optional*) -- name of weight parameter

  * **n_power_iterations** (*int**, **optional*) -- number of
    power iterations to calculate spectral norm

  * **eps** (*float**, **optional*) -- epsilon for numerical
    stability in calculating norms

  * **dim** (*int**, **optional*) -- dimension corresponding to
    number of outputs, the default is "0", except for modules that
    are instances of ConvTranspose{1,2,3}d, when it is "1"

Returns:
      The original module with the spectral norm hook
Return type:
      T_module
Note:
 This function has been reimplemented as
 "torch.nn.utils.parametrizations.spectral_norm()" using the new
 parametrization functionality in
 "torch.nn.utils.parametrize.register_parametrization()". Please
 use the newer version. This function will be deprecated in a
 future version of PyTorch.

Example:
torch.roll
torch.roll(input, shifts, dims=None) -> Tensor
Roll the tensor "input" along the given dimension(s). Elements that
   are shifted beyond the last position are re-introduced at the first
   position. If "dims" is None, the tensor will be flattened before
   rolling and then restored to the original shape.
Parameters:
      * input (Tensor) -- the input tensor.
  * **shifts** (*int** or **tuple of ints*) -- The number of
    places by which the elements of the tensor are shifted. If
    shifts is a tuple, dims must be a tuple of the same size, and
    each dimension will be rolled by the corresponding value

  * **dims** (*int** or **tuple of ints*) -- Axis along which to
    roll

Example:
  >>> x = torch.tensor([1, 2, 3, 4, 5, 6, 7, 8]).view(4, 2)
  >>> x
  tensor([[1, 2],
          [3, 4],
          [5, 6],
          [7, 8]])
  >>> torch.roll(x, 1)
  tensor([[8, 1],
          [2, 3],
          [4, 5],
[2, 3],
              [4, 5],
              [6, 7]])
      >>> torch.roll(x, 1, 0)
      tensor([[7, 8],
              [1, 2],
              [3, 4],
              [5, 6]])
      >>> torch.roll(x, -1, 0)
      tensor([[3, 4],
              [5, 6],
              [7, 8],
              [1, 2]])
      >>> torch.roll(x, shifts=(2, 1), dims=(0, 1))
      tensor([[6, 5],
              [8, 7],
              [2, 1],
              [4, 3]])
torch.Tensor.new_tensor
Tensor.new_tensor(data, *, dtype=None, device=None, requires_grad=False, layout=torch.strided, pin_memory=False) -> Tensor
Returns a new Tensor with "data" as the tensor data. By default,
   the returned Tensor has the same "torch.dtype" and "torch.device"
   as this tensor.
Warning:
 "new_tensor()" always copies "data". If you have a Tensor "data"
 and want to avoid a copy, use "torch.Tensor.requires_grad_()" or
 "torch.Tensor.detach()". If you have a numpy array and want to
 avoid a copy, use "torch.from_numpy()".

Warning:
 When data is a tensor *x*, "new_tensor()" reads out 'the data'
 from whatever it is passed, and constructs a leaf variable.
 Therefore "tensor.new_tensor(x)" is equivalent to
 "x.clone().detach()" and "tensor.new_tensor(x,
 requires_grad=True)" is equivalent to
 "x.clone().detach().requires_grad_(True)". The equivalents using
 "clone()" and "detach()" are recommended.