shrinath-suresh/blogs-docs-splitted huggingface dataset:
use different precision for backward.


"stride" controls the stride for the cross-correlation, a single
     number or a tuple.


"padding" controls the amount of padding applied to the input. It
     can be either a string {'valid', 'same'} or an int / a tuple of
     ints giving the amount of implicit padding applied on both sides.


"dilation" controls the spacing between the kernel points; also
     known as the Ã  trous algorithm. It is harder to describe, but
     this link has a nice visualization of what "dilation" does.


"groups" controls the connections between inputs and outputs.
     "in_channels" and "out_channels" must both be divisible by
     "groups". For example,
* At groups=1, all inputs are convolved to all outputs.

* At groups=2, the operation becomes equivalent to having two
  conv layers side by side, each seeing half the input
  channels and producing half the output channels, and both
  subsequently concatenated.
subsequently concatenated.
    * At groups= "in_channels", each input channel is convolved
      with its own set of filters (of size
      \frac{\text{out\_channels}}{\text{in\_channels}}).

The parameters "kernel_size", "stride", "padding", "dilation" can
   either be:
  * a single "int" -- in which case the same value is used for the
    height and width dimension

  * a "tuple" of two ints -- in which case, the first *int* is
    used for the height dimension, and the second *int* for the
    width dimension

Note:
 When *groups == in_channels* and *out_channels == K *
 in_channels*, where *K* is a positive integer, this operation is
 also known as a "depthwise convolution".In other words, for an
 input of size (N, C_{in}, L_{in}), a depthwise convolution with a
 depthwise multiplier *K* can be performed with the arguments
 (C_\text{in}=C_\text{in}, C_\text{out}=C_\text{in} \times
 \text{K}, ..., \text{groups}=C_\text{in}).
\text{K}, ..., \text{groups}=C_\text{in}).
Note:
 In some circumstances when given tensors on a CUDA device and
 using CuDNN, this operator may select a nondeterministic
 algorithm to increase performance. If this is undesirable, you
 can try to make the operation deterministic (potentially at a
 performance cost) by setting "torch.backends.cudnn.deterministic
 = True". See Reproducibility for more information.

Note:
 "padding='valid'" is the same as no padding. "padding='same'"
 pads the input so the output has the shape as the input. However,
 this mode doesn't support any stride values other than 1.

Note:
 This module supports complex data types i.e. "complex32,
 complex64, complex128".

Parameters:
      * in_channels (int) -- Number of channels in the input
        image
  * **out_channels** (*int*) -- Number of channels produced by the
    convolution

  * **kernel_size** (*int** or **tuple*) -- Size of the convolving
kernel
  * **stride** (*int** or **tuple**, **optional*) -- Stride of the
    convolution. Default: 1

  * **padding** (*int**, **tuple** or **str**, **optional*) --
    Padding added to all four sides of the input. Default: 0

  * **padding_mode** (*str**, **optional*) -- "'zeros'",
    "'reflect'", "'replicate'" or "'circular'". Default: "'zeros'"

  * **dilation** (*int** or **tuple**, **optional*) -- Spacing
    between kernel elements. Default: 1

  * **groups** (*int**, **optional*) -- Number of blocked
    connections from input channels to output channels. Default: 1

  * **bias** (*bool**, **optional*) -- If "True", adds a learnable
    bias to the output. Default: "True"

Shape:
      * Input: (N, C_{in}, H_{in}, W_{in}) or (C_{in}, H_{in}, W_{in})
  * Output: (N, C_{out}, H_{out}, W_{out}) or (C_{out}, H_{out},
    W_{out}), where

       H_{out} = \left\lfloor\frac{H_{in}  + 2 \times
\text{padding}[0] - \text{dilation}[0]           \times
           (\text{kernel_size}[0] - 1) - 1}{\text{stride}[0]} +
           1\right\rfloor
       W_{out} = \left\lfloor\frac{W_{in}  + 2 \times
       \text{padding}[1] - \text{dilation}[1]           \times
       (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} +
       1\right\rfloor

Variables:
      * weight (Tensor) -- the learnable weights of the module
        of shape (\text{out_channels},
        \frac{\text{in_channels}}{\text{groups}},
        \text{kernel_size[0]}, \text{kernel_size[1]}). The values of
        these weights are sampled from \mathcal{U}(-\sqrt{k},
        \sqrt{k}) where k = \frac{groups}{C_\text{in} *
        \prod_{i=0}^{1}\text{kernel_size}[i]}
  * **bias** (*Tensor*) -- the learnable bias of the module of
    shape (out_channels). If "bias" is "True", then the values of
    these weights are sampled from \mathcal{U}(-\sqrt{k},
\sqrt{k}) where k = \frac{groups}{C_\text{in} *
        \prod_{i=0}^{1}\text{kernel_size}[i]}
-[ Examples ]-



With square kernels and equal stride
m = nn.Conv2d(16, 33, 3, stride=2)
non-square kernels and unequal stride and with padding
m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
non-square kernels and unequal stride and with padding and dilation
m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
input = torch.randn(20, 16, 50, 100)
output = m(input)
torch.mode
torch.mode(input, dim=- 1, keepdim=False, *, out=None)
Returns a namedtuple "(values, indices)" where "values" is the mode
   value of each row of the "input" tensor in the given dimension
   "dim", i.e. a value which appears most often in that row, and
   "indices" is the index location of each mode value found.
By default, "dim" is the last dimension of the "input" tensor.
If "keepdim" is "True", the output tensors are of the same size as
   "input" except in the dimension "dim" where they are of size 1.
   Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensors having 1 fewer dimension than "input".
Note:
 This function is not defined for "torch.cuda.Tensor" yet.

Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int*) -- the dimension to reduce.

  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not.

Keyword Arguments:
retained or not.
Keyword Arguments:
      out (tuple, optional) -- the result tuple of two
      output tensors (values, indices)
Example:
  >>> a = torch.randint(10, (5,))
  >>> a
  tensor([6, 5, 1, 0, 2])
  >>> b = a + (torch.randn(50, 1) * 5).long()
  >>> torch.mode(b, 0)
  torch.return_types.mode(values=tensor([6, 5, 1, 0, 2]), indices=tensor([2, 2, 2, 2, 2]))
torch.signal.windows.cosine
torch.signal.windows.cosine(M, *, sym=True, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Computes a window with a simple cosine waveform. Also known as the
   sine window.
The cosine window is defined as follows:
  w_n = \cos{\left(\frac{\pi n}{M} - \frac{\pi}{2}\right)} =
  \sin{\left(\frac{\pi n}{M}\right)}

The window is normalized to 1 (maximum value is 1). However, the 1
   doesn't appear if "M" is even and "sym" is True.
Parameters:
      M (int) -- the length of the window. In other words, the
      number of points of the returned window.
Keyword Arguments:
      * sym (bool, optional) -- If False, returns a
        periodic window suitable for use in spectral analysis. If
        True, returns a symmetric window suitable for use in filter
        design. Default: True.
  * **dtype** ("torch.dtype", optional) -- the desired data type
of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Return type:
      Tensor
Examples:
  >>> # Generates a symmetric cosine window.
  >>> torch.signal.windows.cosine(10)
  tensor([0.1564, 0.4540, 0.7071, 0.8910, 0.9877, 0.9877, 0.8910, 0.7071, 0.4540, 0.1564])

  >>> # Generates a periodic cosine window.
torch.pow
torch.pow(input, exponent, *, out=None) -> Tensor
Takes the power of each element in "input" with "exponent" and
   returns a tensor with the result.
"exponent" can be either a single "float" number or a Tensor with
   the same number of elements as "input".
When "exponent" is a scalar value, the operation applied is:
  \text{out}_i = x_i ^ \text{exponent}

When "exponent" is a tensor, the operation applied is:
  \text{out}_i = x_i ^ {\text{exponent}_i}

When "exponent" is a tensor, the shapes of "input" and "exponent"
   must be broadcastable.
Parameters:
      * input (Tensor) -- the input tensor.
  * **exponent** (*float** or **tensor*) -- the exponent value

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([ 0.4331,  1.2475,  0.6834, -0.2791])
  >>> torch.pow(a, 2)
  tensor([ 0.1875,  1.5561,  0.4670,  0.0779])
tensor([ 0.1875,  1.5561,  0.4670,  0.0779])
      >>> exp = torch.arange(1., 5.)
  >>> a = torch.arange(1., 5.)
  >>> a
  tensor([ 1.,  2.,  3.,  4.])
  >>> exp
  tensor([ 1.,  2.,  3.,  4.])
  >>> torch.pow(a, exp)
  tensor([   1.,    4.,   27.,  256.])

torch.pow(self, exponent, *, out=None) -> Tensor
"self" is a scalar "float" value, and "exponent" is a tensor. The
   returned tensor "out" is of the same shape as "exponent"
The operation applied is:
  \text{out}_i = \text{self} ^ {\text{exponent}_i}

Parameters:
      * self (float) -- the scalar base value for the power
        operation
  * **exponent** (*Tensor*) -- the exponent tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> exp = torch.arange(1., 5.)
  >>> base = 2
  >>> torch.pow(base, exp)
  tensor([  2.,   4.,   8.,  16.])
torch.logsumexp
torch.logsumexp(input, dim, keepdim=False, *, out=None)
Returns the log of summed exponentials of each row of the "input"
   tensor in the given dimension "dim". The computation is numerically
   stabilized.
For summation index j given by dim and other indices i, the
   result is
     \text{logsumexp}(x)_{i} = \log \sum_j \exp(x_{ij})

If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension(s) "dim" where it is of size 1.
   Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensor having 1 (or "len(dim)") fewer dimension(s).
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int** or **tuple of ints**, **optional*) -- the
    dimension or dimensions to reduce. If "None", all dimensions
    are reduced.

  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not.

Keyword Arguments:
retained or not.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(3, 3)
  >>> torch.logsumexp(a, 1)
  tensor([1.4907, 1.0593, 1.5696])
  >>> torch.dist(torch.logsumexp(a, 1), torch.log(torch.sum(torch.exp(a), 1)))
  tensor(1.6859e-07)
torch.cuda.set_rng_state
torch.cuda.set_rng_state(new_state, device='cuda')
Sets the random number generator state of the specified GPU.
Parameters:
      * new_state (torch.ByteTensor) -- The desired state
  * **device** (*torch.device** or **int**, **optional*) -- The
    device to set the RNG state. Default: "'cuda'" (i.e.,
    "torch.device('cuda')", the current CUDA device).
torch.unbind
torch.unbind(input, dim=0) -> seq
Removes a tensor dimension.
Returns a tuple of all slices along a given dimension, already
   without it.
Parameters:
      * input (Tensor) -- the tensor to unbind
  * **dim** (*int*) -- dimension to remove

Example:
  >>> torch.unbind(torch.tensor([[1, 2, 3],
  >>>                            [4, 5, 6],
  >>>                            [7, 8, 9]]))
  (tensor([1, 2, 3]), tensor([4, 5, 6]), tensor([7, 8, 9]))
LnStructured
class torch.nn.utils.prune.LnStructured(amount, n, dim=- 1)
Prune entire (currently unpruned) channels in a tensor based on
   their L"n"-norm.
Parameters:
      * amount (int or float) -- quantity of channels to
        prune. If "float", should be between 0.0 and 1.0 and represent
        the fraction of parameters to prune. If "int", it represents
        the absolute number of parameters to prune.
  * **n** (*int**, **float**, **inf**, **-inf**, **'fro'**,
    **'nuc'*) -- See documentation of valid entries for argument
    "p" in "torch.norm()".

  * **dim** (*int**, **optional*) -- index of the dim along which
    we define channels to prune. Default: -1.

classmethod apply(module, name, amount, n, dim, importance_scores=None)
  Adds the forward pre-hook that enables pruning on the fly and
  the reparametrization of a tensor in terms of the original
  tensor and the pruning mask.

  Parameters:
Parameters:
         * module (nn.Module) -- module containing the tensor to
           prune
     * **name** (*str*) -- parameter name within "module" on which
       pruning will act.

     * **amount** (*int** or **float*) -- quantity of parameters
       to prune. If "float", should be between 0.0 and 1.0 and
       represent the fraction of parameters to prune. If "int", it
       represents the absolute number of parameters to prune.

     * **n** (*int**, **float**, **inf**, **-inf**, **'fro'**,
       **'nuc'*) -- See documentation of valid entries for
       argument "p" in "torch.norm()".

     * **dim** (*int*) -- index of the dim along which we define
       channels to prune.

     * **importance_scores** (*torch.Tensor*) -- tensor of
       importance scores (of same shape as module parameter) used
       to compute mask for pruning. The values in this tensor
       indicate the importance of the corresponding elements in
the parameter being pruned. If unspecified or None, the
           module parameter will be used in its place.
apply_mask(module)
  Simply handles the multiplication between the parameter being
  pruned and the generated mask. Fetches the mask and the original
  tensor from the module and returns the pruned version of the
  tensor.

  Parameters:
     **module** (*nn.Module*) -- module containing the tensor to
     prune

  Returns:
     pruned version of the input tensor

  Return type:
     pruned_tensor (torch.Tensor)

compute_mask(t, default_mask)
  Computes and returns a mask for the input tensor "t". Starting
  from a base "default_mask" (which should be a mask of ones if
  the tensor has not been pruned yet), generate a mask to apply on
  top of the "default_mask" by zeroing out the channels along the
  specified dim with the lowest L"n"-norm.

  Parameters:
Parameters:
         * t (torch.Tensor) -- tensor representing the parameter
           to prune
     * **default_mask** (*torch.Tensor*) -- Base mask from
       previous pruning iterations, that need to be respected
       after the new mask is applied.  Same dims as "t".

  Returns:
     mask to apply to "t", of same dims as "t"

  Return type:
     mask (torch.Tensor)

  Raises:
     **IndexError** -- if "self.dim >= len(t.shape)"

prune(t, default_mask=None, importance_scores=None)
  Computes and returns a pruned version of input tensor "t"
  according to the pruning rule specified in "compute_mask()".

  Parameters:
     * **t** (*torch.Tensor*) -- tensor to prune (of same
       dimensions as "default_mask").

     * **importance_scores** (*torch.Tensor*) -- tensor of
       importance scores (of same shape as "t") used to compute
       mask for pruning "t". The values in this tensor indicate
the importance of the corresponding elements in the "t"
           that is being pruned. If unspecified or None, the tensor
           "t" will be used in its place.
     * **default_mask** (*torch.Tensor**, **optional*) -- mask
       from previous pruning iteration, if any. To be considered
       when determining what portion of the tensor that pruning
       should act on. If None, default to a mask of ones.

  Returns:
     pruned version of tensor "t".

remove(module)
  Removes the pruning reparameterization from a module. The pruned
  parameter named "name" remains permanently pruned, and the
  parameter named "name+'_orig'" is removed from the parameter
  list. Similarly, the buffer named "name+'_mask'" is removed from
  the buffers.

  Note:

    Pruning itself is NOT undone or reversed!
LazyConv2d
class torch.nn.LazyConv2d(out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)
A "torch.nn.Conv2d" module with lazy initialization of the
   "in_channels" argument of the "Conv2d" that is inferred from the
   "input.size(1)". The attributes that will be lazily initialized are
   weight and bias.
Check the "torch.nn.modules.lazy.LazyModuleMixin" for further
   documentation on lazy modules and their limitations.
Parameters:
      * out_channels (int) -- Number of channels produced by the
        convolution
  * **kernel_size** (*int** or **tuple*) -- Size of the convolving
    kernel

  * **stride** (*int** or **tuple**, **optional*) -- Stride of the
    convolution. Default: 1

  * **padding** (*int** or **tuple**, **optional*) -- Zero-padding
    added to both sides of the input. Default: 0

  * **padding_mode** (*str**, **optional*) -- "'zeros'",
"'reflect'", "'replicate'" or "'circular'". Default: "'zeros'"
  * **dilation** (*int** or **tuple**, **optional*) -- Spacing
    between kernel elements. Default: 1

  * **groups** (*int**, **optional*) -- Number of blocked
    connections from input channels to output channels. Default: 1

  * **bias** (*bool**, **optional*) -- If "True", adds a learnable
    bias to the output. Default: "True"

See also:
 "torch.nn.Conv2d" and "torch.nn.modules.lazy.LazyModuleMixin"

cls_to_become
  alias of "Conv2d"
torch.kron
torch.kron(input, other, *, out=None) -> Tensor
Computes the Kronecker product, denoted by \otimes, of "input" and
   "other".
If "input" is a (a_0 \times a_1 \times \dots \times a_n) tensor and
   "other" is a (b_0 \times b_1 \times \dots \times b_n) tensor, the
   result will be a (a_0b_0 \times a_1b_1 \times \dots \times
   a_n*b_n) tensor with the following entries:
  (\text{input} \otimes \text{other})_{k_0, k_1, \dots, k_n} =
  \text{input}_{i_0, i_1, \dots, i_n} * \text{other}_{j_0, j_1,
  \dots, j_n},

where k_t = i_t * b_t + j_t for 0 \leq t \leq n. If one tensor has
   fewer dimensions than the other it is unsqueezed until it has the
   same number of dimensions.
Supports real-valued and complex-valued inputs.
Note:
 This function generalizes the typical definition of the Kronecker
 product for two matrices to two tensors, as described above. When
 "input" is a (m \times n) matrix and "other" is a (p \times q)
matrix, the result will be a (pm \times qn) block matrix:
    \mathbf{A} \otimes \mathbf{B}=\begin{bmatrix} a_{11}
    \mathbf{B} & \cdots & a_{1 n} \mathbf{B} \\ \vdots & \ddots &
    \vdots \\ a_{m 1} \mathbf{B} & \cdots & a_{m n} \mathbf{B}
    \end{bmatrix}

 where "input" is \mathbf{A} and "other" is \mathbf{B}.

Parameters:
      * input (Tensor) --
  * **other** (*Tensor*) --

Keyword Arguments:
      out (Tensor, optional) -- The output tensor. Ignored
      if "None". Default: "None"
Examples:
  >>> mat1 = torch.eye(2)
  >>> mat2 = torch.ones(2, 2)
  >>> torch.kron(mat1, mat2)
  tensor([[1., 1., 0., 0.],
          [1., 1., 0., 0.],
          [0., 0., 1., 1.],
          [0., 0., 1., 1.]])

  >>> mat1 = torch.eye(2)
  >>> mat2 = torch.arange(1, 5).reshape(2, 2)
  >>> torch.kron(mat1, mat2)
  tensor([[1., 2., 0., 0.],
          [3., 4., 0., 0.],
          [0., 0., 1., 2.],
torch.fft.ihfft
torch.fft.ihfft(input, n=None, dim=- 1, norm=None, *, out=None) -> Tensor
Computes the inverse of "hfft()".
"input" must be a real-valued signal, interpreted in the Fourier
   domain. The IFFT of a real signal is Hermitian-symmetric, "X[i] =
   conj(X[-i])". "ihfft()" represents this in the one-sided form where
   only the positive frequencies below the Nyquist frequency are
   included. To compute the full output, use "ifft()".
Note:
 Supports torch.half on CUDA with GPU Architecture SM53 or
 greater. However it only supports powers of 2 signal length in
 every transformed dimension.

Parameters:
      * input (Tensor) -- the real input tensor
  * **n** (*int**, **optional*) -- Signal length. If given, the
    input will either be zero-padded or trimmed to this length
    before computing the Hermitian IFFT.

  * **dim** (*int**, **optional*) -- The dimension along which to
    take the one dimensional Hermitian IFFT.
take the one dimensional Hermitian IFFT.
  * **norm** (*str**, **optional*) --

    Normalization mode. For the backward transform ("ihfft()"),
    these correspond to:

    * ""forward"" - no normalization

    * ""backward"" - normalize by "1/n"

    * ""ortho"" - normalize by "1/sqrt(n)" (making the IFFT
      orthonormal)

    Calling the forward transform ("hfft()") with the same
    normalization mode will apply an overall normalization of
    "1/n" between the two transforms. This is required to make
    "ihfft()" the exact inverse.

    Default is ""backward"" (normalize by "1/n").

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-



t = torch.arange(5)
t
   tensor([0, 1, 2, 3, 4])
torch.fft.ihfft(t)
   tensor([ 2.0000-0.0000j, -0.5000-0.6882j, -0.5000-0.1625j])



Compare against the full output from "ifft()":



torch.fft.ifft(t)
torch.nn.modules.module.register_module_full_backward_hook
torch.nn.modules.module.register_module_full_backward_hook(hook)
Registers a backward hook common to all the modules.
Warning:
 This adds global state to the *nn.module* module and it is only
 intended for debugging/profiling purposes.

The hook will be called every time the gradients with respect to a
   module are computed, i.e. the hook will execute if and only if the
   gradients with respect to module outputs are computed. The hook
   should have the following signature:
  hook(module, grad_input, grad_output) -> Tensor or None

The "grad_input" and "grad_output" are tuples. The hook should not
   modify its arguments, but it can optionally return a new gradient
   with respect to the input that will be used in place of
   "grad_input" in subsequent computations. "grad_input" will only
   correspond to the inputs given as positional arguments and all
kwarg arguments will not appear in the hook. Entries in
   "grad_input" and "grad_output" will be "None" for all non-Tensor
   arguments.
For technical reasons, when this hook is applied to a Module, its
   forward function will receive a view of each Tensor passed to the
   Module. Similarly the caller will receive a view of each Tensor
   returned by the Module's forward function.
Global hooks are called before hooks registered with
   register_backward_hook
Returns:
      a handle that can be used to remove the added hook by calling
      "handle.remove()"
Return type:
      "torch.utils.hooks.RemovableHandle"
torch.jit.annotate
torch.jit.annotate(the_type, the_value)
This method is a pass-through function that returns the_value,
   used to hint TorchScript compiler the type of the_value. It is a
   no-op when running outside of TorchScript.
Though TorchScript can infer correct type for most Python
   expressions, there are some cases where type inference can be
   wrong, including:


Empty containers like [] and {}, which TorchScript assumes to
     be container of Tensor


Optional types like Optional[T] but assigned a valid value of
     type T, TorchScript would assume it is type T rather than
     Optional[T]


Note that annotate() does not help in init method of
   torch.nn.Module subclasses because it is executed in eager mode.
   To annotate types of torch.nn.Module attributes, use "Annotate()"
   instead.
Example:
  import torch
  from typing import Dict

  @torch.jit.script
  def fn():