shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.fft.fftshift
torch.fft.fftshift(input, dim=None) -> Tensor
Reorders n-dimensional FFT data, as provided by "fftn()", to have
   negative frequency terms first.
This performs a periodic shift of n-dimensional data such that the
   origin "(0, ..., 0)" is moved to the center of the tensor.
   Specifically, to "input.shape[dim] // 2" in each selected
   dimension.
Note:
 By convention, the FFT returns positive frequency terms first,
 followed by the negative frequencies in reverse order, so that
 "f[-i]" for all 0 < i \leq n/2 in Python gives the negative
 frequency terms. "fftshift()" rearranges all frequencies into
 ascending order from negative to positive with the zero-frequency
 term in the center.

Note:
 For even lengths, the Nyquist frequency at "f[n/2]" can be
 thought of as either negative or positive. "fftshift()" always
 puts the Nyquist term at the 0-index. This is the same convention
 used by "fftfreq()".
used by "fftfreq()".
Parameters:
      * input (Tensor) -- the tensor in FFT order
  * **dim** (*int**, **Tuple**[**int**]**, **optional*) -- The
    dimensions to rearrange. Only dimensions specified here will
    be rearranged, any other dimensions will be left in their
    original order. Default: All dimensions of "input".

-[ Example ]-



f = torch.fft.fftfreq(4)
f
   tensor([ 0.0000,  0.2500, -0.5000, -0.2500])
torch.fft.fftshift(f)
   tensor([-0.5000, -0.2500,  0.0000,  0.2500])



Also notice that the Nyquist frequency term at "f[2]" was moved to
   the beginning of the tensor.
This also works for multi-dimensional transforms:



x = torch.fft.fftfreq(5, d=1/5) + 0.1 * torch.fft.fftfreq(5, d=1/5).unsqueeze(1)
x
   tensor([[ 0.0000,  1.0000,  2.0000, -2.0000, -1.0000],
           [ 0.1000,  1.1000,  2.1000, -1.9000, -0.9000],
           [ 0.2000,  1.2000,  2.2000, -1.8000, -0.8000],
[-0.2000,  0.8000,  1.8000, -2.2000, -1.2000],
           [-0.1000,  0.9000,  1.9000, -2.1000, -1.1000]])



torch.fft.fftshift(x)
   tensor([[-2.2000, -1.2000, -0.2000,  0.8000,  1.8000],
           [-2.1000, -1.1000, -0.1000,  0.9000,  1.9000],
           [-2.0000, -1.0000,  0.0000,  1.0000,  2.0000],
           [-1.9000, -0.9000,  0.1000,  1.1000,  2.1000],
           [-1.8000, -0.8000,  0.2000,  1.2000,  2.2000]])



"fftshift()" can also be useful for spatial data. If our data is
   defined on a centered grid ("[-(N//2), (N-1)//2]") then we can use
   the standard FFT defined on an uncentered grid ("[0, N)") by first
   applying an "ifftshift()".



x_centered = torch.arange(-5, 5)
x_uncentered = torch.fft.ifftshift(x_centered)
fft_uncentered = torch.fft.fft(x_uncentered)



Similarly, we can convert the frequency domain components to
   centered convention by applying "fftshift()".



fft_centered = torch.fft.fftshift(fft_uncentered)
The inverse transform, from centered Fourier space back to centered
   spatial data, can be performed by applying the inverse shifts in
   reverse order:



x_centered_2 = torch.fft.fftshift(torch.fft.ifft(torch.fft.ifftshift(fft_centered)))
torch.testing.assert_close(x_centered.to(torch.complex64), x_centered_2, check_stride=False)
torch.nn.functional.gumbel_softmax
torch.nn.functional.gumbel_softmax(logits, tau=1, hard=False, eps=1e-10, dim=- 1)
Samples from the Gumbel-Softmax distribution (Link 1  Link 2) and
   optionally discretizes.
Parameters:
      * logits (Tensor) -- [..., num_features] unnormalized
        log probabilities
  * **tau** (*float*) -- non-negative scalar temperature

  * **hard** (*bool*) -- if "True", the returned samples will be
    discretized as one-hot vectors, but will be differentiated as
    if it is the soft sample in autograd

  * **dim** (*int*) -- A dimension along which softmax will be
    computed. Default: -1.

Returns:
      Sampled tensor of same shape as logits from the Gumbel-Softmax
      distribution. If "hard=True", the returned samples will be one-
      hot, otherwise they will be probability distributions that sum
      to 1 across dim.
Return type:
      Tensor
Note:
Return type:
      Tensor
Note:
 This function is here for legacy reasons, may be removed from
 nn.Functional in the future.

Note:
 The main trick for *hard* is to do  *y_hard - y_soft.detach() +
 y_soft*It achieves two things: - makes the output value exactly
 one-hot (since we add then subtract y_soft value) - makes the
 gradient equal to y_soft gradient (since we strip all other
 gradients)

Examples::
      >>> logits = torch.randn(20, 32)
      >>> # Sample soft categorical using reparametrization trick:
      >>> F.gumbel_softmax(logits, tau=1, hard=False)
      >>> # Sample hard categorical using "Straight-through" trick:
      >>> F.gumbel_softmax(logits, tau=1, hard=True)
torch.nn.utils.parametrize.register_parametrization
torch.nn.utils.parametrize.register_parametrization(module, tensor_name, parametrization, *, unsafe=False)
Adds a parametrization to a tensor in a module.
Assume that "tensor_name="weight"" for simplicity. When accessing
   "module.weight", the module will return the parametrized version
   "parametrization(module.weight)". If the original tensor requires a
   gradient, the backward pass will differentiate through
   "parametrization", and the optimizer will update the tensor
   accordingly.
The first time that a module registers a parametrization, this
   function will add an attribute "parametrizations" to the module of
   type "ParametrizationList".
The list of parametrizations on the tensor "weight" will be
   accessible under "module.parametrizations.weight".
The original tensor will be accessible under
   "module.parametrizations.weight.original".
"module.parametrizations.weight.original".
Parametrizations may be concatenated by registering several
   parametrizations on the same attribute.
The training mode of a registered parametrization is updated on
   registration to match the training mode of the host module
Parametrized parameters and buffers have an inbuilt caching system
   that can be activated using the context manager "cached()".
A "parametrization" may optionally implement a method with
   signature
  def right_inverse(self, X: Tensor) -> Union[Tensor, Sequence[Tensor]]

This method is called on the unparametrized tensor when the first
   parametrization is registered to compute the initial value of the
   original tensor. If this method is not implemented, the original
   tensor will be just the unparametrized tensor.
If all the parametrizations registered on a tensor implement
   right_inverse it is possible to initialize a parametrized tensor
   by assigning to it, as shown in the example below.
It is possible for the first parametrization to depend on several
   inputs. This may be implemented returning a tuple of tensors from
   "right_inverse" (see the example implementation of a "RankOne"
   parametrization below).
In this case, the unconstrained tensors are also located under
   "module.parametrizations.weight" with names "original0",
   "original1",...
Note:
 If unsafe=False (default) both the forward and right_inverse
 methods will be called once to perform a number of consistency
 checks. If unsafe=True, then right_inverse will be called if the
 tensor is not parametrized, and nothing will be called otherwise.

Note:
 In most situations, "right_inverse" will be a function such that
 "forward(right_inverse(X)) == X" (see right inverse). Sometimes,
 when the parametrization is not surjective, it may be reasonable
 to relax this.

Warning:
 If a parametrization depends on several inputs,
"register_parametrization()" will register a number of new
     parameters. If such parametrization is registered after the
     optimizer is created, these new parameters will need to be added
     manually to the optimizer. See
     "torch.Optimizer.add_param_group()".
Parameters:
      * module (nn.Module) -- module on which to register the
        parametrization
  * **tensor_name** (*str*) -- name of the parameter or buffer on
    which to register the parametrization

  * **parametrization** (*nn.Module*) -- the parametrization to
    register

Keyword Arguments:
      unsafe (bool) -- a boolean flag that denotes whether the
      parametrization may change the dtype and shape of the tensor.
      Default: False Warning: the parametrization is not checked for
      consistency upon registration. Enable this flag at your own
      risk.
Raises:
      ValueError -- if the module does not have a parameter or a
      buffer named "tensor_name"
buffer named "tensor_name"
Return type:
      Module
-[ Examples ]-



import torch
import torch.nn as nn
import torch.nn.utils.parametrize as P
class Symmetric(nn.Module):
    def forward(self, X):
        return X.triu() + X.triu(1).T  # Return a symmetric matrix
def right_inverse(self, A):
    return A.triu()

m = nn.Linear(5, 5)
P.register_parametrization(m, "weight", Symmetric())
print(torch.allclose(m.weight, m.weight.T))  # m.weight is now symmetric
   True
A = torch.rand(5, 5)
A = A + A.T   # A is now symmetric
m.weight = A  # Initialize the weight to be the symmetric matrix A
print(torch.allclose(m.weight, A))
   True
class RankOne(nn.Module):
    def forward(self, x, y):
        # Form a rank 1 matrix multiplying two vectors
        return x.unsqueeze(-1) @ y.unsqueeze(-2)
def right_inverse(self, Z):
def right_inverse(self, Z):
    # Project Z onto the rank 1 matrices
    U, S, Vh = torch.linalg.svd(Z, full_matrices=False)
    # Return rescaled singular vectors
    s0_sqrt = S[0].sqrt().unsqueeze(-1)
    return U[..., :, 0] * s0_sqrt, Vh[..., 0, :] * s0_sqrt

linear_rank_one = P.register_parametrization(nn.Linear(4, 4), "weight", RankOne())
print(torch.linalg.matrix_rank(linear_rank_one.weight).item())
   1
default_qat_qconfig_v2
torch.quantization.qconfig.default_qat_qconfig_v2
alias of QConfig(activation=functools.partial(,
   observer=,
   quant_min=0, quant_max=255, dtype=torch.quint8){},
   weight=functools.partial(, observer=,
   quant_min=-128, quant_max=127, dtype=torch.qint8,
   qscheme=torch.per_tensor_symmetric){})
torch.fft.rfft2
torch.fft.rfft2(input, s=None, dim=(- 2, - 1), norm=None, *, out=None) -> Tensor
Computes the 2-dimensional discrete Fourier transform of real
   "input". Equivalent to "rfftn()" but FFTs only the last two
   dimensions by default.
The FFT of a real signal is Hermitian-symmetric, "X[i, j] =
   conj(X[-i, -j])", so the full "fft2()" output contains redundant
   information. "rfft2()" instead omits the negative frequencies in
   the last dimension.
Note:
 Supports torch.half on CUDA with GPU Architecture SM53 or
 greater. However it only supports powers of 2 signal length in
 every transformed dimensions.

Parameters:
      * input (Tensor) -- the input tensor
  * **s** (*Tuple**[**int**]**, **optional*) -- Signal size in the
    transformed dimensions. If given, each dimension "dim[i]" will
    either be zero-padded or trimmed to the length "s[i]" before
    computing the real FFT. If a length "-1" is specified, no
padding is done in that dimension. Default: "s =
        [input.size(d) for d in dim]"
  * **dim** (*Tuple**[**int**]**, **optional*) -- Dimensions to be
    transformed. Default: last two dimensions.

  * **norm** (*str**, **optional*) --

    Normalization mode. For the forward transform ("rfft2()"),
    these correspond to:

    * ""forward"" - normalize by "1/n"

    * ""backward"" - no normalization

    * ""ortho"" - normalize by "1/sqrt(n)" (making the real FFT
      orthonormal)

    Where "n = prod(s)" is the logical FFT size. Calling the
    backward transform ("irfft2()") with the same normalization
    mode will apply an overall normalization of "1/n" between the
    two transforms. This is required to make "irfft2()" the exact
    inverse.

    Default is ""backward"" (no normalization).

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-



t = torch.rand(10, 10)
-[ Example ]-



t = torch.rand(10, 10)
rfft2 = torch.fft.rfft2(t)
rfft2.size()
   torch.Size([10, 6])



Compared against the full output from "fft2()", we have all
   elements up to the Nyquist frequency.



fft2 = torch.fft.fft2(t)
torch.testing.assert_close(fft2[..., :6], rfft2, check_stride=False)



The discrete Fourier transform is separable, so "rfft2()" here is
   equivalent to a combination of "fft()" and "rfft()":



two_ffts = torch.fft.fft(torch.fft.rfft(t, dim=1), dim=0)
torch.testing.assert_close(rfft2, two_ffts, check_stride=False)
GRU
class torch.ao.nn.quantized.dynamic.GRU(args, *kwargs)
Applies a multi-layer gated recurrent unit (GRU) RNN to an input
   sequence.
For each element in the input sequence, each layer computes the
   following function:
  \begin{array}{ll}     r_t = \sigma(W_{ir} x_t + b_{ir} + W_{hr}
  h_{(t-1)} + b_{hr}) \\     z_t = \sigma(W_{iz} x_t + b_{iz} +
  W_{hz} h_{(t-1)} + b_{hz}) \\     n_t = \tanh(W_{in} x_t +
  b_{in} + r_t * (W_{hn} h_{(t-1)}+ b_{hn})) \\     h_t = (1 -
  z_t) * n_t + z_t * h_{(t-1)} \end{array}

where h_t is the hidden state at time t, x_t is the input at time
   t, h_{(t-1)} is the hidden state of the layer at time t-1 or
   the initial hidden state at time 0, and r_t, z_t, n_t are the
   reset, update, and new gates, respectively. \sigma is the sigmoid
   function, and * is the Hadamard product.
In a multilayer GRU, the input x^{(l)}_t of the l -th layer (l >=
   2) is the hidden state h^{(l-1)}_t of the previous layer multiplied
by dropout \delta^{(l-1)}_t where each \delta^{(l-1)}_t is a
   Bernoulli random variable which is 0 with probability "dropout".
Parameters:
      * input_size -- The number of expected features in the input
        x
  * **hidden_size** -- The number of features in the hidden state
    *h*

  * **num_layers** -- Number of recurrent layers. E.g., setting
    "num_layers=2" would mean stacking two GRUs together to form a
    *stacked GRU*, with the second GRU taking in outputs of the
    first GRU and computing the final results. Default: 1

  * **bias** -- If "False", then the layer does not use bias
    weights *b_ih* and *b_hh*. Default: "True"

  * **batch_first** -- If "True", then the input and output
    tensors are provided as (batch, seq, feature). Default:
    "False"

  * **dropout** -- If non-zero, introduces a *Dropout* layer on
    the outputs of each GRU layer except the last layer, with
dropout probability equal to "dropout". Default: 0
  * **bidirectional** -- If "True", becomes a bidirectional GRU.
    Default: "False"

Inputs: input, h_0
      * input of shape (seq_len, batch, input_size): tensor
        containing the features of the input sequence. The input can
        also be a packed variable length sequence. See
        "torch.nn.utils.rnn.pack_padded_sequence()" for details.
  * **h_0** of shape *(num_layers * num_directions, batch,
    hidden_size)*: tensor containing the initial hidden state for
    each element in the batch. Defaults to zero if not provided.
    If the RNN is bidirectional, num_directions should be 2, else
    it should be 1.

Outputs: output, h_n
      * output of shape (seq_len, batch, num_directions *
        hidden_size): tensor containing the output features h_t from
        the last layer of the GRU, for each t. If a
        "torch.nn.utils.rnn.PackedSequence" has been given as the
input, the output will also be a packed sequence. For the
        unpacked case, the directions can be separated using
        "output.view(seq_len, batch, num_directions, hidden_size)",
        with forward and backward being direction 0 and 1
        respectively.
    Similarly, the directions can be separated in the packed case.

  * **h_n** of shape *(num_layers * num_directions, batch,
    hidden_size)*: tensor containing the hidden state for *t =
    seq_len*

    Like *output*, the layers can be separated using
    "h_n.view(num_layers, num_directions, batch, hidden_size)".

Shape:
      * Input1: (L, N, H_{in}) tensor containing input features where
        H_{in}=\text{input_size} and L represents a sequence
        length.
  * Input2: (S, N, H_{out}) tensor containing the initial hidden
    state for each element in the batch.
    H_{out}=\text{hidden\_size} Defaults to zero if not provided.
where S=\text{num_layers} * \text{num_directions} If the RNN
        is bidirectional, num_directions should be 2, else it should
        be 1.
  * Output1: (L, N, H_{all}) where H_{all}=\text{num\_directions}
    * \text{hidden\_size}

  * Output2: (S, N, H_{out}) tensor containing the next hidden
    state for each element in the batch

Variables:
      * weight_ih_l[k] -- the learnable input-hidden weights of
        the \text{k}^{th} layer (W_ir|W_iz|W_in), of shape
        (3hidden_size, input_size) for k = 0. Otherwise, the
        shape is (3hidden_size, num_directions * hidden_size)
  * **weight_hh_l[k]** -- the learnable hidden-hidden weights of
    the \text{k}^{th} layer (W_hr|W_hz|W_hn), of shape
    *(3*hidden_size, hidden_size)*

  * **bias_ih_l[k]** -- the learnable input-hidden bias of the
    \text{k}^{th} layer (b_ir|b_iz|b_in), of shape
    *(3*hidden_size)*
(3hidden_size)*
  * **bias_hh_l[k]** -- the learnable hidden-hidden bias of the
    \text{k}^{th} layer (b_hr|b_hz|b_hn), of shape
    *(3*hidden_size)*

Note:
 All the weights and biases are initialized from
 \mathcal{U}(-\sqrt{k}, \sqrt{k}) where k =
 \frac{1}{\text{hidden\_size}}

Note:
 If the following conditions are satisfied: 1) cudnn is enabled,
 2) input data is on the GPU 3) input data has dtype
 "torch.float16" 4) V100 GPU is used, 5) input data is not in
 "PackedSequence" format persistent algorithm can be selected to
 improve performance.

Examples:
  >>> rnn = nn.GRU(10, 20, 2)
  >>> input = torch.randn(5, 3, 10)
  >>> h0 = torch.randn(2, 3, 20)
  >>> output, hn = rnn(input, h0)
torch.Tensor.crow_indices
Tensor.crow_indices() -> IntTensor
Returns the tensor containing the compressed row indices of the
   "self" tensor when "self" is a sparse CSR tensor of layout
   "sparse_csr". The "crow_indices" tensor is strictly of shape
   ("self".size(0) + 1) and of type "int32" or "int64". When using MKL
   routines such as sparse matrix multiplication, it is necessary to
   use "int32" indexing in order to avoid downcasting and potentially
   losing information.
Example::
      >>> csr = torch.eye(5,5).to_sparse_csr()
      >>> csr.crow_indices()
      tensor([0, 1, 2, 3, 4, 5], dtype=torch.int32)
torch.Tensor.add
Tensor.add(other, *, alpha=1) -> Tensor
Add a scalar or tensor to "self" tensor. If both "alpha" and
   "other" are specified, each element of "other" is scaled by "alpha"
   before being used.
When "other" is a tensor, the shape of "other" must be
   broadcastable with the shape of the underlying tensor
See "torch.add()"
torch.cuda.get_device_name
torch.cuda.get_device_name(device=None)
Gets the name of a device.
Parameters:
      device (torch.device or int, optional) -- device
      for which to return the name. This function is a no-op if this
      argument is a negative integer. It uses the current device,
      given by "current_device()", if "device" is "None" (default).
Returns:
      the name of the device
Return type:
      str
torch.autograd.functional.vjp
torch.autograd.functional.vjp(func, inputs, v=None, create_graph=False, strict=False)
Function that computes the dot product between a vector "v" and the
   Jacobian of the given function at the point given by the inputs.
Parameters:
      * func (function) -- a Python function that takes Tensor
        inputs and returns a tuple of Tensors or a Tensor.
  * **inputs** (*tuple of Tensors** or **Tensor*) -- inputs to the
    function "func".

  * **v** (*tuple of Tensors** or **Tensor*) -- The vector for
    which the vector Jacobian product is computed.  Must be the
    same size as the output of "func". This argument is optional
    when the output of "func" contains a single element and (if it
    is not provided) will be set as a Tensor containing a single
    "1".

  * **create_graph** (*bool**, **optional*) -- If "True", both the
output and result will be computed in a differentiable way.
        Note that when "strict" is "False", the result can not require
        gradients or be disconnected from the inputs.  Defaults to
        "False".
  * **strict** (*bool**, **optional*) -- If "True", an error will
    be raised when we detect that there exists an input such that
    all the outputs are independent of it. If "False", we return a
    Tensor of zeros as the vjp for said inputs, which is the
    expected mathematical value. Defaults to "False".

Returns:
      tuple with:
         func_output (tuple of Tensors or Tensor): output of
         "func(inputs)"
     vjp (tuple of Tensors or Tensor): result of the dot product
     with the same shape as the inputs.

Return type:
      output (tuple)
-[ Example ]-



def exp_reducer(x):
   ...     return x.exp().sum(dim=1)
inputs = torch.rand(4, 4)
v = torch.ones(4)
vjp(exp_reducer, inputs, v)
vjp(exp_reducer, inputs, v)
   (tensor([5.7817, 7.2458, 5.7830, 6.7782]),
    tensor([[1.4458, 1.3962, 1.3042, 1.6354],
           [2.1288, 1.0652, 1.5483, 2.5035],
           [2.2046, 1.1292, 1.1432, 1.3059],
           [1.3225, 1.6652, 1.7753, 2.0152]]))
vjp(exp_reducer, inputs, v, create_graph=True)
   (tensor([5.7817, 7.2458, 5.7830, 6.7782], grad_fn=),
    tensor([[1.4458, 1.3962, 1.3042, 1.6354],
           [2.1288, 1.0652, 1.5483, 2.5035],
           [2.2046, 1.1292, 1.1432, 1.3059],
           [1.3225, 1.6652, 1.7753, 2.0152]], grad_fn=))
def adder(x, y):
   ...     return 2 * x + 3 * y
inputs = (torch.rand(2), torch.rand(2))
v = torch.ones(2)
vjp(adder, inputs, v)
   (tensor([2.4225, 2.3340]),
    (tensor([2., 2.]), tensor([3., 3.])))
torch.nan_to_num
torch.nan_to_num(input, nan=0.0, posinf=None, neginf=None, *, out=None) -> Tensor
Replaces "NaN", positive infinity, and negative infinity values in
   "input" with the values specified by "nan", "posinf", and "neginf",
   respectively. By default, "NaN"s are replaced with zero, positive
   infinity is replaced with the greatest finite value representable
   by "input"'s dtype, and negative infinity is replaced with the
   least finite value representable by "input"'s dtype.
Parameters:
      * input (Tensor) -- the input tensor.
  * **nan** (*Number**, **optional*) -- the value to replace
    "NaN"s with. Default is zero.

  * **posinf** (*Number**, **optional*) -- if a Number, the value
    to replace positive infinity values with. If None, positive
    infinity values are replaced with the greatest finite value
    representable by "input"'s dtype. Default is None.
neginf (Number, optional) -- if a Number, the value
        to replace negative infinity values with. If None, negative
        infinity values are replaced with the lowest finite value
        representable by "input"'s dtype. Default is None.

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> x = torch.tensor([float('nan'), float('inf'), -float('inf'), 3.14])
  >>> torch.nan_to_num(x)
  tensor([ 0.0000e+00,  3.4028e+38, -3.4028e+38,  3.1400e+00])
  >>> torch.nan_to_num(x, nan=2.0)
  tensor([ 2.0000e+00,  3.4028e+38, -3.4028e+38,  3.1400e+00])
  >>> torch.nan_to_num(x, nan=2.0, posinf=1.0)
  tensor([ 2.0000e+00,  1.0000e+00, -3.4028e+38,  3.1400e+00])
FractionalMaxPool3d
class torch.nn.FractionalMaxPool3d(kernel_size, output_size=None, output_ratio=None, return_indices=False, _random_samples=None)
Applies a 3D fractional max pooling over an input signal composed
   of several input planes.
Fractional MaxPooling is described in detail in the paper
   Fractional MaxPooling by Ben Graham
The max-pooling operation is applied in kT \times kH \times kW
   regions by a stochastic step size determined by the target output
   size. The number of output features is equal to the number of input
   planes.
Parameters:
      * kernel_size (Union[int, Tuple[int, int,
        int]]) -- the size of the window to take a max over.
        Can be a single number k (for a square kernel of k x k x k) or
        a tuple (kt x kh x kw)
  * **output_size** (*Union**[**int**, **Tuple**[**int**, **int**,
    **int**]**]*) -- the target output size of the image of the