shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.func.functional_call
torch.func.functional_call(module, parameter_and_buffer_dicts, args, kwargs=None, *, tie_weights=True)
Performs a functional call on the module by replacing the module
   parameters and buffers with the provided ones.
Note:
 If the module has active parametrizations, passing a value in the
 "parameters_and_buffers" argument with the name set to the
 regular parameter name will completely disable the
 parametrization. If you want to apply the parametrization
 function to the value passed please set the key as
 "{submodule_name}.parametrizations.{parameter_name}.original".

Note:
 If the module performs in-place operations on parameters/buffers,
 these will be reflected in the "parameters_and_buffers" input.

    Example:

       >>> a = {'foo': torch.zeros(())}
       >>> mod = Foo()  # does self.foo = self.foo + 1
       >>> print(mod.foo)  # tensor(0.)
print(mod.foo)  # tensor(0.)
           >>> functional_call(mod, a, torch.ones(()))
           >>> print(mod.foo)  # tensor(0.)
           >>> print(a['foo'])  # tensor(1.)



Note:
 If the module has tied weights, whether or not functional_call
 respects the tying is determined by the tie_weights flag.Example:

    >>> a = {'foo': torch.zeros(())}
    >>> mod = Foo()  # has both self.foo and self.foo_tied which are tied. Returns x + self.foo + self.foo_tied
    >>> print(mod.foo)  # tensor(1.)
    >>> mod(torch.zeros(()))  # tensor(2.)
    >>> functional_call(mod, a, torch.zeros(()))  # tensor(0.) since it will change self.foo_tied too
    >>> functional_call(mod, a, torch.zeros(()), tie_weights=False)  # tensor(1.)--self.foo_tied is not updated
    >>> new_a = {'foo', torch.zeros(()), 'foo_tied': torch.zeros(())}
    >>> functional_call(mod, new_a, torch.zeros()) # tensor(0.)

An example of passing mutliple dictionaries
An example of passing mutliple dictionaries
  a = ({'weight': torch.ones(1, 1)}, {'buffer': torch.zeros(1)})  # two separate dictionaries
  mod = nn.Bar(1, 1)  # return self.weight @ x + self.buffer
  print(mod.weight)  # tensor(...)
  print(mod.buffer)  # tensor(...)
  x = torch.randn((1, 1))
  print(x)
  functional_call(mod, a, x)  # same as x
  print(mod.weight)  # same as before functional_call

And here is an example of applying the grad transform over the
   parameters of a model.
  import torch
  import torch.nn as nn
  from torch.func import functional_call, grad

  x = torch.randn(4, 3)
  t = torch.randn(4, 3)
  model = nn.Linear(3, 3)

  def compute_loss(params, x, t):
      y = functional_call(model, params, x)
      return nn.functional.mse_loss(y, t)

  grad_weights = grad(compute_loss)(dict(model.named_parameters()), x, t)

Note:
 If the user does not need grad tracking outside of grad
transforms, they can detach all of the parameters for better
     performance and memory usageExample:
    >>> detached_params = {k: v.detach() for k, v in model.named_parameters()}
    >>> grad_weights = grad(compute_loss)(detached_params, x, t)
    >>> grad_weights.grad_fn  # None--it's not tracking gradients outside of grad

 This means that the user cannot call "grad_weight.backward()".
 However, if they don't need autograd tracking outside of the
 transforms, this will result in less memory usage and faster
 speeds.

Parameters:
      * module (torch.nn.Module) -- the module to call
  * **parameters_and_buffers** (*Dict**[**str**,**Tensor**] or
    **tuple of Dict**[**str**, **Tensor**]*) -- the parameters
    that will be used in the module call. If given a tuple of
    dictionaries, they must have distinct keys so that all
    dictionaries can be used together

  * **args** (*Any** or **tuple*) -- arguments to be passed to the
module call. If not a tuple, considered a single argument.
  * **kwargs** (*dict*) -- keyword arguments to be passed to the
    module call

  * **tie_weights** (*bool**, **optional*) -- If True, then
    parameters and buffers tied in the original model will be
    treated as tied in the reparamaterized version. Therefore, if
    True and different values are passed for the tied paramaters
    and buffers, it will error. If False, it will not respect the
    originally tied parameters and buffers unless the values
    passed for both weights are the same. Default: True.

Returns:
      the result of calling "module".
Return type:
      Any
torch.linalg.eig
torch.linalg.eig(A, *, out=None)
Computes the eigenvalue decomposition of a square matrix if it
   exists.
Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, the eigenvalue
   decomposition of a square matrix A \in \mathbb{K}^{n \times n}
   (if it exists) is defined as
  A = V \operatorname{diag}(\Lambda) V^{-1}\mathrlap{\qquad V \in
  \mathbb{C}^{n \times n}, \Lambda \in \mathbb{C}^n}

This decomposition exists if and only if A is diagonalizable. This
   is the case when all its eigenvalues are different.
Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
Note:
 The eigenvalues and eigenvectors of a real matrix may be complex.

Note:
 When inputs are on a CUDA device, this function synchronizes that
 device with the CPU.

Warning:
device with the CPU.
Warning:
 This function assumes that "A" is diagonalizable (for example,
 when all the eigenvalues are different). If it is not
 diagonalizable, the returned eigenvalues will be correct but A
 \neq V \operatorname{diag}(\Lambda)V^{-1}.

Warning:
 The returned eigenvectors are normalized to have norm *1*. Even
 then, the eigenvectors of a matrix are not unique, nor are they
 continuous with respect to "A". Due to this lack of uniqueness,
 different hardware and software may compute different
 eigenvectors.This non-uniqueness is caused by the fact that
 multiplying an eigenvector by by e^{i \phi}, \phi \in \mathbb{R}
 produces another set of valid eigenvectors of the matrix.  For
 this reason, the loss function shall not depend on the phase of
 the eigenvectors, as this quantity is not well-defined. This is
 checked when computing the gradients of this function. As such,
when inputs are on a CUDA device, this function synchronizes that
     device with the CPU when computing the gradients. This is checked
     when computing the gradients of this function. As such, when
     inputs are on a CUDA device, the computation of the gradients of
     this function synchronizes that device with the CPU.
Warning:
 Gradients computed using the *eigenvectors* tensor will only be
 finite when "A" has distinct eigenvalues. Furthermore, if the
 distance between any two eigenvalues is close to zero, the
 gradient will be numerically unstable, as it depends on the
 eigenvalues \lambda_i through the computation of \frac{1}{\min_{i
 \neq j} \lambda_i - \lambda_j}.

See also:
 "torch.linalg.eigvals()" computes only the eigenvalues. Unlike
 "torch.linalg.eig()", the gradients of "eigvals()" are always
 numerically stable.

 "torch.linalg.eigh()" for a (faster) function that computes the
eigenvalue decomposition for Hermitian and symmetric matrices.
 "torch.linalg.svd()" for a function that computes another type of
 spectral decomposition that works on matrices of any shape.

 "torch.linalg.qr()" for another (much faster) decomposition that
 works on matrices of any shape.

Parameters:
      A (Tensor) -- tensor of shape (, n, n)* where *** is
      zero or more batch dimensions consisting of diagonalizable
      matrices.
Keyword Arguments:
      out (tuple, optional) -- output tuple of two tensors.
      Ignored if None. Default: None.
Returns:
      A named tuple (eigenvalues, eigenvectors) which corresponds to
      \Lambda and V above.
  *eigenvalues* and *eigenvectors* will always be complex-valued,
  even when "A" is real. The eigenvectors will be given by the
  columns of *eigenvectors*.

Examples:
  >>> A = torch.randn(2, 2, dtype=torch.complex128)
  >>> A
A
      tensor([[ 0.9828+0.3889j, -0.4617+0.3010j],
              [ 0.1662-0.7435j, -0.6139+0.0562j]], dtype=torch.complex128)
      >>> L, V = torch.linalg.eig(A)
      >>> L
      tensor([ 1.1226+0.5738j, -0.7537-0.1286j], dtype=torch.complex128)
      >>> V
      tensor([[ 0.9218+0.0000j,  0.1882-0.2220j],
              [-0.0270-0.3867j,  0.9567+0.0000j]], dtype=torch.complex128)
      >>> torch.dist(V @ torch.diag(L) @ torch.linalg.inv(V), A)
      tensor(7.7119e-16, dtype=torch.float64)



  >>> A = torch.randn(3, 2, 2, dtype=torch.float64)
  >>> L, V = torch.linalg.eig(A)
  >>> torch.dist(V @ torch.diag_embed(L) @ torch.linalg.inv(V), A)
  tensor(3.2841e-16, dtype=torch.float64)
DataParallel
class torch.nn.DataParallel(module, device_ids=None, output_device=None, dim=0)
Implements data parallelism at the module level.
This container parallelizes the application of the given "module"
   by splitting the input across the specified devices by chunking in
   the batch dimension (other objects will be copied once per device).
   In the forward pass, the module is replicated on each device, and
   each replica handles a portion of the input. During the backwards
   pass, gradients from each replica are summed into the original
   module.
The batch size should be larger than the number of GPUs used.
Warning:
 It is recommended to use "DistributedDataParallel", instead of
 this class, to do multi-GPU training, even if there is only a
 single node. See: Use nn.parallel.DistributedDataParallel instead
 of multiprocessing or nn.DataParallel and Distributed Data
 Parallel.

Arbitrary positional and keyword inputs are allowed to be passed
into DataParallel but some types are specially handled. tensors
   will be scattered on dim specified (default 0). tuple, list and
   dict types will be shallow copied. The other types will be shared
   among different threads and can be corrupted if written to in the
   model's forward pass.
The parallelized "module" must have its parameters and buffers on
   "device_ids[0]" before running this "DataParallel" module.
Warning:
 In each forward, "module" is **replicated** on each device, so
 any updates to the running module in "forward" will be lost. For
 example, if "module" has a counter attribute that is incremented
 in each "forward", it will always stay at the initial value
 because the update is done on the replicas which are destroyed
 after "forward". However, "DataParallel" guarantees that the
 replica on "device[0]" will have its parameters and buffers
 sharing storage with the base parallelized "module". So **in-
place** updates to the parameters or buffers on "device[0]" will
     be recorded. E.g., "BatchNorm2d" and "spectral_norm()" rely on
     this behavior to update the buffers.
Warning:
 Forward and backward hooks defined on "module" and its submodules
 will be invoked "len(device_ids)" times, each with inputs located
 on a particular device. Particularly, the hooks are only
 guaranteed to be executed in correct order with respect to
 operations on corresponding devices. For example, it is not
 guaranteed that hooks set via "register_forward_pre_hook()" be
 executed before *all* "len(device_ids)" "forward()" calls, but
 that each such hook be executed before the corresponding
 "forward()" call of that device.

Warning:
 When "module" returns a scalar (i.e., 0-dimensional tensor) in
 "forward()", this wrapper will return a vector of length equal to
 number of devices used in data parallelism, containing the result
 from each device.

Note:
from each device.
Note:
 There is a subtlety in using the "pack sequence -> recurrent
 network -> unpack sequence" pattern in a "Module" wrapped in
 "DataParallel". See My recurrent network doesn't work with data
 parallelism section in FAQ for details.

Parameters:
      * module (Module) -- module to be parallelized
  * **device_ids** (*list of python:int** or **torch.device*) --
    CUDA devices (default: all devices)

  * **output_device** (*int** or **torch.device*) -- device
    location of output (default: device_ids[0])

Variables:
      module (Module) -- the module to be parallelized
Example:
  >>> net = torch.nn.DataParallel(model, device_ids=[0, 1, 2])
  >>> output = net(input_var)  # input_var can be on any device, including CPU
GLU
class torch.nn.GLU(dim=- 1)
Applies the gated linear unit function {GLU}(a, b)= a \otimes
   \sigma(b) where a is the first half of the input matrices and b is
   the second half.
Parameters:
      dim (int) -- the dimension on which to split the input.
      Default: -1
Shape:
      * Input: (\ast_1, N, \ast_2) where *** means, any number of
        additional dimensions
  * Output: (\ast_1, M, \ast_2) where M=N/2

Examples:
  >>> m = nn.GLU()
  >>> input = torch.randn(4, 2)
  >>> output = m(input)
ReflectionPad1d
class torch.nn.ReflectionPad1d(padding)
Pads the input tensor using the reflection of the input boundary.
For N-dimensional padding, use "torch.nn.functional.pad()".
Parameters:
      padding (int, tuple) -- the size of the padding. If is
      int, uses the same padding in all boundaries. If a 2-tuple,
      uses (\text{padding_left}, \text{padding_right})
Shape:
      * Input: (C, W_{in}) or (N, C, W_{in}).
  * Output: (C, W_{out}) or (N, C, W_{out}), where

    W_{out} = W_{in} + \text{padding\_left} +
    \text{padding\_right}

Examples:
  >>> m = nn.ReflectionPad1d(2)
  >>> input = torch.arange(8, dtype=torch.float).reshape(1, 2, 4)
  >>> input
  tensor([[[0., 1., 2., 3.],
           [4., 5., 6., 7.]]])
  >>> m(input)
  tensor([[[2., 1., 0., 1., 2., 3., 2., 1.],
           [6., 5., 4., 5., 6., 7., 6., 5.]]])
  >>> # using different paddings for different sides
conv1d
class torch.ao.nn.quantized.functional.conv1d(input, weight, bias, stride=1, padding=0, dilation=1, groups=1, padding_mode='zeros', scale=1.0, zero_point=0, dtype=torch.quint8)
Applies a 1D convolution over a quantized 1D input composed of
   several input planes.
See "Conv1d" for details and output shape.
Parameters:
      * input -- quantized input tensor of shape (\text{minibatch}
        , \text{in_channels} , iW)
  * **weight** -- quantized filters of shape (\text{out\_channels}
    , \frac{\text{in\_channels}}{\text{groups}} , iW)

  * **bias** -- **non-quantized** bias tensor of shape
    (\text{out\_channels}). The tensor type must be *torch.float*.

  * **stride** -- the stride of the convolving kernel. Can be a
    single number or a tuple *(sW,)*. Default: 1

  * **padding** -- implicit paddings on both sides of the input.
    Can be a single number or a tuple *(padW,)*. Default: 0
dilation -- the spacing between kernel elements. Can be a
        single number or a tuple (dW,). Default: 1


groups -- split input into groups, \text{in_channels}
    should be divisible by the number of groups. Default: 1


padding_mode -- the padding mode to use. Only "zeros" is
    supported for quantized convolution at the moment. Default:
    "zeros"


scale -- quantization scale for the output. Default: 1.0


zero_point -- quantization zero_point for the output.
    Default: 0


dtype -- quantization data type to use. Default:
    "torch.quint8"




Examples:
  >>> from torch.ao.nn.quantized import functional as qF
  >>> filters = torch.randn(33, 16, 3, dtype=torch.float)
  >>> inputs = torch.randn(20, 16, 50, dtype=torch.float)
  >>> bias = torch.randn(33, dtype=torch.float)
  >>>
  >>> scale, zero_point = 1.0, 0
  >>> dtype_inputs = torch.quint8
dtype_inputs = torch.quint8
      >>> dtype_filters = torch.qint8
      >>>
      >>> q_filters = torch.quantize_per_tensor(filters, scale, zero_point, dtype_filters)
      >>> q_inputs = torch.quantize_per_tensor(inputs, scale, zero_point, dtype_inputs)
      >>> qF.conv1d(q_inputs, q_filters, bias, padding=1, scale=scale, zero_point=zero_point)
conv2d
class torch.ao.nn.quantized.functional.conv2d(input, weight, bias, stride=1, padding=0, dilation=1, groups=1, padding_mode='zeros', scale=1.0, zero_point=0, dtype=torch.quint8)
Applies a 2D convolution over a quantized 2D input composed of
   several input planes.
See "Conv2d" for details and output shape.
Parameters:
      * input -- quantized input tensor of shape (\text{minibatch}
        , \text{in_channels} , iH , iW)
  * **weight** -- quantized filters of shape (\text{out\_channels}
    , \frac{\text{in\_channels}}{\text{groups}} , kH , kW)

  * **bias** -- **non-quantized** bias tensor of shape
    (\text{out\_channels}). The tensor type must be *torch.float*.

  * **stride** -- the stride of the convolving kernel. Can be a
    single number or a tuple *(sH, sW)*. Default: 1

  * **padding** -- implicit paddings on both sides of the input.
    Can be a single number or a tuple *(padH, padW)*. Default: 0
dilation -- the spacing between kernel elements. Can be a
        single number or a tuple (dH, dW). Default: 1


groups -- split input into groups, \text{in_channels}
    should be divisible by the number of groups. Default: 1


padding_mode -- the padding mode to use. Only "zeros" is
    supported for quantized convolution at the moment. Default:
    "zeros"


scale -- quantization scale for the output. Default: 1.0


zero_point -- quantization zero_point for the output.
    Default: 0


dtype -- quantization data type to use. Default:
    "torch.quint8"




Examples:
  >>> from torch.ao.nn.quantized import functional as qF
  >>> filters = torch.randn(8, 4, 3, 3, dtype=torch.float)
  >>> inputs = torch.randn(1, 4, 5, 5, dtype=torch.float)
  >>> bias = torch.randn(8, dtype=torch.float)
  >>>
  >>> scale, zero_point = 1.0, 0
  >>> dtype_inputs = torch.quint8
dtype_inputs = torch.quint8
      >>> dtype_filters = torch.qint8
      >>>
      >>> q_filters = torch.quantize_per_tensor(filters, scale, zero_point, dtype_filters)
      >>> q_inputs = torch.quantize_per_tensor(inputs, scale, zero_point, dtype_inputs)
      >>> qF.conv2d(q_inputs, q_filters, bias, padding=1, scale=scale, zero_point=zero_point)
torch.manual_seed
torch.manual_seed(seed)
Sets the seed for generating random numbers. Returns a
   torch.Generator object.
Parameters:
      seed (int) -- The desired seed. Value must be within the
      inclusive range [-0x8000_0000_0000_0000,
      0xffff_ffff_ffff_ffff]. Otherwise, a RuntimeError is raised.
      Negative inputs are remapped to positive values with the formula
      0xffff_ffff_ffff_ffff + seed.
Return type:
      Generator
torch.Tensor.register_hook
Tensor.register_hook(hook)
Registers a backward hook.
The hook will be called every time a gradient with respect to the
   Tensor is computed. The hook should have the following signature:
  hook(grad) -> Tensor or None

The hook should not modify its argument, but it can optionally
   return a new gradient which will be used in place of "grad".
This function returns a handle with a method "handle.remove()" that
   removes the hook from the module.
Note:
 See Backward Hooks execution for more information on how when
 this hook is executed, and how its execution is ordered relative
 to other hooks.

Example:
  >>> v = torch.tensor([0., 0., 0.], requires_grad=True)
  >>> h = v.register_hook(lambda grad: grad * 2)  # double the gradient
  >>> v.backward(torch.tensor([1., 2., 3.]))
  >>> v.grad

   2
   4
   6
  [torch.FloatTensor of size (3,)]

  >>> h.remove()  # removes the hook
torch.set_warn_always
torch.set_warn_always(b)
When this flag is False (default) then some PyTorch warnings may
   only appear once per process. This helps avoid excessive warning
   information. Setting it to True causes these warnings to always
   appear, which may be helpful when debugging.
Parameters:
      b ("bool") -- If True, force warnings to always be emitted
      If False, set to the default behaviour
torch.nn.functional.pixel_unshuffle
torch.nn.functional.pixel_unshuffle(input, downscale_factor) -> Tensor
Reverses the "PixelShuffle" operation by rearranging elements in a
   tensor of shape (, C, H \times r, W \times r) to a tensor of shape
   (, C \times r^2, H, W), where r is the "downscale_factor".
See "PixelUnshuffle" for details.
Parameters:
      * input (Tensor) -- the input tensor
  * **downscale_factor** (*int*) -- factor to increase spatial
    resolution by

Examples:
  >>> input = torch.randn(1, 1, 12, 12)
  >>> output = torch.nn.functional.pixel_unshuffle(input, 3)
  >>> print(output.size())
  torch.Size([1, 9, 4, 4])
Conv2d
class torch.ao.nn.quantized.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)
Applies a 2D convolution over a quantized input signal composed of
   several quantized input planes.
For details on input arguments, parameters, and implementation see
   "Conv2d".
Note:
 Only *zeros* is supported for the "padding_mode" argument.

Note:
 Only *torch.quint8* is supported for the input data type.

Variables:
      * weight (Tensor) -- packed tensor derived from the
        learnable weight parameter.
  * **scale** (*Tensor*) -- scalar for the output scale

  * **zero_point** (*Tensor*) -- scalar for the output zero point

See "Conv2d" for other attributes.
Examples:
  >>> # With square kernels and equal stride
  >>> m = nn.quantized.Conv2d(16, 33, 3, stride=2)
  >>> # non-square kernels and unequal stride and with padding
m = nn.quantized.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
      >>> # non-square kernels and unequal stride and with padding and dilation
      >>> m = nn.quantized.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
      >>> input = torch.randn(20, 16, 50, 100)
      >>> # quantize input to quint8
      >>> q_input = torch.quantize_per_tensor(input, scale=1.0, zero_point=0, dtype=torch.quint8)
      >>> output = m(q_input)



classmethod from_float(mod)
  Creates a quantized module from a float module or qparams_dict.

  Parameters:
     **mod** (*Module*) -- a float module, either produced by
     torch.ao.quantization utilities or provided by the user
torch.bitwise_or
torch.bitwise_or(input, other, *, out=None) -> Tensor
Computes the bitwise OR of "input" and "other". The input tensor
   must be of integral or Boolean types. For bool tensors, it computes
   the logical OR.
Parameters:
      * input -- the first input tensor
  * **other** -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.bitwise_or(torch.tensor([-1, -2, 3], dtype=torch.int8), torch.tensor([1, 0, 3], dtype=torch.int8))
  tensor([-1, -2,  3], dtype=torch.int8)
  >>> torch.bitwise_or(torch.tensor([True, True, False]), torch.tensor([False, True, False]))
  tensor([ True, True, False])
torch.unsqueeze
torch.unsqueeze(input, dim) -> Tensor
Returns a new tensor with a dimension of size one inserted at the
   specified position.
The returned tensor shares the same underlying data with this
   tensor.
A "dim" value within the range "[-input.dim() - 1, input.dim() +
   1)" can be used. Negative "dim" will correspond to "unsqueeze()"
   applied at "dim" = "dim + input.dim() + 1".
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int*) -- the index at which to insert the singleton
    dimension

Example:
  >>> x = torch.tensor([1, 2, 3, 4])
  >>> torch.unsqueeze(x, 0)
  tensor([[ 1,  2,  3,  4]])
  >>> torch.unsqueeze(x, 1)
  tensor([[ 1],
          [ 2],
          [ 3],
          [ 4]])
torch.square
torch.square(input, *, out=None) -> Tensor
Returns a new tensor with the square of the elements of "input".
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([-2.0755,  1.0226,  0.0831,  0.4806])
  >>> torch.square(a)
  tensor([ 4.3077,  1.0457,  0.0069,  0.2310])