shrinath-suresh/blogs-docs-splitted huggingface dataset:
QConfig
class torch.quantization.qconfig.QConfig(activation, weight)
Describes how to quantize a layer or a part of the network by
   providing settings (observer classes) for activations and weights
   respectively.
Note that QConfig needs to contain observer classes (like
   MinMaxObserver) or a callable that returns instances on invocation,
   not the concrete observer instances themselves. Quantization
   preparation function will instantiate observers multiple times for
   each of the layers.
Observer classes have usually reasonable default arguments, but
   they can be overwritten with with_args method (that behaves like
   functools.partial):
  my_qconfig = QConfig(
      activation=MinMaxObserver.with_args(dtype=torch.qint8),
      weight=default_observer.with_args(dtype=torch.qint8))
torch.nn.functional.gelu
torch.nn.functional.gelu(input, approximate='none') -> Tensor
When the approximate argument is 'none', it applies element-wise
   the function \text{GELU}(x) = x * \Phi(x)
where \Phi(x) is the Cumulative Distribution Function for Gaussian
   Distribution.
When the approximate argument is 'tanh', Gelu is estimated with
  \text{GELU}(x) = 0.5 * x * (1 + \text{Tanh}(\sqrt(2 / \pi) * (x
  + 0.044715 * x^3)))

See Gaussian Error Linear Units (GELUs).
torch.linalg.ldl_factor
torch.linalg.ldl_factor(A, *, hermitian=False, out=None)
Computes a compact representation of the LDL factorization of a
   Hermitian or symmetric (possibly indefinite) matrix.
When "A" is complex valued it can be Hermitian ("hermitian"=
   True) or symmetric ("hermitian"= False).
The factorization is of the form the form A = L D L^T. If
   "hermitian" is True then transpose operation is the conjugate
   transpose.
L (or U) and D are stored in compact form in "LD". They follow the
   format specified by LAPACK's sytrf function. These tensors may be
   used in "torch.linalg.ldl_solve()" to solve linear systems.
Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
Note:
 When inputs are on a CUDA device, this function synchronizes that
device with the CPU. For a version of this function that does not
     synchronize, see "torch.linalg.ldl_factor_ex()".
Parameters:
      A (Tensor) -- tensor of shape (*, n, n) where * is zero or
      more batch dimensions consisting of symmetric or Hermitian
      matrices. (*, n, n) where *** is one or more batch dimensions.
Keyword Arguments:
      * hermitian (bool, optional) -- whether to consider
        the input to be Hermitian or symmetric. For real-valued
        matrices, this switch has no effect. Default: False.
  * **out** (*tuple**, **optional*) -- tuple of two tensors to
    write the output to. Ignored if *None*. Default: *None*.

Returns:
      A named tuple (LD, pivots).
Examples:
  >>> A = torch.randn(3, 3)
  >>> A = A @ A.mT # make symmetric
  >>> A
  tensor([[7.2079, 4.2414, 1.9428],
          [4.2414, 3.4554, 0.3264],
          [1.9428, 0.3264, 1.3823]])
  >>> LD, pivots = torch.linalg.ldl_factor(A)
LD, pivots = torch.linalg.ldl_factor(A)
      >>> LD
      tensor([[ 7.2079,  0.0000,  0.0000],
              [ 0.5884,  0.9595,  0.0000],
              [ 0.2695, -0.8513,  0.1633]])
      >>> pivots
      tensor([1, 2, 3], dtype=torch.int32)
torch.cuda.change_current_allocator
torch.cuda.change_current_allocator(allocator)
Changes the currently used memory allocator to be the one provided.
   If the current allocator has already been used/initialized, this
   function will error.
Parameters:
      allocator (torch.cuda.memory._CUDAAllocator) -- allocator
      to be set as the active one.
Note:
 See Memory management for details on creating and using a custom
 allocator
torch.bitwise_not
torch.bitwise_not(input, *, out=None) -> Tensor
Computes the bitwise NOT of the given input tensor. The input
   tensor must be of integral or Boolean types. For bool tensors, it
   computes the logical NOT.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.bitwise_not(torch.tensor([-1, -2, 3], dtype=torch.int8))
  tensor([ 0,  1, -4], dtype=torch.int8)
torch.atleast_2d
torch.atleast_2d(*tensors)
Returns a 2-dimensional view of each input tensor with zero
   dimensions. Input tensors with two or more dimensions are returned
   as-is.
Parameters:
      input (Tensor or list of Tensors) --
Returns:
      output (Tensor or tuple of Tensors)
Example:
  >>> x = torch.tensor(1.)
  >>> x
  tensor(1.)
  >>> torch.atleast_2d(x)
  tensor([[1.]])
  >>> x = torch.arange(4).view(2, 2)
  >>> x
  tensor([[0, 1],
          [2, 3]])
  >>> torch.atleast_2d(x)
  tensor([[0, 1],
          [2, 3]])
  >>> x = torch.tensor(0.5)
  >>> y = torch.tensor(1.)
  >>> torch.atleast_2d((x, y))
  (tensor([[0.5000]]), tensor([[1.]]))
torch.nn.functional.dropout1d
torch.nn.functional.dropout1d(input, p=0.5, training=True, inplace=False)
Randomly zero out entire channels (a channel is a 1D feature map,
   e.g., the j-th channel of the i-th sample in the batched input is a
   1D tensor \text{input}[i, j]) of the input tensor). Each channel
   will be zeroed out independently on every forward call with
   probability "p" using samples from a Bernoulli distribution.
See "Dropout1d" for details.
Parameters:
      * p (float) -- probability of a channel to be zeroed.
        Default: 0.5
  * **training** (*bool*) -- apply dropout if is "True". Default:
    "True"

  * **inplace** (*bool*) -- If set to "True", will do this
    operation in-place. Default: "False"

Return type:
      Tensor
torch.signal.windows.exponential
torch.signal.windows.exponential(M, *, center=None, tau=1.0, sym=True, dtype=None, layout=torch.strided, device=None, requires_grad=False)
Computes a window with an exponential waveform. Also known as
   Poisson window.
The exponential window is defined as follows:
  w_n = \exp{\left(-\frac{|n - c|}{\tau}\right)}

where c is the "center" of the window.
The window is normalized to 1 (maximum value is 1). However, the 1
   doesn't appear if "M" is even and "sym" is True.
Parameters:
      M (int) -- the length of the window. In other words, the
      number of points of the returned window.
Keyword Arguments:
      * center (float, optional) -- where the center of the
        window will be located. Default: M / 2 if sym is False,
        else (M - 1) / 2.
  * **tau** (*float**, **optional*) -- the decay value. Tau is
    generally associated with a percentage, that means, that the
value should vary within the interval (0, 100]. If tau is 100,
        it is considered the uniform window. Default: 1.0.
  * **sym** (*bool**, **optional*) -- If *False*, returns a
    periodic window suitable for use in spectral analysis. If
    *True*, returns a symmetric window suitable for use in filter
    design. Default: *True*.

  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
    returned Tensor. Default: "torch.strided".

  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.
tensor types.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Return type:
      Tensor
Examples:
  >>> # Generates a symmetric exponential window of size 10 and with a decay value of 1.0.
  >>> # The center will be at (M - 1) / 2, where M is 10.
  >>> torch.signal.windows.exponential(10)
  tensor([0.0111, 0.0302, 0.0821, 0.2231, 0.6065, 0.6065, 0.2231, 0.0821, 0.0302, 0.0111])

  >>> # Generates a periodic exponential window and decay factor equal to .5
  >>> torch.signal.windows.exponential(10, sym=False,tau=.5)
  tensor([4.5400e-05, 3.3546e-04, 2.4788e-03, 1.8316e-02, 1.3534e-01, 1.0000e+00, 1.3534e-01, 1.8316e-02, 2.4788e-03, 3.3546e-04])
torch.ne
torch.ne(input, other, *, out=None) -> Tensor
Computes \text{input} \neq \text{other} element-wise.
The second argument can be a number or a tensor whose shape is
   broadcastable with the first argument.
Parameters:
      * input (Tensor) -- the tensor to compare
  * **other** (*Tensor** or **float*) -- the tensor or value to
    compare

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Returns:
      A boolean tensor that is True where "input" is not equal to
      "other" and False elsewhere
Example:
  >>> torch.ne(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
  tensor([[False, True], [True, False]])
torch.set_float32_matmul_precision
torch.set_float32_matmul_precision(precision)
Sets the internal precision of float32 matrix multiplications.
Running float32 matrix multiplications in lower precision may
   significantly increase performance, and in some programs the loss
   of precision has a negligible impact.
Supports three settings:
  * "highest", float32 matrix multiplications use the float32
    datatype for internal computations.

  * "high", float32 matrix multiplications use the TensorFloat32
    or bfloat16_3x datatypes for internal computations, if fast
    matrix multiplication algorithms using those datatypes
    internally are available. Otherwise float32 matrix
    multiplications are computed as if the precision is "highest".

  * "medium", float32 matrix multiplications use the bfloat16
    datatype for internal computations, if a fast matrix
multiplication algorithm using that datatype internally is
        available. Otherwise float32 matrix multiplications are
        computed as if the precision is "high".
Note:
 This does not change the output dtype of float32 matrix
 multiplications, it controls how the internal computation of the
 matrix multiplication is performed.

Note:
 This does not change the precision of convolution operations.
 Other flags, like *torch.backends.cudnn.allow_tf32*, may control
 the precision of convolution operations.

Note:
 This flag currently only affects one native device type: CUDA. If
 "high" or "medium" are set then the TensorFloat32 datatype will
 be used when computing float32 matrix multiplications, equivalent
 to setting *torch.backends.cuda.matmul.allow_tf32 = True*. When
 "highest" (the default) is set then the float32 datatype is used
 for internal computations, equivalent to setting
 *torch.backends.cuda.matmul.allow_tf32 = False*.
Linear
class torch.ao.nn.qat.Linear(in_features, out_features, bias=True, qconfig=None, device=None, dtype=None)
A linear module attached with FakeQuantize modules for weight, used
   for quantization aware training.
We adopt the same interface as torch.nn.Linear, please see
   https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for
   documentation.
Similar to torch.nn.Linear, with FakeQuantize modules initialized
   to default.
Variables:
      weight (torch.Tensor) -- fake quant module for weight
classmethod from_float(mod)
  Create a qat module from a float module or qparams_dict Args:
  *mod* a float module, either produced by torch.ao.quantization
  utilities or directly from user
LazyModuleMixin
class torch.nn.modules.lazy.LazyModuleMixin(args, *kwargs)
A mixin for modules that lazily initialize parameters, also known
   as "lazy modules."
Modules that lazily initialize parameters, or "lazy modules",
   derive the shapes of their parameters from the first input(s) to
   their forward method. Until that first forward they contain
   "torch.nn.UninitializedParameter" s that should not be accessed or
   used, and afterward they contain regular "torch.nn.Parameter" s.
   Lazy modules are convenient since they don't require computing some
   module arguments, like the "in_features" argument of a typical
   "torch.nn.Linear".
After construction, networks with lazy modules should first be
   converted to the desired dtype and placed on the expected device.
   This is because lazy modules only perform shape inference so the
   usual dtype and device placement behavior applies. The lazy modules
   should then perform "dry runs" to initialize all the components in
the module. These "dry runs" send inputs of the correct size,
   dtype, and device through the network and to each one of its lazy
   modules. After this the network can be used as usual.



class LazyMLP(torch.nn.Module):
   ...    def init(self):
   ...        super().init()
   ...        self.fc1 = torch.nn.LazyLinear(10)
   ...        self.relu1 = torch.nn.ReLU()
   ...        self.fc2 = torch.nn.LazyLinear(1)
   ...        self.relu2 = torch.nn.ReLU()
   ...
   ...    def forward(self, input):
   ...        x = self.relu1(self.fc1(input))
   ...        y = self.relu2(self.fc2(x))
   ...        return y
constructs a network with lazy modules
lazy_mlp = LazyMLP()
transforms the network's device and dtype
NOTE: these transforms can and should be applied after construction and before any 'dry runs'
lazy_mlp = lazy_mlp.cuda().double()
lazy_mlp
   LazyMLP( (fc1): LazyLinear(in_features=0, out_features=10, bias=True)
     (relu1): ReLU()
(relu1): ReLU()
     (fc2): LazyLinear(in_features=0, out_features=1, bias=True)
     (relu2): ReLU()
   )



performs a dry run to initialize the network's lazy modules
lazy_mlp(torch.ones(10,10).cuda())
after initialization, LazyLinear modules become regular Linear modules
lazy_mlp
   LazyMLP(
     (fc1): Linear(in_features=10, out_features=10, bias=True)
     (relu1): ReLU()
     (fc2): Linear(in_features=10, out_features=1, bias=True)
     (relu2): ReLU()
   )
attaches an optimizer, since parameters can now be used as usual
optim = torch.optim.SGD(mlp.parameters(), lr=0.01)



A final caveat when using lazy modules is that the order of
   initialization of a network's parameters may change, since the lazy
   modules are always initialized after other modules. For example, if
   the LazyMLP class defined above had a "torch.nn.LazyLinear" module
   first and then a regular "torch.nn.Linear" second, the second
module would be initialized on construction and the first module
   would be initialized during the first dry run. This can cause the
   parameters of a network using lazy modules to be initialized
   differently than the parameters of a network without lazy modules
   as the order of parameter initializations, which often depends on a
   stateful random number generator, is different. Check
   Reproducibility for more details.
Lazy modules can be serialized with a state dict like other
   modules. For example:



lazy_mlp = LazyMLP()
The state dict shows the uninitialized parameters
lazy_mlp.state_dict()
   OrderedDict([('fc1.weight', Uninitialized parameter),
                ('fc1.bias',
                 tensor([-1.8832e+25,  4.5636e-41, -1.8832e+25,  4.5636e-41, -6.1598e-30,
                          4.5637e-41, -1.8788e+22,  4.5636e-41, -2.0042e-31,  4.5637e-41])),
                ('fc2.weight', Uninitialized parameter),
                ('fc2.bias', tensor([0.0019]))])
('fc2.bias', tensor([0.0019]))])
Lazy modules can load regular "torch.nn.Parameter" s (i.e. you can
   serialize/deserialize initialized LazyModules and they will remain
   initialized)



full_mlp = LazyMLP()
Dry run to initialize another module
full_mlp.forward(torch.ones(10, 1))
Load an initialized state into a lazy module
lazy_mlp.load_state_dict(full_mlp.state_dict())
The state dict now holds valid values
lazy_mlp.state_dict()
   OrderedDict([('fc1.weight',
                 tensor([[-0.3837],
                         [ 0.0907],
                         [ 0.6708],
                         [-0.5223],
                         [-0.9028],
                         [ 0.2851],
                         [-0.4537],
                         [ 0.6813],
                         [ 0.5766],
                         [-0.8678]])),
                ('fc1.bias',
                 tensor([-1.8832e+25,  4.5636e-41, -1.8832e+25,  4.5636e-41, -6.1598e-30,
4.5637e-41, -1.8788e+22,  4.5636e-41, -2.0042e-31,  4.5637e-41])),
                ('fc2.weight',
                 tensor([[ 0.1320,  0.2938,  0.0679,  0.2793,  0.1088, -0.1795, -0.2301,  0.2807,
                           0.2479,  0.1091]])),
                ('fc2.bias', tensor([0.0019]))])
Note, however, that the loaded parameters will not be replaced when
   doing a "dry run" if they are initialized when the state is loaded.
   This prevents using initialized modules in different contexts.
has_uninitialized_params()
  Check if a module has parameters that are not initialized

initialize_parameters(args, *kwargs)
  Initialize parameters according to the input batch properties.
  This adds an interface to isolate parameter initialization from
  the forward pass when doing parameter shape inference.
torch.fft.rfft
torch.fft.rfft(input, n=None, dim=- 1, norm=None, *, out=None) -> Tensor
Computes the one dimensional Fourier transform of real-valued
   "input".
The FFT of a real signal is Hermitian-symmetric, "X[i] =
   conj(X[-i])" so the output contains only the positive frequencies
   below the Nyquist frequency. To compute the full output, use
   "fft()"
Note:
 Supports torch.half on CUDA with GPU Architecture SM53 or
 greater. However it only supports powers of 2 signal length in
 every transformed dimension.

Parameters:
      * input (Tensor) -- the real input tensor
  * **n** (*int**, **optional*) -- Signal length. If given, the
    input will either be zero-padded or trimmed to this length
    before computing the real FFT.

  * **dim** (*int**, **optional*) -- The dimension along which to
    take the one dimensional real FFT.

  * **norm** (*str**, **optional*) --
norm (str, optional) --Normalization mode. For the forward transform ("rfft()"),
these correspond to:

* ""forward"" - normalize by "1/n"

* ""backward"" - no normalization

* ""ortho"" - normalize by "1/sqrt(n)" (making the FFT
  orthonormal)

Calling the backward transform ("irfft()") with the same
normalization mode will apply an overall normalization of
"1/n" between the two transforms. This is required to make
"irfft()" the exact inverse.

Default is ""backward"" (no normalization).



Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-



t = torch.arange(4)
t
   tensor([0, 1, 2, 3])
torch.fft.rfft(t)
   tensor([ 6.+0.j, -2.+2.j, -2.+0.j])



Compare against the full output from "fft()":



torch.fft.fft(t)
   tensor([ 6.+0.j, -2.+2.j, -2.+0.j, -2.-2.j])



Notice that the symmetric element "T[-1] == T[1].conj()" is
torch.nanmean
torch.nanmean(input, dim=None, keepdim=False, *, dtype=None, out=None) -> Tensor
Computes the mean of all non-NaN elements along the specified
   dimensions.
This function is identical to "torch.mean()" when there are no
   NaN values in the "input" tensor. In the presence of NaN,
   "torch.mean()" will propagate the NaN to the output whereas
   "torch.nanmean()" will ignore the NaN values (torch.nanmean(a)
   is equivalent to torch.mean(a[~a.isnan()])).
If "keepdim" is "True", the output tensor is of the same size as
   "input" except in the dimension(s) "dim" where it is of size 1.
   Otherwise, "dim" is squeezed (see "torch.squeeze()"), resulting in
   the output tensor having 1 (or "len(dim)") fewer dimension(s).
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int** or **tuple of ints**, **optional*) -- the
    dimension or dimensions to reduce. If "None", all dimensions
    are reduced.
are reduced.
  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not.

Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. If specified, the input tensor is casted
        to "dtype" before the operation is performed. This is useful
        for preventing data type overflows. Default: None.
  * **out** (*Tensor**, **optional*) -- the output tensor.

See also:
 "torch.mean()" computes the mean value, propagating *NaN*.

Example:
  >>> x = torch.tensor([[torch.nan, 1, 2], [1, 2, 3]])
  >>> x.mean()
  tensor(nan)
  >>> x.nanmean()
  tensor(1.8000)
  >>> x.mean(dim=0)
  tensor([   nan, 1.5000, 2.5000])
  >>> x.nanmean(dim=0)
  tensor([1.0000, 1.5000, 2.5000])

  # If all elements in the reduced dimensions are NaN then the result is NaN
  >>> torch.tensor([torch.nan]).nanmean()
  tensor(nan)
Identity
class torch.nn.utils.prune.Identity
Utility pruning method that does not prune any units but generates
   the pruning parametrization with a mask of ones.
classmethod apply(module, name)
  Adds the forward pre-hook that enables pruning on the fly and
  the reparametrization of a tensor in terms of the original
  tensor and the pruning mask.

  Parameters:
     * **module** (*nn.Module*) -- module containing the tensor to
       prune

     * **name** (*str*) -- parameter name within "module" on which
       pruning will act.

apply_mask(module)
  Simply handles the multiplication between the parameter being
  pruned and the generated mask. Fetches the mask and the original
  tensor from the module and returns the pruned version of the
  tensor.

  Parameters:
     **module** (*nn.Module*) -- module containing the tensor to
     prune

  Returns:
     pruned version of the input tensor

  Return type:
Return type:
         pruned_tensor (torch.Tensor)
prune(t, default_mask=None, importance_scores=None)
  Computes and returns a pruned version of input tensor "t"
  according to the pruning rule specified in "compute_mask()".

  Parameters:
     * **t** (*torch.Tensor*) -- tensor to prune (of same
       dimensions as "default_mask").

     * **importance_scores** (*torch.Tensor*) -- tensor of
       importance scores (of same shape as "t") used to compute
       mask for pruning "t". The values in this tensor indicate
       the importance of the corresponding elements in the "t"
       that is being pruned. If unspecified or None, the tensor
       "t" will be used in its place.

     * **default_mask** (*torch.Tensor**, **optional*) -- mask
       from previous pruning iteration, if any. To be considered
       when determining what portion of the tensor that pruning
       should act on. If None, default to a mask of ones.
Returns:
         pruned version of tensor "t".
remove(module)
  Removes the pruning reparameterization from a module. The pruned
  parameter named "name" remains permanently pruned, and the
  parameter named "name+'_orig'" is removed from the parameter
  list. Similarly, the buffer named "name+'_mask'" is removed from
  the buffers.

  Note:

    Pruning itself is NOT undone or reversed!
torch.kthvalue
torch.kthvalue(input, k, dim=None, keepdim=False, *, out=None)
Returns a namedtuple "(values, indices)" where "values" is the "k"
   th smallest element of each row of the "input" tensor in the given
   dimension "dim". And "indices" is the index location of each
   element found.
If "dim" is not given, the last dimension of the input is chosen.
If "keepdim" is "True", both the "values" and "indices" tensors are
   the same size as "input", except in the dimension "dim" where they
   are of size 1. Otherwise, "dim" is squeezed (see
   "torch.squeeze()"), resulting in both the "values" and "indices"
   tensors having 1 fewer dimension than the "input" tensor.
Note:
 When "input" is a CUDA tensor and there are multiple valid "k" th
 values, this function may nondeterministically return "indices"
 for any of them.

Parameters:
      * input (Tensor) -- the input tensor.
  * **k** (*int*) -- k for the k-th smallest element