shrinath-suresh/blogs-docs-splitted huggingface dataset:
BatchNorm2d
class torch.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, device=None, dtype=None)
Applies Batch Normalization over a 4D input (a mini-batch of 2D
   inputs with additional channel dimension) as described in the paper
   Batch Normalization: Accelerating Deep Network Training by Reducing
   Internal Covariate Shift .
  y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}}
  * \gamma + \beta

The mean and standard-deviation are calculated per-dimension over
   the mini-batches and \gamma and \beta are learnable parameter
   vectors of size C (where C is the input size). By default, the
   elements of \gamma are set to 1 and the elements of \beta are set
   to 0. The standard-deviation is calculated via the biased
   estimator, equivalent to torch.var(input, unbiased=False).
Also by default, during training this layer keeps running estimates
   of its computed mean and variance, which are then used for
normalization during evaluation. The running estimates are kept
   with a default "momentum" of 0.1.
If "track_running_stats" is set to "False", this layer then does
   not keep running estimates, and batch statistics are instead used
   during evaluation time as well.
Note:
 This "momentum" argument is different from one used in optimizer
 classes and the conventional notion of momentum. Mathematically,
 the update rule for running statistics here is \hat{x}_\text{new}
 = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times
 x_t, where \hat{x} is the estimated statistic and x_t is the new
 observed value.

Because the Batch Normalization is done over the C dimension,
   computing statistics on (N, H, W) slices, it's common terminology
   to call this Spatial Batch Normalization.
Parameters:
      * num_features (int) -- C from an expected input of size
        (N, C, H, W)
  * **eps** (*float*) -- a value added to the denominator for
numerical stability. Default: 1e-5
  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Can be set to "None" for
    cumulative moving average (i.e. simple average). Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters. Default:
    "True"

  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and
    variance, and when set to "False", this module does not track
    such statistics, and initializes statistics buffers
    "running_mean" and "running_var" as "None". When these buffers
    are "None", this module always uses batch statistics. in both
    training and eval modes. Default: "True"

Shape:
      * Input: (N, C, H, W)
  * Output: (N, C, H, W) (same shape as input)

Examples:
  >>> # With Learnable Parameters
torch.triangular_solve
torch.triangular_solve(b, A, upper=True, transpose=False, unitriangular=False, *, out=None)
Solves a system of equations with a square upper or lower
   triangular invertible matrix A and multiple right-hand sides b.
In symbols, it solves AX = b and assumes A is square upper-
   triangular (or lower-triangular if "upper"= False) and does not
   have zeros on the diagonal.
torch.triangular_solve(b, A) can take in 2D inputs b, A or
   inputs that are batches of 2D matrices. If the inputs are batches,
   then returns batched outputs X
If the diagonal of "A" contains zeros or elements that are very
   close to zero and "unitriangular"= False (default) or if the
   input matrix is badly conditioned, the result may contain NaN s.
Supports input of float, double, cfloat and cdouble data types.
Warning:
 "torch.triangular_solve()" is deprecated in favor of
 "torch.linalg.solve_triangular()" and will be removed in a future
PyTorch release. "torch.linalg.solve_triangular()" has its
     arguments reversed and does not return a copy of one of the
     inputs."X = torch.triangular_solve(B, A).solution" should be
     replaced with
    X = torch.linalg.solve_triangular(A, B)

Parameters:
      * b (Tensor) -- multiple right-hand sides of size (*, m,
        k) where * is zero of more batch dimensions
  * **A** (*Tensor*) -- the input triangular coefficient matrix of
    size (*, m, m) where * is zero or more batch dimensions

  * **upper** (*bool**, **optional*) -- whether A is upper or
    lower triangular. Default: "True".

  * **transpose** (*bool**, **optional*) -- solves *op(A)X = b*
    where *op(A) = A^T* if this flag is "True", and *op(A) = A* if
    it is "False". Default: "False".

  * **unitriangular** (*bool**, **optional*) -- whether A is unit
    triangular. If True, the diagonal elements of A are assumed to
be 1 and not referenced from A. Default: "False".
Keyword Arguments:
      out ((Tensor, Tensor), optional) -- tuple of
      two tensors to write the output to. Ignored if None. Default:
      None.
Returns:
      A namedtuple (solution, cloned_coefficient) where
      cloned_coefficient is a clone of A and solution is the
      solution X to AX = b (or whatever variant of the system of
      equations, depending on the keyword arguments.)
Examples:
  >>> A = torch.randn(2, 2).triu()
  >>> A
  tensor([[ 1.1527, -1.0753],
          [ 0.0000,  0.7986]])
  >>> b = torch.randn(2, 3)
  >>> b
  tensor([[-0.0210,  2.3513, -1.5492],
          [ 1.5429,  0.7403, -1.0243]])
  >>> torch.triangular_solve(b, A)
  torch.return_types.triangular_solve(
  solution=tensor([[ 1.7841,  2.9046, -2.5405],
          [ 1.9320,  0.9270, -1.2826]]),
  cloned_coefficient=tensor([[ 1.1527, -1.0753],
          [ 0.0000,  0.7986]]))
torch.frombuffer
torch.frombuffer(buffer, *, dtype, count=- 1, offset=0, requires_grad=False) -> Tensor
Creates a 1-dimensional "Tensor" from an object that implements the
   Python buffer protocol.
Skips the first "offset" bytes in the buffer, and interprets the
   rest of the raw bytes as a 1-dimensional tensor of type "dtype"
   with "count" elements.
Note that either of the following must be true:


"count" is a positive non-zero number, and the total number of
   bytes in the buffer is less than "offset" plus "count" times the
   size (in bytes) of "dtype".


"count" is negative, and the length (number of bytes) of the
   buffer subtracted by the "offset" is a multiple of the size (in
   bytes) of "dtype".


The returned tensor and buffer share the same memory. Modifications
   to the tensor will be reflected in the buffer and vice versa. The
   returned tensor is not resizable.
Note:
 This function increments the reference count for the object that
owns the shared memory. Therefore, such memory will not be
     deallocated before the returned tensor goes out of scope.
Warning:
 This function's behavior is undefined when passed an object
 implementing the buffer protocol whose data is not on the CPU.
 Doing so is likely to cause a segmentation fault.

Warning:
 This function does not try to infer the "dtype" (hence, it is not
 optional). Passing a different "dtype" than its source may result
 in unexpected behavior.

Parameters:
      buffer (object) -- a Python object that exposes the buffer
      interface.
Keyword Arguments:
      * dtype ("torch.dtype") -- the desired data type of returned
        tensor.
  * **count** (*int**, **optional*) -- the number of desired
    elements to be read. If negative, all the elements (until the
    end of the buffer) will be read. Default: -1.

  * **offset** (*int**, **optional*) -- the number of bytes to
skip at the start of the buffer. Default: 0.
  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

Example:
  >>> import array
  >>> a = array.array('i', [1, 2, 3])
  >>> t = torch.frombuffer(a, dtype=torch.int32)
  >>> t
  tensor([ 1,  2,  3])
  >>> t[0] = -1
  >>> a
  array([-1,  2,  3])

  >>> # Interprets the signed char bytes as 32-bit integers.
  >>> # Each 4 signed char elements will be interpreted as
  >>> # 1 signed 32-bit integer.
  >>> import array
  >>> a = array.array('b', [-1, 0, 0, 0])
  >>> torch.frombuffer(a, dtype=torch.int32)
  tensor([255], dtype=torch.int32)
torch.autograd.Function.forward
static Function.forward(ctx, args, *kwargs)
This function is to be overridden by all subclasses. There are two
   ways to define forward:
Usage 1 (Combined forward and ctx):
  @staticmethod
  def forward(ctx: Any, *args: Any, **kwargs: Any) -> Any:
      pass



It must accept a context ctx as the first argument, followed by
     any number of arguments (tensors or other types).


See Combined or separate forward() and setup_context() for more
     details


Usage 2 (Separate forward and ctx):
  @staticmethod
  def forward(*args: Any, **kwargs: Any) -> Any:
      pass

  @staticmethod
  def setup_context(ctx: Any, inputs: Tuple[Any, ...], output: Any) -> None:
      pass



The forward no longer accepts a ctx argument.


Instead, you must also override the
     "torch.autograd.Function.setup_context()" staticmethod to handle
setting up the "ctx" object. "output" is the output of the
     forward, "inputs" are a Tuple of inputs to the forward.

See Extending torch.autograd for more details

The context can be used to store arbitrary data that can be then
   retrieved during the backward pass. Tensors should not be stored
   directly on ctx (though this is not currently enforced for
   backward compatibility). Instead, tensors should be saved either
   with "ctx.save_for_backward()" if they are intended to be used in
   "backward" (equivalently, "vjp") or "ctx.save_for_forward()" if
   they are intended to be used for in "jvp".
Return type:
      Any
PrepareCustomConfig
class torch.ao.quantization.fx.custom_config.PrepareCustomConfig
Custom configuration for "prepare_fx()" and "prepare_qat_fx()".
Example usage:
  prepare_custom_config = PrepareCustomConfig()             .set_standalone_module_name("module1", qconfig_mapping, example_inputs,                 child_prepare_custom_config, backend_config)             .set_standalone_module_class(MyStandaloneModule, qconfig_mapping, example_inputs,                 child_prepare_custom_config, backend_config)             .set_float_to_observed_mapping(FloatCustomModule, ObservedCustomModule)             .set_non_traceable_module_names(["module2", "module3"])             .set_non_traceable_module_classes([NonTraceableModule1, NonTraceableModule2])             .set_input_quantized_indexes([0])             .set_output_quantized_indexes([0])             .set_preserved_attributes(["attr1", "attr2"])

classmethod from_dict(prepare_custom_config_dict)
Create a "PrepareCustomConfig" from a dictionary with the
      following items:
     "standalone_module_name": a list of (module_name,
     qconfig_mapping, example_inputs, child_prepare_custom_config,
     backend_config) tuples

     "standalone_module_class" a list of (module_class,
     qconfig_mapping, example_inputs, child_prepare_custom_config,
     backend_config) tuples

     "float_to_observed_custom_module_class": a nested dictionary
     mapping from quantization mode to an inner mapping from float
     module classes to observed module classes, e.g. {"static":
     {FloatCustomModule: ObservedCustomModule}}

     "non_traceable_module_name": a list of modules names that are
     not symbolically traceable "non_traceable_module_class": a
     list of module classes that are not symbolically traceable
     "input_quantized_idxs": a list of indexes of graph inputs
     that should be quantized "output_quantized_idxs": a list of
indexes of graph outputs that should be quantized
         "preserved_attributes": a list of attributes that persist
         even if they are not used in "forward"
  This function is primarily for backward compatibility and may be
  removed in the future.

  Return type:
     *PrepareCustomConfig*

set_float_to_observed_mapping(float_class, observed_class, quant_type=QuantType.STATIC)
  Set the mapping from a custom float module class to a custom
  observed module class.

  The observed module class must have a "from_float" class method
  that converts the float module class to the observed module
  class. This is currently only supported for static quantization.

  Return type:
     *PrepareCustomConfig*

set_input_quantized_indexes(indexes)
  Set the indexes of the inputs of the graph that should be
  quantized. Inputs are otherwise assumed to be in fp32 by default
  instead.

  Return type:
     *PrepareCustomConfig*
Return type:
         PrepareCustomConfig
set_non_traceable_module_classes(module_classes)
  Set the modules that are not symbolically traceable, identified
  by class.

  Return type:
     *PrepareCustomConfig*

set_non_traceable_module_names(module_names)
  Set the modules that are not symbolically traceable, identified
  by name.

  Return type:
     *PrepareCustomConfig*

set_output_quantized_indexes(indexes)
  Set the indexes of the outputs of the graph that should be
  quantized. Outputs are otherwise assumed to be in fp32 by
  default instead.

  Return type:
     *PrepareCustomConfig*

set_preserved_attributes(attributes)
  Set the names of the attributes that will persist in the graph
  module even if they are not used in the model's "forward"
  method.

  Return type:
     *PrepareCustomConfig*
Return type:
         PrepareCustomConfig
set_standalone_module_class(module_class, qconfig_mapping, example_inputs, prepare_custom_config, backend_config)
  Set the configuration for running a standalone module identified
  by "module_class".

  If "qconfig_mapping" is None, the parent "qconfig_mapping" will
  be used instead. If "prepare_custom_config" is None, an empty
  "PrepareCustomConfig" will be used. If "backend_config" is None,
  the parent "backend_config" will be used instead.

  Return type:
     *PrepareCustomConfig*

set_standalone_module_name(module_name, qconfig_mapping, example_inputs, prepare_custom_config, backend_config)
  Set the configuration for running a standalone module identified
  by "module_name".

  If "qconfig_mapping" is None, the parent "qconfig_mapping" will
  be used instead. If "prepare_custom_config" is None, an empty
  "PrepareCustomConfig" will be used. If "backend_config" is None,
torch.renorm
torch.renorm(input, p, dim, maxnorm, *, out=None) -> Tensor
Returns a tensor where each sub-tensor of "input" along dimension
   "dim" is normalized such that the p-norm of the sub-tensor is
   lower than the value "maxnorm"
Note:
 If the norm of a row is lower than *maxnorm*, the row is
 unchanged

Parameters:
      * input (Tensor) -- the input tensor.
  * **p** (*float*) -- the power for the norm computation

  * **dim** (*int*) -- the dimension to slice over to get the sub-
    tensors

  * **maxnorm** (*float*) -- the maximum norm to keep each sub-
    tensor under

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> x = torch.ones(3, 3)
  >>> x[1].fill_(2)
  tensor([ 2.,  2.,  2.])
  >>> x[2].fill_(3)
  tensor([ 3.,  3.,  3.])
  >>> x
  tensor([[ 1.,  1.,  1.],
          [ 2.,  2.,  2.],
          [ 3.,  3.,  3.]])
CosineSimilarity
class torch.nn.CosineSimilarity(dim=1, eps=1e-08)
Returns cosine similarity between x_1 and x_2, computed along
   dim.
  \text{similarity} = \dfrac{x_1 \cdot x_2}{\max(\Vert x_1 \Vert
  _2 \cdot \Vert x_2 \Vert _2, \epsilon)}.

Parameters:
      * dim (int, optional) -- Dimension where cosine
        similarity is computed. Default: 1
  * **eps** (*float**, **optional*) -- Small value to avoid
    division by zero. Default: 1e-8

Shape:
      * Input1: (\ast_1, D, \ast_2) where D is at position dim
  * Input2: (\ast_1, D, \ast_2), same number of dimensions as x1,
    matching x1 size at dimension *dim*,
       and broadcastable with x1 at other dimensions.

  * Output: (\ast_1, \ast_2)

Examples::
      >>> input1 = torch.randn(100, 128)
      >>> input2 = torch.randn(100, 128)
      >>> cos = nn.CosineSimilarity(dim=1, eps=1e-6)
      >>> output = cos(input1, input2)
torch.tensor
torch.tensor(data, *, dtype=None, device=None, requires_grad=False, pin_memory=False) -> Tensor
Constructs a tensor with no autograd history (also known as a "leaf
   tensor", see Autograd mechanics) by copying "data".
Warning:
 When working with tensors prefer using "torch.Tensor.clone()",
 "torch.Tensor.detach()", and "torch.Tensor.requires_grad_()" for
 readability. Letting *t* be a tensor, "torch.tensor(t)" is
 equivalent to "t.clone().detach()", and "torch.tensor(t,
 requires_grad=True)" is equivalent to
 "t.clone().detach().requires_grad_(True)".

See also:
 "torch.as_tensor()" preserves autograd history and avoids copies
 where possible. "torch.from_numpy()" creates a tensor that shares
 storage with a NumPy array.

Parameters:
      data (array_like) -- Initial data for the tensor. Can be a
      list, tuple, NumPy "ndarray", scalar, and other types.
Keyword Arguments:
Keyword Arguments:
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", infers data type from
        "data".
  * **device** ("torch.device", optional) -- the device of the
    constructed tensor. If None and data is a tensor then the
    device of data is used. If None and data is not a tensor then
    the result tensor is constructed on the CPU.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> torch.tensor([[0.1, 1.2], [2.2, 3.1], [4.9, 5.2]])
  tensor([[ 0.1000,  1.2000],
          [ 2.2000,  3.1000],
          [ 4.9000,  5.2000]])

  >>> torch.tensor([0, 1])  # Type inference on data
  tensor([ 0,  1])
tensor([ 0,  1])
  >>> torch.tensor([[0.11111, 0.222222, 0.3333333]],
  ...              dtype=torch.float64,
  ...              device=torch.device('cuda:0'))  # creates a double tensor on a CUDA device
  tensor([[ 0.1111,  0.2222,  0.3333]], dtype=torch.float64, device='cuda:0')

  >>> torch.tensor(3.14159)  # Create a zero-dimensional (scalar) tensor
  tensor(3.1416)

  >>> torch.tensor([])  # Create an empty tensor (of size (0,))
  tensor([])
Fold
class torch.nn.Fold(output_size, kernel_size, dilation=1, padding=0, stride=1)
Combines an array of sliding local blocks into a large containing
   tensor.
Consider a batched "input" tensor containing sliding local blocks,
   e.g., patches of images, of shape (N, C \times
   \prod(\text{kernel_size}), L), where N is batch dimension, C
   \times \prod(\text{kernel_size}) is the number of values within a
   block (a block has \prod(\text{kernel_size}) spatial locations
   each containing a C-channeled vector), and L is the total number of
   blocks. (This is exactly the same specification as the output shape
   of "Unfold".) This operation combines these local blocks into the
   large "output" tensor of shape (N, C, \text{output_size}[0],
   \text{output_size}[1], \dots) by summing the overlapping values.
   Similar to "Unfold", the arguments must satisfy
  L = \prod_d \left\lfloor\frac{\text{output\_size}[d] + 2 \times
  \text{padding}[d] %     - \text{dilation}[d] \times
(\text{kernel_size}[d] - 1) - 1}{\text{stride}[d]} +
      1\right\rfloor,
where d is over all spatial dimensions.

"output_size" describes the spatial shape of the large containing
     tensor of the sliding local blocks. It is useful to resolve the
     ambiguity when multiple input shapes map to same number of
     sliding blocks, e.g., with "stride > 0".

The "padding", "stride" and "dilation" arguments specify how the
   sliding blocks are retrieved.


"stride" controls the stride for the sliding blocks.


"padding" controls the amount of implicit zero-paddings on both
     sides for "padding" number of points for each dimension before
     reshaping.


"dilation" controls the spacing between the kernel points; also
     known as the Ã  trous algorithm. It is harder to describe, but
     this link has a nice visualization of what "dilation" does.


Parameters:
      * output_size (int or tuple) -- the shape of the
spatial dimensions of the output (i.e., "output.sizes()[2:]")
  * **kernel_size** (*int** or **tuple*) -- the size of the
    sliding blocks

  * **dilation** (*int** or **tuple**, **optional*) -- a parameter
    that controls the stride of elements within the neighborhood.
    Default: 1

  * **padding** (*int** or **tuple**, **optional*) -- implicit
    zero padding to be added on both sides of input. Default: 0

  * **stride** (*int** or **tuple*) -- the stride of the sliding
    blocks in the input spatial dimensions. Default: 1



If "output_size", "kernel_size", "dilation", "padding" or
     "stride" is an int or a tuple of length 1 then their values will
     be replicated across all spatial dimensions.


For the case of two output spatial dimensions this operation is
     sometimes called "col2im".


Note:
 "Fold" calculates each combined value in the resulting large
 tensor by summing all values from all containing blocks. "Unfold"
extracts the values in the local blocks by copying from the large
     tensor. So, if the blocks overlap, they are not inverses of each
     other.In general, folding and unfolding operations are related as
     follows. Consider "Fold" and "Unfold" instances created with the
     same parameters:
 >>> fold_params = dict(kernel_size=..., dilation=..., padding=..., stride=...)
 >>> fold = nn.Fold(output_size=..., **fold_params)
 >>> unfold = nn.Unfold(**fold_params)

 Then for any (supported) "input" tensor the following equality
 holds:

    fold(unfold(input)) == divisor * input

 where "divisor" is a tensor that depends only on the shape and
 dtype of the "input":

 >>> input_ones = torch.ones(input.shape, dtype=input.dtype)
 >>> divisor = fold(unfold(input_ones))

 When the "divisor" tensor contains no zero elements, then "fold"
 and "unfold" operations are inverses of each other (up to
 constant divisor).

Warning:
constant divisor).
Warning:
 Currently, only unbatched (3D) or batched (4D) image-like output
 tensors are supported.

Shape:
      * Input: (N, C \times \prod(\text{kernel_size}), L) or (C
        \times \prod(\text{kernel_size}), L)
  * Output: (N, C, \text{output\_size}[0], \text{output\_size}[1],
    \dots) or (C, \text{output\_size}[0], \text{output\_size}[1],
    \dots) as described above

Examples:
  >>> fold = nn.Fold(output_size=(4, 5), kernel_size=(2, 2))
  >>> input = torch.randn(1, 3 * 2 * 2, 12)
  >>> output = fold(input)
  >>> output.size()
  torch.Size([1, 3, 4, 5])
torch.nanmedian
torch.nanmedian(input) -> Tensor
Returns the median of the values in "input", ignoring "NaN" values.
This function is identical to "torch.median()" when there are no
   "NaN" values in "input". When "input" has one or more "NaN" values,
   "torch.median()" will always return "NaN", while this function will
   return the median of the non-"NaN" elements in "input". If all the
   elements in "input" are "NaN" it will also return "NaN".
Parameters:
      input (Tensor) -- the input tensor.
Example:
  >>> a = torch.tensor([1, float('nan'), 3, 2])
  >>> a.median()
  tensor(nan)
  >>> a.nanmedian()
  tensor(2.)

torch.nanmedian(input, dim=- 1, keepdim=False, *, out=None)
Returns a namedtuple "(values, indices)" where "values" contains
   the median of each row of "input" in the dimension "dim", ignoring
   "NaN" values, and "indices" contains the index of the median values
   found in the dimension "dim".