shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.cumulative_trapezoid
torch.cumulative_trapezoid(y, x=None, *, dx=None, dim=- 1) -> Tensor
Cumulatively computes the trapezoidal rule along "dim". By default
   the spacing between elements is assumed to be 1, but "dx" can be
   used to specify a different constant spacing, and "x" can be used
   to specify arbitrary spacing along "dim".
For more details, please read "torch.trapezoid()". The difference
   between "torch.trapezoid()" and this function is that,
   "torch.trapezoid()" returns a value for each integration, where as
   this function returns a cumulative value for every spacing within
   the integration. This is analogous to how .sum returns a value
   and .cumsum returns a cumulative sum.
Parameters:
      * y (Tensor) -- Values to use when computing the
        trapezoidal rule.
  * **x** (*Tensor*) -- If specified, defines spacing between
    values as specified above.

Keyword Arguments:
Keyword Arguments:
      * dx (float) -- constant spacing between values. If
        neither "x" or "dx" are specified then this defaults to 1.
        Effectively multiplies the result by its value.
  * **dim** (*int*) -- The dimension along which to compute the
    trapezoidal rule. The last (inner-most) dimension by default.

Examples:
  >>> # Cumulatively computes the trapezoidal rule in 1D, spacing is implicitly 1.
  >>> y = torch.tensor([1, 5, 10])
  >>> torch.cumulative_trapezoid(y)
  tensor([3., 10.5])

  >>> # Computes the same trapezoidal rule directly up to each element to verify
  >>> (1 + 5) / 2
  3.0
  >>> (1 + 10 + 10) / 2
  10.5

  >>> # Cumulatively computes the trapezoidal rule in 1D with constant spacing of 2
  >>> # NOTE: the result is the same as before, but multiplied by 2
  >>> torch.cumulative_trapezoid(y, dx=2)
  tensor([6., 21.])
tensor([6., 21.])
  >>> # Cumulatively computes the trapezoidal rule in 1D with arbitrary spacing
  >>> x = torch.tensor([1, 3, 6])
  >>> torch.cumulative_trapezoid(y, x)
  tensor([6., 28.5])

  >>> # Computes the same trapezoidal rule directly up to each element to verify
  >>> ((3 - 1) * (1 + 5)) / 2
  6.0
  >>> ((3 - 1) * (1 + 5) + (6 - 3) * (5 + 10)) / 2
  28.5

  >>> # Cumulatively computes the trapezoidal rule for each row of a 3x3 matrix
  >>> y = torch.arange(9).reshape(3, 3)
  tensor([[0, 1, 2],
          [3, 4, 5],
          [6, 7, 8]])
  >>> torch.cumulative_trapezoid(y)
  tensor([[ 0.5,  2.],
          [ 3.5,  8.],
          [ 6.5, 14.]])

  >>> # Cumulatively computes the trapezoidal rule for each column of the matrix
  >>> torch.cumulative_trapezoid(y, dim=0)
  tensor([[ 1.5,  2.5,  3.5],
          [ 6.0,  8.0, 10.0]])
[ 6.0,  8.0, 10.0]])
  >>> # Cumulatively computes the trapezoidal rule for each row of a 3x3 ones matrix
  >>> #   with the same arbitrary spacing
  >>> y = torch.ones(3, 3)
  >>> x = torch.tensor([1, 3, 6])
  >>> torch.cumulative_trapezoid(y, x)
  tensor([[2., 5.],
          [2., 5.],
          [2., 5.]])

  >>> # Cumulatively computes the trapezoidal rule for each row of a 3x3 ones matrix
  >>> #   with different arbitrary spacing per row
  >>> y = torch.ones(3, 3)
  >>> x = torch.tensor([[1, 2, 3], [1, 3, 5], [1, 4, 7]])
  >>> torch.cumulative_trapezoid(y, x)
  tensor([[1., 2.],
          [2., 4.],
          [3., 6.]])
ParameterDict
class torch.nn.ParameterDict(parameters=None)
Holds parameters in a dictionary.
ParameterDict can be indexed like a regular Python dictionary, but
   Parameters it contains are properly registered, and will be visible
   by all Module methods. Other objects are treated as would be done
   by a regular Python dictionary
"ParameterDict" is an ordered dictionary. "update()" with other
   unordered mapping types (e.g., Python's plain "dict") does not
   preserve the order of the merged mapping. On the other hand,
   "OrderedDict" or another "ParameterDict" will preserve their
   ordering.
Note that the constructor, assigning an element of the dictionary
   and the "update()" method will convert any "Tensor" into
   "Parameter".
Parameters:
      values (iterable, optional) -- a mapping (dictionary)
      of (string : Any) or an iterable of key-value pairs of type
      (string, Any)
Example:
  class MyModule(nn.Module):
Example:
  class MyModule(nn.Module):
      def __init__(self):
          super(MyModule, self).__init__()
          self.params = nn.ParameterDict({
                  'left': nn.Parameter(torch.randn(5, 10)),
                  'right': nn.Parameter(torch.randn(5, 10))
          })

      def forward(self, x, choice):
          x = self.params[choice].mm(x)
          return x

clear()
  Remove all items from the ParameterDict.

copy()
  Returns a copy of this "ParameterDict" instance.

  Return type:
     *ParameterDict*

fromkeys(keys, default=None)
  Return a new ParameterDict with the keys provided

  Parameters:
     * **keys** (*iterable**, **string*) -- keys to make the new
       ParameterDict from

     * **default** (*Parameter**, **optional*) -- value to set for
       all keys

  Return type:
     *ParameterDict*

get(key, default=None)
get(key, default=None)
  Return the parameter associated with key if present. Otherwise
  return default if provided, None if not.

  Parameters:
     * **key** (*str*) -- key to get from the ParameterDict

     * **default** (*Parameter**, **optional*) -- value to return
       if key not present

  Return type:
     *Any*

items()
  Return an iterable of the ParameterDict key/value pairs.

  Return type:
     *Iterable*[*Tuple*[str, *Any*]]

keys()
  Return an iterable of the ParameterDict keys.

  Return type:
     *Iterable*[str]

pop(key)
  Remove key from the ParameterDict and return its parameter.

  Parameters:
     **key** (*str*) -- key to pop from the ParameterDict

  Return type:
     *Any*

popitem()
  Remove and return the last inserted *(key, parameter)* pair from
  the ParameterDict

  Return type:
     *Tuple*[str, *Any*]

setdefault(key, default=None)
setdefault(key, default=None)
  If key is in the ParameterDict, return its value. If not, insert
  *key* with a parameter *default* and return *default*. *default*
  defaults to *None*.

  Parameters:
     * **key** (*str*) -- key to set default for

     * **default** (*Any*) -- the parameter set to the key

  Return type:
     *Any*

update(parameters)
  Update the "ParameterDict" with the key-value pairs from a
  mapping or an iterable, overwriting existing keys.

  Note:

    If "parameters" is an "OrderedDict", a "ParameterDict", or an
    iterable of key-value pairs, the order of new elements in it
    is preserved.

  Parameters:
     **parameters** (*iterable*) -- a mapping (dictionary) from
     string to "Parameter", or an iterable of key-value pairs of
     type (string, "Parameter")

values()
  Return an iterable of the ParameterDict values.

  Return type:
     *Iterable*[*Any*]
torch.bitwise_and
torch.bitwise_and(input, other, *, out=None) -> Tensor
Computes the bitwise AND of "input" and "other". The input tensor
   must be of integral or Boolean types. For bool tensors, it computes
   the logical AND.
Parameters:
      * input -- the first input tensor
  * **other** -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.bitwise_and(torch.tensor([-1, -2, 3], dtype=torch.int8), torch.tensor([1, 0, 3], dtype=torch.int8))
  tensor([1, 0,  3], dtype=torch.int8)
  >>> torch.bitwise_and(torch.tensor([True, True, False]), torch.tensor([False, True, False]))
  tensor([ False, True, False])
torch.nn.functional.selu
torch.nn.functional.selu(input, inplace=False) -> Tensor
Applies element-wise, \text{SELU}(x) = scale * (\max(0,x) + \min(0,
   \alpha * (\exp(x) - 1))), with
   \alpha=1.6732632423543772848170429916717 and
   scale=1.0507009873554804934193349852946.
See "SELU" for more details.
Return type:
      Tensor
torch.view_as_real
torch.view_as_real(input) -> Tensor
Returns a view of "input" as a real tensor. For an input complex
   tensor of "size" m1, m2, \dots, mi, this function returns a new
   real tensor of size m1, m2, \dots, mi, 2, where the last dimension
   of size 2 represents the real and imaginary components of complex
   numbers.
Warning:
 "view_as_real()" is only supported for tensors with "complex
 dtypes".

Parameters:
      input (Tensor) -- the input tensor.
Example:
  >>> x=torch.randn(4, dtype=torch.cfloat)
  >>> x
  tensor([(0.4737-0.3839j), (-0.2098-0.6699j), (0.3470-0.9451j), (-0.5174-1.3136j)])
  >>> torch.view_as_real(x)
  tensor([[ 0.4737, -0.3839],
          [-0.2098, -0.6699],
          [ 0.3470, -0.9451],
          [-0.5174, -1.3136]])
torch.fft.ifft
torch.fft.ifft(input, n=None, dim=- 1, norm=None, *, out=None) -> Tensor
Computes the one dimensional inverse discrete Fourier transform of
   "input".
Note:
 Supports torch.half and torch.chalf on CUDA with GPU Architecture
 SM53 or greater. However it only supports powers of 2 signal
 length in every transformed dimension.

Parameters:
      * input (Tensor) -- the input tensor
  * **n** (*int**, **optional*) -- Signal length. If given, the
    input will either be zero-padded or trimmed to this length
    before computing the IFFT.

  * **dim** (*int**, **optional*) -- The dimension along which to
    take the one dimensional IFFT.

  * **norm** (*str**, **optional*) --

    Normalization mode. For the backward transform ("ifft()"),
    these correspond to:

    * ""forward"" - no normalization

    * ""backward"" - normalize by "1/n"

    * ""ortho"" - normalize by "1/sqrt(n)" (making the IFFT
orthonormal)
    Calling the forward transform ("fft()") with the same
    normalization mode will apply an overall normalization of
    "1/n" between the two transforms. This is required to make
    "ifft()" the exact inverse.

    Default is ""backward"" (normalize by "1/n").

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
-[ Example ]-



t = torch.tensor([ 6.+0.j, -2.+2.j, -2.+0.j, -2.-2.j])
torch.fft.ifft(t)
   tensor([0.+0.j, 1.+0.j, 2.+0.j, 3.+0.j])
InstanceNorm1d
class torch.nn.InstanceNorm1d(num_features, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False, device=None, dtype=None)
Applies Instance Normalization over a 2D (unbatched) or 3D
   (batched) input as described in the paper Instance Normalization:
   The Missing Ingredient for Fast Stylization.
  y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}}
  * \gamma + \beta

The mean and standard-deviation are calculated per-dimension
   separately for each object in a mini-batch. \gamma and \beta are
   learnable parameter vectors of size C (where C is the number of
   features or channels of the input) if "affine" is "True". The
   standard-deviation is calculated via the biased estimator,
   equivalent to torch.var(input, unbiased=False).
By default, this layer uses instance statistics computed from input
   data in both training and evaluation modes.
If "track_running_stats" is set to "True", during training this
layer keeps running estimates of its computed mean and variance,
   which are then used for normalization during evaluation. The
   running estimates are kept with a default "momentum" of 0.1.
Note:
 This "momentum" argument is different from one used in optimizer
 classes and the conventional notion of momentum. Mathematically,
 the update rule for running statistics here is \hat{x}_\text{new}
 = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times
 x_t, where \hat{x} is the estimated statistic and x_t is the new
 observed value.

Note:
 "InstanceNorm1d" and "LayerNorm" are very similar, but have some
 subtle differences. "InstanceNorm1d" is applied on each channel
 of channeled data like multidimensional time series, but
 "LayerNorm" is usually applied on entire sample and often in NLP
 tasks. Additionally, "LayerNorm" applies elementwise affine
 transform, while "InstanceNorm1d" usually don't apply affine
 transform.

Parameters:
transform.
Parameters:
      * num_features (int) -- number of features or channels C
        of the input
  * **eps** (*float*) -- a value added to the denominator for
    numerical stability. Default: 1e-5

  * **momentum** (*float*) -- the value used for the running_mean
    and running_var computation. Default: 0.1

  * **affine** (*bool*) -- a boolean value that when set to
    "True", this module has learnable affine parameters,
    initialized the same way as done for batch normalization.
    Default: "False".

  * **track_running_stats** (*bool*) -- a boolean value that when
    set to "True", this module tracks the running mean and
    variance, and when set to "False", this module does not track
    such statistics and always uses batch statistics in both
    training and eval modes. Default: "False"

Shape:
      * Input: (N, C, L) or (C, L)
  * Output: (N, C, L) or (C, L) (same shape as input)

Examples:
TransformerEncoder
class torch.nn.TransformerEncoder(encoder_layer, num_layers, norm=None, enable_nested_tensor=True, mask_check=True)
TransformerEncoder is a stack of N encoder layers. Users can build
   the BERT(https://arxiv.org/abs/1810.04805) model with corresponding
   parameters.
Parameters:
      * encoder_layer -- an instance of the
        TransformerEncoderLayer() class (required).
  * **num_layers** -- the number of sub-encoder-layers in the
    encoder (required).

  * **norm** -- the layer normalization component (optional).

  * **enable_nested_tensor** -- if True, input will automatically
    convert to nested tensor (and convert back on output). This
    will improve the overall performance of TransformerEncoder
    when padding rate is high. Default: "True" (enabled).

Examples::
      >>> encoder_layer = nn.TransformerEncoderLayer(d_model=512, nhead=8)
transformer_encoder = nn.TransformerEncoder(encoder_layer, num_layers=6)
      >>> src = torch.rand(10, 32, 512)
      >>> out = transformer_encoder(src)



forward(src, mask=None, src_key_padding_mask=None, is_causal=None)
  Pass the input through the encoder layers in turn.

  Parameters:
     * **src** (*Tensor*) -- the sequence to the encoder
       (required).

     * **mask** (*Optional**[**Tensor**]*) -- the mask for the src
       sequence (optional).

     * **is_causal** (*Optional**[**bool**]*) -- If specified,
       applies a causal mask as mask (optional) and ignores
       attn_mask for computing scaled dot product attention.
       Default: "False".

     * **src_key_padding_mask** (*Optional**[**Tensor**]*) -- the
       mask for the src keys per batch (optional).

  Return type:
     *Tensor*

  Shape:
     see the docs in Transformer class.
torch.atan
torch.atan(input, *, out=None) -> Tensor
Returns a new tensor with the arctangent of the elements of
   "input".
  \text{out}_{i} = \tan^{-1}(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([ 0.2341,  0.2539, -0.6256, -0.6448])
  >>> torch.atan(a)
  tensor([ 0.2299,  0.2487, -0.5591, -0.5727])
LayerNorm
class torch.ao.nn.quantized.LayerNorm(normalized_shape, weight, bias, scale, zero_point, eps=1e-05, elementwise_affine=True, device=None, dtype=None)
This is the quantized version of "LayerNorm".
Additional args:
      * scale - quantization scale of the output, type: double.
  * **zero_point** - quantization zero point of the output, type:
    long.
torch.nn.functional.embedding_bag
torch.nn.functional.embedding_bag(input, weight, offsets=None, max_norm=None, norm_type=2, scale_grad_by_freq=False, mode='mean', sparse=False, per_sample_weights=None, include_last_offset=False, padding_idx=None)
Computes sums, means or maxes of bags of embeddings, without
   instantiating the intermediate embeddings.
See "torch.nn.EmbeddingBag" for more details.
Note:
 This operation may produce nondeterministic gradients when given
 tensors on a CUDA device. See Reproducibility for more
 information.

Parameters:
      * input (LongTensor) -- Tensor containing bags of indices
        into the embedding matrix
  * **weight** (*Tensor*) -- The embedding matrix with number of
    rows equal to the maximum possible index + 1, and number of
    columns equal to the embedding size

  * **offsets** (*LongTensor**, **optional*) -- Only used when
"input" is 1D. "offsets" determines the starting index
        position of each bag (sequence) in "input".
  * **max_norm** (*float**, **optional*) -- If given, each
    embedding vector with norm larger than "max_norm" is
    renormalized to have norm "max_norm". Note: this will modify
    "weight" in-place.

  * **norm_type** (*float**, **optional*) -- The "p" in the
    "p"-norm to compute for the "max_norm" option. Default "2".

  * **scale_grad_by_freq** (*bool**, **optional*) -- if given,
    this will scale gradients by the inverse of frequency of the
    words in the mini-batch. Default "False". Note: this option is
    not supported when "mode="max"".

  * **mode** (*str**, **optional*) -- ""sum"", ""mean"" or
    ""max"". Specifies the way to reduce the bag. Default:
    ""mean""

  * **sparse** (*bool**, **optional*) -- if "True", gradient
    w.r.t. "weight" will be a sparse tensor. See Notes under
"torch.nn.Embedding" for more details regarding sparse
        gradients. Note: this option is not supported when
        "mode="max"".
  * **per_sample_weights** (*Tensor**, **optional*) -- a tensor of
    float / double weights, or None to indicate all weights should
    be taken to be 1. If specified, "per_sample_weights" must have
    exactly the same shape as input and is treated as having the
    same "offsets", if those are not None.

  * **include_last_offset** (*bool**, **optional*) -- if "True",
    the size of offsets is equal to the number of bags + 1. The
    last element is the size of the input, or the ending index
    position of the last bag (sequence).

  * **padding_idx** (*int**, **optional*) -- If specified, the
    entries at "padding_idx" do not contribute to the gradient;
    therefore, the embedding vector at "padding_idx" is not
    updated during training, i.e. it remains as a fixed "pad".
Note that the embedding vector at "padding_idx" is excluded
        from the reduction.
Return type:
      Tensor
Shape:
      * "input" (LongTensor) and "offsets" (LongTensor, optional)
    * If "input" is 2D of shape *(B, N)*, it will be treated as
      "B" bags (sequences) each of fixed length "N", and this will
      return "B" values aggregated in a way depending on the
      "mode". "offsets" is ignored and required to be "None" in
      this case.

    * If "input" is 1D of shape *(N)*, it will be treated as a
      concatenation of multiple bags (sequences). "offsets" is
      required to be a 1D tensor containing the starting index
      positions of each bag in "input". Therefore, for "offsets"
      of shape *(B)*, "input" will be viewed as having "B" bags.
      Empty bags (i.e., having 0-length) will have returned
      vectors filled by zeros.

  * "weight" (Tensor): the learnable weights of the module of
shape (num_embeddings, embedding_dim)
  * "per_sample_weights" (Tensor, optional). Has the same shape as
    "input".

  * "output": aggregated embedding values of shape *(B,
    embedding_dim)*

Examples:
  >>> # an Embedding module containing 10 tensors of size 3
  >>> embedding_matrix = torch.rand(10, 3)
  >>> # a batch of 2 samples of 4 indices each
  >>> input = torch.tensor([1, 2, 4, 5, 4, 3, 2, 9])
  >>> offsets = torch.tensor([0, 4])
  >>> F.embedding_bag(input, embedding_matrix, offsets)
  tensor([[ 0.3397,  0.3552,  0.5545],
          [ 0.5893,  0.4386,  0.5882]])

  >>> # example with padding_idx
  >>> embedding_matrix = torch.rand(10, 3)
  >>> input = torch.tensor([2, 2, 2, 2, 4, 3, 2, 9])
  >>> offsets = torch.tensor([0, 4])
  >>> F.embedding_bag(input, embedding_matrix, offsets, padding_idx=2, mode='sum')
  tensor([[ 0.0000,  0.0000,  0.0000],
          [-0.7082,  3.2145, -2.6251]])
Identity
class torch.nn.Identity(args, *kwargs)
A placeholder identity operator that is argument-insensitive.
Parameters:
      * args (Any) -- any argument (unused)
  * **kwargs** (*Any*) -- any keyword argument (unused)

Shape:
      * Input: (*), where * means any number of dimensions.
  * Output: (*), same shape as the input.

Examples:
  >>> m = nn.Identity(54, unused_argument1=0.1, unused_argument2=False)
  >>> input = torch.randn(128, 20)
  >>> output = m(input)
  >>> print(output.size())
  torch.Size([128, 20])
torch.jit.script_if_tracing
torch.jit.script_if_tracing(fn)
Compiles "fn" when it is first called during tracing.
   "torch.jit.script" has a non-negligible start up time when it is
   first called due to lazy-initializations of many compiler builtins.
   Therefore you should not use it in library code. However, you may
   want to have parts of your library work in tracing even if they use
   control flow. In these cases, you should use
   "@torch.jit.script_if_tracing" to substitute for
   "torch.jit.script".
Parameters:
      fn -- A function to compile.
Returns:
      If called during tracing, a "ScriptFunction" created by
      torch.jit.script is returned. Otherwise, the original function
      fn is returned.
Dropout3d
class torch.nn.Dropout3d(p=0.5, inplace=False)
Randomly zero out entire channels (a channel is a 3D feature map,
   e.g., the j-th channel of the i-th sample in the batched input is a
   3D tensor \text{input}[i, j]). Each channel will be zeroed out
   independently on every forward call with probability "p" using
   samples from a Bernoulli distribution.
Usually the input comes from "nn.Conv3d" modules.
As described in the paper Efficient Object Localization Using
   Convolutional Networks , if adjacent pixels within feature maps are
   strongly correlated (as is normally the case in early convolution
   layers) then i.i.d. dropout will not regularize the activations and
   will otherwise just result in an effective learning rate decrease.
In this case, "nn.Dropout3d()" will help promote independence
   between feature maps and should be used instead.
Parameters:
      * p (float, optional) -- probability of an element to
        be zeroed.
be zeroed.
  * **inplace** (*bool**, **optional*) -- If set to "True", will
    do this operation in-place

Shape:
      * Input: (N, C, D, H, W) or (C, D, H, W).
  * Output: (N, C, D, H, W) or (C, D, H, W) (same shape as input).

Examples:
  >>> m = nn.Dropout3d(p=0.2)
  >>> input = torch.randn(20, 16, 4, 32, 32)
  >>> output = m(input)
ASGD
class torch.optim.ASGD(params, lr=0.01, lambd=0.0001, alpha=0.75, t0=1000000.0, weight_decay=0, foreach=None, maximize=False, differentiable=False)
Implements Averaged Stochastic Gradient Descent.
It has been proposed in Acceleration of stochastic approximation by
   averaging.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    1e-2)

  * **lambd** (*float**, **optional*) -- decay term (default:
    1e-4)

  * **alpha** (*float**, **optional*) -- power for eta update
    (default: 0.75)

  * **t0** (*float**, **optional*) -- point at which to start
    averaging (default: 1e6)

  * **weight_decay** (*float**, **optional*) -- weight decay (L2
    penalty) (default: 0)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used. If unspecified by the
user (so foreach is None), we will try to use foreach over the
        for-loop implementation on CUDA, since it is usually
        significantly more performant. (default: None)
  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **differentiable** (*bool**, **optional*) -- whether autograd
    should occur through the optimizer step in training.
    Otherwise, the step() function runs in a torch.no_grad()
    context. Setting to True can impair performance, so leave it
    False if you don't intend to run autograd through this
    instance (default: False)

add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters: