shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.sign
torch.sign(input, *, out=None) -> Tensor
Returns a new tensor with the signs of the elements of "input".
  \text{out}_{i} = \operatorname{sgn}(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([0.7, -1.2, 0., 2.3])
  >>> a
  tensor([ 0.7000, -1.2000,  0.0000,  2.3000])
  >>> torch.sign(a)
  tensor([ 1., -1.,  0.,  1.])
torch.swapaxes
torch.swapaxes(input, axis0, axis1) -> Tensor
Alias for "torch.transpose()".
This function is equivalent to NumPy's swapaxes function.
Examples:
  >>> x = torch.tensor([[[0,1],[2,3]],[[4,5],[6,7]]])
  >>> x
  tensor([[[0, 1],
          [2, 3]],

          [[4, 5],
          [6, 7]]])
  >>> torch.swapaxes(x, 0, 1)
  tensor([[[0, 1],
          [4, 5]],

          [[2, 3],
          [6, 7]]])
  >>> torch.swapaxes(x, 0, 2)
  tensor([[[0, 4],
          [2, 6]],

          [[1, 5],
          [3, 7]]])
torch.nn.utils.remove_spectral_norm
torch.nn.utils.remove_spectral_norm(module, name='weight')
Removes the spectral normalization reparameterization from a
   module.
Parameters:
      * module (Module) -- containing module
  * **name** (*str**, **optional*) -- name of weight parameter

Return type:
      T_module
-[ Example ]-



m = spectral_norm(nn.Linear(40, 10))
remove_spectral_norm(m)
torch.cuda.seed
torch.cuda.seed()
Sets the seed for generating random numbers to a random number for
   the current GPU. It's safe to call this function if CUDA is not
   available; in that case, it is silently ignored.
Warning:
 If you are working with a multi-GPU model, this function will
 only initialize the seed on one GPU.  To initialize all GPUs, use
 "seed_all()".
torch.nn.functional.cross_entropy
torch.nn.functional.cross_entropy(input, target, weight=None, size_average=None, ignore_index=- 100, reduce=None, reduction='mean', label_smoothing=0.0)
This criterion computes the cross entropy loss between input logits
   and target.
See "CrossEntropyLoss" for details.
Parameters:
      * input (Tensor) -- Predicted unnormalized logits; see
        Shape section below for supported shapes.
  * **target** (*Tensor*) -- Ground truth class indices or class
    probabilities; see Shape section below for supported shapes.

  * **weight** (*Tensor**, **optional*) -- a manual rescaling
    weight given to each class. If given, has to be a Tensor of
    size *C*

  * **size_average** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged over each
    loss element in the batch. Note that for some losses, there
multiple elements per sample. If the field "size_average" is
        set to "False", the losses are instead summed for each
        minibatch. Ignored when reduce is "False". Default: "True"
  * **ignore_index** (*int**, **optional*) -- Specifies a target
    value that is ignored and does not contribute to the input
    gradient. When "size_average" is "True", the loss is averaged
    over non-ignored targets. Note that "ignore_index" is only
    applicable when the target contains class indices. Default:
    -100

  * **reduce** (*bool**, **optional*) -- Deprecated (see
    "reduction"). By default, the losses are averaged or summed
    over observations for each minibatch depending on
    "size_average". When "reduce" is "False", returns a loss per
    batch element instead and ignores "size_average". Default:
    "True"

  * **reduction** (*str**, **optional*) -- Specifies the reduction
to apply to the output: "'none'" | "'mean'" | "'sum'".
        "'none'": no reduction will be applied, "'mean'": the sum of
        the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Note:
        "size_average" and "reduce" are in the process of being
        deprecated, and in the meantime, specifying either of those
        two args will override "reduction". Default: "'mean'"
  * **label_smoothing** (*float**, **optional*) -- A float in
    [0.0, 1.0]. Specifies the amount of smoothing when computing
    the loss, where 0.0 means no smoothing. The targets become a
    mixture of the original ground truth and a uniform
    distribution as described in Rethinking the Inception
    Architecture for Computer Vision. Default: 0.0.

Return type:
      Tensor
Shape:
      * Input: Shape (C), (N, C) or (N, C, d_1, d_2, ..., d_K) with K
        \geq 1 in the case of K-dimensional loss.
Target: If containing class indices, shape (), (N) or (N, d_1,
        d_2, ..., d_K) with K \geq 1 in the case of K-dimensional loss
        where each value should be between [0, C). If containing class
        probabilities, same shape as the input and each value should
        be between [0, 1].
where:
 \begin{aligned}     C ={} & \text{number of classes} \\     N
 ={} & \text{batch size} \\ \end{aligned}



Examples:
  >>> # Example of target with class indices
  >>> input = torch.randn(3, 5, requires_grad=True)
  >>> target = torch.randint(5, (3,), dtype=torch.int64)
  >>> loss = F.cross_entropy(input, target)
  >>> loss.backward()
  >>>
  >>> # Example of target with class probabilities
  >>> input = torch.randn(3, 5, requires_grad=True)
  >>> target = torch.randn(3, 5).softmax(dim=1)
  >>> loss = F.cross_entropy(input, target)
  >>> loss.backward()
torch.conj
torch.conj(input) -> Tensor
Returns a view of "input" with a flipped conjugate bit. If "input"
   has a non-complex dtype, this function just returns "input".
Note:
 "torch.conj()" performs a lazy conjugation, but the actual
 conjugated tensor can be materialized at any time using
 "torch.resolve_conj()".

Warning:
 In the future, "torch.conj()" may return a non-writeable view for
 an "input" of non-complex dtype. It's recommended that programs
 not modify the tensor returned by "torch.conj_physical()" when
 "input" is of non-complex dtype to be compatible with this
 change.

Parameters:
      input (Tensor) -- the input tensor.
Example:
  >>> x = torch.tensor([-1 + 1j, -2 + 2j, 3 - 3j])
  >>> x.is_conj()
  False
  >>> y = torch.conj(x)
  >>> y.is_conj()
  True
torch.nn.utils.remove_weight_norm
torch.nn.utils.remove_weight_norm(module, name='weight')
Removes the weight normalization reparameterization from a module.
Parameters:
      * module (Module) -- containing module
  * **name** (*str**, **optional*) -- name of weight parameter

Return type:
      T_module
-[ Example ]-



m = weight_norm(nn.Linear(20, 40))
remove_weight_norm(m)
StepLR
class torch.optim.lr_scheduler.StepLR(optimizer, step_size, gamma=0.1, last_epoch=- 1, verbose=False)
Decays the learning rate of each parameter group by gamma every
   step_size epochs. Notice that such decay can happen simultaneously
   with other changes to the learning rate from outside this
   scheduler. When last_epoch=-1, sets initial lr as lr.
Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
  * **step_size** (*int*) -- Period of learning rate decay.

  * **gamma** (*float*) -- Multiplicative factor of learning rate
    decay. Default: 0.1.

  * **last_epoch** (*int*) -- The index of last epoch. Default:
    -1.

  * **verbose** (*bool*) -- If "True", prints a message to stdout
    for each update. Default: "False".

-[ Example ]-



Assuming optimizer uses lr = 0.05 for all groups
lr = 0.05     if epoch < 30
lr = 0.005    if 30 <= epoch < 60
lr = 0.0005   if 60 <= epoch < 90
...
...
scheduler = StepLR(optimizer, step_size=30, gamma=0.1)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()



get_last_lr()
  Return last computed learning rate by current scheduler.

load_state_dict(state_dict)
  Loads the schedulers state.

  Parameters:
     **state_dict** (*dict*) -- scheduler state. Should be an
     object returned from a call to "state_dict()".

print_lr(is_verbose, group, lr, epoch=None)
  Display the current learning rate.

state_dict()
  Returns the state of the scheduler as a "dict".

  It contains an entry for every variable in self.__dict__ which
  is not the optimizer.
PReLU
class torch.nn.PReLU(num_parameters=1, init=0.25, device=None, dtype=None)
Applies the element-wise function:
  \text{PReLU}(x) = \max(0,x) + a * \min(0,x)

or
  \text{PReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0 \\ ax,
  & \text{ otherwise } \end{cases}

Here a is a learnable parameter. When called without arguments,
   nn.PReLU() uses a single parameter a across all input channels.
   If called with nn.PReLU(nChannels), a separate a is used for each
   input channel.
Note:
 weight decay should not be used when learning a for good
 performance.

Note:
 Channel dim is the 2nd dim of input. When input has dims < 2,
 then there is no channel dim and the number of channels = 1.

Parameters:
      * num_parameters (int) -- number of a to learn. Although
        it takes an int as input, there is only two values are
        legitimate: 1, or the number of channels at input. Default: 1
init (float) -- the initial value of a. Default: 0.25

Shape:
      * Input: ( *) where *** means, any number of additional
        dimensions.
  * Output: (*), same shape as the input.

Variables:
      weight (Tensor) -- the learnable weights of shape
      ("num_parameters").
[image]
Examples:
  >>> m = nn.PReLU()
  >>> input = torch.randn(2)
  >>> output = m(input)
torch.transpose
torch.transpose(input, dim0, dim1) -> Tensor
Returns a tensor that is a transposed version of "input". The given
   dimensions "dim0" and "dim1" are swapped.
If "input" is a strided tensor then the resulting "out" tensor
   shares its underlying storage with the "input" tensor, so changing
   the content of one would change the content of the other.
If "input" is a sparse tensor then the resulting "out" tensor does
   not share the underlying storage with the "input" tensor.
If "input" is a sparse tensor with compressed layout (SparseCSR,
   SparseBSR, SparseCSC or SparseBSC) the arguments "dim0" and "dim1"
   must be both batch dimensions, or must both be sparse dimensions.
   The batch dimensions of a sparse tensor are the dimensions
   preceding the sparse dimensions.
Note:
 Transpositions which interchange the sparse dimensions of a
 *SparseCSR* or *SparseCSC* layout tensor will result in the
layout changing between the two options. Transposition of the
     sparse dimensions of a SparseBSR or SparseBSC layout tensor
     will likewise generate a result with the opposite layout.
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim0** (*int*) -- the first dimension to be transposed

  * **dim1** (*int*) -- the second dimension to be transposed

Example:
  >>> x = torch.randn(2, 3)
  >>> x
  tensor([[ 1.0028, -0.9893,  0.5809],
          [-0.1669,  0.7299,  0.4942]])
  >>> torch.transpose(x, 0, 1)
  tensor([[ 1.0028, -0.1669],
          [-0.9893,  0.7299],
          [ 0.5809,  0.4942]])

See also "torch.t()".
torch.cdist
torch.cdist(x1, x2, p=2.0, compute_mode='use_mm_for_euclid_dist_if_necessary')
Computes batched the p-norm distance between each pair of the two
   collections of row vectors.
Parameters:
      * x1 (Tensor) -- input tensor of shape B \times P \times
        M.
  * **x2** (*Tensor*) -- input tensor of shape B \times R \times
    M.

  * **p** (*float*) -- p value for the p-norm distance to
    calculate between each vector pair \in [0, \infty].

  * **compute_mode** (*str*) --
    'use_mm_for_euclid_dist_if_necessary' - will use matrix
    multiplication approach to calculate euclidean distance (p =
    2) if P > 25 or R > 25 'use_mm_for_euclid_dist' - will always
    use matrix multiplication approach to calculate euclidean
    distance (p = 2) 'donot_use_mm_for_euclid_dist' - will never
    use matrix multiplication approach to calculate euclidean
    distance (p = 2) Default: use_mm_for_euclid_dist_if_necessary.
Return type:
      Tensor
If x1 has shape B \times P \times M and x2 has shape B \times R
   \times M then the output will have shape B \times P \times R.
This function is equivalent to
   scipy.spatial.distance.cdist(input,'minkowski', p=p) if p \in (0,
   \infty). When p = 0 it is equivalent to
   scipy.spatial.distance.cdist(input, 'hamming') * M. When p =
   \infty, the closest scipy function is
   scipy.spatial.distance.cdist(xn, lambda x, y: np.abs(x -
   y).max()).
-[ Example ]-



a = torch.tensor([[0.9041,  0.0196], [-0.3108, -2.4423], [-0.4821,  1.059]])
a
   tensor([[ 0.9041,  0.0196],
           [-0.3108, -2.4423],
           [-0.4821,  1.0590]])
b = torch.tensor([[-2.1763, -0.4713], [-0.6986,  1.3702]])
b
   tensor([[-2.1763, -0.4713],
           [-0.6986,  1.3702]])
torch.cdist(a, b, p=2)
   tensor([[3.1193, 2.0959],
           [2.7138, 3.8322],
           [2.2830, 0.3791]])
torch.split
torch.split(tensor, split_size_or_sections, dim=0)
Splits the tensor into chunks. Each chunk is a view of the original
   tensor.
If "split_size_or_sections" is an integer type, then "tensor" will
   be split into equally sized chunks (if possible). Last chunk will
   be smaller if the tensor size along the given dimension "dim" is
   not divisible by "split_size".
If "split_size_or_sections" is a list, then "tensor" will be split
   into "len(split_size_or_sections)" chunks with sizes in "dim"
   according to "split_size_or_sections".
Parameters:
      * tensor (Tensor) -- tensor to split.
  * **split_size_or_sections** (*int**) or **(**list**(**int**)*)
    -- size of a single chunk or list of sizes for each chunk

  * **dim** (*int*) -- dimension along which to split the tensor.

Return type:
      List[Tensor]
Example:
  >>> a = torch.arange(10).reshape(5, 2)
  >>> a
  tensor([[0, 1],
          [2, 3],
tensor([[0, 1],
              [2, 3],
              [4, 5],
              [6, 7],
              [8, 9]])
      >>> torch.split(a, 2)
      (tensor([[0, 1],
               [2, 3]]),
       tensor([[4, 5],
               [6, 7]]),
       tensor([[8, 9]]))
      >>> torch.split(a, [1, 4])
      (tensor([[0, 1]]),
       tensor([[2, 3],
               [4, 5],
               [6, 7],
               [8, 9]]))
torch.cuda.jiterator._create_jit_fn
torch.cuda.jiterator._create_jit_fn(code_string, **kwargs)
Create a jiterator-generated cuda kernel for an elementwise op.
The code string has to be a valid CUDA function that describes the
   computation for a single element. The code string has to follow the
   c++ template pattern, as shown in the example below. This function
   will be inlined into elementwise kernel template, and compiled on
   the fly. Compiled kernel will be cached in memory, as well as local
   temp dir.
Jiterator-generated kernels accepts noncontiguous tensors, and
   supports boardcasting and type promotion.
Parameters:
      * code_string (str) -- CUDA code string to be compiled by
        jiterator. The entry functor must return by value.
  * **kwargs** (*Dict**, **optional*) -- Keyword arguments for
    generated function

Return type:
      Callable
Example:
Return type:
      Callable
Example:
  code_string = "template <typename T> T my_kernel(T x, T y, T alpha) { return -x + alpha * y; }"
  jitted_fn = create_jit_fn(code_string, alpha=1.0)
  a = torch.rand(3, device='cuda')
  b = torch.rand(3, device='cuda')
  # invoke jitted function like a regular python function
  result = jitted_fn(a, b, alpha=3.14)

code_string also allows multiple function definitions, and the last
   function will be treated as the entry function.
Example:
  code_string = "template <typename T> T util_fn(T x, T y) { return ::sin(x) + ::cos(y); }"
  code_string += "template <typename T> T my_kernel(T x, T y, T val) { return ::min(val, util_fn(x, y)); }"
  jitted_fn = create_jit_fn(code_string, val=0.0)
  a = torch.rand(3, device='cuda')
  b = torch.rand(3, device='cuda')
  # invoke jitted function like a regular python function
  result = jitted_fn(a, b)  # using default val=0.0
Jiterator can be used together with python registration to override
   an operator's cuda kernel. Following example is overriding gelu's
   cuda kernel with relu.
Example:
  code_string = "template <typename T> T my_gelu(T a) { return a > 0 ? a : 0; }"
  my_gelu = create_jit_fn(code_string)
  my_lib = torch.library.Library("aten", "IMPL")
  my_lib.impl('aten::gelu', my_gelu, "CUDA")
  # torch.nn.GELU and torch.nn.function.gelu are now overridden
  a = torch.rand(3, device='cuda')
  torch.allclose(torch.nn.functional.gelu(a), torch.nn.functional.relu(a))

Warning:
 This API is in beta and may change in future releases.

Warning:
 This API only supports up to 8 inputs and 1 output

Warning:
 All input tensors must live in CUDA device
torch.logaddexp
torch.logaddexp(input, other, *, out=None) -> Tensor
Logarithm of the sum of exponentiations of the inputs.
Calculates pointwise \log\left(e^x + e^y\right). This function is
   useful in statistics where the calculated probabilities of events
   may be so small as to exceed the range of normal floating point
   numbers. In such cases the logarithm of the calculated probability
   is stored. This function allows adding probabilities stored in such
   a fashion.
This op should be disambiguated with "torch.logsumexp()" which
   performs a reduction on a single tensor.
Parameters:
      * input (Tensor) -- the input tensor.
  * **other** (*Tensor*) -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.logaddexp(torch.tensor([-1.0]), torch.tensor([-1.0, -2, -3]))
  tensor([-0.3069, -0.6867, -0.8731])
tensor([-0.3069, -0.6867, -0.8731])
      >>> torch.logaddexp(torch.tensor([-100.0, -200, -300]), torch.tensor([-1.0, -2, -3]))
      tensor([-1., -2., -3.])
      >>> torch.logaddexp(torch.tensor([1.0, 2000, 30000]), torch.tensor([-1.0, -2, -3]))
      tensor([1.1269e+00, 2.0000e+03, 3.0000e+04])
torch.nn.utils.prune.random_structured
torch.nn.utils.prune.random_structured(module, name, amount, dim)
Prunes tensor corresponding to parameter called "name" in "module"
   by removing the specified "amount" of (currently unpruned) channels
   along the specified "dim" selected at random. Modifies module in
   place (and also return the modified module) by:


adding a named buffer called "name+'_mask'" corresponding to the
      binary mask applied to the parameter "name" by the pruning
      method.


replacing the parameter "name" by its pruned version, while the
      original (unpruned) parameter is stored in a new parameter named
      "name+'_orig'".


Parameters:
      * module (nn.Module) -- module containing the tensor to
        prune
  * **name** (*str*) -- parameter name within "module" on which
    pruning will act.

  * **amount** (*int** or **float*) -- quantity of parameters to
prune. If "float", should be between 0.0 and 1.0 and represent
        the fraction of parameters to prune. If "int", it represents
        the absolute number of parameters to prune.
  * **dim** (*int*) -- index of the dim along which we define
    channels to prune.

Returns:
      modified (i.e. pruned) version of the input module
Return type:
      module (nn.Module)
-[ Examples ]-



m = prune.random_structured(
   ...     nn.Linear(5, 3), 'weight', amount=3, dim=1
   ... )
columns_pruned = int(sum(torch.sum(m.weight, dim=0) == 0))
print(columns_pruned)
   3
torch.argmax
torch.argmax(input) -> LongTensor
Returns the indices of the maximum value of all elements in the
   "input" tensor.
This is the second value returned by "torch.max()". See its
   documentation for the exact semantics of this method.
Note:
 If there are multiple maximal values then the indices of the
 first maximal value are returned.

Parameters:
      input (Tensor) -- the input tensor.
Example:
  >>> a = torch.randn(4, 4)
  >>> a
  tensor([[ 1.3398,  0.2663, -0.2686,  0.2450],
          [-0.7401, -0.8805, -0.3402, -1.1936],
          [ 0.4907, -1.3948, -1.0691, -0.3132],
          [-1.6092,  0.5419, -0.2993,  0.3195]])
  >>> torch.argmax(a)
  tensor(0)

torch.argmax(input, dim, keepdim=False) -> LongTensor
Returns the indices of the maximum values of a tensor across a
   dimension.
This is the second value returned by "torch.max()". See its
   documentation for the exact semantics of this method.
Parameters:
      * input (Tensor) -- the input tensor.
  * **dim** (*int*) -- the dimension to reduce. If "None", the
    argmax of the flattened input is returned.

  * **keepdim** (*bool*) -- whether the output tensor has "dim"
    retained or not. Ignored if "dim=None".

Example:
  >>> a = torch.randn(4, 4)
  >>> a
  tensor([[ 1.3398,  0.2663, -0.2686,  0.2450],
          [-0.7401, -0.8805, -0.3402, -1.1936],
          [ 0.4907, -1.3948, -1.0691, -0.3132],
          [-1.6092,  0.5419, -0.2993,  0.3195]])
  >>> torch.argmax(a, dim=1)
  tensor([ 0,  2,  0,  1])
QuantWrapper
class torch.quantization.QuantWrapper(module)
A wrapper class that wraps the input module, adds QuantStub and
   DeQuantStub and surround the call to module with call to quant and
   dequant modules.
This is used by the quantization utility functions to add the
   quant and dequant modules, before convert function QuantStub
   will just be observer, it observes the input tensor, after
   convert, QuantStub will be swapped to nnq.Quantize which does
   actual quantization. Similarly for DeQuantStub.
AdaptiveAvgPool3d
class torch.nn.AdaptiveAvgPool3d(output_size)
Applies a 3D adaptive average pooling over an input signal composed
   of several input planes.
The output is of size D x H x W, for any input size. The number of
   output features is equal to the number of input planes.
Parameters:
      output_size (Union[int, None,
      Tuple[Optional[int], Optional[int],
      Optional[int]]]) -- the target output size of the
      form D x H x W. Can be a tuple (D, H, W) or a single number D
      for a cube D x D x D. D, H and W can be either a "int", or
      "None" which means the size will be the same as that of the
      input.
Shape:
      * Input: (N, C, D_{in}, H_{in}, W_{in}) or (C, D_{in}, H_{in},
        W_{in}).
  * Output: (N, C, S_{0}, S_{1}, S_{2}) or (C, S_{0}, S_{1},
    S_{2}), where S=\text{output\_size}.

-[ Examples ]-



target output size of 5x7x9
target output size of 5x7x9
m = nn.AdaptiveAvgPool3d((5, 7, 9))
input = torch.randn(1, 64, 8, 9, 10)
output = m(input)
target output size of 7x7x7 (cube)
m = nn.AdaptiveAvgPool3d(7)
input = torch.randn(1, 64, 10, 9, 8)
output = m(input)
target output size of 7x9x8
m = nn.AdaptiveAvgPool3d((7, None, None))
input = torch.randn(1, 64, 10, 9, 8)
output = m(input)
torch.bitwise_right_shift
torch.bitwise_right_shift(input, other, *, out=None) -> Tensor
Computes the right arithmetic shift of "input" by "other" bits. The
   input tensor must be of integral type. This operator supports
   broadcasting to a common shape and type promotion.
The operation applied is:
  \text{out}_i = \text{input}_i >> \text{other}_i

Parameters:
      * input (Tensor or Scalar) -- the first input tensor
  * **other** (*Tensor** or **Scalar*) -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> torch.bitwise_right_shift(torch.tensor([-2, -7, 31], dtype=torch.int8), torch.tensor([1, 0, 3], dtype=torch.int8))
  tensor([-1, -7,  3], dtype=torch.int8)
celu
class torch.ao.nn.quantized.functional.celu(input, scale, zero_point, alpha=1.)
Applies the quantized CELU function element-wise.
  \text{CELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x / \alpha)
  - 1))

Parameters:
      * input (Tensor) -- quantized input
  * **alpha** (*float*) -- the \alpha value for the CELU
    formulation. Default: 1.0

Return type:
      Tensor