shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.nn.functional.max_pool1d
torch.nn.functional.max_pool1d(input, kernel_size, stride=None, padding=0, dilation=1, ceil_mode=False, return_indices=False)
Applies a 1D max pooling over an input signal composed of several
   input planes.
Note:
 The order of "ceil_mode" and "return_indices" is different from
 what seen in "MaxPool1d", and will change in a future release.

See "MaxPool1d" for details.
Parameters:
      * input -- input tensor of shape (\text{minibatch} ,
        \text{in_channels} , iW), minibatch dim optional.
  * **kernel_size** -- the size of the window. Can be a single
    number or a tuple *(kW,)*

  * **stride** -- the stride of the window. Can be a single number
    or a tuple *(sW,)*. Default: "kernel_size"

  * **padding** -- Implicit negative infinity padding to be added
    on both sides, must be >= 0 and <= kernel_size / 2.

  * **dilation** -- The stride between elements within a sliding
window, must be > 0.
  * **ceil_mode** -- If "True", will use *ceil* instead of *floor*
    to compute the output shape. This ensures that every element
    in the input tensor is covered by a sliding window.

  * **return_indices** -- If "True", will return the argmax along
    with the max values. Useful for
    "torch.nn.functional.max_unpool1d" later
torch.Tensor.to_sparse_csc
Tensor.to_sparse_csc() -> Tensor
Convert a tensor to compressed column storage (CSC) format.  Except
   for strided tensors, only works with 2D tensors.  If the "self" is
   strided, then the number of dense dimensions could be specified,
   and a hybrid CSC tensor will be created, with dense_dim dense
   dimensions and self.dim() - 2 - dense_dim batch dimension.
Parameters:
      dense_dim (int, optional) -- Number of dense
      dimensions of the resulting CSC tensor.  This argument should be
      used only if "self" is a strided tensor, and must be a value
      between 0 and dimension of "self" tensor minus two.
Example:
  >>> dense = torch.randn(5, 5)
  >>> sparse = dense.to_sparse_csc()
  >>> sparse._nnz()
  25

  >>> dense = torch.zeros(3, 3, 1, 1)
  >>> dense[0, 0] = dense[1, 2] = dense[2, 1] = 1
  >>> dense.to_sparse_csc(dense_dim=2)
  tensor(ccol_indices=tensor([0, 1, 2, 3]),
torch.save
torch.save(obj, f, pickle_module=pickle, pickle_protocol=DEFAULT_PROTOCOL, _use_new_zipfile_serialization=True)
Saves an object to a disk file.
See also: Saving and loading tensors
Parameters:
      * obj (object) -- saved object
  * **f** (*Union**[**str**, **PathLike**, **BinaryIO**,
    **IO**[**bytes**]**]*) -- a file-like object (has to implement
    write and flush) or a string or os.PathLike object containing
    a file name

  * **pickle_module** (*Any*) -- module used for pickling metadata
    and objects

  * **pickle_protocol** (*int*) -- can be specified to override
    the default protocol

Note:
 A common PyTorch convention is to save tensors using .pt file
 extension.

Note:
 PyTorch preserves storage sharing across serialization. See
 Saving and loading tensors preserves views for more details.

Note:
 The 1.6 release of PyTorch switched "torch.save" to use a new
zipfile-based file format. "torch.load" still retains the ability
     to load files in the old format. If for any reason you want
     "torch.save" to use the old format, pass the kwarg
     "_use_new_zipfile_serialization=False".
-[ Example ]-



Save to file
x = torch.tensor([0, 1, 2, 3, 4])
torch.save(x, 'tensor.pt')
Save to io.BytesIO buffer
buffer = io.BytesIO()
torch.save(x, buffer)
torch.triu
torch.triu(input, diagonal=0, *, out=None) -> Tensor
Returns the upper triangular part of a matrix (2-D tensor) or batch
   of matrices "input", the other elements of the result tensor "out"
   are set to 0.
The upper triangular part of the matrix is defined as the elements
   on and above the diagonal.
The argument "diagonal" controls which diagonal to consider. If
   "diagonal" = 0, all elements on and above the main diagonal are
   retained. A positive value excludes just as many diagonals above
   the main diagonal, and similarly a negative value includes just as
   many diagonals below the main diagonal. The main diagonal are the
   set of indices \lbrace (i, i) \rbrace for i \in [0, \min{d_{1},
   d_{2}} - 1] where d_{1}, d_{2} are the dimensions of the matrix.
Parameters:
      * input (Tensor) -- the input tensor.
  * **diagonal** (*int**, **optional*) -- the diagonal to consider

Keyword Arguments:
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(3, 3)
  >>> a
  tensor([[ 0.2309,  0.5207,  2.0049],
          [ 0.2072, -1.0680,  0.6602],
          [ 0.3480, -0.5211, -0.4573]])
  >>> torch.triu(a)
  tensor([[ 0.2309,  0.5207,  2.0049],
          [ 0.0000, -1.0680,  0.6602],
          [ 0.0000,  0.0000, -0.4573]])
  >>> torch.triu(a, diagonal=1)
  tensor([[ 0.0000,  0.5207,  2.0049],
          [ 0.0000,  0.0000,  0.6602],
          [ 0.0000,  0.0000,  0.0000]])
  >>> torch.triu(a, diagonal=-1)
  tensor([[ 0.2309,  0.5207,  2.0049],
          [ 0.2072, -1.0680,  0.6602],
          [ 0.0000, -0.5211, -0.4573]])

  >>> b = torch.randn(4, 6)
  >>> b
  tensor([[ 0.5876, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
          [-0.2447,  0.9556, -1.2919,  1.3378, -0.1768, -1.0857],
          [ 0.4333,  0.3146,  0.6576, -1.0432,  0.9348, -0.4410],
[-0.9888,  1.0679, -1.3337, -1.6556,  0.4798,  0.2830]])
      >>> torch.triu(b, diagonal=1)
      tensor([[ 0.0000, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
              [ 0.0000,  0.0000, -1.2919,  1.3378, -0.1768, -1.0857],
              [ 0.0000,  0.0000,  0.0000, -1.0432,  0.9348, -0.4410],
              [ 0.0000,  0.0000,  0.0000,  0.0000,  0.4798,  0.2830]])
      >>> torch.triu(b, diagonal=-1)
      tensor([[ 0.5876, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
              [-0.2447,  0.9556, -1.2919,  1.3378, -0.1768, -1.0857],
              [ 0.0000,  0.3146,  0.6576, -1.0432,  0.9348, -0.4410],
              [ 0.0000,  0.0000, -1.3337, -1.6556,  0.4798,  0.2830]])
torch.linalg.svdvals
torch.linalg.svdvals(A, *, driver=None, out=None) -> Tensor
Computes the singular values of a matrix.
Supports input of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if "A" is a batch of matrices
   then the output has the same batch dimensions.
The singular values are returned in descending order.
Note:
 This function is equivalent to NumPy's *linalg.svd(A,
 compute_uv=False)*.

Note:
 When inputs are on a CUDA device, this function synchronizes that
 device with the CPU.

See also:
 "torch.linalg.svd()" computes the full singular value
 decomposition.

Parameters:
      A (Tensor) -- tensor of shape (, m, n)* where *** is
      zero or more batch dimensions.
Keyword Arguments:
      * driver (str, optional) -- name of the cuSOLVER
        method to be used. This keyword argument only works on CUDA
inputs. Available options are: None, gesvd, gesvdj, and
        gesvda. Check "torch.linalg.svd()" for details. Default:
        None.
  * **out** (*Tensor**, **optional*) -- output tensor. Ignored if
    *None*. Default: *None*.

Returns:
      A real-valued tensor, even when "A" is complex.
Examples:
  >>> A = torch.randn(5, 3)
  >>> S = torch.linalg.svdvals(A)
  >>> S
  tensor([2.5139, 2.1087, 1.1066])

  >>> torch.dist(S, torch.linalg.svd(A, full_matrices=False).S)
  tensor(2.4576e-07)
AdamW
class torch.optim.AdamW(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.01, amsgrad=False, *, maximize=False, foreach=None, capturable=False, differentiable=False)
Implements AdamW algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma \text{(lr)}, \: \beta_1,
  \beta_2          \text{(betas)}, \: \theta_0 \text{(params)}, \:
  f(\theta) \text{(objective)},          \: \epsilon \text{
  (epsilon)}                                                    \\
  &\hspace{13mm}      \lambda \text{(weight decay)},  \:
  \textit{amsgrad},          \: \textit{maximize}
  \\      &\textbf{initialize} : m_0 \leftarrow 0 \text{ (first
  moment)}, v_0 \leftarrow 0          \text{ ( second moment)}, \:
  \widehat{v_0}^{max}\leftarrow 0              \\[-1.ex]
  &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\
\textbf{do}                         \
      &\hspace{5mm}\textbf{if} \: \textit{maximize}:
      \      &\hspace{10mm}g_t           \leftarrow
      -\nabla_{\theta} f_t (\theta_{t-1})          \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}g_t           \leftarrow   \nabla_{\theta}
      f_t (\theta_{t-1})           \      &\hspace{5mm} \theta_t
      \leftarrow \theta_{t-1} - \gamma \lambda \theta_{t-1}         \
      &\hspace{5mm}m_t           \leftarrow   \beta_1 m_{t-1} + (1 -
      \beta_1) g_t          \      &\hspace{5mm}v_t
      \leftarrow   \beta_2 v_{t-1} + (1-\beta_2) g^2_t          \
      &\hspace{5mm}\widehat{m_t} \leftarrow   m_t/\big(1-\beta_1^t
      \big)                   \      &\hspace{5mm}\widehat{v_t}
      \leftarrow   v_t/\big(1-\beta_2^t \big)                   \
      &\hspace{5mm}\textbf{if} \: amsgrad
      \      &\hspace{10mm}\widehat{v_t}^{max} \leftarrow
      \mathrm{max}(\widehat{v_t}^{max},          \widehat{v_t})
\      &\hspace{10mm}\theta_t \leftarrow \theta_t - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}^{max}} +
      \epsilon \big)                                 \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}\theta_t \leftarrow \theta_t - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}} + \epsilon
      \big)                                       \
      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
For further details regarding the algorithm we refer to Decoupled
   Weight Decay Regularization.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    1e-3)

  * **betas** (*Tuple**[**float**, **float**]**, **optional*) --
    coefficients used for computing running averages of gradient
and its square (default: (0.9, 0.999))
  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **weight_decay** (*float**, **optional*) -- weight decay
    coefficient (default: 1e-2)

  * **amsgrad** (*bool**, **optional*) -- whether to use the
    AMSGrad variant of this algorithm from the paper On the
    Convergence of Adam and Beyond (default: False)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used. If unspecified by the
    user (so foreach is None), we will try to use foreach over the
    for-loop implementation on CUDA, since it is usually
    significantly more performant. (default: None)

  * **capturable** (*bool**, **optional*) -- whether this instance
is safe to capture in a CUDA graph. Passing True can impair
        ungraphed performance, so if you don't intend to graph capture
        this instance, leave it False (default: False)
  * **differentiable** (*bool**, **optional*) -- whether autograd
    should occur through the optimizer step in training.
    Otherwise, the step() function runs in a torch.no_grad()
    context. Setting to True can impair performance, so leave it
    False if you don't intend to run autograd through this
    instance (default: False)

add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.

  This can be useful when fine tuning a pre-trained network as
  frozen layers can be made trainable and added to the "Optimizer"
  as training progresses.

  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
     **state_dict** (*dict*) -- optimizer state. Should be an
     object returned from a call to "state_dict()".

register_step_post_hook(hook)
  Register an optimizer step post hook which will be called after
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None

  The "optimizer" argument is the optimizer instance being used.

  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:
     "torch.utils.hooks.RemoveableHandle"

register_step_pre_hook(hook)
  Register an optimizer step pre hook which will be called before
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None or modified args and kwargs
The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:
     "torch.utils.hooks.RemoveableHandle"

state_dict()
  Returns the state of the optimizer as a "dict".

  It contains two entries:

  * state - a dict holding current optimization state. Its content
       differs between optimizer classes.

  * param_groups - a list containing all parameter groups where
    each
       parameter group is a dict

zero_grad(set_to_none=False)
  Sets the gradients of all optimized "torch.Tensor" s to zero.

  Parameters:
FuseCustomConfig
class torch.ao.quantization.fx.custom_config.FuseCustomConfig
Custom configuration for "fuse_fx()".
Example usage:
  fuse_custom_config = FuseCustomConfig().set_preserved_attributes(["attr1", "attr2"])

classmethod from_dict(fuse_custom_config_dict)
  Create a "ConvertCustomConfig" from a dictionary with the
  following items:

     "preserved_attributes": a list of attributes that persist
     even if they are not used in "forward"

  This function is primarily for backward compatibility and may be
  removed in the future.

  Return type:
     *FuseCustomConfig*

set_preserved_attributes(attributes)
  Set the names of the attributes that will persist in the graph
  module even if they are not used in the model's "forward"
  method.

  Return type:
     *FuseCustomConfig*

to_dict()
  Convert this "FuseCustomConfig" to a dictionary with the items
  described in "from_dict()".
Adam
class torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False, *, foreach=None, maximize=False, capturable=False, differentiable=False, fused=None)
Implements Adam algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma \text{ (lr)}, \beta_1,
  \beta_2          \text{ (betas)},\theta_0 \text{
  (params)},f(\theta) \text{ (objective)}          \\
  &\hspace{13mm}      \lambda \text{ (weight decay)},  \:
  \textit{amsgrad},          \:\textit{maximize}
  \\      &\textbf{initialize} :  m_0 \leftarrow 0 \text{ ( first
  moment)},          v_0\leftarrow 0 \text{ (second moment)},\:
  \widehat{v_0}^{max}\leftarrow 0\\[-1.ex]
  &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\
  &\hspace{5mm}\textbf{if} \: \textit{maximize}:
  \\      &\hspace{10mm}g_t           \leftarrow
-\nabla_{\theta} f_t (\theta_{t-1})         \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}g_t           \leftarrow   \nabla_{\theta}
      f_t (\theta_{t-1})          \      &\hspace{5mm}\textbf{if} \:
      \lambda \neq 0                                           \
      &\hspace{10mm} g_t \leftarrow g_t + \lambda  \theta_{t-1}
      \      &\hspace{5mm}m_t           \leftarrow   \beta_1 m_{t-1}
      + (1 - \beta_1) g_t          \      &\hspace{5mm}v_t
      \leftarrow   \beta_2 v_{t-1} + (1-\beta_2) g^2_t          \
      &\hspace{5mm}\widehat{m_t} \leftarrow   m_t/\big(1-\beta_1^t
      \big)                   \      &\hspace{5mm}\widehat{v_t}
      \leftarrow   v_t/\big(1-\beta_2^t \big)                   \
      &\hspace{5mm}\textbf{if} \: amsgrad
      \      &\hspace{10mm}\widehat{v_t}^{max} \leftarrow
      \mathrm{max}(\widehat{v_t}^{max},          \widehat{v_t})
      \      &\hspace{10mm}\theta_t \leftarrow \theta_{t-1} - \gamma
\widehat{m_t}/          \big(\sqrt{\widehat{v_t}^{max}} +
      \epsilon \big)                                 \
      &\hspace{5mm}\textbf{else}
      \      &\hspace{10mm}\theta_t \leftarrow \theta_{t-1} - \gamma
      \widehat{m_t}/          \big(\sqrt{\widehat{v_t}} + \epsilon
      \big)                                       \
      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
For further details regarding the algorithm we refer to Adam: A
   Method for Stochastic Optimization.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    1e-3)

  * **betas** (*Tuple**[**float**, **float**]**, **optional*) --
    coefficients used for computing running averages of gradient
    and its square (default: (0.9, 0.999))
and its square (default: (0.9, 0.999))
  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **weight_decay** (*float**, **optional*) -- weight decay (L2
    penalty) (default: 0)

  * **amsgrad** (*bool**, **optional*) -- whether to use the
    AMSGrad variant of this algorithm from the paper On the
    Convergence of Adam and Beyond (default: False)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used (default: None)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **capturable** (*bool**, **optional*) -- whether this instance
    is safe to capture in a CUDA graph. Passing True can impair
    ungraphed performance, so if you don't intend to graph capture
    this instance, leave it False (default: False)
differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()
        context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)

fused (bool, optional) -- whether the fused
    implementation (CUDA only) is used. Currently,
    torch.float64, torch.float32, torch.float16, and
    torch.bfloat16 are supported. Since the fused implementation
    is usually significantly faster than the for-loop
    implementation, we try to use it whenever possible (all
    parameters are on CUDA and are of a supported type). Else, we
    continue with the for-loop implementation. (default: None)



add_param_group(param_group)
  Add a param group to the "Optimizer" s *param_groups*.
This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
  Parameters:
     **param_group** (*dict*) -- Specifies what Tensors should be
     optimized along with group specific optimization options.

load_state_dict(state_dict)
  Loads the optimizer state.

  Parameters:
     **state_dict** (*dict*) -- optimizer state. Should be an
     object returned from a call to "state_dict()".

register_step_post_hook(hook)
  Register an optimizer step post hook which will be called after
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None

  The "optimizer" argument is the optimizer instance being used.

  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
"handle.remove()"
  Return type:
     "torch.utils.hooks.RemoveableHandle"

register_step_pre_hook(hook)
  Register an optimizer step pre hook which will be called before
  optimizer step. It should have the following signature:

     hook(optimizer, args, kwargs) -> None or modified args and kwargs

  The "optimizer" argument is the optimizer instance being used.
  If args and kwargs are modified by the pre-hook, then the
  transformed values are returned as a tuple containing the
  new_args and new_kwargs.

  Parameters:
     **hook** (*Callable*) -- The user defined hook to be
     registered.

  Returns:
     a handle that can be used to remove the added hook by calling
     "handle.remove()"

  Return type:
     "torch.utils.hooks.RemoveableHandle"

state_dict()
  Returns the state of the optimizer as a "dict".

  It contains two entries:

  * state - a dict holding current optimization state. Its content
differs between optimizer classes.
  * param_groups - a list containing all parameter groups where
    each
       parameter group is a dict

zero_grad(set_to_none=False)
  Sets the gradients of all optimized "torch.Tensor" s to zero.

  Parameters:
     **set_to_none** (*bool*) -- instead of setting to zero, set
     the grads to None. This will in general have lower memory
     footprint, and can modestly improve performance. However, it
     changes certain behaviors. For example: 1. When the user
     tries to access a gradient and perform manual ops on it, a
     None attribute or a Tensor full of 0s will behave
     differently. 2. If the user requests
     "zero_grad(set_to_none=True)" followed by a backward pass,
     ".grad"s are guaranteed to be None for params that did not
     receive a gradient. 3. "torch.optim" optimizers have a
     different behavior if the gradient is 0 or None (in one case
torch.nn.utils.skip_init
torch.nn.utils.skip_init(module_cls, args, *kwargs)
Given a module class object and args / kwargs, instantiates the
   module without initializing parameters / buffers.  This can be
   useful if initialization is slow or if custom initialization will
   be performed, making the default initialization unnecessary. There
   are some caveats to this, due to the way this function is
   implemented:


The module must accept a device arg in its constructor that is
   passed to any parameters or buffers created during construction.


The module must not perform any computation on parameters in its
   constructor except initialization (i.e. functions from
   "torch.nn.init").


If these conditions are satisfied, the module can be instantiated
   with parameter / buffer values uninitialized, as if having been
   created using "torch.empty()".
Parameters:
      * module_cls -- Class object; should be a subclass of
        "torch.nn.Module"
"torch.nn.Module"
  * **args** -- args to pass to the module's constructor

  * **kwargs** -- kwargs to pass to the module's constructor

Returns:
      Instantiated module with uninitialized parameters / buffers
Example:
  >>> import torch
  >>> m = torch.nn.utils.skip_init(torch.nn.Linear, 5, 1)
  >>> m.weight
  Parameter containing:
  tensor([[0.0000e+00, 1.5846e+29, 7.8307e+00, 2.5250e-29, 1.1210e-44]],
         requires_grad=True)
  >>> m2 = torch.nn.utils.skip_init(torch.nn.Linear, in_features=6, out_features=1)
  >>> m2.weight
  Parameter containing:
  tensor([[-1.4677e+24,  4.5915e-41,  1.4013e-45,  0.0000e+00, -1.4677e+24,
            4.5915e-41]], requires_grad=True)