example, "l" will be returned only once.
      Example:
         >>> l = nn.Linear(2, 2)
         >>> net = nn.Sequential(l, l)
         >>> for idx, m in enumerate(net.modules()):
         ...     print(idx, '->', m)
         0 -> Sequential(
           (0): Linear(in_features=2, out_features=2, bias=True)
           (1): Linear(in_features=2, out_features=2, bias=True)
         )
         1 -> Linear(in_features=2, out_features=2, bias=True)
   named_buffers(prefix='', recurse=True, remove_duplicate=True)
      Returns an iterator over module buffers, yielding both the name
      of the buffer as well as the buffer itself.
      Parameters:
         * prefix (str) -- prefix to prepend to all buffer
           names.
         * recurse (bool, optional) -- if True, then yields
           buffers of this module and all submodules. Otherwise,
           yields only buffers that are direct members of this module.
           Defaults to True.


Defaults to True.
         * remove_duplicate (bool, optional) -- whether to
           remove the duplicated buffers in the result. Defaults to
           True.
      Yields:
         (str, torch.Tensor) -- Tuple containing the name and buffer
      Return type:
         Iterator[Tuple[str, Tensor]]
      Example:
         >>> for name, buf in self.named_buffers():
         >>>     if name in ['running_var']:
         >>>         print(buf.size())
   named_children()
      Returns an iterator over immediate children modules, yielding
      both the name of the module as well as the module itself.
      Yields:
         (str, Module) -- Tuple containing a name and child module
      Return type:
         Iterator[Tuple[str, Module]]
      Example:
         >>> for name, module in model.named_children():
         >>>     if name in ['conv4', 'conv5']:
         >>>         print(module)
   named_modules(memo=None, prefix='', remove_duplicate=True)


Returns an iterator over all modules in the network, yielding
      both the name of the module as well as the module itself.
      Parameters:
         * memo (Optional[Set[Module]]) -- a memo to
           store the set of modules already added to the result
         * prefix (str) -- a prefix that will be added to the
           name of the module
         * remove_duplicate (bool) -- whether to remove the
           duplicated module instances in the result or not
      Yields:
         (str, Module) -- Tuple of name and module
      Note:
        Duplicate modules are returned only once. In the following
        example, "l" will be returned only once.
      Example:
         >>> l = nn.Linear(2, 2)
         >>> net = nn.Sequential(l, l)
         >>> for idx, m in enumerate(net.named_modules()):
         ...     print(idx, '->', m)
         0 -> ('', Sequential(
           (0): Linear(in_features=2, out_features=2, bias=True)


(1): Linear(in_features=2, out_features=2, bias=True)
         ))
         1 -> ('0', Linear(in_features=2, out_features=2, bias=True))
   named_parameters(prefix='', recurse=True, remove_duplicate=True)
      Returns an iterator over module parameters, yielding both the
      name of the parameter as well as the parameter itself.
      Parameters:
         * prefix (str) -- prefix to prepend to all parameter
           names.
         * recurse (bool) -- if True, then yields parameters of
           this module and all submodules. Otherwise, yields only
           parameters that are direct members of this module.
         * remove_duplicate (bool, optional) -- whether to
           remove the duplicated parameters in the result. Defaults to
           True.
      Yields:
         (str, Parameter) -- Tuple containing the name and parameter
      Return type:
         Iterator[Tuple[str, Parameter]]
      Example:


Example:
         >>> for name, param in self.named_parameters():
         >>>     if name in ['bias']:
         >>>         print(param.size())
   parameters(recurse=True)
      Returns an iterator over module parameters.
      This is typically passed to an optimizer.
      Parameters:
         recurse (bool) -- if True, then yields parameters of
         this module and all submodules. Otherwise, yields only
         parameters that are direct members of this module.
      Yields:
         Parameter -- module parameter
      Return type:
         Iterator[Parameter]
      Example:
         >>> for param in model.parameters():
         >>>     print(type(param), param.size())
          (20L,)
          (20L, 1L, 5L, 5L)
   register_backward_hook(hook)
      Registers a backward hook on the module.
      This function is deprecated in favor of
      "register_full_backward_hook()" and the behavior of this


function will change in future versions.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemovableHandle"
   register_buffer(name, tensor, persistent=True)
      Adds a buffer to the module.
      This is typically used to register a buffer that should not to
      be considered a model parameter. For example, BatchNorm's
      "running_mean" is not a parameter, but is part of the module's
      state. Buffers, by default, are persistent and will be saved
      alongside parameters. This behavior can be changed by setting
      "persistent" to "False". The only difference between a
      persistent buffer and a non-persistent buffer is that the latter
      will not be a part of this module's "state_dict".
      Buffers can be accessed as attributes using given names.
      Parameters:
         * name (str) -- name of the buffer. The buffer can be


accessed from this module using the given name
         * tensor (Tensor or None) -- buffer to be
           registered. If "None", then operations that run on buffers,
           such as "cuda", are ignored. If "None", the buffer is
           not included in the module's "state_dict".
         * persistent (bool) -- whether the buffer is part of
           this module's "state_dict".
      Example:
         >>> self.register_buffer('running_mean', torch.zeros(num_features))
   register_forward_hook(hook, *, prepend=False, with_kwargs=False)
      Registers a forward hook on the module.
      The hook will be called every time after "forward()" has
      computed an output.
      If "with_kwargs" is "False" or not specified, the input contains
      only the positional arguments given to the module. Keyword
      arguments won't be passed to the hooks and only to the
      "forward". The hook can modify the output. It can modify the


input inplace but it will not have effect on forward since this
      is called after "forward()" is called. The hook should have the
      following signature:
         hook(module, args, output) -> None or modified output
      If "with_kwargs" is "True", the forward hook will be passed the
      "kwargs" given to the forward function and be expected to return
      the output possibly modified. The hook should have the following
      signature:
         hook(module, args, kwargs, output) -> None or modified output
      Parameters:
         * hook (Callable) -- The user defined hook to be
           registered.
         * prepend (bool) -- If "True", the provided "hook" will
           be fired before all existing "forward" hooks on this
           "torch.nn.modules.Module". Otherwise, the provided "hook"
           will be fired after all existing "forward" hooks on this
           "torch.nn.modules.Module". Note that global "forward" hooks


registered with "register_module_forward_hook()" will fire
           before all hooks registered by this method. Default:
           "False"
         * with_kwargs (bool) -- If "True", the "hook" will be
           passed the kwargs given to the forward function. Default:
           "False"
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemovableHandle"
   register_forward_pre_hook(hook, *, prepend=False, with_kwargs=False)
      Registers a forward pre-hook on the module.
      The hook will be called every time before "forward()" is
      invoked.
      If "with_kwargs" is false or not specified, the input contains
      only the positional arguments given to the module. Keyword
      arguments won't be passed to the hooks and only to the
      "forward". The hook can modify the input. User can either return
      a tuple or a single modified value in the hook. We will wrap the


value into a tuple if a single value is returned (unless that
      value is already a tuple). The hook should have the following
      signature:
         hook(module, args) -> None or modified input
      If "with_kwargs" is true, the forward pre-hook will be passed
      the kwargs given to the forward function. And if the hook
      modifies the input, both the args and kwargs should be returned.
      The hook should have the following signature:
         hook(module, args, kwargs) -> None or a tuple of modified input and kwargs
      Parameters:
         * hook (Callable) -- The user defined hook to be
           registered.
         * prepend (bool) -- If true, the provided "hook" will
           be fired before all existing "forward_pre" hooks on this
           "torch.nn.modules.Module". Otherwise, the provided "hook"
           will be fired after all existing "forward_pre" hooks on
           this "torch.nn.modules.Module". Note that global


"forward_pre" hooks registered with
           "register_module_forward_pre_hook()" will fire before all
           hooks registered by this method. Default: "False"
         * with_kwargs (bool) -- If true, the "hook" will be
           passed the kwargs given to the forward function. Default:
           "False"
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemovableHandle"
   register_full_backward_hook(hook, prepend=False)
      Registers a backward hook on the module.
      The hook will be called every time the gradients with respect to
      a module are computed, i.e. the hook will execute if and only if
      the gradients with respect to module outputs are computed. The
      hook should have the following signature:
         hook(module, grad_input, grad_output) -> tuple(Tensor) or None
      The "grad_input" and "grad_output" are tuples that contain the


gradients with respect to the inputs and outputs respectively.
      The hook should not modify its arguments, but it can optionally
      return a new gradient with respect to the input that will be
      used in place of "grad_input" in subsequent computations.
      "grad_input" will only correspond to the inputs given as
      positional arguments and all kwarg arguments are ignored.
      Entries in "grad_input" and "grad_output" will be "None" for all
      non-Tensor arguments.
      For technical reasons, when this hook is applied to a Module,
      its forward function will receive a view of each Tensor passed
      to the Module. Similarly the caller will receive a view of each
      Tensor returned by the Module's forward function.
      Warning:
        Modifying inputs or outputs inplace is not allowed when using
        backward hooks and will raise an error.
      Parameters:
         * hook (Callable) -- The user-defined hook to be
           registered.


registered.
         * prepend (bool) -- If true, the provided "hook" will
           be fired before all existing "backward" hooks on this
           "torch.nn.modules.Module". Otherwise, the provided "hook"
           will be fired after all existing "backward" hooks on this
           "torch.nn.modules.Module". Note that global "backward"
           hooks registered with
           "register_module_full_backward_hook()" will fire before all
           hooks registered by this method.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemovableHandle"
   register_full_backward_pre_hook(hook, prepend=False)
      Registers a backward pre-hook on the module.
      The hook will be called every time the gradients for the module
      are computed. The hook should have the following signature:
         hook(module, grad_output) -> Tensor or None


The "grad_output" is a tuple. The hook should not modify its
      arguments, but it can optionally return a new gradient with
      respect to the output that will be used in place of
      "grad_output" in subsequent computations. Entries in
      "grad_output" will be "None" for all non-Tensor arguments.
      For technical reasons, when this hook is applied to a Module,
      its forward function will receive a view of each Tensor passed
      to the Module. Similarly the caller will receive a view of each
      Tensor returned by the Module's forward function.
      Warning:
        Modifying inputs inplace is not allowed when using backward
        hooks and will raise an error.
      Parameters:
         * hook (Callable) -- The user-defined hook to be
           registered.
         * prepend (bool) -- If true, the provided "hook" will
           be fired before all existing "backward_pre" hooks on this
           "torch.nn.modules.Module". Otherwise, the provided "hook"


will be fired after all existing "backward_pre" hooks on
           this "torch.nn.modules.Module". Note that global
           "backward_pre" hooks registered with
           "register_module_full_backward_pre_hook()" will fire before
           all hooks registered by this method.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemovableHandle"
   register_load_state_dict_post_hook(hook)
      Registers a post hook to be run after module's "load_state_dict"
      is called.
      It should have the following signature::
         hook(module, incompatible_keys) -> None
      The "module" argument is the current module that this hook is
      registered on, and the "incompatible_keys" argument is a
      "NamedTuple" consisting of attributes "missing_keys" and
      "unexpected_keys". "missing_keys" is a "list" of "str"


containing the missing keys and "unexpected_keys" is a "list" of
      "str" containing the unexpected keys.
      The given incompatible_keys can be modified inplace if needed.
      Note that the checks performed when calling "load_state_dict()"
      with "strict=True" are affected by modifications the hook makes
      to "missing_keys" or "unexpected_keys", as expected. Additions
      to either set of keys will result in an error being thrown when
      "strict=True", and clearing out both missing and unexpected keys
      will avoid an error.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemovableHandle"
   register_module(name, module)
      Alias for "add_module()".
   register_parameter(name, param)
      Adds a parameter to the module.
      The parameter can be accessed as an attribute using given name.
      Parameters:


Parameters:
         * name (str) -- name of the parameter. The parameter
           can be accessed from this module using the given name
         * param (Parameter or None) -- parameter to be
           added to the module. If "None", then operations that run on
           parameters, such as "cuda", are ignored. If "None", the
           parameter is not included in the module's "state_dict".
   register_state_dict_pre_hook(hook)
      These hooks will be called with arguments: "self", "prefix", and
      "keep_vars" before calling "state_dict" on "self". The
      registered hooks can be used to perform pre-processing before
      the "state_dict" call is made.
   requires_grad_(requires_grad=True)
      Change if autograd should record operations on parameters in
      this module.
      This method sets the parameters' "requires_grad" attributes in-
      place.
      This method is helpful for freezing part of the module for


finetuning or training parts of a model individually (e.g., GAN
      training).
      See Locally disabling gradient computation for a comparison
      between .requires_grad_() and several similar mechanisms that
      may be confused with it.
      Parameters:
         requires_grad (bool) -- whether autograd should record
         operations on parameters in this module. Default: "True".
      Returns:
         self
      Return type:
         Module
   save(f, extra_files={})
      See "torch.jit.save" for details.
   set_extra_state(state)
      This function is called from "load_state_dict()" to handle any
      extra state found within the state_dict. Implement this
      function and a corresponding "get_extra_state()" for your module
      if you need to store extra state within its state_dict.
      Parameters:
         state (dict) -- Extra state from the state_dict
   share_memory()
      See "torch.Tensor.share_memory()"
      Return type:
         T


Return type:
         T
   state_dict(args, destination=None, prefix='', keep_vars=False)
      Returns a dictionary containing references to the whole state of
      the module.
      Both parameters and persistent buffers (e.g. running averages)
      are included. Keys are corresponding parameter and buffer names.
      Parameters and buffers set to "None" are not included.
      Note:
        The returned object is a shallow copy. It contains references
        to the module's parameters and buffers.
      Warning:
        Currently "state_dict()" also accepts positional arguments for
        "destination", "prefix" and "keep_vars" in order. However,
        this is being deprecated and keyword arguments will be
        enforced in future releases.
      Warning:
        Please avoid the use of argument "destination" as it is not
        designed for end-users.
      Parameters:
         * destination (dict, optional*) -- If provided, the


state of module will be updated into the dict and the same
           object is returned. Otherwise, an "OrderedDict" will be
           created and returned. Default: "None".
         * prefix (str, optional) -- a prefix added to
           parameter and buffer names to compose the keys in
           state_dict. Default: "''".
         * keep_vars (bool, optional) -- by default the
           "Tensor" s returned in the state dict are detached from
           autograd. If it's set to "True", detaching will not be
           performed. Default: "False".
      Returns:
         a dictionary containing a whole state of the module
      Return type:
         dict
      Example:
         >>> module.state_dict().keys()
         ['bias', 'weight']
   to(args, *kwargs)
      Moves and/or casts the parameters and buffers.
      This can be called as
      to(device=None, dtype=None, non_blocking=False)
      to(dtype, non_blocking=False)
      to(tensor, non_blocking=False)


to(tensor, non_blocking=False)
      to(memory_format=torch.channels_last)
      Its signature is similar to "torch.Tensor.to()", but only
      accepts floating point or complex "dtype"s. In addition, this
      method will only cast the floating point or complex parameters
      and buffers to "dtype" (if given). The integral parameters and
      buffers will be moved "device", if that is given, but with
      dtypes unchanged. When "non_blocking" is set, it tries to
      convert/move asynchronously with respect to the host if
      possible, e.g., moving CPU Tensors with pinned memory to CUDA
      devices.
      See below for examples.
      Note:
        This method modifies the module in-place.
      Parameters:
         * device ("torch.device") -- the desired device of the
           parameters and buffers in this module
         * dtype ("torch.dtype") -- the desired floating point or
           complex dtype of the parameters and buffers in this module


tensor (torch.Tensor) -- Tensor whose dtype and
           device are the desired dtype and device for all parameters
           and buffers in this module
         * memory_format ("torch.memory_format") -- the desired
           memory format for 4D parameters and buffers in this module
           (keyword only argument)
      Returns:
         self
      Return type:
         Module
      Examples:
         >>> linear = nn.Linear(2, 2)
         >>> linear.weight
         Parameter containing:
         tensor([[ 0.1913, -0.3420],
                 [-0.5113, -0.2325]])
         >>> linear.to(torch.double)
         Linear(in_features=2, out_features=2, bias=True)
         >>> linear.weight
         Parameter containing:
         tensor([[ 0.1913, -0.3420],
                 [-0.5113, -0.2325]], dtype=torch.float64)
         >>> gpu1 = torch.device("cuda:1")
         >>> linear.to(gpu1, dtype=torch.half, non_blocking=True)
         Linear(in_features=2, out_features=2, bias=True)


linear.weight
         Parameter containing:
         tensor([[ 0.1914, -0.3420],
                 [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1')
         >>> cpu = torch.device("cpu")
         >>> linear.to(cpu)
         Linear(in_features=2, out_features=2, bias=True)
         >>> linear.weight
         Parameter containing:
         tensor([[ 0.1914, -0.3420],
                 [-0.5112, -0.2324]], dtype=torch.float16)
         >>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble)
         >>> linear.weight
         Parameter containing:
         tensor([[ 0.3741+0.j,  0.2382+0.j],
                 [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128)
         >>> linear(torch.ones(3, 2, dtype=torch.cdouble))
         tensor([[0.6122+0.j, 0.1150+0.j],
                 [0.6122+0.j, 0.1150+0.j],
                 [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)
   to_empty(*, device)
      Moves the parameters and buffers to the specified device without
      copying storage.


copying storage.
      Parameters:
         device ("torch.device") -- The desired device of the
         parameters and buffers in this module.
      Returns:
         self
      Return type:
         Module
   train(mode=True)
      Sets the module in training mode.
      This has any effect only on certain modules. See documentations
      of particular modules for details of their behaviors in
      training/evaluation mode, if they are affected, e.g. "Dropout",
      "BatchNorm", etc.
      Parameters:
         mode (bool) -- whether to set training mode ("True") or
         evaluation mode ("False"). Default: "True".
      Returns:
         self
      Return type:
         Module
   type(dst_type)
      Casts all parameters and buffers to "dst_type".
      Note:
        This method modifies the module in-place.
      Parameters:
         dst_type (type or string) -- the desired type
      Returns:
         self
      Return type:
         Module
   xpu(device=None)


Module
   xpu(device=None)
      Moves all model parameters and buffers to the XPU.
      This also makes associated parameters and buffers different
      objects. So it should be called before constructing optimizer if
      the module will live on XPU while being optimized.
      Note:
        This method modifies the module in-place.
      Parameters:
         device (int, optional) -- if specified, all
         parameters will be copied to that device
      Returns:
         self
      Return type:
         Module
   zero_grad(set_to_none=False)
      Sets gradients of all model parameters to zero. See similar
      function under "torch.optim.Optimizer" for more context.
      Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. See "torch.optim.Optimizer.zero_grad()"
         for details.


torch.nn.functional.avg_pool2dtorch.nn.functional.avg_pool2d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True, divisor_override=None) -> Tensor
   Applies 2D average-pooling operation in kH \times kW regions by
   step size sH \times sW steps. The number of output features is
   equal to the number of input planes.
   See "AvgPool2d" for details and output shape.
   Parameters:
      * input -- input tensor (\text{minibatch} ,
        \text{in_channels} , iH , iW)
      * kernel_size -- size of the pooling region. Can be a single
        number or a tuple (kH, kW)
      * stride -- stride of the pooling operation. Can be a single
        number or a tuple (sH, sW). Default: "kernel_size"
      * padding -- implicit zero paddings on both sides of the
        input. Can be a single number or a tuple (padH, padW).
        Default: 0
      * ceil_mode -- when True, will use ceil instead of floor


in the formula to compute the output shape. Default: "False"
      * count_include_pad -- when True, will include the zero-
        padding in the averaging calculation. Default: "True"
      * divisor_override -- if specified, it will be used as
        divisor, otherwise size of the pooling region will be used.
        Default: None


torch.Tensor.valuesTensor.values() -> Tensor
   Return the values tensor of a sparse COO tensor.
   Warning:
     Throws an error if "self" is not a sparse COO tensor.
   See also "Tensor.indices()".
   Note:
     This method can only be called on a coalesced sparse tensor. See
     "Tensor.coalesce()" for details.


torch.Tensor.to_sparseTensor.to_sparse(sparseDims) -> Tensor
   Returns a sparse copy of the tensor.  PyTorch supports sparse
   tensors in coordinate format.
   Parameters:
      sparseDims (int, optional) -- the number of sparse
      dimensions to include in the new sparse tensor
   Example:
      >>> d = torch.tensor([[0, 0, 0], [9, 0, 10], [0, 0, 0]])
      >>> d
      tensor([[ 0,  0,  0],
              [ 9,  0, 10],
              [ 0,  0,  0]])
      >>> d.to_sparse()
      tensor(indices=tensor([[1, 1],
                             [0, 2]]),
             values=tensor([ 9, 10]),
             size=(3, 3), nnz=2, layout=torch.sparse_coo)
      >>> d.to_sparse(1)
      tensor(indices=tensor([[1]]),
             values=tensor([[ 9,  0, 10]]),
             size=(3, 3), nnz=1, layout=torch.sparse_coo)
   to_sparse(*, layout=None, blocksize=None, dense_dim=None) -> Tensor
   Returns a sparse tensor with the specified layout and blocksize.


If the "self" is strided, the number of dense dimensions could be
   specified, and a hybrid sparse tensor will be created, with
   dense_dim dense dimensions and self.dim() - 2 - dense_dim batch
   dimension.
   Note:
     If the "self" layout and blocksize parameters match with the
     specified layout and blocksize, return "self". Otherwise, return
     a sparse tensor copy of "self".
   Parameters:
      * layout ("torch.layout", optional) -- The desired sparse
        layout. One of "torch.sparse_coo", "torch.sparse_csr",
        "torch.sparse_csc", "torch.sparse_bsr", or "torch.sparse_bsc".
        Default: if "None", "torch.sparse_coo".
      * blocksize (list, tuple, "torch.Size", optional) -- Block
        size of the resulting BSR or BSC tensor. For other layouts,
        specifying the block size that is not "None" will result in a
        RuntimeError exception.  A block size must be a tuple of
        length two such that its items evenly divide the two sparse
        dimensions.


dimensions.
      * dense_dim (int, optional) -- Number of dense
        dimensions of the resulting CSR, CSC, BSR or BSC tensor.  This
        argument should be used only if "self" is a strided tensor,
        and must be a value between 0 and dimension of "self" tensor
        minus two.
   Example:
      >>> x = torch.tensor([[1, 0], [0, 0], [2, 3]])
      >>> x.to_sparse(layout=torch.sparse_coo)
      tensor(indices=tensor([[0, 2, 2],
                             [0, 0, 1]]),
             values=tensor([1, 2, 3]),
             size=(3, 2), nnz=3, layout=torch.sparse_coo)
      >>> x.to_sparse(layout=torch.sparse_bsr, blocksize=(1, 2))
      tensor(crow_indices=tensor([0, 1, 1, 2]),
             col_indices=tensor([0, 0]),
             values=tensor([[[1, 0]],
                            [[2, 3]]]), size=(3, 2), nnz=2, layout=torch.sparse_bsr)
      >>> x.to_sparse(layout=torch.sparse_bsr, blocksize=(2, 1))
      RuntimeError: Tensor size(-2) 3 needs to be divisible by blocksize[0] 2


x.to_sparse(layout=torch.sparse_csr, blocksize=(3, 1))
      RuntimeError: to_sparse for Strided to SparseCsr conversion does not use specified blocksize
      >>> x = torch.tensor([[[1], [0]], [[0], [0]], [[2], [3]]])
      >>> x.to_sparse(layout=torch.sparse_csr, dense_dim=1)
      tensor(crow_indices=tensor([0, 1, 1, 3]),
             col_indices=tensor([0, 0, 1]),
             values=tensor([[1],
                            [2],
                            [3]]), size=(3, 2, 1), nnz=3, layout=torch.sparse_csr)


torch.Tensor.ccol_indicesTensor.ccol_indices()


torch.Tensor.selectTensor.select(dim, index) -> Tensor
   See "torch.select()"


Adamaxclass torch.optim.Adamax(params, lr=0.002, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, foreach=None, *, maximize=False, differentiable=False)
   Implements Adamax algorithm (a variant of Adam based on infinity
   norm).
      \begin{aligned}      &\rule{110mm}{0.4pt}
      \      &\textbf{input}      : \gamma \text{ (lr)}, \beta_1,
      \beta_2          \text{ (betas)},\theta_0 \text{
      (params)},f(\theta) \text{ (objective)},          \: \lambda
      \text{ (weight decay)},
      \      &\hspace{13mm}    \epsilon \text{ (epsilon)}
      \      &\textbf{initialize} :  m_0 \leftarrow 0 \text{ ( first
      moment)},          u_0 \leftarrow 0 \text{ ( infinity norm)}
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
      \textbf{do}                         \      &\hspace{5mm}g_t
      \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \
      &\hspace{5mm}if \: \lambda \neq 0


&\hspace{5mm}if \: \lambda \neq 0
      \      &\hspace{10mm} g_t \leftarrow g_t + \lambda
      \theta_{t-1}                            \      &\hspace{5mm}m_t
      \leftarrow   \beta_1 m_{t-1} + (1 - \beta_1) g_t
      \      &\hspace{5mm}u_t      \leftarrow   \mathrm{max}(\beta_2
      u_{t-1}, |g_{t}|+\epsilon)   \      &\hspace{5mm}\theta_t
      \leftarrow \theta_{t-1} - \frac{\gamma m_t}{(1-\beta^t_1) u_t}
      \      &\rule{110mm}{0.4pt}
      \[-1.ex]      &\bf{return} \:  \theta_t
      \[-1.ex]      &\rule{110mm}{0.4pt}
      \[-1.ex] \end{aligned}
   For further details regarding the algorithm we refer to Adam: A
   Method for Stochastic Optimization.
   Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
      * lr (float, optional) -- learning rate (default:
        2e-3)
      * betas (Tuple[float, float], optional) --


coefficients used for computing running averages of gradient
        and its square
      * eps (float, optional) -- term added to the
        denominator to improve numerical stability (default: 1e-8)
      * weight_decay (float, optional) -- weight decay (L2
        penalty) (default: 0)
      * foreach (bool, optional) -- whether foreach
        implementation of optimizer is used. If unspecified by the
        user (so foreach is None), we will try to use foreach over the
        for-loop implementation on CUDA, since it is usually
        significantly more performant. (default: None)
      * maximize (bool, optional) -- maximize the params
        based on the objective, instead of minimizing (default: False)
      * differentiable (bool, optional) -- whether autograd
        should occur through the optimizer step in training.
        Otherwise, the step() function runs in a torch.no_grad()


context. Setting to True can impair performance, so leave it
        False if you don't intend to run autograd through this
        instance (default: False)
   add_param_group(param_group)
      Add a param group to the "Optimizer" s param_groups.
      This can be useful when fine tuning a pre-trained network as
      frozen layers can be made trainable and added to the "Optimizer"
      as training progresses.
      Parameters:
         param_group (dict) -- Specifies what Tensors should be
         optimized along with group specific optimization options.
   load_state_dict(state_dict)
      Loads the optimizer state.
      Parameters:
         state_dict (dict) -- optimizer state. Should be an
         object returned from a call to "state_dict()".
   register_step_post_hook(hook)
      Register an optimizer step post hook which will be called after
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None


hook(optimizer, args, kwargs) -> None
      The "optimizer" argument is the optimizer instance being used.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   register_step_pre_hook(hook)
      Register an optimizer step pre hook which will be called before
      optimizer step. It should have the following signature:
         hook(optimizer, args, kwargs) -> None or modified args and kwargs
      The "optimizer" argument is the optimizer instance being used.
      If args and kwargs are modified by the pre-hook, then the
      transformed values are returned as a tuple containing the
      new_args and new_kwargs.
      Parameters:
         hook (Callable) -- The user defined hook to be
         registered.
      Returns:


registered.
      Returns:
         a handle that can be used to remove the added hook by calling
         "handle.remove()"
      Return type:
         "torch.utils.hooks.RemoveableHandle"
   state_dict()
      Returns the state of the optimizer as a "dict".
      It contains two entries:
      * state - a dict holding current optimization state. Its content
           differs between optimizer classes.
      * param_groups - a list containing all parameter groups where
        each
           parameter group is a dict
   zero_grad(set_to_none=False)
      Sets the gradients of all optimized "torch.Tensor" s to zero.
      Parameters:
         set_to_none (bool) -- instead of setting to zero, set
         the grads to None. This will in general have lower memory
         footprint, and can modestly improve performance. However, it
         changes certain behaviors. For example: 1. When the user
         tries to access a gradient and perform manual ops on it, a


None attribute or a Tensor full of 0s will behave
         differently. 2. If the user requests
         "zero_grad(set_to_none=True)" followed by a backward pass,
         ".grad"s are guaranteed to be None for params that did not
         receive a gradient. 3. "torch.optim" optimizers have a
         different behavior if the gradient is 0 or None (in one case
         it does the step with a gradient of 0 and in the other it
         skips the step altogether).


torch.foreach_cos_torch._foreach_cos(self: List[Tensor]) -> None
   Apply "torch.cos()" to each Tensor of the input list.


ModuleDictclass torch.nn.ModuleDict(modules=None)
   Holds submodules in a dictionary.
   "ModuleDict" can be indexed like a regular Python dictionary, but
   modules it contains are properly registered, and will be visible by
   all "Module" methods.
   "ModuleDict" is an ordered dictionary that respects
   * the order of insertion, and
   * in "update()", the order of the merged "OrderedDict", "dict"
     (started from Python 3.6) or another "ModuleDict" (the argument
     to "update()").
   Note that "update()" with other unordered mapping types (e.g.,
   Python's plain "dict" before Python version 3.6) does not preserve
   the order of the merged mapping.
   Parameters:
      modules (iterable, optional) -- a mapping (dictionary)
      of (string: module) or an iterable of key-value pairs of type
      (string, module)
   Example:
      class MyModule(nn.Module):
          def init(self):
              super(MyModule, self).init()


super(MyModule, self).init()
              self.choices = nn.ModuleDict({
                      'conv': nn.Conv2d(10, 10, 3),
                      'pool': nn.MaxPool2d(3)
              })
              self.activations = nn.ModuleDict([
                      ['lrelu', nn.LeakyReLU()],
                      ['prelu', nn.PReLU()]
              ])
          def forward(self, x, choice, act):
              x = self.choiceschoice
              x = self.activationsact
              return x
   clear()
      Remove all items from the ModuleDict.
   items()
      Return an iterable of the ModuleDict key/value pairs.
      Return type:
         Iterable[Tuple[str, Module]]
   keys()
      Return an iterable of the ModuleDict keys.
      Return type:
         Iterable[str]
   pop(key)
      Remove key from the ModuleDict and return its module.
      Parameters:
         key (str) -- key to pop from the ModuleDict
      Return type:
         Module
   update(modules)


Module
   update(modules)
      Update the "ModuleDict" with the key-value pairs from a mapping
      or an iterable, overwriting existing keys.
      Note:
        If "modules" is an "OrderedDict", a "ModuleDict", or an
        iterable of key-value pairs, the order of new elements in it
        is preserved.
      Parameters:
         modules (iterable) -- a mapping (dictionary) from
         string to "Module", or an iterable of key-value pairs of type
         (string, "Module")
   values()
      Return an iterable of the ModuleDict values.
      Return type:
         Iterable[Module]


torch.Tensor.tensor_splitTensor.tensor_split(indices_or_sections, dim=0) -> List of Tensors
   See "torch.tensor_split()"


OneCycleLRclass torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr, total_steps=None, epochs=None, steps_per_epoch=None, pct_start=0.3, anneal_strategy='cos', cycle_momentum=True, base_momentum=0.85, max_momentum=0.95, div_factor=25.0, final_div_factor=10000.0, three_phase=False, last_epoch=- 1, verbose=False)
   Sets the learning rate of each parameter group according to the
   1cycle learning rate policy. The 1cycle policy anneals the learning
   rate from an initial learning rate to some maximum learning rate
   and then from that maximum learning rate to some minimum learning
   rate much lower than the initial learning rate. This policy was
   initially described in the paper Super-Convergence: Very Fast
   Training of Neural Networks Using Large Learning Rates.
   The 1cycle learning rate policy changes the learning rate after
   every batch. step should be called after a batch has been used
   for training.
   This scheduler is not chainable.


This scheduler is not chainable.
   Note also that the total number of steps in the cycle can be
   determined in one of two ways (listed in order of precedence):
   1. A value for total_steps is explicitly provided.
   2. A number of epochs (epochs) and a number of steps per epoch
      (steps_per_epoch) are provided. In this case, the number of
      total steps is inferred by total_steps = epochs *
      steps_per_epoch
   You must either provide a value for total_steps or provide a value
   for both epochs and steps_per_epoch.
   The default behaviour of this scheduler follows the fastai
   implementation of 1cycle, which claims that "unpublished work has
   shown even better results by using only two phases". To mimic the
   behaviour of the original paper instead, set "three_phase=True".
   Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
      * max_lr (float or list) -- Upper learning rate
        boundaries in the cycle for each parameter group.


total_steps (int) -- The total number of steps in the
        cycle. Note that if a value is not provided here, then it must
        be inferred by providing a value for epochs and
        steps_per_epoch. Default: None
epochs (int) -- The number of epochs to train for. This
    is used along with steps_per_epoch in order to infer the total
    number of steps in the cycle if a value for total_steps is not
    provided. Default: None
steps_per_epoch (int) -- The number of steps per epoch
    to train for. This is used along with epochs in order to infer
    the total number of steps in the cycle if a value for
    total_steps is not provided. Default: None
pct_start (float) -- The percentage of the cycle (in
    number of steps) spent increasing the learning rate. Default:
    0.3
anneal_strategy (str) -- {'cos', 'linear'} Specifies the
    annealing strategy: "cos" for cosine annealing, "linear" for


linear annealing. Default: 'cos'
      * cycle_momentum (bool) -- If "True", momentum is cycled
        inversely to learning rate between 'base_momentum' and
        'max_momentum'. Default: True
      * base_momentum (float or list) -- Lower momentum
        boundaries in the cycle for each parameter group. Note that
        momentum is cycled inversely to learning rate; at the peak of
        a cycle, momentum is 'base_momentum' and learning rate is
        'max_lr'. Default: 0.85
      * max_momentum (float or list) -- Upper momentum
        boundaries in the cycle for each parameter group.
        Functionally, it defines the cycle amplitude (max_momentum -
        base_momentum). Note that momentum is cycled inversely to
        learning rate; at the start of a cycle, momentum is
        'max_momentum' and learning rate is 'base_lr' Default: 0.95
      * div_factor (float) -- Determines the initial learning


rate via initial_lr = max_lr/div_factor Default: 25
      * final_div_factor (float) -- Determines the minimum
        learning rate via min_lr = initial_lr/final_div_factor
        Default: 1e4
      * three_phase (bool) -- If "True", use a third phase of
        the schedule to annihilate the learning rate according to
        'final_div_factor' instead of modifying the second phase (the
        first two phases will be symmetrical about the step indicated
        by 'pct_start').
      * last_epoch (int) -- The index of the last batch. This
        parameter is used when resuming a training job. Since step()
        should be invoked after each batch instead of after each
        epoch, this number represents the total number of batches
        computed, not the total number of epochs computed. When
        last_epoch=-1, the schedule is started from the beginning.
        Default: -1
      * verbose (bool) -- If "True", prints a message to stdout


for each update. Default: "False".
   -[ Example ]-

data_loader = torch.utils.data.DataLoader(...)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
scheduler = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr=0.01, steps_per_epoch=len(data_loader), epochs=10)
for epoch in range(10):
    for batch in data_loader:
        train_batch(...)
        scheduler.step()
   get_last_lr()
      Return last computed learning rate by current scheduler.
   load_state_dict(state_dict)
      Loads the schedulers state.
      Parameters:
         state_dict (dict) -- scheduler state. Should be an
         object returned from a call to "state_dict()".
   print_lr(is_verbose, group, lr, epoch=None)
      Display the current learning rate.
   state_dict()
      Returns the state of the scheduler as a "dict".
      It contains an entry for every variable in self.dict which
      is not the optimizer.


torch.Tensor.untyped_storageTensor.untyped_storage() -> torch.UntypedStorage
   Returns the underlying "UntypedStorage".


InstanceNorm3dclass torch.ao.nn.quantized.InstanceNorm3d(num_features, weight, bias, scale, zero_point, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False, device=None, dtype=None)
   This is the quantized version of "InstanceNorm3d".
   Additional args:
      * scale - quantization scale of the output, type: double.
      * zero_point - quantization zero point of the output, type:
        long.


torch.nn.functional.avg_pool1dtorch.nn.functional.avg_pool1d(input, kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True) -> Tensor
   Applies a 1D average pooling over an input signal composed of
   several input planes.
   See "AvgPool1d" for details and output shape.
   Parameters:
      * input -- input tensor of shape (\text{minibatch} ,
        \text{in_channels} , iW)
      * kernel_size -- the size of the window. Can be a single
        number or a tuple (kW,)
      * stride -- the stride of the window. Can be a single number
        or a tuple (sW,). Default: "kernel_size"
      * padding -- implicit zero paddings on both sides of the
        input. Can be a single number or a tuple (padW,). Default: 0
      * ceil_mode -- when True, will use ceil instead of floor
        to compute the output shape. Default: "False"
      * count_include_pad -- when True, will include the zero-


padding in the averaging calculation. Default: "True"
   Examples:
      >>> # pool of square window of size=3, stride=2
      >>> input = torch.tensor([[[1, 2, 3, 4, 5, 6, 7]]], dtype=torch.float32)
      >>> F.avg_pool1d(input, kernel_size=3, stride=2)
      tensor([[[ 2.,  4.,  6.]]])


torch.Tensor.bitwise_not_Tensor.bitwise_not_() -> Tensor
   In-place version of "bitwise_not()"


torch.Tensor.sparse_dimTensor.sparse_dim() -> int
   Return the number of sparse dimensions in a sparse tensor "self".
   Note:
     Returns "0" if "self" is not a sparse tensor.
   See also "Tensor.dense_dim()" and hybrid tensors.


torch.Tensor.hypotTensor.hypot(other) -> Tensor
   See "torch.hypot()"


torch.scattertorch.scatter(input, dim, index, src) -> Tensor
   Out-of-place version of "torch.Tensor.scatter_()"


torch.swapdimstorch.swapdims(input, dim0, dim1) -> Tensor
   Alias for "torch.transpose()".
   This function is equivalent to NumPy's swapaxes function.
   Examples:
      >>> x = torch.tensor([[[0,1],[2,3]],[[4,5],[6,7]]])
      >>> x
      tensor([[[0, 1],
              [2, 3]],
              [[4, 5],
              [6, 7]]])
      >>> torch.swapdims(x, 0, 1)
      tensor([[[0, 1],
              [4, 5]],
              [[2, 3],
              [6, 7]]])
      >>> torch.swapdims(x, 0, 2)
      tensor([[[0, 4],
              [2, 6]],
              [[1, 5],
              [3, 7]]])


torch.Tensor.true_divide_Tensor.true_divide_(value) -> Tensor
   In-place version of "true_divide_()"


torch.Tensor.fmaxTensor.fmax(other) -> Tensor
   See "torch.fmax()"


torch.is_storagetorch.is_storage(obj)
   Returns True if obj is a PyTorch storage object.
   Parameters:
      obj (Object) -- Object to test


Generatorclass torch.Generator(device='cpu')
   Creates and returns a generator object that manages the state of
   the algorithm which produces pseudo random numbers. Used as a
   keyword argument in many In-place random sampling functions.
   Parameters:
      device ("torch.device", optional) -- the desired device for
      the generator.
   Returns:
      An torch.Generator object.
   Return type:
      Generator
   Example:
      >>> g_cpu = torch.Generator()
      >>> g_cuda = torch.Generator(device='cuda')
   device
      Generator.device -> device
      Gets the current device of the generator.
      Example:
         >>> g_cpu = torch.Generator()
         >>> g_cpu.device
         device(type='cpu')
   get_state() -> Tensor
      Returns the Generator state as a "torch.ByteTensor".
      Returns:
         A "torch.ByteTensor" which contains all the necessary bits to
         restore a Generator to a specific point in time.
      Return type:
         Tensor
      Example:


Return type:
         Tensor
      Example:
         >>> g_cpu = torch.Generator()
         >>> g_cpu.get_state()
   initial_seed() -> int
      Returns the initial seed for generating random numbers.
      Example:
         >>> g_cpu = torch.Generator()
         >>> g_cpu.initial_seed()
         2147483647
   manual_seed(seed) -> Generator
      Sets the seed for generating random numbers. Returns a
      torch.Generator object. It is recommended to set a large seed,
      i.e. a number that has a good balance of 0 and 1 bits. Avoid
      having many 0 bits in the seed.
      Parameters:
         seed (int) -- The desired seed. Value must be within
         the inclusive range [-0x8000_0000_0000_0000,
         0xffff_ffff_ffff_ffff]. Otherwise, a RuntimeError is raised.
         Negative inputs are remapped to positive values with the
         formula 0xffff_ffff_ffff_ffff + seed.
      Returns:
         An torch.Generator object.
      Return type:
         Generator
      Example:


Generator
      Example:
         >>> g_cpu = torch.Generator()
         >>> g_cpu.manual_seed(2147483647)
   seed() -> int
      Gets a non-deterministic random number from std::random_device
      or the current time and uses it to seed a Generator.
      Example:
         >>> g_cpu = torch.Generator()
         >>> g_cpu.seed()
         1516516984916
   set_state(new_state) -> void
      Sets the Generator state.
      Parameters:
         new_state (torch.ByteTensor) -- The desired state.
      Example:
         >>> g_cpu = torch.Generator()
         >>> g_cpu_other = torch.Generator()
         >>> g_cpu.set_state(g_cpu_other.get_state())


torch.Tensor.ge_Tensor.ge_(other) -> Tensor
   In-place version of "ge()".


torch.Tensor.pin_memoryTensor.pin_memory() -> Tensor
   Copies the tensor to pinned memory, if it's not already pinned.


torch.Tensor.gtTensor.gt(other) -> Tensor
   See "torch.gt()".


torch.cummaxtorch.cummax(input, dim, , out=None)
   Returns a namedtuple "(values, indices)" where "values" is the
   cumulative maximum of elements of "input" in the dimension "dim".
   And "indices" is the index location of each maximum value found in
   the dimension "dim".
      y_i = max(x_1, x_2, x_3, \dots, x_i)
   Parameters:
      * input (Tensor) -- the input tensor.
      * dim (int) -- the dimension to do the operation over
   Keyword Arguments:
      out (tuple, optional*) -- the result tuple of two
      output tensors (values, indices)
   Example:
      >>> a = torch.randn(10)
      >>> a
      tensor([-0.3449, -1.5447,  0.0685, -1.5104, -1.1706,  0.2259,  1.4696, -1.3284,
           1.9946, -0.8209])
      >>> torch.cummax(a, dim=0)
      torch.return_types.cummax(
          values=tensor([-0.3449, -0.3449,  0.0685,  0.0685,  0.0685,  0.2259,  1.4696,  1.4696,
           1.9946,  1.9946]),
          indices=tensor([0, 0, 2, 2, 2, 5, 6, 6, 8, 8]))


torch.nn.functional.upsample_nearesttorch.nn.functional.upsample_nearest(input, size=None, scale_factor=None)
   Upsamples the input, using nearest neighbours' pixel values.
   Warning:
     This function is deprecated in favor of
     "torch.nn.functional.interpolate()". This is equivalent with
     "nn.functional.interpolate(..., mode='nearest')".
   Currently spatial and volumetric upsampling are supported (i.e.
   expected inputs are 4 or 5 dimensional).
   Parameters:
      * input (Tensor) -- input
      * size (int or Tuple[int, int] or
        Tuple[int, int, int]) -- output spatia size.
      * scale_factor (int) -- multiplier for spatial size. Has
        to be an integer.
   Note:
     This operation may produce nondeterministic gradients when given
     tensors on a CUDA device. See Reproducibility for more
     information.


torch.Tensor.index_put_Tensor.index_put_(indices, values, accumulate=False) -> Tensor
   Puts values from the tensor "values" into the tensor "self" using
   the indices specified in "indices" (which is a tuple of Tensors).
   The expression "tensor.index_put_(indices, values)" is equivalent
   to "tensor[indices] = values". Returns "self".
   If "accumulate" is "True", the elements in "values" are added to
   "self". If accumulate is "False", the behavior is undefined if
   indices contain duplicate elements.
   Parameters:
      * indices (tuple of LongTensor) -- tensors used to index
        into self.
      * values (Tensor) -- tensor of same dtype as self.
      * accumulate (bool) -- whether to accumulate into self


FloatFunctionalclass torch.ao.nn.quantized.FloatFunctional
   State collector class for float operations.
   The instance of this class can be used instead of the "torch."
   prefix for some operations. See example usage below.
   Note:
     This class does not provide a "forward" hook. Instead, you must
     use one of the underlying functions (e.g. "add").
   Examples:
      >>> f_add = FloatFunctional()
      >>> a = torch.tensor(3.0)
      >>> b = torch.tensor(4.0)
      >>> f_add.add(a, b)  # Equivalent to torch.add(a, b)
   Valid operation names:
      * add
      * cat
      * mul
      * add_relu
      * add_scalar
      * mul_scalar


torch.Tensor.hypot_Tensor.hypot_(other) -> Tensor
   In-place version of "hypot()"


torch.Tensor.mmTensor.mm(mat2) -> Tensor
   See "torch.mm()"


ELUclass torch.nn.ELU(alpha=1.0, inplace=False)
   Applies the Exponential Linear Unit (ELU) function, element-wise,
   as described in the paper: Fast and Accurate Deep Network Learning
   by Exponential Linear Units (ELUs).
   ELU is defined as:
      \text{ELU}(x) = \begin{cases} x, & \text{ if } x > 0\ \alpha *
      (\exp(x) - 1), & \text{ if } x \leq 0 \end{cases}
   Parameters:
      * alpha (float) -- the \alpha value for the ELU
        formulation. Default: 1.0
      * inplace (bool) -- can optionally do the operation in-
        place. Default: "False"
   Shape:
      * Input: (), where * means any number of dimensions.
      * Output: (), same shape as the input.
   [image]
   Examples:
      >>> m = nn.ELU()
      >>> input = torch.randn(2)
      >>> output = m(input)


torch.Tensor.swapdimsTensor.swapdims(dim0, dim1) -> Tensor
   See "torch.swapdims()"


torch.Tensor.atanTensor.atan() -> Tensor
   See "torch.atan()"


torch.optim.Optimizer.stepOptimizer.step(closure)
   Performs a single optimization step (parameter update).
   Parameters:
      closure (Callable) -- A closure that reevaluates the model
      and returns the loss. Optional for most optimizers.
   Note:
     Unless otherwise specified, this function should not modify the
     ".grad" field of the parameters.


torch.Tensor.is_quantizedTensor.is_quantized
   Is "True" if the Tensor is quantized, "False" otherwise.


torch.Tensor.arcsinhTensor.arcsinh() -> Tensor
   See "torch.arcsinh()"


torch.baddbmmtorch.baddbmm(input, batch1, batch2, , beta=1, alpha=1, out=None) -> Tensor
   Performs a batch matrix-matrix product of matrices in "batch1" and
   "batch2". "input" is added to the final result.
   "batch1" and "batch2" must be 3-D tensors each containing the same
   number of matrices.
   If "batch1" is a (b \times n \times m) tensor, "batch2" is a (b
   \times m \times p) tensor, then "input" must be broadcastable with
   a (b \times n \times p) tensor and "out" will be a (b \times n
   \times p) tensor. Both "alpha" and "beta" mean the same as the
   scaling factors used in "torch.addbmm()".
      \text{out}_i = \beta\ \text{input}_i + \alpha\ (\text{batch1}_i
      \mathbin{@} \text{batch2}_i)
   If "beta" is 0, then "input" will be ignored, and nan and inf
   in it will not be propagated.
   For inputs of type FloatTensor or DoubleTensor*, arguments
   "beta" and "alpha" must be real numbers, otherwise they should be
   integers.


integers.
   This operator supports TensorFloat32.
   On certain ROCm devices, when using float16 inputs this module will
   use different precision for backward.
   Parameters:
      * input (Tensor) -- the tensor to be added
      * batch1 (Tensor) -- the first batch of matrices to be
        multiplied
      * batch2 (Tensor) -- the second batch of matrices to be
        multiplied
   Keyword Arguments:
      * beta (Number, optional) -- multiplier for "input"
        (\beta)
      * alpha (Number, optional) -- multiplier for
        \text{batch1} \mathbin{@} \text{batch2} (\alpha)
      * out (Tensor, optional) -- the output tensor.
   Example:
      >>> M = torch.randn(10, 3, 5)
      >>> batch1 = torch.randn(10, 3, 4)
      >>> batch2 = torch.randn(10, 4, 5)
      >>> torch.baddbmm(M, batch1, batch2).size()
      torch.Size([10, 3, 5])


HistogramObserverclass torch.quantization.observer.HistogramObserver(bins=2048, upsample_rate=128, dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None, factory_kwargs=None, eps=1.1920928955078125e-07)
   The module records the running histogram of tensor values along
   with min/max values. "calculate_qparams" will calculate scale and
   zero_point.
   Parameters:
      * bins (int) -- Number of bins to use for the histogram
      * upsample_rate (int) -- Factor by which the histograms
        are upsampled, this is used to interpolate histograms with
        varying ranges across observations
      * dtype (dtype) -- dtype argument to the quantize node
        needed to implement the reference model spec
      * qscheme -- Quantization scheme to be used
      * reduce_range -- Reduces the range of the quantized data
        type by 1 bit
      * eps (Tensor) -- Epsilon value for float32, Defaults to


torch.finfo(torch.float32).eps.
   The scale and zero point are computed as follows:
   1. Create the histogram of the incoming inputs.
         The histogram is computed continuously, and the ranges per
         bin change with every new tensor observed.
   2. Search the distribution in the histogram for optimal min/max
      values.
         The search for the min/max values ensures the minimization of
         the quantization error with respect to the floating point
         model.
   3. Compute the scale and zero point the same way as in the
         "MinMaxObserver"


torch.promote_typestorch.promote_types(type1, type2) -> dtype
   Returns the "torch.dtype" with the smallest size and scalar kind
   that is not smaller nor of lower kind than either type1 or
   type2. See type promotion documentation for more information on
   the type promotion logic.
   Parameters:
      * type1 ("torch.dtype") --
      * type2 ("torch.dtype") --
   Example:
      >>> torch.promote_types(torch.int32, torch.float32)
      torch.float32
      >>> torch.promote_types(torch.uint8, torch.long)
      torch.long


torch.Tensor.resolve_negTensor.resolve_neg() -> Tensor
   See "torch.resolve_neg()"


torch.nn.functional.threshold_torch.nn.functional.threshold_(input, threshold, value) -> Tensor
   In-place version of "threshold()".


torch.linalg.lstsqtorch.linalg.lstsq(A, B, rcond=None, , driver=None)
   Computes a solution to the least squares problem of a system of
   linear equations.
   Letting \mathbb{K} be \mathbb{R} or \mathbb{C}, the least squares
   problem for a linear system AX = B with A \in \mathbb{K}^{m
   \times n}, B \in \mathbb{K}^{m \times k} is defined as
      \min_{X \in \mathbb{K}^{n \times k}} |AX - B|_F
   where |-|_F denotes the Frobenius norm.
   Supports inputs of float, double, cfloat and cdouble dtypes. Also
   supports batches of matrices, and if the inputs are batches of
   matrices then the output has the same batch dimensions.
   "driver" chooses the backend function that will be used. For CPU
   inputs the valid values are 'gels', 'gelsy', 'gelsd,
   'gelss'*. To choose the best driver on CPU consider:
   * If "A" is well-conditioned (its condition number is not too
     large), or you do not mind some precision loss.


For a general matrix: 'gelsy' (QR with pivoting) (default)
If "A" is full-rank: 'gels' (QR)

If "A" is not well-conditioned.
'gelsd' (tridiagonal reduction and SVD)
But if you run into memory issues: 'gelss' (full SVD).
   For CUDA input, the only valid driver is 'gels', which assumes
   that "A" is full-rank.
   See also the full description of these drivers
   "rcond" is used to determine the effective rank of the matrices in
   "A" when "driver" is one of ('gelsy', 'gelsd', 'gelss'). In
   this case, if \sigma_i are the singular values of A in decreasing
   order, \sigma_i will be rounded down to zero if \sigma_i \leq
   \text{rcond} \cdot \sigma_1. If "rcond"= None (default), "rcond"
   is set to the machine precision of the dtype of "A" times max(m,
   n).
   This function returns the solution to the problem and some extra
   information in a named tuple of four tensors *(solution, residuals,


rank, singular_values). For inputs "A", "B" of shape (, m, n),
   (, m, k) respectively, it contains
   * solution: the least squares solution. It has shape (, n, k).
   * residuals: the squared residuals of the solutions, that is,
     |AX - B|_F^2. It has shape equal to the batch dimensions of
     "A". It is computed when m > n and every matrix in "A" is full-
     rank, otherwise, it is an empty tensor. If "A" is a batch of
     matrices and any matrix in the batch is not full rank, then an
     empty tensor is returned. This behavior may change in a future
     PyTorch release.
   * rank: tensor of ranks of the matrices in "A". It has shape
     equal to the batch dimensions of "A". It is computed when
     "driver" is one of ('gelsy', 'gelsd', 'gelss'), otherwise
     it is an empty tensor.
   * singular_values: tensor of singular values of the matrices in
     "A". It has shape (, min(m, n))*. It is computed when "driver"


is one of ('gelsd', 'gelss'), otherwise it is an empty
     tensor.
   Note:
     This function computes X = "A".pinverse() @ "B" in a faster
     and more numerically stable way than performing the computations
     separately.
   Warning:
     The default value of "rcond" may change in a future PyTorch
     release. It is therefore recommended to use a fixed value to
     avoid potential breaking changes.
   Parameters:
      * A (Tensor) -- lhs tensor of shape (, m, n) where ***
        is zero or more batch dimensions.
      * B (Tensor) -- rhs tensor of shape (, m, k) where ***
        is zero or more batch dimensions.
      * rcond (float, optional) -- used to determine the
        effective rank of "A". If "rcond"= None, "rcond" is set to
        the machine precision of the dtype of "A" times max(m, n).
        Default: None.
   Keyword Arguments:
      driver (str, optional) -- name of the LAPACK/MAGMA


method to be used. If None, 'gelsy' is used for CPU inputs
      and 'gels' for CUDA inputs. Default: None.
   Returns:
      A named tuple (solution, residuals, rank, singular_values).
   Examples:
      >>> A = torch.randn(1,3,3)
      >>> A
      tensor([[[-1.0838,  0.0225,  0.2275],
           [ 0.2438,  0.3844,  0.5499],
           [ 0.1175, -0.9102,  2.0870]]])
      >>> B = torch.randn(2,3,3)
      >>> B
      tensor([[[-0.6772,  0.7758,  0.5109],
           [-1.4382,  1.3769,  1.1818],
           [-0.3450,  0.0806,  0.3967]],
          [[-1.3994, -0.1521, -0.1473],
           [ 1.9194,  1.0458,  0.6705],
           [-1.1802, -0.9796,  1.4086]]])
      >>> X = torch.linalg.lstsq(A, B).solution # A is broadcasted to shape (2, 3, 3)
      >>> torch.dist(X, torch.linalg.pinv(A) @ B)
      tensor(1.5152e-06)
      >>> S = torch.linalg.lstsq(A, B, driver='gelsd').singular_values
      >>> torch.dist(S, torch.linalg.svdvals(A))
      tensor(2.3842e-07)


tensor(2.3842e-07)
      >>> A[:, 0].zero_()  # Decrease the rank of A
      >>> rank = torch.linalg.lstsq(A, B).rank
      >>> rank
      tensor([2])


LinearLRclass torch.optim.lr_scheduler.LinearLR(optimizer, start_factor=0.3333333333333333, end_factor=1.0, total_iters=5, last_epoch=- 1, verbose=False)
   Decays the learning rate of each parameter group by linearly
   changing small multiplicative factor until the number of epoch
   reaches a pre-defined milestone: total_iters. Notice that such
   decay can happen simultaneously with other changes to the learning
   rate from outside this scheduler. When last_epoch=-1, sets initial
   lr as lr.
   Parameters:
      * optimizer (Optimizer) -- Wrapped optimizer.
      * start_factor (float) -- The number we multiply learning
        rate in the first epoch. The multiplication factor changes
        towards end_factor in the following epochs. Default: 1./3.
      * end_factor (float) -- The number we multiply learning
        rate at the end of linear changing process. Default: 1.0.
      * total_iters (int) -- The number of iterations that


multiplicative factor reaches to 1. Default: 5.
      * last_epoch (int) -- The index of the last epoch.
        Default: -1.
      * verbose (bool) -- If "True", prints a message to stdout
        for each update. Default: "False".
   -[ Example ]-

Assuming optimizer uses lr = 0.05 for all groups
lr = 0.025    if epoch == 0
lr = 0.03125  if epoch == 1
lr = 0.0375   if epoch == 2
lr = 0.04375  if epoch == 3
lr = 0.05    if epoch >= 4
scheduler = LinearLR(self.opt, start_factor=0.5, total_iters=4)
for epoch in range(100):
    train(...)
    validate(...)
    scheduler.step()
   get_last_lr()
      Return last computed learning rate by current scheduler.
   load_state_dict(state_dict)
      Loads the schedulers state.
      Parameters:
         state_dict (dict) -- scheduler state. Should be an
         object returned from a call to "state_dict()".
   print_lr(is_verbose, group, lr, epoch=None)


print_lr(is_verbose, group, lr, epoch=None)
      Display the current learning rate.
   state_dict()
      Returns the state of the scheduler as a "dict".
      It contains an entry for every variable in self.dict which
      is not the optimizer.


torch.sigmoidtorch.sigmoid(input, *, out=None) -> Tensor
   Alias for "torch.special.expit()".


LazyBatchNorm2dclass torch.nn.LazyBatchNorm2d(eps=1e-05, momentum=0.1, affine=True, track_running_stats=True, device=None, dtype=None)
   A "torch.nn.BatchNorm2d" module with lazy initialization of the
   "num_features" argument of the "BatchNorm2d" that is inferred from
   the "input.size(1)". The attributes that will be lazily initialized
   are weight, bias, running_mean and running_var.
   Check the "torch.nn.modules.lazy.LazyModuleMixin" for further
   documentation on lazy modules and their limitations.
   Parameters:
      * eps (float) -- a value added to the denominator for
        numerical stability. Default: 1e-5
      * momentum (float) -- the value used for the running_mean
        and running_var computation. Can be set to "None" for
        cumulative moving average (i.e. simple average). Default: 0.1
      * affine (bool) -- a boolean value that when set to
        "True", this module has learnable affine parameters. Default:
        "True"


"True"
      * track_running_stats (bool) -- a boolean value that when
        set to "True", this module tracks the running mean and
        variance, and when set to "False", this module does not track
        such statistics, and initializes statistics buffers
        "running_mean" and "running_var" as "None". When these buffers
        are "None", this module always uses batch statistics. in both
        training and eval modes. Default: "True"
   cls_to_become
      alias of "BatchNorm2d"


torch.copysigntorch.copysign(input, other, , out=None) -> Tensor
   Create a new floating-point tensor with the magnitude of "input"
   and the sign of "other", elementwise.
      \text{out}{i} = \begin{cases}     -|\text{input}| &
      \text{if } \text{other}{i} \leq -0.0 \      |\text{input}|
      & \text{if } \text{other}_{i} \geq 0.0 \ \end{cases}
   Supports broadcasting to a common shape, and integer and float
   inputs.
   Parameters:
      * input (Tensor) -- magnitudes.
      * other (Tensor or Number) -- contains value(s) whose
        signbit(s) are applied to the magnitudes in "input".
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Example:
      >>> a = torch.randn(5)
      >>> a
      tensor([-1.2557, -0.0026, -0.5387,  0.4740, -0.9244])
      >>> torch.copysign(a, 1)
      tensor([1.2557, 0.0026, 0.5387, 0.4740, 0.9244])
      >>> a = torch.randn(4, 4)
      >>> a


a = torch.randn(4, 4)
      >>> a
      tensor([[ 0.7079,  0.2778, -1.0249,  0.5719],
              [-0.0059, -0.2600, -0.4475, -1.3948],
              [ 0.3667, -0.9567, -2.5757, -0.1751],
              [ 0.2046, -0.0742,  0.2998, -0.1054]])
      >>> b = torch.randn(4)
      tensor([ 0.2373,  0.3120,  0.3190, -1.1128])
      >>> torch.copysign(a, b)
      tensor([[ 0.7079,  0.2778,  1.0249, -0.5719],
              [ 0.0059,  0.2600,  0.4475, -1.3948],
              [ 0.3667,  0.9567,  2.5757, -0.1751],
              [ 0.2046,  0.0742,  0.2998, -0.1054]])
      >>> a = torch.tensor([1.])
      >>> b = torch.tensor([-0.])
      >>> torch.copysign(a, b)
      tensor([-1.])
   Note:
     copysign handles signed zeros. If the other argument has a
     negative zero (-0), the corresponding output value will be
     negative.


torch.Tensor.histcTensor.histc(bins=100, min=0, max=0) -> Tensor
   See "torch.histc()"


torch.pca_lowranktorch.pca_lowrank(A, q=None, center=True, niter=2)
   Performs linear Principal Component Analysis (PCA) on a low-rank
   matrix, batches of such matrices, or sparse matrix.
   This function returns a namedtuple "(U, S, V)" which is the nearly
   optimal approximation of a singular value decomposition of a
   centered matrix A such that A = U diag(S) V^T.
   Note:
     The relation of "(U, S, V)" to PCA is as follows:
     * A is a data matrix with "m" samples and "n" features
     * the V columns represent the principal directions
     * S ** 2 / (m - 1) contains the eigenvalues of A^T A / (m - 1)
       which is the covariance of "A" when "center=True" is provided.
     * "matmul(A, V[:, :k])" projects data to the first k principal
       components
   Note:
     Different from the standard SVD, the size of returned matrices
     depend on the specified rank and q values as follows:
        * U is m x q matrix
        * S is q-vector
        * V is n x q matrix
   Note:


V is n x q matrix
   Note:
     To obtain repeatable results, reset the seed for the pseudorandom
     number generator
   Parameters:
A (Tensor) -- the input tensor of size (*, m, n)
q (int, optional) -- a slightly overestimated rank
    of A. By default, "q = min(6, m, n)".
center (bool, optional) -- if True, center the input
    tensor, otherwise, assume that the input is centered.
niter (int, optional) -- the number of subspace
    iterations to conduct; niter must be a nonnegative integer,
    and defaults to 2.
   Return type:
  Tuple[Tensor, Tensor, Tensor]
   References:
Nathan Halko, Per-Gunnar Martinsson, and Joel Tropp, Finding
    structure with randomness: probabilistic algorithms for
    constructing approximate matrix decompositions,
    arXiv:0909.4061 [math.NA; math.PR], 2009 (available at
    arXiv <http://arxiv.org/abs/0909.4061>_).


torch.uniquetorch.unique(input, sorted=True, return_inverse=False, return_counts=False, dim=None) -> Tuple[Tensor, Tensor, Tensor]
   Returns the unique elements of the input tensor.
   Note:
     This function is different from "torch.unique_consecutive()" in
     the sense that this function also eliminates non-consecutive
     duplicate values.
   Note:
     Currently in the CUDA implementation and the CPU implementation
     when dim is specified, torch.unique always sort the tensor at
     the beginning regardless of the sort argument. Sorting could be
     slow, so if your input tensor is already sorted, it is
     recommended to use "torch.unique_consecutive()" which avoids the
     sorting.
   Parameters:
      * input (Tensor) -- the input tensor
      * sorted (bool) -- Whether to sort the unique elements in
        ascending order before returning as output.
      * return_inverse (bool) -- Whether to also return the


indices for where elements in the original input ended up in
        the returned unique list.
      * return_counts (bool) -- Whether to also return the
        counts for each unique element.
      * dim (int) -- the dimension to apply unique. If "None",
        the unique of the flattened input is returned. default: "None"
   Returns:
      A tensor or a tuple of tensors containing
         * output (Tensor): the output list of unique scalar
           elements.
         * inverse_indices (Tensor): (optional) if
           "return_inverse" is True, there will be an additional
           returned tensor (same shape as input) representing the
           indices for where elements in the original input map to in
           the output; otherwise, this function will only return a
           single tensor.
         * counts (Tensor): (optional) if "return_counts" is
           True, there will be an additional returned tensor (same


shape as output or output.size(dim), if dim was specified)
           representing the number of occurrences for each unique
           value or tensor.
   Return type:
      (Tensor, Tensor (optional), Tensor (optional))
   Example:
      >>> output = torch.unique(torch.tensor([1, 3, 2, 3], dtype=torch.long))
      >>> output
      tensor([1, 2, 3])
      >>> output, inverse_indices = torch.unique(
      ...     torch.tensor([1, 3, 2, 3], dtype=torch.long), sorted=True, return_inverse=True)
      >>> output
      tensor([1, 2, 3])
      >>> inverse_indices
      tensor([0, 2, 1, 2])
      >>> output, inverse_indices = torch.unique(
      ...     torch.tensor([[1, 3], [2, 3]], dtype=torch.long), sorted=True, return_inverse=True)
      >>> output
      tensor([1, 2, 3])
      >>> inverse_indices
      tensor([[0, 2],
              [1, 2]])


AdaptiveLogSoftmaxWithLossclass torch.nn.AdaptiveLogSoftmaxWithLoss(in_features, n_classes, cutoffs, div_value=4.0, head_bias=False, device=None, dtype=None)
   Efficient softmax approximation as described in Efficient softmax
   approximation for GPUs by Edouard Grave, Armand Joulin, Moustapha
   CissÃ©, David Grangier, and HervÃ© JÃ©gou.
   Adaptive softmax is an approximate strategy for training models
   with large output spaces. It is most effective when the label
   distribution is highly imbalanced, for example in natural language
   modelling, where the word frequency distribution approximately
   follows the Zipf's law.
   Adaptive softmax partitions the labels into several clusters,
   according to their frequency. These clusters may contain different
   number of targets each. Additionally, clusters containing less
   frequent labels assign lower dimensional embeddings to those
   labels, which speeds up the computation. For each minibatch, only


clusters for which at least one target is present are evaluated.
   The idea is that the clusters which are accessed frequently (like
   the first one, containing most frequent labels), should also be
   cheap to compute -- that is, contain a small number of assigned
   labels.
   We highly recommend taking a look at the original paper for more
   details.
   * "cutoffs" should be an ordered Sequence of integers sorted in the
     increasing order. It controls number of clusters and the
     partitioning of targets into clusters. For example setting
     "cutoffs = [10, 100, 1000]" means that first 10 targets will be
     assigned to the 'head' of the adaptive softmax, targets 11, 12,
     ..., 100 will be assigned to the first cluster, and targets
     101, 102, ..., 1000 will be assigned to the second cluster,
     while targets 1001, 1002, ..., n_classes - 1 will be assigned
     to the last, third cluster.
   * "div_value" is used to compute the size of each additional


cluster, which is given as \left\lfloor\frac{\texttt{in_feature
     s}}{\texttt{div_value}^{idx}}\right\rfloor, where idx is the
     cluster index (with clusters for less frequent words having
     larger indices, and indices starting from 1).
   * "head_bias" if set to True, adds a bias term to the 'head' of the
     adaptive softmax. See paper for details. Set to False in the
     official implementation.
   Warning:
     Labels passed as inputs to this module should be sorted according
     to their frequency. This means that the most frequent label
     should be represented by the index 0, and the least frequent
     label should be represented by the index n_classes - 1.
   Note:
     This module returns a "NamedTuple" with "output" and "loss"
     fields. See further documentation for details.
   Note:
     To compute log-probabilities for all classes, the "log_prob"
     method can be used.
   Parameters:
      * in_features (int) -- Number of features in the input
        tensor


tensor
      * n_classes (int) -- Number of classes in the dataset
      * cutoffs (Sequence) -- Cutoffs used to assign targets to
        their buckets
      * div_value (float, optional) -- value used as an
        exponent to compute sizes of the clusters. Default: 4.0
      * head_bias (bool, optional) -- If "True", adds a bias
        term to the 'head' of the adaptive softmax. Default: "False"
   Returns:
      * output is a Tensor of size "N" containing computed target
        log probabilities for each example
      * loss is a Scalar representing the computed negative log
        likelihood loss
   Return type:
      "NamedTuple" with "output" and "loss" fields
   Shape:
      * input: (N, \texttt{in_features}) or (\texttt{in_features})
      * target: (N) or () where each value satisfies 0 <=
        \texttt{target[i]} <= \texttt{n_classes}
      * output1: (N) or ()
      * output2: "Scalar"
   log_prob(input)


output2: "Scalar"
   log_prob(input)
      Computes log probabilities for all \texttt{n_classes}
      Parameters:
         input (Tensor) -- a minibatch of examples
      Returns:
         log-probabilities of for each class c in range 0 <= c <=
         \texttt{n_classes}, where \texttt{n_classes} is a parameter
         passed to "AdaptiveLogSoftmaxWithLoss" constructor.
      Return type:
         Tensor
      Shape:
         * Input: (N, \texttt{in_features})
         * Output: (N, \texttt{n_classes})
   predict(input)
      This is equivalent to self.log_prob(input).argmax(dim=1), but
      is more efficient in some cases.
      Parameters:
         input (Tensor) -- a minibatch of examples
      Returns:
         a class with the highest probability for each example
      Return type:
         output (Tensor)
      Shape:
         * Input: (N, \texttt{in_features})
         * Output: (N)


torch.atleast_1dtorch.atleast_1d(tensors)
   Returns a 1-dimensional view of each input tensor with zero
   dimensions. Input tensors with one or more dimensions are returned
   as-is.
   Parameters:
      input (Tensor or list of Tensors*) --
   Returns:
      output (Tensor or tuple of Tensors)
   Example:
      >>> x = torch.arange(2)
      >>> x
      tensor([0, 1])
      >>> torch.atleast_1d(x)
      tensor([0, 1])
      >>> x = torch.tensor(1.)
      >>> x
      tensor(1.)
      >>> torch.atleast_1d(x)
      tensor([1.])
      >>> x = torch.tensor(0.5)
      >>> y = torch.tensor(1.)
      >>> torch.atleast_1d((x, y))
      (tensor([0.5000]), tensor([1.]))


torch.set_flush_denormaltorch.set_flush_denormal(mode) -> bool
   Disables denormal floating numbers on CPU.
   Returns "True" if your system supports flushing denormal numbers
   and it successfully configures flush denormal mode.
   "set_flush_denormal()" is only supported on x86 architectures
   supporting SSE3.
   Parameters:
      mode (bool) -- Controls whether to enable flush denormal
      mode or not
   Example:
      >>> torch.set_flush_denormal(True)
      True
      >>> torch.tensor([1e-323], dtype=torch.float64)
      tensor([ 0.], dtype=torch.float64)
      >>> torch.set_flush_denormal(False)
      True
      >>> torch.tensor([1e-323], dtype=torch.float64)
      tensor(9.88131e-324 *
             [ 1.0000], dtype=torch.float64)


SiLUclass torch.nn.SiLU(inplace=False)
   Applies the Sigmoid Linear Unit (SiLU) function, element-wise. The
   SiLU function is also known as the swish function.
      \text{silu}(x) = x * \sigma(x), \text{where } \sigma(x) \text{
      is the logistic sigmoid.}
   Note:
     See Gaussian Error Linear Units (GELUs) where the SiLU (Sigmoid
     Linear Unit) was originally coined, and see Sigmoid-Weighted
     Linear Units for Neural Network Function Approximation in
     Reinforcement Learning and Swish: a Self-Gated Activation
     Function where the SiLU was experimented with later.
   Shape:
      * Input: (), where * means any number of dimensions.
      * Output: (), same shape as the input.
   [image]
   Examples:
      >>> m = nn.SiLU()
      >>> input = torch.randn(2)
      >>> output = m(input)


torch.Tensor.flattenTensor.flatten(start_dim=0, end_dim=- 1) -> Tensor
   See "torch.flatten()"


ELUclass torch.ao.nn.quantized.ELU(scale, zero_point, alpha=1.0)
   This is the quantized equivalent of "ELU".
   Parameters:
      * scale -- quantization scale of the output tensor
      * zero_point -- quantization zero point of the output tensor
      * alpha (float) -- the alpha constant


torch.nn.functional.torch.nn.parallel.data_paralleltorch.nn.parallel.data_parallel(module, inputs, device_ids=None, output_device=None, dim=0, module_kwargs=None)
   Evaluates module(input) in parallel across the GPUs given in
   device_ids.
   This is the functional version of the DataParallel module.
   Parameters:
      * module (Module) -- the module to evaluate in parallel
      * inputs (Tensor) -- inputs to the module
      * device_ids (list of python:int or torch.device) --
        GPU ids on which to replicate module
      * output_device (list of python:int or torch.device)
        -- GPU location of the output  Use -1 to indicate the CPU.
        (default: device_ids[0])
   Returns:
      a Tensor containing the result of module(input) located on
      output_device


torch.cuda.inittorch.cuda.init()
   Initialize PyTorch's CUDA state.  You may need to call this
   explicitly if you are interacting with PyTorch via its C API, as
   Python bindings for CUDA functionality will not be available until
   this initialization takes place.  Ordinary users should not need
   this, as all of PyTorch's CUDA methods automatically initialize
   CUDA state on-demand.
   Does nothing if the CUDA state is already initialized.


torch.Tensor.realTensor.real
   Returns a new tensor containing real values of the "self" tensor
   for a complex-valued input tensor. The returned tensor and "self"
   share the same underlying storage.
   Returns "self" if "self" is a real-valued tensor tensor.
   Example::
      >>> x=torch.randn(4, dtype=torch.cfloat)
      >>> x
      tensor([(0.3100+0.3553j), (-0.5445-0.7896j), (-1.6492-0.0633j), (-0.0638-0.8119j)])
      >>> x.real
      tensor([ 0.3100, -0.5445, -1.6492, -0.0638])


torch.signal.windows.bartletttorch.signal.windows.bartlett(M, , sym=True, dtype=None, layout=torch.strided, device=None, requires_grad=False)
   Computes the Bartlett window.
   The Bartlett window is defined as follows:
      w_n = 1 - \left| \frac{2n}{M - 1} - 1 \right| = \begin{cases}
      \frac{2n}{M - 1} & \text{if } 0 \leq n \leq \frac{M - 1}{2} \
      2 - \frac{2n}{M - 1} & \text{if } \frac{M - 1}{2} < n < M \
      \end{cases}
   The window is normalized to 1 (maximum value is 1). However, the 1
   doesn't appear if "M" is even and "sym" is True.
   Parameters:
      M (int) -- the length of the window. In other words, the
      number of points of the returned window.
   Keyword Arguments:
      * sym (bool, optional) -- If False, returns a
        periodic window suitable for use in spectral analysis. If
        True, returns a symmetric window suitable for use in filter
        design. Default: True*.


design. Default: True.
      * dtype ("torch.dtype", optional) -- the desired data type
        of returned tensor. Default: if "None", uses a global default
        (see "torch.set_default_tensor_type()").
      * layout ("torch.layout", optional) -- the desired layout of
        returned Tensor. Default: "torch.strided".
      * device ("torch.device", optional) -- the desired device of
        returned tensor. Default: if "None", uses the current device
        for the default tensor type (see
        "torch.set_default_tensor_type()"). "device" will be the CPU
        for CPU tensor types and the current CUDA device for CUDA
        tensor types.
      * requires_grad (bool, optional) -- If autograd should
        record operations on the returned tensor. Default: "False".
   Return type:
      Tensor
   Examples:
      >>> # Generates a symmetric Bartlett window.
      >>> torch.signal.windows.bartlett(10)


torch.signal.windows.bartlett(10)
      tensor([0.0000, 0.2222, 0.4444, 0.6667, 0.8889, 0.8889, 0.6667, 0.4444, 0.2222, 0.0000])
      >>> # Generates a periodic Bartlett window.
      >>> torch.signal.windows.bartlett(10, sym=False)
      tensor([0.0000, 0.2000, 0.4000, 0.6000, 0.8000, 1.0000, 0.8000, 0.6000, 0.4000, 0.2000])


torch.poissontorch.poisson(input, generator=None) -> Tensor
   Returns a tensor of the same size as "input" with each element
   sampled from a Poisson distribution with rate parameter given by
   the corresponding element in "input" i.e.,
      \text{out}_i \sim \text{Poisson}(\text{input}_i)
   "input" must be non-negative.
   Parameters:
      input (Tensor) -- the input tensor containing the rates of
      the Poisson distribution
   Keyword Arguments:
      generator ("torch.Generator", optional) -- a pseudorandom
      number generator for sampling
   Example:
      >>> rates = torch.rand(4, 4) * 5  # rate parameter between 0 and 5
      >>> torch.poisson(rates)
      tensor([[9., 1., 3., 5.],
              [8., 6., 6., 0.],
              [0., 4., 5., 3.],
              [2., 1., 4., 2.]])


torch.asintorch.asin(input, , out=None) -> Tensor
   Returns a new tensor with the arcsine of the elements of "input".
      \text{out}{i} = \sin^{-1}(\text{input})
   Parameters:
      input (Tensor) -- the input tensor.
   Keyword Arguments:
      out (Tensor, optional*) -- the output tensor.
   Example:
      >>> a = torch.randn(4)
      >>> a
      tensor([-0.5962,  1.4985, -0.4396,  1.4525])
      >>> torch.asin(a)
      tensor([-0.6387,     nan, -0.4552,     nan])


torch.Tensor.arcsin_Tensor.arcsin_() -> Tensor
   In-place version of "arcsin()"


torch.Tensor.geqrfTensor.geqrf()
   See "torch.geqrf()"


torch.Tensor.whereTensor.where(condition, y) -> Tensor
   "self.where(condition, y)" is equivalent to "torch.where(condition,
   self, y)". See "torch.where()"


torch.sym_mintorch.sym_min(a, b)
   SymInt-aware utility for max().


torch.index_addtorch.index_add(input, dim, index, source, *, alpha=1, out=None) -> Tensor
   See "index_add_()" for function description.


HuberLossclass torch.nn.HuberLoss(reduction='mean', delta=1.0)
   Creates a criterion that uses a squared term if the absolute
   element-wise error falls below delta and a delta-scaled L1 term
   otherwise. This loss combines advantages of both "L1Loss" and
   "MSELoss"; the delta-scaled L1 region makes the loss less sensitive
   to outliers than "MSELoss", while the L2 region provides smoothness
   over "L1Loss" near 0. See Huber loss for more information.
   For a batch of size N, the unreduced loss can be described as:
      \ell(x, y) = L = {l_1, ..., l_N}^T
   with
      l_n = \begin{cases} 0.5 (x_n - y_n)^2, & \text{if } |x_n - y_n|
      < delta \ delta * (|x_n - y_n| - 0.5 * delta), &
      \text{otherwise } \end{cases}
   If reduction is not none, then:
      \ell(x, y) = \begin{cases}     \operatorname{mean}(L), &
      \text{if reduction} = \text{mean';}\\
      \operatorname{sum}(L),  &  \text{if reduction} = \text{sum'.}
      \end{cases}
   Note:


\end{cases}
   Note:
     When delta is set to 1, this loss is equivalent to
     "SmoothL1Loss". In general, this loss differs from "SmoothL1Loss"
     by a factor of delta (AKA beta in Smooth L1). See "SmoothL1Loss"
     for additional discussion on the differences in behavior between
     the two losses.
   Parameters:
      * reduction (str, optional) -- Specifies the reduction
        to apply to the output: "'none'" | "'mean'" | "'sum'".
        "'none'": no reduction will be applied, "'mean'": the sum of
        the output will be divided by the number of elements in the
        output, "'sum'": the output will be summed. Default: "'mean'"
      * delta (float, optional) -- Specifies the threshold
        at which to change between delta-scaled L1 and L2 loss. The
        value must be positive.  Default: 1.0
   Shape:
      * Input: () where * means any number of dimensions.
      * Target: (), same shape as the input.


Target: (*), same shape as the input.
Output: scalar. If "reduction" is "'none'", then (*), same
    shape as the input.


Moduleclass torch.nn.Module
   Base class for all neural network modules.
   Your models should also subclass this class.
   Modules can also contain other Modules, allowing to nest them in a
   tree structure. You can assign the submodules as regular
   attributes:
      import torch.nn as nn
      import torch.nn.functional as F
      class Model(nn.Module):
          def init(self):
              super().init()
              self.conv1 = nn.Conv2d(1, 20, 5)
              self.conv2 = nn.Conv2d(20, 20, 5)
          def forward(self, x):
              x = F.relu(self.conv1(x))
              return F.relu(self.conv2(x))
   Submodules assigned in this way will be registered, and will have
   their parameters converted too when you call "to()", etc.
   Note:
     As per the example above, an "init()" call to the parent
     class must be made before assignment on the child.
   Variables:
      training (bool) -- Boolean represents whether this module


is in training or evaluation mode.
   add_module(name, module)
      Adds a child module to the current module.
      The module can be accessed as an attribute using the given name.
      Parameters:
         * name (str) -- name of the child module. The child
           module can be accessed from this module using the given
           name
         * module (Module) -- child module to be added to the
           module.
   apply(fn)
      Applies "fn" recursively to every submodule (as returned by
      ".children()") as well as self. Typical use includes
      initializing the parameters of a model (see also torch.nn.init).
      Parameters:
         fn ("Module" -> None) -- function to be applied to each
         submodule
      Returns:
         self
      Return type:
         Module
      Example:
         >>> @torch.no_grad()
         >>> def init_weights(m):
         >>>     print(m)
         >>>     if type(m) == nn.Linear:
         >>>         m.weight.fill_(1.0)


    m.weight.fill_(1.0)
     >>>         print(m.weight)
     >>> net = nn.Sequential(nn.Linear(2, 2), nn.Linear(2, 2))
     >>> net.apply(init_weights)
     Linear(in_features=2, out_features=2, bias=True)
     Parameter containing:
     tensor([[1., 1.],
             [1., 1.]], requires_grad=True)
     Linear(in_features=2, out_features=2, bias=True)
     Parameter containing:
     tensor([[1., 1.],
             [1., 1.]], requires_grad=True)
     Sequential(
       (0): Linear(in_features=2, out_features=2, bias=True)
       (1): Linear(in_features=2, out_features=2, bias=True)
     )

bfloat16()
      Casts all floating point parameters and buffers to "bfloat16"
      datatype.
      Note:
        This method modifies the module in-place.
      Returns:
         self
      Return type:
         Module
   buffers(recurse=True)
      Returns an iterator over module buffers.
      Parameters:


Parameters:
         recurse (bool) -- if True, then yields buffers of this
         module and all submodules. Otherwise, yields only buffers
         that are direct members of this module.
      Yields:
         torch.Tensor -- module buffer
      Return type:
         Iterator[Tensor]
      Example:
         >>> for buf in model.buffers():
         >>>     print(type(buf), buf.size())
          (20L,)
          (20L, 1L, 5L, 5L)
   children()
      Returns an iterator over immediate children modules.
      Yields:
         Module -- a child module
      Return type:
         Iterator[Module]
   cpu()
      Moves all model parameters and buffers to the CPU.
      Note:
        This method modifies the module in-place.
      Returns:
         self
      Return type:
         Module
   cuda(device=None)
      Moves all model parameters and buffers to the GPU.
      This also makes associated parameters and buffers different


objects. So it should be called before constructing optimizer if
      the module will live on GPU while being optimized.
      Note:
        This method modifies the module in-place.
      Parameters:
         device (int, optional) -- if specified, all
         parameters will be copied to that device
      Returns:
         self
      Return type:
         Module
   double()
      Casts all floating point parameters and buffers to "double"
      datatype.
      Note:
        This method modifies the module in-place.
      Returns:
         self
      Return type:
         Module
   eval()
      Sets the module in evaluation mode.
      This has any effect only on certain modules. See documentations
      of particular modules for details of their behaviors in
      training/evaluation mode, if they are affected, e.g. "Dropout",
      "BatchNorm", etc.
      This is equivalent with "self.train(False)".
      See Locally disabling gradient computation for a comparison


between .eval() and several similar mechanisms that may be
      confused with it.
      Returns:
         self
      Return type:
         Module
   extra_repr()
      Set the extra representation of the module
      To print customized extra information, you should re-implement
      this method in your own modules. Both single-line and multi-line
      strings are acceptable.
      Return type:
         str
   float()
      Casts all floating point parameters and buffers to "float"
      datatype.
      Note:
        This method modifies the module in-place.
      Returns:
         self
      Return type:
         Module
   forward(*input)
      Defines the computation performed at every call.
      Should be overridden by all subclasses.
      Note:
        Although the recipe for forward pass needs to be defined
        within this function, one should call the "Module" instance
        afterwards instead of this since the former takes care of


running the registered hooks while the latter silently ignores
        them.
   get_buffer(target)
      Returns the buffer given by "target" if it exists, otherwise
      throws an error.
      See the docstring for "get_submodule" for a more detailed
      explanation of this method's functionality as well as how to
      correctly specify "target".
      Parameters:
         target (str) -- The fully-qualified string name of the
         buffer to look for. (See "get_submodule" for how to specify a
         fully-qualified string.)
      Returns:
         The buffer referenced by "target"
      Return type:
         torch.Tensor
      Raises:
         AttributeError -- If the target string references an
         invalid     path or resolves to something that is not a
         buffer
   get_extra_state()
      Returns any extra state to include in the module's state_dict.
      Implement this and a corresponding "set_extra_state()" for your


module if you need to store extra state. This function is called
      when building the module's state_dict().
      Note that extra state should be picklable to ensure working
      serialization of the state_dict. We only provide provide
      backwards compatibility guarantees for serializing Tensors;
      other objects may break backwards compatibility if their
      serialized pickled form changes.
      Returns:
         Any extra state to store in the module's state_dict
      Return type:
         object
   get_parameter(target)
      Returns the parameter given by "target" if it exists, otherwise
      throws an error.
      See the docstring for "get_submodule" for a more detailed
      explanation of this method's functionality as well as how to
      correctly specify "target".
      Parameters:
         target (str) -- The fully-qualified string name of the
         Parameter to look for. (See "get_submodule" for how to
         specify a fully-qualified string.)
      Returns:


Returns:
         The Parameter referenced by "target"
      Return type:
         torch.nn.Parameter
      Raises:
         AttributeError -- If the target string references an
         invalid     path or resolves to something that is not an
         "nn.Parameter"
   get_submodule(target)
      Returns the submodule given by "target" if it exists, otherwise
      throws an error.
      For example, let's say you have an "nn.Module" "A" that looks
      like this:
         A(
             (net_b): Module(
                 (net_c): Module(
                     (conv): Conv2d(16, 33, kernel_size=(3, 3), stride=(2, 2))
                 )
                 (linear): Linear(in_features=100, out_features=200, bias=True)
             )
         )
      (The diagram shows an "nn.Module" "A". "A" has a nested
      submodule "net_b", which itself has two submodules "net_c" and
      "linear". "net_c" then has a submodule "conv".)
      To check whether or not we have the "linear" submodule, we would


call "get_submodule("net_b.linear")". To check whether we have
      the "conv" submodule, we would call
      "get_submodule("net_b.net_c.conv")".
      The runtime of "get_submodule" is bounded by the degree of
      module nesting in "target". A query against "named_modules"
      achieves the same result, but it is O(N) in the number of
      transitive modules. So, for a simple check to see if some
      submodule exists, "get_submodule" should always be used.
      Parameters:
         target (str) -- The fully-qualified string name of the
         submodule to look for. (See above example for how to specify
         a fully-qualified string.)
      Returns:
         The submodule referenced by "target"
      Return type:
         torch.nn.Module
      Raises:
         AttributeError -- If the target string references an
         invalid     path or resolves to something that is not an
         "nn.Module"
   half()
      Casts all floating point parameters and buffers to "half"


datatype.
      Note:
        This method modifies the module in-place.
      Returns:
         self
      Return type:
         Module
   ipu(device=None)
      Moves all model parameters and buffers to the IPU.
      This also makes associated parameters and buffers different
      objects. So it should be called before constructing optimizer if
      the module will live on IPU while being optimized.
      Note:
        This method modifies the module in-place.
      Parameters:
         device (int, optional) -- if specified, all
         parameters will be copied to that device
      Returns:
         self
      Return type:
         Module
   load_state_dict(state_dict, strict=True)
      Copies parameters and buffers from "state_dict" into this module
      and its descendants. If "strict" is "True", then the keys of
      "state_dict" must exactly match the keys returned by this
      module's "state_dict()" function.
      Parameters: