torch.solve(input, A, *, out=None) -> (Tensor, Tensor)
This function returns the solution to the system of linear equations represented by AX=BAX = B  and the LU factorization of A, in order as a namedtuple solution, LU. LU contains L and U factors for LU factorization of A. torch.solve(B, A) can take in 2D inputs B, A or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs solution, LU. Supports real-valued and complex-valued inputs.  Note Irrespective of the original strides, the returned matrices solution and LU will be transposed, i.e. with strides like B.contiguous().transpose(-1, -2).stride() and A.contiguous().transpose(-1, -2).stride() respectively.   Parameters

input (Tensor) – input matrix BB  of size (∗,m,k)(*, m, k)  , where ∗*  is zero or more batch dimensions.
A (Tensor) – input square matrix of size (∗,m,m)(*, m, m) , where ∗*  is zero or more batch dimensions.   Keyword Arguments
out ((Tensor, Tensor), optional) – optional output tuple.   Example: >>> A = torch.tensor([[6.80, -2.11,  5.66,  5.97,  8.23],
...                   [-6.05, -3.30,  5.36, -4.44,  1.08],
...                   [-0.45,  2.58, -2.70,  0.27,  9.04],
...                   [8.32,  2.71,  4.35,  -7.17,  2.14],
...                   [-9.67, -5.14, -7.26,  6.08, -6.87]]).t()
>>> B = torch.tensor([[4.02,  6.19, -8.22, -7.57, -3.03],
...                   [-1.56,  4.00, -8.67,  1.75,  2.86],
...                   [9.81, -4.09, -4.57, -8.61,  8.99]]).t()
>>> X, LU = torch.solve(B, A)
>>> torch.dist(B, torch.mm(A, X))
tensor(1.00000e-06 *
       7.0977)

>>> # Batched solver example
>>> A = torch.randn(2, 3, 1, 4, 4)
>>> B = torch.randn(2, 3, 1, 4, 6)
>>> X, LU = torch.solve(B, A)
>>> torch.dist(B, A.matmul(X))
tensor(1.00000e-06 *
   3.6386)


torch.sort(input, dim=-1, descending=False, *, out=None) -> (Tensor, LongTensor)
Sorts the elements of the input tensor along a given dimension in ascending order by value. If dim is not given, the last dimension of the input is chosen. If descending is True then the elements are sorted in descending order by value. A namedtuple of (values, indices) is returned, where the values are the sorted values and indices are the indices of the elements in the original input tensor.  Parameters

input (Tensor) – the input tensor.
dim (int, optional) – the dimension to sort along
descending (bool, optional) – controls the sorting order (ascending or descending)   Keyword Arguments
out (tuple, optional) – the output tuple of (Tensor, LongTensor) that can be optionally given to be used as output buffers   Example: >>> x = torch.randn(3, 4)
>>> sorted, indices = torch.sort(x)
>>> sorted
tensor([[-0.2162,  0.0608,  0.6719,  2.3332],
        [-0.5793,  0.0061,  0.6058,  0.9497],
        [-0.5071,  0.3343,  0.9553,  1.0960]])
>>> indices
tensor([[ 1,  0,  2,  3],
        [ 3,  1,  0,  2],
        [ 0,  3,  1,  2]])

>>> sorted, indices = torch.sort(x, 0)
>>> sorted
tensor([[-0.5071, -0.2162,  0.6719, -0.5793],
        [ 0.0608,  0.0061,  0.9497,  0.3343],
        [ 0.6058,  0.9553,  1.0960,  2.3332]])
>>> indices
tensor([[ 2,  0,  0,  1],
        [ 0,  1,  1,  2],
        [ 1,  2,  2,  0]])


torch.sparse Introduction PyTorch provides torch.Tensor to represent a multi-dimensional array containing elements of a single data type. By default, array elements are stored contiguously in memory leading to efficient implementations of various array processing algorithms that relay on the fast access to array elements. However, there exists an important class of multi-dimensional arrays, so-called sparse arrays, where the contiguous memory storage of array elements turns out to be suboptimal. Sparse arrays have a property of having a vast portion of elements being equal to zero which means that a lot of memory as well as processor resources can be spared if only the non-zero elements are stored or/and processed. Various sparse storage formats (such as COO, CSR/CSC, LIL, etc.) have been developed that are optimized for a particular structure of non-zero elements in sparse arrays as well as for specific operations on the arrays.  Note When talking about storing only non-zero elements of a sparse array, the usage of adjective “non-zero” is not strict: one is allowed to store also zeros in the sparse array data structure. Hence, in the following, we use “specified elements” for those array elements that are actually stored. In addition, the unspecified elements are typically assumed to have zero value, but not only, hence we use the term “fill value” to denote such elements.   Note Using a sparse storage format for storing sparse arrays can be advantageous only when the size and sparsity levels of arrays are high. Otherwise, for small-sized or low-sparsity arrays using the contiguous memory storage format is likely the most efficient approach.   Warning The PyTorch API of sparse tensors is in beta and may change in the near future.  Sparse COO tensors Currently, PyTorch implements the so-called Coordinate format, or COO format, as the default sparse storage format for storing sparse tensors. In COO format, the specified elements are stored as tuples of element indices and the corresponding values. In particular,  the indices of specified elements are collected in indices tensor of size (ndim, nse) and with element type torch.int64, the corresponding values are collected in values tensor of size (nse,) and with an arbitrary integer or floating point number element type,  where ndim is the dimensionality of the tensor and nse is the number of specified elements.  Note The memory consumption of a sparse COO tensor is at least (ndim *
8 + <size of element type in bytes>) * nse bytes (plus a constant overhead from storing other tensor data). The memory consumption of a strided tensor is at least product(<tensor shape>) * <size of element type in bytes>. For example, the memory consumption of a 10 000 x 10 000 tensor with 100 000 non-zero 32-bit floating point numbers is at least (2 * 8 + 4) * 100 000 = 2 000 000 bytes when using COO tensor layout and 10 000 * 10 000 * 4 = 400 000 000 bytes when using the default strided tensor layout. Notice the 200 fold memory saving from using the COO storage format.  Construction A sparse COO tensor can be constructed by providing the two tensors of indices and values, as well as the size of the sparse tensor (when it cannot be inferred from the indices and values tensors) to a function torch.sparse_coo_tensor(). Suppose we want to define a sparse tensor with the entry 3 at location (0, 2), entry 4 at location (1, 0), and entry 5 at location (1, 2). Unspecified elements are assumed to have the same value, fill value, which is zero by default. We would then write: >>> i = [[0, 1, 1],
         [2, 0, 2]]
>>> v =  [3, 4, 5]
>>> s = torch.sparse_coo_tensor(i, v, (2, 3))
>>> s
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3, 4, 5]),
       size=(2, 3), nnz=3, layout=torch.sparse_coo)
>>> s.to_dense()
tensor([[0, 0, 3],
        [4, 0, 5]])
 Note that the input i is NOT a list of index tuples. If you want to write your indices this way, you should transpose before passing them to the sparse constructor: >>> i = [[0, 2], [1, 0], [1, 2]]
>>> v =  [3,      4,      5    ]
>>> s = torch.sparse_coo_tensor(list(zip(*i)), v, (2, 3))
>>> # Or another equivalent formulation to get s
>>> s = torch.sparse_coo_tensor(torch.tensor(i).t(), v, (2, 3))
>>> torch.sparse_coo_tensor(i.t(), v, torch.Size([2,3])).to_dense()
tensor([[0, 0, 3],
        [4, 0, 5]])
 An empty sparse COO tensor can be constructed by specifying its size only: >>> torch.sparse_coo_tensor(size=(2, 3))
tensor(indices=tensor([], size=(2, 0)),
       values=tensor([], size=(0,)),
       size=(2, 3), nnz=0, layout=torch.sparse_coo)
 Hybrid sparse COO tensors Pytorch implements an extension of sparse tensors with scalar values to sparse tensors with (contiguous) tensor values. Such tensors are called hybrid tensors. PyTorch hybrid COO tensor extends the sparse COO tensor by allowing the values tensor to be a multi-dimensional tensor so that we have:  the indices of specified elements are collected in indices tensor of size (sparse_dims, nse) and with element type torch.int64, the corresponding (tensor) values are collected in values tensor of size (nse, dense_dims) and with an arbitrary integer or floating point number element type.   Note We use (M + K)-dimensional tensor to denote a N-dimensional hybrid sparse tensor, where M and K are the numbers of sparse and dense dimensions, respectively, such that M + K == N holds.  Suppose we want to create a (2 + 1)-dimensional tensor with the entry [3, 4] at location (0, 2), entry [5, 6] at location (1, 0), and entry [7, 8] at location (1, 2). We would write >>> i = [[0, 1, 1],
         [2, 0, 2]]
>>> v =  [[3, 4], [5, 6], [7, 8]]
>>> s = torch.sparse_coo_tensor(i, v, (2, 3, 2))
>>> s
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([[3, 4],
                      [5, 6],
                      [7, 8]]),
       size=(2, 3, 2), nnz=3, layout=torch.sparse_coo)
 >>> s.to_dense()
tensor([[[0, 0],
         [0, 0],
         [3, 4]],
        [[5, 6],
         [0, 0],
         [7, 8]]])
 In general, if s is a sparse COO tensor and M =
s.sparse_dim(), K = s.dense_dim(), then we have the following invariants:
M + K == len(s.shape) == s.ndim - dimensionality of a tensor is the sum of the number of sparse and dense dimensions,
s.indices().shape == (M, nse) - sparse indices are stored explicitly,
s.values().shape == (nse,) + s.shape[M : M + K] - the values of a hybrid tensor are K-dimensional tensors,
s.values().layout == torch.strided - values are stored as strided tensors.   Note Dense dimensions always follow sparse dimensions, that is, mixing of dense and sparse dimensions is not supported.  Uncoalesced sparse COO tensors PyTorch sparse COO tensor format permits uncoalesced sparse tensors, where there may be duplicate coordinates in the indices; in this case, the interpretation is that the value at that index is the sum of all duplicate value entries. For example, one can specify multiple values, 3 and 4, for the same index 1, that leads to an 1-D uncoalesced tensor: >>> i = [[1, 1]]
>>> v =  [3, 4]
>>> s=torch.sparse_coo_tensor(i, v, (3,))
>>> s
tensor(indices=tensor([[1, 1]]),
       values=tensor(  [3, 4]),
       size=(3,), nnz=2, layout=torch.sparse_coo)
 while the coalescing process will accumulate the multi-valued elements into a single value using summation: >>> s.coalesce()
tensor(indices=tensor([[1]]),
       values=tensor([7]),
       size=(3,), nnz=1, layout=torch.sparse_coo)
 In general, the output of torch.Tensor.coalesce() method is a sparse tensor with the following properties:  the indices of specified tensor elements are unique, the indices are sorted in lexicographical order,
torch.Tensor.is_coalesced() returns True.   Note For the most part, you shouldn’t have to care whether or not a sparse tensor is coalesced or not, as most operations will work identically given a coalesced or uncoalesced sparse tensor. However, some operations can be implemented more efficiently on uncoalesced tensors, and some on coalesced tensors. For instance, addition of sparse COO tensors is implemented by simply concatenating the indices and values tensors: >>> a = torch.sparse_coo_tensor([[1, 1]], [5, 6], (2,))
>>> b = torch.sparse_coo_tensor([[0, 0]], [7, 8], (2,))
>>> a + b
tensor(indices=tensor([[0, 0, 1, 1]]),
       values=tensor([7, 8, 5, 6]),
       size=(2,), nnz=4, layout=torch.sparse_coo)
 If you repeatedly perform an operation that can produce duplicate entries (e.g., torch.Tensor.add()), you should occasionally coalesce your sparse tensors to prevent them from growing too large. On the other hand, the lexicographical ordering of indices can be advantageous for implementing algorithms that involve many element selection operations, such as slicing or matrix products.  Working with sparse COO tensors Let’s consider the following example: >>> i = [[0, 1, 1],
         [2, 0, 2]]
>>> v =  [[3, 4], [5, 6], [7, 8]]
>>> s = torch.sparse_coo_tensor(i, v, (2, 3, 2))
 As mentioned above, a sparse COO tensor is a torch.Tensor instance and to distinguish it from the Tensor instances that use some other layout, on can use torch.Tensor.is_sparse or torch.Tensor.layout properties: >>> isinstance(s, torch.Tensor)
True
>>> s.is_sparse
True
>>> s.layout == torch.sparse_coo
True
 The number of sparse and dense dimensions can be acquired using methods torch.Tensor.sparse_dim() and torch.Tensor.dense_dim(), respectively. For instance: >>> s.sparse_dim(), s.dense_dim()
(2, 1)
 If s is a sparse COO tensor then its COO format data can be acquired using methods torch.Tensor.indices() and torch.Tensor.values().  Note Currently, one can acquire the COO format data only when the tensor instance is coalesced: >>> s.indices()
RuntimeError: Cannot get indices on an uncoalesced tensor, please call .coalesce() first
 For acquiring the COO format data of an uncoalesced tensor, use torch.Tensor._values() and torch.Tensor._indices(): >>> s._indices()
tensor([[0, 1, 1],
        [2, 0, 2]])
  Constructing a new sparse COO tensor results a tensor that is not coalesced: >>> s.is_coalesced()
False
 but one can construct a coalesced copy of a sparse COO tensor using the torch.Tensor.coalesce() method: >>> s2 = s.coalesce()
>>> s2.indices()
tensor([[0, 1, 1],
       [2, 0, 2]])
 When working with uncoalesced sparse COO tensors, one must take into an account the additive nature of uncoalesced data: the values of the same indices are the terms of a sum that evaluation gives the value of the corresponding tensor element. For example, the scalar multiplication on an uncoalesced sparse tensor could be implemented by multiplying all the uncoalesced values with the scalar because c *
(a + b) == c * a + c * b holds. However, any nonlinear operation, say, a square root, cannot be implemented by applying the operation to uncoalesced data because sqrt(a + b) == sqrt(a) + sqrt(b) does not hold in general. Slicing (with positive step) of a sparse COO tensor is supported only for dense dimensions. Indexing is supported for both sparse and dense dimensions: >>> s[1]
tensor(indices=tensor([[0, 2]]),
       values=tensor([[5, 6],
                      [7, 8]]),
       size=(3, 2), nnz=2, layout=torch.sparse_coo)
>>> s[1, 0, 1]
tensor(6)
>>> s[1, 0, 1:]
tensor([6])
 In PyTorch, the fill value of a sparse tensor cannot be specified explicitly and is assumed to be zero in general. However, there exists operations that may interpret the fill value differently. For instance, torch.sparse.softmax() computes the softmax with the assumption that the fill value is negative infinity. Supported Linear Algebra operations The following table summarizes supported Linear Algebra operations on sparse matrices where the operands layouts may vary. Here T[layout] denotes a tensor with a given layout. Similarly, M[layout] denotes a matrix (2-D PyTorch tensor), and V[layout] denotes a vector (1-D PyTorch tensor). In addition, f denotes a scalar (float or 0-D PyTorch tensor), * is element-wise multiplication, and @ is matrix multiplication.
PyTorch operation Sparse grad? Layout signature
torch.mv() no M[sparse_coo] @ V[strided] -> V[strided]
torch.matmul() no M[sparse_coo] @ M[strided] -> M[strided]
torch.mm() no M[sparse_coo] @ M[strided] -> M[strided]
torch.sparse.mm() yes M[sparse_coo] @ M[strided] -> M[strided]
torch.smm() no M[sparse_coo] @ M[strided] -> M[sparse_coo]
torch.hspmm() no M[sparse_coo] @ M[strided] -> M[hybrid sparse_coo]
torch.bmm() no T[sparse_coo] @ T[strided] -> T[strided]
torch.addmm() no f * M[strided] + f * (M[sparse_coo] @ M[strided]) -> M[strided]
torch.sparse.addmm() yes f * M[strided] + f * (M[sparse_coo] @ M[strided]) -> M[strided]
torch.sspaddmm() no f * M[sparse_coo] + f * (M[sparse_coo] @ M[strided]) -> M[sparse_coo]
torch.lobpcg() no GENEIG(M[sparse_coo]) -> M[strided], M[strided]
torch.pca_lowrank() yes PCA(M[sparse_coo]) -> M[strided], M[strided], M[strided]
torch.svd_lowrank() yes SVD(M[sparse_coo]) -> M[strided], M[strided], M[strided]   where “Sparse grad?” column indicates if the PyTorch operation supports backward with respect to sparse matrix argument. All PyTorch operations, except torch.smm(), support backward with respect to strided matrix arguments.  Note Currently, PyTorch does not support matrix multiplication with the layout signature M[strided] @ M[sparse_coo]. However, applications can still compute this using the matrix relation D @
S == (S.t() @ D.t()).t().
class torch.Tensor
The following methods are specific to sparse tensors:
is_sparse
Is True if the Tensor uses sparse storage layout, False otherwise.

dense_dim() → int
Return the number of dense dimensions in a sparse tensor self.  Warning Throws an error if self is not a sparse tensor.  See also Tensor.sparse_dim() and hybrid tensors.

sparse_dim() → int
Return the number of sparse dimensions in a sparse tensor self.  Warning Throws an error if self is not a sparse tensor.  See also Tensor.dense_dim() and hybrid tensors.

sparse_mask(mask) → Tensor
Returns a new sparse tensor with values from a strided tensor self filtered by the indices of the sparse tensor mask. The values of mask sparse tensor are ignored. self and mask tensors must have the same shape.  Note The returned sparse tensor has the same indices as the sparse tensor mask, even when the corresponding values in self are zeros.   Parameters
mask (Tensor) – a sparse tensor whose indices are used as a filter   Example: >>> nse = 5
>>> dims = (5, 5, 2, 2)
>>> I = torch.cat([torch.randint(0, dims[0], size=(nse,)),
...                torch.randint(0, dims[1], size=(nse,))], 0).reshape(2, nse)
>>> V = torch.randn(nse, dims[2], dims[3])
>>> S = torch.sparse_coo_tensor(I, V, dims).coalesce()
>>> D = torch.randn(dims)
>>> D.sparse_mask(S)
tensor(indices=tensor([[0, 0, 0, 2],
                       [0, 1, 4, 3]]),
       values=tensor([[[ 1.6550,  0.2397],
                       [-0.1611, -0.0779]],

                      [[ 0.2326, -1.0558],
                       [ 1.4711,  1.9678]],

                      [[-0.5138, -0.0411],
                       [ 1.9417,  0.5158]],

                      [[ 0.0793,  0.0036],
                       [-0.2569, -0.1055]]]),
       size=(5, 5, 2, 2), nnz=4, layout=torch.sparse_coo)

sparse_resize_(size, sparse_dim, dense_dim) → Tensor
Resizes self sparse tensor to the desired size and the number of sparse and dense dimensions.  Note If the number of specified elements in self is zero, then size, sparse_dim, and dense_dim can be any size and positive integers such that len(size) == sparse_dim +
dense_dim. If self specifies one or more elements, however, then each dimension in size must not be smaller than the corresponding dimension of self, sparse_dim must equal the number of sparse dimensions in self, and dense_dim must equal the number of dense dimensions in self.   Warning Throws an error if self is not a sparse tensor.   Parameters

size (torch.Size) – the desired size. If self is non-empty sparse tensor, the desired size cannot be smaller than the original size.
sparse_dim (int) – the number of sparse dimensions
dense_dim (int) – the number of dense dimensions

sparse_resize_and_clear_(size, sparse_dim, dense_dim) → Tensor
Removes all specified elements from a sparse tensor self and resizes self to the desired size and the number of sparse and dense dimensions.  Parameters

size (torch.Size) – the desired size.
sparse_dim (int) – the number of sparse dimensions
dense_dim (int) – the number of dense dimensions

to_dense() → Tensor
Creates a strided copy of self.  Warning Throws an error if self is a strided tensor.  Example: >>> s = torch.sparse_coo_tensor(
...        torch.tensor([[1, 1],
...                      [0, 2]]),
...        torch.tensor([9, 10]),
...        size=(3, 3))
>>> s.to_dense()
tensor([[ 0,  0,  0],
        [ 9,  0, 10],
        [ 0,  0,  0]])

to_sparse(sparseDims) → Tensor
Returns a sparse copy of the tensor. PyTorch supports sparse tensors in coordinate format.  Parameters
sparseDims (int, optional) – the number of sparse dimensions to include in the new sparse tensor   Example: >>> d = torch.tensor([[0, 0, 0], [9, 0, 10], [0, 0, 0]])
>>> d
tensor([[ 0,  0,  0],
        [ 9,  0, 10],
        [ 0,  0,  0]])
>>> d.to_sparse()
tensor(indices=tensor([[1, 1],
                       [0, 2]]),
       values=tensor([ 9, 10]),
       size=(3, 3), nnz=2, layout=torch.sparse_coo)
>>> d.to_sparse(1)
tensor(indices=tensor([[1]]),
       values=tensor([[ 9,  0, 10]]),
       size=(3, 3), nnz=1, layout=torch.sparse_coo)

coalesce() → Tensor
Returns a coalesced copy of self if self is an uncoalesced tensor. Returns self if self is a coalesced tensor.  Warning Throws an error if self is not a sparse COO tensor.

is_coalesced() → bool
Returns True if self is a sparse COO tensor that is coalesced, False otherwise.  Warning Throws an error if self is not a sparse COO tensor.  See coalesce() and uncoalesced tensors.

indices() → Tensor
Return the indices tensor of a sparse COO tensor.  Warning Throws an error if self is not a sparse COO tensor.  See also Tensor.values().  Note This method can only be called on a coalesced sparse tensor. See Tensor.coalesce() for details.

values() → Tensor
Return the values tensor of a sparse COO tensor.  Warning Throws an error if self is not a sparse COO tensor.  See also Tensor.indices().  Note This method can only be called on a coalesced sparse tensor. See Tensor.coalesce() for details.

 The following torch.Tensor methods support sparse COO tensors: add() add_() addmm() addmm_() any() asin() asin_() arcsin() arcsin_() bmm() clone() deg2rad() deg2rad_() detach() detach_() dim() div() div_() floor_divide() floor_divide_() get_device() index_select() isnan() log1p() log1p_() mm() mul() mul_() mv() narrow_copy() neg() neg_() negative() negative_() numel() rad2deg() rad2deg_() resize_as_() size() pow() sqrt() square() smm() sspaddmm() sub() sub_() t() t_() transpose() transpose_() zero_() Sparse tensor functions
torch.sparse_coo_tensor(indices, values, size=None, *, dtype=None, device=None, requires_grad=False) → Tensor
Constructs a sparse tensor in COO(rdinate) format with specified values at the given indices.  Note This function returns an uncoalesced tensor.   Parameters

indices (array_like) – Initial data for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types. Will be cast to a torch.LongTensor internally. The indices are the coordinates of the non-zero values in the matrix, and thus should be two-dimensional where the first dimension is the number of tensor dimensions and the second dimension is the number of non-zero values.
values (array_like) – Initial values for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types.
size (list, tuple, or torch.Size, optional) – Size of the sparse tensor. If not provided the size will be inferred as the minimum size big enough to hold all non-zero elements.   Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, infers data type from values.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> i = torch.tensor([[0, 1, 1],
...                   [2, 0, 2]])
>>> v = torch.tensor([3, 4, 5], dtype=torch.float32)
>>> torch.sparse_coo_tensor(i, v, [2, 4])
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       size=(2, 4), nnz=3, layout=torch.sparse_coo)

>>> torch.sparse_coo_tensor(i, v)  # Shape inference
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       size=(2, 3), nnz=3, layout=torch.sparse_coo)

>>> torch.sparse_coo_tensor(i, v, [2, 4],
...                         dtype=torch.float64,
...                         device=torch.device('cuda:0'))
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       device='cuda:0', size=(2, 4), nnz=3, dtype=torch.float64,
       layout=torch.sparse_coo)

# Create an empty sparse tensor with the following invariants:
#   1. sparse_dim + dense_dim = len(SparseTensor.shape)
#   2. SparseTensor._indices().shape = (sparse_dim, nnz)
#   3. SparseTensor._values().shape = (nnz, SparseTensor.shape[sparse_dim:])
#
# For instance, to create an empty sparse tensor with nnz = 0, dense_dim = 0 and
# sparse_dim = 1 (hence indices is a 2D tensor of shape = (1, 0))
>>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), [], [1])
tensor(indices=tensor([], size=(1, 0)),
       values=tensor([], size=(0,)),
       size=(1,), nnz=0, layout=torch.sparse_coo)

# and to create an empty sparse tensor with nnz = 0, dense_dim = 1 and
# sparse_dim = 1
>>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), torch.empty([0, 2]), [1, 2])
tensor(indices=tensor([], size=(1, 0)),
       values=tensor([], size=(0, 2)),
       size=(1, 2), nnz=0, layout=torch.sparse_coo)

torch.sparse.sum(input, dim=None, dtype=None) [source]

Returns the sum of each row of the sparse tensor input in the given dimensions dim. If dim is a list of dimensions, reduce over all of them. When sum over all sparse_dim, this method returns a dense tensor instead of a sparse tensor. All summed dim are squeezed (see torch.squeeze()), resulting an output tensor having dim fewer dimensions than input. During backward, only gradients at nnz locations of input will propagate back. Note that the gradients of input is coalesced.  Parameters

input (Tensor) – the input sparse tensor
dim (int or tuple of python:ints) – a dimension or a list of dimensions to reduce. Default: reduce over all dims.
dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: dtype of input.    Example: >>> nnz = 3
>>> dims = [5, 5, 2, 3]
>>> I = torch.cat([torch.randint(0, dims[0], size=(nnz,)),
                   torch.randint(0, dims[1], size=(nnz,))], 0).reshape(2, nnz)
>>> V = torch.randn(nnz, dims[2], dims[3])
>>> size = torch.Size(dims)
>>> S = torch.sparse_coo_tensor(I, V, size)
>>> S
tensor(indices=tensor([[2, 0, 3],
                       [2, 4, 1]]),
       values=tensor([[[-0.6438, -1.6467,  1.4004],
                       [ 0.3411,  0.0918, -0.2312]],

                      [[ 0.5348,  0.0634, -2.0494],
                       [-0.7125, -1.0646,  2.1844]],

                      [[ 0.1276,  0.1874, -0.6334],
                       [-1.9682, -0.5340,  0.7483]]]),
       size=(5, 5, 2, 3), nnz=3, layout=torch.sparse_coo)

# when sum over only part of sparse_dims, return a sparse tensor
>>> torch.sparse.sum(S, [1, 3])
tensor(indices=tensor([[0, 2, 3]]),
       values=tensor([[-1.4512,  0.4073],
                      [-0.8901,  0.2017],
                      [-0.3183, -1.7539]]),
       size=(5, 2), nnz=3, layout=torch.sparse_coo)

# when sum over all sparse dim, return a dense tensor
# with summed dims squeezed
>>> torch.sparse.sum(S, [0, 1, 3])
tensor([-2.6596, -1.1450])

torch.sparse.addmm(mat, mat1, mat2, beta=1.0, alpha=1.0) [source]

This function does exact same thing as torch.addmm() in the forward, except that it supports backward for sparse matrix mat1. mat1 need to have sparse_dim = 2. Note that the gradients of mat1 is a coalesced sparse tensor.  Parameters

mat (Tensor) – a dense matrix to be added
mat1 (Tensor) – a sparse matrix to be multiplied
mat2 (Tensor) – a dense matrix to be multiplied
beta (Number, optional) – multiplier for mat (β\beta )
alpha (Number, optional) – multiplier for mat1@mat2mat1 @ mat2  (α\alpha )

torch.sparse.mm(mat1, mat2) [source]

Performs a matrix multiplication of the sparse matrix mat1 and the (sparse or strided) matrix mat2. Similar to torch.mm(), If mat1 is a (n×m)(n \times m)  tensor, mat2 is a (m×p)(m \times p)  tensor, out will be a (n×p)(n \times p)  tensor. mat1 need to have sparse_dim = 2. This function also supports backward for both matrices. Note that the gradients of mat1 is a coalesced sparse tensor.  Parameters

mat1 (SparseTensor) – the first sparse matrix to be multiplied
mat2 (Tensor) – the second matrix to be multiplied, which could be sparse or dense     Shape:

The format of the output tensor of this function follows: - sparse x sparse -> sparse - sparse x dense -> dense   Example: >>> a = torch.randn(2, 3).to_sparse().requires_grad_(True)
>>> a
tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                       [0, 1, 2, 0, 1, 2]]),
       values=tensor([ 1.5901,  0.0183, -0.6146,  1.8061, -0.0112,  0.6302]),
       size=(2, 3), nnz=6, layout=torch.sparse_coo, requires_grad=True)

>>> b = torch.randn(3, 2, requires_grad=True)
>>> b
tensor([[-0.6479,  0.7874],
        [-1.2056,  0.5641],
        [-1.1716, -0.9923]], requires_grad=True)

>>> y = torch.sparse.mm(a, b)
>>> y
tensor([[-0.3323,  1.8723],
        [-1.8951,  0.7904]], grad_fn=<SparseAddmmBackward>)
>>> y.sum().backward()
>>> a.grad
tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                       [0, 1, 2, 0, 1, 2]]),
       values=tensor([ 0.1394, -0.6415, -2.1639,  0.1394, -0.6415, -2.1639]),
       size=(2, 3), nnz=6, layout=torch.sparse_coo)

torch.sspaddmm(input, mat1, mat2, *, beta=1, alpha=1, out=None) → Tensor
Matrix multiplies a sparse tensor mat1 with a dense tensor mat2, then adds the sparse tensor input to the result. Note: This function is equivalent to torch.addmm(), except input and mat1 are sparse.  Parameters

input (Tensor) – a sparse matrix to be added
mat1 (Tensor) – a sparse matrix to be matrix multiplied
mat2 (Tensor) – a dense matrix to be matrix multiplied   Keyword Arguments

beta (Number, optional) – multiplier for mat (β\beta )
alpha (Number, optional) – multiplier for mat1@mat2mat1 @ mat2  (α\alpha )
out (Tensor, optional) – the output tensor.

torch.hspmm(mat1, mat2, *, out=None) → Tensor
Performs a matrix multiplication of a sparse COO matrix mat1 and a strided matrix mat2. The result is a (1 + 1)-dimensional hybrid COO matrix.  Parameters

mat1 (Tensor) – the first sparse matrix to be matrix multiplied
mat2 (Tensor) – the second strided matrix to be matrix multiplied   Keyword Arguments
{out} –

torch.smm(input, mat) → Tensor
Performs a matrix multiplication of the sparse matrix input with the dense matrix mat.  Parameters

input (Tensor) – a sparse matrix to be matrix multiplied
mat (Tensor) – a dense matrix to be matrix multiplied

torch.sparse.softmax(input, dim, dtype=None) [source]

Applies a softmax function. Softmax is defined as: Softmax(xi)=exp(xi)∑jexp(xj)\text{Softmax}(x_{i}) = \frac{exp(x_i)}{\sum_j exp(x_j)}  where i,ji, j  run over sparse tensor indices and unspecified entries are ignores. This is equivalent to defining unspecified entries as negative infinity so that exp(xk)=0exp(x_k) = 0  when the entry with index kk  has not specified. It is applied to all slices along dim, and will re-scale them so that the elements lie in the range [0, 1] and sum to 1.  Parameters

input (Tensor) – input
dim (int) – A dimension along which softmax will be computed.
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None

torch.sparse.log_softmax(input, dim, dtype=None) [source]

Applies a softmax function followed by logarithm. See softmax for more details.  Parameters

input (Tensor) – input
dim (int) – A dimension along which softmax will be computed.
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None
 Other functions The following torch functions support sparse COO tensors: cat() dstack() empty() empty_like() hstack() index_select() is_complex() is_floating_point() is_nonzero() is_same_size() is_signed() is_tensor() lobpcg() mm() native_norm() pca_lowrank() select() stack() svd_lowrank() unsqueeze() vstack() zeros() zeros_like()


torch.sparse.addmm(mat, mat1, mat2, beta=1.0, alpha=1.0) [source]

This function does exact same thing as torch.addmm() in the forward, except that it supports backward for sparse matrix mat1. mat1 need to have sparse_dim = 2. Note that the gradients of mat1 is a coalesced sparse tensor.  Parameters

mat (Tensor) – a dense matrix to be added
mat1 (Tensor) – a sparse matrix to be multiplied
mat2 (Tensor) – a dense matrix to be multiplied
beta (Number, optional) – multiplier for mat (β\beta )
alpha (Number, optional) – multiplier for mat1@mat2mat1 @ mat2  (α\alpha )


torch.sparse.log_softmax(input, dim, dtype=None) [source]

Applies a softmax function followed by logarithm. See softmax for more details.  Parameters

input (Tensor) – input
dim (int) – A dimension along which softmax will be computed.
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None


torch.sparse.mm(mat1, mat2) [source]

Performs a matrix multiplication of the sparse matrix mat1 and the (sparse or strided) matrix mat2. Similar to torch.mm(), If mat1 is a (n×m)(n \times m)  tensor, mat2 is a (m×p)(m \times p)  tensor, out will be a (n×p)(n \times p)  tensor. mat1 need to have sparse_dim = 2. This function also supports backward for both matrices. Note that the gradients of mat1 is a coalesced sparse tensor.  Parameters

mat1 (SparseTensor) – the first sparse matrix to be multiplied
mat2 (Tensor) – the second matrix to be multiplied, which could be sparse or dense     Shape:

The format of the output tensor of this function follows: - sparse x sparse -> sparse - sparse x dense -> dense   Example: >>> a = torch.randn(2, 3).to_sparse().requires_grad_(True)
>>> a
tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                       [0, 1, 2, 0, 1, 2]]),
       values=tensor([ 1.5901,  0.0183, -0.6146,  1.8061, -0.0112,  0.6302]),
       size=(2, 3), nnz=6, layout=torch.sparse_coo, requires_grad=True)

>>> b = torch.randn(3, 2, requires_grad=True)
>>> b
tensor([[-0.6479,  0.7874],
        [-1.2056,  0.5641],
        [-1.1716, -0.9923]], requires_grad=True)

>>> y = torch.sparse.mm(a, b)
>>> y
tensor([[-0.3323,  1.8723],
        [-1.8951,  0.7904]], grad_fn=<SparseAddmmBackward>)
>>> y.sum().backward()
>>> a.grad
tensor(indices=tensor([[0, 0, 0, 1, 1, 1],
                       [0, 1, 2, 0, 1, 2]]),
       values=tensor([ 0.1394, -0.6415, -2.1639,  0.1394, -0.6415, -2.1639]),
       size=(2, 3), nnz=6, layout=torch.sparse_coo)


torch.sparse.softmax(input, dim, dtype=None) [source]

Applies a softmax function. Softmax is defined as: Softmax(xi)=exp(xi)∑jexp(xj)\text{Softmax}(x_{i}) = \frac{exp(x_i)}{\sum_j exp(x_j)}  where i,ji, j  run over sparse tensor indices and unspecified entries are ignores. This is equivalent to defining unspecified entries as negative infinity so that exp(xk)=0exp(x_k) = 0  when the entry with index kk  has not specified. It is applied to all slices along dim, and will re-scale them so that the elements lie in the range [0, 1] and sum to 1.  Parameters

input (Tensor) – input
dim (int) – A dimension along which softmax will be computed.
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None


torch.sparse.sum(input, dim=None, dtype=None) [source]

Returns the sum of each row of the sparse tensor input in the given dimensions dim. If dim is a list of dimensions, reduce over all of them. When sum over all sparse_dim, this method returns a dense tensor instead of a sparse tensor. All summed dim are squeezed (see torch.squeeze()), resulting an output tensor having dim fewer dimensions than input. During backward, only gradients at nnz locations of input will propagate back. Note that the gradients of input is coalesced.  Parameters

input (Tensor) – the input sparse tensor
dim (int or tuple of python:ints) – a dimension or a list of dimensions to reduce. Default: reduce over all dims.
dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: dtype of input.    Example: >>> nnz = 3
>>> dims = [5, 5, 2, 3]
>>> I = torch.cat([torch.randint(0, dims[0], size=(nnz,)),
                   torch.randint(0, dims[1], size=(nnz,))], 0).reshape(2, nnz)
>>> V = torch.randn(nnz, dims[2], dims[3])
>>> size = torch.Size(dims)
>>> S = torch.sparse_coo_tensor(I, V, size)
>>> S
tensor(indices=tensor([[2, 0, 3],
                       [2, 4, 1]]),
       values=tensor([[[-0.6438, -1.6467,  1.4004],
                       [ 0.3411,  0.0918, -0.2312]],

                      [[ 0.5348,  0.0634, -2.0494],
                       [-0.7125, -1.0646,  2.1844]],

                      [[ 0.1276,  0.1874, -0.6334],
                       [-1.9682, -0.5340,  0.7483]]]),
       size=(5, 5, 2, 3), nnz=3, layout=torch.sparse_coo)

# when sum over only part of sparse_dims, return a sparse tensor
>>> torch.sparse.sum(S, [1, 3])
tensor(indices=tensor([[0, 2, 3]]),
       values=tensor([[-1.4512,  0.4073],
                      [-0.8901,  0.2017],
                      [-0.3183, -1.7539]]),
       size=(5, 2), nnz=3, layout=torch.sparse_coo)

# when sum over all sparse dim, return a dense tensor
# with summed dims squeezed
>>> torch.sparse.sum(S, [0, 1, 3])
tensor([-2.6596, -1.1450])


torch.sparse_coo_tensor(indices, values, size=None, *, dtype=None, device=None, requires_grad=False) → Tensor
Constructs a sparse tensor in COO(rdinate) format with specified values at the given indices.  Note This function returns an uncoalesced tensor.   Parameters

indices (array_like) – Initial data for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types. Will be cast to a torch.LongTensor internally. The indices are the coordinates of the non-zero values in the matrix, and thus should be two-dimensional where the first dimension is the number of tensor dimensions and the second dimension is the number of non-zero values.
values (array_like) – Initial values for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types.
size (list, tuple, or torch.Size, optional) – Size of the sparse tensor. If not provided the size will be inferred as the minimum size big enough to hold all non-zero elements.   Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, infers data type from values.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.    Example: >>> i = torch.tensor([[0, 1, 1],
...                   [2, 0, 2]])
>>> v = torch.tensor([3, 4, 5], dtype=torch.float32)
>>> torch.sparse_coo_tensor(i, v, [2, 4])
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       size=(2, 4), nnz=3, layout=torch.sparse_coo)

>>> torch.sparse_coo_tensor(i, v)  # Shape inference
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       size=(2, 3), nnz=3, layout=torch.sparse_coo)

>>> torch.sparse_coo_tensor(i, v, [2, 4],
...                         dtype=torch.float64,
...                         device=torch.device('cuda:0'))
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       device='cuda:0', size=(2, 4), nnz=3, dtype=torch.float64,
       layout=torch.sparse_coo)

# Create an empty sparse tensor with the following invariants:
#   1. sparse_dim + dense_dim = len(SparseTensor.shape)
#   2. SparseTensor._indices().shape = (sparse_dim, nnz)
#   3. SparseTensor._values().shape = (nnz, SparseTensor.shape[sparse_dim:])
#
# For instance, to create an empty sparse tensor with nnz = 0, dense_dim = 0 and
# sparse_dim = 1 (hence indices is a 2D tensor of shape = (1, 0))
>>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), [], [1])
tensor(indices=tensor([], size=(1, 0)),
       values=tensor([], size=(0,)),
       size=(1,), nnz=0, layout=torch.sparse_coo)

# and to create an empty sparse tensor with nnz = 0, dense_dim = 1 and
# sparse_dim = 1
>>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), torch.empty([0, 2]), [1, 2])
tensor(indices=tensor([], size=(1, 0)),
       values=tensor([], size=(0, 2)),
       size=(1, 2), nnz=0, layout=torch.sparse_coo)


torch.split(tensor, split_size_or_sections, dim=0) [source]

Splits the tensor into chunks. Each chunk is a view of the original tensor. If split_size_or_sections is an integer type, then tensor will be split into equally sized chunks (if possible). Last chunk will be smaller if the tensor size along the given dimension dim is not divisible by split_size. If split_size_or_sections is a list, then tensor will be split into len(split_size_or_sections) chunks with sizes in dim according to split_size_or_sections.  Parameters

tensor (Tensor) – tensor to split.
split_size_or_sections (int) or (list(int)) – size of a single chunk or list of sizes for each chunk
dim (int) – dimension along which to split the tensor.     Example::

>>> a = torch.arange(10).reshape(5,2)
>>> a
tensor([[0, 1],
        [2, 3],
        [4, 5],
        [6, 7],
        [8, 9]])
>>> torch.split(a, 2)
(tensor([[0, 1],
         [2, 3]]),
 tensor([[4, 5],
         [6, 7]]),
 tensor([[8, 9]]))
>>> torch.split(a, [1,4])
(tensor([[0, 1]]),
 tensor([[2, 3],
         [4, 5],
         [6, 7],
         [8, 9]]))


torch.sqrt(input, *, out=None) → Tensor
Returns a new tensor with the square-root of the elements of input.  outi=inputi\text{out}_{i} = \sqrt{\text{input}_{i}}
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([-2.0755,  1.0226,  0.0831,  0.4806])
>>> torch.sqrt(a)
tensor([    nan,  1.0112,  0.2883,  0.6933])


torch.square(input, *, out=None) → Tensor
Returns a new tensor with the square of the elements of input.  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([-2.0755,  1.0226,  0.0831,  0.4806])
>>> torch.square(a)
tensor([ 4.3077,  1.0457,  0.0069,  0.2310])


torch.squeeze(input, dim=None, *, out=None) → Tensor
Returns a tensor with all the dimensions of input of size 1 removed. For example, if input is of shape: (A×1×B×C×1×D)(A \times 1 \times B \times C \times 1 \times D)  then the out tensor will be of shape: (A×B×C×D)(A \times B \times C \times D) . When dim is given, a squeeze operation is done only in the given dimension. If input is of shape: (A×1×B)(A \times 1 \times B) , squeeze(input, 0) leaves the tensor unchanged, but squeeze(input, 1) will squeeze the tensor to the shape (A×B)(A \times B) .  Note The returned tensor shares the storage with the input tensor, so changing the contents of one will change the contents of the other.   Warning If the tensor has a batch dimension of size 1, then squeeze(input) will also remove the batch dimension, which can lead to unexpected errors.   Parameters

input (Tensor) – the input tensor.
dim (int, optional) – if given, the input will be squeezed only in this dimension   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> x = torch.zeros(2, 1, 2, 1, 2)
>>> x.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x)
>>> y.size()
torch.Size([2, 2, 2])
>>> y = torch.squeeze(x, 0)
>>> y.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x, 1)
>>> y.size()
torch.Size([2, 2, 1, 2])


torch.sspaddmm(input, mat1, mat2, *, beta=1, alpha=1, out=None) → Tensor
Matrix multiplies a sparse tensor mat1 with a dense tensor mat2, then adds the sparse tensor input to the result. Note: This function is equivalent to torch.addmm(), except input and mat1 are sparse.  Parameters

input (Tensor) – a sparse matrix to be added
mat1 (Tensor) – a sparse matrix to be matrix multiplied
mat2 (Tensor) – a dense matrix to be matrix multiplied   Keyword Arguments

beta (Number, optional) – multiplier for mat (β\beta )
alpha (Number, optional) – multiplier for mat1@mat2mat1 @ mat2  (α\alpha )
out (Tensor, optional) – the output tensor.


torch.stack(tensors, dim=0, *, out=None) → Tensor
Concatenates a sequence of tensors along a new dimension. All tensors need to be of the same size.  Parameters

tensors (sequence of Tensors) – sequence of tensors to concatenate
dim (int) – dimension to insert. Has to be between 0 and the number of dimensions of concatenated tensors (inclusive)   Keyword Arguments
out (Tensor, optional) – the output tensor.


torch.std(input, unbiased=True) → Tensor
Returns the standard-deviation of all elements in the input tensor. If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.  Parameters

input (Tensor) – the input tensor.
unbiased (bool) – whether to use the unbiased estimation or not    Example: >>> a = torch.randn(1, 3)
>>> a
tensor([[-0.8166, -1.3802, -0.3560]])
>>> torch.std(a)
tensor(0.5130)

torch.std(input, dim, unbiased=True, keepdim=False, *, out=None) → Tensor
 Returns the standard-deviation of each row of the input tensor in the dimension dim. If dim is a list of dimensions, reduce over all of them. If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s). If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.  Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
unbiased (bool) – whether to use the unbiased estimation or not
keepdim (bool) – whether the output tensor has dim retained or not.   Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.2035,  1.2959,  1.8101, -0.4644],
        [ 1.5027, -0.3270,  0.5905,  0.6538],
        [-1.5745,  1.3330, -0.5596, -0.6548],
        [ 0.1264, -0.5080,  1.6420,  0.1992]])
>>> torch.std(a, dim=1)
tensor([ 1.0311,  0.7477,  1.2204,  0.9087])


torch.std_mean(input, unbiased=True) -> (Tensor, Tensor)
Returns the standard-deviation and mean of all elements in the input tensor. If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.  Parameters

input (Tensor) – the input tensor.
unbiased (bool) – whether to use the unbiased estimation or not    Example: >>> a = torch.randn(1, 3)
>>> a
tensor([[0.3364, 0.3591, 0.9462]])
>>> torch.std_mean(a)
(tensor(0.3457), tensor(0.5472))

torch.std_mean(input, dim, unbiased=True, keepdim=False) -> (Tensor, Tensor)
 Returns the standard-deviation and mean of each row of the input tensor in the dimension dim. If dim is a list of dimensions, reduce over all of them. If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s). If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.  Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
unbiased (bool) – whether to use the unbiased estimation or not
keepdim (bool) – whether the output tensor has dim retained or not.    Example: >>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.5648, -0.5984, -1.2676, -1.4471],
        [ 0.9267,  1.0612,  1.1050, -0.6014],
        [ 0.0154,  1.9301,  0.0125, -1.0904],
        [-1.9711, -0.7748, -1.3840,  0.5067]])
>>> torch.std_mean(a, 1)
(tensor([0.9110, 0.8197, 1.2552, 1.0608]), tensor([-0.6871,  0.6229,  0.2169, -0.9058]))


torch.stft(input, n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=None, return_complex=None) [source]

Short-time Fourier transform (STFT).  Warning From version 1.8.0, return_complex must always be given explicitly for real inputs and return_complex=False has been deprecated. Strongly prefer return_complex=True as in a future pytorch release, this function will only return complex tensors. Note that torch.view_as_real() can be used to recover a real tensor with an extra last dimension for real and imaginary components.  The STFT computes the Fourier transform of short overlapping windows of the input. This giving frequency components of the signal as they change over time. The interface of this function is modeled after the librosa stft function. Ignoring the optional batch dimension, this method computes the following expression:  X[m,ω]=∑k=0win_length-1window[k] input[m×hop_length+k]exp(−j2π⋅ωkwin_length),X[m, \omega] = \sum_{k = 0}^{\text{win\_length-1}}% \text{window}[k]\ \text{input}[m \times \text{hop\_length} + k]\ % \exp\left(- j \frac{2 \pi \cdot \omega k}{\text{win\_length}}\right),
where mm  is the index of the sliding window, and ω\omega  is the frequency that 0≤ω<n_fft0 \leq \omega < \text{n\_fft} . When onesided is the default value True,
input must be either a 1-D time sequence or a 2-D batch of time sequences. If hop_length is None (default), it is treated as equal to floor(n_fft / 4). If win_length is None (default), it is treated as equal to n_fft.
window can be a 1-D tensor of size win_length, e.g., from torch.hann_window(). If window is None (default), it is treated as if having 11  everywhere in the window. If win_length<n_fft\text{win\_length} < \text{n\_fft} , window will be padded on both sides to length n_fft before being applied. If center is True (default), input will be padded on both sides so that the tt -th frame is centered at time t×hop_lengtht \times \text{hop\_length} . Otherwise, the tt -th frame begins at time t×hop_lengtht \times \text{hop\_length} .
pad_mode determines the padding method used on input when center is True. See torch.nn.functional.pad() for all available options. Default is "reflect". If onesided is True (default for real input), only values for ω\omega  in [0,1,2,…,⌊n_fft2⌋+1]\left[0, 1, 2, \dots, \left\lfloor \frac{\text{n\_fft}}{2} \right\rfloor + 1\right]  are returned because the real-to-complex Fourier transform satisfies the conjugate symmetry, i.e., X[m,ω]=X[m,n_fft−ω]∗X[m, \omega] = X[m, \text{n\_fft} - \omega]^* . Note if the input or window tensors are complex, then onesided output is not possible. If normalized is True (default is False), the function returns the normalized STFT results, i.e., multiplied by (frame_length)−0.5(\text{frame\_length})^{-0.5} . If return_complex is True (default if input is complex), the return is a input.dim() + 1 dimensional complex tensor. If False, the output is a input.dim() + 2 dimensional real tensor where the last dimension represents the real and imaginary components.  Returns either a complex tensor of size (∗×N×T)(* \times N \times T)  if return_complex is true, or a real tensor of size (∗×N×T×2)(* \times N \times T \times 2) . Where ∗*  is the optional batch size of input, NN  is the number of frequencies where STFT is applied and TT  is the total number of frames used.  Warning This function changed signature at version 0.4.1. Calling with the previous signature may cause error or return incorrect result.   Parameters

input (Tensor) – the input tensor
n_fft (int) – size of Fourier transform
hop_length (int, optional) – the distance between neighboring sliding window frames. Default: None (treated as equal to floor(n_fft / 4))
win_length (int, optional) – the size of window frame and STFT filter. Default: None (treated as equal to n_fft)
window (Tensor, optional) – the optional window function. Default: None (treated as window of all 11  s)
center (bool, optional) – whether to pad input on both sides so that the tt -th frame is centered at time t×hop_lengtht \times \text{hop\_length} . Default: True

pad_mode (string, optional) – controls the padding method used when center is True. Default: "reflect"

normalized (bool, optional) – controls whether to return the normalized STFT results Default: False

onesided (bool, optional) – controls whether to return half of results to avoid redundancy for real inputs. Default: True for real input and window, False otherwise.
return_complex (bool, optional) – whether to return a complex tensor, or a real tensor with an extra last dimension for the real and imaginary components.   Returns
A tensor containing the STFT result with shape described above  Return type
Tensor


torch.Storage A torch.Storage is a contiguous, one-dimensional array of a single data type. Every torch.Tensor has a corresponding storage of the same data type.
class torch.FloatStorage(*args, **kwargs) [source]

bfloat16()
Casts this storage to bfloat16 type

bool()
Casts this storage to bool type

byte()
Casts this storage to byte type

char()
Casts this storage to char type

clone()
Returns a copy of this storage

complex_double()
Casts this storage to complex double type

complex_float()
Casts this storage to complex float type

copy_()

cpu()
Returns a CPU copy of this storage if it’s not already on the CPU

cuda(device=None, non_blocking=False, **kwargs)
Returns a copy of this object in CUDA memory. If this object is already in CUDA memory and on the correct device, then no copy is performed and the original object is returned.  Parameters

device (int) – The destination GPU id. Defaults to the current device.
non_blocking (bool) – If True and the source is in pinned memory, the copy will be asynchronous with respect to the host. Otherwise, the argument has no effect.
**kwargs – For compatibility, may contain the key async in place of the non_blocking argument.

data_ptr()

device

double()
Casts this storage to double type

dtype

element_size()

fill_()

float()
Casts this storage to float type

static from_buffer()

static from_file(filename, shared=False, size=0) → Storage
If shared is True, then memory is shared between all processes. All changes are written to the file. If shared is False, then the changes on the storage do not affect the file. size is the number of elements in the storage. If shared is False, then the file must contain at least size * sizeof(Type) bytes (Type is the type of storage). If shared is True the file will be created if needed.  Parameters

filename (str) – file name to map
shared (bool) – whether to share memory
size (int) – number of elements in the storage

get_device()

half()
Casts this storage to half type

int()
Casts this storage to int type

is_cuda: bool = False

is_pinned()

is_shared()

is_sparse: bool = False

long()
Casts this storage to long type

new()

pin_memory()
Copies the storage to pinned memory, if it’s not already pinned.

resize_()

share_memory_()
Moves the storage to shared memory. This is a no-op for storages already in shared memory and for CUDA storages, which do not need to be moved for sharing across processes. Storages in shared memory cannot be resized. Returns: self

short()
Casts this storage to short type

size()

tolist()
Returns a list containing the elements of this storage

type(dtype=None, non_blocking=False, **kwargs)
Returns the type if dtype is not provided, else casts this object to the specified type. If this is already of the correct type, no copy is performed and the original object is returned.  Parameters

dtype (type or string) – The desired type
non_blocking (bool) – If True, and the source is in pinned memory and destination is on the GPU or vice versa, the copy is performed asynchronously with respect to the host. Otherwise, the argument has no effect.
**kwargs – For compatibility, may contain the key async in place of the non_blocking argument. The async arg is deprecated.


torch.sub(input, other, *, alpha=1, out=None) → Tensor
Subtracts other, scaled by alpha, from input.  outi=inputi−alpha×otheri\text{{out}}_i = \text{{input}}_i - \text{{alpha}} \times \text{{other}}_i
Supports broadcasting to a common shape, type promotion, and integer, float, and complex inputs.  Parameters

input (Tensor) – the input tensor.
other (Tensor or Scalar) – the tensor or scalar to subtract from input
   Keyword Arguments

alpha (Scalar) – the scalar multiplier for other

out (Tensor, optional) – the output tensor.    Example: >>> a = torch.tensor((1, 2))
>>> b = torch.tensor((0, 1))
>>> torch.sub(a, b, alpha=2)
tensor([1, 0])


torch.subtract(input, other, *, alpha=1, out=None) → Tensor
Alias for torch.sub().


torch.sum(input, *, dtype=None) → Tensor
Returns the sum of all elements in the input tensor.  Parameters
input (Tensor) – the input tensor.  Keyword Arguments
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.   Example: >>> a = torch.randn(1, 3)
>>> a
tensor([[ 0.1133, -0.9567,  0.2958]])
>>> torch.sum(a)
tensor(-0.5475)

torch.sum(input, dim, keepdim=False, *, dtype=None) → Tensor
 Returns the sum of each row of the input tensor in the given dimension dim. If dim is a list of dimensions, reduce over all of them. If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).  Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.   Keyword Arguments
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.   Example: >>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.0569, -0.2475,  0.0737, -0.3429],
        [-0.2993,  0.9138,  0.9337, -1.6864],
        [ 0.1132,  0.7892, -0.1003,  0.5688],
        [ 0.3637, -0.9906, -0.4752, -1.5197]])
>>> torch.sum(a, 1)
tensor([-0.4598, -0.1381,  1.3708, -2.6217])
>>> b = torch.arange(4 * 5 * 6).view(4, 5, 6)
>>> torch.sum(b, (2, 1))
tensor([  435.,  1335.,  2235.,  3135.])


torch.svd(input, some=True, compute_uv=True, *, out=None) -> (Tensor, Tensor, Tensor)
Computes the singular value decomposition of either a matrix or batch of matrices input. The singular value decomposition is represented as a namedtuple (U,S,V), such that input = U diag(S) Vᴴ, where Vᴴ is the transpose of V for the real-valued inputs, or the conjugate transpose of V for the complex-valued inputs. If input is a batch of tensors, then U, S, and V are also batched with the same batch dimensions as input. If some is True (default), the method returns the reduced singular value decomposition i.e., if the last two dimensions of input are m and n, then the returned U and V matrices will contain only min(n, m) orthonormal columns. If compute_uv is False, the returned U and V will be zero-filled matrices of shape (m × m) and (n × n) respectively, and the same device as input. The some argument has no effect when compute_uv is False. Supports input of float, double, cfloat and cdouble data types. The dtypes of U and V are the same as input’s. S will always be real-valued, even if input is complex.  Warning torch.svd() is deprecated. Please use torch.linalg.svd() instead, which is similar to NumPy’s numpy.linalg.svd.   Note Differences with torch.linalg.svd():
some is the opposite of torch.linalg.svd()’s full_matricies. Note that default value for both is True, so the default behavior is effectively the opposite.
torch.svd() returns V, whereas torch.linalg.svd() returns Vᴴ. If compute_uv=False, torch.svd() returns zero-filled tensors for U and Vh, whereas torch.linalg.svd() returns empty tensors.    Note The singular values are returned in descending order. If input is a batch of matrices, then the singular values of each matrix in the batch is returned in descending order.   Note The implementation of SVD on CPU uses the LAPACK routine ?gesdd (a divide-and-conquer algorithm) instead of ?gesvd for speed. Analogously, the SVD on GPU uses the cuSOLVER routines gesvdj and gesvdjBatched on CUDA 10.1.243 and later, and uses the MAGMA routine gesdd on earlier versions of CUDA.   Note The returned matrix U will be transposed, i.e. with strides U.contiguous().transpose(-2, -1).stride().   Note Gradients computed using U and V may be unstable if input is not full rank or has non-unique singular values.   Note When some = False, the gradients on U[..., :, min(m, n):] and V[..., :, min(m, n):] will be ignored in backward as those vectors can be arbitrary bases of the subspaces.   Note The S tensor can only be used to compute gradients if compute_uv is True.   Note With the complex-valued input the backward operation works correctly only for gauge invariant loss functions. Please look at Gauge problem in AD for more details.   Note Since U and V of an SVD is not unique, each vector can be multiplied by an arbitrary phase factor eiϕe^{i \phi}  while the SVD result is still correct. Different platforms, like Numpy, or inputs on different device types, may produce different U and V tensors.   Parameters

input (Tensor) – the input tensor of size (*, m, n) where * is zero or more batch dimensions consisting of (m × n) matrices.
some (bool, optional) – controls whether to compute the reduced or full decomposition, and consequently the shape of returned U and V. Defaults to True.
compute_uv (bool, optional) – option whether to compute U and V or not. Defaults to True.   Keyword Arguments
out (tuple, optional) – the output tuple of tensors   Example: >>> a = torch.randn(5, 3)
>>> a
tensor([[ 0.2364, -0.7752,  0.6372],
        [ 1.7201,  0.7394, -0.0504],
        [-0.3371, -1.0584,  0.5296],
        [ 0.3550, -0.4022,  1.5569],
        [ 0.2445, -0.0158,  1.1414]])
>>> u, s, v = torch.svd(a)
>>> u
tensor([[ 0.4027,  0.0287,  0.5434],
        [-0.1946,  0.8833,  0.3679],
        [ 0.4296, -0.2890,  0.5261],
        [ 0.6604,  0.2717, -0.2618],
        [ 0.4234,  0.2481, -0.4733]])
>>> s
tensor([2.3289, 2.0315, 0.7806])
>>> v
tensor([[-0.0199,  0.8766,  0.4809],
        [-0.5080,  0.4054, -0.7600],
        [ 0.8611,  0.2594, -0.4373]])
>>> torch.dist(a, torch.mm(torch.mm(u, torch.diag(s)), v.t()))
tensor(8.6531e-07)
>>> a_big = torch.randn(7, 5, 3)
>>> u, s, v = torch.svd(a_big)
>>> torch.dist(a_big, torch.matmul(torch.matmul(u, torch.diag_embed(s)), v.transpose(-2, -1)))
tensor(2.6503e-06)


torch.svd_lowrank(A, q=6, niter=2, M=None) [source]

Return the singular value decomposition (U, S, V) of a matrix, batches of matrices, or a sparse matrix AA  such that A≈Udiag(S)VTA \approx U diag(S) V^T . In case MM  is given, then SVD is computed for the matrix A−MA - M .  Note The implementation is based on the Algorithm 5.1 from Halko et al, 2009.   Note To obtain repeatable results, reset the seed for the pseudorandom number generator   Note The input is assumed to be a low-rank matrix.   Note In general, use the full-rank SVD implementation torch.svd for dense matrices due to its 10-fold higher performance characteristics. The low-rank SVD will be useful for huge sparse matrices that torch.svd cannot handle.   Args::

A (Tensor): the input tensor of size (∗,m,n)(*, m, n)  q (int, optional): a slightly overestimated rank of A.  niter (int, optional): the number of subspace iterations to

conduct; niter must be a nonnegative integer, and defaults to 2  M (Tensor, optional): the input tensor’s mean of size

(∗,1,n)(*, 1, n) .    References::

 Nathan Halko, Per-Gunnar Martinsson, and Joel Tropp, Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions, arXiv:0909.4061 [math.NA; math.PR], 2009 (available at arXiv).


torch.swapaxes(input, axis0, axis1) → Tensor
Alias for torch.transpose(). This function is equivalent to NumPy’s swapaxes function. Examples: >>> x = torch.tensor([[[0,1],[2,3]],[[4,5],[6,7]]])
>>> x
tensor([[[0, 1],
        [2, 3]],

        [[4, 5],
        [6, 7]]])
>>> torch.swapaxes(x, 0, 1)
tensor([[[0, 1],
        [4, 5]],

        [[2, 3],
        [6, 7]]])
>>> torch.swapaxes(x, 0, 2)
tensor([[[0, 4],
        [2, 6]],

        [[1, 5],
        [3, 7]]])


torch.swapdims(input, dim0, dim1) → Tensor
Alias for torch.transpose(). This function is equivalent to NumPy’s swapaxes function. Examples: >>> x = torch.tensor([[[0,1],[2,3]],[[4,5],[6,7]]])
>>> x
tensor([[[0, 1],
        [2, 3]],

        [[4, 5],
        [6, 7]]])
>>> torch.swapdims(x, 0, 1)
tensor([[[0, 1],
        [4, 5]],

        [[2, 3],
        [6, 7]]])
>>> torch.swapdims(x, 0, 2)
tensor([[[0, 4],
        [2, 6]],

        [[1, 5],
        [3, 7]]])


torch.symeig(input, eigenvectors=False, upper=True, *, out=None) -> (Tensor, Tensor)
This function returns eigenvalues and eigenvectors of a real symmetric matrix input or a batch of real symmetric matrices, represented by a namedtuple (eigenvalues, eigenvectors). This function calculates all eigenvalues (and vectors) of input such that input=Vdiag(e)VT\text{input} = V \text{diag}(e) V^T . The boolean argument eigenvectors defines computation of both eigenvectors and eigenvalues or eigenvalues only. If it is False, only eigenvalues are computed. If it is True, both eigenvalues and eigenvectors are computed. Since the input matrix input is supposed to be symmetric, only the upper triangular portion is used by default. If upper is False, then lower triangular portion is used.  Note The eigenvalues are returned in ascending order. If input is a batch of matrices, then the eigenvalues of each matrix in the batch is returned in ascending order.   Note Irrespective of the original strides, the returned matrix V will be transposed, i.e. with strides V.contiguous().transpose(-1, -2).stride().   Warning Extra care needs to be taken when backward through outputs. Such operation is only stable when all eigenvalues are distinct and becomes less stable the smaller mini≠j∣λi−λj∣\min_{i \neq j} |\lambda_i - \lambda_j|  is.   Parameters

input (Tensor) – the input tensor of size (∗,n,n)(*, n, n)  where * is zero or more batch dimensions consisting of symmetric matrices.
eigenvectors (bool, optional) – controls whether eigenvectors have to be computed
upper (boolean, optional) – controls whether to consider upper-triangular or lower-triangular region   Keyword Arguments
out (tuple, optional) – the output tuple of (Tensor, Tensor)  Returns
A namedtuple (eigenvalues, eigenvectors) containing
eigenvalues (Tensor): Shape (∗,m)(*, m) . The eigenvalues in ascending order.
eigenvectors (Tensor): Shape (∗,m,m)(*, m, m) . If eigenvectors=False, it’s an empty tensor. Otherwise, this tensor contains the orthonormal eigenvectors of the input.   Return type
(Tensor, Tensor)   Examples: >>> a = torch.randn(5, 5)
>>> a = a + a.t()  # To make a symmetric
>>> a
tensor([[-5.7827,  4.4559, -0.2344, -1.7123, -1.8330],
        [ 4.4559,  1.4250, -2.8636, -3.2100, -0.1798],
        [-0.2344, -2.8636,  1.7112, -5.5785,  7.1988],
        [-1.7123, -3.2100, -5.5785, -2.6227,  3.1036],
        [-1.8330, -0.1798,  7.1988,  3.1036, -5.1453]])
>>> e, v = torch.symeig(a, eigenvectors=True)
>>> e
tensor([-13.7012,  -7.7497,  -2.3163,   5.2477,   8.1050])
>>> v
tensor([[ 0.1643,  0.9034, -0.0291,  0.3508,  0.1817],
        [-0.2417, -0.3071, -0.5081,  0.6534,  0.4026],
        [-0.5176,  0.1223, -0.0220,  0.3295, -0.7798],
        [-0.4850,  0.2695, -0.5773, -0.5840,  0.1337],
        [ 0.6415, -0.0447, -0.6381, -0.0193, -0.4230]])
>>> a_big = torch.randn(5, 2, 2)
>>> a_big = a_big + a_big.transpose(-2, -1)  # To make a_big symmetric
>>> e, v = a_big.symeig(eigenvectors=True)
>>> torch.allclose(torch.matmul(v, torch.matmul(e.diag_embed(), v.transpose(-2, -1))), a_big)
True


torch.t(input) → Tensor
Expects input to be <= 2-D tensor and transposes dimensions 0 and 1. 0-D and 1-D tensors are returned as is. When input is a 2-D tensor this is equivalent to transpose(input, 0, 1).  Parameters
input (Tensor) – the input tensor.   Example: >>> x = torch.randn(())
>>> x
tensor(0.1995)
>>> torch.t(x)
tensor(0.1995)
>>> x = torch.randn(3)
>>> x
tensor([ 2.4320, -0.4608,  0.7702])
>>> torch.t(x)
tensor([ 2.4320, -0.4608,  0.7702])
>>> x = torch.randn(2, 3)
>>> x
tensor([[ 0.4875,  0.9158, -0.5872],
        [ 0.3938, -0.6929,  0.6932]])
>>> torch.t(x)
tensor([[ 0.4875,  0.3938],
        [ 0.9158, -0.6929],
        [-0.5872,  0.6932]])


torch.take(input, index) → Tensor
Returns a new tensor with the elements of input at the given indices. The input tensor is treated as if it were viewed as a 1-D tensor. The result takes the same shape as the indices.  Parameters

input (Tensor) – the input tensor.
indices (LongTensor) – the indices into tensor    Example: >>> src = torch.tensor([[4, 3, 5],
...                     [6, 7, 8]])
>>> torch.take(src, torch.tensor([0, 2, 5]))
tensor([ 4,  5,  8])


torch.tan(input, *, out=None) → Tensor
Returns a new tensor with the tangent of the elements of input.  outi=tan(inputi)\text{out}_{i} = \tan(\text{input}_{i})
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([-1.2027, -1.7687,  0.4412, -1.3856])
>>> torch.tan(a)
tensor([-2.5930,  4.9859,  0.4722, -5.3366])


torch.tanh(input, *, out=None) → Tensor
Returns a new tensor with the hyperbolic tangent of the elements of input.  outi=tanh(inputi)\text{out}_{i} = \tanh(\text{input}_{i})
 Parameters
input (Tensor) – the input tensor.  Keyword Arguments
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4)
>>> a
tensor([ 0.8986, -0.7279,  1.1745,  0.2611])
>>> torch.tanh(a)
tensor([ 0.7156, -0.6218,  0.8257,  0.2553])