shrinath-suresh/blogs-docs-splitted huggingface dataset:
+----------------------+----------------------+
| "torch.is_tensor()"  | None                 |
+----------------------+----------------------+
| "Tensor.item()"      | None                 |
+----------------------+----------------------+
| "Tensor.kthvalue()", | Removes dimensions   |
| "torch.kthvalue()"   |                      |
+----------------------+----------------------+
| "Tensor.le()",       | Unifies names from   |
| "torch.le()"         | inputs               |
+----------------------+----------------------+
| "Tensor.log()",      | Keeps input names    |
| "torch.log()"        |                      |
+----------------------+----------------------+
| "Tensor.log10()",    | Keeps input names    |
| "torch.log10()"      |                      |
+----------------------+----------------------+
| "Tensor.log10_()"    | None                 |
+----------------------+----------------------+
| "Tensor.log1p()",    | Keeps input names    |
| "torch.log1p()"      |                      |
| "torch.log1p()"      |                      |
+----------------------+----------------------+
| "Tensor.log1p_()"    | None                 |
+----------------------+----------------------+
| "Tensor.log2()",     | Keeps input names    |
| "torch.log2()"       |                      |
+----------------------+----------------------+
| "Tensor.log2_()"     | None                 |
+----------------------+----------------------+
| "Tensor.log_()"      | None                 |
+----------------------+----------------------+
| "Tensor.log_normal_  | None                 |
| ()"                  |                      |
+----------------------+----------------------+
| "Tensor.logical_not  | Keeps input names    |
| ()", "torch.logical  |                      |
| not()"              |                      |
+----------------------+----------------------+
| "Tensor.logical_not  | None                 |
| ()"                 |                      |
+----------------------+----------------------+
+----------------------+----------------------+
| "Tensor.logsumexp()  | Removes dimensions   |
| ",                   |                      |
| "torch.logsumexp()"  |                      |
+----------------------+----------------------+
| "Tensor.long()"      | Keeps input names    |
+----------------------+----------------------+
| "Tensor.lt()",       | Unifies names from   |
| "torch.lt()"         | inputs               |
+----------------------+----------------------+
| "torch.manual_seed(  | None                 |
| )"                   |                      |
+----------------------+----------------------+
| "Tensor.masked_fill  | Keeps input names    |
| ()", "torch.masked_  |                      |
| fill()"              |                      |
+----------------------+----------------------+
| "Tensor.masked_fill  | None                 |
| _()"                 |                      |
+----------------------+----------------------+
| "Tensor.masked_sele  | Aligns mask up to    |
| "Tensor.masked_sele  | Aligns mask up to    |
| ct()", "torch.maske  | input and then unif  |
| d_select()"          | ies_names_from_inpu  |
|                      | t_tensors            |
+----------------------+----------------------+
| "Tensor.matmul()",   | Contracts away dims  |
| "torch.matmul()"     |                      |
+----------------------+----------------------+
| "Tensor.mean()",     | Removes dimensions   |
| "torch.mean()"       |                      |
+----------------------+----------------------+
| "Tensor.median()",   | Removes dimensions   |
| "torch.median()"     |                      |
+----------------------+----------------------+
| "Tensor.nanmedian()  | Removes dimensions   |
| ",                   |                      |
| "torch.nanmedian()"  |                      |
+----------------------+----------------------+
| "Tensor.mm()",       | Contracts away dims  |
| "torch.mm()"         |                      |
+----------------------+----------------------+
+----------------------+----------------------+
| "Tensor.mode()",     | Removes dimensions   |
| "torch.mode()"       |                      |
+----------------------+----------------------+
| "Tensor.mul()",      | Unifies names from   |
| "torch.mul()"        | inputs               |
+----------------------+----------------------+
| "Tensor.mul_()"      | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.mv()",       | Contracts away dims  |
| "torch.mv()"         |                      |
+----------------------+----------------------+
| "Tensor.names"       | See documentation    |
+----------------------+----------------------+
| "Tensor.narrow()",   | Keeps input names    |
| "torch.narrow()"     |                      |
+----------------------+----------------------+
| "Tensor.ndim"        | None                 |
+----------------------+----------------------+
| "Tensor.ndimension(  | None                 |
| "Tensor.ndimension(  | None                 |
| )"                   |                      |
+----------------------+----------------------+
| "Tensor.ne()",       | Unifies names from   |
| "torch.ne()"         | inputs               |
+----------------------+----------------------+
| "Tensor.neg()",      | Keeps input names    |
| "torch.neg()"        |                      |
+----------------------+----------------------+
| "Tensor.neg_()"      | None                 |
+----------------------+----------------------+
| "torch.normal()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.normal_()"   | None                 |
+----------------------+----------------------+
| "Tensor.numel()",    | None                 |
| "torch.numel()"      |                      |
+----------------------+----------------------+
| "torch.ones()"       | Factory functions    |
+----------------------+----------------------+
| "Tensor.pow()",      | Unifies names from   |
| "Tensor.pow()",      | Unifies names from   |
| "torch.pow()"        | inputs               |
+----------------------+----------------------+
| "Tensor.pow_()"      | None                 |
+----------------------+----------------------+
| "Tensor.prod()",     | Removes dimensions   |
| "torch.prod()"       |                      |
+----------------------+----------------------+
| "Tensor.rad2deg()",  | Keeps input names    |
| "torch.rad2deg()"    |                      |
+----------------------+----------------------+
| "Tensor.rad2deg_()"  | None                 |
+----------------------+----------------------+
| "torch.rand()"       | Factory functions    |
+----------------------+----------------------+
| "torch.rand()"       | Factory functions    |
+----------------------+----------------------+
| "torch.randn()"      | Factory functions    |
+----------------------+----------------------+
| "torch.randn()"      | Factory functions    |
+----------------------+----------------------+
+----------------------+----------------------+
| "Tensor.random_()"   | None                 |
+----------------------+----------------------+
| "Tensor.reciprocal(  | Keeps input names    |
| )",                  |                      |
| "torch.reciprocal()" |                      |
+----------------------+----------------------+
| "Tensor.reciprocal_  | None                 |
| ()"                  |                      |
+----------------------+----------------------+
| "Tensor.refine_name  | See documentation    |
| s()"                 |                      |
+----------------------+----------------------+
| "Tensor.register_ho  | None                 |
| ok()"                |                      |
+----------------------+----------------------+
| "Tensor.rename()"    | See documentation    |
+----------------------+----------------------+
| "Tensor.rename_()"   | See documentation    |
+----------------------+----------------------+
| "Tensor.requires_gr  | None                 |
| "Tensor.requires_gr  | None                 |
| ad"                  |                      |
+----------------------+----------------------+
| "Tensor.requires_gr  | None                 |
| ad_()"               |                      |
+----------------------+----------------------+
| "Tensor.resize_()"   | Only allow resizes   |
|                      | that do not change   |
|                      | shape                |
+----------------------+----------------------+
| "Tensor.resize_as_(  | Only allow resizes   |
| )"                   | that do not change   |
|                      | shape                |
+----------------------+----------------------+
| "Tensor.round()",    | Keeps input names    |
| "torch.round()"      |                      |
+----------------------+----------------------+
| "Tensor.round_()"    | None                 |
+----------------------+----------------------+
| "Tensor.rsqrt()",    | Keeps input names    |
| "torch.rsqrt()"      |                      |
| "torch.rsqrt()"      |                      |
+----------------------+----------------------+
| "Tensor.rsqrt_()"    | None                 |
+----------------------+----------------------+
| "Tensor.select()",   | Removes dimensions   |
| "torch.select()"     |                      |
+----------------------+----------------------+
| "Tensor.short()"     | Keeps input names    |
+----------------------+----------------------+
| "Tensor.sigmoid()",  | Keeps input names    |
| "torch.sigmoid()"    |                      |
+----------------------+----------------------+
| "Tensor.sigmoid_()"  | None                 |
+----------------------+----------------------+
| "Tensor.sign()",     | Keeps input names    |
| "torch.sign()"       |                      |
+----------------------+----------------------+
| "Tensor.sign_()"     | None                 |
+----------------------+----------------------+
| "Tensor.sgn()",      | Keeps input names    |
| "torch.sgn()"        |                      |
| "torch.sgn()"        |                      |
+----------------------+----------------------+
| "Tensor.sgn_()"      | None                 |
+----------------------+----------------------+
| "Tensor.sin()",      | Keeps input names    |
| "torch.sin()"        |                      |
+----------------------+----------------------+
| "Tensor.sin_()"      | None                 |
+----------------------+----------------------+
| "Tensor.sinh()",     | Keeps input names    |
| "torch.sinh()"       |                      |
+----------------------+----------------------+
| "Tensor.sinh_()"     | None                 |
+----------------------+----------------------+
| "Tensor.asinh()",    | Keeps input names    |
| "torch.asinh()"      |                      |
+----------------------+----------------------+
| "Tensor.asinh_()"    | None                 |
+----------------------+----------------------+
| "Tensor.size()"      | None                 |
+----------------------+----------------------+
+----------------------+----------------------+
| "Tensor.softmax()",  | Keeps input names    |
| "torch.softmax()"    |                      |
+----------------------+----------------------+
| "Tensor.split()",    | Keeps input names    |
| "torch.split()"      |                      |
+----------------------+----------------------+
| "Tensor.sqrt()",     | Keeps input names    |
| "torch.sqrt()"       |                      |
+----------------------+----------------------+
| "Tensor.sqrt_()"     | None                 |
+----------------------+----------------------+
| "Tensor.squeeze()",  | Removes dimensions   |
| "torch.squeeze()"    |                      |
+----------------------+----------------------+
| "Tensor.std()",      | Removes dimensions   |
| "torch.std()"        |                      |
+----------------------+----------------------+
| "torch.std_mean()"   | Removes dimensions   |
+----------------------+----------------------+
| "Tensor.stride()"    | None                 |
| "Tensor.stride()"    | None                 |
+----------------------+----------------------+
| "Tensor.sub()",      | Unifies names from   |
| "torch.sub()"        | inputs               |
+----------------------+----------------------+
| "Tensor.sub_()"      | Unifies names from   |
|                      | inputs               |
+----------------------+----------------------+
| "Tensor.sum()",      | Removes dimensions   |
| "torch.sum()"        |                      |
+----------------------+----------------------+
| "Tensor.tan()",      | Keeps input names    |
| "torch.tan()"        |                      |
+----------------------+----------------------+
| "Tensor.tan_()"      | None                 |
+----------------------+----------------------+
| "Tensor.tanh()",     | Keeps input names    |
| "torch.tanh()"       |                      |
+----------------------+----------------------+
| "Tensor.tanh_()"     | None                 |
+----------------------+----------------------+
+----------------------+----------------------+
| "Tensor.atanh()",    | Keeps input names    |
| "torch.atanh()"      |                      |
+----------------------+----------------------+
| "Tensor.atanh_()"    | None                 |
+----------------------+----------------------+
| "torch.tensor()"     | Factory functions    |
+----------------------+----------------------+
| "Tensor.to()"        | Keeps input names    |
+----------------------+----------------------+
| "Tensor.topk()",     | Removes dimensions   |
| "torch.topk()"       |                      |
+----------------------+----------------------+
| "Tensor.transpose()  | Permutes dimensions  |
| ",                   |                      |
| "torch.transpose()"  |                      |
+----------------------+----------------------+
| "Tensor.trunc()",    | Keeps input names    |
| "torch.trunc()"      |                      |
+----------------------+----------------------+
| "Tensor.trunc_()"    | None                 |
| "Tensor.trunc_()"    | None                 |
+----------------------+----------------------+
| "Tensor.type()"      | None                 |
+----------------------+----------------------+
| "Tensor.type_as()"   | Keeps input names    |
+----------------------+----------------------+
| "Tensor.unbind()",   | Removes dimensions   |
| "torch.unbind()"     |                      |
+----------------------+----------------------+
| "Tensor.unflatten()" | See documentation    |
+----------------------+----------------------+
| "Tensor.uniform_()"  | None                 |
+----------------------+----------------------+
| "Tensor.var()",      | Removes dimensions   |
| "torch.var()"        |                      |
+----------------------+----------------------+
| "torch.var_mean()"   | Removes dimensions   |
+----------------------+----------------------+
| "Tensor.zero_()"     | None                 |
+----------------------+----------------------+
| "torch.zeros()"      | Factory functions    |
| "torch.zeros()"      | Factory functions    |
+----------------------+----------------------+
Keeps input names
All pointwise unary functions follow this rule as well as some other
unary functions.


Check names: None


Propagate names: input tensor's names are propagated to the output.





x = torch.randn(3, 3, names=('N', 'C'))
x.abs().names
   ('N', 'C')



Removes dimensions
All reduction ops like "sum()" remove dimensions by reducing over the
desired dimensions. Other operations like "select()" and "squeeze()"
remove dimensions.
Wherever one can pass an integer dimension index to an operator, one
can also pass a dimension name. Functions that take lists of dimension
indices can also take in a list of dimension names.


Check names: If "dim" or "dims" is passed in as a list of names,
  check that those names exist in "self".


Propagate names: If the dimensions of the input tensor specified by
"dim" or "dims" are not present in the output tensor, then the
  corresponding names of those dimensions do not appear in
  "output.names".



x = torch.randn(1, 3, 3, 3, names=('N', 'C', 'H', 'W'))
x.squeeze('N').names
   ('C', 'H', 'W')
x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
x.sum(['N', 'C']).names
   ('H', 'W')



# Reduction ops with keepdim=True don't actually remove dimensions.



x = torch.randn(3, 3, 3, 3, names=('N', 'C', 'H', 'W'))
x.sum(['N', 'C'], keepdim=True).names
   ('N', 'C', 'H', 'W')



Unifies names from inputs
All binary arithmetic ops follow this rule. Operations that broadcast
still broadcast positionally from the right to preserve compatibility
with unnamed tensors. To perform explicit broadcasting by names, use
"Tensor.align_as()".

Check names: All names must match positionally from the right. i.e.,
  in "tensor + other", "match(tensor.names[i], other.names[i])" must
be true for all "i" in "(-min(tensor.dim(), other.dim()) + 1, -1]".


Check names: Furthermore, all named dimensions must be aligned from
  the right. During matching, if we match a named dimension "A" with
  an unnamed dimension "None", then "A" must not appear in the tensor
  with the unnamed dimension.


Propagate names: unify pairs of names from the right from both
  tensors to produce output names.


For example,
# tensor: Tensor[   N, None]
   # other:  Tensor[None,    C]



tensor = torch.randn(3, 3, names=('N', None))
other = torch.randn(3, 3, names=(None, 'C'))
(tensor + other).names
   ('N', 'C')



Check names:


"match(tensor.names[-1], other.names[-1])" is "True"


"match(tensor.names[-2], tensor.names[-2])" is "True"


Because we matched "None" in "tensor" with "'C'", check to make sure
  "'C'" doesn't exist in "tensor" (it does not).


Check to make sure "'N'" doesn't exists in "other" (it does not).


Finally, the output names are computed with "[unify('N', None),
unify(None, 'C')] = ['N', 'C']"
More examples:
# Dimensions don't match from the right:
   # tensor: Tensor[N, C]
   # other:  Tensor[   N]



tensor = torch.randn(3, 3, names=('N', 'C'))
other = torch.randn(3, names=('N',))
(tensor + other).names
   RuntimeError: Error when attempting to broadcast dims ['N', 'C'] and dims
   ['N']: dim 'C' and dim 'N' are at the same position from the right but do
   not match.



# Dimensions aren't aligned when matching tensor.names[-1] and other.names[-1]:
   # tensor: Tensor[N, None]
   # other:  Tensor[      N]



tensor = torch.randn(3, 3, names=('N', None))
other = torch.randn(3, names=('N',))
(tensor + other).names
   RuntimeError: Misaligned dims when attempting to broadcast dims ['N'] and
   dims ['N', None]: dim 'N' appears in a different position from the right
   across both lists.



Note:
In both of the last examples, it is possible to align the tensors by
names and then perform the addition. Use "Tensor.align_as()" to
  align tensors by name or "Tensor.align_to()" to align tensors to a
  custom dimension ordering.
Permutes dimensions
Some operations, like "Tensor.t()", permute the order of dimensions.
Dimension names are attached to individual dimensions so they get
permuted as well.
If the operator takes in positional index "dim", it is also able to
take a dimension name as "dim".


Check names: If "dim" is passed as a name, check that it exists in
  the tensor.


Propagate names: Permute dimension names in the same way as the
  dimensions that are being permuted.





x = torch.randn(3, 3, names=('N', 'C'))
x.transpose('N', 'C').names
   ('C', 'N')



Contracts away dims
Matrix multiply functions follow some variant of this. Let's go
through "torch.mm()" first and then generalize the rule for batch
matrix multiplication.
For "torch.mm(tensor, other)":

Check names: None
Check names: None


Propagate names: result names are "(tensor.names[-2],
  other.names[-1])".





x = torch.randn(3, 3, names=('N', 'D'))
y = torch.randn(3, 3, names=('in', 'out'))
x.mm(y).names
   ('N', 'out')



Inherently, a matrix multiplication performs a dot product over two
dimensions, collapsing them. When two tensors are matrix-multiplied,
the contracted dimensions disappear and do not show up in the output
tensor.
"torch.mv()", "torch.dot()" work in a similar way: name inference does
not check input names and removes the dimensions that are involved in
the dot product:



x = torch.randn(3, 3, names=('N', 'D'))
y = torch.randn(3, names=('something',))
x.mv(y).names
   ('N',)



Now, let's take a look at "torch.matmul(tensor, other)". Assume that
"tensor.dim() >= 2" and "other.dim() >= 2".

Check names: Check that the batch dimensions of the inputs are
  aligned and broadcastable. See Unifies names from inputs for what it
  means for the inputs to be aligned.
means for the inputs to be aligned.

Propagate names: result names are obtained by unifying the batch
  dimensions and removing the contracted dimensions:
  "unify(tensor.names[:-2], other.names[:-2]) + (tensor.names[-2],
  other.names[-1])".

Examples:
# Batch matrix multiply of matrices Tensor['C', 'D'] and Tensor['E', 'F'].
   # 'A', 'B' are batch dimensions.



x = torch.randn(3, 3, 3, 3, names=('A', 'B', 'C', 'D'))
y = torch.randn(3, 3, 3, names=('B', 'E', 'F'))
torch.matmul(x, y).names
   ('A', 'B', 'C', 'F')



Finally, there are fused "add" versions of many matmul functions.
i.e., "addmm()" and "addmv()". These are treated as composing name
inference for i.e. "mm()" and name inference for "add()".
Factory functions
Factory functions now take a new "names" argument that associates a
name with each dimension.



torch.zeros(2, 3, names=('N', 'C'))
   tensor([[0., 0., 0.],
           [0., 0., 0.]], names=('N', 'C'))



out function and in-place variants
out function and in-place variants
A tensor specified as an "out=" tensor has the following behavior:


If it has no named dimensions, then the names computed from the
  operation get propagated to it.


If it has any named dimensions, then the names computed from the
  operation must be exactly equal to the existing names. Otherwise,
  the operation errors.


All in-place methods modify inputs to have names equal to the computed
names from name inference. For example:



x = torch.randn(3, 3)
y = torch.randn(3, 3, names=('N', 'C'))
x.names
   (None, None)
x += y
x.names
   ('N', 'C')
torch.library
Python operator registration API provides capabilities for extending
PyTorch's core library of operators with user defined operators.
Currently, this can be done in two ways:


Creating new libraries


Lets you to register new operators and kernels for various
     backends and functionalities by specifying the appropriate
     dispatch keys. For example,
* Consider registering a new operator "add" in your newly
  created namespace "foo". You can access this operator using
  the "torch.ops" API and calling into by calling
  "torch.ops.foo.add". You can also access specific registered
  overloads by calling "torch.ops.foo.add.{overload_name}".

* If you registered a new kernel for the "CUDA" dispatch key
  for this operator, then your custom defined function will be
  called for CUDA tensor inputs.



This can be done by creating Library class objects of ""DEF""
     kind.
kind.


Extending existing C++ libraries (e.g., aten)


Lets you register kernels for existing operators
     corresponding to various backends and functionalities by
     specifying the appropriate dispatch keys.


This may come in handy to fill up spotty operator support for a
     feature implemented through a dispatch key. For example.,
* You can add operator support for Meta Tensors (by
  registering function to the "Meta" dispatch key).



This can be done by creating Library class objects of ""IMPL""
     kind.


A tutorial that walks you through some examples on how to use this API
is available on Google Colab.
Warning:
Dispatcher is a complicated PyTorch concept and having a sound
  understanding of Dispatcher is crucial to be able to do anything
  advanced with this API. This blog post is a good starting point to
  learn about Dispatcher.
class torch.library.Library(ns, kind, dispatch_key='')
A class to create libraries that can be used to register new