Functions and operators for Dot and Matrix multiplication and Element-wise calculation in PyTorch
Tags: pytorch, function, matrix, multiplication
Published: 2024-03-21 00:37:51

*Memos:
- [My post](https://dev.to/hyperkai/matrix-and-element-wise-multiplication-in-pytorch-25le) explains Matrix and Element-wise multiplication in PyTorch.
- [My post](https://dev.to/hyperkai/dot-and-matrix-vector-multiplication-in-pytorch-3859) explains Dot and Matrix-vector multiplication in PyTorch.
- [My post](https://dev.to/hyperkai/matmul-and-dot-in-pytorch-4e8p) explains [matmul()](https://pytorch.org/docs/stable/generated/torch.matmul.html) and [dot()](https://pytorch.org/docs/stable/generated/torch.dot.html)
- [My post](https://dev.to/hyperkai/mv-mm-and-bmm-in-pytorch-57nh) explains [mv()](https://pytorch.org/docs/stable/generated/torch.mv.html), [mm()](https://pytorch.org/docs/stable/generated/torch.mm.html) and [bmm()](https://pytorch.org/docs/stable/generated/torch.bmm.html).
- [My post](https://dev.to/hyperkai/add-sub-and-mul-in-pytorch-1m1f) explains [add()](https://pytorch.org/docs/stable/generated/torch.add.html), [sub()](https://pytorch.org/docs/stable/generated/torch.sub.html) and [mul()](https://pytorch.org/docs/stable/generated/torch.mul.html).
- [My post](https://dev.to/hyperkai/div-remainder-and-fmod-in-pytorch-1eh1) explains [div()](https://pytorch.org/docs/stable/generated/torch.div.html), [remainder()](https://pytorch.org/docs/stable/generated/torch.remainder.html) and [fmod()](https://pytorch.org/docs/stable/generated/torch.fmod.html).

### <**Dot multiplication**>

- [dot()](https://pytorch.org/docs/stable/generated/torch.dot.html) can multiply 1D tensors:

```python
import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor
   
torch.dot(tensor1, tensor2) # tensor(53)
tensor1.dot(tensor2) # tensor(53)
```

- `@` or [matmul()](https://pytorch.org/docs/stable/generated/torch.matmul.html) can multiply 1D or more D tensors:

```python
import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

tensor1 @ tensor2 # tensor(53)

torch.matmul(tensor1, tensor2) # tensor(53)
tensor1.matmul(tensor2) # tensor(53)
```

### <**Matrix multiplication**>

- [mv()](https://pytorch.org/docs/stable/generated/torch.mv.html) can multiply a 2D and 1D tensor:

```python
import torch

tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor
tensor2 = torch.tensor([5, 0, 8]) # 1D tensor

torch.mv(tensor1, tensor2) # tensor([42, 56])
tensor1.mv(tensor2) # tensor([42, 56])
```

- [mm()](https://pytorch.org/docs/stable/generated/torch.mm.html) can multiply 2D tensors:

```python
import torch

tensor1 = torch.tensor([[2, 7, 4], [8, 3, 2]]) # 2D tensor
tensor2 = torch.tensor([[5, 0, 8, 6], # 2D tensor
                        [3, 6, 1, 7],
                        [1, 4, 9, 2]])

torch.mm(tensor1, tensor2)
# tensor([[35, 58, 59, 69], [51, 26, 85, 73]])
tensor1.mm(tensor2)
# tensor([[35, 58, 59, 69], [51, 26, 85, 73]])
```

- [bmm()](https://pytorch.org/docs/stable/generated/torch.bmm.html) can multiply 3D tensors:

```python
import torch

tensor1 = torch.tensor([[[2, 7]], [[8, 3]]]) # 3D tensor
tensor2 = torch.tensor([[[5, 9], [3, 6]], # 3D tensor
                        [[7, 2], [1, 4]]])
  
torch.bmm(tensor1, tensor2) # tensor([[[31, 60]], [[59, 28]]])
tensor1.bmm(tensor2) # tensor([[[31, 60]], [[59, 28]]])
```

- `@` or `matmul()` can multiply 1D or more D tensors by dot or matrix multiplication:

```python
import torch
  
tensor1 = torch.tensor([[2, 7], [8, 3]]) # 2D tensor
tensor2 = torch.tensor([[[[5, 9], [3, 6]], [[7, 2], [1, 4]]],
                        [[[6, 0], [4, 6]], [[2, 9], [8, 1]]]])
                       # 4D tensor
tensor1 @ tensor2
# tensor([[[[31, 60], [49, 90]], [[21, 32], [59, 28]]],
#         [[[40, 42], [60, 18]], [[60, 25], [40, 75]]]])
torch.matmul(tensor1, tensor2)
# tensor([[[[31, 60], [49, 90]], [[21, 32], [59, 28]]],
#         [[[40, 42], [60, 18]], [[60, 25], [40, 75]]]])
tensor1.matmul(tensor2)
# tensor([[[[31, 60], [49, 90]], [[21, 32], [59, 28]]],
#         [[[40, 42], [60, 18]], [[60, 25], [40, 75]]]])
```

### <**Element-wise calculation**>

- `*` or [mul()](https://pytorch.org/docs/stable/generated/torch.mul.html) can do multiplication with 0D or more D tensors. *`mul()` and [multiply()](https://pytorch.org/docs/stable/generated/torch.multiply.html) are the same because `multiply()` is the alias of `mul()`:

```python
import torch
  
tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

tensor1 * tensor2 # tensor([12, 21, 20])
torch.mul(tensor1, tensor2) # tensor([12, 21, 20])
tensor1.mul(tensor2) # tensor([12, 21, 20])
```

- `/` or [div()](https://pytorch.org/docs/stable/generated/torch.div.html) can do division with 0D or more D tensors:
*Memos:
- [divide()](https://pytorch.org/docs/stable/generated/torch.divide.html) is the alias of `div()`.
- [true_divide()](https://pytorch.org/docs/stable/generated/torch.true_divide.html) is the alias of `div()` with `rounding_mode=None`.
- [floor_divide()](https://pytorch.org/docs/stable/generated/torch.floor_divide.html) is the same as `div()` with `rounding_mode="trunc"` as long as I experimented:

```python
import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

tensor1 / tensor2 # tensor([0.3333, 2.3333, 0.8000])
torch.div(tensor1, tensor2) # tensor([0.3333, 2.3333, 0.8000])
tensor1.div(tensor2) # tensor([0.3333, 2.3333, 0.8000])
```

- `+` or [add()](https://pytorch.org/docs/stable/generated/torch.add.html) can do addition with 0D or more D tensors:

```python
import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

tensor1 + tensor2 # tensor([8, 10, 9])
torch.add(tensor1, tensor2) # tensor([8, 10, 9])
tensor1.add(tensor2) # tensor([8, 10, 9])
```

- `-` or [sub()](https://pytorch.org/docs/stable/generated/torch.sub.html) can do subtraction with 0D or more D tensors. *`sub()` and [subtract()](https://pytorch.org/docs/stable/generated/torch.subtract.html) are the same because `subtract()` is the alias of `sub()`:

```python
import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

tensor1 - tensor2 # tensor([-4, 4, -1])
torch.subtract(tensor1, tensor2) # tensor([-4, 4, -1])
tensor1.subtract(tensor2) # tensor([-4, 4, -1])
```

- `%` or [remainder()](https://pytorch.org/docs/stable/generated/torch.remainder.html) can do modulo(mod) calculation with 0D or more D tensors:

```python
import torch

tensor1 = torch.tensor([2, 7, 4]) # 1D tensor
tensor2 = torch.tensor([6, 3, 5]) # 1D tensor

tensor1 % tensor2 # tensor([2, 1, 4])
torch.remainder(tensor1, tensor2) # tensor([2, 1, 4])
tensor1.remainder(tensor2) # tensor([2, 1, 4])
```