Tensors for busy people
Tags: tensors, machinelearning, beginners, hpc
Published: 2019-05-27 01:12:09

---
title: Tensors for busy people
published: true
description: Intro to Tensors
tags: tensors, machinelearning, beginners, hpc
---

# Wat?

- Whats Scalar?.
- Whats Vector?.
- Whats Matrix?.
- Whats Tensor?.

_This is a quick and short intro to Tensors for busy people..._
_I will keep it [KISS](https://en.wikipedia.org/wiki/KISS_principle) and to the minimum possible for easy reading._

![](https://thepracticaldev.s3.amazonaws.com/i/0ot9j9oqg6p0vih8gtwp.jpg)

Scalar is the minimum possible unit we are working with, like an integer.

Vector is a bunch of Scalars in a particular axis, can be drawn as a line of boxes or items, lets say is a list or array collection of items.

Matrix is a 2 dimensions arrays, usually represented as a table.

Tensor is a multi-dimensional array, usually can be represented graphically as a cube.


# Rank & Dimension

- Whats Rank?.
- Whats Dimension?.

![](https://thepracticaldev.s3.amazonaws.com/i/bp6ux6ppf5t5amwkxklq.jpg)

You can imagine lists of integers, accommodated on different layouts, on a 3D space, or 2D space, etc thats the Rank.

_How it looks like on Code?._


# Scalar

```nim
let myscalar = 42
```

Scalar can be a variety of things, usually numeric values, to keep things simple and easy to understand we will use an integer here, `42` is our Scalar.

# Vector

```nim
let myvector = [1, 2, 3]
```

Vector is a collection of items, we continue using integers, it can be seen on the code as an array or list, you can draw it as a Rank 1 Vector.

# Matrix

```nim
let mymatrix = [
                 [1, 2, 3],
                 [4, 5, 6],
               ]
```

We continue adding dimensions then we end up with the Matrix, a 2D Tensor, can be simplified on code as a list with lists inside.

# Tensor


```nim
let mytensor = [
                 [
                   [1, 2, 3],
                   [4, 5, 6],
                 ],
                 [
                   [7,   8,  9],
                   [10, 11, 12],
                 ],
               ]
```

Wow, we reached the crazy cube, a multiple dimensions array of integers,
we need to convert this jam of lists into a `Tensor` object!.


# Tensor Arraymancer

```nim
import arraymancer

let mytensor = [
                 [
                   [1, 2, 3],
                   [4, 5, 6],
                 ],
                 [
                   [7,   8,  9],
                   [10, 11, 12],
                 ],
               ].toTensor
```

Done, congrats you coded your first `Tensor`!.


# Playing with Tensors

_What can I do with it?._

```nim
echo mytensor is Tensor  # Is really a Tensor type?
echo sizeOf(mytensor)    # Whats the Size of it?
echo mytensor.rank       # Whats the Rank? 
echo mytensor.shape      # Whats the Shape of my Tensor?
echo mytensor.strides    # Strides?
echo mytensor.offset     # Offset? (if any)
```


# Human readable representation

Print a human readable representation using:

```nim
echo mytensor
```

This should print on the standard output or REPL:

```nim
Tensor[int] of shape [2, 2, 3] of type "int" on backend "CPU":
| |     1       2       3 |     7       8       9 |
| |     4       5       6 |     10      11      12|

```

On this basic example I am using `CPU` as Backend, for compatibility,
yours can also say `CUDA` or `OpenCL` or `OpenMP` (if you have the hardware),
because GPUs are comfy massaging numbers.


# Tensor Slicing

You can read or write a value from the Tensor:

(Comments added just to explain the indexes, but not required)

![](https://thepracticaldev.s3.amazonaws.com/i/map0jj8vaek7l8h0bxl1.jpg)

```nim
import arraymancer

var mytensor = [
    [1, 2, 3, 4, 5] # 0
    #0  1  2  3  4
  ].toTensor

echo mytensor[0, 2]  # Read the value "3"
mytensor[0, 2] = 42  # Write the value "42"
```

Remember that stuff declared with `var` is mutable, `let` is immutable, 
a Tensor can also be Compile-Time pre-computed declaring it with `const`.

We can overwrite that Scalar value, lets see if we can get the whole Column?.

(Comments added to explain indexes, not required, we will take it away soon)

![](https://thepracticaldev.s3.amazonaws.com/i/ijblg9wdw6neyexie3p8.jpg)

```nim
import arraymancer

var mytensor = [
    [1,   2,  3,  4,  5],   # 0
    [6,   7,  8,  9, 10],   # 1
    [11, 12, 13, 14, 15],   # 2
    [16, 17, 18, 19, 20],   # 3
    [21, 22, 23, 24, 25],   # 4
    # 0   1   2   3   4
  ].toTensor

echo mytensor[_, 2]
```

The only change is the `_` on the index to select the whole **Column**.

Lets try get an entire **Row** now, without comments this time...

![](https://thepracticaldev.s3.amazonaws.com/i/vrxiui4215ui6ep7t015.jpg)


```nim
import arraymancer

var mytensor = [
    [1,   2,  3,  4,  5], 
    [6,   7,  8,  9, 10],  
    [11, 12, 13, 14, 15],  
    [16, 17, 18, 19, 20], 
    [21, 22, 23, 24, 25],  
  ].toTensor

echo mytensor[3, _]
```

I hope was not too complicated, just the `_` for the **Row**.

Now lets grab just the center without touching the rest of it.

![](https://thepracticaldev.s3.amazonaws.com/i/8js58xj7901adngb8tqv.jpg)

```nim
import arraymancer

var mytensor = [
    [1,   2,  3,  4,  5],  
    [6,   7,  8,  9, 10],  
    [11, 12, 13, 14, 15],  
    [16, 17, 18, 19, 20],  
    [21, 22, 23, 24, 25],  
  ].toTensor

echo mytensor[2, 2]
```


# Going 3D

Lets chop off a piece of this pie...

![](https://thepracticaldev.s3.amazonaws.com/i/wr7d9cvr08nquietjnr9.jpg)

```nim
import arraymancer

var mytensor = [
    [
      [ 1,  2,  3,  4,  5],      # 1st Matrix.
      [ 6,  7,  8,  9, 10],
      [11, 12, 13, 14, 15],
      [16, 17, 18, 19, 20],
      [21, 22, 23, 24, 25],
    ],
    [
      [26, 27, 28, 29, 30],      # 2nd Matrix.
      [31, 32, 33, 34, 35],
      [36, 37, 38, 39, 40],
      [41, 42, 43, 44, 45],
      [46, 47, 48, 49, 50],
    ],
    [
      [51, 52, 53, 54, 55],      # 3rd Matrix.
      [56, 57, 58, 59, 60],
      [61, 62, 63, 64, 65],
      [66, 67, 68, 69, 70],
      [71, 72, 73, 74, 75],
    ],
    [
      [76, 77, 78, 79,  80],     # 4th Matrix.
      [81, 82, 83, 84,  85],
      [86, 87, 88, 89,  90],
      [91, 92, 93, 94,  95],
      [96, 97, 98, 99, 100],
    ],
    [
      [101, 102, 103, 104, 105], # 5th Matrix.
      [106, 107, 108, 109, 110],
      [111, 112, 113, 114, 115],
      [116, 117, 118, 119, 120],
      [121, 122, 123, 124, 125],
    ]
  ].toTensor

echo mytensor[3, _, _]
```

What if we want the corner slice?.

![](https://thepracticaldev.s3.amazonaws.com/i/uvouj2twedexuk26czsf.jpg)

```nim
import arraymancer

var mytensor = [
    [
      [ 1,  2,  3,  4,  5],      # 1st Matrix.
      [ 6,  7,  8,  9, 10],
      [11, 12, 13, 14, 15],
      [16, 17, 18, 19, 20],
      [21, 22, 23, 24, 25],
    ],
    [
      [26, 27, 28, 29, 30],      # 2nd Matrix.
      [31, 32, 33, 34, 35],
      [36, 37, 38, 39, 40],
      [41, 42, 43, 44, 45],
      [46, 47, 48, 49, 50],
    ],
    [
      [51, 52, 53, 54, 55],      # 3rd Matrix.
      [56, 57, 58, 59, 60],
      [61, 62, 63, 64, 65],
      [66, 67, 68, 69, 70],
      [71, 72, 73, 74, 75],
    ],
    [
      [76, 77, 78, 79,  80],     # 4th Matrix.
      [81, 82, 83, 84,  85],
      [86, 87, 88, 89,  90],
      [91, 92, 93, 94,  95],
      [96, 97, 98, 99, 100],
    ],
    [
      [101, 102, 103, 104, 105], # 5th Matrix.
      [106, 107, 108, 109, 110],
      [111, 112, 113, 114, 115],
      [116, 117, 118, 119, 120],
      [121, 122, 123, 124, 125],
    ]
  ].toTensor

echo mytensor[_, 1, 4]
```

A last one slicing...

![](https://thepracticaldev.s3.amazonaws.com/i/p5wge2vzewvzkr2khvpr.jpg)

```nim
import arraymancer

var mytensor = [
    [
      [ 1,  2,  3,  4,  5],      # 1st Matrix.
      [ 6,  7,  8,  9, 10],
      [11, 12, 13, 14, 15],
      [16, 17, 18, 19, 20],
      [21, 22, 23, 24, 25],
    ],
    [
      [26, 27, 28, 29, 30],      # 2nd Matrix.
      [31, 32, 33, 34, 35],
      [36, 37, 38, 39, 40],
      [41, 42, 43, 44, 45],
      [46, 47, 48, 49, 50],
    ],
    [
      [51, 52, 53, 54, 55],      # 3rd Matrix.
      [56, 57, 58, 59, 60],
      [61, 62, 63, 64, 65],
      [66, 67, 68, 69, 70],
      [71, 72, 73, 74, 75],
    ],
    [
      [76, 77, 78, 79,  80],     # 4th Matrix.
      [81, 82, 83, 84,  85],
      [86, 87, 88, 89,  90],
      [91, 92, 93, 94,  95],
      [96, 97, 98, 99, 100],
    ],
    [
      [101, 102, 103, 104, 105], # 5th Matrix.
      [106, 107, 108, 109, 110],
      [111, 112, 113, 114, 115],
      [116, 117, 118, 119, 120],
      [121, 122, 123, 124, 125],
    ]
  ].toTensor

echo mytensor[2, 0, 2]
```


# Tensor Exercises

To reduce repetition lets assume this base Tensor:

```nim
import arraymancer

var mytensor = [
    [1, 2],
    [3, 4],
  ].toTensor
```

# Tensor Addition

```nim
echo mytensor + mytensor 
```

# Tensor Substraction

```nim
echo mytensor - mytensor  
```

- Multiplication is left to the reader as exercise.

# Tensor Iterator

```nim
# Values only.
for value in mytensor:
  echo value   

# Coordinates and Values.
for coordinates, value in mytensor:
  echo coordinates
  echo value
```

- Iteration of `pairs` and `items` is left to the reader as exercise.

# Tensor Broadcasting

```nim
# Broadcasting with operations are made beginning with a dot "."
echo mytensor .+ mytensor    # Addition
echo mytensor .- mytensor    # Substraction
```

- Broadcasting other operations is left to the reader as exercise.

# Tensor from list

```nim
# Tensor from vanilla std lib sequence.
let mytensor = toSeq(1..24).toTensor.reshape(2, 3, 4) 
```

- Try reshaping your Tensors to new shapes!

# Why Arraymancer?

- C Speed.
- Python like syntax, Numpy like ergonomics.
- Accelerated with Intel MKL/OpenBLAS/CUDA/OpenCL/OpenMP.
- Dependency free distribution.
- 1 Language for the whole stack from experimental quick prototypes to Performance critical Production.


# This is just the beginning, is not the end

This is barely the building block, but you can learn more:

- https://github.com/mratsim/Arraymancer#4-reasons-why-arraymancer
- https://mratsim.github.io/Arraymancer
- https://github.com/mratsim/Arraymancer#installation

_It really wont take more than a `nimble install arraymancer` to get started on CPU!._
👑