What’s a Tensor?. Unpacking a common data science... | by Giovanni Rosati | Towards Data Science

The concept of a Tensor was created in 1900 by two Italian mathematicians, Tullio Levi-Civita and Gregorio Ricci-Curbastro, and as is common, they built upon the work of other mathematicians. The Wikipedia page for “Tensor” begins:
In mathematics, a tensor is an algebraic object that describes a (multilinear) relationship between sets of algebraic objects related to a vector space.
For non-mathematicians, that’s a lot to unpack!
Fortunately, it’s a lot easier to grasp what a tensor is in a practical sense and how we manipulate them in data science.
I find it helpful to visualize a tensor as nested “lists” of numbers, with any number of “levels.” Another way of visualizing this is to imagine each cell in an Excel spreadsheet containing all the contents from another entire spreadsheet. Then, in that second spreadsheet, each cell there is also linked to another entire spreadsheet, and so on. Each spreadsheet could be thought of as an additional “dimension.”
A “first-order” tensor has just one dimension. In computer science, this would be called a “vector,” an ordered list of numbers. A “second-order” tensor would be a matrix, like an Excel spreadsheet, and have two dimensions. As the dimensions increase, we can think of this information as vectors, matrices, or arrays, nested within arrays, with any number of dimensions (n-dimension).
Here’s how a third-order tensor might be represented:
Each of the three “top-level” groups of values is represented by a different color. In this example, the digits of each value correspond to that value’s particular spot in the overall data structure. The right-most digit of each value corresponds to the top-level group to which it belongs. All values in red are in top-level group number 1, green values are in group 2, and red values are in group 3.
Within each top-level group is a matrix (two-dimensions). The left-most digit of each value corresponds to the “row” it resides in, and the middle digit corresponds to its “column.” In this way, the order of all the values is maintained and allows us to perform calculations on the entire structure using other vectors, matrices, or arrays.
Here’s the same structure, visualized with different information:
Each letter represents a unique position in the structure, with the same color-coding used to differentiate the top-level matrices.
A more complex visualization is possible with a 3-dimensional cube. Each vertical layer of the cube would contain the data from one of these “top-level” groups, creating a 3x3x3 cube with 27 unique “locations.” But once we go beyond three dimensions, our ability to visualize a tensor using graphics breaks down.
Because of that limitation, let’s instead look at how we would encode the same data using computer software. Using Python and Numpy, we could create an array with the numerical data from the first example:
from numpy import arrayT = array([[[111,121,131], [211,221,231], [311,321,331]],[[112,122,132], [212,222,232], [312,322,332]],[[113,123,133], [213,223,233], [313,323,333]],])
Encoding the data in this way makes it possible to have any number of dimensions, which is of tremendous value when we have gigantic amounts of complex data!
I hope this has helped explain why, at least in data science, tensors can be thought of as “nested” arrays.
In a future post, I’ll cover “element-wise” addition, subtraction, multiplication, division, etc. of tensors and the difference between a “Hadamard product” and a “Tensor product” — one requires tensors of identical dimensions, and one does not.
There are so many great resources for data science education on the internet. Here are a few more of my favorite resources:
Machine Learning Mastery, by Jason Brownlee
FREE neural networks introductory course, by 3Blue1Brown
Flatiron School (paid programs)
I’m optimistic that AI will increasingly be used to make life better for all of us. Do you have a data science problem/challenge? I’d love to help; let’s connect!
Thanks for reading!