Tensors and Arrays. What’s The Difference? | by Kurtis Pykes | Towards Data Science

So I’m working through the Deep Learning Specialization on Coursera. Andrew Ng, the course instructor, has been going on and on about Numpy Arrays — to the point where I could picture what it looks like in my head.
Suddenly, he said, “this is a tensor”. I looked at the output... Stopped. And thought... “Dude, it’s just a bloody array... Why couldn't you just say it’s an array. Showoff!”. I genuinely thought he was just flexing his extensive knowledge of elaborate deep learning jargon on me but the hairs on my back begun to rise each time he mentioned tensors which forced me to investigate the difference between the two.
To get to grips with Arrays, it’s best we visit the NumPy library in Python. The NumPy (Numerical Python) library is an open-source Python library and the de-facto standard for working with numerical data in Python. Thereby, NumPy is a very important part of much of Data Science work and is used extensively in other libraries such as Pandas, SciPy, Matplotlib, and Scikit-Learn.
The reason I suggested we refer to NumPy is that the array data structure is central to the NumPy library. According to the NumPy Documentation, an array can be described as “a grid of values and it contains information about the raw data, how to locate an element, and how to interpret an element. It has a grid of elements that can be indexed in various ways. The elements are all of the same type, referred to as the array dtype.”.
import numpy as np array = np.array([[[1,2,3],    [4,5,6],    [7,8,9]],[[11,12,13], [14,15,16], [17,18,19]],[[21,22,23], [24,25,26], [27,28,29]]])print(f"Data type: {type(array)}\nShape:{array.shape}\nArray:\n{array}")Data type: <class 'numpy.ndarray'>Shape: (3, 3, 3) Array: [[[ 1  2  3]   [ 4  5  6]   [ 7  8  9]][[11 12 13]   [14 15 16]   [17 18 19]][[21 22 23]   [24 25 26]   [27 28 29]]]
This differs from a standard Python list because a Python list can contain different data types within a single list, whereas all the elements in a NumPy array must be homogenous. Other differential factors include, arrays being faster, more compact, less memory intensive, and more convenient than Python list hence allowing for code to be much further optimized.
Similarly, we are going to look at TensorFlow to explore tensors. The TensorFlow Documentation states “TensorFlow is an end-to-end open-source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications.”.
Okay, not much about tensors there but the overall goal of the framework is to make Machine Learning easy accessible.
The tf.Tensor the documentation describes a tensor as a multidimensional array — Can you see why I was getting frustrated with our teacher Andrew Ng. It’s basically the same thing as a NumPy array, no? Let’s start by converting our array from the previous section to a tensor.
import tensorflow as tftensor = tf.convert_to_tensor(array, dtype=tf.int32)print(f"Data type: {type(tensor)}\nShape: {tensor.shape}\nTensor:\n{tensor}")Data type: <class 'tensorflow.python.framework.ops.EagerTensor'>Shape: (3, 3, 3)Tensor: [[[ 1  2  3]   [ 4  5  6]   [ 7  8  9]]   [[11 12 13]   [14 15 16]   [17 18 19]]   [[21 22 23]   [24 25 26]   [27 28 29]]]
Hold up. What on earth is that datatype? At this point, I was baffled yet curious to find answers.
With some swift googling skills, by no coincidence, I came across the reputable Mr. Jason Brownlee’s blog — MachineLearningMastery — and here’s what he had to say-
“A tensor is a generalization of vectors and matrices and is easily understood as a multidimensional array. A vector is a one-dimensional or first-order tensor and a matrix is a two-dimensional or second-order tensor.” — Introduction To Tensors For Machine Learning
This still didn’t quite tell me why I’d rather use a tensor over an array. Fortunately, I came across a post by Apoorv Yadav — Do NumPy arrays Differ From Tensors — where he performed the test we are going to perform below and gave two declarative statements:
A tensor is a more suitable choice if you’re going to be using GPU’s as it can reside in accelerators memory.Tensors are immutable
A tensor is a more suitable choice if you’re going to be using GPU’s as it can reside in accelerators memory.
Tensors are immutable
Literally, all I needed to know.
To reproduce the experiment done by Apoorv, we first must create a varied shape from [500, 500] to [10000, 10000] with step 100 and then let the magic begin (code provided below) ...
After running the code, here’s the output in a visual format...
As you can see the tensor stored in the GPU is much faster than its counterparts.
“Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other specialized hardware to accelerate computing.” — Pytorch Documentation
In addition to this, I also found a nice StackExchange discussion that went into more depth than about the differences between tensors and multi-dimensional arrays on a more technical aspect. I guess the general gist is that if you’re unsure about something then and it’s bugging you then you’re better off following your curiosity — it is a major sigh of relief now I know there’s a difference between arrays and tensors.
I hope you found this useful. Connect with me on LinkedIn or Twitter.