A hitchhiker guide to python NumPy Arrays | by Deepak K Gupta (DAKSH) | Towards Data Science

If you’re using python for data science, either you have used NumPy or must have heard about it. Most of the statistical analysis which needs data to be stored in memory uses Numpy.
Questions may be asked as why NumPy? Isn’t python lists or other data structures can do the same things?
Well Yes and No, there is nothing in NumPy which can’t be done via python lists or by using other data structures, however, NumPy provides an efficient storage and better way handling of the data for mathematical operations using simple API’s, the benefit of which far out-weights writing your own.
NumPy provides efficient storage and better ways of handling data for Mathematical Operations
NumPy is meant for creating homogeneous n-dimensional arrays (n = 1..n). Unlike Python lists, all elements of a NumPy array should be of same type. so the following code is not valid if data type is provided
numpy_arr = np.array([1,2,"Hello",3,"World"], dtype=np.int32)  # Error
However, for python lists, this is a valid code
py_arr = [1,2,"Hello",3,"World"]  # Valid Code
NumPy arrays are made to be created as homogeneous arrays, considering the mathematical operations that can be performed on them. It would not be possible with heterogeneous data sets.
Let’s see what additional benefits NumPy provides us and how it eases our programming life, especially the ones dealing with mathematical calculations.
The NumPy arrays takes significantly less amount of memory as compared to python lists. It also provides a mechanism of specifying the data types of the contents, which allows further optimisation of the code.
As an example, we can create a simple array of six elements using a python list as well as by using numpy.array(...) , The difference in amount of memory occupied by it is quite astounding. See the example below
If this difference seems intimidating then prepare to have more. It’s possible to do further optimisation on the NumPy arrays if we know the maximum number of individual data we’re planning to have.
In the example above, NumPy by default considers these integers as 8 Bytes integers, however, we can provide data types with NumPy arrays if we know the maximum range of the data. For example, we can use 1 Byte integer for storing numbers upto 255 and 2 Bytes integer for numbers upto 65535
The optimized memory allocation is not limited to storing numbers, it also expands to storing strings. However, since NumPy arrays are mostly used for mathematical calculations, string will be stored and used rarely. Nevertheless, it’s good to know if the requirement arises
Similarly, NumPy arrays provide considerable savings in storing floating point values.
Now we know that memory benefits of NumPy, lets now move on to understand how it allows us to create multi dimension arrays
An n-dimension array is generally used for creating a matrix or tensors, again mainly for the mathematical calculation purpose. Compare to python list base n-dimension arrays, NumPy not only saves the memory usage, it provide a significant number of additional benefits which makes it easy to mathematical calculations
Here is a list of things we can do with NumPy n-dimensional arrays which is otherwise difficult to do.
The dimensions of the array can be changed at runtime as long as the multiplicity factor produces the same number of elements. For example, a 2 * 5 matrix can be converted into 5 * 2 and a 1 * 4 into 2 * 2. This can be done by calling the NumPy .reshape(...) function on the arrays
NumPy can also generate a predefined set of number for an array. A predefined set of numbers with steps (including decimal steps can be generated using numpy.arange(...) function. The output of this function will always be a single dimension set of numbers. However, we can use reshape on this output to generate dimension of our choice.
NumPy provides the API for creating n-dimension arrays using pre-filled ones and zeros where all members of the matrix are either zero or one. One of the most probable usage of this is to create a Sparse or Dense matrix for machine learning .
By default, NumPy uses floating point float64 data type for creating ones and zeros, however, the data type can be changed to integer using dtype option as depicted below
As .reshape(x,y) can convert an array into multi dimensional array, similarly, its possible to create a single dimensional array from any any multi dimensional array using the function .ravel()
As stated earlier, NumPy is not only about efficient storing the data, it also makes it extremely easy to perform mathematical operations on it. Any actions on n-dimension arrays behaves exactly similar to mathematical operations.
NumPy n-dimensional arrays makes it extremely easy to perform mathematical operations on it
This is the main USP of NumPy because of which it’s widely used in data analytic community. The python lists are nowhere near to what it can do. Let for example, consider multiplying a python list by 2. Here is what we’ll get
py_arr = [1,2,3] * 2# Generates [1,2,3,1,2,3]
Whereas a numpy array produces the output, which each array elements are multiplied by 2
np_arr = np.array([1,2,3]) * 2# Generates [2,4,6]
Similarly, we can also do other mathematical operations on numpy arrays like addition, subtractions and divisions
np_arr = np.array([1,2,3]) + 2# Generates [3,4,5]
A Multi dimensional matrix operations are as easy as we do using two numbers
np_arr1 = np.array([[1,2,3],[4,5,6]])np_arr2 = np.array([[1,2,3],[4,5,6]])np_arr3 = np_arr1 * np_arr2#Generates# [[ 1  4  9]# [16 25 36]]np_arr3 = np_arr1 + np_arr2#Generates# [[ 2  4  6]# [ 8 10 12]]
NumPy built-in mathematical functions also allow us to perform complex mathematical operations like sqrt, mean and median.
np.sqrt(np_arr1)#generates# [1.         1.41421356 1.73205081]# [2.         2.23606798 2.44948974]]np.mean(np_arr1) #generates 3.5np.median(np_arr1) #generates 3.5
There are additional in-built members functions to get more details of the arrays, including sum, min and max
np_arr1.sum()  # 21np_arr1.min()  # 1np_arr1.max()  # 6
While working with data sets there will be times when we need to find specific data from the available data set. Though NumPy provides multiple functions for the same, three of them will be used more often than others. They are where, nonzeroand count_nonzero
Where and nonzero functions returns the index associated with the True statement. for example
np_arr = np.array([1,2,0,4,5])find = np.where(np_arr > 2)#returns [3,4]
and
np_arr = np.array([1,2,0,4,5])find = np.nonzero(np_arr)# return [0,1,3,4]
And finally one can count the number of non zero elements in a numpy array by using count_nonzero(...) function
n_arr = np.array([1,2,3,0,3,0,2,0,0,2])np.count_nonzero(n_arr) # returns 6
These methods are very useful in cases like calculating the sparsity or the density of a matrix.
The final method which has its own usage in machine learning is to find the shape of the NumPy array. This is done using .shape . For example
n_arr = np.array([1,2,3,0,3,0,2,0,0,2])n_arr.shape# Generates => (10,)n_arr = np.array([[1,2],[3,0]])# Generates => (2,2)
The examples provided above depicts only a minimal set of NumPy functionalities. Though not comprehensive, but should give enough feel about what is NumPy and why should we be learning and using it.
That’s all for this particular post. Will come up with another set of interesting NumPy functionalities in another post
Thanks for Reading ...!!!