A NumPy affair: Broadcasting. This article explores the concept of... | by Richmond Alake | Towards Data Science

This article explores the concept of Broadcasting that is utilized in the scientific computing package that is NumPy.
We will introduce the concept of Broadcasting by giving a simple definition, along with some examples to illustrate how Broadcasting is implemented in Python by leveraging NumPy arrays. A further detailed explanation will be provided along with a more in-depth definition of what Broadcasting is, its rules, benefits, and limitations.
Broadcasting is the methodology adopted in NumPy used to perform arithmetic operations on arrays with differing dimensions. General arithmetic operations such as addition, multiplication, subtraction, etc. tend to broadcast arrays before performing the operations on arrays with variations in size.
To provide a context of reasoning for the definition above, the code below depicts how an arithmetic operation would work without broadcasting example #1 and the same arithmetic operation is shown in examples #2, but this time with broadcasting.
Example #1 — A world without broadcasting
import numpy as npa = np.array([1,2,3], dtype=float)b = 5c = np.empty([3])for idx, i in enumerate(a):    c[idx] = i * bprint(c)>> [5. 10. 15.]
In example #1 we have a NumPy array containing three elements: 1, 2, and 3; this array is assigned to the variable ‘a’. Variable ‘b’ is assigned to the scalar 5, and ‘c’ is initialized as an empty array that has 3 elements. The shape of ‘c’ is the predetermined expected shape of the result of an arithmetic operation between ‘a’ and ‘b’.
The arithmetic operation performed is a multiplication. In this operation, the scalar ‘b’ is multiplied by each element in the array ‘a’. In more technical terms, we are performing an element-wise operation between the values held in the variables ‘a’ and ‘b’.
To perform an element-wise operation, we iterate through the array ‘a’ and multiply each element with the scalar held in ‘b’. The operation result is placed in the variable ‘c’ at a corresponding index position of the element within ‘a’. So a multiplication of the element in index 0 in ‘a’ (a[0] = 2) and the scalar held in ‘b’ will be placed at index 1 in ‘c’ (c[0] = 1* 5 = 5).
Example #2 — A world with Broadcasting
import numpy as npa = np.array([1,2,3], dtype=float)b = 5c = a * bprint(c)>> [5. 10. 15.]
In example #2 we can see that we have less written code, and also we do not require a loop to achieve the same operation result as example #1. This is due to the utilization of broadcasting to perform the arithmetic operation.
The diagram above depicts visually what is occurring in example #2. An arithmetic operation between an array and a scalar is the simplest method of explaining how broadcasting works. In example #2, the scalar value 5 assigned to variable ‘b’ has been ‘broadcast’ to all the elements in the array in variable ‘a’. In layman terms, the value 5 has been duplicated several times to match the dimensions of the shape of the array ‘a’, to enable suitable conditions for the execution of the operations.
Redefining broadcasting more purposefully and intuitively, we can come up with a description that is as follows: Broadcasting is a method used in python that provides a wrapper for array operations that enables the optimization of loop-based operations through vectorization.
‘Wrapper’ refers to the fact that the operation carried out through broadcasting is one that is abstracted through python, as the actual executing occurs in a lower-level language, namely C.
‘Vectorization’ refers to an operation that is performed over several elements without any modification to the operation. This operation could be a multiplication arithmetic operation shown in example #2.
Earlier on, we stated that the scalar value in example #2 is ‘duplicated’ to match the dimensions of the array in variable ‘a’ for the objective of both data having the same dimensions to enable suitable element-wise operation. This is not entirely correct.
What’s happening is not duplication, but instead, the iterator over the extended dimension does not actually move, or rather iterate. To give a more simplistic description of what is occurring, the scalar ‘5’ or any data value, be it an array, that is extended to meet the dimensions of another array, has an iterator that simply does not move during the broadcasting operation. This occurs because the stride of the iterator of the scalar 5 in variable ‘b’ is set to 0 in NumPy core. The following images below show the time sequence of the iteration operation between the data values held in variable ‘a’ and ‘b’, and also, the operation result at each time step is shown.
Now we understand at some level how broadcasting works in NumPy. To further hone in on what broadcasting is and how it is utilized, we will explore rules and conditions that enable suitable broadcasting, along with practical examples of how broadcasting can be leveraged.
Broadcasting rules and conditions:
For a set of arrays to be deemed ‘broadcastable’, there are a set of conditions that are met.
The shape of the arrays are the same or the ending dimensions for both arrays matchIf an array has a different number of dimension to the another array, then the array with the lesser dimension is extended by 1 until the number of dimension of both arrays are equalIf two arrays are to be broadcastable, then the array with dimension size of 1 is to behave in a manner that enables the array with lesser dimension to be expanded by duplication to the length of the second array.
The shape of the arrays are the same or the ending dimensions for both arrays match
If an array has a different number of dimension to the another array, then the array with the lesser dimension is extended by 1 until the number of dimension of both arrays are equal
If two arrays are to be broadcastable, then the array with dimension size of 1 is to behave in a manner that enables the array with lesser dimension to be expanded by duplication to the length of the second array.
Condition #1
# Condition 1: The shape of the arrays are the same or the ending dimensions for both arrays matchx = np.array([3,2,1])y = np.array([6,5,2])z = x * yprint(x.shape)>> (3,)print(y.shape)>> (3,)print(x.shape == y.shape)>> Trueprint(z)>> [18 10 2]
Condition #2
x = np.random.randn(4,3)y = np.arange(4).reshape((4,1))print(x.shape)>> (4,3)print(y.shape)>> (4,1)z = x * yprint(z)>>[[-0.         -0.          0.        ] [ 2.2984589   0.27385878 -1.17348763] [-1.96979462 -4.9748125   0.65956746] [-1.24697399  0.80710713  1.61002339]]
Condition #3
x = np.random.randn(5,4)y = np.random.randn(1,4)# The array 'x' has a length of 5 and the array 'y' has length of 1, by following condition 3, the array 'y' has been stretched 5 times to match the array 'x' during broadcasting.print(len(x))>> 5print(len(y))>> 1print(x + y)>>[[-0.99329397 -0.98391026  0.85963453  0.28137122] [-2.17210589  0.99278479  1.98294275  1.11116366] [ 0.92228273 -0.39603659  1.99637842  2.31311734] [-1.29068518  0.22292541  1.56178367  2.07585643] [ 2.42435639 -0.07977165  0.28020364  1.42663066]]
Lastly, broadcasting provides some benefits in terms of efficiency as the looping that occurs in broadcasting happens in C as opposed to Python; this means that instructions are not interpreted before execution in the CPU, thus increasing efficiency. Also, as shown in the diagram earlier that illustrates how broadcasting works, we can observe the memory efficiency provided by broadcasting as we do not make needless copies of data.
Broadcasting can be useful in practice, especially in data analysis. An example is when tasked with demeaning data, we can leverage broadcasting. Demeaning data is making the mean of a group of dataset zero, by subtracting each element of the dataset by the mean of the grouped of the dataset. The code snippet provides how this is implemented.
# Broadcasting can be used for demeaning data.# Demeaning is the process of making the mean of a group of data zero by subtracting each element of the data by the mean of the grouped data.arr = np.array([[1,2,3],                [4,5,6],                [7,8,9],                [10,11,12]])# Take the mean of each column within the arrayarr_mean = arr.mean(0)# Now we use broadcasting to perform element wise subtraction between the array and the meandemeaned_data = arr - arr_mean# To prove that array has been demeaned, we should get a mean of zero for each column when computing the meandemeaned_data.mean(0)>> array([0., 0., 0.])
Feel free to learn more about broadcasting from the links provided below.