Python for data analysis- NumPy (Based on udemy)
Published: 2022-09-19 16:11:56

**What is Numpy?**
-NumPy (or Numpy) is a Linear Algebra Library for Python, the reason it is so important for Data Science with Python is that almost all of the libraries in the PyData Ecosystem rely on NumPy as one of their main building blocks.
Using NumPy
Once you've installed NumPy you can import it as a library:

import numpy as np
Numpy has many built-in functions and capabilities. We won't cover them all but instead we will focus on some of the most important aspects of Numpy: vectors,arrays,matrices, and number generation. Let's start by discussing arrays.

**1.Numpy Arrays**
NumPy arrays are the main way we will use Numpy throughout the course. Numpy arrays essentially come in two flavors: vectors and matrices. Vectors are strictly 1-d arrays and matrices are 2-d (but you should note a matrix can still have only one row or one column).

Let's begin our introduction by exploring how to create NumPy arrays.

**Creating NumPy Arrays**
From a Python List
We can create an array by directly converting a list or list of lists:

my_list = [1,2,3]
my_list
[1, 2, 3]
np.array(my_list)
array([1, 2, 3])
my_matrix = [[1,2,3],[4,5,6],[7,8,9]]
my_matrix
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
np.array(my_matrix)
array([[1, 2, 3],
       [4, 5, 6],
       [7, 8, 9]])
Built-in Methods
There are lots of built-in ways to generate Arrays

**arange**
Return evenly spaced values within a given interval.

np.arange(0,10)
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
np.arange(0,11,2)
array([ 0,  2,  4,  6,  8, 10])
**zeros and ones**
Generate arrays of zeros or ones

np.zeros(3)
array([ 0.,  0.,  0.])
np.zeros((5,5))
array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])
np.ones(3)
array([ 1.,  1.,  1.])
np.ones((3,3))
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.],
       [ 1.,  1.,  1.]])
**linspace**
Return evenly spaced numbers over a specified interval.

np.linspace(0,10,3)
array([  0.,   5.,  10.])
np.linspace(0,10,50)
array([  0.        ,   0.20408163,   0.40816327,   0.6122449 ,
         0.81632653,   1.02040816,   1.2244898 ,   1.42857143,
         1.63265306,   1.83673469,   2.04081633,   2.24489796,
         2.44897959,   2.65306122,   2.85714286,   3.06122449,
         3.26530612,   3.46938776,   3.67346939,   3.87755102,
         4.08163265,   4.28571429,   4.48979592,   4.69387755,
         4.89795918,   5.10204082,   5.30612245,   5.51020408,
         5.71428571,   5.91836735,   6.12244898,   6.32653061,
         6.53061224,   6.73469388,   6.93877551,   7.14285714,
         7.34693878,   7.55102041,   7.75510204,   7.95918367,
         8.16326531,   8.36734694,   8.57142857,   8.7755102 ,
         8.97959184,   9.18367347,   9.3877551 ,   9.59183673,
         9.79591837,  10.        ])
**eye**
Creates an identity matrix

np.eye(4)
array([[ 1.,  0.,  0.,  0.],
       [ 0.,  1.,  0.,  0.],
       [ 0.,  0.,  1.,  0.],
       [ 0.,  0.,  0.,  1.]])
**Random**
**Numpy also has lots of ways to create random number arrays:**

**rand**
Create an array of the given shape and populate it with random samples from a uniform distribution over [0, 1).

np.random.rand(2)
array([ 0.11570539,  0.35279769])
np.random.rand(5,5)
array([[ 0.66660768,  0.87589888,  0.12421056,  0.65074126,  0.60260888],
       [ 0.70027668,  0.85572434,  0.8464595 ,  0.2735416 ,  0.10955384],
       [ 0.0670566 ,  0.83267738,  0.9082729 ,  0.58249129,  0.12305748],
       [ 0.27948423,  0.66422017,  0.95639833,  0.34238788,  0.9578872 ],
       [ 0.72155386,  0.3035422 ,  0.85249683,  0.30414307,  0.79718816]])
**randn**
Return a sample (or samples) from the "standard normal" distribution. Unlike rand which is uniform:

np.random.randn(2)
array([-0.27954018,  0.90078368])
np.random.randn(5,5)
array([[ 0.70154515,  0.22441999,  1.33563186,  0.82872577, -0.28247509],
       [ 0.64489788,  0.61815094, -0.81693168, -0.30102424, -0.29030574],
       [ 0.8695976 ,  0.413755  ,  2.20047208,  0.17955692, -0.82159344],
       [ 0.59264235,  1.29869894, -1.18870241,  0.11590888, -0.09181687],
       [-0.96924265, -1.62888685, -2.05787102, -0.29705576,  0.68915542]])
**randint**
Return random integers from low (inclusive) to high (exclusive).

np.random.randint(1,100)
44
np.random.randint(1,100,10)
array([13, 64, 27, 63, 46, 68, 92, 10, 58, 24])
Array Attributes and Methods
Let's discuss some useful attributes and methods or an array:

arr = np.arange(25)
ranarr = np.random.randint(0,50,10)
arr
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24])
**ranarr**
array([10, 12, 41, 17, 49,  2, 46,  3, 19, 39])
**Reshape**
Returns an array containing the same data with a new shape.

arr.reshape(5,5)
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])
**max,min,argmax,argmin**
These are useful methods for finding max or min values. Or to find their index locations using argmin or argmax

**ranarr**
array([10, 12, 41, 17, 49,  2, 46,  3, 19, 39])
ranarr.max()
49
ranarr.argmax()
4
ranarr.min()
2
ranarr.argmin()
5
**Shape**
Shape is an attribute that arrays have (not a method):

**# Vector**
arr.shape
(25,)
# Notice the two sets of brackets
arr.reshape(1,25)
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
        17, 18, 19, 20, 21, 22, 23, 24]])
arr.reshape(1,25).shape
(1, 25)
arr.reshape(25,1)
array([[ 0],
       [ 1],
       [ 2],
       [ 3],
       [ 4],
       [ 5],
       [ 6],
       [ 7],
       [ 8],
       [ 9],
       [10],
       [11],
       [12],
       [13],
       [14],
       [15],
       [16],
       [17],
       [18],
       [19],
       [20],
       [21],
       [22],
       [23],
       [24]])
arr.reshape(25,1).shape
(25, 1)
**dtype**
You can also grab the data type of the object in the array:

arr.dtype
dtype('int64')

**2. NumPy array indexing and selection**
NumPy Indexing and Selection
In this lecture we will discuss how to select elements or groups of elements from an array.

import numpy as np
#Creating sample array
arr = np.arange(0,11)
#Show
arr
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
Bracket Indexing and Selection
The simplest way to pick one or some elements of an array looks very similar to python lists:

**#Get a value at an index**
arr[8]
8
#Get values in a range
arr[1:5]
array([1, 2, 3, 4])
#Get values in a range
arr[0:5]
array([0, 1, 2, 3, 4])
Broadcasting
_Numpy arrays differ from a normal Python list because of their ability to broadcast:_

#Setting a value with index range (Broadcasting)
arr[0:5]=100
​
#Show
arr
array([100, 100, 100, 100, 100,   5,   6,   7,   8,   9,  10])
# Reset array, we'll see why I had to reset in  a moment
arr = np.arange(0,11)
​
#Show
arr
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
#Important notes on Slices
slice_of_arr = arr[0:6]
​
#Show slice
slice_of_arr
array([0, 1, 2, 3, 4, 5])
#Change Slice
slice_of_arr[:]=99
​
#Show Slice again
slice_of_arr
array([99, 99, 99, 99, 99, 99])
Now note the changes also occur in our original array!

arr
array([99, 99, 99, 99, 99, 99,  6,  7,  8,  9, 10])
Data is not copied, it's a view of the original array! This avoids memory problems!

_#To get a copy, need to be explicit_
arr_copy = arr.copy()
​
arr_copy
array([99, 99, 99, 99, 99, 99,  6,  7,  8,  9, 10])
Indexing a 2D array (matrices)
The general format is arr_2d[row][col] or arr_2d[row,col]. I recommend usually using the comma notation for clarity.

arr_2d = np.array(([5,10,15],[20,25,30],[35,40,45]))
​
#Show
arr_2d
array([[ 5, 10, 15],
       [20, 25, 30],
       [35, 40, 45]])
#Indexing row
arr_2d[1]
​
array([20, 25, 30])
# Format is arr_2d[row][col] or arr_2d[row,col]
​
# Getting individual element value
arr_2d[1][0]
20
# Getting individual element value
arr_2d[1,0]
20
# 2D array slicing
​
#Shape (2,2) from top right corner
arr_2d[:2,1:]
array([[10, 15],
       [25, 30]])
#Shape bottom row
arr_2d[2]
array([35, 40, 45])
#Shape bottom row
arr_2d[2,:]
array([35, 40, 45])
_Fancy Indexing
Fancy indexing allows you to select entire rows or columns out of order,to show this, let's quickly build out a numpy array:
_
#Set up matrix
arr2d = np.zeros((10,10))
#Length of array
arr_length = arr2d.shape[1]
#Set up array
​
for i in range(arr_length):
    arr2d[i] = i
    
arr2d
array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.],
       [ 2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.],
       [ 3.,  3.,  3.,  3.,  3.,  3.,  3.,  3.,  3.,  3.],
       [ 4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.],
       [ 5.,  5.,  5.,  5.,  5.,  5.,  5.,  5.,  5.,  5.],
       [ 6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.],
       [ 7.,  7.,  7.,  7.,  7.,  7.,  7.,  7.,  7.,  7.],
       [ 8.,  8.,  8.,  8.,  8.,  8.,  8.,  8.,  8.,  8.],
       [ 9.,  9.,  9.,  9.,  9.,  9.,  9.,  9.,  9.,  9.]])
Fancy indexing allows the following

arr2d[[2,4,6,8]]
array([[ 2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.],
       [ 4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.],
       [ 6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.],
       [ 8.,  8.,  8.,  8.,  8.,  8.,  8.,  8.,  8.,  8.]])
#Allows in any order
arr2d[[6,4,2,7]]
array([[ 6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.,  6.],
       [ 4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.,  4.],
       [ 2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.,  2.],
       [ 7.,  7.,  7.,  7.,  7.,  7.,  7.,  7.,  7.,  7.]])
More Indexing Help
_Indexing a 2d matrix can be a bit confusing at first, especially when you start to add in step size. Try google image searching NumPy indexing to fins useful images, like this one:
_

_**Selection**
Let's briefly go over how to use brackets for selection based off of comparison operators.
_
arr = np.arange(1,11)
arr
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
arr > 4
array([False, False, False, False,  True,  True,  True,  True,  True,  True], dtype=bool)
bool_arr = arr>4
bool_arr
array([False, False, False, False,  True,  True,  True,  True,  True,  True], dtype=bool)
arr[bool_arr]
array([ 5,  6,  7,  8,  9, 10])
arr[arr>2]
array([ 3,  4,  5,  6,  7,  8,  9, 10])
x = 2
arr[arr>x]
array([ 3,  4,  5,  6,  7,  8,  9, 10])

**3.NumPy Operations**
Arithmetic
You can easily perform array with array arithmetic, or scalar with array arithmetic. Let's see some examples:

import numpy as np
arr = np.arange(0,10)
arr + arr
array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18])
arr * arr
array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])
arr - arr
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
# Warning on division by zero, but not an error!
# Just replaced with nan
arr/arr
/Users/marci/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:1: RuntimeWarning: invalid value encountered in true_divide
  if __name__ == '__main__':
array([ nan,   1.,   1.,   1.,   1.,   1.,   1.,   1.,   1.,   1.])
# Also warning, but not an error instead infinity
1/arr
/Users/marci/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:1: RuntimeWarning: divide by zero encountered in true_divide
  if __name__ == '__main__':
array([        inf,  1.        ,  0.5       ,  0.33333333,  0.25      ,
        0.2       ,  0.16666667,  0.14285714,  0.125     ,  0.11111111])
arr**3
array([  0,   1,   8,  27,  64, 125, 216, 343, 512, 729])
Universal Array Functions
Numpy comes with many universal array functions, which are essentially just mathematical operations you can use to perform the operation across the array. Let's show some common ones:

#Taking Square Roots
np.sqrt(arr)
array([ 0.        ,  1.        ,  1.41421356,  1.73205081,  2.        ,
        2.23606798,  2.44948974,  2.64575131,  2.82842712,  3.        ])
#Calcualting exponential (e^)
np.exp(arr)
array([  1.00000000e+00,   2.71828183e+00,   7.38905610e+00,
         2.00855369e+01,   5.45981500e+01,   1.48413159e+02,
         4.03428793e+02,   1.09663316e+03,   2.98095799e+03,
         8.10308393e+03])
np.max(arr) #same as arr.max()
9
np.sin(arr)
array([ 0.        ,  0.84147098,  0.90929743,  0.14112001, -0.7568025 ,
       -0.95892427, -0.2794155 ,  0.6569866 ,  0.98935825,  0.41211849])
np.log(arr)
/Users/marci/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:1: RuntimeWarning: divide by zero encountered in log
  if __name__ == '__main__':
array([       -inf,  0.        ,  0.69314718,  1.09861229,  1.38629436,
        1.60943791,  1.79175947,  1.94591015,  2.07944154,  2.19722458])