Python | Replace NaN values with average of columns - GeeksforGeeks

20 Mar, 2019
In machine learning and data analytics data visualization is one of the most important steps. Cleaning and arranging data is done by different algorithms. Sometimes in data sets, we get NaN (not a number) values which are not possible to use for data visualization.
To solve this problem, one possible method is to replace nan values with an average of columns. Given below are a few methods to solve this problem.
 Method #1: Using np.colmean and np.take
# Python code to demonstrate# to replace nan values# with an average of columns  import numpy as np  # Initialising numpy arrayini_array = np.array([[1.3, 2.5, 3.6, np.nan],                       [2.6, 3.3, np.nan, 5.5],                      [2.1, 3.2, 5.4, 6.5]])  # printing initial arrayprint ("initial array", ini_array)  # column meancol_mean = np.nanmean(ini_array, axis = 0)  # printing column meanprint ("columns mean", str(col_mean))  # find indices where nan value is presentinds = np.where(np.isnan(ini_array))  # replace inds with avg of columnini_array[inds] = np.take(col_mean, inds[1])  # printing final arrayprint ("final array", ini_array)
Output:
initial array [[ 1.3  2.5  3.6  nan]
 [ 2.6  3.3  nan  5.5]
 [ 2.1  3.2  5.4  6.5]]
columns mean [ 2.   3.   4.5  6. ]

final array [[ 1.3  2.5  3.6  6. ]
 [ 2.6  3.3  4.5  5.5]
 [ 2.1  3.2  5.4  6.5]]

Method #2: Using np.ma and np.where
# Python code to demonstrate# to replace nan values# with average of columns  import numpy as np  # Initialising numpy arrayini_array = np.array([[1.3, 2.5, 3.6, np.nan],                      [2.6, 3.3, np.nan, 5.5],                      [2.1, 3.2, 5.4, 6.5]])  # printing initial arrayprint ("initial array", ini_array)  # replace nan with col meansres = np.where(np.isnan(ini_array), np.ma.array(ini_array,               mask = np.isnan(ini_array)).mean(axis = 0), ini_array)     # printing final arrayprint ("final array", res)
Output:
initial array [[ 1.3  2.5  3.6  nan]
 [ 2.6  3.3  nan  5.5]
 [ 2.1  3.2  5.4  6.5]]
final array [[ 1.3  2.5  3.6  6. ]
 [ 2.6  3.3  4.5  5.5]
 [ 2.1  3.2  5.4  6.5]]

Method #3: Using Naive and zip
# Python code to demonstrate# to replace nan values# with average of columns  import numpy as np  # Initialising numpy arrayini_array = np.array([[1.3, 2.5, 3.6, np.nan],                      [2.6, 3.3, np.nan, 5.5],                      [2.1, 3.2, 5.4, 6.5]])  # printing initial arrayprint ("initial array", ini_array)  # indices where values is nan in arrayindices = np.where(np.isnan(ini_array))  # Iterating over numpy array to replace nan with valuesfor row, col in zip(*indices):    ini_array[row, col] = np.mean(ini_array[           ~np.isnan(ini_array[:, col]), col])  # printing final arrayprint ("final array", ini_array)
Output:
initial array [[ 1.3  2.5  3.6  nan]
 [ 2.6  3.3  nan  5.5]
 [ 2.1  3.2  5.4  6.5]]
final array [[ 1.3  2.5  3.6  6. ]
 [ 2.6  3.3  4.5  5.5]
 [ 2.1  3.2  5.4  6.5]]

Python numpy-program
Python-numpy
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Check if element exists in list in Python
How To Convert Python Dictionary To JSON?
Python Classes and Objects
How to drop one or multiple columns in Pandas Dataframe
Defaultdict in Python
Python | Get unique values from a list
Python | os.path.join() method
Create a directory in Python
Python | Pandas dataframe.groupby()