Count NaN or missing values in Pandas DataFrame - GeeksforGeeks

02 Jul, 2020
In this article, we will see how to Count NaN or missing values in Pandas DataFrame using isnull() and sum() method of the DataFrame.
Pandas isnull() function detect missing values in the given object. It return a boolean same-sized object indicating if the values are NA. Missing values gets mapped to True and non-missing value gets mapped to False.
Syntax: DataFrame.isnull()
Parameters: None
Return Type: Dataframe of Boolean values which are True for NaN values otherwise False.
Pandas sum() function return the sum of the values for the requested axis. If the input is index axis then it adds all the values in a column and repeats the same for all the columns and returns a series containing the sum of all the values in each column. It also provides support to skip the missing values while calculating the.
Syntax: DataFrame.sum(axis=None, skipna=None, level=None, numeric_only=None, min_count=0, **kwargs)
Parameters :
axis : {index (0), columns (1)}
skipna : Exclude NA/null values when computing the result.
level : If the axis is a MultiIndex (hierarchical), count along a particular level, collapsing into a Series
numeric_only : Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. Not implemented for Series.
min_count : The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA.
Returns : sum of Series or DataFrame (if level specified).
Let’s create a pandas dataframe.
# import numpy library as npimport numpy as np  # import pandas library as pdimport pandas as pd  # List of Tuplesstudents = [('Ankit', 22, 'Up', 'Geu'),           ('Ankita', np.NaN, 'Delhi', np.NaN),           ('Rahul', 16, 'Tokyo', 'Abes'),           ('Simran', 41, 'Delhi', 'Gehu'),           ('Shaurya', np.NaN, 'Delhi', 'Geu'),           ('Shivangi', 35, 'Mumbai', np.NaN ),           ('Swapnil', 35, np.NaN, 'Geu'),           (np.NaN, 35, 'Uk', 'Geu'),           ('Jeet', 35, 'Guj', 'Gehu'),           (np.NaN, np.NaN, np.NaN, np.NaN)            ]  # Create a DataFrame object from# list of tuples with columns# and indices.details = pd.DataFrame(students, columns =['Name', 'Age',                                            'Place', 'College'],                        index =['a', 'b', 'c', 'd', 'e',                                 'f', 'g', 'i', 'j', 'k'])  details
Output:
Example 1 : Count total NaN at each column in DataFrame.
# import numpy library as npimport numpy as np  # import pandas library as pdimport pandas as pd    # List of Tuplesstudents = [('Ankit', 22, 'Up', 'Geu'),           ('Ankita', np.NaN, 'Delhi', np.NaN),           ('Rahul', 16, 'Tokyo', 'Abes'),           ('Simran', 41, 'Delhi', 'Gehu'),           ('Shaurya', np.NaN, 'Delhi', 'Geu'),           ('Shivangi', 35, 'Mumbai', np.NaN ),           ('Swapnil', 35, np.NaN, 'Geu'),           (np.NaN, 35, 'Uk', 'Geu'),           ('Jeet', 35, 'Guj', 'Gehu'),           (np.NaN, np.NaN, np.NaN, np.NaN)            ]  # Create a DataFrame object from list of tuples # with columns and indices.details = pd.DataFrame(students, columns =['Name', 'Age',                                           'Place', 'College'],                        index =['a', 'b', 'c', 'd', 'e',                                 'f', 'g', 'i', 'j', 'k'])  # show the boolean dataframe            print(" \nshow the boolean Dataframe : \n\n", details.isnull())  # Count total NaN at each column in a DataFrameprint(" \nCount total NaN at each column in a DataFrame : \n\n",      details.isnull().sum())
Output:
Example 2 : Count total NaN at each row in DataFrame .
# import numpy library as npimport numpy as np  # import pandas library as pdimport pandas as pd    # List of Tuplesstudents = [('Ankit', 22, 'Up', 'Geu'),           ('Ankita', np.NaN, 'Delhi', np.NaN),           ('Rahul', 16, 'Tokyo', 'Abes'),           ('Simran', 41, 'Delhi', 'Gehu'),           ('Shaurya', np.NaN, 'Delhi', 'Geu'),           ('Shivangi', 35, 'Mumbai', np.NaN ),           ('Swapnil', 35, np.NaN, 'Geu'),           (np.NaN, 35, 'Uk', 'Geu'),           ('Jeet', 35, 'Guj', 'Gehu'),           (np.NaN, np.NaN, np.NaN, np.NaN)            ]  # Create a DataFrame object from# list of tuples with columns# and indices.details = pd.DataFrame(students, columns =['Name', 'Age',                                            'Place', 'College'],                        index =['a', 'b', 'c', 'd', 'e',                                'f', 'g', 'i', 'j', 'k'])  # show the boolean dataframe            print(" \nshow the boolean Dataframe : \n\n", details.isnull())  # index attribute of a dataframe# gives index list   # Count total NaN at each row in a DataFramefor i in range(len(details.index)) :    print(" Total NaN in row", i + 1, ":",          details.iloc[i].isnull().sum())
Output:
Example 3 : Count total NaN in DataFrame .
# import numpy library as npimport numpy as np  # import pandas library as pdimport pandas as pd    # List of Tuplesstudents = [('Ankit', 22, 'Up', 'Geu'),           ('Ankita', np.NaN, 'Delhi', np.NaN),           ('Rahul', 16, 'Tokyo', 'Abes'),           ('Simran', 41, 'Delhi', 'Gehu'),           ('Shaurya', np.NaN, 'Delhi', 'Geu'),           ('Shivangi', 35, 'Mumbai', np.NaN ),           ('Swapnil', 35, np.NaN, 'Geu'),           (np.NaN, 35, 'Uk', 'Geu'),           ('Jeet', 35, 'Guj', 'Gehu'),           (np.NaN, np.NaN, np.NaN, np.NaN)            ]  # Create a DataFrame object from# list of tuples with columns# and indices.details = pd.DataFrame(students, columns =['Name', 'Age',                                            'Place', 'College'],                        index =['a', 'b', 'c', 'd', 'e',                                'f', 'g', 'i', 'j', 'k'])  # show the boolean dataframe            print(" \nshow the boolean Dataframe : \n\n", details.isnull())  # Count total NaN in a DataFrameprint(" \nCount total NaN in a DataFrame : \n\n",       details.isnull().sum().sum())
Output:
Python pandas-dataFrame
Python-pandas
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Python Dictionary
Read a file line by line in Python
Enumerate() in Python
How to Install PIP on Windows ?
Iterate over a list in Python
Different ways to create Pandas Dataframe
Python String | replace()
Create a Pandas DataFrame from Lists
Python program to convert a list to string
Reading and Writing to text files in Python