How To Get The Row Count Of a Pandas DataFrame | Towards Data Science

In today’s short guide we will discuss a few ways for computing the row count of pandas DataFrames. Additionally, we will showcase how to omit null values when deriving the counts. Finally, we will observe the performance of each of the methods introduced in this article and identify the most efficient way for computing the number of rows.
First, let’s create an example DataFrame that we’ll reference throughout this guide in order to demonstrate a few concepts.
import pandas as pddf = pd.DataFrame({    'colA':[1, 2, None, 4, 5],     'colB': [None, 'b', 'c', 'd', 'e'],    'colC': [True, False, False, True, None],})print(df)   colA  colB   colC0   1.0  None   True1   2.0     b  False2   NaN     c  False3   4.0     d   True4   5.0     e   None
The most simple and clear way to compute the row count of a DataFrame is to use len() built-in method:
>>> len(df)5
Note that you can even pass df.index for slightly improved performance (more on this in the final section of the article):
>>> len(df.index)5
Alternatively, you can even use pandas.DataFrame.shape that returns a tuple representing the dimensionality of the DataFrame. The first element of the tuple corresponds to the number of rows while the second element represents the number of columns.
>>> df.shape[0]5
You can also unpack the result of df.shape and infer the row count as shown below:
>>> n_rows, _ = df.shape>>> n_rows5
The third option you have when it comes to computing row counts in pandas is pandas.DataFrame.count() method that returns the count for non-NA entries.
Let’s assume that we want to count all the rows which have no null values under a certain column. The following should do the trick for us:
>>> df[df.columns[1]].count()4
This method should only be used when you want to ignore null values. If this is not the case then you should use either len() or shape.
Now that we know a few different ways for computing the count of rows in DataFrames, it would be interesting to discuss the performance implications around them. To do so, we are going to create a larger DataFrame than the one we used so far in this guide.
import numpy as npimport pandas as pddf = pd.DataFrame(    np.random.randint(0, 100, size=(10000, 4)),    columns=list('ABCD'))print(df)       A   B   C   D0     61  38   2  391     96  65  20  472     35  56  97   83     71  31  80  254     20  63  99  34   ..  ..  ..  ..9995  96  81   6  439996  43  22  83  479997  62  92  42  269998  11  48  91  859999  55  47  77  66[10000 rows x 4 columns]
In order to evaluate performance, we are going to use timeit which is useful when it comes to timing small bits of Python code.
>>> %timeit len(df)548 ns ± 24.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)>>> %timeit len(df.index)358 ns ± 10.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)>>> %timeit df.shape[0]904 ns ± 48 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)>>> %timeit df[df.columns[1]].count()81.9 μs ± 4.91 μs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
We can see from the results above that the most efficient way for counting rows in pandas is len() method. By providing just the index (len(df.index)) is even faster.
The least efficient way is count() and thus you should only be using this method only if you need to exclude null entries from the counts.
In today’s article we discussed how to compute row counts of pandas DataFrames using
len()
shape
and count()
methods. Note that count() should only be used if you want to omit null rows at certain column(s). Finally, we discussed the difference of each of these methods in terms of performance.
Become a member and read every story on Medium. Your membership fee directly supports me and other writers you read.
You may also like