Getting started with Pandas time-series functionality | by Tom Waterman | Towards Data Science

Pandas has exceptional features for analyzing time-series data, including automatic datetime parsing, advanced filtering capabilities, and several datetime-specific plotting functions.
I find myself using those features almost every day, but it took me a long time to discover them: many of Pandas datetime capabilities are not immediately obvious, and I didn’t use those features when I was starting out.
But using the features is easy, and will help make your analysis faster whenever you’re dealing with time-series data.
To help you get started, here are the 3 time-series features that I use most often.
Pandas can automatically parse columns in a dataset into time-series data, without requiring you to specify any regex patterns.
Suppose you’re analyzing a dataset where the first five rows look like this.
You can see that the column date looks like a time-series, and it makes sense for us to convert the values in that column into the Pandas datetime type.
To instruct Pandas to convert the values, use the parse_dates argument when loading the data.
Note: the parse_dates argument is available in all of Pandas data loading functions, including read_csv.
>>> import pandas as pd>>> df = pd.read_csv('http://bit.ly/30iosS6', parse_dates=['date'])>>> df.info()<class 'pandas.core.frame.DataFrame'>RangeIndex: 360 entries, 0 to 359Data columns (total 2 columns): #   Column  Non-Null Count  Dtype---  ------  --------------  ----- 0   dau     360 non-null    int64 1   date    360 non-null    datetime64[ns]dtypes: datetime64[ns](1), int64(1)memory usage: 5.8 KB
Notice how the Dtype of the column date is datetime64[ns]. You can now use all of Pandas’ special datetime methods on the date column.
>>> df.date.dt.day_name()0      Tuesday1    Wednesday2     Thursday3       Friday4     Saturday...
Pandas provides convenient filtering methods for a DataFrame when its index is a datetime type.
>>> df.set_index('date', inplace=True)>>> df.indexDatetimeIndex(['2019-01-01', ... ,'2019-12-26'])
For example, you can select a specific date by passing a string to the DataFrame’s loc accessor.
>>> df.loc['2019-02-01']dau    554Name: 2019-02-01 00:00:00, dtype: int64
You can also use strings to select ranges of data, ie to slice the DataFrame.
>>> df.loc['2019-02-01':'2019-03-01']            daudate2019-02-01  5542019-02-02  798...2019-02-28  5692019-03-01  599
Notice that the index is inclusive of the last value. We used 2019-03-01 as the end of the range we selected, so the row labelled2019–03–01 is included in the resulting DataFrame.
You can even pass in partial strings to select data for a specific range.
>>> df.loc['2019-03']            daudate2019-03-01  5992019-03-02  724...2019-03-30  7242019-03-31  638
Last but not least, you can easily plot the time-series data with Pandas plot function.
>>> df.plot()
Notice how Pandas has used the DataFrame’s index for the X-axis.
Of course, this chart isn’t very helpful. Let’s use an aggregate view to produce something more readable.
To do that, we can use the resample method of the DataFrame to aggregate the timeseries index by month. Then, we’ll calculate the mean of each month and create a bar plot of the result.
>>> df.resample('M').mean().plot.bar()
For more details on how to quickly generate charts in Pandas, you can read my article “Fast Plotting with Pandas”.
towardsdatascience.com
Thanks for reading! I often write about Data Science and programming tricks on Medium, so follow me to get more articles like this one.