How to deal with missing values in a Timeseries in Python?

05 Nov, 2021
In this article, we will discuss how to deal with missing values in a time series using the Python programming language.
Time series is a sequence of observations recorded at regular time intervals. Time series analysis can be useful to see how a given asset, security, or economic variable changes over time. Another big question here is why we need to deal with missing values in the dataset and why the missing values are present in the data?
The handling of missing data is very important during the preprocessing of the dataset as many machine learning algorithms do not support missing values.
Time series are subject to have missing points due to problems in reading or recording the data.
Why can’t we change the missing values with global mean because the time series data might have some like seasonality or trend?  A conventional method such as mean and mode imputation, deletion, and other methods are not good enough to handle missing values as those methods can cause bias to the data. Estimation or imputation of the missing data with the values produced by some procedures or algorithms can be the best possible solution to minimize the bias effect of the conventional method of the data. So that at last, the data will be completed and ready to use for another step of analysis or data mining. 
 The method fills missing values according to sequence and conditions. It means that the method replaces ‘nan’s value with the last observed non-nan value or the next observed non-nan value.
backfill – bfill : according to the last observed valueforwardfill – ffill : according to the next observed value
backfill – bfill : according to the last observed value
forwardfill – ffill : according to the next observed value
Python3
# import the librariesimport pandas as pdimport numpy as np  # dataframe with index as timeseriestime_sdata = pd.date_range("09/10/2021", periods=9, freq="W")  df = pd.DataFrame(index=time_sdata)print(df)  # there are four missing valuesdf["example"] = [10001.0, 10002.0, 10003.0, np.nan,                 10004.0, np.nan, np.nan, 10005.0, np.nan]  gfg1 = df.ffill()print("Using ffill() function:-")print(gfg1)  # here we are doing Backfill Missing Values# in the output the last value has NaN because # there is no backward value for thatgfg2 = df.bfill()print("Using bfill() function:-")print(gfg2)
Output:
The method is more complex than the above fillna() method. It consists of different methodologies, including ‘linear’, ‘quadratic’, ‘nearest’. Interpolation is a powerful method to fill missing values in time-series data. Go through the below link provided for a few more examples. 
Python3
# import the librariesimport pandas as pdimport numpy as np  # dataframe with index as timeseriestime_sdata = pd.date_range("09/10/2021", periods=9, freq="W")  df = pd.DataFrame(index=time_sdata)print(df)  # there are four missing valuesdf["example"] = [10001.0, 10002.0, 10003.0, np.nan,                 10004.0, np.nan, np.nan, 10005.0, np.nan]  # using interpolate() to fill the missing # values in a specific order# dealing with missing valuesdataframe1 = df.interpolate()print(dataframe1)
Output:
This is the maximum number of consecutive NaN values to forward/backward fill. In other words, if there is a gap with more than this number of consecutive NaNs, it will only be partially filled.
Syntax:   
DataFrame.interpolate(method=’linear’, axis=0, limit=None, inplace=False, limit_direction=None, limit_area=None, downcast=None, **kwargs) 
Note: Only method=’linear’ is supported for DataFrame/Series with a MultiIndex.
Python3
# import the librariesimport pandas as pdimport numpy as np  # dataframe with index as timeseriestime_sdata = pd.date_range("09/10/2021", periods=9, freq="W")  df = pd.DataFrame(index=time_sdata)print(df)  # there are four missing valuesdf["example"] = [10001.0, 10002.0, 10003.0, np.nan,                 10004.0, np.nan, np.nan, 10005.0, np.nan]  # Interpolating Missing Values to two valuesdataframe = df.interpolate(limit=2, limit_direction="forward")print(dataframe)
Output:
Picked
Python-pandas
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
How to Install PIP on Windows ?
Python Classes and Objects
Python OOPs Concepts
Introduction To PYTHON
Python | os.path.join() method
How to drop one or multiple columns in Pandas Dataframe
How To Convert Python Dictionary To JSON?
Check if element exists in list in Python
Python | Get unique values from a list
Create a directory in Python