Stock value prediction with time series analysis
By: Ben Ward
Tags: Python, Data Science, Time Series Analysis

Stock market data is inherently unpredictable, there are too many different external factors influencing the pricing of so many different kinds of companies stock. Especially with such a small time frame of the market’s trends I thought it would be interesting to see how much variance a simple time series model would be able to pickup. The majority of stocks values go up gradually over time, whether it is a few months or years, so I felt like overall the trend of the aggregated data would be fairly positive. I was less sure, however, of the models ability to pickup on the sudden drops and spikes in the market so I set off to find out.

Reading the data in, this is what the data looked like:

Head of data set

#Seeing how long each count should be

len(stocks_df)

length of data set

stocks_df.describe()

statistical description of data set

As you can see, there were very few missing values and because stock data generally flows together over time I felt comfortable interpolating the few missing values.

stocks_df = stocks_df.interpolate()

The first thing I wanted to do with this luckily clean data set is create a new metric of the average value the stock had during the day by averaging high and low prices in each row. I did that simply as follows:

stocks_df['average_price'] = ((stocks_df['high']+stocks_df['low'])/2)

Next, I needed to create the metrics that would help me carry out my time series analysis: days and months since the beginning of the data availability.

stocks_df['days_since_2013_02_08'] = (stocks_df['date'] - pd.Timestamp('2013-02-08'))/pd.Timedelta(1,'D') stocks_df['months_since_2013_02'] = (stocks_df['date'] - pd.Timestamp('2013-02-08'))/pd.Timedelta(1,'M')

I also wanted to create a column which indicated the month of each pricing for any seasonal analysis.

stocks_df['month'] = pd.DatetimeIndex(stocks_df['date']).month

Finally, I needed to group the dataset by date to be able to get an overview of all stock prices at once.

stocks_date_grouped = stocks_df.groupby('date').mean()

I first plotted both the opening and closing prices as well as high and low values during the day to see if there were obvious disparities on this time scale.

open and closing prices

high and low values averaged

There was nothing that could be justified as significant between either one of these plots, though there was a noticeable split between the high and low prices for each day. This is something I was very curious to see if the model could pick up.

Another way I tried to see variance in the data was by creating a rolling mean and comparing that to the average price. That went as follows:

stocks_date_grouped['rolling_mean_average'] = stocks_date_grouped['average_price'].rolling(12).mean()

Then I visualized this new metric:

rolling mean vs average price

There is certainly some disparity between the rolling mean and true value. Something promising for the validity of the prediction. Before that, I wanted to get a picture of the monthly trends and influence on the value of the stock. This was done by creating a simple linear model.

sm0 = smf.ols('average_price ~ months_since_2013_02 + C(month)',data=stocks_date_grouped).fit()



sm0.summary()

linear model output

This output was quite interesting, the coefficients for each month are almost all negative. From what we saw in the previous trends this does not add up as there has been a positive trend over time with price. This may be the first glimpse into the overall unpredictable nature of the data.

Let’s finally take a look at what the model shows.

predicting values until 2025

I plotted three separate types of prediction models. The first is a simplified model which was just based off of the average price and months since opening. The second is a detrended model which was the rolling mean vs months since opening. Finally, the model that took in the most variance was the one called fixed effects. This took in the average price vs months opening but also took into account the variance month by month. Those models each look as follows under the hood:

m0 = smf.ols('average_price ~ months_since_opening',data=stocks_date_grouped).fit() m1 = smf.ols('rolling_mean_average ~ months_since_opening',data=stocks_date_grouped).fit() m2 = smf.ols('average_price ~ months_since_opening + C(month)',data=stocks_date_grouped).fit()

This really was not a very effective model which predicted the large dips and peaks that was contained in the real data, but then again it would not be very ‘statistically accurate’ model if it did wildly show large dups and spikes. As stated in the beginning, stock data is inherently random and there are a myriad of factors influencing the pricing every day.

For a future study I would try and gather data from further back in history including both major market crashes in the past century. I suspect it would still produce a model similar to this, though. The only way to truly get even close to an accurate prediction would be to take in thousands of other financial and political factors and use the one showing the highest statistical correlation.

If anything, this brief study validates how complex stock data is and why no one has been able to crack the code of prediction (and why no one probably ever will). Stocks at their root are influenced by human behavior and as long as that is prevalent it will be hard to reach a perfect model of prediction.