Are Stock Returns Normally Distributed? | by Tony Yiu | Towards Data Science

While painful, the chaos in financial markets recently provides a good opportunity for us to question our assumptions. It’s very common in the investments industry to model the potential range of an investment’s future returns with a normal distribution. Any time we can model something with normal distributions, it makes life a lot easier. For example, the return of a portfolio consisting of many investments (each with normally distributed returns) is also normally distributed. And to describe an investment, we only need 2 values: the mean (a.k.a. the investment’s expected return) and the standard deviation (a.k.a. the investment’s risk). If we want to be thorough, we should also record the investment’s correlation with our overall portfolio.
I wrote previously about how the finance industry models the risk of an investment. The X-axis location of the peak of the bell curve is the expected return and the width of the bell curve proxies its risk:
But do risk estimates made with these assumptions actually make sense? Are stock returns actually normal? And does the assumption of normality understate, properly state, or overstate the frequency of market disasters (like what we experienced over the past few weeks)?
Since 1950, the average annual return of the S&P 500 has been approximately 8% and the standard deviation of that return has been 12%. I want to look at monthly returns so let’s translate these to monthly:
Monthly Expected Return = 8%/12 = 0.66%Monthly Standard Deviation = 12%/(12^0.5) = 3.50%
Let’s overlay the actual returns on top of a theoretical normal distribution with a mean of 0.66% and a standard deviation of 3.5%:
It looks approximately normal but if we look to the left of the distribution, we can see the famous fat tails. The fat tails mean that extreme events occur more frequently in reality than what a normal distribution would predict. More evidence of that is how the actual distribution of monthly S&P 500 returns is skinnier in its center than the normal distribution. The skinny middle and the fat tails imply that the normal distribution might not be the best describer of stock returns. Rather, there seem to be 2 regimes — a calm regime where we spend most of the time that is normally distributed (but with a lower volatility than 12%) and a regime with high volatility and terrible returns.
Another way to check for normality is with a QQ plot (I also wrote a blog detailing how QQ plots work). Let’s check out the QQ plot for monthly S&P 500 returns:
Deviations from the red 45 degree line represent differences from the normal distribution. While most of the observations do fall more or less on the red line, we can see significant deviations on the left tail and smaller ones on the right tail. The value on the X-axis (Theoretical Quantiles) tells us how frequently we expect to see an observation of that magnitude on a normal distribution (they are Z-scores, a.k.a. standard deviations away from the mean, which implies a probability). And the value on the Y-axis (Sample Quantiles, also in Z-scores) tells us how frequently we actually see it.
For example, take the 2 dots on the left that are obvious outliers. Both of those represent S&P 500 returns of worse than -20%. In the dataset, there are 843 monthly observations in total. So the 2 outlier dots represent a mere 0.237% of our observations. We can use the following line of code to find the point on a normal distribution where 0.237% of the observations lie to the left:
In:from scipy.stats import norm# Multiply by 2 to account for probabilities in right tail alsotheoretical_z_score = norm.interval(1-(0.00237*2))[0]print('Theoretical Z = ' + str(round(theoretical_z_score, 2)))Out:Theoretical Z = -2.82
This means that on a normal distribution (with mean=0 and standard deviation=1), we would expect 0.237% of the observations to lie to the left of -2.82 (this value is an example of a Z-score). The Z-score we just calculated is the X-axis position of the second-worst return on the QQ plot.
We can confirm this via the cumulative density function (CDF method), which tells us, for a given distribution, the sum of the probabilities that lie to the left of the Z-score:
In:prob_left = norm.cdf(theoretical_z_score)print('Probability to left = ' + str(round(prob_left, 5)))Out:Probability to left = 0.00237
The Z-score is a metric that connects magnitudes with probabilities. It’s formula is:
Z-score = (observed - mean)/standard_deviation
A Z-score of -2.82 means the observed value was -2.82 standard deviations below the mean (the further it is from the mean in either direction, the less probable the observation). The -2.82 is a theoretical Z-score, a.k.a. the value below which we expect 0.237% of our observations to lie on a normal distribution.
Now let’s calculate the Z-score of our actual data. The 2 outlier dots represent disastrous monthly returns of -20.4% (2008 Financial Crisis) and -22.5% (this past month). So we can use -20.4% to calculate our Z-score (since 2 out of the 842 observations are -20.4% or worse) along with the mean and standard deviation of the S&P 500’s monthly returns:
In:actual_z_score = (-0.204 - 0.0066)/0.035print('Actual Z = ' + str(round(actual_z_score, 2)))Out:Actual Z = -6.02
Wow, a -20% monthly return is a 6 sigma event (6 standard deviations below the mean)! Let’s frame this in terms of probabilities that we can more easily comprehend:
A -2.82 sigma (or worse) event occurs with 0.237% frequency. That means we would expect it to happen once every 1/0.00237 months. So we expect it to happen once every 422 months, or once every 35 years.
A -6.02 sigma (or worse) event occurs with 8.87*10^-8% frequency. Yeah, that number doesn’t make sense to me either so let’s rephrase it. In terms of years, if stock returns were truly normal, then we would expect a 6 sigma event like this one to occur once every 93,884,861 years!
That’s why the QQ plot highlights those 2 points especially (the -6.02 sigma is the Y-axis value of the second worst return on the QQ plot). It’s saying that we are observing 6 sigma events (massively improbably events) in our data at a much higher than expected frequency (approximately 3 sigma frequency). It’s trying to tell us:
It’s saying that we are observing 6 sigma events (massively improbably events) in our data at a much higher than expected frequency (approximately 3 sigma frequency). It’s trying to tell us:
“Hey based on the mean and standard deviation of our data and most critically the assumption that our data is normally distributed, what we are observing here is super duper abnormal!”
And we observed 2 returns worse than -20%! That’s 2 six sigma events (once in 90 million year type events) in a dataset that is only 70 years long. So we can pretty confidently state (no hypothesis test needed!) that NO stock returns are not normal. Moreover, assuming that they are causing us to understate the likelihood of disastrous market returns.
Do we scrap all our models and try to start again from scratch? I don’t think we need to go all the way there. Stock returns are roughly normal after all and a lot of the benefits of investment theory such as diversification hold true even in a world of less than normal stock returns and fat tails (perhaps even more so).
But when we stress test our portfolios (as well as our own mental expectations of what the future might hold), we should definitely be cognizant of the supposed 4, 5, and 6 sigma events that actually seem to occur once every business cycle.
We can also rethink specifically the way we estimate the frequency of disastrous market returns. One issue is that financial markets have just not been around long enough. Therefore we don’t have enough observations to be confident that our estimates of mean, standard deviation, etc. are truly representative of the true distribution. So we should acknowledge the possibility that the inferences we make (using the market data that we do have) will sometimes be woefully incorrect. It reminds me a little of earthquake forecasting where scientists are trying to predict the magnitude and frequency of huge earthquakes (the likes of which might never have been recorded before) using a dataset dominated by quakes of small and medium magnitude. Perhaps the finance industry can borrow a page or 2 from them.
Previous Posts Referenced In This Article:
Understanding Investment Risk
Understanding The Normal Distribution
What In The World Are QQ Plots?