Efficient Frontier Portfolio Optimisation in Python | by Ricky Kim | Towards Data Science

My personal interest in finance has led me to take an online course on investment management in Coursera. It is a 5-course specialisation by the University of Geneva partnered with UBS. It is not specifically for financial modelling, but more for general introduction in investment strategies and the theories surrounding them. As someone who doesn’t have any experience in the industry, the course is really helpful to understand the big picture. I am currently on the 3rd course within the specialisation, and I learned something very interesting called “Modern Portfolio Theory”
While I was going through the course, I thought it would be a very good material to practice my Python skills. Even though the course did not provide any technical details of how to actually implement it, with some digging I found a couple of very useful blog posts I can refer to.
Series of Medium blog post by Bernard Brenyah
Markowitz’s Efficient Frontier in Python [Part 1/2]
Markowitz’s Efficient Frontier in Python [Part 2/2]
Blog post by Bradford Lynch
Investment Portfolio Optimization
Based on what I have learned through the course, and also from the above blog posts, I have tried to replicate it in my own way, tweaking bit and pieces along the way.
*In addition to short code blocks I will attach, you can find the link for the whole Jupyter Notebook at the end of this post.
A good portfolio is more than a long list of good stocks and bonds. It is a balanced whole, providing the investor with protections and opportunities with respect to a wide range of contingencies. – Harry Markowitz
Modern Portfolio Theory (MPT) is an investment theory developed by Harry Markowitz and published under the title “Portfolio Selection” in the Journal of Finance in 1952.
There are a few underlying concepts that can help you understand MPT. If you are familiar with finance, you might know what the acronym “TANSTAAFL” stands for. It is a famous acronym for “There Ain’t No Such Thing As A Free Lunch”. This concept is also closely related to ‘risk-return trade-off’.
Higher risk is associated with greater probability of higher return and lower risk with a greater probability of smaller return. MPT assumes that investors are risk-averse, meaning that given two portfolios that offer the same expected return, investors will prefer the less risky one. Thus, an investor will take on increased risk only if compensated by higher expected returns.
Another factor comes in to play in MPT is “diversification”. Modern portfolio theory says that it is not enough to look at the expected risk and return of one particular stock. By investing in more than one stock, an investor can reap the benefits of diversification — chief among them, a reduction in the riskiness of the portfolio.
What you need to understand is that “risk of a portfolio is not equal to average/weighted-average of individual stocks in the portfolio”. In terms of return, yes it is the average/weighted average of individual stock’s returns, but that’s not the case for risk. The risk is about how volatile the asset is, if you have more than one stock in your portfolio, then you have to take count of how these stocks movement correlates with each other. The beauty of diversification is that you can even get lower risk than a stock with the lowest risk in your portfolio, by optimising the allocation.
I will try to explain as I go along with the actual code. First, let’s start by importing some libraries we need. “Quandl” is a financial platform which also offers Python library. If you haven’t installed it before, of course, you first need to install the package in your command line “pip install quandl”, and before you can use it, you also need to get an API key on Quandl’s website. Sign-up and getting an API key is free but has some limits. As a logged-in free user, you will be able to call 2,000 calls per 10 minutes maximum (speed limit), and 50,000 calls per day (volume limit).
import pandas as pd  import numpy as npimport matplotlib.pyplot as pltimport seaborn as snsimport quandlimport scipy.optimize as scoplt.style.use('fivethirtyeight')np.random.seed(777)%matplotlib inline%config InlineBackend.figure_format = 'retina'
In order to run the below code block, you will need your own API key. The stocks selected for this post is Apple, Amazon, Google, Facebook. Below code block will get daily adjusted closing price of each stock from 01/01/2016 to 31/12/2017 (2 years’ price data).
quandl.ApiConfig.api_key = 'your_api_key_here'stocks = ['AAPL','AMZN','GOOGL','FB']data = quandl.get_table('WIKI/PRICES', ticker = stocks,                        qopts = { 'columns': ['date', 'ticker', 'adj_close'] },                        date = { 'gte': '2016-1-1', 'lte': '2017-12-31' }, paginate=True)data.head()
data.info()
By looking at the info() of data, it seems like the “date” column is already in datetime format. Let’s transform the data a little bit to make it easier to work with.
df = data.set_index('date')table = df.pivot(columns='ticker')# By specifying col[1] in below list comprehension# You can select the stock names under multi-level columntable.columns = [col[1] for col in table.columns]table.head()
It looks much better now. Let’s first look at how the price of each stock has evolved within the given time frame.
plt.figure(figsize=(14, 7))for c in table.columns.values:    plt.plot(table.index, table[c], lw=3, alpha=0.8,label=c)plt.legend(loc='upper left', fontsize=12)plt.ylabel('price in $')
It looks like that Amazon and Google’s stock price is relatively more expensive than those of Facebook and Apple. But since Facebook and Apple are squashed at the bottom, it is hard to see the movement of these two.
Another way to plot this is plotting daily returns (percent change compared to the day before). By plotting daily returns instead of actual prices, we can see the stocks’ volatility.
returns = table.pct_change()plt.figure(figsize=(14, 7))for c in returns.columns.values:    plt.plot(returns.index, returns[c], lw=3, alpha=0.8,label=c)plt.legend(loc='upper right', fontsize=12)plt.ylabel('daily returns')
Amazon has two distinctive positive spikes and a couple of negative ones. Facebook has one highest positive spike. And Apple also has some spikes stand out from the plot. From the above plot, we can roughly see that Amazon looks like a quite risky stock, and Google seems to be the most stable one among them.
We have 4 stocks in our portfolio. One decision we have to make is how we should allocate our budget to each of stock in our portfolio. If our total budget is 1, then we can decide the weights for each stock, so that the sum of weights will be 1. And the value for weights will be the portion of budget we allocate to a specific stock. For example, if weight is 0.5 for Amazon, it means that we allocate 50% of our budget to Amazon.
Let’s define some functions to simulate random weights to each stock in the portfolio, then calculate the portfolio’s overall annualised returns and annualised volatility.
“portfolio_annualised_performance” function will calculate the returns and volatility, and to make it as an annualised calculation I take into account 252 as the number of trading days in one year. “random_portfolios” function will generate portfolios with random weights assigned to each stock, and by giving num_portfolios argument, you can decide how many random portfolios you want to generate.
From the above code block, there are two things I want to point out.
The first is the calculation for portfolio’s volatility in “portfolio_annualised_performance” function. If you look up “portfolio standard deviation formula”, you will come across formulas as below.
This formula can be simplified if we make use of matrix notation. Again Bernard Brenyah, whom I mentioned at the beginning of the post, has provided a clear explanation of how the above formula can be expressed in matrix calculation in one of his blog posts.
With the above matrix calculation, we get the part inside the square root in the original formula. Now, all we need to do is put them inside the square root. Same as the annualised return, I took into account of 252 trading days (in this case, the square root of 252) to calculate the annualised standard deviation of a portfolio.
The second thing I would like to point out is the Sharpe ratio. In order to understand the Sharpe ratio, it is essential to understand the broader concept of risk-adjusted return. Risk-adjusted return refines an investment’s return by measuring how much risk is involved in producing that return, which is generally expressed as a number or rating. There could be a number of different methods of expressing risk-adjusted return, and the Sharpe ratio is one of them.
The Sharpe ratio was derived in 1966 by William Sharpe, another winner of a Nobel Memorial Prize in Economic Sciences.
The ratio describes how much excess return you are receiving for the extra volatility that you endure for holding a riskier asset. The Sharpe ratio can be expressed in below formula.
There are some criticisms on how the Sharpe ratio uses the standard deviation of returns as a denominator, which assumes the normal distribution of the returns. However, more often than not, the returns on financial assets tend to deviate from a normal distribution and may make interpretations of the Sharpe ratio misleading. It is for this reason that there are other methods which adjust or modify the original Sharpe ratio. But it is a more advanced topic, and I will stick to the traditional Sharpe ratio for this post.
Now let’s get the needed argument values for our functions. You can easily get daily returns by calling pct_change on the data frame with the price data. And the mean daily returns, the covariance matrix of returns are needed to calculate portfolio returns and volatility. We will generate 25,000 random portfolios. Finally, the risk-free rate has been taken from U.S. Department of The Treasury. The rate of 1.78% is the 52week treasury bill rates at the start of 2018. The rationale behind this is that the historical price data is from 2016~2017, and if I assume that I implement this analysis at the start of 2018, the most updated Treasury bill rate is from the start of 2018. And I also chose 52weeks treasury bill rates to match with the annualised return and risk I calculated.
returns = table.pct_change()mean_returns = returns.mean()cov_matrix = returns.cov()num_portfolios = 25000risk_free_rate = 0.0178
Let me briefly explain what below function is doing. First, it generates random portfolio and gets the results (portfolio returns, portfolio volatility, portfolio Sharpe ratio) and weights for the corresponding result. Then by locating the one with the highest Sharpe ratio portfolio, it displays maximum Sharpe ratio portfolio as red star sign. And does similar steps for minimum volatility portfolio, and displays it as a green star on the plot. All the randomly generated portfolios will be also plotted with colour map applied to them based on the Sharpe ratio. Bluer, higher the Sharpe ratio.
And for these two optimal portfolios, it will also show how it allocates the budget within the portfolio.
display_simulated_ef_with_random(mean_returns, cov_matrix, num_portfolios, risk_free_rate)
For minimum risk portfolio, we can see that more than half of our budget is allocated to Google. If you take another look at the daily return plot from earlier, you can see that Google is the least volatile stock of four, so allocating a large percentage to Google for minimum risk portfolio makes intuitive sense.
If we are willing to take higher risk for higher return, one that gives us the best risk-adjusted return is the one with maximum Sharpe ratio. In this scenario, we are allocating a significant portion to Amazon and Facebook, which are quite volatile stocks from the previous plot of daily returns. And Google which had more than 50% in the case of minimum risk portfolio, has less than 1% budget allocated to it.
From the plot of the randomly simulated portfolios, we can see it forms a shape of an arch line on the top of clustered blue dots. This line is called efficient frontier. Why is it efficient? Because points along the line will give you the lowest risk for a given target return. All the other dots right to the line will give you higher risk with same returns. If the expected returns are the same, why would you take an extra risk when there’s an option with lower risk?
The way we found the two kinds of optimal portfolio above was by simulating many possible random choices and pick the best ones (either minimum risk or maximum risk-adjusted return). We can also implement this by using Scipy’s optimize function.
If you are an advanced Excel user, you might be familiar with ‘solver’ function in excel. Scipy’s optimize function is doing the similar task when given what to optimize, and what are constraints and bounds.
Below functions are to get the maximum Sharpe ratio portfolio. In Scipy’s optimize function, there’s no ‘maximize’, so as an objective function you need to pass something that should be minimized. That is why the first “neg_sharpe_ratio” is computing the negative Sharpe ratio. Now we can use this as our objective function to minimize. In “max_sharpe_ratio” function, you first define arguments (this should not include the variables you would like to change for optimisation, in this case, “weights”). At first, the construction of constraints was a bit difficult for me to understand, due to the way it is stated.
constraints = ({‘type’: ‘eq’, ‘fun’: lambda x: np.sum(x) — 1})
The above constraint is saying that sum of x should be equal to 1. You can think of the ‘fun’ part construction as ‘1’ on the right side of equal sign has been moved to the left side of the equal sign.
np.sum(x) == 1 has become np.sum(x)-1
And what does this mean? It simply means that the sum of all the weights should be equal to 1. You cannot allocate more than 100% of your budget in total.
“bounds” is giving another limit to assign random weights, by saying any weight should be inclusively between 0 and 1. You cannot give minus budget allocation to a stock or more than 100% allocation to a stock.
We can also define an optimising function for calculating minimum volatility portfolio. This time we really do minimise objective function. What do we want to minimise? We want to minimise volatility by trying different weights. “constraints” and “bounds” are same as the above.
As I already mentioned above we can also draw a line which depicts where the efficient portfolios for a given risk rate should be. This is called “efficient frontier”. Below I define other functions to compute efficient frontier. The first function “efficient_return” is calculating the most efficient portfolio for a given target return, and the second function “efficient_frontier” will take a range of target returns and compute efficient portfolio for each return level.
Let’s try to plot the portfolio choices with maximum Sharpe ratio and minimum volatility also with all the randomly generated portfolios. But this time, we are not picking the optimal ones from the randomly generated portfolios, but we are actually calculating by using Scipy’s ‘minimize’ function. And the below function will also plot the efficient frontier line.
display_calculated_ef_with_random(mean_returns, cov_matrix, num_portfolios, risk_free_rate)
We have almost the same result as what we have simulated by picking from the randomly generated portfolios. The slight difference is that the Scipy’s “optimize” function has not allocated any budget at all for Google on maximum Sharpe ratio portfolio, while one we chose from the randomly generated samples has 0.45% of allocation for Google. There are some differences in the decimal places but more or less same.
Instead of plotting every randomly generated portfolio, we can plot each individual stocks on the plot with the corresponding values of each stock’s annual return and annual risk. This way we can see and compare how diversification is lowering the risk by optimising the allocation.
display_ef_with_selected(mean_returns, cov_matrix, risk_free_rate)
As you can see from the above plot, the stock with the least risk is Google at around 0.18. But with portfolio optimisation, we can achieve even lower risk at 0.16, and still with a higher return than Google. And if we are willing to take slightly more risk at around the similar level of risk of Google, we can achieve a much higher return of 0.30 with portfolio optimisation.
Considering how vast and the deep the finance field is, I’ve probably only scratched the surface. But I had fun going through coding and trying to understand the concept. And I’m learning every day. After finishing this implementation, I definitely know better than yesterday’s me. And if I keep on going and learning, in about a couple of year’s time, I will know a whole lot more than today’s me. If you have any comments or questions, feel free to leave a comment. Any feedback would be appreciated.
Thank you for reading. You can find the Jupyter Notebook from the below link.