Machine Learning Models For Improved Startup Valuation. | by flo.tausend  | Towards Data Science

Determining the valuation of an early-stage Startup is in most cases very challenging due limited historical data, little to no existing revenues, market uncertainty and many more. Traditional valuation techniques, such as Discounted Cash Flow (DCF) or Multiples (CCA), therefore often lead to inappropriate results. On the other hand, alternative valuation methods remain subject to an individual’s subjective assessment and a black box for others.
Therefore, the underlying study leverages machine learning algorithms to predict a fair, data-driven and comprehensible startup valuations. Three different data sources are merged and applied to three regression models. The regression models’ performance is compared on defined measurements and the final outcome is a continuous numeric value — the predicted startup valuation.
In order to build a machine learning model and get significant results, it is essential to have a solid data base. In this case the final analytical data set is a combination of public and private, anonymised data sources. The study was enabled by partnering with investory.io to get access to anonymised key performance indicators, as well as cap tab le data from early-stage companies. In combination with Crunchbase it is possible to map this data to funding stages, funding rounds, locations and the team size. Last but not least we used social media data, as a previous study (“Social is the new financial”, Jin et al 2017) found a relationship between likelihood of closing an investment round and Twitter activities. At this point a big shoutout to Storyclash for access to their amazing social media analytics platform.
Most of the effort had to be spent merging the different data sources together and cleaning them. While the investory.io data came already in a clean numeric format, Crunchbase data for example was not pulled through their API and therefor came in a very unclean format.
In the very beginning an overview of the different tables and information was generated with a simple database client DBEAVER. After that relevant information was extracted, transformed and loaded mainly with the help of Python programming.
As this generated countless lines of code, we will not go into every detail, but pick some examples as data normalisation below. But just to give you a brief idea: There were companies from more than 50 countries included, with various different currencies and local peculiarities. It is often said that data preparation, cleaning and transformation consumes 80% of a data scientist’s time. This statement also proves itself true in this case.
Normalisation & Scaling
On important step to prepare the dataset for our machine learning algorithms was normalisation. The idea is to bring the numeric values in a common scale. This is necessary as we have headcount and revenue as two features. While revenue may take a numeric value with 6 digits, it is very unlikely that the headcount has values above 100. The features have a very different range and revenue would influence our model more only due to its higher values.
#Data Normalizationfrom sklearn.preprocessing import MinMaxScaler#Select numerical columns which needs to be normalizedtrain_norm = x_traintest_norm = x_test# Normalize Training Data scaler = MinMaxScaler(feature_range=(0, 1))x_train_norm = scaler.fit_transform(train_norm)#Converting numpy array to dataframetraining_norm_col = pd.DataFrame(x_train_norm, index=train_norm.index, columns=train_norm.columns) x_train.update(training_norm_col)print (x_train.head())# Normalize Testing Data by using mean and SD of training setx_test_norm = scaler.transform(test_norm)testing_norm_col = pd.DataFrame(x_test_norm, index=test_norm.index, columns=test_norm.columns) x_test.update(testing_norm_col)print (x_train.head())
The MinMax Scaler is very sensitive to outliers. If your data has some severe outliers, please take care of that beforehand. The range of the normalized data is usually between zero and one as seen in the code above.
Within the project there were three regression models applied to the dataset. They were chosen according to their characteristics and usual purpose of application. Each regression model was evaluated with the help of three indicators: Mean Absolute Error, Root Mean Squared Error and R-Squared.
Linear RegressionA linear regression describes the process of finding a straight line that is as close as possible to the given data points. The model then tries to minimize the squared errors. As it is assumed that the company valuation increases with better performance gradually, the linear regression is applied to the given dataset. Furthermore, as there are multiple numerous features given in the dataset for this project, it is not a simple but a multiple linear regression. As this project is set up with Python, the module statsmodels is used to perform regression and evaluate the predictive results. While there are several classes available, the ordinary least squares method was chosen as it is mostly used for linear relationships in the training data.
# Linear Regressionimport pandas as pdfrom sklearn import linear_modelimport statsmodels.api as sm# with sklearnregr = linear_model.LinearRegression()regr.fit(x_train, y_train)print('Intercept: \n', regr.intercept_)print('Coefficients: \n', regr.coef_)# with statsmodelsx_train = sm.add_constant(x_train) # adding a constant model_OLS = sm.OLS(y_train, x_train).fit()#predictions = model.predict(x_test)  print_model = model_OLS.summary()print(print_model)
The constant (const) in the first line indicates the estimated valuation if all other variables are zero. This constant is calculated at 986.000 €. Furthermore, the Revenue coefficient is with a p-value of 0.003 statistically significant for the model. After removing one coefficient after the other, always the one with the most non-significant value, there were to coefficients clearly driving the model: Revenue and headcount with a p-value of 0.000 and 0.004.
Poly RegressionThe polynomial regression is performed by two steps. First of all, the data has to be transformed into a polynomial. A degree of one would create a simple linear regression. To create a polynomial regression the degree must be higher than one and chosen carefully, because if the value is chosen too high, the model may be overfitted.
The final code for the polynomial regression looks the following:
# Poly Regressionimport operatorimport pandas as pdimport matplotlib.pyplot as pltfrom sklearn.linear_model import LinearRegressionfrom sklearn.metrics import mean_squared_error, r2_score, mean_absolute_errorfrom sklearn.preprocessing import PolynomialFeatures# with sklearnpolynomial_features= PolynomialFeatures(degree=2)x_poly = polynomial_features.fit_transform(x_train)model = LinearRegression()model.fit(x_poly, y_train)y_poly_pred = model.predict(x_poly)MAE = mean_absolute_error(y_train,y_poly_pred)rmse = np.sqrt(mean_squared_error(y_train,y_poly_pred))r2 = r2_score(y_train,y_poly_pred)print(MAE)print(rmse)print(r2)
Neural Network Regression
Neural networks are well known for outperforming traditional learning algorithms, especially when it comes to image recognition or classification problems. Applying neural networks to regression problems is not found that often in existing literature. This probably results from the fact that a neural network for regression may be considered as over engineered for a regression, which is in many cases simpler and does not require that much computational power. But on the other hand, it is also argued that deep neural networks mimic the human brain and therefore perform better at complex tasks. In conclusion to the fact that the prevailing dataset is very complex and has many features, the idea is to apply a neural network for performance improvements competing with the previously applied traditional models.
## Neural Networkmodel = Sequential()model.add(Dense(256, input_dim=10, kernel_initializer='normal', activation='relu'))model.add(Dense(512, activation='relu'))model.add(Dense(512, activation='relu'))model.add(Dense(1, activation='relu'))model.summary()model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'])
In this specific case the model has one input layer, two hidden layers with 512 nodes and one output layer. The activation function of choice is the rectified linear unit function (ReLU). The function is widely used due and usually a good first choice due to its good performance with low computational power and its ability to not activate all Neurons at the same time. The last layer is the output layer with only one node which is supposed to do the final output. In this case the private company valuation. In this case the best results were achieved with the mean squared error as loss measurement and the optimiser adam. In contrast to the stochastic gradient descent optimisation, the adam optimiser manages the learning rate for each parameter (weight) in the network and individually adjusts them. The final neural network model looks the following:
Evaluating Model PerformanceR2, MAE and RMSE are used determine the overall performance of each model and compare them. , interpreting the coefficients helps to find the drivers of a model. Before interpreting the results, it can be seen that the polynomial regression was outperforming the Neural Network Regression and the Linear Regression.
Furthermore, the Revenue coefficient is with a p-value of 0.003 statistically significant for the model. After removing one coefficient after the other, always the one with the most non-significant value, there were two coefficients clearly driving the model: Revenue and headcount with a p-value of 0.000 and 0.004
The best performing machine learning model was the polynomial regression model. It had an explained variance of 28% percent which is acceptable with regards to mimicking human behaviour. Furthermore the mean error was 830k Euros and the KPI features “Revenue” and “Headcount” have statistically significant influence on the model.
The underlying applied machine learning model takes only structured performance data into consideration. Important factors that usually drive a startup valuation enormously, such as management team experience, technological advantage, operating market growth and many more, are not yet considered. This also explains the moderate r-squared value. It needs to be stated that there is no standard guideline defined for the level of acceptance when it comes to r-squared - Especially when the model mimics human behaviour.
The research shows that with a little amount of structured private and public company data, a simple regression model is able to produce useful valuations. To further develop the machine learning model and improve its performance, it is highly recommended to take more features into consideration as a next step. Unstructured data from the areas mentioned above in combination with additional growth indicators are very likely to boost model performance significantly.
***************************************************************
Special thanks to Guillermo Falco from investory.io and Prof. Dr. Barbara Sprick from SRH Heidelberg supporting this study.
***************************************************************
Articles you may enjoy reading as well:
Learn How To Predict Customer Churn
Generate Your Sample Dataset — A Must Have Skill For Data Scientists.
Setup Your Data Environment With Docker
Eliminating Churn is Growth Hacking 2.0
Misleading with Data & Statistics