Use Machine Learning to Predict Horse Racing | by Cullen Sun | Towards Data Science

I am a part-time student of master of computer science at HKU. After last semester exams, I was thinking it would be fun and cool to apply machine learning on a real world dataset.
It took me no time to find out the great website for machine learning, Kaggle. Kaggle has a lot of interesting datasets for data science students and professionals. I searched for “Hong Kong” because I would like to find a dataset that is close/related to me, and I got the dataset for Horse Racing in HK. Great thanks to Graham Daley for posting the dataset on Kaggle.
I was very excited to work on my first real world machine learning project. I think horse racing is predictable and it’s a typical classification problem.
In this article, I am going to share with you how to do data preprocessing, neural network model building and training. We are going to use common machine learning packages like pandas, tensorflow, numpy, sklearn, etc. At the end, I will also share some thoughts that I came out after this project.
The dataset contains two csv files: races.csv and runs.csv. The two tables are related by column “race_id”. We will need to join the two tables to make meaningful data for training.
Line 1-6: Import the packages we are going to use for the project.
Line 8–9: Use pandas to read the csv file. Then, select 6 columns that I think are important features.
Line 11–13: It’s very common that the dataset contains NaN, which means some field is missing. Here, let’s just drop them.
Line 15–23: Some columns are strings. However, the neural network model only deals with numbers, so we need to encode them. String columns have different types: nominal and ordinal. We use LabelEncoder for column “venue”, while we use OrdinalEncoder for columns “config” and “going”.
Till now, we are happy with the data from races.csv. The dataframe head looks like below.
The runs.csv file contains data of horses for every race, associated with races.csv by race_id. Usually, each race has 14 or less horses racing together. The code is very similar to what we did for races.csv file, so I am not going to explain line by line. One thing to note is that, in line 10–13 above, we need to delete some noisy data, and the reason is explained in the code inline comment. After the processing, the dataframe head looks like below.
You might think that we are done with the runs.csv file, but not really because we need to think how we are going to feed the data into the neural network. Basically, we want one row from races.csv plus all the rows from runs.csv which belong to the same race, and put them together as one sample input for the neural network. The following code is to reshape runs dataframe so that rows of same race will form into one row.
Line 1–5: We define a small function her for sorting the columns.
Line 7: The magic pivot method reshapes the runs.csv data so that rows of same race goes into one row. The columns are ordered like [(horse_age, 1), (horse_age, 2), (horse_age, 3) ... ]. However, I want the columns to be like [(horse_age, 1), (horse_country, 1), (horse_type, 1) ...(horse_age, 2), (horse_country, 2), (horse_type, 2) ... ], so that we will have horse-1-features, horse-2-features, horse-3-features ... in order. It took me a lot of time to figure out which panda API to use for this. Have to mention that we need to avoid using loops to strive for better computation efficiency.
Line 8–10: Sort columns with the function defined in line 1–5. Apart from grouping the horse features, it also put the “result” columns to the end. So the right most columns will be [... (result, 11), (result, 12), (result, 13), (result, 14)]. Put them at the end because they are the outputs but not feature inputs.
Line 12–14: As the inline comment has explained, our neural network will need 14 horses data as input. Here, we simply fill dummy horse features with 0s.
You see, it’s a lot of work for the data preprocessing. Luckily, we are almost there. Now, let’s do the last bit to get the data ready for the neural network.
Line 1: Join the two dataframes we have prepared above by “race_id”.
Line 2: Select all the columns except last 14 columns as inputs X.
Line 3–4: Apply the feature scaling technique, standardisation, which makes training easier. Without that it’s difficult for the network to learn parameters for feature “declared_weight” (1100 sth) and feature “horse_age” (around 3–5).
Line 6: Convert the column “result” to the output y. This is a binary classification. So we put 1 if the horse got 1st place, else put 0.
Line 8–9: Simply print the shape of input and output to take a look. We can see that we have 6348 races, each of which we have 104 features and 14 outputs.
(6348, 104)(6348, 14)
Line 11–12: Use a handy util method from sklearn to split the data to train and test(validation) sets.
Firstly, I’d like to show you the architecture of the neural network as below. This is the core of the project. All the data preprocessing we have done so far is to feed into the neural network.
Input layer: 104 nodes, which takes 6 from the races.csv file and 7x14 from the runs.csv file.
Hidden layer: Just one with 96 nodes. I think this is enough for our project because the function of this classification shall not be that complicated.
Output layer: 14 nodes, each of them indicate whether a horse won the race.
Code is shown below, which is fairly simple, thanks to keras. Please pay attention to metrics. I chose precision, precision = TP/(TP+FP), as an evaluation parameter because it's our interest to bet on win.
The model detail is printed out as below. Trainable parameters are 11,438, which is the power of a neural network. It means the network will learn and adjust each of these parameters in order to find a function y = f(X) to fit our data.
Line 1–4: Further twist on the data before feed data for training.
Line 6–8: The training itself. During training the console prints the progress.
Line 10–26: Plot the performance, and the graphs are shown below.
The network is powerful enough to fit the training dataset well. However, there is big overfitting. The best performance we can reach on the validation dataset is about precision=0.3, which happens around epoch 60~80. Precision=0.3 means when we can get 3 out 10 win-bets correctly. This performance is not as high as I expected. The reason might be most races are handicaps, which means runners carry different weights to equalise their chance of winning the race, see reference Racing 101.
One shall study the domain knowledge before data analysis. My lesson is I didn’t understand the horse racing terms like “win_odds” and “actual_weight” so I didn’t know which columns are the useful features. After study the guide, things got much clearer.
Do more data analysis before write code. I started coding very quickly after found the dataset. You can see another notebook of mine, in which I made a big mistake that I treated one row of runs.csv file as one sample. Whether a horse can win a race is relative to other horses in the same race, so we have to put 14 horses’ data of one race together as one sample.
I have heard people saying that data preprocessing usually takes more time than model building and training. I found it’s true for this project. Pandas have hundreds of powerful APIs for you to manipulate the dataframe, and it will take time to get familiar with them in order to choose the right API to get your job done quickly.
One can really learn a lot by doing the project. It’s much much more than the course assignments.
In this article, I have shared with you how to apply machine learning skills end to end on a real world dataset from Kaggle. My full notebook can be found here. Thank you so much for reading it and welcome your comments.