Predicting Horse Racing Results with Deep Learning | by Victor Sim | Towards Data Science

I am interested in the application of machine learning in different fields, where analysis of large amounts of data is necessary to come to accurate conclusions. One such field is sports analytics. Unlike other forms of gambling, the environment and the actors within it are sure to have an effect on the result, making it a problem that can be solved with machine learning.
I wanted to try to implementing deep learning in this field and so I looked online for datasets about sports. Eventually, I found a dataset on Hong Kong horse racing that contained 6000+ races. Unlike other games like basketball and football that are more complex, horse racing is just 14 horses racing against each otehr. This means that there are far fewer variables to take into account.
The model should be able to predict which horse will win, when given data about the race and the horse (sex,age,type).
With the theoretical concept in mind, this is the code that I used:
import osos.getcwd()
This script gets the current working directory that the programming is running this. I run this function so I can copy a template of the path, so as to easily describe the path to the dataset.
os.chdir('Desktop\\Files\\Data')
After that, I move to the directory where the file is stored.
import pandas as pddf = pd.read_csv('races.csv')
The dataset consists of two csv files, the races and runs csv. The races csv contains information about each race and the runs csv contains information about each of the horses that participate in each race. The goal is to encode all non-numerical data, and concatenate both of these csv files into one big dataframe. This will make it easier to get the model to train the model.
df = pd.read_csv('races.csv')df = df.drop('date',axis = 1)df = df.drop('going',axis = 1)df = df.drop('surface',axis = 1)df = df.drop('prize',axis = 1)df = df.drop('race_no',axis = 1)for column in df.columns:    if 'sec' in column or 'time' in column or 'place' in column or 'win' in column:        df = df.drop(column,axis = 1)df
This script removes all the unnecessary information from the dataset. Here is the dataset after removing all the unnecessary information:
df2 = pd.read_csv('runs.csv')
This script opens the second csv file, that contains all the information on the horses that participated in the race.
df2 = pd.read_csv('runs.csv')df2 = df2.drop('horse_id',axis = 1)df2 = df2.drop('result',axis = 1)df2 = df2.drop('horse_gear',axis = 1)df2 = df2.drop('win_odds',axis = 1)df2 = df2.drop('place_odds',axis = 1)df2 = df2.drop('trainer_id',axis = 1)df2 = df2.drop('jockey_id',axis = 1)df2['race_id'] = df2['race_id'] for column in df2.columns:    if 'time' in column or 'behind' in column or 'position' in column:        df2 = df2.drop(column,axis = 1)        df2
This script removes all the unnecessary data from the csv file. Here is the dataset after that:
import warningswarnings.filterwarnings('ignore')import numpy as nptrue_df = []for i in range(len(df['race_id'])):    matches = list(df2[df2['race_id']==i].drop(['race_id'],axis=1).drop(['horse_no'],axis=1).drop(['won'],axis=1).values)    horse_no = len(matches)    matches.insert(0,df[df['race_id']==i].values[0])    matches = flatten(matches)    true_df.append(matches)
This script concatenates both CSV files together: For every race, it finds all the horses that participated in the race. This data is flattened an added to information about the race. The result of this is that you end up with a dataframe, with each row being a race. Each row has 104 values, containing information about the race and each of the horses in the race.
true_df = pd.DataFrame(true_df)true_df
After that, we can convert the list into a dataframe. Here is the full dataframe:
All the NaN values are there because some races have 14 horses, while others only have 12 horses. For those 12 horses races, the other values are filled with zeros.
winners = []for i in range(len(df['race_id'])):    try:        winner = df2[df2['race_id']==i][df2['won']==1]['horse_no'].values[0]    except:        print(df2[df2['race_id']==i][df2['won']==1])        winner = 1    winners.append(winner)
I then collect all the results from each race, and add this data to the end of every row.
true_df['winners'] = winnerstrue_df = pd.DataFrame(true_df).fillna(0)true_df.to_csv('Desktop\\Files\\Data\\insert_data.csv',index=False)
I then replace all instances of NaN and None with 0, so it does not have influence on the model’s training. The data frame is then saved as a csv file.
import osos.chdir('C:\\Users\\v_sim\\Desktop\\Files\\Data')import pandas as pddf = pd.read_csv('insert_data.csv')winners = df['winners'].valuesdf = df.drop('winners',axis=1).fillna(0)df = df.drop('0',axis = 1)
After saving the data frame as a csv file, this script opens the csv files, records the winners of each race, and then removes them from the dataframe. This is so that the rest of the data can be directly converted to the X-values.
def create_dict(array):    array = array.astype(str)    unique = np.unique(array)    encode_dictionary = dict()    decode_dictionary = dict()    for i in range(len(unique)):        encode_dictionary[unique[i]] = i        decode_dictionary[i] = unique[i]    return encode_dictionary,decode_dictionarydef encode_df(df):    columns = df.columns    dtypes = df.dtypes    for i in range(len(columns)):        if dtypes[i] == 'object':            encode,decode = create_dict(df[columns[i]].values)            df[columns[i]] = df[columns[i]].map(encode)    return dfdf = encode_df(df)df = df.fillna(0)
This script contains two functions: One that encodes information within a list, and another that applies the function for columns in a dataframe, where the data is non-numerical.
X = df.values.reshape(len(X),103,1)y = np.array(y)
This function defines the X and y values, based on the operations applied on the initial dataset.
from keras.models import Sequentialfrom keras.layers import Densefrom keras.layers import Flatten,BatchNormalizationfrom keras.layers import Dropoutfrom keras.layers.convolutional import Conv1Dfrom keras.layers.convolutional import MaxPooling1Dfrom keras.optimizers import Adamimport kerasmodel = Sequential()model.add(Conv1D(filters=256, kernel_size=2, activation='relu', input_shape=(103,1)))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Conv1D(filters=512, kernel_size=2, activation='relu'))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Conv1D(filters=1024, kernel_size=2, activation='relu'))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Flatten())model.add(Dense(64,activation = 'relu'))model.add(Dense(128,activation = 'relu'))model.add(Dense(256,activation = 'relu'))model.add(BatchNormalization())model.add(Dense(14, activation='softmax'))model.compile(loss='categorical_crossentropy', optimizer='adam',metrics = ['accuracy'])
This is the model that I ended up going with. It is a typical convolutional network that is used for image classification. I adapted it for 1 dimensional arrays, and for multi-class classification.
from keras.models import Sequential, load_model, model_from_jsonfrom keras import callbacks, optimizerssymbol = 'horse_racing'h5 = symbol + '_best_model' + '.h5'checkpoint = callbacks.ModelCheckpoint(h5,                                       monitor='loss',                                       verbose=0,                                       save_best_only=True,                                       save_weights_only=True,                                       mode='auto',                                       period=1)callback = [checkpoint]json = symbol + '_best_model' + '.json'model_json = model.to_json()with open(json, "w") as json_file:    json_file.write(model_json)model.fit(X,y,epochs = 5000,callbacks = callback,validation_split = 0.1)
This script trains the model, with the checkpoint callback, so that the weights of the best iteration of the model can be saved and reloaded. This prevents the waste of computational resources.
After training the model for a while, I the model had a 92% accuracy on training data, but the accuracy on validation data was relatively poor.
from keras.models import Sequentialfrom keras.layers import Densefrom keras.layers import Flatten,BatchNormalizationfrom keras.layers import Dropoutfrom keras.layers.convolutional import Conv1Dfrom keras.layers.convolutional import MaxPooling1Dfrom keras.optimizers import Adamimport kerasmodel = Sequential()model.add(Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(103,1)))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Dropout(0.3))model.add(Conv1D(filters=128, kernel_size=2, activation='relu'))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Dropout(0.3))model.add(Conv1D(filters=256, kernel_size=2, activation='relu'))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Dropout(0.3))model.add(Conv1D(filters=512, kernel_size=2, activation='relu', input_shape=(103,1)))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Dropout(0.3))model.add(Conv1D(filters=1024, kernel_size=2, activation='relu'))model.add(MaxPooling1D(pool_size=2))model.add(BatchNormalization())model.add(Dropout(0.3))model.add(Conv1D(filters=2048, kernel_size=2, activation='relu'))model.add(BatchNormalization())model.add(Dropout(0.3))model.add(Flatten())model.add(Dense(64,activation = 'relu'))model.add(Dense(128,activation = 'relu'))model.add(Dense(256,activation = 'relu'))model.add(Dropout(0.3))model.add(BatchNormalization())model.add(Dense(14, activation='softmax'))model.compile(loss='categorical_crossentropy', optimizer='adam',metrics = ['accuracy'])
This model is the model that I used to combat this: The model is much deeper and also much wider. The model is deeper so that more complex patterns can be found and it is wider so that it works better with dropout.
This project is just a basic implementation of machine learning for sports analytics, just for the fun of it. I think that it was reasonably successful, but overfitting on the data is still a problem.
If you want to see more of my content, click this link.