Machine Learning for Sports Betting: It’s Not a Basic Classification Problem. | by Charles Malafosse | Towards Data Science

Sports betting is one of these perfect problems for machine learning algorithms and specifically classification neural networks. Tons of data available and a clear objective of picking the winner! Nonetheless, classic classification models are not well suited for betting strategies, and one needs to use a custom loss function in his neural network to achieve better profitability. We explain why below.
Let’s implement basic betting strategies based on odds from betting exchanges. Decimal odds are the ratio of the full payout to the stake. For instance, odds of 2, means that for a 1$ bet you could win 2$ (including your stake). Note that odds inverse gives the implied probability of being right. Odds of 2 means a 50% chance of winning (1/odds). Hence you want to bet on teams with the lowest odds, i.e., the highest probability of winning.
For the sake of illustration, we design two basics strategies:
“Win”: Bet on team with the lowest odds, the one with the highest probability of winning.
“Win or Draw”: Bet on team with the lowest odds value and bet on a draw.
Assume that we have odds 5.5–1.61 for the game Bournemouth vs. Chelsea. Chelsea has odds of 1.61. Hence we bet on Chelsea’s victory for the “Win” strategy, and a victory and draw for the “Win or Draw” strategy.
Now let’s compare the two betting strategies, “Win” vs. “Win or Draw.”For the data, we used the Betfair API. Betfair is one of the biggest betting exchange, and its API contains exchange markets navigation, odds retrieval, and bet placement operations.
The bar charts above represent the accuracy and profit achieved by both betting systems. Accuracy means the number of times our bets were correct, divided by the total number of bets (200 in that case). Profit represents the amount you could have earned by investing 1£ in each bet.The first chart on the left shows an accuracy ratio of 61% for the “Win” strategy vs. 80% for the “Win or Draw.” In that case, the “Win or Draw” seems a lot better. However, the second chart shows that the profit it generates is far less than the “Win” strategies. 1£ vs. 12.
Therefore we shouldn’t aim at better accuracy in predicting games outcomes, but instead, we should focus on maximizing our overall profit.
Much data is involved when deciding on which team to put our money. For this reason, betting is an ideal subject to apply one of the most popular machine learning techniques, Neural Networks.
In particular, we could use a classification neural network. A classification NN is ideal when applied to problems for which there is a discrete outcome, or said otherwise when identifying for which category belongs a particular observation. As examples, think of the funny “Hotdog, not Hotdog” or Sentiment Analysis that attempts to classify text into positive or negative sentiment.
Applied to sports betting, we could devise a neural network with three simple categories. Below is the architecture of such a network.
However, from our previous example with two simple betting strategies, it is clear that we are not trying to predict the outcome of the game, but rather what bet would be the most profitable. Applied to a classification neural network, this would result in the following architecture. Note that in this case we even allow a “no bet” category.
We end up with a multi-label classification problem (not to be confused with multi-class classification) as the outcome of a game could result in one or two predictions being correct. For instance, a win by the home team would result in two winnings bets: “Win Home” and “Win Home or Draw”.
Not all bets provide the same reward. A bet with odds of 2 can generate 1£ of profit, while odds of 1.1 generates a lower profit of 0.1£. Both bets, however, exposed to the same loss of 1£ if not successful. Therefore they are not equal; risking 1£ to make 1£ is not the same as risking 1£ to make 0.1£.
To take this into account in our neural network, we need to use a custom loss function. A loss function (or objective function) is a measure of “how good” a neural network does, given its training sample and the expected output. In standard classification neural network, we use loss functions such as the categorical cross-entropy. However, this kind of functions would give similar weights to all bets, ignoring the profitability discrepancies.
In our case we want the model to maximize the overall gain of the strategy. Thus the input of our custom loss function must incorporate the potential profit of each bet.
We set up our custom loss function with Keras on top of TensorFlow.
In Keras, a loss function takes two arguments:
y_true: Vector of true labels (Win Home, Win Home or Draw, Draw, Win Away, Win Away or Draw, No bet). Based on our neural network architecture, this takes the form of a vector of 1 and 0. For instance, a game resulting in a home team victory has the following y_true vector (1,1,0,0,0,0).
y_pred: Vector of Predictions. It is the output of our neural network classifier.
Because we cannot pass the game’s odds in the loss function due to Keras limitations, we have to pass them as additional items of the y_true vector.
Below is our custom loss function written in Python and Keras.In a nutshell, it measures the profit/loss average over the input for a unit stake. Steps are the following for each observation (each game) :
Get odds from the y_true input
Compute the potential profit of each bet using odds.
Concatenate profits of winning bets and losses of the ones.
We end up with the expected P&L of this observation. We multiply this by -1 to get a “loss” to minimize (and not a gain to maximize).
def odds_loss(y_true, y_pred):    """    The function implements the custom loss function        Inputs    true : a vector of dimension batch_size, 7. A label encoded version of the output and the backp1_a and backp1_b    pred : a vector of probabilities of dimension batch_size , 5.        Returns     the loss value    """    win_home_team = y_true[:, 0:1]    win_home_or_draw = y_true[:, 1:2]    win_away = y_true[:, 2:3]    win_away_or_draw = y_true[:, 3:4]    draw = y_true[:, 4:5]    no_bet = y_true[:, 5:6]    odds_a = y_true[:, 6:7]    odds_b = y_true[:, 7:8]    gain_loss_vector = K.concatenate([win_home_team * (odds_a - 1) + (1 - win_home_team) * -1,      win_home_or_draw * (1/(1 -1/odds_b) - 1) + (1 - win_home_or_draw) * -1,      win_away * (odds_b - 1) + (1 - win_away) * -1,      win_away_or_draw * (1/(1 -1/odds_a) - 1) + (1 - win_away_or_draw) * -1,      draw * (1/(1 - 1/odds_a - 1/odds_b) - 1) + (1 - draw) * -1,      K.zeros_like(odds_a)], axis=1)    return -1 * K.mean(K.sum(gain_loss_vector * y_pred, axis=1))
For our data we take a list of 200 games from the English Premier League, season 2018–2019, August to December 2018. It contains descriptive game data such as team names, odds from Betfair, and a sentiment score (representing the percentage of positive tweets over the positive and negative tweets). Data and Jupyter notebook available on my github page.
Our data contain the outcome of each game in the form of 1, 2 ot 3:
1: Home win
2:Away win
3:Draw
This needs to be converted to a one-hot encoding vector representing the output layer of our neural network. Plus we add the odds of each team as elements of this vector. This is exactly what we do below.
def get_data():    data = pd.read_csv('extract-betsentiment-com.csv')    X = data.values[:, 5:-5]    y = data.values[:, -1]    y_full = np.zeros((X.shape[0], 8))    for i, y_i in enumerate(y):        if y_i == 1:            y_full[i, 0] = 1.0            y_full[i, 1] = 1.0        if y_i == 2:            y_full[i, 2] = 1.0            y_full[i, 3] = 1.0        if y_i == 3:            y_full[i, 1] = 1.0            y_full[i, 3] = 1.0            y_full[i, 4] = 1.0        y_full[i, 6] = X[i, 1] # ADD ODDS OF HOME TEAM        y_full[i, 7] = X[i, 2] # ADD ODDS OF AWAY TEAM    return X, y_full, yX, y, outcome = get_data()# SPLIT THE DATA IN TRAIN AND TEST DATASET.train_x, test_x, train_y, test_y, = train_test_split(X,  y)
Before training the model, we need first to define it. We use a fully connected neural network, with two hidden layers. We use BatchNormalization to normalize weights and eliminate the vanishing gradient problem.
Then we train the model using a set of arbitrary parameters.
def get_model(input_dim, output_dim, base=1000, multiplier=0.25, p=0.2):    inputs = Input(shape=(input_dim,))    l = BatchNormalization()(inputs)    l = Dropout(p)(l)    n = base    l = Dense(n, activation='relu')(l)    l = BatchNormalization()(l)    l = Dropout(p)(l)    n = int(n * multiplier)    l = Dense(n, activation='relu')(l)    l = BatchNormalization()(l)    l = Dropout(p)(l)    n = int(n * multiplier)    l = Dense(n, activation='relu')(l)    outputs = Dense(output_dim, activation='softmax')(l)    model = Model(inputs=inputs, outputs=outputs)    model.compile(optimizer='Nadam', loss=odds_loss)    return modelmodel = get_model(21, 6, 1000, 0.9, 0.7)history = model.fit(train_x, train_y, validation_data=(test_x, test_y),          epochs=200, batch_size=5, callbacks=[EarlyStopping(patience=25),ModelCheckpoint('odds_loss.hdf5',save_best_only=True)])
Once the training has completed, we look at the performance of our model with the following print command:
print('Training Loss : {}\nValidation Loss : {}'.format(model.evaluate(train_x, train_y), model.evaluate(test_x, test_y)))150/150 [==============================] - 0s 271us/step 50/50 [==============================] - 0s 318us/step Training Loss : -0.05027060141166051Validation Loss : -0.08301501810550689
As we can see, we end up with a training loss of -0.05. Keep in mind that we’re trying to minimize our loss function, which is the opposite of our P&L. This number tells us that, on average, each bet would generate a profit of 0.05 for each stake of 1£. Our validation dataset, shows an average profit of 0.08 for each pound. Not bad considering we just provided basic data to our neural network. Over 200 games, our theoretical NN betting strategy would have generated 10 to 16.6£, assuming we bet 1£ on each game.
We presented a way to include bets p&l into a neural network classifier using a custom loss function. It goes beyond the accuracy ratio that can be misleading when designing betting systems. We believe this is useful for anyone looking to use machine learning for sports.
Feel free to contact me for more information or questions. Also, note that we’ll be happy to share our data if you find any interest in them.
https://medium.com/@charlesmalafosse/membership
Github page with data and notebook. Use google colab to test it in no time.