Trying to do a linear regression with multiple inputs and one output

ErikVats: <p>I am trying to create a linear regression model that predicts injury time in a football match, however my results are quite bad:<br>
Error on validation data<br>
MSE: 1.1520577669143677<br>
MAE: 0.7984767556190491<br>
MAPE: 35.94094467163086 %<br>
Error on test data<br>
MSE: 1.2277499437332153<br>
MAE: 0.8027499914169312<br>
MAPE: 41.30732345581055 %.</p>
<p>Can somebody help me with my code to improve my results?</p>
<p>import matplotlib.pyplot as plt</p>
<p>import pandas as pd</p>
<p>import torch</p>
<p>from torch.utils.data import DataLoader</p>
<p>from math import sqrt</p>
<h1>We set a fixed seed for repeatability</h1>
<p>random_seed = 12345  # This seed is also used in the pandas sample() method below</p>
<p>torch.manual_seed(random_seed)</p>
<p>df = pd.read_csv(‘data/injuryTimeDataset.csv’, index_col=0)</p>
<p>print(df)</p>
<p><span class="hashtag">#Split</span> data into a train, validation and test set</p>
<p><span class="hashtag">#Test</span> set</p>
<p>test_set = df.iloc[8000:12000]</p>
<p><span class="hashtag">#Make</span> a cope of the dataset and remove test set</p>
<p>train_val_set = df.copy().drop(test_set.index)</p>
<p><span class="hashtag">#Random</span> sample validation data without replacement(10%)</p>
<p>val_set = train_val_set.sample(frac=0.1, replace = False, random_state = random_seed)</p>
<p><span class="hashtag">#Remaining</span> data used for training (90%)</p>
<p>train_set = train_val_set.copy().drop(val_set.index)</p>
<p><span class="hashtag">#Check</span> numbers add up</p>
<p>n_points = len(train_set) + len(val_set) + len(test_set)</p>
<p><span class="hashtag">#print</span>(f’{len(df)} = {len(train_set)} + {len(val_set)} + {len(test_set)} = {n_points}’)</p>
<p><span class="hashtag">#Plot</span> the sets</p>
<h1>plt.figure(figsize=(16, 9))</h1>
<h1>plt.scatter(train_set.index, train_set[‘declared_inj_time’], color=‘black’, label=‘Train’)</h1>
<h1>plt.scatter(val_set.index, val_set[‘declared_inj_time’], color=‘green’, label=‘Val’)</h1>
<h1>plt.scatter(test_set.index, test_set[‘declared_inj_time’], color=‘red’, label=‘Test’)</h1>
<h1>plt.legend()</h1>
<p><span class="hashtag">#Inputs</span> and outputs</p>
<p>INPUT_COLS = [‘goals’,‘corners’, ‘free_kicks’,‘substitutions’]</p>
<p>OUTPUT_COL = [‘declared_inj_time’]</p>
<p><span class="hashtag">#Linear</span> regression model</p>
<p>class LinearRegression(torch.nn.Module):</p>
<pre><code>def __init__(self):

    super().__init__()

    self.device = 'cuda' if torch.cuda.is_available() else 'cpu'

    self.linear = torch.nn.Linear(in_features=len(INPUT_COLS), out_features=len(OUTPUT_COL))



def forward(self, x):

    y_pred = self.linear(x)

    return y_pred
</code></pre>
<p><span class="hashtag">#Training</span> loop</p>
<p>def train(</p>
<pre><code>model: torch.nn.Module,

train_loader: DataLoader,

val_loader: DataLoader,

n_epochs: int,

lr: float,
</code></pre>
<p>) -&gt; torch.nn.Module:</p>
<pre><code>#Loss and optimizer

criterion = torch.nn.MSELoss(reduction='mean')

optimizer = torch.optim.Adam(model.parameters(),lr=lr)

#Train weigths

for epoch in range(n_epochs):

    for inputs, labels in train_loader:

        #Zero the parameter gradients

        optimizer.zero_grad()

        #Forward propagation

        pred_y = model(inputs)

        

        #Compute loss

        loss = criterion(pred_y,labels)

        #Backward propagration to compute gradient

        loss.backward()

        #Update parameters

        optimizer.step()

        # Evaluate model on validation data

    mse_val = 0

    for inputs, labels in val_loader:

        mse_val += torch.sum(torch.pow(labels - model(inputs), 2)).item()

    mse_val /= len(val_loader.dataset)

    print(f'Epoch: {epoch + 1}: Val MSE: {mse_val}')



return model
</code></pre>
<p><span class="hashtag">#Prepare</span> data for training</p>
<p>x_train = torch.from_numpy(train_set[INPUT_COLS].values).to(torch.float)</p>
<p>y_train = torch.from_numpy(train_set[OUTPUT_COL].values).to(torch.float)</p>
<p>x_val = torch.from_numpy(val_set[INPUT_COLS].values).to(torch.float)</p>
<p>y_val = torch.from_numpy(val_set[OUTPUT_COL].values).to(torch.float)</p>
<p><span class="hashtag">#Create</span> dataset loaders</p>
<p>train_dataset = torch.utils.data.TensorDataset(x_train, y_train)</p>
<p>train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=10,shuffle=True)</p>
<p>val_dataset = torch.utils.data.TensorDataset(x_val,y_val)</p>
<p>val_loader = torch.utils.data.DataLoader(val_dataset, batch_size=len(val_set),shuffle=False)</p>
<p><span class="hashtag">#Initialize</span> mmodel</p>
<p>model = LinearRegression()</p>
<p><span class="hashtag">#Train</span> model</p>
<p>n_epochs = 100</p>
<p>lr = 0.0001</p>
<p>model = train(model, train_loader, val_loader, n_epochs, lr)</p>
<p><span class="hashtag">#Evaluate</span> model</p>
<h1>Predict on validation data</h1>
<p>pred_val = model(x_val)</p>
<h1>Compute MSE, MAE and MAPE on validation data</h1>
<p>print(‘Error on validation data’)</p>
<p>mse_val = torch.mean(torch.pow(pred_val - y_val, 2))</p>
<p>print(f’MSE: {mse_val.item()}’)</p>
<p>mae_val = torch.mean(torch.abs(pred_val - y_val))</p>
<p>print(f’MAE: {mae_val.item()}’)</p>
<p>mape_val = 100*torch.mean(torch.abs(torch.div(pred_val - y_val, y_val)))</p>
<p>print(f’MAPE: {mape_val.item()} %’)</p>
<p><span class="hashtag">#Evaluate</span> model on test data</p>
<h1>Get input and output as torch tensors</h1>
<p>x_test = torch.from_numpy(test_set[INPUT_COLS].values).to(torch.float)</p>
<p>y_test = torch.from_numpy(test_set[OUTPUT_COL].values).to(torch.float)</p>
<h1>Make prediction</h1>
<p>pred_test = model(x_test)</p>
<h1>Compute MSE, MAE and MAPE on test data</h1>
<p>print(‘Error on test data’)</p>
<p>mse_test = torch.mean(torch.pow(pred_test - y_test, 2))</p>
<p>print(f’MSE: {mse_test.item()}’)</p>
<p>mae_test = torch.mean(torch.abs(pred_test - y_test))</p>
<p>print(f’MAE: {mae_test.item()}’)</p>
<p>mape_test = 100*torch.mean(torch.abs(torch.div(pred_test - y_test, y_test)))</p>
<p>print(f’MAPE: {mape_test.item()} %’)</p>

googlebot: <p>Your target variables are conceptually discrete, so gradient descent for MSE loss struggles, as it implicitly fits means of gaussian distributions. Poisson regression (see PoissonNLLLoss) or reformulating the task as classification should work better.</p>
<p>Note that you could use something like LinearRegression from sklearn, that returns a closed form solution without doing mini-batch gradient descent.</p>

ErikVats: <p>Thank you very much!<br>
If I should reformulate the task as a classification task, how would this look? The goal of my task is to get a measure of how much each event affects injurytime.</p>
<p>Would a neural network give a better estimate?</p>
<p>Also, are there any non-gaussian response loss function that I could use?</p>
<p>I did change the criterion to PoissonNLLLoss and the results did not improve, any other tips?:<br>
Error on validation data<br>
MSE: 4.964473247528076<br>
MAE: 1.8417545557022095<br>
MAPE: 61.19207763671875 %<br>
Error on test data<br>
MSE: 4.665085792541504<br>
MAE: 1.7789844274520874<br>
MAPE: 60.499691009521484 %</p>

googlebot: <p>poisson loss fits log(input) to a poisson rate, so exp(input) should be used to recover [positive] rates. And I’m not sure how good these metrics are with a poisson distribution.</p>
<aside class="quote no-group" data-username="ErikVats" data-post="3" data-topic="98584">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/e/ea5d25/48.png" class="avatar"> ErikVats:</div>
<blockquote>
<p>If I should reformulate the task as a classification task, how would this look? The goal of my task is to get a measure of how much each event affects injurytime.</p>
</blockquote>
</aside>
<p>For that goal, maybe it is not that good, as per category coefficients would be introduced.</p>
<aside class="quote no-group" data-username="ErikVats" data-post="3" data-topic="98584">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/e/ea5d25/48.png" class="avatar"> ErikVats:</div>
<blockquote>
<p>Would a neural network give a better estimate?</p>
</blockquote>
</aside>
<p>Without hidden layers, it is worse than a least squares minimizer, as the latter instantly gets the global minimum. (and I don’t think a hidden layer will help in your case, as your feature space is too small).</p>
<p>It is actually possible that your initial result converged and the fit line is as good as you can get (with weak predictors). Big MSE is to be expected with separated targets.</p>