Error in metric evaluation

username: Rexedoziem
cooked: <p>found input variables with inconsistent numbers of samples: [558, 14] trying to place valid_labels and predictions into the mean squared error metric, the shape valid_labels = (558,6) and predictions = (14, 6). How do I go about it?</p>
<p>def MCRMSE(y_trues, y_preds):<br>
scores = []<br>
idxes = y_trues.shape[1]<br>
for i in range(idxes):<br>
y_true = y_trues[:,i]<br>
y_pred = y_preds[:,i]<br>
score = mean_squared_error(y_true, y_pred, squared=False) # RMSE<br>
scores.append(score)<br>
mcrmse_score = np.mean(scores)<br>
return mcrmse_score, scores</p>
<p>def get_score(y_trues, y_preds):<br>
mcrmse_score, scores = MCRMSE(y_trues, y_preds)<br>
return mcrmse_score, scores</p>
<p>valid_labels = valid_df[config.targets].values</p>
<p>def eval_fx(data_loader, model, device):<br>
model.eval()<br>
valid_loss = 0</p>
<pre><code>#final_targets = []
final_outputs = []
with torch.no_grad():
    for bi, (d, targets) in tqdm(enumerate(data_loader)):
        input_ids = d['input_ids']
        attention_mask = d['attention_mask']
        #token_type_ids = d['token_type_ids']
        targets = targets
    
        input_ids = input_ids.to(device, dtype=torch.long)
        attention_mask = attention_mask.to(device, dtype=torch.long)
        #token_type_ids = token_type_ids.to(device, dtype=torch.long)
        targets = targets.to(device, dtype=torch.float)
        # putting them into the device
        outputs = model(input_ids, attention_mask)
    
        loss = loss_fn(outputs, targets)
        #criterion = RMSELoss
        #loss = criterion(outputs,targets)
        valid_loss += loss.item() * d['input_ids'].size(0)
    valid_loss = valid_loss / len(data_loader.sampler)
        
    final_outputs.append(outputs.cpu().numpy())
    predictions = np.concatenate(final_outputs)
    
return valid_loss, predictions
</code></pre>
<p>import numpy as np<br>
from tqdm import tqdm<br>
save_model = False<br>
best_score = np.inf<br>
early_stopping = 5<br>
early_stopping_counter = 0<br>
for epoch in range(config.EPOCHS):<br>
train_loss = train_fx(train_data_loader, model, optimizer, scheduler, device)<br>
valid_loss, predictions = eval_fx(valid_data_loader, model, device)</p>
<pre><code>score, scores = get_score(valid_labels, predictions)
</code></pre>

username: Rexedoziem
cooked: <p>The error came in from scores and I don’t know how to go about it?</p>

username: bradmacer
cooked: <aside class="quote no-group" data-username="Rexedoziem" data-post="1" data-topic="164730">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rexedoziem/48/52029_2.png" class="avatar"> Rexedoziem:</div>
<blockquote>
<p>found input variables with inconsistent numbers of samples</p>
</blockquote>
</aside>
<p>Sounds like the shapes of your labels and predictions are not in alignment. I faced a similar problem while fitting a <a href="http://net-informations.com/ds/mla/default.htm" rel="noopener nofollow ugc">linear regression</a> model . The problem in my case was, Number of rows in X was not equal to number of rows in y. In most case, x as your feature parameter and y as your predictor. But your feature parameter should not be 1D. So check the shape of x and if it is 1D, then convert it from 1D to 2D.</p>
<p><code>x.reshape(-1,1)</code></p>
<p>Also, you likely get problems because you remove rows containing nulls in X_train and y_train independent of each other. y_train probably has few, or no nulls and X_train probably has some. So when you remove a row in X_train and the same row is not removed in y_train it will cause your data to be unsynced and have different lenghts. Instead you should remove nulls before you separate X and y.</p>