NN: Invalid index in gather

username: lolarose
cooked: <p>I am new to PyTorch and am getting the following error when building a neural network. Please let me know if you need any additional details. Thanks in advance for your help!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/8/d8d400d3045d1033b44d0ce780d44ec73296c5a5.png" data-download-href="https://discuss.pytorch.org/uploads/default/d8d400d3045d1033b44d0ce780d44ec73296c5a5" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/8/d8d400d3045d1033b44d0ce780d44ec73296c5a5.png" alt="image" data-base62-sha1="uW9lBkpUqjW1sL1Q7ozphOVmmot" width="690" height="237" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">871×300 9.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>(I have also tested generating a random result data set of the same shape and the model works<br>
y = torch.randint(0, 4, (X_train.shape[0], 1), dtype=torch.long)<br>
)</p>
<p>Error:</p>
<h2><a name="re-initializing-module-because-the-following-parameters-were-re-set-num_classes-predictor-re-initializing-optimizer-1" class="anchor" href="#re-initializing-module-because-the-following-parameters-were-re-set-num_classes-predictor-re-initializing-optimizer-1"></a>Re-initializing module because the following parameters were re-set: num_classes, predictor.<br>
Re-initializing optimizer.</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in </p>
<p>----&gt; 1 skorch_model.fit(X_train_torch,y_train_torch)</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in fit(self, X, y, **fit_params)<br>
846             self.initialize()<br>
847<br>
 → 848         self.partial_fit(X, y, **fit_params)<br>
849         return self<br>
850</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in partial_fit(self, X, y, classes, **fit_params)<br>
805         self.notify(‘on_train_begin’, X=X, y=y)<br>
806         try:<br>
 → 807             self.fit_loop(X, y, **fit_params)<br>
808         except KeyboardInterrupt:<br>
809             pass</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in fit_loop(self, X, y, epochs, **fit_params)<br>
737                 yi_res = yi if not y_train_is_ph else None<br>
738                 self.notify(‘on_batch_begin’, X=Xi, y=yi_res, training=True)<br>
 → 739                 step = self.train_step(Xi, yi, **fit_params)<br>
740                 train_batch_count += 1<br>
741                 self.history.record_batch(‘train_loss’, step[‘loss’].item())</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in train_step(self, Xi, yi, **fit_params)<br>
662             step_accumulator.store_step(step)<br>
663             return step[‘loss’]<br>
 → 664         self.optimizer_.step(step_fn)<br>
665         self.optimizer_.zero_grad()<br>
666         return step_accumulator.get_step()</p>
<p>~/.local/lib/python3.6/site-packages/torch/optim/sgd.py in step(self, closure)<br>
78         loss = None<br>
79         if closure is not None:<br>
—&gt; 80             loss = closure()<br>
81<br>
82         for group in self.param_groups:</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in step_fn()<br>
659         step_accumulator = self.get_train_step_accumulator()<br>
660         def step_fn():<br>
 → 661             step = self.train_step_single(Xi, yi, **fit_params)<br>
662             step_accumulator.store_step(step)<br>
663             return step[‘loss’]</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in train_step_single(self, Xi, yi, **fit_params)<br>
602         self.module_.train()<br>
603         y_pred = self.infer(Xi, **fit_params)<br>
 → 604         loss = self.get_loss(y_pred, yi, X=Xi, training=True)<br>
605         loss.backward()<br>
606</p>
<p>~/.local/lib/python3.6/site-packages/skorch/net.py in get_loss(self, y_pred, y_true, X, training)<br>
1097         “”"<br>
1098         y_true = to_tensor(y_true, device=self.device)<br>
 → 1099         return self.criterion_(y_pred, y_true)<br>
1100<br>
1101     def get_dataset(self, X, y=None):</p>
<p>~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in <strong>call</strong>(self, *input, **kwargs)<br>
539             result = self._slow_forward(*input, **kwargs)<br>
540         else:<br>
 → 541             result = self.forward(*input, **kwargs)<br>
542         for hook in self._forward_hooks.values():<br>
543             hook_result = hook(self, input, result)</p>
<p>~/.local/lib/python3.6/site-packages/spacecutter/losses.py in forward(self, y_pred, y_true)<br>
106         return cumulative_link_loss(y_pred, y_true,<br>
107                                     reduction=self.reduction,<br>
 → 108                                     class_weights=self.class_weights)</p>
<p>~/.local/lib/python3.6/site-packages/spacecutter/losses.py in cumulative_link_loss(y_pred, y_true, reduction, class_weights)<br>
66     “”"<br>
67     eps = 1e-15<br>
—&gt; 68     likelihoods = torch.clamp(torch.gather(y_pred, 1, y_true), eps, 1 - eps)<br>
69     neg_log_likelihood = -torch.log(likelihoods)<br>
70</p>
<p>RuntimeError: Invalid index in gather at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:657</p>

username: ptrblck
cooked: <p>Could you post the code for <code>cumulative_link_loss</code> and if possible the tensors you are passing to it (you can also create dummy random values for them)?<br>
Apparently the <code>torch.gather</code> call throws an invalid indexing error.</p>

username: lolarose
cooked: <p>That is from the spacecutter library</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/EthanRosenthal/spacecutter" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars2.githubusercontent.com/u/7435500?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/EthanRosenthal/spacecutter" target="_blank" rel="nofollow noopener">EthanRosenthal/spacecutter</a></h3>

<p>Ordinal regression models in PyTorch. Contribute to EthanRosenthal/spacecutter development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>from skorch import NeuralNet<br>
import numpy as np<br>
import torch<br>
from torch import nn<br>
from sklearn.model_selection import train_test_split<br>
from spacecutter.callbacks import AscensionCallback<br>
from spacecutter.losses import CumulativeLinkLoss<br>
from spacecutter.models import OrdinalLogisticModel</p>
<p>predictor = nn.Sequential(<br>
nn.Linear(num_features, num_features),<br>
nn.ReLU(),<br>
nn.Linear(num_features, 1)<br>
)</p>
<p>from <a href="http://skorch.net" rel="nofollow noopener">skorch.net</a> import NeuralNet<br>
skorch_model = NeuralNet(<br>
module=OrdinalLogisticModel,<br>
module__predictor=predictor,<br>
module__num_classes=num_classes,<br>
criterion=CumulativeLinkLoss,<br>
train_split=None,<br>
callbacks=[<br>
(‘ascension’, AscensionCallback()),<br>
],<br>
)</p>