Engine run with multi inputs

Artur_Punch: <p>I am solving multilabel segmentation task. I want to train model with images and tabular data. So i have custom model class with multi inputs</p>
<pre><code class="lang-auto">class ContextNet(Module):
  def __init__(self, classes=10, features=10, dropout=0.1):
    super(ContextNet, self).__init__()
....
  def forward(self, image, predictions):
....
    return 
</code></pre>
<p>so i need data generator which returns image, tabular data and mask. What is correct way to do that?<br>
When i do such generator:</p>
<pre><code class="lang-auto">class gen_context(Dataset):
    def __init__(self, root, val=False):
        self.val = val
        self.map = array(listdir(root + 'Images/'))

    def __len__(self):
        return len(self.map)

    def __getitem__(self, index):
...
        return image, tabular, mask
</code></pre>
<p>i got</p>
<pre><code class="lang-auto">  File "/EFSDATA/fermata_classification/src/modules/train/train.py", line 143, in main_trainer
    trainer.run(train_loader, max_epochs=Params.epochs_to_stop)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 704, in run
    return self._internal_run()
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 783, in _internal_run
    self._handle_exception(e)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 466, in _handle_exception
    raise e
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 753, in _internal_run
    time_taken = self._run_once_on_dataset()
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 854, in _run_once_on_dataset
    self._handle_exception(e)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 466, in _handle_exception
    raise e
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 840, in _run_once_on_dataset
    self.state.output = self._process_function(self, self.state.batch)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 101, in update
    x, y = prepare_batch(batch, device=device, non_blocking=non_blocking)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 37, in _prepare_batch
    x, y = batch
ValueError: too many values to unpack (expected 2)
</code></pre>
<p>in other variants i got</p>
<pre><code class="lang-auto">  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/engine.py", line 840, in _run_once_on_dataset
    self.state.output = self._process_function(self, self.state.batch)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 102, in update
    y_pred = model(x)
  File "/home/devuser/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'predictions'
</code></pre>

vfdev-5: <p>Hi <a class="mention" href="/u/artur_punch">@Artur_Punch</a>,</p>
<p>according to traceback line:</p>
<pre><code class="lang-auto">File "/home/devuser/miniconda3/lib/python3.9/site-packages/ignite/engine/__init__.py", line 102, in update
</code></pre>
<p>you should be using <code>create_supervised_trainer</code> which is a helper function to create a trainer for a simple supervised task.</p>
<p>In your case you can create a trainer directly using <a href="https://pytorch.org/ignite/generated/ignite.engine.engine.Engine.html#ignite.engine.engine.Engine" rel="noopener nofollow ugc">Engine API</a>:</p>
<pre><code class="lang-python">model = ...
model = model.cuda()
optimized = ...
criterion = ...


def train_step(engine, batch):
    model.train()
    optimizer.zero_grad()

    image, tabular, mask = batch[0].cuda(), batch[1].cuda(), batch[2].cuda()

    outputs = model(image, tabular)   # or whatever logic you have for your model
    loss = criterion(outputs, mask)
    loss.backward()
    optimizer.step()

    return loss.item()
    
trainer = Engine(train_step)
</code></pre>
<p>Let me know if it works for you?</p>