Non-ML models with ignite

username: pytorchnewbie
cooked: <p>In research, a lot of the time we will be comparing machine learning approaches to non-machine learning approaches. For the latter, there isn’t usually a notion of “iterations” or “epochs”. The model does what it does and outputs an error. Since the idea would be to code all the experiments in the same framework, I was wondering how to use ignite for non-learning based approaches/if it makes sense to do that. For instance, does it make sense to use ignite’s “metrics” class for a non-learning based approach? Does using any of ignite make sense for non-learning approaches?</p>
<p>Thanks!</p>

username: vfdev-5
cooked: <p>Hi <a class="mention" href="/u/pytorchnewbie">@pytorchnewbie</a></p>
<blockquote>
<p>The model does what it does and outputs an error. Since the idea would be to code all the experiments in the same framework, I was wondering how to use ignite for non-learning based approaches/if it makes sense to do that. For instance, does it make sense to use ignite’s “metrics” class for a non-learning based approach? Does using any of ignite make sense for non-learning approaches?</p>
</blockquote>
<p>Probably, it depends on details etc (if you could provide some, it would help), but I think this can be just considered as inference phase: a model (any kind of) is used to compute predictions over the batches of the test dataset and we wish to save the outputs</p>
<pre data-code-wrap="python"><code class="lang-python">def inference_step(engine, batch):
    sample, meta_data, what_ever_data = batch
    result = non_ml_model(sample)
    return {
        "result": result,        
    }

inference_engine = Engine(inference_step)

@inference_engine.on(Events.ITERATION_COMPLETED)
def save_result(engine):
    sample, meta_data, what_ever_data = engine.state.batch
    result = engine.state.output["result"]
    # do something to save the result

</code></pre>
<p>Another example of using Engine can be to compute mean/std of the dataset (no ML-model at all):</p>
<pre><code class="lang-auto">from ignite.metrics import Average

def compute_mean_std(engine, batch):
    b, c, *_ = batch['image'].shape
    data = batch['image'].reshape(b, c, -1).to(dtype=torch.float64)
    mean = torch.mean(data, dim=-1).sum(dim=0)
    mean2 = torch.mean(data ** 2, dim=-1).sum(dim=0)
    return {"mean": mean, "mean^2": mean2}

compute_engine = Engine(compute_mean_std)
img_mean = Average(output_transform=lambda output: output['mean'])
img_mean.attach(compute_engine, 'mean')
img_mean2 = Average(output_transform=lambda output: output['mean^2'])
img_mean2.attach(compute_engine, 'mean2')
state = compute_engine.run(train_loader)
state.metrics['std'] = torch.sqrt(state.metrics['mean2'] - state.metrics['mean'] ** 2)
mean = state.metrics['mean'].tolist()
std = state.metrics['std'].tolist()
</code></pre>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://pytorch.org/ignite/engine.html#ignite.engine.engine.Engine">
  <header class="source">
      <img src="https://pytorch.org/ignite/_static/ignite_logomark.svg" class="site-icon" width="250" height="250">

      <a href="https://pytorch.org/ignite/engine.html#ignite.engine.engine.Engine" target="_blank" rel="noopener nofollow ugc">PyTorch-Ignite</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/344;"><img src="https://raw.githubusercontent.com/pytorch/ignite/master/assets/logo/ignite_logo.png" class="thumbnail" width="690" height="344"></div>

<h3><a href="https://pytorch.org/ignite/engine.html#ignite.engine.engine.Engine" target="_blank" rel="noopener nofollow ugc">ignite.engine — PyTorch-Ignite v0.4.13 Documentation</a></h3>

  <p>High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Let me know if this helps. Otherwise, please provide a minimal code snippet to understand better the question…</p>

username: pytorchnewbie
cooked: <p>Thank you so much! I am figuring out something about the non-ml model and will respond to this in a day or two with more details.</p>