How to parallelize the evaluation of several versions of the same model

username: pascal_notsawo
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fdaee889295684da26fa72a83a90b0b5ee836f53.png" data-download-href="https://discuss.pytorch.org/uploads/default/fdaee889295684da26fa72a83a90b0b5ee836f53" title="algo"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdaee889295684da26fa72a83a90b0b5ee836f53_2_690x323.png" alt="algo" data-base62-sha1="AcbxrwTEd58cImMwV2q1FVp1jk7" width="690" height="323" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdaee889295684da26fa72a83a90b0b5ee836f53_2_690x323.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdaee889295684da26fa72a83a90b0b5ee836f53_2_1035x484.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/d/fdaee889295684da26fa72a83a90b0b5ee836f53_2_1380x646.png 2x" data-dominant-color="FAFAFA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">algo</span><span class="informations">2025×949 128 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I would like to know how to parallelize this algorithm.</p>
<p>In my current implementation :</p>
<ul>
<li>I am looping over <code>x</code> and <code>y</code> like in the pseudo code, and it takes too much time when <code>n</code> and <code>m</code> are high.</li>
<li>I am using <code>torch.nn.utils.parameters_to_vector</code> to convert my parameters to vector, then disrupting and converting back to parameter with <code>torch.nn.utils.vector_to_parameters</code>.</li>
<li>I am using pytorch lightning for evaluation, <code>loss = trainer.validate(model, dataloader)</code></li>
</ul>
<p>It is even slower in this second case.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/b/cb88070bc85c1be699e5c546060b13a28caafcb2.png" data-download-href="https://discuss.pytorch.org/uploads/default/cb88070bc85c1be699e5c546060b13a28caafcb2" title="algo2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb88070bc85c1be699e5c546060b13a28caafcb2_2_690x427.png" alt="algo2" data-base62-sha1="t2wmzHtJlQMmMFXNvjqUFsdTKJY" width="690" height="427" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb88070bc85c1be699e5c546060b13a28caafcb2_2_690x427.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb88070bc85c1be699e5c546060b13a28caafcb2_2_1035x640.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/b/cb88070bc85c1be699e5c546060b13a28caafcb2_2_1380x854.png 2x" data-dominant-color="FBFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">algo2</span><span class="informations">1761×1090 163 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>