Current state of MPS

adamjstewart: <p>I’m considering purchasing a new MacBook Pro and trying to decide whether or not it’s worth it to shell out for a better GPU.</p>
<p>Last I looked at PyTorch’s MPS support, the majority of operators had not yet been ported to MPS, and <code>PYTORCH_ENABLE_MPS_FALLBACK</code> was required to train just about any model. The additional overhead of data transfer between MPS and CPU resulted in MPS training actually being slower than CPU training. I also saw some reports that there were issues with training accuracy on MPS.</p>
<p>Obviously, a lot of hard work has happened since then, and more operators are ported to MPS every week. So I’m wondering if anyone down in the trenches can give a “State of the Union” for MPS support. I’m particularly interested in the following questions:</p>
<ol>
<li>Is MPS still slower than CPU due to <code>PYTORCH_ENABLE_MPS_FALLBACK</code> being required?</li>
<li>Is there an approximate timeline for how long you think it will be until MPS is faster than CPU?</li>
<li>Is there an approximate timeline for how long you think it will be until (almost) all operators have been ported to MPS?</li>
<li>What’s the theoretical performance bump we can reasonably expect once 3 is complete?</li>
</ol>
<p>My workflow is relatively simple. Using PyTorch Lightning and TorchGeo to train ResNet and ViT models from timm and segmentation-models-pytorch. Bonus points if anyone has actually done benchmarking for CPU vs. MPS performance.</p>

oat: <p>Same here. I also want to know if it’s worthwhile to buy M2 Ultra for training with MPS utilized.</p>

AlexP4499: <p>This is anecdotal, and running on a custom data.</p>
<p>I am running training on a model using a Dell XPS 9710 (3060), the training process is only using around 600mb of GPU memory. I am finding that an epoch is taking around 14second.</p>
<p>Using a Macbook Pro M1 Max(32), I am finding the same epoch to take around 40seconds.</p>
<p>The delta is significantly larger than I expected, and I’m still looking into it - the power draw to the Apple silicon seems too low… and I’m not entirely sure why this is. The Nvidia 3060(mobile) draws 62/63watts which is about as high as Dell allow it to pull, possibly a couple of watts below running a benchmark/graphics - but pretty much flat out, so the code is ok.</p>
<p>For comparison, the M1 Max CPU is taking around 300 seconds per epoch, and seems to be struggling to get the fans going either.</p>
<p>Perhaps it is me being an Apple noob; but I have switched it to the Max power setting, and plugged it into &gt;=100watts.</p>
<p>Also, check out Alex’s videos (different Alex, not a plug) - he runs a few benchmarks that are of interest. Generally, Apple Silicon performs well…</p><div class="youtube-onebox lazy-video-container" data-video-id="0wxobKsnf2o" data-video-title="Apple M3 Max MLX beats RTX4090m" data-video-start-time="" data-provider-name="youtube">
  <a href="https://www.youtube.com/watch?v=0wxobKsnf2o" target="_blank" class="video-thumbnail" rel="noopener nofollow ugc">
    <img class="youtube-thumbnail" src="https://img.youtube.com/vi/0wxobKsnf2o/maxresdefault.jpg" title="Apple M3 Max MLX beats RTX4090m" width="690" height="388">
  </a>
</div>