Pytorch support for M1 Mac GPU

sssingh: <p>Hi,</p>
<p>Sometime back in Sept 2021, a post said that PyTorch support for M1 Mac GPUs is being worked on and should be out soon. Do we have any further updates on this, please?</p>
<p>Thanks.<br>
Sunil</p>

tcapelle: <p>For the moment, TF works pretty well:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://wandb.ai/tcapelle/apple_m1_pro/reports/Deep-Learning-on-the-M1-Pro-with-Apple-Silicon---VmlldzoxMjQ0NjY3">
  <header class="source">
      <img src="https://cdn.wandb.ai/production/296bb56c9/favicon.png" class="site-icon" width="32" height="32">

      <a href="https://wandb.ai/tcapelle/apple_m1_pro/reports/Deep-Learning-on-the-M1-Pro-with-Apple-Silicon---VmlldzoxMjQ0NjY3" target="_blank" rel="noopener nofollow ugc" title="03:08PM - 19 November 2021">W&amp;B – 19 Nov 21</a>
  </header>

  <article class="onebox-body">
    <img src="https://storage.googleapis.com/wandb-production.appspot.com/tcapelle/images/views/1244667/preview.png?md5=8e318138ba671d375f4ac9f1e99e1411" class="thumbnail onebox-avatar" width="300" height="300">

<h3><a href="https://wandb.ai/tcapelle/apple_m1_pro/reports/Deep-Learning-on-the-M1-Pro-with-Apple-Silicon---VmlldzoxMjQ0NjY3" target="_blank" rel="noopener nofollow ugc">Deep Learning on the M1 Pro with Apple Silicon</a></h3>

  <p>Let's take my new Macbook Pro for a spin and see how well it performs, shall we?. Made by Thomas Capelle using Weights &amp; Biases</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>even pure numpy is really fast with the right compiler flags</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://tlkh.dev/benchmarking-the-apple-m1-max">
  <header class="source">
      <img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1611242173172/AOX1gE2jc.png" class="site-icon" width="32" height="32">

      <a href="https://tlkh.dev/benchmarking-the-apple-m1-max" target="_blank" rel="noopener nofollow ugc">Timothy Liu's Blog</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/362;"><img src="https://hashnode.com/post/cover/ckvzio0r901ye1bs1ddfx98e1" class="thumbnail" width="690" height="362"></div>

<h3><a href="https://tlkh.dev/benchmarking-the-apple-m1-max" target="_blank" rel="noopener nofollow ugc">Benchmarking the Apple M1 Max</a></h3>

  <p>Understanding the Hardware Capabilities of Apple's flagship SOC</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Hope to see PyTorch soon, I am loving the new DataPipes and functorch.</p>

Gregg: <p>I’d like to draw attention to this question since I would also like it answered! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

khasmamad: <p>I am curious as well.</p>

FrankHeijden: <p>Same! Would love to see support for this <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

Lenno93: <p>Same here! GPU Support on M1 with PyTorch would be a game changer for me</p>

paantya: <p>its supports!</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/">
  <header class="source">
      <img src="https://pytorch.org/favicon.ico?" class="site-icon" width="32" height="32">

      <a href="https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/388;"><img src="https://pytorch.org//assets/images/METAPT-002-BarGraph-02-static.png" class="thumbnail" width="690" height="388"></div>

<h3><a href="https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/" target="_blank" rel="noopener nofollow ugc">Introducing Accelerated PyTorch Training on Mac</a></h3>

  <p>In collaboration with the Metal engineering team at Apple, we are excited to announce support for GPU-accelerated PyTorch training on Mac. Until now, PyTorch training on Mac only leveraged the CPU, but with the upcoming  PyTorch v1.12 release,...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


patniemeyer: <p>I am getting mixed results:  On my Mac Studio Ultra I have installed the latest conda and the nightly torch build.  Out of the box in CPU mode one little experiment was running about 8x faster than before (also CPU) so that is nice.  However asking for “mps” the performance dropped to 8x <em>worse</em> than the old CPU perf…  (64x worse than the new CPU perf).   On other projects I got errors related to broadcast incompatibilities… All of these could be my problems but these do work with CUDA (on another machine).  Anyway, just FYI.  Looking forward to using this as it improves.  Thanks to the team who made this possible!</p>

Maxhirez: <p>Are there any new attributes for using M1/Metal native?  I got the latest nightly and played with it a little bit, but couldn’t find any evidence that the SOC GPU was engaged.  For example, torch.cuda.is_available() is still false, which I guess you’d expect, but if I create a tensor, tensor.device still shows CPU.  All torch.device() are still set to cuda.  Any pointers?  Is it working on a metal GPU now without saying it’s working on a gpu or something?<br>
Thanks!</p>

PBAL: <p>Similar here: With MBP M1 Max 10 CPU core, 32 GPU core, 64GB RAM, the new PyTorch nightly build 1.13.0.dev20220620 is 3x faster with the CPU than my old version 1.10.0 using the same CPUs. Speed using GPU is terrible in comparison.</p>
<p>Also interesting, when looking at the 10 CPU cores’ usage, with 1.13 they are using ~15%. With the 1.10 version, they are all maxed out.</p>
<p>Clearly more is being optimized for the M1 chip in the new nightly build than just GPU support. The results are startling.</p>

patniemeyer: <p>Tried again with the latest nightly builds.  This time I used the pytorch <code>transformer_tutorial.py</code> example.  Same general results: an RTX3090 runs it in 1 minute (rounding up).  “cpu” mode on the Ultra M1 is 32 minutes and “mps” mode is 46 minutes.  I am concerned / hopeful that I am just doing something wrong in setup because I have yet to see an example that runs faster in “mps” mode than “cpu” mode.  If anyone has any suggestions I’d appreciate them.  Thanks.  (Again, all praise to the team working on this and I hope this feedback is constructive).</p>