CPU/GPU/TPU and the beloved PyTorch

username: blackbirdbarber
cooked: <p>Is there anything more except CPU/GPU/TPU where PyTorch can work?<br>
Simple and clear but I cannot find the answer for now.</p>

username: Kushaj
cooked: <p>It can work on microcontrollers, maybe fpga (with some hacks).</p>

username: blackbirdbarber
cooked: <p>Aha, is there any link on where PyTorch can work?</p>

username: Kushaj
cooked: <p>For microcontrollers I have mostly seen onnx exported models or torchscript models. You can check <a href="https://github.com/ai-techsystems/deepC" rel="nofollow noopener">deepC</a>. They use onnx exported models which is not pytorch directly but you can kind of use pytorch to onnx export.</p>
<p>I was referring to examples more on the deployment side where you can use c++ to deploy models.</p>

username: blackbirdbarber
cooked: <p><a class="mention" href="/u/kushaj">@Kushaj</a> it can work on arm <a href="https://discuss.pytorch.org/t/does-pytorch-support-arm-processor-aarch64/1786" class="inline-onebox">Does Pytorch support ARM processor(aarch64)?</a>, which is Raspberry cake architecture. Some guy <span class="mention">@smt</span> h confirmed that.</p>
<p>I am interested if can work on amd as well.<br>
I found some <a href="https://discuss.pytorch.org/t/pytorch-usability-on-amd-hardware/32895/2" class="inline-onebox">PyTorch usability on AMD hardware?</a> link but they don’t say if it can work or no.</p>
<p>Also I am interested if can work on DSP processors. When I find documents <a href="https://on-demand.gputechconf.com/gtc-cn/2019/pdf/CN9624/presentation.pdf" rel="nofollow noopener">https://on-demand.gputechconf.com/gtc-cn/2019/pdf/CN9624/presentation.pdf</a> like this I am completely lost. There is a mention but no feedback.</p>
<p>Also I am interested to know if PyTorch can work on FPGA. There is a link I found <a href="https://discuss.pytorch.org/t/pytorch-and-fpga/14687" class="inline-onebox">Pytorch and FPGA</a> but still very confused.</p>
<p>Also I am interested if can work on anything else.<br>
Thanks</p>

username: blackbirdbarber
cooked: <p>So what you are saying is it  can work on other devices <strong>just</strong> via ONNX?</p>

username: tom
cooked: <p>I’d take a look at <a href="https://tvm.ai/" rel="nofollow noopener">TVM</a>. The target FPGA and you can take PyTorch JITed models as the basis.</p>

username: blackbirdbarber
cooked: <p><a class="mention" href="/u/tom">@tom</a>, thanks, TVM is still not the buzz word. At least for me, and I think I saw it but I haven’t realized the importance of this project.</p>
<p>From what I can see one of the main components it has is VTA.</p>
<p>Certainly CUDA is an option, on the other side, it is unclear to me how this can be plugged on HHVM, but OK. Metal may be the Apple GPU, like HIP is for Amd, which is missing in this diagram.</p>
<p>So you run in VTA graph optimization, and you just send the script export from PyTorch. Right. So the whole idea of TVM is export scripts from PyTorch to JIT. Right.</p>
<p>What I heard on some classes of FastAi they downgraded JIT technology and this gave some bias on my understanding of things.</p>
<p>What you are saying in other words, JIT is the future. Right.</p>
<p>Thanks</p>

username: Kushaj
cooked: <p>You can use PyTorch on AMD. It requires quite some work (if you do not want to use the official conda build). I will be getting my new PC in 7 days which has AMD CPU. I will document the entire process to setup PyTorch on that.</p>
<p>PyTorch can work on ARM. I did this a long time ago. Took a rasberry pi and installed pytorch on it and it worked. I installed from a wheel that I found online (you can compile also).</p>

username: blackbirdbarber
cooked: <aside class="quote no-group" data-username="Kushaj" data-post="9" data-topic="91379">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kushaj/48/58838_2.png" class="avatar"> Kushaj:</div>
<blockquote>
<p>I will be getting my new PC in 7 days which has AMD CPU. I will document the entire process to setup PyTorch on that.</p>
</blockquote>
</aside>
<p>Very proactive. I am glad you will do that, and share the experience. Should we except also some testing AMD HIP for AMD GPU.</p>

username: tom
cooked: <aside class="quote no-group" data-username="blackbirdbarber" data-post="10" data-topic="91379">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/6de8d8/48.png" class="avatar"> blackbirdbarber:</div>
<blockquote>
<p>Very proactive. I am glad you will do that, and share the experience. Should we except also some testing AMD HIP for AMD GPU.</p>
</blockquote>
</aside>
<p>Note that not all AMD GPUs work with HIP/ROCm: Vega and Polaris do, notably Navi does not.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Kushaj
cooked: <aside class="quote no-group" data-username="blackbirdbarber" data-post="10" data-topic="91379">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/b/6de8d8/48.png" class="avatar"> blackbirdbarber:</div>
<blockquote>
<p>Should we except also some testing AMD HIP for AMD GPU.</p>
</blockquote>
</aside>
<p>For the GPU, I personally think Nvidia has a monopoly (or you can say <strong>THE ONLY OPTION</strong>), if you want to keep your life easy.</p>
<p>For the CPU, I think Intel is the much much easier option (you just install from conda and you don’t have to worry about anything). When doing research for AMD setup on Ubuntu I have already seen some links that make me feel it would be a long long day.</p>
<p>I think for the CPU it is money v/s convenience thing.</p>