Correct machine to buy that works with PyTorch

username: paulocoutinho
cooked: <p>Hi,</p>
<p>I want buy a machine (PC) with GPU that works with PyTorch and have great performance for my Ai project.</p>
<p>Today, my server in my home is running Ai things with CPU what take a long time to finish (around 1 minute). I want solve with a decent PC with GPU that works with PyTorch.</p>
<p>What is the correct GPU and system (Windows, Linux etc) to get better performance?</p>
<p>Can someone help me with this?</p>
<p>Thanks.</p>

username: JuanFMontesinos
cooked: <p>It all depends on your budget <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>Linux is for sure the OS to go.</p>

username: paulocoutinho
cooked: <p>Yes, surely, but something that i can start.</p>
<p>Whats is a recommended machine decent for generate somethings, in my case i need generate 1.000 images by minute.</p>
<p>Example: Ubuntu + RTX 2080 8GB + 16GB RAM + core i7 3Ghz</p>
<p>It will sufficient? It is hard to me have an idea about need resource.</p>
<p>Furthermore, there is still the compatibility problem. Some video cards don’t have the right driver for Linux or have a similar problem, so I don’t really know what makes sense and is compatible on Linux (Ubuntu).</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="paulocoutinho" data-post="3" data-topic="205724">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/paulocoutinho/48/71182_2.png" class="avatar"> paulocoutinho:</div>
<blockquote>
<p>Furthermore, there is still the compatibility problem. Some video cards don’t have the right driver for Linux or have a similar problem, so I don’t really know what makes sense and is compatible on Linux (Ubuntu).</p>
</blockquote>
</aside>
<p>It’s unclear what this refers to, but the current PyTorch binaries support all NVIDIA GPUs from Maxwell to the current Hopper generation. Obviously I would not recommend trying to buy a 10 year old Maxwell GPU.</p>
<aside class="quote no-group" data-username="paulocoutinho" data-post="3" data-topic="205724">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/paulocoutinho/48/71182_2.png" class="avatar"> paulocoutinho:</div>
<blockquote>
<p>in my case i need generate 1.000 images by minute.</p>
</blockquote>
</aside>
<p>Search for public benchmarks similar to your setup.</p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/paulocoutinho">@paulocoutinho</a> just to add to what <a class="mention" href="/u/ptrblck">@ptrblck</a> i have successfully run pytorch on even i3-8GB + RTX 1060Ti 4GB <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

username: paulocoutinho
cooked: <p>I want test on my notebook now with “NVIDIA Geforce MX110” with 2GB. Is a Lenovo Ideapad S145. I only want test that it will work to i buy a real machine after.</p>

username: JuanFMontesinos
cooked: <p>So in short, what typically bounds you is GPU memory.<br>
If you don’t have enough, there is not much you can easily do.<br>
From that point of view, you can try to acquire a 4090 that shuld have 24 Gb RAM. Or 3090 second hand and so on. I’d say you can train anything but huge LLMs, still you can finetune many with fancy techniques.</p>
<p>You should have no problem regarding compatibility, all Nvidia GPUs are supported but too old models. Something from the last 2 gens should be ok.</p>
<p>Another very important point is the hard disk, you may want to have a ssd m2 nvme so that you have fast data loading. A bad hard disk can reaaally bottleneck your process.</p>
<p>So given a budget, the most expensive component would try to get one with decent like 12 - 24 Gb.<br>
Then save some decent money for the hard disk and lastly fit the CPU and remaining components.</p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/juanfmontesinos">@JuanFMontesinos</a>  on the same note, have you given a thought to having multiple cheap GPUs in comparison to having an expensive large GPU</p>

username: paulocoutinho
cooked: <p>Thanks guys. I tried in my 2GB MX110 but have only error about low memory.</p>
<p>What im trying now is use “runpod” service to use GPU as a service.</p>
<p>I will post here the results.</p>

username: JuanFMontesinos
cooked: <p>They used to be worth when nvlink was available on consumer-grade GPUs. They removed this feature from most consumer-grade gpus to push ML practitioners towards more expensive models.</p>
<p>IMO it’s a trade off. With several GPUs you can do several experiments in parallel, but less memory implies you cannot train larger models. Most typical multi-gpu schema works by duplicating the model per GPU, therefore a model should fit in each GPU. But ofc, having tons of memory is useless if not enough compute capabilities as it would take ages to train.</p>