Spec'ing a machine for Deep Learning

username: Mole_Turner
cooked: <p>Hi all,</p>
<p>Apologies if this isn’t an appropriate post for the forum, but I figured there would be a wealth of knowledge to tap to.</p>
<p>What specifications would people recommend when buying/building a computer that will mainly be used for machine learning. The project I’m working on has some funding and I want to put together a reasonable machine without going overboard, but also one that won’t end up being underpowered and need updating again in a year.</p>

username: JuanFMontesinos
cooked: <p>Well it’s a tricky question, depends on the budget and what do you work on.<br>
As everything in the life it’s a trade-off.</p>
<p>First think you should take into account is what do you want to work on.<br>
Audio?<br>
Video?<br>
graphs?</p>
<p>I would go for threadrippers (they have plenty of cores at a good quality/price) to take advantage of multiprocessing.<br>
A nvme hard disk, at least SSD, no hdd.<br>
Realise PCIe 4.0 is out.</p>
<p>The delicated thing is how many of everything.<br>
1st thing how many gpus you can afford.<br>
You will need a power supply of 400*n + 200 more or less.</p>
<p>If you are gonna work with video I would get .3090s and at least 64 Gb of RAM.<br>
But in the end there are no rules carved in stone, depends on your budget, how many ppl will use it, research field…</p>
<p>Let me know extra details for extra help.</p>

username: Mole_Turner
cooked: <p>Thanks for the in-depth reply!</p>
<p>I’m currently waiting to find out how much of a budget we have to spend. I’m really hoping it’s at least &lt;=£2k.</p>
<p>With the SSD - how much of a bottleneck will it create if it’s pulling the data from a HDD?</p>
<p>I’ll be working solely with multi-channel audio synthesis/multivariate regression - one of the reasons I’ve struggled trying to spec it myself is that most of the bench marks are done using computer vision models and I’m finding it hard to transfer it across into how they would perform on audio. Though I would assume most systems that perform well on image/video would be sufficient for audio.</p>
<p>How do the threadrippers stack up to the Intel stuff - I’ve no real experience of AMD?</p>
<p>I would assume that unless I go for older GPUs I’ll probably only be able to afford one.</p>
<p>I think at present I’d probably be the only person running stuff on it…however it would be ideal if I could run a couple of models at a time for time efficiency.</p>

username: antonp
cooked: <p>SSD is a must. Remember that majority of your work is preprocessing and transformations which benefit a lot from fast storage. I have 2.5TB of pci-e 3.0 Nvme storage and 3TB HDD. Every time I need to deal with the data on HDD it always bogs down my workflow to the rage inflicting point.</p>
<p>CPU is only marginally important these days as long as you got midrange and up. I have two desktop machines that I’m working on (a few generations old at this point), one is a total beast of it’s time i9-9900k with 5Ghz over clock on all 8cores, the other one is i5-10400 locked at 4.3 on 6cores. Both are hyper-threaded so double the amount for logical cores. I don’t really feel the speed difference whatsoever not in day to day activities nor crunching numbers.</p>
<p>If you can squeeze threadpipper into your budget - you won’t bottlenecking yourself on CPU whatsoever.</p>
<p>Now the main challenge these days is to get a GPU. Grab as many as you can, going 3090 is actually worth it cause it’s not as efficient for mining as lower cards, thus, the demand markups are less. Good luck</p>

username: JuanFMontesinos
cooked: <p>Okay so let me add some extra comments.</p>
<p>If you think more ppl will work with the computer, take into account that datasets may ocuppy a lot.<br>
For working with audio the datasets I’ve played with are very contained (like MUSDB and so).<br>
So you can start with 1 Tb nvme. If you go for audiovisual and need to store video you will lack hard disk.</p>
<p>I think you can go for 32 Gb of RAM in audio field.<br>
Using HDD is a very strong bottleneck. Prob 2 times slower.<br>
You can just check nvme specs vs sdd too and you will see nvme is way faster.</p>
<p>Given your budget, PCIe 4.0 is too expensive atm.<br>
Threadrippers are from  12 to 16 cores (2 threads/core) thus between 24-32 threads.<br>
Multiprocessing lauches 1 process per thread. Therefore the more cores the faster dataloading.</p>
<p>AMD usually offers more pci lanes than intel, where many of their cpus are capped. Although I didn’t try such config like <a class="mention" href="/u/anton">@Anton</a> suggest so cannot give u more hints. However, as he mentions, you don’t need to buy last gen necesarily.</p>
<p>For the motherboard you can look for something with 2 gpu slots so that you can buy 1 gpu when you have some extra money.</p>
<p>In terms of GPUs, 3090 is awesome for both, the amount of VRAM, and the speed.<br>
It’s kinda 50% faster than previous gens but it’s very expensive too.</p>
<p>You need something with at least 12 Gb of VRAM as a U-Net already burns 9 or so.<br>
Atm I find the market a bit ill. New gen GPUs are weirdly expensive and not really worth for deep learning due to the VRAM but 3090. Besides the market is kinda out of stock.</p>
<p>The 1080 ti used to be really worth in terms of quality/price. In fact Nvidia retired those due to that. I saw  this in amazon USA<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f45043ee32178eaff1183e35226c32f051bb935.png" data-download-href="https://discuss.pytorch.org/uploads/default/9f45043ee32178eaff1183e35226c32f051bb935" title="imagen"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f45043ee32178eaff1183e35226c32f051bb935.png" alt="imagen" data-base62-sha1="mIXMG7Gsfi7A3fUhY2r1rwblx5P" width="443" height="500" data-dominant-color="F7F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">imagen</span><span class="informations">671×756 27.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>If that were the price, you can get 2 of these until you can afford.<br>
Otherwise I wouldn’t know what to recommend you.</p>
<p>With a 3090 I guess it’s around 2k5 or so. Dunno if you have to pay taxes or not. W/o taxes can be 2k more or less.</p>
<p>Good luck</p>

username: Mole_Turner
cooked: <p>Unsurprisingly it seems to be the GPU area I’m struggling in. I think ideally I’d want at least 10GB VRAM which incurs a huge price hike compared to the 8GB cards currently available, and that’s the only thing which may push me over budget. But I’m not sure if there are many/any benefits with getting a machine with a single 8GB GPU over just using some of the cloud services. I’ve played around on Colab but the limited runtimes and fair usage policy means I don’t want to rely on it for my entire project.</p>

username: Mole_Turner
cooked: <p>So, my lab has a machine with the below specs, and looking at things like “compute capability” I’m not really sure what they mean.</p>
<p>Processor - dual Xeon e5-2670<br>
RAM - 256GB<br>
GPU - Quatro FX 5800</p>
<p>I’m aware this is all pretty old. But wondering if it’s not too out of date to still be useful.</p>

username: JuanFMontesinos
cooked: <p>Soo the GPU is not useful but it was a server cpu so it was very powerful back ago.<br>
It’s prob DDR3 memory soo the bandwith is less but you can try to plug a modern GPU and do a benchmark to check.</p>
<p>Audio data loading is not really demanding so you may be fine if you get a ssd/nvme disk.</p>

username: Mole_Turner
cooked: <p>This website was recommended to me and also might be of use to others. It’s a pretty comprehensive guide to putting together hardware for ML/DL applications and where people commonly go wrong/spend money in the wrong places.</p>
<aside class="onebox allowlistedgeneric">
  <header class="source">
      <img src="https://i0.wp.com/timdettmers.com/wp-content/uploads/2020/09/cropped-tim_dettmers_square-min.png?fit=32%2C32&amp;ssl=1" class="site-icon" width="16" height="16">
      <a href="https://timdettmers.com/2018/12/16/deep-learning-hardware-guide/#Hard_driveSSD" target="_blank" rel="noopener nofollow ugc" title="06:25PM - 16 December 2018">Tim Dettmers – 16 Dec 18</a>
  </header>
  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:603/406;"><img src="https://i2.wp.com/timdettmers.com/wp-content/uploads/2015/03/cpu_underclocking2.png?fit=603%2C406&amp;ssl=1" class="thumbnail" width="603" height="406"></div>

<h3><a href="https://timdettmers.com/2018/12/16/deep-learning-hardware-guide/#Hard_driveSSD" target="_blank" rel="noopener nofollow ugc">A Full Hardware Guide to Deep Learning — Tim Dettmers</a></h3>

<p>In this guide I analyse hardware from CPU to SSD and their impact on performance for deep learning so that you can choose the hardware that you really need.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: Parveen_Kr90
cooked: <p>For a solid DL, ML machine:</p>
<p>CPU: AMD Ryzen 9 7950X or Intel i9-13900K – good core count and performance.<br>
GPU: NVIDIA RTX 3090 or 4090 – 24GB VRAM is great for ML workloads.<br>
RAM: 64GB (or 128GB if budget allows) – ensures smooth multitasking.<br>
Storage: 1TB NVMe SSD (for OS and projects) + larger SSD/HDD for data.</p>
<p>This setup balances performance and future-proofing without going overboard.</p>
<p>Also, check these recommended workstations for <a href="https://www.proxpc.com/workstation/artificial-intelligence" rel="noopener nofollow ugc">Machine Learning and Deep Learning</a>  from ProX PC <a href="https://www.proxpc.com/" rel="noopener nofollow ugc">www.proxpc.com</a></p>

username: Emily_Smith
cooked: <aside class="quote no-group" data-username="Mole_Turner" data-post="1" data-topic="120411" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mole_turner/48/37337_2.png" class="avatar"> Mole_Turner:</div>
<blockquote>
<p>Hi all,</p>
<p>Apologies if this isn’t an appropriate post for the forum, but I figured there would be a wealth of knowledge to tap to.</p>
<p>What specifications would people recommend when buying/building a computer that will mainly be used for machine learning. The project I’m working on has some funding and I want to put together a reasonable machine without going overboard, but also one that won’t end up being underpowered and need updating again in a year.</p>
</blockquote>
</aside>
<p>No need to apologize—this is a great question! When spec’ing a machine for deep learning, here are some key recommendations to balance performance and cost:</p>
<ol>
<li><strong>GPU:</strong> The most critical component for deep learning. Go for an NVIDIA GPU with CUDA support, like the RTX 4090, 4080, or A6000 for high performance. For budget options, look at the RTX 3060/3070.</li>
<li><strong>CPU:</strong> A decent multi-core CPU like AMD Ryzen 9 or Intel i7/i9 is sufficient. Deep learning doesn’t heavily rely on CPU, so prioritize the GPU instead.</li>
<li><strong>RAM:</strong> At least 32GB, but 64GB is better for handling large datasets.</li>
<li><strong>Storage:</strong> Use an NVMe SSD with at least 1TB for fast read/write speeds, paired with an HDD for additional storage if needed.</li>
<li><strong>Power Supply:</strong> Ensure your PSU can handle the GPU power requirements—around 850W+ is usually sufficient.</li>
<li><strong>Cooling:</strong> Invest in good cooling (liquid or air) to keep temperatures under control during long training sessions.</li>
<li><strong>Motherboard:</strong> Pick one with ample PCIe lanes to support your GPU and future upgrades.</li>
</ol>
<p>If you’re working with large models, consider cloud services like AWS or Google Cloud for scalability. This way, you can avoid overspending upfront and scale as needed.</p>
<p>Good luck building your machine! Let us know how it goes. <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20"></p>