An Affordable GPU for Data Scientists
By: Naser Tamimi
Tags: AI, Data Science, Machine Learning, Gpu, Data Scientist

NVIDIA GPUs are evolving through time and the main aspect of this evolution is their architecture. NVIDIA GPU architectures from the oldest to the newest are Tesla (released in 2006), Fermi (2010), Kepler (2012), Maxwell (2014), Pascal (2016), Volta (2017), Turing (2018), and Ampere (2020). Some of these architectures are toward data centers and professional users and some of them are more toward regular consumers and especially gamers. These days, most of the NVIDIA GPUs that are available are Pascal, Volta, Turing, or Ampere based.

An important innovation in NVIDIA GPU architecture that took the attention of data scientists and artificial intelligence (AI) experts was the introduction of Tensor Cores. In layman’s terms, tensor cores speed up matrix multiplication and addition. As a data scientist, you know the importance of matrix multiplication and addition in AI technologies such as deep learning. So, to have the best bang for your buck, I highly suggest choosing GPUs that are built on Volta, Turing, and Ampere architecture. It is important to mention that not every graphic card based on these architectures have tensor cores (you see an example later). The graphic cards that have tensor cores are usually denoted with RTX. The graphic cards that do not have tensor cores are usually (not always) denoted with GTX.

Tesla (released in 2006), Fermi (2010), Kepler (2012), Maxwell (2014), Pascal (2016), Volta (2017), Turing (2018), and Ampere (2020) are NVIDIA GPUs architectures

In addition to architecture, NVIDIA graphic cards have different brands or line of products for different customer targets. The three most important ones for us are GeForce, Quadro, and Titan. Although NVIDIA recommends Quadro and Titan for AI and deep learning projects, their price tags ($2500 and above) make them unaffordable for individual or hobbyist data scientists. On the other hand, we have the GeForce brand that targets gamers and is more affordable and still uses the same advanced architecture technologies like Turing or Ampere. Therefore, I recommend to choose from GeForce graphic cards rather than spending too much on expensive cards like Quadro brand cards.

GeForce cards are coming in different series. The most famous ones are GeForce 10 series, GeForce 16 series, GeForce 20 series, and new GeForce 30 Series. GeForce 10 series are Pascal-based cards. As mentioned before, Pascal-based GPUs do not have tensor core technology, therefore these graphic cards are usually called something like GeForce GTX MODEL NUMBER (example GeForce GTX 1060). Sometimes you might see “Ti” or “Super” in front of graphic card names in this series, it just shows a small improvement to the non-Ti or non-Super versions. We also have the GeForce 16 series. This series of graphics cards are built with Turing-based GPUs but the Tensor and RT (Ray Tracing) cores are not included. Therefore, like Series 10, they have GTX in their name. An example of this card is GeForce GTX 1650. Although GeForce 10 and 16 series do not have tensor cores, they are still good options for those who want to learn deep learning and parallel computing. Lack of tensor cores slows down the training and inference of many machine learning methods including DL algorithms, but still having NVIDIA GPUs has a significant advantage to run on CPU only. Series 10 and 16 start from $160 and is recommended if you want to learn.

NVIDIA GeForce GPUs. The table compares 10, 16, 20, and 30 Series. For deep learning and parallel computing, I recommend RTX graphics cards(20 and 30 Series).

If you are thinking of a more robust graphic card that helps you to not only learn parallel computing and deep learning but also running heavier DL projects, you should consider GeForce 20 series. GeForce 20 series are Turing-based graphic cards that have tensor cores. As you guessed correctly, these cards have RTX in their name. For example, you can buy a GeForce RTX 2060 graphic card, which has 240 tensor cores, for $330. This is a graphic card that I recommend for individual data scientists who want an affordable but still robust GPU system.

NVIDIA RTX 2060 Founders Edition.

And finally, we have the newly released GeForce 30 series cards (including 3070, 3080, and 3090) are Ampere-based cards. Graphic cards with Ampere-based GPUs were released in September 2020. Ampere-based cards enjoy 3rd generation tensor cores. Although their price starts from $499 (a good price for the latest technology), they require heavy investment in other PC components such as CPU and PSU (power supply unit). In other words, if you want to buy such a high-end graphics card, probably your affordable PC has a processing bottleneck somewhere else that you cannot use the maximum power of your graphic card. Therefore, if you decide to build an affordable PC or upgrade your PC with a new graphic card, it is better to avoid expensive and high-end graphics cards such as GeForce Series 30, unless you want to upgrade everything and build an expensive but powerful PC.

NVIDIA RTX 3070, 3080, and 3090 (from left to right) Founders Edition.

In addition to architecture, brand, and models, you may find NVIDIA GPUs on different graphic cards made by different manufacturers. NVIDIA makes its own graphic cards and they call them Founders Edition. In addition, GIGABYTE, EVGA, ASUS, and MSI are a few examples of third party manufacturers that build graphic cards using NVIDIA GPUs. In terms of performance, usually, we don’t see too much difference between different manufacturers if they use the same NVIDIA GPUs.

Top NVIDIA Graphics Card Manufacturers (Source)

To sum up, if you are looking for an affordable but powerful graphic card, I suggest RTX 2060. It starts from $330 and has tensor cores for accelerating deep learning projects.