Why use GPU with Neural Networks?
Channel: CodeEmporium
Date: 2020-04-06

Start with an analogy. Then delve into CUDA with some pytorch code to demonstrate why we use GPUs instead of just CPUs.

REFERENCES
[1] Why GPU’s work well in deep learning: https://www.quora.com/Why-are-GPUs-well-suited-to-deep-learning
[2] Matrix math on a GPU:   http://www.ijcee.org/vol9/949-E1621.pdf
[3] GPUs are not necessarily faster than CPUs: https://www.quora.com/Why-are-GPUs-more-powerful-than-CPUs
[4] Code for difference: https://medium.com/analytics-vidhya/using-pytorch-and-cuda-for-large-computation-in-google-colabs-f1c026c17673
[5] GPU memory architecture: https://medium.com/@ashanpriyadarshana/cuda-gpu-memory-architecture-8c3ac644bd64