Introduction to deep learning hardware in the cloud
Tags: aws, ai, machinelearning, datascience
Published: 2023-04-17 15:24:23

For more than a decade, organizations are using machine learning for various use cases such as predictions, assistance in the decision-making process, and more.  
Due to the demand for high computational resources and in many cases expensive hardware requirements, the public cloud became one of the better ways for running machine learning or deep learning processes.  
## Terminology
Before we dive into the topic of this post, let us begin with some terminology:  
* **Artificial Intelligence** – "The ability of a computer program or a machine to think and learn", Wikipedia  
* **Machine Learning** – "The task of making computers more intelligent without explicitly teaching them how to behave", Bill Brock, VP of Engineering at Very  
* **Deep Learning** – "A branch of machine learning that uses neural networks with many layers. A deep neural network analyzes data with learned representations like the way a person would look at a problem", Bill Brock, VP of Engineering at Very  

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9t2x1s12iisbalmu873b.png)

Source: [https://www.simplilearn.com/tutorials/artificial-intelligence-tutorial/ai-vs-machine-learning-vs-deep-learning](https://www.simplilearn.com/tutorials/artificial-intelligence-tutorial/ai-vs-machine-learning-vs-deep-learning)  

### Public use case of deep learning
* [Disney makes its archive accessible using deep learning built on AWS](https://aws.amazon.com/blogs/media/in-the-news-disney-makes-its-archive-accessible-using-deep-learning-built-on-aws/)  

In this blog post, I will focus on deep learning and hardware available in the cloud for achieving deep learning.  

## Deep Learning workflow
The deep learning process is made of the following steps:  
1.	Prepare – Store data in a repository (such as object storage or a database)  
2.	Build – Choose a machine learning framework (such as [TensorFlow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [Apache MXNet](https://mxnet.apache.org/), etc.)  
3.	Train – Choose hardware (compute, network, storage) to train the model you have built ("learn" and optimize model from data)  
4.	Inference – Using the trained model (on large scale) to make a prediction  

## Deep Learning processors comparison (Training phase)
Below is a comparison table for the various processors available in the public cloud, dedicated to the deep learning training phase:  

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bjtuhfs0naftux3z6pd8.png)
### Additional References
* [Amazon EC2 - Accelerated Computing](https://aws.amazon.com/ec2/instance-types/#Accelerated_Computing)  
* [AWS EC2 Instances Powered by Gaudi Accelerators for Training Deep Learning Models](https://aws.amazon.com/blogs/aws/new-ec2-instances-powered-by-gaudi-accelerators-for-training-deep-learning-models/)  
* [AWS Trainium](https://aws.amazon.com/machine-learning/trainium/)  
* [NVIDIA T4 Tensor Core GPU](https://www.nvidia.com/en-us/data-center/tesla-t4/)  
* [NVIDIA A10 Tensor Core GPU](https://www.nvidia.com/en-us/data-center/products/a10-gpu/)  
* [NVIDIA A100 Tensor Core GPU](https://www.nvidia.com/en-us/data-center/a100/)  

## Deep Learning processors comparison (Inference phase)
Below is a comparison table for the various processors available in the public cloud, dedicated to the deep learning inference phase:  

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mqzca2mubuy8u89o4h7f.png)

### Additional References
* [Amazon EC2 - Accelerated Computing](https://aws.amazon.com/ec2/instance-types/#Accelerated_Computing)  
* [AWS Inferentia](https://aws.amazon.com/machine-learning/inferentia/)  

## Summary
In this blog post, I have shared information about the various alternatives for using hardware available in the public cloud to run deep learning processes.  
I recommend you to keep reading and expand your knowledge on both machine learning and deep learning, what services are available in the cloud and what are the use cases to achieve outcomes from deep learning.  

### Additional References
* [AWS Machine Learning Infrastructure](https://aws.amazon.com/machine-learning/infrastructure/)  
* [AWS - Select the right ML instance for your training and inference jobs](https://pages.awscloud.com/rs/112-TZM-766/images/AL-ML%20for%20Startups%20-%20Select%20the%20Right%20ML%20Instance.pdf)  
* [AWS - Accelerate deep learning and innovate faster with AWS Trainium](https://d1.awsstatic.com/events/Summits/reinvent2022/CMP313_Accelerate-deep-learning-and-innovate-faster-with-AWS-Trainium.pdf)  

### About the Author
Eyal Estrin is a cloud and information security architect, the owner of the blog [Security & Cloud 24/7](https://security-24-7.com/) and the author of the book [Cloud Security Handbook](https://amzn.to/3xMI4Ak), with more than 20 years in the IT industry.  
Eyal is an [AWS Community Builder](https://aws.amazon.com/developer/community/community-builders/) since 2020.  
You can connect with him on [Twitter](https://twitter.com/eyalestrin) and [LinkedIn](https://www.linkedin.com/in/eyalestrin/).