Deep Learning Workflow in PyTorch
Tags: pytorch, deeplearning, workflow, development
Published: 2024-06-08 22:18:03

1. Prepare data(true data).
2. Prepare (untrained) model.
3. Train model.
4. Test model.
5. Save model.

### 1. Prepare data(true data).

1. Get data(true data) like images, video, sound, text, etc.
2. Divide the data(true data) into the one for training(**Train data**) and the one for testing(**Test data**). *Basically, train data is 80% and test data is 20%.

### 2. Prepare (untrained) model.

1. Select the suitable layers for the data. *There are layers in PyTorch such as [Linear()](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear), [Conv2d()](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d), [MaxPool2d()](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html#torch.nn.MaxPool2d), etc according to [the doc](https://pytorch.org/docs/stable/nn.html#convolution-layers).
2. Select the activation function for the data if necesarry. *There are layers in PyTorch such as [ReLU()](https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU), [Sigmoid()](https://pytorch.org/docs/stable/generated/torch.nn.Sigmoid.html#torch.nn.Sigmoid), [Softmax()](https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html#torch.nn.Softmax), etc according to [the doc](https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity).

### 3. Train model.

1. Select the suitable loss function and optimizer for the data(true data).
*Memos:
  - A loss function is the function which can get the mean(average) of the sum of the losses(differences) between a model's predictions and true values(train or test data) to optimize a model during training or to evaluate how good a model is during testing.
  - An optimizer is the function which can adjust a model's parameters(`weight` and `bias`) by **gradient descent** to minimize the mean(average) of the sum of the losses(differences) between the model's predictions and true values(train data) during training. *Gradient Descent(GD) is the function which can find the minimum(or maximum) gradient(slope) of a function.
  - Loss Function is also called Cost Function or Error Function.
  - There are loss functions in PyTorch such as [L1Loss()](https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html#torch.nn.L1Loss), [MSELoss()](https://pytorch.org/docs/stable/generated/torch.nn.MSELoss.html#torch.nn.MSELoss), [CrossEntropyLoss()](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss), etc according to [the doc](https://pytorch.org/docs/stable/nn.html#loss-functions).
  - There are optimizers in PyTorch such as [SGD()](https://pytorch.org/docs/stable/generated/torch.optim.SGD.html#torch.optim.SGD), [Adam()](https://pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam), [Adadelta()](https://pytorch.org/docs/stable/generated/torch.optim.Adadelta.html#torch.optim.Adadelta), etc according to [the doc](https://pytorch.org/docs/stable/optim.html#algorithms).
2. Calculate a model's predictions with true values(train data), working from input layer to output layer. &#42;This calculation is called **Forward Propagation**.
3. Calculate the mean(average) of the sum of the losses(differences) between a model's predictions and true values(train data) using a loss function.
4. Zero out the gradients of all tensors every epoch for proper calculation. *The gradients are accumulated in buffers, then they are not overwritten until [backward()](https://pytorch.org/docs/stable/generated/torch.Tensor.backward.html) is called.
5. Calculate a gradient using the mean(average) of the sum of the losses(differences) calculated by `3.`, working from output layer to input layer. &#42;This calculation is called **Backpropagation**.
6. Update the model's parameters(`weight` and `bias`) by gradient descent using the gradient calculated by `5.` to minimize the mean(average) of the sum of the losses(differences) between a model's predictions and true values(train data) using an optimizer.

&#42;Repeat the epoch of `2.`, `3.`, `4.`, `5.` and `6.` with a `for` loop to minimize the mean(average) of the sum of the losses(differences) between a model's predictions and true values(train data).

### 4. Test model.

1. Calculate a model's predictions with true values(test data).
2. Calculate the mean(average) of the sum of the losses(differences) between a model's predictions and true values(test data) with a loss function.
3. Show each mean(average) of the sum of the losses(differences) with true values(train and test data) by text or graph.

### 5. Save model.

Finally, save the model if the model is the enough quality which you want.