# Hands-On Fundamentals of 1D Convolutional Neural Networks--A Tutorial for Beginner Users
## 5 Backpropagation  
The training process of the network comprises both feedforward and backpropagation, as schematically depicted in Figure 5.  
Figure 5: Schematization of the training process. Backpropagation aims to minimize a loss function by updating the network's parameters (e.g., weights and biases) so that the network can correctly map arbitrary inputs to outputs. The choice of loss function depends on the specific task at hand, as different tasks require different loss functions to effectively measure and optimize model performance. Our literature review indicates that regression and classification are among the most commonly used tasks. Typically, a regression task aims to predict the value of a continuous variable, such as a price or probability. For instance, a 1D CNN can be used in regression tasks to predict the concentration of a specific compound in a sample based on its absorption spectrum. In this example, the input 1D signals are represented by absorbance values recorded at various wavelengths (spectra). Additionally, the same spectral data can be employed for classification tasks by making use of different loss function. In this case, the goal is to categorize different chemical compounds into predefined categories. While the underlying architecture of 1D CNNs for both regression and classification tasks involves convolutional layers to extract features from sequential data, the key differences lie in the output layer configuration, the choice of activation functions, the loss functions used for training, and the evaluation metrics. For regression, the output is a continuous value and the model is trained to minimize the error between predicted and actual values. For classification, the output is categorical, and the model is trained to correctly classify inputs into one of several classes. Backpropagation is the process used in training neural networks, where the error (or loss) is propagated backward through the layers of the network, starting from the output layer and moving toward the input layer. Its main purpose is to calculate the gradients (partial derivatives) of the loss function with respect to each parameter (weight) in the network. These gradients indicate how much each parameter needs to change to reduce the loss. The derivatives can be efficiently computed using the chain rule and can be used in the gradient descent algorithm to update the network's parameters in a way that minimizes the loss function. In this section, the gradients of each parameter are computed from end to start (e.g., from FCL to \(CL_{1}\)). In regression problems, the loss function (\(L\)) is computed by comparing the actual target \(y_{t}\) with the predicted output values \(y_{out}\). One of the most popular loss functions is represented by the mean squared error (MSE):  
\[L=\frac{1}{N}\sum_{i=1}^{N}(y_{out}(i)-y_{t}(i))^{2} \tag{13}\]  
where \(i\) runs from 1 to the dimension of the output, in this case \(N=2\). Let us begin with the FCL. In this case, the parameters to be updated are the weight matrix \(W\) and the bias \(b\). The first gradient to be calculated is \(\Delta\)W, with each entry represented by  
\[ \begin{split}\Delta W(i,j)&=\frac{\partial L}{ \partial W(i,j)}\\ &=\frac{\partial L}{\partial y_{out}(i)}\cdot\frac{\partial y_{ out}(i)}{\partial W(i,j)}\\ &=(y_{out}(i)-y_{t}(i))\cdot\frac{\partial}{\partial W(i,j)} \sigma\left(\sum_{j=1}^{18}W(i,j)f(j)+b(i)\right)\end{split}  \tag{14}\]  
where \" \(\cdot\) \" refers to the element-wise product. The last derivative in Equation (14) can be calculated once an activation function is provided. Discussing the advantages and disadvantages of using a specific activation function over another is beyond the scope of this work. For this purpose, we refer the curiosity of the reader to the existing literature on this topic [7; 125; 126; 127; 128; 129]. In this output layer, the sigmoid activation function has been considered [130], while other functions will be discussed in Appendix A.