# Hands-On Fundamentals of 1D Convolutional Neural Networks--A Tutorial for Beginner Users
## 5 Backpropagation
\[\Delta K_{j,l}^{(2)}(u) =\frac{\partial L}{\partial K_{j,l}^{(2)}(u)} \tag{21}\] \[=\sum_{i=1}^{6}\frac{\partial L}{\partial C_{l,\sigma}^{(2)}(i)} \cdot\frac{\partial C_{l,\sigma}^{(2)}(i)}{\partial K_{j,l}^{(2)}(u)}\] \[=\sum_{i=1}^{6}\Delta C_{l,\sigma}^{(2)}(i)\cdot\frac{\partial \sigma(C_{l}^{(2)}(i))}{\partial K_{j,l}^{(2)}(u)}\] \[=\sum_{i=1}^{6}\Delta C_{l,\sigma}^{(2)}(i)\cdot\frac{\partial \sigma(C_{l}^{(2)}(i))}{\partial C_{l}^{(2)}(i)}\cdot\frac{\partial C_{l}^{( 2)}(i)}{\partial K_{j,l}^{(2)}(u)}\] \[=\sum_{i=1}^{6}\Delta C_{l,\sigma}^{(2)}(i)\cdot C_{l,\sigma}^{( 2)}(i)\cdot(1-C_{l,\sigma}^{(2)}(i))\cdot\frac{\partial C_{l}^{(2)}(i)}{ \partial K_{j,l}^{(2)}(u)}\] Figure 6.0  
where \(\Delta C_{l}^{(2)}=\Delta C_{l,\sigma}^{(2)}\cdot C_{l,\sigma}^{(2)}\cdot(1-C _{l,\sigma}^{(2)})\) has been obtained, considering sigmoid as the activation function (\(\sigma\)). In these hidden layers, other activation functions may be used, these cases will be discussed in the Appendix A.  
Recalling Equations (3) and (9), the last derivative in (21) can be written as  
Figure 6.1 Figure 6.2 Figure 6.3  
and since only \(S_{j}^{(1)}\cdot K_{j,l}^{(2)}\) is relevant for the computation, we obtain  
Figure 6.4  
Each element of the error \(\Delta K_{j,l}^{(2)}\) can now be written as  
Figure 6.5  
and then the error is  
Figure 6.6  
where \"\(*\)\" represents the convolution operation. In the present case, \(S_{j}^{(1)}\) and \(\Delta C_{l}^{(2)}\) are \(8\times 1\) and \(6\times 1\) 1D columns, respectively. The convolution in Equation (23) yields a \(3\times 1\) vector, corresponding to the filter dimension selected in \(CL_{2}\) (\(s^{(2)}\)). Now, great attention should be paid when using Equation (25). In the literature, there are many examples of similar formulae in which the vector \(S_{j}^{(1)}\) is used in reverse [120], without providing details. The reversing operation is required only when the convolution definition in (2) is used. Following a similar procedure, it is possible to compute the bias \(b_{l}^{(2)}\) error. Since \(C_{l}^{(2)}\) is linear with the bias, \(\frac{\partial C_{l}^{(2)}}{\partial b_{l}^{(2)}}=1\), then the bias error is computed as follows:  
\[ \begin{split}\Delta b_{l}^{(2)}&=\frac{\partial L}{ \partial b_{l}^{(2)}}\\ &=\sum_{i=1}^{6}\frac{\partial L}{\partial C_{l,\sigma}^{(2)}(i)} \cdot\frac{\partial C_{l,\sigma}^{(2)}(i)}{\partial b_{l}^{(2)}}\\ &=\sum_{i=1}^{6}\Delta C_{l,\sigma}^{(2)}(i)\cdot\frac{\partial \sigma(C_{l}^{(2)}(i))}{\partial b_{l}^{(2)}}\\ &=\sum_{i=1}^{6}\Delta C_{l,\sigma}^{(2)}(i)\cdot\frac{\partial \sigma(C_{l}^{(2)}(i))}{\partial C_{l}^{(2)}(i)}\cdot\frac{\partial C_{l}^{( 2)}(i)}{\partial b_{l}^{(2)}}\\ &=\sum_{i=1}^{6}\Delta C_{l,\sigma}^{(2)}(i)\cdot C_{l,\sigma}^{ (2)}(i)(1-C_{l,\sigma}^{(2)}(i))\\ &=\sum_{i=1}^{6}\Delta C_{l}^{(2)}(i).\end{split}  \tag{26}\]  
Let us now enter the \(CL_{1}\). The object here is to update filters \(K_{1,l}^{(1)}\) and bias \(b_{j}^{(1)}\). To achieve this, the error \(\Delta S_{j}^{(1)}\) must be calculated first. Using Equations (1) and (9), \(C_{l}^{(2)}(i)\) can be expressed as a function of \(S_{j}^{(2)}(i+u-1)\). Introducing a new index \(I=i+u-1\), we find that \(C_{l}^{(2)}(I-u+1)\) can be represented as a function of \(S_{j}^{(2)}(I)\). Moreover, for simplicity, if we denote \(I=i\), \(C_{l}^{(2)}(i-u+1)\) can be written as a function of \(S_{j}^{(2)}(i)\). This result is utilized when computing each element of the error \(\Delta S_{j}^{(1)}\) as follows:  
\[ \begin{split}\Delta S_{j}^{(1)}(i)&=\frac{\partial L }{\partial S_{j}^{(1)}(i)}\\ &=\sum_{l=1}^{q=6}\sum_{u=1}^{s^{(2)}}\frac{\partial L}{\partial C _{l}^{(2)}(i-u+1)}\cdot\frac{\partial C_{l}^{(2)}(i-u+1)}{\partial S_{j}^{(1 )}(i)}\\ &=\sum_{l=1}^{q=6}\sum_{u=1}^{s^{(2)}}\Delta C_{l}^{(2)}(i-u+1) \cdot\frac{\partial C_{l}^{(2)}(i-u+1)}{\partial S_{j}^{(1)}(i)}\end{split}  \tag{27}\]  
where \(s^{(2)}=3\) is the filter size used in \(CL_{2}\). The partial derivative in Equation (27) can be computed by making use of Equation (9) evaluated in \(i-u+1\).