# Pallet Recognition with Multi-Task Learning for Automated Guided Vehicles
## 3 Experiments
### Evaluation of the Performance of Pallet Recognition  
The deep learning models have two classification tasks (angle and distance), and both must be correct to function in real-world situations. If even one label is wrong, the AGV cannot engage the pallet. Thus, we used an exact match ratio (subset accuracy), indicating  
Figure 4: (**a,b**) are correctly placed pallets in different workplaces. (**c**) is an example of a pallet moved 20 cm to the right. (**d**) is an example of a pallet moved 20 cm to the left and rotated three degrees. Figure 5: Four types of images in training dataset. the percentage of samples that have all their labels classified correctly. The exact match ratio is defined as the following product of two indicating functions of each label:  
\[\textit{exact match ratio}=\frac{1}{n}\ \sum_{i=1}^{n}I(Y^{(i)}_{\textit{angle}}= \widehat{Y}^{(i)}_{\textit{angle}})I(Y^{(i)}_{\textit{distance}}=\widehat{Y}^ {(i)}_{\textit{distance}}), \tag{2}\]  
where \(Y^{(i)}\) is a true label and \(\widehat{Y}^{(i)}\) is the predicted label of \(Y^{(i)}\) for \(i\)th sample. \(I\) is an indicator function that returns one if the condition in the parenthesis is true and zero otherwise. The exact match ratio is a strict measure because zero is returned if any of the predicted labels is wrong. Although our tasks are classification problems, each label can be considered as a continuous one. Therefore, we used the following mean absolute errors that measure the amount of difference between the actual and predicted values:  
\[\textit{mean absolute error}=\frac{1}{n}\ \sum_{i=1}^{n}\Bigl{|}Z^{(i)}- \widehat{Z}^{(i)}\Bigr{|}, \tag{3}\]  
where \(Z^{(i)}\) and \(\widehat{Z}^{(i)}\) are values of the predicted and true labels for \(i\)th sample. We conducted experiments with multi-task models and single-task models in five deep learning architectures. As reported in Table 5, the multi-task model outperformed the single-task models in terms of exact match ratios. For task-specific accuracy, the results are different as shown in Table 6. In the case of Task 1, both models showed high accuracy, which meant the task is easy to learn. The single-task model showed slightly better performance than the multi-task model, but the difference is less than 2%. On the other hand, in Task 2, both models were less accurate than in Task 1, and the multi-task model showed better accuracy. The multi-task structure showed benefits in learning Task 2, which is relatively difficult to learn. As shown in Figure 6, training and validation losses for Task 1 had similar curves regardless of the model. However, for Task 2, the training losses of the multi-task model converged earlier, and the validation losses had a lower value. Compared with the single-task model, the gap between training and validation losses is smaller, which is consistent with the characteristics of a multi-task model that is robust to overfitting. As shown in Figure 7, the same results are shown in terms of accuracy. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{3}{*}{**Methods**} & \multicolumn{8}{c}{**Architectures**} \\ \cline{2-10}  & **AlexNet** & **VGG11** & **ResNet-18** & **ResNet-50** & **ResNet-101** \\ \cline{2-10}  & **Task 1** & **Task 2** & **Task 1** & **Task 2** & **Task 1** & **Task 2** & **Task 1** & **Task 2** \\ \hline \multirow{3}{*}{Single-task} & **0.95** & 0.61 & **0.97** & 0.69 & **0.97** & 0.78 & **0.97** & 0.72 & 0.96 & 0.71 \\  & **(0.02)** & (0.09) & **(0.02)** & (0.08) & **(0.03)** & (0.06) & **(0.02)** & (0.07) & (0.02) & (0.06) \\ \cline{1-1}  & 0.93 & **0.75** & 0.95 & **0.83** & 0.96 & **0.84** & 0.96 & **0.84** & **0.97** & **0.79** \\ \cline{1-1}  & (0.03) & **(0.08)** & (0.03) & **(0.06)** & (0.02) & **(0.06)** & (0.03) & **(0.06)** & **(0.02)** & **(0.11)** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparisons with single models and multi-task models, in terms of task-specific accuracy. The best values for each architecture are in bold. The mean accuracy and standard deviation are reported.